U md# @sdZddlmZddlZddlmZd?ddZddZ d d Z d Z d Z d dZ ddZddZedkrdZejjdedfddefdefdefdefgZejjedfddefdefgZddlZejjjeefdd Zeed!feZ ej!j"D]Z#ee#e e#<qej!j"D]Z#ee#e e#<qed"e \Z$Z"ed#e \Z$Z"e'd$d%e"DZ(e()d!d&ejjedZ*e+e*e(,Z-e&e-j.e&e e e-e'd'd%ed(e"DZ(e()d!d&ejjedZ*e+e*e(,Z-e&e-j.e&e e e-e!d)efd*efd+efd,efd-efd.efd/efd0efgZ/ej0d1e/d2dd3Z1e&d4d5d%e1j!j"De1j2e3Z4e45d6e6e1j!j"7d!Z8e1e8ddfj9Z:e&e:j;e&e:j!ed7e:\Ze:d0Z?e+e?e>,Z@e&e@j.e&e e e@d:Ae1j!j"dd6ZBed;e:\ZCZDe'dd%eeHeDDZIe:d0ZJe+eJeI,ZKe&e e eKq|dS)@a convenience functions for ANOVA type analysis with OLS Note: statistical results of ANOVA are not checked, OLS is checked but not whether the reported results are the ones used in ANOVA includes form2design for creating dummy variables TODO: * ... * )lmapNFcCs^|}t|}|r0|dddf|ktS|dddf|ktddddfSdS)z|convert array of categories to dummy variables by default drops dummy variable for last category uses ravel, 1d onlyN)Zravelnpuniqueastypeint)x returnallgroupsr e/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/statsmodels/sandbox/regression/try_ols_anova.py data2dummys  r cCsLttt|}||dddddfkdjtddddfS)zcreates product dummy variables from 2 columns of 2d array drops last dummy variable, but not from each category singular with simple dummy variable but not with constant quickly written, no safeguards Nr) rrrtupletolistallTrr)rr r r r data2proddummys rcCs.|jdkr|dddf}t|dd}||S)zcreate dummy continuous variable Parameters ---------- x1 : 1d array label or group array x2 : 1d array (float) continuous variable Notes ----- useful for group specific slope coefficients in regression NT)r )ndimr )x1Zx2dummyr r r data2groupcont.s  raW ANOVA statistics (model sum of squares excludes constant) Source DF Sum Squares Mean Square F Value Pr > F Model %(df_model)i %(ess)f %(mse_model)f %(fvalue)f %(f_pvalue)f Error %(df_resid)i %(ssr)f %(mse_resid)f CTotal %(nobs)i %(uncentered_tss)f %(mse_total)f R squared %(rsquared)f a] ANOVA statistics (model sum of squares includes constant) Source DF Sum Squares Mean Square F Value Pr > F Model %(df_model)i %(ssmwithmean)f %(mse_model)f %(fvalue)f %(f_pvalue)f Error %(df_resid)i %(ssr)f %(mse_resid)f CTotal %(nobs)i %(uncentered_tss)f %(mse_total)f R squared %(rsquared)f c Csbi}||jddddddddd d d g }|D]}t||||<q.|jj|d <|j|j|d <|S)zjupdate regression results dictionary with ANOVA specific statistics not checked for completeness Zdf_modelZdf_residZessssruncentered_tssZ mse_modelZ mse_residZ mse_totalZfvalueZf_pvalueZrsquarednobsZ ssmwithmean)update__dict__getattrmodelrrr)resadZ anova_attrkeyr r r anovadict[s   r"cCshi}g}|D]L}|dkr>t|jd|d<|dqd|kr^||||<||q|dddkr|dd}t||||<||q|ddd kr|ddd }ttj||d||df|d |<|d |q|ddd krV|ddd }t ||d||d|d |<|d |qt d q||fS)aconvert string formula to data dictionary ss : str * I : add constant * varname : for simple varnames data is used as is * F:varname : create dummy variables for factor varname * P:varname1*varname2 : create product dummy variables for varnames * G:varname1*varname2 : create product between factor and continuous variable data : dict or structured array data set, access of variables by name as in dictionaries Returns ------- vars : dictionary dictionary of variables with converted dummy variables names : list list of names, product (P:) and grouped continuous variables (G:) have name by joining individual names sorted according to input Examples -------- >>> xx, n = form2design('I a F:b P:c*d G:c*f', testdata) >>> xx.keys() ['a', 'b', 'const', 'cf', 'cd'] >>> n ['const', 'a', 'b', 'cd', 'cf'] Notes ----- with sorted dict, separate name list would not be necessary Irconst:NzF:rzP:*zG:zunknown expression in formula) splitrZonesshapeappendr rZc_joinr ValueError)ssdatavarsnamesitemvr r r form2designls.$    ,$ r4cCs(|dd}|D]}||q|S)zwdrop names from a list of strings, names to drop are in space delimited list does not change original list N)r)remove)r.ZliZnewlir2r r r dropnames   r6__main__i)sizeabcdr&efT)flattenrz I a F:b P:c*dzI a F:b P:c*d G:a*e fcCsg|] }t|qSr xx.0nnr r r srGg{Gz?cCsg|] }t|qSr rBrDr r r rGszae fZbreedZsexZlitterpenpigZageZbageyz dftest3.data.)missingZusemaskrLcCsg|]}tj|qSr )dtamasksum)rEkr r r rGsrz I F:sex agecCsg|] }t|qSr )xx_b1rDr r r rGsr( z'I F:breed F:sex F:litter F:pen age bagecCsg|] }t|qSr xx_b1arDr r r rGsz Results droppingcCsg|] }t|qSr rSrDr r r rGs)F)L__doc__Zstatsmodels.compat.pythonrnumpyrZstatsmodels.apiapismr rrZ anova_str0Z anova_strr"r4r6__name__rrandomrandintviewrZ testdataintnormalfloatZ testdatacontZnumpy.lib.recfunctionslibZ recfunctionsZ zip_descrZdt2emptyZtestdataZdtyper1namerCnprintZ column_stackXrOrJZOLSfitZrest1paramsZdt_bZ genfromtxtrMrNboolmZreshapelenanyZdroprowsr/Z dta_use_b1r*rQZnames_b1ZX_b1Zy_b1Zrest_b1r,ZallexogrTZ names_b1aZX_b1aZy_b1aZrest_b1aZdropnZX_b1a_Zy_b1a_Z rest_b1a_r r r r s     = 2$