U md2 @sdZddlZddlmZddlmZddZddZd d Z d d Z d dZ ddZ ddZ ddZddZddZddZedkrdZerPdZedZejddZeddd dfZeeeeedddfZd d d gZee eeeeeeedddfeZeeeeeeeed e eeeeedd!lm Z m!Z!d"d#Z"d$d%Z#d&d'Z$d(d)Z%d*d+Z&d,d-Z'ered.ee!j(e"d d/d d0dd1ed2ed dd eded3ed dd edee!j(e"d4d5d d6dd1ed3ed4dd ed7ee!j(e"d4d5d d8dd1ed3ed4dded7ee!j(e"d4d5d d9dd1ed3ed4d ded7ee!j(e#d d5d d:dd1ee!j(e$dd5d d;dd1d<\ZZ)Z*Z+ed3eee)e*ed7ee!j(e#e)d5d ee*e+fdd1ee!j(e$e*d5d ee)e+fdd1ed=ee!j(e%d d/d d>dd1eed dd e d?\ZZ)Z*ed3eee)e*e ee!j(e&e)d5d ee*fdd1ee!j(e'e*d5d ee)fdd1d@\ZZ)Z*ed3eee)e*e ee!j(e&e)d5d ee*fdd1ee!j(e'e*d5d ee)fdd1edAee d e,dd gdBe"d dd dBdCee d e,dd gde"d dd ddCeed e,dd gde"d dd dee d e,ddDgde"dEddEdeed e,dd gdBe"d dd dBee d e,dd gde"d ddEe-dFdddGl.m/Z/d>dHdIdJgZe0dKdLdMZ1eD]t\Z)Z*e!j(e&e)d5d e1e*fdd1Z2e!j(e'e*d5d e1e)fdd1Z3ee1e)e*e Z4e/e2e4ddNdOdPe/e3e4d dNdQdPqdRD]Z+eD]\Z)Z*e!j(e#e)d5d e1e*e+fdd1Z2e!j(e$e*d5d e1e)e+fdd1Z3ee1e)e*ee+Z4e/e2e4ddSdOdPe/e3e4d dSdQdPe/ee1e,e)e*dge+e"e1e)e*e+dNdTdPe/e e1e,e)e*dge+e"e1e)e*e-e+dLe+e+dNdTdPqqdS)Uagradient/Jacobian of normal and t loglikelihood use chain rule normal derivative wrt mu, sigma and beta new version: loc-scale distributions, derivative wrt loc, scale also includes "standardized" t distribution (for use in GARCH) TODO: * use sympy for derivative of loglike wrt shape parameters it works for df of t distribution dlog(gamma(a))da = polygamma(0,a) check polygamma is available in scipy.special * get loc-scale example to work with mean = X*b * write some full unit test examples A: josef-pktd N)special)gammalncCs<|j\}}dtdtjt|||d|}|S)aynormal loglikelihood given observations and mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows Returns ------- lls : ndarray contribution to loglikelihood for each observation g)Tnplogpi)yparamsmusigma2llsr]/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/statsmodels/sandbox/regression/tools.pynorm_llss .rcCsB|j\}}|||}||d|dt|}t||fS)aBJacobian of normal loglikelihood wrt mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows Returns ------- grad : array (nobs, 2) derivative of loglikelihood for each observation wrt mean in first column, and wrt variance in second column Notes ----- this is actually the derivative wrt sigma not sigma**2, but evaluated with parameter sigma2 = sigma**2 r)rrsqrt column_stack)r r r r ZdllsdmuZ dllsdsigma2rrr norm_lls_grad/s  rcCs|S)z-gradient/Jacobian for d (x*beta)/ d beta r)xbetarrr mean_gradKsrc Cs|dd}|dtt|df}t||}t||}t||f}t||}t|ddddf||ddddff} | S)aJacobian of normal loglikelihood wrt mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable with mean x*beta, and variance sigma2 x : ndarray, 2d explanatory variables, observation in rows, variables in columns params : array_like, (nvars + 1) array of coefficients and variance (beta, sigma2) Returns ------- grad : array (nobs, 2) derivative of loglikelihood for each observation wrt mean in first column, and wrt scale (sigma) in second column assume params = (beta, sigma2) Notes ----- TODO: for heteroscedasticity need sigma to be a 1d array Nr)roneslenrdotrr) r rr rr Zdmudbetar Zparams2ZdllsdmsZgradrrrnormgradPs    2rcCs|j\}}|d}t|ddt|ddt|dtj}||ddtd||d|d|dt|8}|S)at loglikelihood given observations and mean mu and variance sigma2 = 1 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows df : int degrees of freedom of the t distribution Returns ------- lls : ndarray contribution to loglikelihood for each observation Notes ----- parametrized for garch ?r@?r)rrrrrr r dfr r r rrrtstd_llsts  4@r"cCs| S)z?derivative of log pdf of standard normal with respect to y r)r rrr norm_dlldysr#cCspt|d}tt|ddt|dt|dtj}|d|d|d|dd}|S)zIpdf for standardized (not standard) t distribution, variance is one rrrr)rarrayexprrrr)rr!rZPxrrrtstd_pdfs:$r'cCst||||j\}}|d}t|ddt|ddt|tj}||ddtd||d||dt|8}|S)at loglikelihood given observations and mean mu and variance sigma2 = 1 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows df : int degrees of freedom of the t distribution Returns ------- lls : ndarray contribution to loglikelihood for each observation Notes ----- parametrized for garch normalized/rescaled so that sigma2 is the variance >>> df = 10; sigma = 1. >>> stats.t.stats(df, loc=0., scale=sigma.*np.sqrt((df-2.)/df)) (array(0.0), array(1.0)) >>> sigma = np.sqrt(2.) >>> stats.t.stats(df, loc=0., scale=sigma*np.sqrt((df-2.)/df)) (array(0.0), array(2.0)) rrrrr)printrrrrrr rrrts_llss   0<r)cCs*|d}|d |d|d||S)aderivative of log pdf of standard t with respect to y Parameters ---------- y : array_like data points of random variable at which loglike is evaluated df : array_like degrees of freedom,shape parameters of log-likelihood function of t distribution Returns ------- dlldy : ndarray derivative of loglikelihood wrt random variable y evaluated at the points given in y Notes ----- with mean 0 and scale 1, but variance is df/(df-2) rrrrr r!rrrts_dlldysr+cCs*|d |dd|d|d|S)aderivative of log pdf of standardized t with respect to y Parameters ---------- y : array_like data points of random variable at which loglike is evaluated df : array_like degrees of freedom,shape parameters of log-likelihood function of t distribution Returns ------- dlldy : ndarray derivative of loglikelihood wrt random variable y evaluated at the points given in y Notes ----- parametrized for garch, standardized to variance=1 rrrrr*rrr tstd_dlldysr,cGsN|||}||f| |}d|||f||||d}||fS)aderivative of log-likelihood with respect to location and scale Parameters ---------- y : array_like data points of random variable at which loglike is evaluated loc : float location parameter of distribution scale : float scale parameter of distribution dlldy : function derivative of loglikelihood fuction wrt. random variable x args : array_like shape parameters of log-likelihood function Returns ------- dlldloc : ndarray derivative of loglikelihood wrt location evaluated at the points given in y dlldscale : ndarray derivative of loglikelihood wrt scale evaluated at the points given in y grr)r locscaleZdlldyargsZystZdlldlocZ dlldscalerrr locscale_grads &r0__main__g?r r)statsmisccCsttjj||||dSN)r-r.rrr4tpdf)r r-r.r!rrrllt2sr:cCsttjj||||dSr6r7)r-r r.r!rrrlltloc4sr;cCsttjj||||dSr6r7)r.r r-r!rrrlltscale6sr<cCsttjj|||dSr6rrr4Znormr9)r r-r.rrrllnorm9sr>cCsttjj|||dSr6r=)r-r r.rrr llnormloc;sr?cCsttjj|||dSr6r=)r.r r-rrr llnormscale=sr@z gradient of tgư>)rrr2)Zdxnr/orderzt ts?g|=)rrrE)rrrE)rrrE)rDrrE)rDrrE)rDrrrEz gradient of norm)rr)rDrr)rDrrz loglike of tdzdifferently standardizedg?rg?)assert_almost_equal)rr)gr)rrgr z deriv loc)err_msgz deriv scale)r3r2rFZloglike)5__doc__numpyrZscipyrZ scipy.specialrrrrrr"r#r'r)r+r,r0__name__verbosesigrrrandomZrandnZrvsrrr r r(Z dllfdbetar4r5r:r;r<r>r?r@Z derivativer-r.r!r$rZ numpy.testingrGZlinspaceZytZdlldloZdlldscgrrrrrs  $ '          ((&&&0