U md@sddlmZddlmZddlmZddlmZddlZddl Z ddl Z ddl mZddlmZdd lmZdd lmZd d lmZe je jd ddZedddZeedddZeejdddZeejdddZ ddee!ejdddZ"dS) )urlopen) HTTPError)ZipFile)BinaryION)sparse) _download)settings)logging)check_datasetdir_exists) dataframereturncCs6|dd}|dk|t|kB}|jdd|fS)NcSs t|SN)lenunique)xr^/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/scanpy/datasets/_ebi_expression_atlas.pyz _filter_boring..r )applyrloc)r Z unique_valsZ is_boringrrr_filter_boringsr) accessionc Csbd|d}zt|W5QRXWn8tk r\}z|jd|d|_W5d}~XYnXdS)Nz)https://www.ebi.ac.uk/gxa/sc/experiments//z ())rrmsg)rbase_urlerrr sniff_urls  r cCsft|d|}|d}|d}tj|}|jdddt|d|dt|d|d dS) Nz(https://www.ebi.ac.uk/gxa/sc/experiment/z/download?accessKey=&fileType=z"/download/zip?accessKey=&fileType=T)parentsexist_okzexperiment-designexperimental_design.tsvzquantification-rawexpression_archive.zip)r r datasetdirmkdirr)rrZ design_urlZmtx_urlexperiment_dirrrrdownload_experiment"s    r()streamrc Cs|}|dr|}qdd|dddD\}}}ttjj}||ks\||krdtj}ntj}tj |dd||tj dd}t j |d |d d |d d ff||fd }|S) N%css|]}t|VqdSr)int).0rrrr ;sz'read_mtx_from_stream.. z\s+)rr r)sepheaderZdtyperr r)shape) readline startswithsplitnpZiinfoZint32maxZint64pdread_csvZfloat32r csr_matrix) r)Zcurlinenm_Z max_int32Z coord_dtypedataZmtxrrrread_mtx_from_stream7s   " .r?)archiverc Cs|}t|dksttdd|D}tdd|D}tdd|D}||d}t|}W5QRX||d}tj|dddd }W5QRX||d$}tj|dddjddd f}W5QRXt |} || _ || _ | S) Ncss|]}|jdr|VqdS)z.mtxNfilenameendswithr,irrrr-Ps z/read_expression_from_archive..css|]}|jdr|VqdS)z .mtx_rowsNrBrErrrr-Qs css|]}|jdr|VqdS)z .mtx_colsNrBrErrrr-Rs r )r0r1r r) infolistrAssertionErrornextopenr?r8r9ZilocanndataAnnDataZ var_namesZ obs_names) r@infoZ mtx_data_infoZ mtx_rows_infoZ mtx_cols_infofexprvarnameZobsnameadatarrrread_expression_from_archiveMs( rTF) filter_boring)rrUrc Cstj|}||d}z t|}|r4t|j|_|WStk rLYnXt|t d|d| t |dd}t |}W5QRXt j|dddd }||j|j<|j|d d |rt|j|_|S) a Load a dataset from the `EBI Single Cell Expression Atlas `__ Downloaded datasets are saved in the directory specified by :attr:`~scanpy._settings.ScanpyConfig.datasetdir`. Params ------ accession Dataset accession. Like ``E-GEOD-98816`` or ``E-MTAB-4888``. This can be found in the url on the datasets page, for example https://www.ebi.ac.uk/gxa/sc/experiments/E-GEOD-98816/results/tsne. filter_boring Whether boring labels in `.obs` should be automatically removed, such as labels with a single or :attr:`~anndata.AnnData.n_obs` distinct values. Example ------- >>> import scanpy as sc >>> adata = sc.datasets.ebi_expression_atlas("E-MTAB-4888") z.h5adz Downloaded z to r$rGr#rHr)r0Z index_colgzip) compression)r r%rMreadrobsOSErrorr(loggrOabsoluterrTr8r9columnswrite)rrUr'Z dataset_pathrSrPrYrrrebi_expression_atlas`s&     r_)#urllib.requestr urllib.errorrzipfilertypingrrMZpandasr8numpyr6ZscipyrZ readwriterZ _settingsr r r[_utilsr Z DataFramerstrr r(r:r?rNrTboolr_rrrrs.