U md7@sddlmZddlmZddlZddlZddlZddl Z ddl m Z ddl mZmZddlmZddlmZdd lmZmZd d lmZmZeejZd.eeeee jdddZ ee jdddZ!e jdddZ"ee jdddZ#ee jdddZ$e jdddZ%ee jdddZ&eee jdd d!Z'eee jdd"d#Z(d/e)e)eee*d%d&d'Z+ed0d$d)ed*e*e jd+d,d-Z,dS)1)Path)OptionalN)version)logging_utils)Literal)settings)read read_visium)check_datasetdir_existsfilter_oldformatwarning ?) n_variables n_centers cluster_stdn_observationsreturncCs@ddl}|jj||||dd\}}tj|t|td|jdS)a Gaussian Blobs. Parameters ---------- n_variables Dimension of feature space. n_centers Number of cluster centers. cluster_std Standard deviation of clusters. n_observations Number of observations. By default, this is the same observation number as in :func:`scanpy.datasets.krumsiek11`. Returns ------- Annotated data matrix containing a observation annotation 'blobs' that indicates cluster identity. rN)Z n_samplesZ n_featuresZcentersrZ random_state)blobs)obsdtype) Zsklearn.datasetsZdatasetsZ make_blobsadAnnDatadictastypestrr)rrrrZsklearnXyr"R/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/scanpy/datasets/_datasets.pyrs r)rcCstjd}d}t||d}|S)a9 Bulk data with conditions ulcerative colitis (UC) and Crohn's disease (CD). The study assesses transcriptional profiles in peripheral blood mononuclear cells from 42 healthy individuals, 59 CD patients, and 26 UC patients by hybridization to microarrays interrogating more than 22,000 sequences. Reference --------- Burczynski et al., "Molecular classification of Crohn's disease and ulcerative colitis patients using transcriptional profiles in peripheral blood mononuclear cells" J Mol Diagn 8, 51 (2006). PMID:16436634. z!burczynski06/GDS1615_full.soft.gzzQftp://ftp.ncbi.nlm.nih.gov/geo/datasets/GDS1nnn/GDS1615/soft/GDS1615_full.soft.gz backup_url)r datasetdirr )filenameurladatar"r"r# burczynski069s  r*cCstd}tj}dt_t|dd}|t_d|jd<ddd d d d }||jd <tddt|jD}d|dd<d |dd<d |dd<d |dd<||j d<t ||S)a Simulated myeloid progenitors [Krumsiek11]_. The literature-curated boolean network from [Krumsiek11]_ was used to simulate the data. It describes development to four cell fates: 'monocyte', 'erythrocyte', 'megakaryocyte' and 'neutrophil'. See also the discussion of this data in [Wolf19]_. Simulate via :func:`~scanpy.tl.sim`. Returns ------- Annotated data matrix. zkrumsiek11.txterrorTZfirst_column_namesrirootZStemZMoEryZMkZNeu)ri?iik highlightscSsg|]}dqS)Z progenitorr".0ir"r"r# gszkrumsiek11..Pi@iii0r cell_type) HEREr verbosityr unsnparrayrangeZn_obsrrsanitize_anndata)r'Zverbosity_saver)Z fate_labelsr8r"r"r# krumsiek11Os          r@cstjd}d}t|d|d}t|jddddg}|d d |f}d |jd <d d ddddfdd|jD|j d<t j |j dt d|j d<t |jd<|S)zp Hematopoiesis in early mouse embryos [Moignard15]_. Returns ------- Annotated data matrix. zmoignard15/nbt.3154-S3.xlsxzrhttps://static-content.springer.com/esm/art%3A10.1038%2Fnbt.3154/MediaObjects/41587_2015_BFnbt3154_MOESM4_ESM.xlsxzdCt_values.txt)Zsheetr%ZEif2b1ZMrpl19ZPolr2aZUbcNir-z#D7A83Ez#7AAE5Dz#497ABCz#AF353Az#765099)ZHFZNPZPSZ4SGZ4SFGcs&g|]tfddDqS)c3s|]}|r|VqdS)N) startswith)r2gnameZsnamer"r# s z(moignard15...)nextkeys)r2groupsrCr#r4szmoignard15..Z exp_groups) categoriesZexp_groups_colors)r r&r r<Zin1d var_namescopyr;Z obs_namesrpdZ CategoricallistrFvalues)r'r%r)Z gene_subsetr"rGr# moignard15qs*     rOc s@tdddl}tjd}|jjddd}t||| |d^}|d d }|d d  t }|d d  t }|d d  t }|dd  t }W5QRXtj|tjd} || _|| _ddgd7fdd|D| jd<t| dd| jD| _t|| j}| dd|f} d| jd<| S)a2 Development of Myeloid Progenitors [Paul15]_. Non-logarithmized raw data. The data has been sent out by Email from the Amit Lab. An R version for loading the data can be found here https://github.com/theislab/scAnalysisTutorial Returns ------- Annotated data matrix. zWIn Scanpy 0.*, this returned logarithmized data. Now it returns non-logarithmized data.rNzpaul15/paul15.h5Texist_okz"http://falexwolf.de/data/paul15.h5rzdata.debatchedr"zdata.debatched_rownameszdata.debatched_colnamesz cluster.idzinfo.genes_strings)rr.z3MEP Mk GMP GMP DC Baso Baso Mo Mo Neu Neu Eos Lymphcs g|]}||dqS)r r"r1r8r"r#r4szpaul15..Zpaul15_clusterscSsg|]}|ddqS);r)split)r2Zgnr"r"r#r4siHr-)loggwarningh5pyr r&parentmkdirrcheck_presence_downloadFilerrflattenintrrZ transposer<Zfloat32rJZ row_namesrVrr?Z intersect1dr;) rYr'r%fr Z gene_names cell_namesZclustersZinfogenes_namesr)r"rTr#paul15s4       rbcCs"td}t|dd}d|jd<|S)z Simulated toggleswitch. Data obtained simulating a simple toggleswitch [Gardner00]_ Simulate via :func:`~scanpy.tl.sim`. Returns ------- Annotated data matrix. ztoggleswitch.txtTr,rr-)r9r r;)r'r)r"r"r# toggleswitchs   rcc Cs@td}t&tjdtddt|W5QRSQRXdS)a Subsampled and processed 68k PBMCs. 10x PBMC 68k dataset from https://support.10xgenomics.com/single-cell-gene-expression/datasets The original PBMC 68k dataset was preprocessed using scanpy and was saved keeping only 724 cells and 221 highly variable genes. The saved file contains the annotation of cell types (key: `'bulk_labels'`), UMAP coordinates, louvain clustering and gene rankings based on the `bulk_labels`. Returns ------- Annotated data matrix. z10x_pbmc68k_reduced.h5adignoreanndatacategorymoduleN)r9warningscatch_warningsfilterwarnings FutureWarningr )r'r"r"r#pbmc68k_reduceds rmcCsd}ttjd|d}|S)a 3k PBMCs from 10x Genomics. The data consists in 3k PBMCs from a Healthy Donor and is freely available from 10x Genomics (`here `__ from this `webpage `__). The exact same data is also used in Seurat's `basic clustering tutorial `__. .. note:: This downloads 5.9 MB of data upon the first call of the function and stores it in `./data/pbmc3k_raw.h5ad`. The following code was run to produce the file. .. code:: python adata = sc.read_10x_mtx( # the directory with the `.mtx` file './data/filtered_gene_bc_matrices/hg19/', # use gene symbols for the variable names (variables-axis index) var_names='gene_symbols', # write a cache file for faster subsequent reading cache=True, ) adata.var_names_make_unique() # this is unnecessary if using 'gene_ids' adata.write('write/pbmc3k_raw.h5ad', compression='gzip') Returns ------- Annotated data matrix. z(http://falexwolf.de/data/pbmc3k_raw.h5adzpbmc3k_raw.h5adr$)r r r&)r(r)r"r"r#pbmc3ks'rnc CsBt0tjdtddttjdddW5QRSQRXdS)zProcessed 3k PBMCs from 10x Genomics. Processed using the `basic tutorial `__. Returns ------- Annotated data matrix. rdrerfzpbmc3k_processed.h5adz[https://raw.githubusercontent.com/chanzuckerberg/cellxgene/main/example-dataset/pbmc3k.h5adr$N)rirjrkrlr r r&r"r"r"r#pbmc3k_processed&s roF) sample_idspaceranger_versionbase_dirdownload_imagec Csddl}|dkrtj}d|d|d}||}|jdd|d}||}tj|||d||*} | D]} || jsp| | |qpW5QRXtj|d ||d d|rtj|d ||d ddS) z Params ------ sample_id String name of example visium dataset. base_dir Where to download the dataset to. download_image Whether to download the high-resolution tissue section. rNz/https://cf.10xgenomics.com/samples/spatial-exp//TrPz_spatial.tar.gz)r'r%zfiltered_feature_bc_matrix.h5z_filtered_feature_bc_matrix.h5 image.tifz _image.tif) tarfiler r&r[rr\opennameexistsextract) rprqrrrsrvZ url_prefixZ sample_dirZ tar_filenameZtar_pthr`elr"r"r#_download_visium_dataset9s2     r|"V1_Breast_Cancer_Block_A_Section_1)include_hires_tiff)r}Z"V1_Breast_Cancer_Block_A_Section_2ZV1_Human_HeartZV1_Human_Lymph_NodeZV1_Mouse_KidneyZV1_Adult_Mouse_BrainZ!V1_Mouse_Brain_Sagittal_PosteriorZ+V1_Mouse_Brain_Sagittal_Posterior_Section_2Z V1_Mouse_Brain_Sagittal_AnteriorZ*V1_Mouse_Brain_Sagittal_Anterior_Section_2ZV1_Human_Brain_Section_1ZV1_Human_Brain_Section_2Z&V1_Adult_Mouse_Brain_Coronal_Section_1Z&V1_Adult_Mouse_Brain_Coronal_Section_2Z-Targeted_Visium_Human_Cerebellum_NeuroscienceZParent_Visium_Human_CerebellumZ-Targeted_Visium_Human_SpinalCord_NeuroscienceZParent_Visium_Human_SpinalCordZ-Targeted_Visium_Human_Glioblastoma_Pan_CancerZ Parent_Visium_Human_GlioblastomaZ-Targeted_Visium_Human_BreastCancer_ImmunologyZ Parent_Visium_Human_BreastCancerZ.Targeted_Visium_Human_OvarianCancer_Pan_CancerZ.Targeted_Visium_Human_OvarianCancer_ImmunologyZ!Parent_Visium_Human_OvarianCancerZ4Targeted_Visium_Human_ColorectalCancer_GeneSignatureZ$Parent_Visium_Human_ColorectalCancer)rpr~rcCsTd|krd}nd}t|||d|rBttj|tj|dd}nttj|}|S)u Processed Visium Spatial Gene Expression data from 10x Genomics. Database: https://support.10xgenomics.com/spatial-gene-expression/datasets Parameters ---------- sample_id The ID of the data sample in 10x’s spatial database. include_hires_tiff Download and include the high-resolution tissue image (tiff) in `adata.uns["spatial"][sample_id]["metadata"]["source_image_path"]`. Returns ------- Annotated data matrix. ZV1_z1.1.0z1.2.0)rsru)Zsource_image_path)r|r r r&)rpr~rqr)r"r"r# visium_sgels2 r)rrrr)NF)r})-pathlibrtypingrrinumpyr<ZpandasrLrer packagingrrrWrZ_compatrZ _settingsr Z readwriter r r r__file__rZr9r_floatrrr*r@rOrbrcrmrnrorboolr|rr"r"r"r#st       &"%6* 3