U vId@sddlmZddlmZddlmZddlmZddlmZm Z m Z ddl Z ddl Z ddlmZd d lmZmZz$ddlZdd lmZmZmZmZWn(ek reeeef\ZZZZYnXGd d d eZddZddZGdddeZdS))issparse)ceil)copy)partial)DictUnionSequenceN)AnnData) AnnCollection_ConcatViewMixin)Sampler BatchSamplerDataset DataLoaderc@s&eZdZd ddZddZddZdS) BatchIndexSamplerFcCs(||_||kr|n||_||_||_dSN)n_obs batch_sizeshuffle drop_last)selfrrrrr`/home/sam/Atlas/atlas_env/lib/python3.8/site-packages/anndata/experimental/pytorch/_annloader.py__init__szBatchIndexSampler.__init__ccsx|jrtj|j}ntt|j}td|j|jD]:}||t ||j|j}t ||jkrl|j rlq8|Vq8dS)Nr) rnprandomZ permutationrtolistlistrangerminlenr)rindicesibatchrrr__iter__szBatchIndexSampler.__iter__cCs(|jr|j|j}nt|j|j}|Sr)rrrr)rlengthrrr__len__-szBatchIndexSampler.__len__N)FF)__name__ __module__ __qualname__rr&r(rrrrrs rcCst|tjr(|r|}q|r|}nZ|jjdkrt|jtj rt |rT| }|rhtj |dd}nt |}|r~|n|}|S)Ncategorycuda)Zdevice) isinstancetorchZTensorr- pin_memoryZdtypenamerZ issubdtypenumberrZtoarrayZtensor)arruse_cudar0rrrdefault_converter7s    r5cszdkr}nhtr*fdd}|}nLi}|D]B}|krH||<q2t|trXd}n||}t||||<q2|S)Ncs |Srr)r3convert top_convertrrcompose_convertMsz(_convert_on_top..compose_convert)callabler.r_convert_on_top)r7r8 attrs_keysZ new_convertr9attrZas_ksrr6rr;Hs  r;csDeZdZdZdeeeeeeffe e e e dfdd Z Z S) AnnLoadera PyTorch DataLoader for AnnData objects. Builds DataLoader from a sequence of AnnData objects, from an :class:`~anndata.experimental.AnnCollection` object or from an `AnnCollectionView` object. Takes care of the required conversions. Parameters ---------- adatas `AnnData` objects or an `AnnCollection` object from which to load the data. batch_size How many samples per batch to load. shuffle Set to `True` to have the data reshuffled at every epoch. use_default_converter Use the default converter to convert arrays to pytorch tensors, transfer to the default cuda device (if `use_cuda=True`), do memory pinning (if `pin_memory=True`). If you pass an AnnCollection object with prespecified converters, the default converter won't overwrite these converters but will be applied on top of them. use_cuda Transfer pytorch tensors to the default cuda device after conversion. Only works if `use_default_converter=True` **kwargs Arguments for PyTorch DataLoader. If `adatas` is not an `AnnCollection` object, then also arguments for `AnnCollection` initialization. FT)adatasrruse_default_converterr4c st|tr|g}t|ts.t|ts.t|tr|dd}|dd}|dd} |dd} |dd} |dd} |dd } |d d }t|||| | | | | |d }nt|trt|}nt d |r|d d}t t ||d}t |j |t|jgd|_ d|k}d|k}d|ko"|ddk }d|ko8|ddk}|pB|}|dk r|dkr|s|s|dd}|r|d}t|||d}ntt||||}tj|fd|d|ntj|f||d|dS)Njoin_obsinner join_obsmlabelkeys index_uniquer7harmonize_dtypesTindices_strict)rBrDrErFrGr7rHrIz1adata should be of type AnnData or AnnCollection.r0F)r4r0)XsamplerZ batch_samplerZworker_init_fnZ num_workersrr?r)rr)rrK)rr)r.r rtupledictpopr r r ValueErrorrr5r;r7r<rrr"superr)rr@rrrAr4kwargsrBrDrErFrGr7rHrIZdatasetr0Z _converterZ has_samplerZhas_batch_samplerZhas_worker_init_fnZ has_workersZ use_parallelrrK __class__rrr}s                zAnnLoader.__init__)r?FTF) r)r*r+__doc__rrr rstrintboolr __classcell__rrrRrr>`sr>)Z scipy.sparsermathrr functoolsrtypingrrrnumpyrwarningsZ _core.anndatar Zmulti_files._anncollectionr r r/Ztorch.utils.datarrrr ImportErrorobjectrr5r;r>rrrrs"