Integrate scRNA-seq datasets#
!lamin load test-scrna
Show code cell output
💡 found cached instance metadata: /home/runner/.lamin/instance--testuser1--test-scrna.env
✅ loaded instance: testuser1/test-scrna
import lamindb as ln
import lnschema_bionty as lb
import pandas as pd
import anndata as ad
✅ loaded instance: testuser1/test-scrna (lamindb 0.51.0)
ln.track()
💡 notebook imports: anndata==0.9.2 lamindb==0.51.0 lnschema_bionty==0.30.0 pandas==1.5.3
✅ saved: Transform(id='agayZTonayqAz8', name='Integrate scRNA-seq datasets', short_name='scrna2', version='0', type=notebook, updated_at=2023-08-28 13:51:37, created_by_id='DzTjkKse')
✅ saved: Run(id='cMROWvjBDVTAPpvUiwf5', run_at=2023-08-28 13:51:37, transform_id='agayZTonayqAz8', created_by_id='DzTjkKse')
Query files based on metadata#
# lookup objects for auto-complete
assays = lb.ExperimentalFactor.lookup()
species = lb.Species.lookup()
query = ln.File.filter(
experimental_factors=assays.single_cell_rna_sequencing, # scRNA-seq
species=species.human, # human
cell_types__name__contains="monocyte", # monocyte
).distinct()
query.df()
| storage_id | key | suffix | accessor | description | version | initial_version_id | size | hash | hash_type | transform_id | run_id | updated_at | created_by_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||||
| L7srPtuIfV1AWTBQTWYo | 7gYw68gC | None | .h5ad | AnnData | Conde22 | None | None | 28049505 | WEFcMZxJNmMiUOFrcSTaig | md5 | Nv48yAceNSh8z8 | ujzl8FtsURX7meXQWLrn | 2023-08-28 13:51:20 | DzTjkKse |
| SIdlfiN2VEwYVeGfIcBS | 7gYw68gC | None | .h5ad | AnnData | 10x reference pbmc68k | None | None | 589484 | eKVXV5okt5YRYjySMTKGEw | md5 | Nv48yAceNSh8z8 | ujzl8FtsURX7meXQWLrn | 2023-08-28 13:51:31 | DzTjkKse |
Intersect measured genes between two datasets#
# get file objects
file1, file2 = query.list()
file1.describe()
💡 File(id='L7srPtuIfV1AWTBQTWYo', key=None, suffix='.h5ad', accessor='AnnData', description='Conde22', version=None, size=28049505, hash='WEFcMZxJNmMiUOFrcSTaig', hash_type='md5', created_at=2023-08-28 13:51:20, updated_at=2023-08-28 13:51:20)
Provenance:
🗃️ storage: Storage(id='7gYw68gC', root='/home/runner/work/lamin-usecases/lamin-usecases/docs/test-scrna', type='local', updated_at=2023-08-28 13:51:35, created_by_id='DzTjkKse')
📔 transform: Transform(id='Nv48yAceNSh8z8', name='Validate & register scRNA-seq datasets', short_name='scrna', version='0', type='notebook', updated_at=2023-08-28 13:51:31, created_by_id='DzTjkKse')
👣 run: Run(id='ujzl8FtsURX7meXQWLrn', run_at=2023-08-28 13:50:41, transform_id='Nv48yAceNSh8z8', created_by_id='DzTjkKse')
👤 created_by: User(id='DzTjkKse', handle='testuser1', email='testuser1@lamin.ai', name='Test User1', updated_at=2023-08-28 13:51:35)
Features:
var (X):
🔗 index (36503, bionty.Gene.id): ['0tqWIZ0EwOF6', 'rT0Xjh7mbeht', '3WrrzHSaNKiX', 'S0H0s3WM12iQ', 'z7loK3Eqm6rq'...]
obs (metadata):
🔗 cell_type (32, bionty.CellType): ['naive B cell', 'effector memory CD4-positive, alpha-beta T cell', 'regulatory T cell', 'animal cell', 'gamma-delta T cell']
🔗 assay (4, bionty.ExperimentalFactor): ["10x 5' v2", "10x 3' v3", "10x 5' v1", 'single-cell RNA sequencing']
🔗 tissue (17, bionty.Tissue): ['lamina propria', 'blood', 'duodenum', 'bone marrow', 'spleen']
🔗 donor (12, core.Label): ['621B', 'A29', 'A35', '637C', 'A36']
file1.view_lineage()
file2.describe()
💡 File(id='SIdlfiN2VEwYVeGfIcBS', key=None, suffix='.h5ad', accessor='AnnData', description='10x reference pbmc68k', version=None, size=589484, hash='eKVXV5okt5YRYjySMTKGEw', hash_type='md5', created_at=2023-08-28 13:51:31, updated_at=2023-08-28 13:51:31)
Provenance:
🗃️ storage: Storage(id='7gYw68gC', root='/home/runner/work/lamin-usecases/lamin-usecases/docs/test-scrna', type='local', updated_at=2023-08-28 13:51:35, created_by_id='DzTjkKse')
📔 transform: Transform(id='Nv48yAceNSh8z8', name='Validate & register scRNA-seq datasets', short_name='scrna', version='0', type='notebook', updated_at=2023-08-28 13:51:31, created_by_id='DzTjkKse')
👣 run: Run(id='ujzl8FtsURX7meXQWLrn', run_at=2023-08-28 13:50:41, transform_id='Nv48yAceNSh8z8', created_by_id='DzTjkKse')
👤 created_by: User(id='DzTjkKse', handle='testuser1', email='testuser1@lamin.ai', name='Test User1', updated_at=2023-08-28 13:51:35)
Features:
var (X):
🔗 index (695, bionty.Gene.id): ['VPG6Ybxhk9ss', 'zOUVvOZ5PDec', '3z0yr6iybn0l', 'R0KxhGBHlynU', 'VSc0IwLJsfrD'...]
external:
🔗 assay (1, bionty.ExperimentalFactor): ['single-cell RNA sequencing']
🔗 species (1, bionty.Species): ['human']
obs (metadata):
🔗 cell_type (9, bionty.CellType): ['conventional dendritic cell', 'CD8-positive, CD25-positive, alpha-beta regulatory T cell', 'CD14-positive, CD16-negative classical monocyte', 'dendritic cell', 'cytotoxic T cell']
file2.view_lineage()
Load files into memory:
file1_adata = file1.load()
file2_adata = file2.load()
💡 adding file L7srPtuIfV1AWTBQTWYo as input for run cMROWvjBDVTAPpvUiwf5, adding parent transform Nv48yAceNSh8z8
💡 adding file SIdlfiN2VEwYVeGfIcBS as input for run cMROWvjBDVTAPpvUiwf5, adding parent transform Nv48yAceNSh8z8
Here we compute shared genes without loading files:
file1_genes = file1.features["var"]
file2_genes = file2.features["var"]
shared_genes = file1_genes & file2_genes
len(shared_genes)
695
shared_genes.list("symbol")[:10]
['BTG1',
'EFHD2',
'TNFSF13B',
'SFPQ',
'GSTP1',
'POP5',
'ELOVL5',
'GIMAP7',
'CFP',
'PPIA']
We also need to convert the ensembl_gene_id to symbol for file2 so that they can be concatenated:
mapper = pd.DataFrame(shared_genes.values_list("ensembl_gene_id", "symbol")).set_index(
0
)[1]
mapper.head()
0
ENSG00000133639 BTG1
ENSG00000142634 EFHD2
ENSG00000102524 TNFSF13B
ENSG00000116560 SFPQ
ENSG00000084207 GSTP1
Name: 1, dtype: object
file2_adata.var.rename(index=mapper, inplace=True)
Intersect cell types#
file1_celltypes = file1.cell_types.all()
file2_celltypes = file2.cell_types.all()
shared_celltypes = file1_celltypes & file2_celltypes
shared_celltypes_names = shared_celltypes.list("name")
shared_celltypes_names
['conventional dendritic cell',
'CD16-positive, CD56-dim natural killer cell, human']
We can now subset the two datasets by shared cell types:
file1_adata_subset = file1_adata[
file1_adata.obs["cell_type"].isin(shared_celltypes_names)
]
file2_adata_subset = file2_adata[
file2_adata.obs["cell_type"].isin(shared_celltypes_names)
]
Concatenate subseted datasets:
adata_concat = ad.concat(
[file1_adata_subset, file2_adata_subset],
label="file",
keys=[file1.description, file2.description],
)
adata_concat
AnnData object with n_obs × n_vars = 126 × 0
obs: 'cell_type', 'file'
obsm: 'X_umap'
adata_concat.obs.value_counts()
cell_type file
CD16-positive, CD56-dim natural killer cell, human Conde22 114
conventional dendritic cell Conde22 7
CD16-positive, CD56-dim natural killer cell, human 10x reference pbmc68k 3
conventional dendritic cell 10x reference pbmc68k 2
dtype: int64
Show code cell content
# clean up test instance
!lamin delete --force test-scrna
!rm -r ./test-scrna
💡 deleting instance testuser1/test-scrna
✅ deleted instance settings file: /home/runner/.lamin/instance--testuser1--test-scrna.env
✅ instance cache deleted
✅ deleted '.lndb' sqlite file
❗ consider manually deleting your stored data: /home/runner/work/lamin-usecases/lamin-usecases/docs/test-scrna