# 1 Overview

In home maintenance, shims are little wedges of wood that you stick into wobbly entities to make them more stable.

We need things like this to deal with diverse data resources in genomics. Here’s an example of the problem:

cm = TxRegInfra:::basicCfieldsMap()
cmdf = as.data.frame(cm)
names(cmdf) = names(cm)
cmdf
##          eQTL    FP         HS
## chrom     chr   chr      chrom
## start snp_pos start chromStart
## end   snp_pos   end   chromEnd

The rownames of this data frame are target annotation terms for features of GRanges: chrom, start, end. The columns are different assay types. Entry i,j is the notation for feature i on assay type j. Thus for eQTL data the start and end can be determined from the source attribute ‘snp_pos’, while for footprints (FP) the footprint end is denoted ‘end’ and for hotspots (HS) the footprint end is denoted ‘chromEnd’.

In order to leave data in its original state but simplify downstream integration, we use shims like basicCfieldsMap to map attribute names to a common vocabulary.

# 2 Application to RaggedMongoExpt instances

We use RaggedMongoExpt instances to work with contents of a remote MongoDB that holds large volumes of genomic annotation.

## 2.1 Construction

con1 = mongo(url=URL_txregInAWS(), db="txregnet")
cd = TxRegInfra::basicColData
rme0 = RaggedMongoExpt(con1, colData=cd)
rme0
## class: RaggedMongoExpt
## dim: 2640 2640
## assays(0):
## rownames: NULL
##   iPS_19_11_DS15153_hg19_FP vHMEC_DS18406_hg19_FP
## colData names(6): base type ... type mid

Here rme0 holds a reference to a MongoDB database, coordinated with the colData component. (The package includes a unit test for correspondence between collection names in the txregnet database and the colData element names.)

## 2.2 Basic motivation

We’ll step back for a moment to give a sense of basic motivations. We want to use MongoDB to manage data about eQTL, DnaseI hypersensitive regions and so forth, without curating the related file contents. Here’s an illustration of the basic functionality for eQTL:

mycon = mongo(db="txregnet",
url=URL_txregInAWS(),   # ATLAS deployment in AWS
collection="Lung_allpairs_v7_eQTL")
mycon\$find(q=rjson::toJSON(list(chr=17)), limit=2)
##             gene_id       variant_id tss_distance ma_samples ma_count      maf
## 1 ENSG00000272636.1 17_36551_T_C_b37         5124        286      367 0.479112
## 2 ENSG00000272636.1 17_36718_A_G_b37         5291        246      305 0.398172
##   pval_nominal    slope  slope_se     qvalue chr snp_pos A1 A2 build
## 1  0.000892921 0.176774 0.0527027 0.06317724  17   36551  T  C   b37
## 2  0.001601720 0.170962 0.0537081 0.09719138  17   36718  A  G   b37

We’ll need different q components for assays of different types, because the internal notation used for chromosomes differs between the assay types. Other aspects of diversified annotation can emerge, and the shim concept helps deliver to the user a more unified interface in the face of this diversity.

## 2.3 The sbov function

At present, the main workhorse for retrieving assay results from RaggedMongoExpt instances is sbov, which is an approach to a subsetByOverlaps functionality. We’ll illustrate this with extractions from lung-related eQTL, Dnase hotspot, and digital genomic footprinting results.

lname_eqtl = "Lung_allpairs_v7_eQTL"
lname_dhs = "ENCFF001SSA_hg19_HS" # see dnmeta
lname_fp = "fLung_DS14724_hg19_FP"
si17 = GenomeInfoDb::Seqinfo(genome="hg19")["chr17"]
si17n = si17
GenomeInfoDb::seqlevelsStyle(si17n) = "NCBI"
s1 = sbov(rme0[,lname_eqtl], GRanges("17", IRanges(38.06e6, 38.15e6),
seqinfo=si17n))
## .
s2 = sbov(rme0[,lname_dhs], GRanges("chr17", IRanges(38.06e6, 38.15e6),
seqinfo=si17))
## .
s3 = sbov(rme0[,lname_fp], GRanges("chr17", IRanges(38.06e6, 38.15e6),
seqinfo=si17))
## .

In principle we could avoid the seqlevelsStyle manipulation by checking assay type within sbov, but at the moment the user must shoulder this responsibility.

To see more about how to work with sbov outputs, check the main vignette.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 1, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.29360997676849365, "perplexity": 18260.13667726976}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2021-04/segments/1610704839214.97/warc/CC-MAIN-20210128071759-20210128101759-00143.warc.gz"}