variant-mapper TLDR;#

Here is a quick overview of what you would need to do to get doing.

Install the package#

This can be installed using Pypi or conda

To install using pypi:

pip install variant-mapper

To install using conda:

conda install -c cfin -c conda-forge variant-mapper

Get the mapping files#

In addition to installing you will need to download one or more mapping files (these are on my pCloud so please don’t hammer it).

The common mapping file can be downloaded (~9G): mapping-v20220402.
Contact me for the full mapping file can be downloaded (~60G).

You can also build them from scratch if you want. Click the links to learn more about what is in the mapping files and the mapping process.

Run on the command line#

You can use the variant-mapper via the command line or the API. The full command line options are given in the command line docs and examples are given below.

To mapping a large file, use the scan mapper. Your input must be sorted on chromosome (as a string) and start position (as an integer).

variant-map-scan -vv -i input-file.txt.gz \
                 --chr-name 'chr_name' \
                 --start-pos 'start_pos' \
                 --ref-allele 'other_allele' \
                 --alt-allele 'effect_allele' \
                 /data/mapping-file.v20220402.common.biallelic.vep.b38.vcf.gz > ~/scan-test.txt

If you have a large file that is not presorted, you could do something like this. Add this function to your ~/bashrc

# The body command to pass over a file header
body() {
    IFS= read -r header
    printf '%s\n' "$header"
    "$@"
}
export -f body

Then, source in . ~/.bashrc and you can sort into the variant mapper:

# chr_name is column 1 and start_pos is column 2
# body passes the header over sort into variant-map-scan
zcat input-file.txt.gz |
    body sort -t$'\t' -k1,1 -k2n,2 |
    header -n1001 |
    variant-map-scan -vv --chr-name 'chr_name' \
                 --start-pos 'start_pos' \
                 --ref-allele 'other_allele' \
                 --alt-allele 'effect_allele' \
                 /data/mapping-file.v20220402.common.biallelic.vep.b38.vcf.gz > ~/scan-test.txt
=== variant-mapper-scan (variant_mapper v0.1) ===
[info] 13:52:10 > aaf_method value: mean
[info] 13:52:10 > alt_allele value: other_allele
[info] 13:52:10 > chr_name value: chr_name
[info] 13:52:10 > chr_pos_spec value: None
[info] 13:52:10 > chunksize value: 100000
[info] 13:52:10 > comment_char value: ##
[info] 13:52:10 > debug value: None
[info] 13:52:10 > decode_map_info value: False
[info] 13:52:10 > delimiter value:
[info] 13:52:10 > infile value: None
[info] 13:52:10 > list_pops value: False
[info] 13:52:10 > mapping_file value: /data/mapping.v20220402.common.biallelic.vep.b38.vcf.gz
[info] 13:52:10 > no_sort value: True
[info] 13:52:10 > outfile value: None
[info] 13:52:10 > pops value: None
[info] 13:52:10 > ref_allele value: effect_allele
[info] 13:52:10 > ref_assembly value: None
[info] 13:52:10 > start_pos value: start_pos
[info] 13:52:10 > strand value: None
[info] 13:52:10 > tabix value: None
[info] 13:52:10 > tmp_dir value: None
[info] 13:52:10 > var_id value: None
[info] 13:52:10 > vcf value: False
[info] 13:52:10 > verbose value: True
[info] 13:52:10 > weights length: 0
[info] mapping input file: 1000 input rows [00:01, 642.90 input rows/s]
[info] 13:52:12 > *** END ***

Map an unsorted input file using the tabix mapper.

# Map the first 1000 rows, you can also spcifiy -i
zcat input-file.txt.gz |
   head -n 1000 |
   variant-map-tabix -vv --chr-name 'chr_name' \
                     --start-pos 'start_pos' \
                     --ref-allele 'effect_allele' \
                     --alt-allele 'other_allele' \
                     /data/gwas_norm.v20220402.biallelic.vep.b38.vcf.gz > ~/tabix-test.txt
=== variant-mapper-tabix (variant_mapper v0.1) ===
[info] 14:15:01 > aaf_method value: mean
[info] 14:15:01 > alt_allele value: other_allele
[info] 14:15:01 > chr_name value: chr_name
[info] 14:15:01 > chr_pos_spec value: None
[info] 14:15:01 > comment_char value: ##
[info] 14:15:01 > debug value: None
[info] 14:15:01 > decode_map_info value: False
[info] 14:15:01 > delimiter value:
[info] 14:15:01 > infile value: None
[info] 14:15:01 > list_pops value: False
[info] 14:15:01 > mapping_file value: /data/mapping.v20220402.biallelic.vep.b38.vcf.gz
[info] 14:15:01 > outfile value: None
[info] 14:15:01 > pops value: None
[info] 14:15:01 > ref_allele value: effect_allele
[info] 14:15:01 > ref_assembly value: None
[info] 14:15:01 > start_pos value: start_pos
[info] 14:15:01 > strand value: None
[info] 14:15:01 > tmp_dir value: None
[info] 14:15:01 > var_id value: None
[info] 14:15:01 > verbose value: True
[info] 14:15:01 > weights length: 0
[info] mapping input file: 1000 input rows [00:02, 355.51 input rows/s]
[info] 14:15:04 > *** END ***

If you have no mapping files, then you can use the Ensembl mapper:

# Map the first 100 rows, mapping again Ensembl is very slow
zcat input-file.txt.gz |
   head -n 100 |
   variant-map-ensembl -vv --chr-name 'chr_name' \
                       --start-pos 'start_pos' \
                       --ref-allele 'effect_allele' \
                       --alt-allele 'other_allele' -vv > ~/ensembl-test.txt
=== variant-mapper-ensembl (variant_mapper v0.1) ===
[info] 14:20:52 > aaf_method value: mean
[info] 14:20:52 > alt_allele value: other_allele
[info] 14:20:52 > chr_name value: chr_name
[info] 14:20:52 > chr_pos_spec value: None
[info] 14:20:52 > comment_char value: ##
[info] 14:20:52 > debug value: None
[info] 14:20:52 > decode_map_info value: False
[info] 14:20:52 > delimiter value:
[info] 14:20:52 > infile value: None
[info] 14:20:52 > list_pops value: False
[info] 14:20:52 > outfile value: None
[info] 14:20:52 > pops value: None
[info] 14:20:52 > ref_allele value: effect_allele
[info] 14:20:52 > ref_assembly value: None
[info] 14:20:52 > rest_url value: https://rest.ensembl.org
[info] 14:20:52 > start_pos value: start_pos
[info] 14:20:52 > strand value: None
[info] 14:20:52 > tmp_dir value: None
[info] 14:20:52 > var_id value: None
[info] 14:20:52 > verbose value: True
[info] 14:20:52 > weights length: 0
[info] mapping input file: 100 input rows [01:36,  1.04 input rows/s]
[info] 14:22:28 > *** END ***

Run usung Python using the API#

There is an API for mapping in your own Python code. It is documented in the API docs. There are also some example notebooks in the cloned repo, at ./resources/examples. These can also be seen in the example documentation.

Using the Bash scripts#

If you clone the repository there are various Bash scripts in ./resources/bin, these are mostly used in the creation of the mapping files and most people will not be interested in them. However, if you are and want to use them you will need to add the full path to ./resources/bin to you PATH in your ~/.bashrc (where . is the root of the cloned git repository). These scripts all also need the `bash-helpers <https://gitlab.com/cfinan/bash-helpers`_ repo in your PATH and also shflags <https://github.com/kward/shflags>`_ (which is a very nice Bash command line argument handler). To put something in your PATH involves editing either your ~/.bashrc file or your ~/.bash_profile file (depending on what you use). For example, for adding the bash scripts for building the mapping files you should add something like this:

`bash PATH="/path/to/gwas-norm/resources/bin:${PATH}" export PATH `

where /path/to/ is where you cloned the repository. If you didn’t clone the repository and installed via conda, then it is easiest just to clone it (but not install it with pip) and just use the ./resources/bin scripts.

variant-mapper TLDR;

Contents