Results 1 to 7 of 7

Thread: How to get AdmixTools running

  1. #1
    Registered Users
    Posts
    232
    Sex
    Location
    Papantla, Mexico

    Ireland Ireland England England Ireland Munster European Union

    How to get AdmixTools running

    A little guide on how to get started with the gold standard program used by the leading population geneticists. With this you'll be able to run D-stats to find evidence of admixture as well as qpAdm for admixture coefficients, as well as the other tools included in the qpAdm package. It's actually quite easy once you know where to get all the components from. A big obstacle for some may be having to use a linux machine but here I'll show you how to run Linux from usb,meaning no change at all to your OS,files etc.

    Downloading required programs

    Linux

    Some Linux distributions have problems with AdmixTools. Fedora25 is one that definitely works.

    1) If you want to run from USB, making sure you have a clean USB (all existing files will be erased after Fedora is written to it) go to: https://getfedora.org/en/workstation/download/

    Haven’t tried versions newer than 25, so if you want that, click on link below download button where it says ‘Need instructions? Or a different version’

    2) A media writer will download so you can turn your usb into a linux boot device. From here it’s self-explanatory

    AdmixTools

    From here: https://github.com/DReichLab/AdmixTools click the green ‘clone or download’ button. This will download an AdmixTools-master zipfile. Unzip and put on a separate USB. Create a new folder inside and call it ‘bin’
    You also need to download the specific programs that will actually run what you want to do. One this same page, click the ‘releases’ tab and you’ll see here all the diff programs that AdmixTools can run. Download one of these files and put in the AdmixTools-master bin file

    Datasets: Can be run straight out of the box with example data from here: https://reich.hms.harvard.edu/software


    Or alternatively, download one of the recent datasets from here:
    https://reich.hms.harvard.edu/datasets

    Once in Linux-running the data examples in D-stats

    Starting off in D-stats as it’s the first thing I learned to do and is pretty straightforward. Navigate to your USB with all the AdmixTools stuff and find the Data folder downloaded before. In this folder you’ll find 3 important files: allmap.ind allmap.geno and allmap.snp.These are EIGENSTRAT files and are the filetypes that the program works with. Copy or move these into your AdmixTools-master/bin folder

    In the main AdmixTools-master folder, go into the ‘examples’ subfolder and find a file called ‘parqpDstat’. This file is in the format needed to run qpDstat. The file cannot be used as is so you need to fill it with your own information. This is the parameter file or ‘parfile’


    DIR: here you must enter the path to the directory where your .ind/geno/snp files are located
    SSS: enter just the file name(in this case allmap)
    indivname: put here directory path to the folder where the .ind file is located
    snpname: put here directory path to the folder where the .ind file is located
    genotypename: put here directory path to the folder where the .geno file is located
    poplistname: just the name of the textfile with the populations that you want to run

    The poplistname is just a text file with 4 pops, one on each line, that you want to run with:

    Mbuti
    Uygur
    French
    Han

    You can also run multiple D-stats at one time (number depending on machine memory). If you want to do this, replace ‘poplistname’ with ‘popfilename’ and create a text file with 4 pops on each line

    3) Once the file is filled out, it should look something like this:

    DIR: /run/media/liveuser/0E3F552A/AdmixTools-master/bin
    SSS: allmap
    indivname: /run/media/liveuser/0E3F552A/AdmixTools-master/bin/allmap.ind
    snpname: /run/media/liveuser/0E3F552A/AdmixTools-master/bin/allmap.snp
    genotypename: /run/media/liveuser/0E3F552A/AdmixTools-master/bin/allmap.geno
    popfilename: list_qpDstatmultiple

    Save as something like: parqpDstat1.

    4) This should also be in the bin folder. About the .geno/snp/ind folders, don’t touch any folder apart from the .ind folder. This contains the individual names of each sample. It’s a good idea to use the example data files here just to get a feel for the program because all samples here are named. When merging later you will have to rename a lot of samples as during the merging process, the individual names, while keeping their ids, are renamed to a generic ‘control’. It’s this name-not the id-that you are needed to specify when making parfiles. Now ready for the first run

    Running the program

    Open up a terminal. This is a command line window that Linux uses. If no icon is present, search for it in the search box.
    First we need to make sure we are in the directory where admixtools-master/bin is located. To do this easily, right click a file from this folder and copy the file path.

    Type in ‘cd’ and then paste file path. Press enter. The input here should look like this:
    cd /run/media/liveuser/0E3F552A/AdmixTools-master/bin

    Re-paste the file path, but this time, add ‘qpDstat -p parqpDstat1 [-l lo] [-h hi] >logfile

    so:

    cd /run/media/liveuser/0E3F552A/AdmixTools-master/bin/qpDstat -p parqpDstat1 [-l lo] [-h hi] >logfile

    Press enter and the program should run. It will look like it has frozen but after a few minutes the run will complete and the output will be written to Admix-Tools-master/bin

    In some cases a warning message may appear about a missing package. Commonly ‘libblas’ or ‘liblapack’. To download both of them, make sure you are connected to internet (bluetooth tethering with smartphone also works) and type in the terminal:

    Sudo dnf install lapack-devel

    (in some versions ‘yum’ replaces ‘dnf’)

    The necessary components will download. For me I have to do this everytime I use qpAdm, as I'm using Linux on a USB, and is a real pain but it takes 30 mins and for every run after that it's fine.

    qpAdm

    qpWave is supposed to be used before qpAdm, to verify the number of admixing populations into the target pop. To be honest I never used it and got results that were consistent with published papers but it should be used before.

    As to running qpAdm, same idea as with qpDstat. All the commands for running qpAdm will be found in the material downloaded from reich lab. The Right pops list is essential for qpAdm, make sure you pick pops from every possible contributing admixtureangle in relation to the donor pops in the left pop list. So for modelling a European target, this means right pops like WHGs,EHGs,Anatolia_N,CHG,Levant_N, Iran_N,Natufians,Kostenki,Ust-Ishim and possibly any more that are relevant to European genetic pre-history to get every possible admixture angle covered, as well as Mota as a base,going at the top of the right pop list (mentioned once in a paper-I think Lazaridis 2016 Basal Eurasian-that it is important to have Mota as a base)
    Last edited by Bas; 12-12-2018 at 08:52 PM.

  2. The Following 6 Users Say Thank You to Bas For This Useful Post:

     digital_noise (03-20-2019),  E_M81_I3A (03-28-2019),  Kale (03-17-2019),  NiloSaharan (03-24-2019),  poi (03-21-2019),  TuaMan (12-13-2018)

  3. #2
    Registered Users
    Posts
    1,364
    Sex
    Omitted

    I have both gsl and openblas installed in Fedora25 (via "dnf install gsl" and "dnf install openblas"), which are needing for compiling AdmixTools. During the process of compiling though, gsl and openblas cannot be found. What gives?
    The binaries on github are not the newest versions, to get the newest versions you have to compile them yourself from what I understand.
    Also, the qpgraph program needs one or both of those things to run as well, and can't find them.
    Why can't Linux things ever just work?
    Last edited by Kale; 03-19-2019 at 03:28 AM.
    Collection of 14,000 d-stats: Hidden Content Part 2: Hidden Content Part 3: Hidden Content PM me for d-stats, qpadm, qpgraph, or f3-outgroup nmonte models.

  4. The Following User Says Thank You to Kale For This Useful Post:

     Bas (03-19-2019)

  5. #3
    Registered Users
    Posts
    1,364
    Sex
    Omitted

    To follow up on the above issue. If 'installing' gsl straight from online or whatever doesn't work, try downloading the source files and build/compile it yourself, that worked for me. I can now confirm AdmixTools can be compiled & run on Debian9.8.0 as well.
    Collection of 14,000 d-stats: Hidden Content Part 2: Hidden Content Part 3: Hidden Content PM me for d-stats, qpadm, qpgraph, or f3-outgroup nmonte models.

  6. The Following 3 Users Say Thank You to Kale For This Useful Post:

     anglesqueville (03-21-2019),  Bas (03-22-2019),  NiloSaharan (03-24-2019)

  7. #4
    Moderator
    Posts
    5,845
    Sex
    Location
    Normandy
    Ethnicity
    northwesterner
    mtDNA (M)
    H5a1
    Y-DNA (P)
    R-BY3604-Z275

    Normandie Netherlands Friesland Finland Orkney
    Quote Originally Posted by Kale View Post
    To follow up on the above issue. If 'installing' gsl straight from online or whatever doesn't work, try downloading the source files and build/compile it yourself, that worked for me. I can now confirm AdmixTools can be compiled & run on Debian9.8.0 as well.
    No problem at all with Debian 9.8? No issue of library sharing? I ask this question because I'm intending to buy a new system devoted to Linux and genetics only in the upcoming months, and I'm wondering about the Linux distribution. If Debian works perfectly, including with the updated versions of Admixtools, it's for me a very good news. Is it true with Eigensoft as well?
    En North alom, de North venom
    En North fum naiz, en North manom

    (Roman de Rou, Wace, 1160-1170)

  8. The Following User Says Thank You to anglesqueville For This Useful Post:

     JMcB (03-28-2019)

  9. #5
    Registered Users
    Posts
    1,364
    Sex
    Omitted

    I have not tried the newest version of admixtools. I ran the script available here...
    http://docs.hpc.shef.ac.uk/en/latest...dmixtools.html
    But out of the 6 steps you'll see at the bottom of the page
    1) Just downloads admixtools from github, anyone can do that
    2) Installs GSL, it didn't do that right so I had to do it manually
    3-4) Builds & installs admixtools... I don't remember if the script did that or I did it manually
    5-6) nothing really
    Also, what's eigensoft, what does that do?
    Collection of 14,000 d-stats: Hidden Content Part 2: Hidden Content Part 3: Hidden Content PM me for d-stats, qpadm, qpgraph, or f3-outgroup nmonte models.

  10. #6
    Moderator
    Posts
    5,845
    Sex
    Location
    Normandy
    Ethnicity
    northwesterner
    mtDNA (M)
    H5a1
    Y-DNA (P)
    R-BY3604-Z275

    Normandie Netherlands Friesland Finland Orkney
    Eigensoft is Patterson's package that contains smartpca and convertf (this one is precious for the conversions between usual formats, in particular eigenstrat and plink/packedped). About the newest versions of admixtools: I installed them without any problem on fedora25, but was unable to do the same on Helgenes' fedora27 (or 28,don't remember), because of very painful issues of shared libraries. If I build a new system on a very recent PCU (16 cores, perhaps 32, depending on the price of the AMD new processors which are awaited for the upcoming summer), I believe that it will not be a good idea to choose an outdated distribution of Linux. On the other hand, if I choose an updated distribution but am unable to install admixtools on it... well, some months of reflexion.
    En North alom, de North venom
    En North fum naiz, en North manom

    (Roman de Rou, Wace, 1160-1170)

  11. The Following 2 Users Say Thank You to anglesqueville For This Useful Post:

     Helgenes50 (03-28-2019),  JMcB (03-28-2019)

  12. #7
    Registered Users
    Posts
    1,364
    Sex
    Omitted

    For convertf I always just used the binary from github which worked out-of-box on Fedora25. I can't imagine convertf requires many dependencies.
    When I get some time, which will probably be next Sunday, I'll try and get the newest version of admixtools working and give convertf a run.

    Hmm... off-topic maybe maybe not. Weird issue with qpgraph. Working with a single graph file for all instances...
    1) Original: worst outlier -3.297
    2) Turned of lsqmode in the parfile: worst outlier -3.328 (basically no change)
    3) Turned lsqmode back on, changed initmix from 1000 to 3000: worst outlier 13.973, proportions obviously wrong in multiple instances.
    4) Changed initmix back to 1000 (all settings now back to original): result is not back to original, still looks like 3.
    It's messed up in a way I have seen happen before when putting edges in certain places that are too unanchored maybe? for it to process right.

    Ok nevermind... if you run into that issue close the terminal I guess and open a new one?
    Last edited by Kale; 03-29-2019 at 05:16 AM.
    Collection of 14,000 d-stats: Hidden Content Part 2: Hidden Content Part 3: Hidden Content PM me for d-stats, qpadm, qpgraph, or f3-outgroup nmonte models.

Similar Threads

  1. How to run Admixtools
    By Bas in forum Inquiries Corner
    Replies: 17
    Last Post: 10-13-2017, 09:30 PM
  2. Plink, Admixtools ( and their friends)
    By anglesqueville in forum Autosomal (auDNA)
    Replies: 7
    Last Post: 05-24-2017, 01:39 PM
  3. Admixtools & Ubuntu
    By anglesqueville in forum Inquiries Corner
    Replies: 23
    Last Post: 03-02-2017, 05:42 PM
  4. Replies: 6
    Last Post: 05-06-2014, 05:25 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •