Page 1 of 18 12311 ... LastLast
Results 1 to 10 of 176

Thread: Automated qpGraph: possibilities for the consideration of technically savvy Users?

  1. #1
    Registered Users
    Posts
    28
    Sex
    Omitted

    Automated qpGraph: possibilities for the consideration of technically savvy Users?

    David (of Eurogenes fame) recently posted something of considerable interest:

    https://eurogenes.blogspot.com/2020/...ploration.html

    Rather significant, because it makes this type of analysis quite accessible for the rest of us. I myself dived in immediately... but alas, my laptop does not have enough juice! (I did have a desktop (a few months back) that would've been perfect for this)

    I know that this forum is home to multiple individuals with a solid grasp of qpGraph (Kale, Chad, David/Generalissimo, etc), but I'm also quite aware that manually exploring topologies is one hell of a pain (and perhaps somewhat problematic from a conceptual standpoint).

    With this no longer being an issue, can the experts here say whether any among them could attempt a comprehensive qpGraph topology that involves a substantive sampling of the aDNA record (with respect to Paleolithic/Mesolithic/Neolithic samples)?

    When I was attempting this myself, my goal was an automated graph involving:

    Archaic Humans: Neanderthal, Denisovan

    Africans: Shum_Laka_8000BP, Taforalt

    Broadly Eurasian: Ust_Ishim

    Broadly "West Eurasian": Kostenki14, Sunghir, GoyetQ116-1, Villabruna, Yana, MA1

    ENA: Tianyuan, LAO_Hoabinhian, MNG_North_N

    Near Eastern: Natufians, Pinarbasi_HG, CHG (Kotias + Satsurblia), Iran_N (Ganj_Dareh), Sarazm_EN, Shahr_I_Sokhta_BA2 (IVC-related)

    If anyone with the means could attempt this, that'd be pretty awesome.

  2. The Following 10 Users Say Thank You to durmang For This Useful Post:

     Angoliga (01-11-2021),  beyoku (01-01-2021),  David Bush (01-01-2021),  discreetmaverick (01-08-2021),  jadegreg (01-29-2021),  Justnotyou (01-15-2021),  Mnemonics (01-01-2021),  opacity (03-19-2021),  sheepslayer (01-02-2021),  theplayer (01-01-2021)

  3. #2
    Registered Users
    Posts
    2,070
    Sex
    Omitted

    You don't know how bad I'm wanting to do this! I just can't figure out how to get it up and running.
    Right now I'm stuck at why does qpgraph need block_lengths.rds if F2 statistics are already computed?

    Basically I have 1500 lines of this (output from qpfstat)... Linux command-line qpgraph understands this, how do I make R-qpgraph understand?
     

    Neanderthal.DG Neanderthal.DG 119.340
    Neanderthal.DG Mbuti.DG 61.162
    Neanderthal.DG Dinka.DG 61.136
    Neanderthal.DG Mende.DG 61.376
    Neanderthal.DG Mota.SG 61.233
    Neanderthal.DG South_Africa_2000BP.SG 61.447
    Neanderthal.DG Cameroon_ShumLaka_8000BP.DG 61.200
    Neanderthal.DG Taforalt 62.099
    Neanderthal.DG Loschbour.DG 62.937
    Neanderthal.DG Onge.DG 62.926
    Mbuti.DG Mbuti.DG 158.395
    Mbuti.DG Dinka.DG 151.343
    Mbuti.DG Mende.DG 148.330
    Mbuti.DG Mota.SG 147.789
    Mbuti.DG South_Africa_2000BP.SG 140.026
    Last edited by Kale; 01-01-2021 at 07:02 AM.
    Collection of 14,000 d-stats: Hidden Content Part 2: Hidden Content Part 3: Hidden Content PM me for d-stats, qpadm, qpgraph, or f3-outgroup nmonte models.

  4. The Following 3 Users Say Thank You to Kale For This Useful Post:

     durmang (01-01-2021),  Mnemonics (01-01-2021),  sheepslayer (01-02-2021)

  5. #3
    Registered Users
    Posts
    28
    Sex
    Omitted

    Quote Originally Posted by Kale View Post
    You don't know how bad I'm wanting to do this! I just can't figure out how to get it up and running.
    We're undoubtedly in the same boat! It's an exciting enhancement of qpGraph (and I have so many ideas that I'm chomping at the bit to explore).

    I'm gonna keep at it, until I figure out a way. But my problem is a bit more basic than yours: my computer goes black and becomes unresponsive, when I try to get the f2 blocks. Lol

    In the meantime, my hope is that some kind soul finds enough mercy in their heart to do my work for me (by attempting an admixture graph involving the populations that I've listed).

  6. #4
    Registered Users
    Posts
    247
    Sex
    Location
    Canada
    Ethnicity
    Somali
    Y-DNA (P)
    T-L208
    mtDNA (M)
    N1b2

    Canada Somaliland
    Quote Originally Posted by Kale View Post
    You don't know how bad I'm wanting to do this! I just can't figure out how to get it up and running.
    Right now I'm stuck at why does qpgraph need block_lengths.rds if F2 statistics are already computed?

    Basically I have 1500 lines of this (output from qpfstat)... Linux command-line qpgraph understands this, how do I make R-qpgraph understand?
     

    Neanderthal.DG Neanderthal.DG 119.340
    Neanderthal.DG Mbuti.DG 61.162
    Neanderthal.DG Dinka.DG 61.136
    Neanderthal.DG Mende.DG 61.376
    Neanderthal.DG Mota.SG 61.233
    Neanderthal.DG South_Africa_2000BP.SG 61.447
    Neanderthal.DG Cameroon_ShumLaka_8000BP.DG 61.200
    Neanderthal.DG Taforalt 62.099
    Neanderthal.DG Loschbour.DG 62.937
    Neanderthal.DG Onge.DG 62.926
    Mbuti.DG Mbuti.DG 158.395
    Mbuti.DG Dinka.DG 151.343
    Mbuti.DG Mende.DG 148.330
    Mbuti.DG Mota.SG 147.789
    Mbuti.DG South_Africa_2000BP.SG 140.026
    Are you trying parse_qpDstat_output on your own precomputed dstats?

  7. #5
    Registered Users
    Posts
    2,070
    Sex
    Omitted

    Quote Originally Posted by durmang View Post
    But my problem is a bit more basic than yours: my computer goes black and becomes unresponsive, when I try to get the f2 blocks. Lol
    Mine does that too, your problem is you haven't given up on trying to do that yet like I have

    Quote Originally Posted by Mnemonics View Post
    Are you trying parse_qpDstat_output on your own precomputed dstats?
    Those aren't dstats, that is the output of qpfstat, which on the Linux version is what qpgraph operates on. You give it that and a proposed graph and whammo it's off to the races.
    neither parse_qpDstat_output, parse_qp3pop_output, nor parse_qpf4ratio_output work on that file and there is no parse_qpFstat_output command
    read_f2 doesn't work, it's looking for the weirdo .rds generated things.
    Last edited by Kale; 01-01-2021 at 04:43 PM.
    Collection of 14,000 d-stats: Hidden Content Part 2: Hidden Content Part 3: Hidden Content PM me for d-stats, qpadm, qpgraph, or f3-outgroup nmonte models.

  8. The Following User Says Thank You to Kale For This Useful Post:

     durmang (01-01-2021)

  9. #6
    Registered Users
    Posts
    642
    Sex
    Ethnicity
    Celto-Germanic, Near East
    Nationality
    Murcan
    Y-DNA (P)
    R1b-U152
    mtDNA (M)
    T2a1a

    United States of America
    Can this qpGraph run thru R create "Ghost" populations (ie, Basal Human, Archaic)? Or does it only create branches to/from available samples?

    Also the Reich dataset gives you .geno, .ind, and .snp files after unzipping... is the .geno file the one you feed into qpgraph? Are there any primers on how to use this Windows qpGraph/admixtools software package?

    I have used the R program before, but only for G25 ADMIXTURE back before the online G25 tools were available...

  10. #7
    Registered Users
    Posts
    2,070
    Sex
    Omitted

    Quote Originally Posted by K33 View Post
    Can this qpGraph run thru R create "Ghost" populations (ie, Basal Human, Archaic)? Or does it only create branches to/from available samples?

    Also the Reich dataset gives you .geno, .ind, and .snp files after unzipping... is the .geno file the one you feed into qpgraph? Are there any primers on how to use this Windows qpGraph/admixtools software package?

    I have used the R program before, but only for G25 ADMIXTURE back before the online G25 tools were available...
    Yes it can create ghosts, and the reich dataset needs? (or at least definitely should) be converted to packedancestrymap format.
    Info for how to run the new admixtools is here...
    https://uqrmaie1.github.io/admixtools/

    Step0) Download it
    Step0.5) Invoke it's mighty power
    library(admixtools)

    Step 1) get all of your f2 stats...
    extract_f2('C:/whatever/is/your/directory/with/dataset', 'C:/wherever/you/want/the/f2/to/go', pops = c('pop1, 'pop2.DG', 'pop3.SG'), maxmem = 1000)
    You must create the directory you want the f2's to go, it won't make it by itself.
    Maxmem is however much memory you can give it, 1000 works well for me.

    Step 2a) let it start from scratch
    opt_results = find_graphs('C:/wherever/you/told/the/f2/to/go', pops = c('pop1, 'pop2.DG', 'pop3.SG'), outpop = NULL, numrep = 1, numgraphs = 50, numgen = 20, numsel = 5, numadmix = 3, numstart = 1, stop_after = 1800, verbose = TRUE)
    winner = opt_results %>% top_n(1, -jitter(score))
    winner$score[[1]]
    write_dot(winner$edges[[1]])
    plot_graph(winner$edges[[1]])


    Step2b) or give it a starter graph to work with
    opt_results = find_graphs('C:/wherever/you/told/the/f2/to/go', outpop = NULL, numrep = 1, numgraphs = 50, numgen = 20, numsel = 5, numadmix = 12, numstart = 1, keep = c("best"), initgraphs = 'C:/wherever/you/have/a/graphfile.txt', stop_after = 1800, verbose = TRUE)
    winner = opt_results %>% top_n(1, -jitter(score))
    winner$score[[1]]
    write_dot(winner$edges[[1]])
    plot_graph(winner$edges[[1]])
    Collection of 14,000 d-stats: Hidden Content Part 2: Hidden Content Part 3: Hidden Content PM me for d-stats, qpadm, qpgraph, or f3-outgroup nmonte models.

  11. The Following 3 Users Say Thank You to Kale For This Useful Post:

     durmang (01-10-2021),  K33 (01-11-2021),  Mnemonics (01-10-2021)

  12. #8
    Registered Users
    Posts
    326
    Sex
    Omitted

    Knowing nothing about R, what would be the ideal version for this new Admixtools? Since R is Windows compatible, that means we can just download a version of R as its own standalone application and run it directly out of our Windows PC/Laptop right?

    EDIT: I see on github the author mentions R version 3.5 or higher, but I know from Linux newer distributions aren't always necessarily across the board better for certain applications.
    Last edited by TuaMan; 01-10-2021 at 04:53 AM.

  13. #9
    Registered Users
    Posts
    2,070
    Sex
    Omitted

    Yep, just download an R binary for Windows and good to go. I was using 3.4.3 to run nmonte, which being less than 3.5 meant I needed a new one, I just got the newest (4.0.3) and it seems to be working fine.
    Collection of 14,000 d-stats: Hidden Content Part 2: Hidden Content Part 3: Hidden Content PM me for d-stats, qpadm, qpgraph, or f3-outgroup nmonte models.

  14. #10
    Registered Users
    Posts
    326
    Sex
    Omitted

    Downloaded latest R, Rtools40, and Admixtools itself. When I run the command to install the interactive app:

    devtools::install_github("uqrmaie1/admixtools", dependencies = TRUE)

    I get this message:

    Skipping install of 'admixtools' from a github remote, the SHA1 (c312fe0a) has not changed since last install.
    Use `force = TRUE` to force installation


    Is this an expected message? Am I going to blow something up if I try forcing the install?

Page 1 of 18 12311 ... LastLast

Similar Threads

  1. An American Wellspring: Concepts and Possibilities
    By Alvah Hicks in forum American
    Replies: 2
    Last Post: 09-20-2020, 07:31 PM
  2. Playing around with qpGraph
    By Chad Rohlfsen in forum Ancient (aDNA)
    Replies: 34
    Last Post: 06-19-2017, 10:45 PM
  3. Technically am I part east African or not?
    By jaderose22 in forum African
    Replies: 15
    Last Post: 03-26-2017, 04:59 AM
  4. Replies: 0
    Last Post: 10-04-2016, 12:01 PM
  5. Future possibilities for AncestryDNA
    By Erik in forum AncestryDNA
    Replies: 0
    Last Post: 08-31-2014, 12:35 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •