PDA

View Full Version : Getting FTDNA to update their Haplotree



TigerMW
08-18-2015, 06:43 PM
I'm not sure I know the best practices on this. Paul D. posted about an error equating DF41 to DF13 phylogenetically.

So I see some M343 backbone tests have come in. For some weird reason FTDNA has put DF41 as been equivalent to DF13 on their "updated" haplogroup tree! Who do I contact in FTDNA to get them to edit their haplogroup tree and put it equivalent to CTS2501?


I've sent a request into FTDNA to fix this specific request but I don't know if there is a more efficient way to do this. I'll see what I can find out. I know they have a "contact" web page and they also have a ytree email ID, or at least they used to.

I was told a couple of months ago they created some internal tool they call a "haplotree editor" so changes should be much quicker, albeit they have validation steps and probably mis-steps to go through on these.

haleaton
08-19-2015, 05:05 AM
If you go to the Y-DNA SNP Map in FTDNA in the scroll menu it has a list of "terminal" haplogroups or perhaps those they wish to plot on maps.

I would think if FTDNA sets some ground rules on what they choose to put on their tree it would avoid some needless haggling, such as does it have to based on data on results from their lab?

What is the process of remove or document(this is science after all) the random SNPs that are said to come from GENO 2.0 but yet have not be found elsewhere?

In the past different parts of FTDNA are separate units I think the Tree people and the Big Y people may be that way. Rather than have FTDNA hire a bunch folks and raise test costs, if they had specific ground rules that the volunteer projects could obey and make changes with rules of engagement it might help.

GTC
08-19-2015, 06:13 AM
FTDNA's policy has been to base its haplotrees on published academic material. They use Doran Behar as the reference for the mt haplotree, and the most recent update of the Y haplotree was based on the work of Spencer Wells at the Genographic Project.

These days, the pace of SNP discovery far outstrips academia's ability to publish (via the traditional peer review method) and hope to be considered current.

FTDNA needs to review its Y tree update policy, or remain far behind the action.

Dubhthach
08-19-2015, 09:25 AM
Well in our case DF41 has been on ISOGG tree since mid 2012, I've forwarded on guts of my "ISOGG submission" to contact in FTDNA that mike cc'd me on. Showing the individuals from other major DF13 subclades (DF13 hadn't been discovered let!) who had tested negative for DF41 during course of early-mid 2012.

The isogg tree published in December 2012 shows position of DF41 with regards to DF13.
http://www.isogg.org/tree/ISOGG_HapgrpR12.html

GTC
08-19-2015, 10:17 AM
The ISOGG tree used to be at the forefront of the action, but since the advent of NGS testing like Big Y it is also falling behind -- though it's still far ahead of FTDNA (and 23andMe for that matter).

The most up-to-date Y haplotree information now resides within the major haplogroup projects.

TigerMW
08-19-2015, 12:49 PM
FTDNA's policy has been to base its haplotrees on published academic material. ....
They are not following that practice now. They have made numerous changes to their haplotree over the past year without academic studies. They definitely update their haplotree when they come out with a new SNP pack. I was able to get a change within L513 by adding an equivalent, CTS5396, but I haven't attempted too much seriously with them as the tree is in such a state of flux. We just had a new round of Big Y tests come in for L21 this week. It seems almost every new Big Y creates a new public branch. We have over 1100 Big Y tests completed within L21.

Since we brought up ISOGG, I made several runs on getting L513 through. I'm very grateful as their lead volunteers on this were able to get my submissions reviewed and updated in the last two months. They are totally swamped. I honestly worry the tsunami of SNPs could facilitate health issues. People are that dedicated. This does remind me of working with FTDNA a bit. Sometimes you have to figure out what format they really want since they haven't pre-built the processes (as far as I know.) Usually, timing is a big issue. If they are swamped you aren't going to get far so you may just have to try again later.... but for FTDNA they should have a defined, published process with some kind of queue/status information.

GTC
08-19-2015, 01:07 PM
They are not following that practice now. They have made numerous changes to their haplotree over the past year without academic studies.

Doesn't surprise me, given that the supply of academic papers pertaining to the Y haplotree has essentially dried up, for the reasons I mentioned earllier.

On that basis, they must be taking Y-tree update advice from haplogroup project admins, as FTDNA itself has no chance of being on top of the vast number of clades and subclades.

Joe B
08-19-2015, 05:32 PM
Doesn't surprise me, given that the supply of academic papers pertaining to the Y haplotree has essentially dried up, for the reasons I mentioned earllier.

On that basis, they must be taking Y-tree update advice from haplogroup project admins, as FTDNA itself has no chance of being on top of the vast number of clades and subclades.
The FTDNA Y haplotree wouldn't be so screwed up if FTDNA actually took advice from haplogroup administators. FTDNA's phylogenetic problems started with Geno 2.0. and were compounded when they brought in the Arpeggi software folks. I do not think there is a FTDNA conspricacy, evil intent or just plain stupidity involved. But something has been going on since late 2012. FTDNA has willfully ignored phylogenetic science from many sources to the point of risking their reputation. I think there was a bad contract between National Geographic and Gene By Gene, Ltd. that prevented them from maintaining a proper y-haplotree. Let's hope that changes soon.
I'm getting very tired of telling project members that the FTDNA Y haplotree is wrong.

Cofgene
08-19-2015, 10:33 PM
The FTDNA Y haplotree wouldn't be so screwed up if FTDNA actually took advice from haplogroup administators. FTDNA's phylogenetic problems started with Geno 2.0. and were compounded when they brought in the Arpeggi software folks. I do not think there is a FTDNA conspricacy, evil intent or just plain stupidity involved. But something has been going on since late 2012. FTDNA has willfully ignored phylogenetic science from many sources to the point of risking their reputation. I think there was a bad contract between National Geographic and Gene By Gene, Ltd. that prevented them from maintaining a proper y-haplotree. Let's hope that changes soon.
I'm getting very tired of telling project members that the FTDNA Y haplotree is wrong.

I believe that FTDNA had a contractual agreement surrounding the display and use of the Geno 2 based tree. That started and then the Big-Y SNP tsunami crashed in. FTDNA is now able to update the tree. The new clade issue will continue to be a problem since essentially every new tested lineage is going to define a new clade. As FTDNA moves forward towards integrating new Big-Y results closer to real time we will still have a problem where the tree needs to be modified due to non-FTDNA results. It is not clear that FTDNA will accept tree changes that are based on non-FTDNA data. Right now the initial versions of SNP panels are the only route for those externally derived clades to be placed onto the draft FTDNA tree for confirmation using FTDNA based samples.

haleaton
08-20-2015, 02:13 PM
Presumably, "later this year" refers to FTDNA's Annual Conference in November.

https://www.genomeweb.com/sequencing-technology/consumer-genomics-firms-hope-lower-costs-new-features-will-make-y-chromosome

A Haplotree editor and software would certainly reduce costs of updating it, but a streamlined procedure and ground rules for submitting changes by volunteers would allow for cheap adjudication by paid staff.

The value of having a good detailed and updated Y Haplotree tree to FTDNA’s business would be to increase sales of other Y tests such Big Y, SNP Packs, SNPs, & STRS. It might be nice to identify which SNPs on tree can be tested in Big Y, SNP Packs, & Individual SNP orders.

Other things FTDNA could do to increase their market and further grow the Y Haplogroup tree at minimal costs based on other’s suggestions:

1) Allow access to Y Haplogroup projects for GENO 2.0 transfers (without STRs) on Project SNP pages. This would help sell STR tests to new clients.

2) Update the Named Y SNPs list periodically based on YBrowse, FGC, & YFull definitions and update status all Big Ys in Named SNPs & Novel Variants.

3) In Big Y Matches they should identify shared Novel Variants that are widely shared outside of Matches and consider using Public NGS datasets which is the great breakthrough of the Citizen Scientists and the process used by FGC & YFull.

4) Further promote future Beta SNP Pack projects underwritten by “Beta” volunteers who are not guaranteed all SNP results, but provide a “SNP Assurance Policy” once they become “non-Beta” and promoted widely on Y-Haplogroup.

5) Streamline Individual SNP order availability through something like YSEQ’s “Wish a SNP.” It may not be cost effective for FTDNA to drop price to YSEQ levels, but FTDNA’s customer base is different.

6) Use portion of FTDNA & The Genographic Project advertising budget to promote Y-SNP testing. Perhaps with one of the Genealogy TV shows using celebrities who get Big Y and establish their own branch of SNPs and learn their ancient ancestry. One thing about NGS is you get new parts of the Human Family Tree, even if there a no matches--yet.

7) Consider including Ancient DNA and perhaps support testing of new discoveries.

Look forward to what Big Y follow on info maybe next November from FTDNA. Hope for pleasant surprises in price and coverage.

Cofgene
08-21-2015, 01:02 PM
7) Consider including Ancient DNA and perhaps support testing of new discoveries.



The proposed U106 backbone panel has 1 testable SNP from RISE98 on it. The other 2 RISE98 novels that are not well covered by Big-Y may need Elite 2 results to check them out.

TigerMW
08-21-2015, 01:49 PM
... It is not clear that FTDNA will accept tree changes that are based on non-FTDNA data. Right now the initial versions of SNP panels are the only route for those externally derived clades to be placed onto the draft FTDNA tree for confirmation using FTDNA based samples.
I am not sure how this works but I think this is a good point and good consideration as far as FTDNA SNP Packs.

There does seem to be some step that FTDNA takes where they query their database before updating the haplotree. This would imply that SNPs under consideration for the haplotree must be tested by testing at FTDNA. In the past, I received a haplotree related response from FTDNA that said they were looking at the ISOGG tree. It may be possible that they will accept the ISOGG tree. I don't know, but I would because Ray Banks is about as strong (strict) a reviewer as there is.

However, is another consideration/work-around. FTDNA definitely accepts SNPs from other sources for inclusion on their SNP Packs. If such academically (or other non-FTDNA) identified SNP is included in a Pack as "under investigation" then FTDNA could easily have their "proof" as the Pack results come in.

BTW, it's very hard to stay on topic because of the various advocacies and forces involved for and against testing companies. I'm moving those kinds of posts over to this thread:
http://www.anthrogenica.com/showthread.php?5217-The-Good-the-Bad-Evil-the-Ugly-of-testing-companies-and-Star-Wars-and-Competition

TigerMW
08-25-2015, 10:59 PM
We are starting to see more movement on the FTDNA haplotree. Every place there has been an SNP pack in L21 has driven an update to the haplotree. We've also noticed that additional L21 SNPs in the R1b-M343 Backbone SNP Pack are now appearing in the top layer of L21's section of the haplotree.

Has the U152 / L2 portions of the haplotree been updated to reflect some of the SNPs in the R1b-M343 Backbone? Are those that are added available for single SNP ordering on the Advanced Tests?

I've envious, but I think all of U106 will get a make-over in the next couple of months.

haleaton
08-25-2015, 11:24 PM
We are starting to see more movement on the FTDNA haplotree. Every place there has been an SNP pack in L21 has driven an update to the haplotree. We've also noticed that additional L21 SNPs in the R1b-M343 Backbone SNP Pack are now appearing in the top layer of L21's section of the haplotree.

Has the U152 / L2 portions of the haplotree been updated to reflect some of the SNPs in the R1b-M343 Backbone? Are those that are added available for single SNP ordering on the Advanced Tests?

I've envious, but I think all of U106 will get a make-over in the next couple of months.

I noticed in U152 that DF103 and DF110 were added to the tree following R1b-M343 Backbone test results. However SNPs on the test that were found in multiple Big Y's they are going slow in adding them to the tree. I think the R1b-M343 Backbone test results may drive any new tree changes when the change is simple.

TigerMW
08-26-2015, 12:31 PM
I noticed in U152 that DF103 and DF110 were added to the tree following R1b-M343 Backbone test results. However SNPs on the test that were found in multiple Big Y's they are going slow in adding them to the tree. I think the R1b-M343 Backbone test results may drive any new tree changes when the change is simple.
I think the activities revolving around the development of a new SNP pack drive evaluation of the pertinent sections to the tree. However, I can say for sure that FTDNA does not take a haplotree change submission from a project administrator and implement it out of hand. They have to go through their own validation process. This is where things seem to slow down, particularly if they have conflicting results in their database.

There is a difference between an SNP's inclusion in a Pack product versus an SNP's inclusion on their haplotree. Inclusion in a Pack can be easy as there is little risk. Inclusion on their haplotree is where they are cautious. However, being cautious does not prevent mistakes.

This is one of things I like about their pack approach. They are not haplotree dependent. That means it's okay to have SNPs "under investigation" included. That also means the packs will include some flakey SNPs that need to be discarded. However, this is needed. It is the trial and error approach exercising phylogenetic consistency versus what I'll call flakiness. Rather than assuming we have complete knowledge of the tree we can include "trial" SNPs in packs and over a series of pack results we'll have validation or rejection. Then, this can be used to update the haplotree correctly.

This is actually the scientific method. Hypothesize, experiment, conclude and then do it again.

ArmandoR1b
09-02-2015, 04:16 PM
I'm not sure I know the best practices on this. Paul D. posted about an error equating DF41 to DF13 phylogenetically.


I've sent a request into FTDNA to fix this specific request but I don't know if there is a more efficient way to do this. I'll see what I can find out. I know they have a "contact" web page and they also have a ytree email ID, or at least they used to.

I was told a couple of months ago they created some internal tool they call a "haplotree editor" so changes should be much quicker, albeit they have validation steps and probably mis-steps to go through on these.

Mike, can you ask FTDNA to update the location of Z198 on the haplotree? It should be phylogenetically equivalent to L176 and upstream from L165 and CTS4188.

Here is a screenshot of the results of a person that is terminal for Z198 based on the R1b-M343 Backbone SNP Pack at FTDNA -

http://i.imgur.com/5LzI5EHh.jpg
http://imgur.com/5LzI5EH

ISOGG, Z198 project results, YFull, and Alex Williamson's Big Tree all agree that Z198 is upstream of L165 and CTS4188.

http://www.isogg.org/tree/ISOGG_HapgrpR.html

https://www.familytreedna.com/public/R1b1c6/default.aspx?section=yresults

http://www.yfull.com/tree/R-Z198/

http://www.ytree.net/DisplayTree.php?blockID=187&star=false

TigerMW
09-02-2015, 10:57 PM
Mike, can you ask FTDNA to update the location of Z198 on the haplotree? It should be phylogenetically equivalent to L176 and upstream from L165 and CTS4188.

Here is a screenshot of the results of a person that is terminal for Z198 based on the R1b-M343 Backbone SNP Pack at FTDNA -

http://i.imgur.com/5LzI5EHh.jpg
http://imgur.com/5LzI5EH

ISOGG, Z198 project results, YFull, and Alex Williamson's Big Tree all agree that Z198 is upstream of L165 and CTS4188.

http://www.isogg.org/tree/ISOGG_HapgrpR.html

https://www.familytreedna.com/public/R1b1c6/default.aspx?section=yresults

http://www.yfull.com/tree/R-Z198/

http://www.ytree.net/DisplayTree.php?blockID=187&star=false

To submit these we need to compile the kit #s tested in FTDNA's system that provides the proof points positioning the SNPs. They actually try to support a chain of evidence and verify the proving kit number results are there.

However, this all works much better, I'm finding during the SNP Pack development process. During that process you have their attention and they research the subclades on their own beyond what specific positioning requests you are bring forward. This is a good point for discussion over on Earl Davis' thread for DF27 packs/panels.
http://www.anthrogenica.com/showthread.php?5277-DF27-Let-s-not-be-rash

paulgill
09-02-2015, 11:45 PM
I tried, but they won't listen at all, they have the J1 halpotree upside down, finally I gave up on them and bought the WGS test from FullGenomes, FTDNA means nothing to me anymore as it has nothing that WGS doesn't cover.

Cofgene
09-03-2015, 01:33 AM
To submit these we need to compile the kit #s tested in FTDNA's system that provides the proof points positioning the SNPs. They actually try to support a chain of evidence and verify the proving kit number results are there.

However, this all works much better, I'm finding during the SNP Pack development process. During that process you have their attention and they research the subclades on their own beyond what specific positioning requests you are bring forward. This is a good point for discussion over on Earl Davis' thread for DF27 packs/panels.
http://www.anthrogenica.com/showthread.php?5277-DF27-Let-s-not-be-rash

Correct! The U106 panels were submitted with FTDNA kit results - primarily Big-Y - for almost all of the haplogroup nodes that we wanted on the FTDNA tree. Providing FTDNA a link to individuals who test positive for a specific haplogroup makes it easier to establish the result utilizing their internal test results.

TigerMW
09-04-2015, 02:36 AM
Correct! The U106 panels were submitted with FTDNA kit results - primarily Big-Y - for almost all of the haplogroup nodes that we wanted on the FTDNA tree. Providing FTDNA a link to individuals who test positive for a specific haplogroup makes it easier to establish the result utilizing their internal test results.
Did you add any "under investigation" or early branching phylogenetic equivalent SNPs to the packs? Its to your subclade's advantage, in my opinion. You might as well stretch FTDNA all you can.

Wing Genealogist
09-04-2015, 11:34 AM
Did you add any "under investigation" or early branching phylogenetic equivalent SNPs to the packs? Its to your subclade's advantage, in my opinion. You might as well stretch FTDNA all you can.

Depending on the SNP pack, we have asked to have "under investigation" (such as the RISE98 SNPs) and currently equivalent SNPs added into the packs. Some of the packs were already very crowded and we needed to prioritize, while others had more room to add in these SNPs.

One of the items we are noting is there appears to be a large bottleneck for several different subclades of U106 where their are a large number of currently equivalent SNPs (such as Z8 Z7, Z326, Z156 and others). We have added multiple equivalents at these areas to see if we can further layer them out.

TigerMW
09-04-2015, 11:58 AM
Depending on the SNP pack, we have asked to have "under investigation" (such as the RISE98 SNPs) and currently equivalent SNPs added into the packs. Some of the packs were already very crowded and we needed to prioritize, while others had more room to add in these SNPs.

One of the items we are noting is there appears to be a large bottleneck for several different subclades of U106 where their are a large number of currently equivalent SNPs (such as Z8 Z7, Z326, Z156 and others). We have added multiple equivalents at these areas to see if we can further layer them out.
Cool. I look at phylogenetic equivalents as having three purposes. One is as a cross-check in case an SNP is flippy or there is a testing problem. The other purpose is to attempt to break phylogenetic blocks. I have found that some phylogenetic blocks are well, well tested and seem to be wast to attempt to break. An example would be L21 and L459. However, many of the lesser tested blocks (maybe most), particularly if they are long strings of SNPs, oftentimes do break as more people test.

The SNP Packs are a different kind of test. They are not haplotree dependent and in fact can ease the burden of proving the tree since the goal is that everyone gets tested for all of the same SNPs in a pack and the results show up readily in public on the Y DNA SNP project report screens.

The third purpose? Not really related to the above points, but of course phylogenetic equivalents can be useful and are a must in branch age estimation SNP counting methods.

vettor
09-04-2015, 07:52 PM
we can only hope that ftdna once they take over completely all the Natgeno data will incorporate the ydna and mtdna trees which natgeno currently use ( these are uptodate current trees ) into thier ( ftdna ) trees

Artmar
09-12-2015, 04:03 PM
I can see that since yesterday(?) FTDNA updated haplotree for R1a branch, Z280 macro-branch in particular. Can't say for the other haplogroups.

paulgill
09-12-2015, 06:04 PM
I can see that since yesterday(?) FTDNA updated haplotree for R1a branch, Z280 macro-branch in particular. Can't say for the other haplogroups.

They have mine upside down, even repeated requests have not changed their mind.

TigerMW
09-14-2015, 08:22 PM
We've been seeing updates in various branches of L21 that seem to be related to new SNP Packs. Typically, FTDNA never gets it all right in one one update and will take multiple updates. It's painful but I'm plugging "proofs" to them ,one by one (action by action).

Some update requests seem to happen within several days, and others - I don't know what happens

razyn
09-14-2015, 09:58 PM
Some update requests seem to happen within several days, and others - I don't know what happens

The Z209 level correction happened quite fast -- again, probably because it was screwing up the M-343 Backbone pack reports, and you told them so.

I wonder idly whose address you have, that I don't seem to have, at FTDNA. They never ask me ("Mr. DF27") my opinion about any of these matters (their haplotree, packs, panels, the phylogeny). I don't think they ever look at the DF27 project's prolifically multiplying groups (in STR Results); nor at its "activity feed," that they set up last spring when they revamped all their project sites. We have been discussing these things quite openly, on the member forum they set up for it -- not just on Anthrogenica, Facebook, and the DF27 Yahoo group.

Janine Cloud in Customer Service answers my emails, and tries to solve my problems with the IT people. Usually, they eventually succeed. But I think the lab people, and maybe their Corporate masters, operate on some higher plane -- to which GAP, as such, does not really give one access. I might add that Thomas Krahn didn't respond, either, when he worked there. I tried for about two years, 2011-13. At his own company, he has been quite accessible. So I think maybe there is a "firewall" of sorts, and some of us -- who should perhaps be heard -- are inaudible to them. If you have somebody's ear, great.

TigerMW
09-15-2015, 05:55 PM
The Z209 level correction happened quite fast -- again, probably because it was screwing up the M-343 Backbone pack reports, and you told them so.

I wonder idly whose address you have, that I don't seem to have, at FTDNA. They never ask me ("Mr. DF27") my opinion about any of these matters (their haplotree, packs, panels, the phylogeny). I don't think they ever look at the DF27 project's prolifically multiplying groups (in STR Results); nor at its "activity feed," that they set up last spring when they revamped all their project sites. We have been discussing these things quite openly, on the member forum they set up for it -- not just on Anthrogenica, Facebook, and the DF27 Yahoo group.
...

I didn't request any updates to the tree for the R-M343 Backbone SNP pack. I sent them a spreadsheet full of SNPs with position numbers, allele changes, synonym names. I didn't send them kit #s proving positioning but I think the Backbone test was mostly very well known SNPs so there was not the same need.

I was surprised in that they did some updating downstream including into the first level below DF13 and I think L2 and it looks like under U106 and DF27 as well. They did that on their own including naming a couple of new SNPs. This is part of the reason I started this thread. I want to communicate that getting them to do an SNP pack does spur action in their haplotree. I noticed a week before they released a pack for my L21 subclade of L513, that they started updating their haplotree for L513. They got some things wrong and some right, which I'm trying to correct. I formally put in a request on Friday for the main L513 change on L193. Nothing has happened since then but it seems like forever.

The one that has been amazingly fast has been BY653 under DF27. I don't know how it got so lucky. FTDNA wasn't planning anything with it that I knew of until about when the M343 Backbone test launched. It's in the pack and on the tree.

Cofgene
09-15-2015, 11:12 PM
we can only hope that ftdna once they take over completely all the Natgeno data will incorporate the ydna and mtdna trees which natgeno currently use ( these are uptodate current trees ) into thier ( ftdna ) trees

That incorporation happened about 18 months ago. We were stuck with their errors in interpreting the Geno2 results until the current panels started modifying the tree. The Geno2 results are out of date compared to what the haplogroup projects know and are putting onto the panels.

vettor
09-15-2015, 11:39 PM
That incorporation happened about 18 months ago. We were stuck with their errors in interpreting the Geno2 results until the current panels started modifying the tree. The Geno2 results are out of date compared to what the haplogroup projects know and are putting onto the panels.

complete take over will be november 2015 when the old owner of natgeno moves on ( as he stated )

vettor
09-23-2015, 07:36 AM
In my T haplotree tree in ftdna, I have positive, negatives and presumed positive SNP's.

Why in it that in the K haplotree and F haplotree does it only show positives and untested and not any negatives?

example
K

M9 P128 P131 P132

Joe B
09-25-2015, 10:54 PM
One of the fustrating aspects to the FTDNA haplotree updates is the reluctance to give up on SNP L150 as a haplogroup definer. Because of back mutations under R1b-CTS7822 and the fact that R1b-PF7562 individuals are L150+, L23- while the rest of us are L23+, L150+, L150 is the likely the cause of most errors on the R1b-Z2103 branch of their tree. I suspect that if L150 is removed from FTDNA's haplotree algorithm the Z2103 branches will self correct. They made some changes today that are nice but will mean more it the tree was correct. L277.1, L584 and Z2106 are brother clades under Z2103/Z2105. Z2109 and CTS7763 are bother clades under Z2106. CTS3937 and CTS7822 are brother clades under Z2109. M64.3 is a branch downstream of CTS3937. CTS9219 is a major branch of CTS7822 and is a loose brother to CTS699.

R1b>M269>PF7562>7563
R1b>M269>L23>Z2103
R1b>M269>L23>Z2103>L277
R1b>M269>L23>Z2103>L584>PF7580
R1b>M269>L23>Z2103>Z2106>CTS7763
R1b>M269>L23>Z2103>Z2106>Z2109>CTS3937>M64
R1b>M269>L23>Z2103>Z2106>Z2109>CTS7822>CTS9219
R1b>M269>L23>Z2103>Z2106>Z2109>CTS7822>CTS699

I know things are getting a little testy for many of us. For me, it's the repeated need to tell people the FTDNA haplotree is wrong. It's almost become a standard salutation is my emails.

morrisondna
09-25-2015, 11:23 PM
R1b>M269>L23>PF7562>7563

Joe, I think you mean R1b>M269>PF7562>PF7563...?

Joe B
09-25-2015, 11:53 PM
Joe, I think you mean R1b>M269>PF7562>PF7563...?Thanks, it's the heat. You are correct. R1b>M269>PF7562>PF7563 is the old L150+, L23- and the most basal branch of R1b>M269 that we know of.

razyn
10-03-2015, 10:10 PM
Has anybody else noticed that some of the BigY Novel Variants are suddenly being called by their new SNP name (e.g. FGC11397), instead of the position on the Y chromosome? That's a new experience for me. The BigY list still doesn't seem to talk to the Haplotree (this particular Rox2 guy is still just called as P312, not even DF27). But it's one small step for mankind.

I wonder if it's the ones that are on SNP packs, or going to be on new ones, or are on the Advanced SNP test menu, or have been requested for that... Something is going on, anyhow, since the last time I looked, which was probably last night. Although as it happens, I was looking at one of the ones for which Shared Novel Variants is currently screwed up (again).

MitchellSince1893
10-03-2015, 10:30 PM
Has anybody else noticed that some of the BigY Novel Variants are suddenly being called by their new SNP name (e.g. FGC11397), instead of the position on the Y chromosome? That's a new experience for me. The BigY list still doesn't seem to talk to the Haplotree (this particular Rox2 guy is still just called as P312, not even DF27). But it's one small step for mankind.

I just checked. I don't have any SNP names in my novel list...just position numbers. When I check my known SNP list I don't have any FGC named SNPs.

razyn
10-03-2015, 10:40 PM
I checked mine, and I don't either. Maybe it's just Rox2 cluster guys (a DF27 subclade). I looked at two different ones and they both have Z2571 and FGC11397 called, as such, in Shared Novel Variants. Could be a case of a squeaky wheel's having been greased, that Rox2 group is pretty proactive (not that I think that's an actual word). Anyway it's a step they formerly refused to take.

TigerMW
10-03-2015, 11:03 PM
Has anybody else noticed that some of the BigY Novel Variants are suddenly being called by their new SNP name (e.g. FGC11397), instead of the position on the Y chromosome? That's a new experience for me. The BigY list still doesn't seem to talk to the Haplotree (this particular Rox2 guy is still just called as P312, not even DF27). But it's one small step for mankind.

I wonder if it's the ones that are on SNP packs, or going to be on new ones, or are on the Advanced SNP test menu, or have been requested for that... Something is going on, anyhow, since the last time I looked, which was probably last night. Although as it happens, I was looking at one of the ones for which Shared Novel Variants is currently screwed up (again).
I asked Bennett Greenspan a couple of months ago if he was planning to lower Big Y pricing and he was emphatic that he he had no plans to do so, etc., etc... Anyway, in the same conversation he said there enhancements coming this fall that will help existing customers. I hope they re-evaluate their "test call" methodology but I don't know that is part of it all. I did gather that they were integrating the database of Big Y SNPs with the regular SNP database and that the number of "known" SNPs would go up dramatically. It looks like that is starting to happen even if they haven't moved SNPs off the Novel Variants tab.

Along this line, I download the Y DNA SNP project report for the L21 project about every week or so and compare it with what I currently have. I've noticed for the last couple of months all of the folks with Big Y results have had changes every download. I don't know if this is the SNP name association process or updates to the haplotree (which drive negative result displays) or both. Something is happening.

razyn
10-04-2015, 01:04 AM
The FTDNA haplotree still has DF27 and L194 on the same level (and some people are buying a SNP test for the latter -- about the longest shot there is, now that M153 isn't directly under P312). We have one kit (N2642) that tested L194+ via WTY in 2009; and later DF27+ (SNP test in 2012). His Geno2 results don't show Z195+, and he's had a Sanger sequenced Z196-. So he is at the very least in our big group E, DF27+ and Z195-. (Which I have started calling ZZ12+ because it takes fewer letters, in a limited caption.) I don't know what else should be required, to get L194 off their list of things every P312* guy might want to buy, just in case.

TigerMW
10-04-2015, 01:59 AM
The FTDNA haplotree still has DF27 and L194 on the same level (and some people are buying a SNP test for the latter -- about the longest shot there is, now that M153 isn't directly under P312). We have one kit (N2642) that tested L194+ via WTY in 2009; and later DF27+ (SNP test in 2012). His Geno2 results don't show Z195+, and he's had a Sanger sequenced Z196-. So he is at the very least in our big group E, DF27+ and Z195-. (Which I have started calling ZZ12+ because it takes fewer letters, in a limited caption.) I don't know what else should be required, to get L194 off their list of things every P312* guy might want to buy, just in case.

They are having a hard time integrating SNP/branch insertions with their legacy haplotree. I know of at least two situations in other haplogroups very similar to what you are describing. I imagine there are more, probably quite a few. I've tried submitting a correction in about three different ways. I know they understand my request.

The ultimate response I get is a software "glitch". Sound like a big gov't system. That doesn't make it any better. It's still a problem as it does screw up what individuals see and think. All I know is they say they are working on it.

haleaton
10-04-2015, 05:13 AM
Has anybody else noticed that some of the BigY Novel Variants are suddenly being called by their new SNP name (e.g. FGC11397), instead of the position on the Y chromosome? That's a new experience for me. The BigY list still doesn't seem to talk to the Haplotree (this particular Rox2 guy is still just called as P312, not even DF27). But it's one small step for mankind.

I wonder if it's the ones that are on SNP packs, or going to be on new ones, or are on the Advanced SNP test menu, or have been requested for that... Something is going on, anyhow, since the last time I looked, which was probably last night. Although as it happens, I was looking at one of the ones for which Shared Novel Variants is currently screwed up (again).

The only one I noticed (for me) was the one Novel Variant FGC5301 < L2 < U152 which became part of the R1b R-M343 SNP Pack, though no one yet has tested positive. They do not go in and update myself and my match R-FGC5301, based on Big Y, because they claim it is not "on the tree" though the Big Y results say that is it "on the tree." I did not want to waste my money and just retest myself, so I found a good candidate and hopefully it will end up on the actual tree.

Odd thing mentioned elsewhere for Big Y, is that all my 13 matches after 8/7/2015 show L858 & L862 as shared novel variants in the pulldown menu. I asked them about this and pointed out that this was likely wrong, but after some initial responses I never heard back from the "Big Y Team." I have clearly tested negative for L858 & L862.

In U152 once somebody is positive in the SNP Pack it does end up on their public tree and then they ask me test for it . . .

Earl Davis
10-04-2015, 07:25 AM
I checked mine, and I don't either. Maybe it's just Rox2 cluster guys (a DF27 subclade). I looked at two different ones and they both have Z2571 and FGC11397 called, as such, in Shared Novel Variants. Could be a case of a squeaky wheel's having been greased, that Rox2 group is pretty proactive (not that I think that's an actual word). Anyway it's a step they formerly refused to take.

I checked mine and FGC17102 and FGC17099 now show up. Their parent FGC17112 currently does not.

Earl.

REWM
10-04-2015, 10:54 AM
I checked mine and FGC17102 and FGC17099 now show up. Their parent FGC17112 currently does not.

Earl.

I checked my shared novel variants and Z29624 now shows up. They now have it as a known SNP . But it's parent Z2573 still currently has it listed as Presumed Negative on their Haplotree for me. I also still have L858 & L862 that I have clearly tested negative for.

corner
10-04-2015, 01:54 PM
Has anybody else noticed that some of the BigY Novel Variants are suddenly being called by their new SNP name (e.g. FGC11397), instead of the position on the Y chromosome? That's a new experience for me. The BigY list still doesn't seem to talk to the Haplotree (this particular Rox2 guy is still just called as P312, not even DF27). But it's one small step for mankind.

I wonder if it's the ones that are on SNP packs, or going to be on new ones, or are on the Advanced SNP test menu, or have been requested for that... Something is going on, anyhow, since the last time I looked, which was probably last night. Although as it happens, I was looking at one of the ones for which Shared Novel Variants is currently screwed up (again).


I checked mine, and I don't either. Maybe it's just Rox2 cluster guys (a DF27 subclade). I looked at two different ones and they both have Z2571 and FGC11397 called, as such, in Shared Novel Variants. Could be a case of a squeaky wheel's having been greased, that Rox2 group is pretty proactive (not that I think that's an actual word). Anyway it's a step they formerly refused to take.I requested FGC11397 at FTDNA some time ago to help identify Rox2 as a single test there. I think someone else (sorry, can't remember who) must have requested Z2571 because it was already on the list there. These SNPs were identified in Rox2 in April 2014 after NGS tests. I have no access to any BigY results and have not taken the test myself, so the SNPs showing up as Shared Novel Variants is news. It is also interesting because we thought FGC11397 was one of the FGC SNPs found in Jim and Bob's FGC results that didn't show up in later BigY results. Somehow, deep DF27>ZZ12 SNP, Z2571, was left out of the DF27 section of the new FTDNA R1b-M343 Backbone SNP Pack, which is unfortunate because identifiable Rox2 and other ZZ12 subclades only came out of that test as P312* with DF27 itself also being absent. Hopefully those formerly P312* should now get a bit more of an informative DF27* result from the backbone test, DF27 itself being added recently, but ideally Z2571 would be in the test like other 'top layer' subclades are.

razyn
10-20-2015, 03:17 PM
Has anybody else noticed that some of the BigY Novel Variants are suddenly being called by their new SNP name... ?

I noticed some new ones today; don't know how long they have been displayed. A431 and A432 are now "Known SNPs," so those positions are no longer showing up as matches in Shared Novel Variants... they have presumably lost their novelty. In my own subclade, CTS10029 is a Known SNP, but none of the FGC157xx series are. Those were found in my sample, but apparently at the wrong company -- though FTDNA has been testing FGC15710 for a year, and it has three subgroups in the DF27 project. Whatever.

TigerMW
11-05-2015, 02:45 AM
I noticed some new ones today; don't know how long they have been displayed. A431 and A432 are now "Known SNPs," so those positions are no longer showing up as matches in Shared Novel Variants... they have presumably lost their novelty. In my own subclade, CTS10029 is a Known SNP, but none of the FGC157xx series are. Those were found in my sample, but apparently at the wrong company -- though FTDNA has been testing FGC15710 for a year, and it has three subgroups in the DF27 project. Whatever.
Now that we have the DF27 SNP packs out we'll have a series of SNPs per every branch (or very nearly) so we will have a good opportunity to maintain FTDNA's attention on updating the haplotree and the haplogroup labeling process.

razyn
11-05-2015, 04:20 AM
Now that we have the DF27 SNP packs out we'll have a series of SNPs per every branch (or very nearly) so we will have a good opportunity to maintain FTDNA's attention on updating the haplotree and the haplogroup labeling process.

The new Z209 pack has several of the FGC157xx SNPs on it, so as more of us turn up, that test should spot them. I am officially Appeased.

kinman
11-05-2015, 04:01 PM
Hi All,
I wish FTDNA would finally get around to adding R-Z142 , a major clade of R-Z49. They have a bunch of little branches for R-Z49, but they don't list a major branch like R-Z142 (which is probably now the fastest growing of all the branches).
---------Ken

haleaton
11-05-2015, 05:44 PM
Hi All,
I wish FTDNA would finally get around to adding R-Z142 , a major clade of R-Z49. They have a bunch of little branches for R-Z49, but they don't list a major branch like R-Z142 (which is probably now the fastest growing of all the branches).
---------Ken

Hopefully, Project Administrators will bring up how to (encourage, help) update tree next week at https://www.familytreedna.com/conference/. A case for Z142 would likely have to be based totally on data they have from their lab including GENO 2.0.

MitchellSince1893
11-05-2015, 06:43 PM
Hopefully, Project Administrators will bring up how to (encourage, help) update tree next week at https://www.familytreedna.com/conference/. A case for Z142 would likely have to be based totally on data they have from their lab including GENO 2.0.
They need to expand what they use from their lab beyond just geno2.0 tested SNPs. In my case FTDNA has two other FTDNA tests confirming I'm Z142 but they don't currently incorporate them.

haleaton
11-05-2015, 06:49 PM
They need to expand what they use from their lab beyond just geno2.0 tested SNPs. In my case FTDNA has two other FTDNA tests confirming I'm Z142 but they don't currently incorporate them.

They should have enough Big Y Z142+ to incorporate into their tree, of course. [Edit: I guess Z49 was not in Big Y? Was not in mine.]

TigerMW
11-06-2015, 12:07 AM
They need to expand what they use from their lab beyond just geno2.0 tested SNPs. In my case FTDNA has two other FTDNA tests confirming I'm Z142 but they don't currently incorporate them.
They've gone beyond Geno 2 SNPs for over a year. They have SNPs from the old Chromo 2 and Big Y testing now on their haplotree.

Regardless of where the SNP comes from they need to see at least two derived occurrences of the SNP in their database so at least two people have to test for it at FTDNA. They also need to see the parent SNPs' derived result and ancestral results in parallel SNPs.

They take Big Y evidence for sure, although they may need to follow their calls, not third party interpretations of BAM files. I'm not sure how they deal with a third party BAM file call of their Big Y data.

The real problem is their Y tree team is swamped/backlogged. The way to get their attention is get an SNP pack out the door for the section of the tree you care about.

Are Z49 and Z142 available via their Advanced Tests menu or in a pack? If so, can your project admin show them two derived results and make your proposal to add just like we would for ISOGG? Remember, they are getting swamped with requests, many from people they barely know. It helps if the project admin making the request is active and known to them.

kinman
11-06-2015, 12:18 AM
I had an entry for Z49 in the complete CSV file. However, in the "derived" column there was just a question mark, under Reference and Genotype it had "G" and "?", and under confidence it said "Unknown". I assume that is what they refer to as a "no call".
-----------Ken


They should have enough Big Y Z142+ to incorporate into their tree, of course. [Edit: I guess Z49 was not in Big Y? Was not in mine.]

haleaton
11-06-2015, 12:34 AM
They've gone beyond Geno 2 SNPs for over a year. They have SNPs from the old Chromo 2 and Big Y testing now on their haplotree.

Regardless of where the SNP comes from they need to see at least two derived occurrences of the SNP in their database so at least two people have to test for it at FTDNA. They also need to see the parent SNPs' derived result and ancestral results in parallel SNPs.

They take Big Y evidence for sure, although they may need to follow their calls, not third party interpretations of BAM files. I'm not sure how they deal with a third party BAM file call of their Big Y data.

The real problem is their Y tree team is swamped/backlogged. The way to get their attention is get an SNP pack out the door for the section of the tree you care about.

Are Z49 and Z142 available via their Advanced Tests menu or in a pack? If so, can your project admin show them two derived results and make your proposal to add just like we would for ISOGG? Remember, they are getting swamped with requests, many from people they barely know. It helps if the project admin making the request is active and known to them.

These are good points and guidelines. The Z142 data may already be there--but it has to be results from FTDNA lab and FTDNA analysis, which I think is a reasonable position for FTDNA to take.

Also, I think if you want to fix something the person who wants it enough needs to pursue it. The focus of the U152 group tree has been comparison of NGS sequences--and it is excellent, but time consuming work. One side issue is many of the SNPs in the tree are not called out by letter names just location numbers and (anc/der). Some of them do have names you can find in YBrowse. Some may have "BY" names but they have not been made public. So far to get on FTDNA tree or to be in a SNP Pack it has to have a name. One way to give a SNP an "A" name and get it at YBrowse is the "Wish a SNP" for a buck at YSEQ.

MitchellSince1893
11-06-2015, 01:15 AM
There are roughly 17 BigY results that are Z142 positive give or take.


They've gone beyond Geno 2 SNPs for over a year. They have SNPs from the old Chromo 2 and Big Y testing now on their haplotree.

Regardless of where the SNP comes from they need to see at least two derived occurrences of the SNP in their database so at least two people have to test for it at FTDNA. They also need to see the parent SNPs' derived result and ancestral results in parallel SNPs.

I guess the problem for us Z142 types is FTDNA's BigY has no calls for Z49 so they currently don't use the data? And in my case they also aren't using my individual Z142 and Z49 tests I took with FTDNA.

I hear you on the overworked staff, but it sounds like it's a case of the squeaky wheel getting the grease as to which SNPs are fixed first?

IIR wasn't Richard Rocca or somebody else working on a U152 SNP package?

kinman
11-06-2015, 02:03 AM
Yes, FTDNA offers single SNP tests for both, Z49 on their haplotree menu and Z142 in their Advanced Tests menu. I knew that I was Z49 from their single SNP test, but didn't verify Z142 and downstream until my recent Big Y test.
There are dozens of positive tests for Z142 (either through the single Z142 test or Big Y). All they need to do is add a Z142 branch between Z49 and the Z150 and Z51 branches that they already have on their haplotree (as well as the rapidly growing third major subclade FGC22963 , which a lot of people have tested positive for in Big Y tests).
----------------Ken
------------------------------------------------------------------------------------



Are Z49 and Z142 available via their Advanced Tests menu or in a pack? If so, can your project admin show them two derived results and make your proposal to add just like we would for ISOGG? Remember, they are getting swamped with requests, many from people they barely know. It helps if the project admin making the request is active and known to them.

TigerMW
11-06-2015, 01:51 PM
.... Also, I think if you want to fix something the person who wants it enough needs to pursue it. The focus of the U152 group tree has been comparison of NGS sequences--and it is excellent, but time consuming work. One side issue is many of the SNPs in the tree are not called out by letter names just location numbers and (anc/der). ...
The side issue of not having an SNP name is not really an inhibitor to getting something on the haplotree. FTDNA works with these SNPs by position-anc-der notation, as you said is common, but they'll add a BY name to it as needed, but they use names that are out there so they seem to be agnostic on SNP naming. Naming an SNP is not a slow down, however, I agree it is a good idea to have SNPs evaluated from multiple standpoints for testing feasibility, region/location stability and phylogenetic recurrency.

GTC
11-12-2015, 02:22 AM
If anyone is attending the upcoming FTDNA Annual Conference, perhaps they could ask during the Q&A session for FTDNA to explain their current process for updating the Y Haplotree.

TigerMW
11-12-2015, 03:48 AM
If anyone is attending the upcoming FTDNA Annual Conference, perhaps they could ask during the Q&A session for FTDNA to explain their current process for updating the Y Haplotree.
It is a good idea to ask directly, but I think understand a little about their process now. They are swamped so it isn't what it should be. That's why I started this thread, to give hints out. The important thing is to get an SNP Pack out for your subclade. You do have to justify it (potential # testers, # SNPs) though.

GTC
11-12-2015, 10:52 AM
It is a good idea to ask directly,

Yes, let FTDNA management tell it to the customer base at large in their own words. Those of us in big, well-resourced haplogroups tend to have better access to the process than the many others who don't.


They are swamped so it isn't what it should be.

If and when they admit that, IMO it would be just the opportunity to suggest that they simply adopt the ISOGG tree. IMO, FTDNA will never have sufficient resources in house to properly maintain the ever expanding Y tree.

lgmayka
11-12-2015, 11:20 AM
If and when they admit that, IMO it would be just the opportunity to suggest that they simply adopt the ISOGG tree. IMO, FTDNA will never have sufficient resources in house to properly maintain the ever expanding Y tree.
Yikes! Due to all the new SNP packs, FTDNA's tree is now "light-years" ahead of the ISOGG tree. ISOGG, as a purely volunteer organization, does not have the resources to keep up with rapid SNP growth.

FTDNA, in contrast, is now making amazing progress in the haplogroups for which SNP packs are offered. All haplogroup administrators should work toward submitting SNP pack proposals to fully cover their haplogroups.

paulgill
11-12-2015, 11:34 AM
Yikes! Due to all the new SNP packs, FTDNA's tree is now "light-years" ahead of the ISOGG tree. ISOGG, as a purely volunteer organization, does not have the resources to keep up with rapid SNP growth.

FTDNA, in contrast, is now making amazing progress in the haplogroups for which SNP packs are offered. All haplogroup administrators should work toward submitting SNP pack proposals to fully cover their haplogroups.

FTDNA's tree is now "light-years" ahead of the ISOGG tree? How come they have mine upside down? I am J1-Z1853* downstream of Z1865, not upstream of Z1865.


PF4629P58More...
J-P58

Z18218
J-Z18218
PF4678Z1889More...
J-PF467
Z1853
J-Z1853
Z1865
J-Z1865
L862
J-L862Add

GTC
11-12-2015, 12:01 PM
Yikes! Due to all the new SNP packs, FTDNA's tree is now "light-years" ahead of the ISOGG tree.
ISOGG, as a purely volunteer organization, does not have the resources to keep up with rapid SNP growth.

I'll posit that there are far more people working on the ISOGG tree than FTDNA has working on theirs, or ever will. Ask FTDNA at their conference how many staff they have assigned to halpotree maintenance.


FTDNA, in contrast, is now making amazing progress in the haplogroups for which SNP packs are offered.

Mine hasn't moved an iota in years. I'm still showing on FTDNA as R1b-Z12. Have done BIG-Y.

lgmayka, what is your haplogroup?

TigerMW
11-12-2015, 12:34 PM
Yes, let FTDNA management tell it to the customer base at large in their own words. Those of us in big, well-resourced haplogroups tend to have better access to the process than the many others who don't.

If and when they admit that, IMO it would be just the opportunity to suggest that they simply adopt the ISOGG tree. IMO, FTDNA will never have sufficient resources in house to properly maintain the ever expanding Y tree.

My guess is they won't be clear on this. There are probably some good reasons, i.e. changes to processes, etc. but I think a lot of times they just want to be cautious in anything that sounds like a commitment they will be pinned down on later.

I would be okay if they accept the ISOGG tree, but I would not advise that. The ISOGG process doesn't follow the scientific process fully. For instance, data used should be published, peer review, etc., etc. That would be excessive for volunteers to provide plus ISOGG can't really certify a chain of evidence.

FTDNA has more data in their database anyway, and they can (and have to) stand behind their own "certified" results. They just have either a poor process or are sorely understaffed... probably both with one feeding the other.

TigerMW
11-12-2015, 12:38 PM
..
Mine hasn't moved an iota in years. I'm still showing on FTDNA as R1b-Z12. Have done BIG-Y.

lgmayka, what is your haplogroup?

My haplogroup label is R-ZW02. I think it is only about 300-500 years old. I had to work very hard to submit the tree for L513 to both ISOGG and FTDNA. I actually spent less time and had faster response time through FTDNA.

However, this is why I suggest the SNP Pack development process is important. FTDNA has a commercial reason to work on the tree when put a new SNP Pack out. Things actually happen, albeit it is easy to get bogged down in their queue.

BTW, working on the tree is always a "works in progress" thing. With FTDNA it seems it is two steps forward, one step backward and on and on. Maybe "baby steps" is a better way to put it.

GTC
11-12-2015, 12:48 PM
The ISOGG process doesn't follow the scientific process fully. For instance, data used should be published, peer review, etc., etc. That would be excessive for volunteers to provide plus ISOGG can't really certify a chain of evidence.

True, but I gather that ISOGG is now being quoted in some scientific papers -- after all, what else is there?

Academia hasn't a ghost of a chance of keeping up with the Y tree changes. I've said before elsewhere that I reckon ISOGG's credentials could be enhanced if it had a governing board of suitably qualified and respected academics overseeing due diligence, etc. However, that would require some of ISOGG's 'upper echelon' to agree to hand over some of their proprietorial control. Politics and all that.


FTDNA has more data in their database anyway, and they can (and have to) stand behind their own "certified" results. They just have either a poor process or are sorely understaffed... probably both with one feeding the other.

Yes, and I cannot see what is essentially a testing lab ever devoting the massive effort required to keeping the tree up to date and relevant across all haplogroups. It's neither their core skill nor their core business.

A cynic may say that FTDNA regards its Y tree as an order generating tool and that's fine, as long as it's correct and relevant to all customers. And there's the rub.

TigerMW
11-12-2015, 12:58 PM
True, but I gather that ISOGG is now being quoted in some scientific papers -- after all, what else is there?

Academia hasn't a ghost of a chance of keeping up with the Y tree changes. I've said before elsewhere that I reckon ISOGG's credentials could be enhanced if it had a governing board of suitably qualified and respected academics overseeing due diligence, etc. However, that would require some of ISOGG's 'upper echelon' to agree to hand over some of their proprietorial control. Politics and all that...
Agreed, but if you were a scientist writing a paper you should not rely on the ISOGG tree for anything that feeds into your conclusions. Citing another source is one thing, but if that source isn't following the rigors of the scientific processes that would be a logic flaw in a peer review or rebuttal. It's a house of cards.

I agree that academia can't keep up with the resolution levels in the tree we are seeking - applicable to genetic genealogy. However, this all means we are left with a mess of draft and experimental trees. Eventually the experimental trees will be purely machine (software) driven and this can work if we are all doing NGS tests and the data is there for the analytics software to sift through. My guess is that it will require a commercial enterprise to pull that off as they can generate the funds to sustain it.

This is a good point. I don't think we can say any tree is 100% correct. We just assume it is based on a high confidence level. We need the software to calculate the confidence/error ranges.

GTC
11-12-2015, 12:58 PM
Further to the bit about an ordering tool its integrity, I should add that FTDNA's haplotree prompts me to order the Z8 SNP pack although they can easily see that I have done BIG Y. Hopeless!

TigerMW
11-12-2015, 01:06 PM
Further to the bit about an ordering tool its integrity, I should add that FTDNA's haplotree prompts me to order the Z8 SNP pack although they can easily see that I have done BIG Y. Hopeless!
Please start a new thread on this. This is more about their targeting/marketing of packs process. They've actually tried to address it, mostly unsuccessfully in my opinion. If you want to talk about this please start another thread. I'll join in.

miiser
11-12-2015, 01:31 PM
I'm not convinced that maintaining the tree is nearly as much effort as it's been made out to be. For R-P312, which contains a significant fraction of the total number of branches in FTDNA's tree, Alex manages to single-handedly identify all new SNPs, define the phylogeny, and format the data into a presentable, useable form. And I'm pretty sure he spends much less than 40 hours a week doing this.

I think the problem is simply that FTDNA perceives that updating the tree does not directly contribute to the bottom line, and resources are therefore not allocated to it.

The "scientific process" is not what some people seem to think it is. Einstein did not walk around with the measurement of Mercury's perihelion in a briefcase handcuffed to his wrist to hand it off to the proper authorities. Academia does not work like you think it does.

The type of work done by Alex is quite sufficient to develop and maintain an effective tree. It may contain a few errors. But the harm of any temporary errors is minimal to none. It's not as if the FDA is validating drug safety based on the correctness of the Y-tree phylogeny. If an error in the tree is discovered, or new data contradicts the current structure, it is a trivial matter to correct the error and update the tree.

The only tricky part is that someone has to be working on it in order for it to get done.

I'm not convinced that FTDNA is allocating even a few hours a week for the task. I believe if they were, someone like Alex would have no difficulty keeping the entire tree up to date.

MacUalraig
11-12-2015, 01:54 PM
True, but I gather that ISOGG is now being quoted in some scientific papers -- after all, what else is there?

Academia hasn't a ghost of a chance of keeping up with the Y tree changes. I've said before elsewhere that I reckon ISOGG's credentials could be enhanced if it had a governing board of suitably qualified and respected academics overseeing due diligence, etc. However, that would require some of ISOGG's 'upper echelon' to agree to hand over some of their proprietorial control. Politics and all that.



Yes, and I cannot see what is essentially a testing lab ever devoting the massive effort required to keeping the tree up to date and relevant across all haplogroups. It's neither their core skill nor their core business.

A cynic may say that FTDNA regards its Y tree as an order generating tool and that's fine, as long as it's correct and relevant to all customers. And there's the rub.

I think you will find scientists have drifted away from the ISOGG tree due to its excessive detail, in favour of this one:

http://onlinelibrary.wiley.com/doi/10.1002/humu.22468/abstract

GTC
11-12-2015, 01:55 PM
Please start a new thread on this. This is more about their targeting/marketing of packs process. They've actually tried to address it, mostly unsuccessfully in my opinion. If you want to talk about this please start another thread. I'll join in.

Okay, here: http://www.anthrogenica.com/showthread.php?5813-FTDNA-s-marketing-of-SNP-Packs&p=120154#post120154

GTC
11-12-2015, 01:58 PM
II'm not convinced that FTDNA is allocating even a few hours a week for the task. I believe if they were, someone like Alex would have no difficulty keeping the entire tree up to date.

Hopefully, someone here is attending the Conference and will ask FTDNA management to enlighten us all about that.

Edit: I should add that we tend to see the job in terms of our haplogroup specialists -- in your case Alex, who we assume has an intimate knowledge of the data that he works with on a regular basis and can make sound classification judgements accordingly.

However, factor in the huge number of haplo- and sub-haplogroups that exist and the associated amount of specialist knowledge involved, and that's why I say that there's no way FTDNA could ever hope to maintain the whole tree in house.

The specialist expertise resides in the project groups and it's those people who are advising FTDNA on the content and variety of SNP Packs that should be made available.

kinman
11-12-2015, 02:38 PM
I only recently got my Big Y results, so my expertise is still pretty limited. However, it seems like a lot of their "Novel Variants" aren't novel at all. I find this even more troubling than not keeping their haplotree up-to-date. Is there some reason that they should be so slow to shift a lot of these SNPs from "novel variants" into Known SNPs? Or is this generally regarded as just a minor annoyance, rather than a major problem?

Táltos
11-12-2015, 03:36 PM
I am happy to say I see progress today in the Q tree at FTDNA! I have been wondering when Q1b would show it's expanded branches.
662266236624
It was announced the SNP pack for Q-L245 should be released in a week or so.

The tree acknowledges my brother's positive Q-YP1003 result, but in the project and the main heading on myFTDNA page only the Q-L245 is listed.

Well I should say in the project they have him under the YP1003 section, the green YP1003 is not triggered there yet. Only the green Q-L245 is.

miiser
11-12-2015, 11:41 PM
Hopefully, someone here is attending the Conference and will ask FTDNA management to enlighten us all about that.

Edit: I should add that we tend to see the job in terms of our haplogroup specialists -- in your case Alex, who we assume has an intimate knowledge of the data that he works with on a regular basis and can make sound classification judgements accordingly.

However, factor in the huge number of haplo- and sub-haplogroups that exist and the associated amount of specialist knowledge involved, and that's why I say that there's no way FTDNA could ever hope to maintain the whole tree in house.

The specialist expertise resides in the project groups and it's those people who are advising FTDNA on the content and variety of SNP Packs that should be made available.

I hear that, and I think it's a fair point. But I'm not convinced it's really intimate knowledge of the tree structure that enables Alex and others to do this job. After all, I don't think Alex remembers the ID labels and location of each SNP in each subclade. Having some knowledge may help a little, but I think it's simply a matter of intelligent analysis - comparing a new kit to the current tree and known SNPs, using the right software tools to quickly identify known and novel SNPs, placing the new kit based on consistency of the new kit's known SNPs with the current tree structure, and identifying new branches based on the kit's novel SNPs. I think it just boils down to having a solid grasp of the data analysis and an efficient routine algorithm established to process each new kit. With a sound algorithm, you don't need to have the tree structure fixed in your own memory. You just apply the same algorithm each time and place each new kit wherever it fits along with its novel SNPs.

And Alex really is processing a significant fraction of the total tree branches, not just a miniscule fraction. I double checked the haplotree to verify that this is really true, and I wasn't making things up. Most of the tree is not nearly as finely structured as R-P312. Take what Alex has done, multiply it by a smallish number, and that's the entirety of the task.

I think it's good that FTDNA consults the project admins for advice on SNP packs. But FTDNA really should have in house capability to do this analysis themselves. I think FTDNA's perspective is that the project admins are free labor. But in order for their Big Y matching to ever be of use, FTDNA needs to have a proper record of the tree in house anyways, apart from any expertise of the project admins. There's no reasonable excuse not to have one, except that they have no real competition yet, so they can continue to make a profit even though the product has major deficiencies. The financial incentive is the missing ingredient, and this is why SNP pack development seems to be the only thing that prompts any progress.

GTC
11-13-2015, 01:10 AM
... I think it's simply a matter of intelligent analysis - comparing a new kit to the current tree and known SNPs, using the right software tools to quickly identify known and novel SNPs, placing the new kit based on consistency of the new kit's known SNPs with the current tree structure, and identifying new branches based on the kit's novel SNPs. I think it just boils down to having a solid grasp of the data analysis and an efficient routine algorithm established to process each new kit. With a sound algorithm, you don't need to have the tree structure fixed in your own memory. You just apply the same algorithm each time and place each new kit wherever it fits along with its novel SNPs.

FTDNA does not have a good record in this department. Some time back it tried to raise the level of sophistication of its automatic Y haplogroup prediction from haplotypes and it failed miserably and had to back out the results.


I think it's good that FTDNA consults the project admins for advice on SNP packs. But FTDNA really should have in house capability to do this analysis themselves. I think FTDNA's perspective is that the project admins are free labor.

IMO, FTDNA could make greater use of project admins by consulting them before it does things. Plenty of companies actively welcome input from their "power" users. I have yet to see FTDNA solicit ideas from the consumer base at their annual conferences rather than telling customers how it's going to be. FTDNA could also benefit from having project admins test their software changes in-house before it is released. If FTDNA cared enough, it could be arranged.


But in order for their Big Y matching to ever be of use, FTDNA needs to have a proper record of the tree in house anyways, apart from any expertise of the project admins. There's no reasonable excuse not to have one, except that they have no real competition yet, so they can continue to make a profit even though the product has major deficiencies. The financial incentive is the missing ingredient, and this is why SNP pack development seems to be the only thing that prompts any progress.

Speaking for U106, FTDNA's Big Y matching is very weak compared to what is available from the project's admin team using spreadsheets. I had expected the influx of Arpeggi people with their supposed expertise in the analysis big data sets to result in better IT effort, but I haven't seen that as yet. Frankly, I can't see the IT function within FTDNA getting better without management change at the top of the company.

lgmayka
11-13-2015, 01:42 AM
FTDNA's tree is now "light-years" ahead of the ISOGG tree? How come they have mine upside down?
Has your haplogroup project administrator proposed a SNP pack appropriate to your part of the tree? Has the SNP pack been released? And has your haplogroup project administrator then made an effort to get as many appropriate project members as possible to order the SNP pack?

lgmayka
11-13-2015, 01:47 AM
Mine hasn't moved an iota in years. I'm still showing on FTDNA as R1b-Z12.
FTDNA offers an R1b-Z8 SNP Pack which covers Z12. These SNPs will get onto the tree after enough customers order the SNP pack, and FTDNA runs the tests.
---
Includes the following SNPs on the haplotree:
U106, Z381, L48, Z9, Z11, Z12, L47, U198, Z156, L1, Z1, Z344, Z343, L148, Z20, Z22, Z23, Z24, Z25, Z26, Z29, Z8, Z301

Includes the following SNPs that are NOT on the haplotree:
Z8432, Z351, BY118, S3511, A299, A300, A321, A5616, A687, A6903, A6904, A6906, A96, Z8175, Z8174, Z8173, Z3, Z4, Z5, Z6, CTS10742, CTS1747, CTS4569, CTS5601, CTS7080, CTS7678, DF101, DF102, F1636, FGC11784, FGC12057, Z346, Z18, Z21, FGC12058, FGC12993, FGC15254, FGC1954, FGC29405, FGC35613, FGC4161, Z338, Z27210, FGC5253, FGC5254, FGC5264, M157, M365, PF889, S10415, S12035, YFS065855, Y15631, S9565, S1355, S16701, S1774, S18890, S18951, S20321, S20422, S26379, S3262, S3510, S5245, S5246, S5633, S6881, S7297, S738, S794
---

lgmayka
11-13-2015, 01:52 AM
Academia hasn't a ghost of a chance of keeping up with the Y tree changes.
I fully agree with this!

Yes, and I cannot see what is essentially a testing lab ever devoting the massive effort required to keeping the tree up to date and relevant across all haplogroups. It's neither their core skill nor their core business.
You seem to be proposing that we all use YFull's haplotree instead. That is their core skill and their core business.

lgmayka
11-13-2015, 01:58 AM
Is there some reason that they should be so slow to shift a lot of these SNPs from "novel variants" into Known SNPs?
Ironically, the first time that FTDNA even talked about doing this, they ran into a storm of opposition from spreadsheet-using experts who didn't want SNPs hopping from one category to another.

lgmayka
11-13-2015, 02:06 AM
And Alex really is processing a significant fraction of the total tree branches, not just a miniscule fraction. I double checked the haplotree to verify that this is really true, and I wasn't making things up. Most of the tree is not nearly as finely structured as R-P312. Take what Alex has done, multiply it by a smallish number, and that's the entirety of the task.
You are ignoring a colossal issue, indeed the "elephant in the room." Alex, like YFull, trusts NGS sequencing to build a haplotree. This is highly controversial. Current ISOGG rules (http://www.isogg.org/tree/ISOGG_SNP_Requirements.html) say:
---
If the evidence for the SNP is based solely on next generation sequencing, the SNP will appear in italics on the tree.
---
Needless to say, the italics are meant to indicate that the entry is not yet trustworthy.

If I understand correctly, the new ISOGG tree coordinator (Ray Banks) is proposing a way to place SNPs on the tree based on NGS sequencing alone, but only under strict technical conditions that require considerable technical expertise to apply.

FTDNA clearly does not trust NGS sequencing alone to build a haplotree, whereas they do trust their SNP pack technology (highly programmable microarrays?).

paulgill
11-13-2015, 02:13 AM
Has your haplogroup project administrator proposed a SNP pack appropriate to your part of the tree? Has the SNP pack been released? And has your haplogroup project administrator then made an effort to get as many appropriate project members as possible to order the SNP pack?

No, as a few of us are J1-Z1853+ and L862-, only one line of Z1853 which is L862+ is covered for now. Anyway, there is nothing at FTDNA to offer me anything, as I already have mine WGS, it is getting analysed right now, and I expect my results any moment.

Gravetto-Danubian
11-13-2015, 02:20 AM
I fully agree with this!

You seem to be proposing that we all use YFull's haplotree instead. That is their core skill and their core business.

Why don't YFull publish formally their tree ? Is it an issue with how samples were acquired ?

lgmayka
11-13-2015, 02:21 AM
I have yet to see FTDNA solicit ideas from the consumer base at their annual conferences rather than telling customers how it's going to be.
On October 17, 2014, FTDNA sent an email to yDNA haplogroup project administrators, specifically asking for SNP pack proposals, including step-by-step instructions. For various reasons, many haplogroups did not respond; and those that did initially encountered a stony silence which, in retrospect, was probably due to staff reallocation to Geno 2.0 Next Gen.

My impression is that after the official announcement of Geno 2.0 Next Gen, FTDNA staff were able to turn much more attention to SNP packs. Just today I submitted a 53-SNP addition to the so-called R1a-Z283 (actually R1a-Z284) SNP Pack. To my pleasant shock, the immediate response was:
---
I will go ahead and make these additions, they should be implemented by the end of the month.
---
This is, of course, merely a reaction from a technical employee, not an official company commitment. Nevertheless, it demonstrates the level of effort that FTDNA is now willing to invest into SNP packs.

lgmayka
11-13-2015, 02:28 AM
No, as a few of us are J1-Z1853+ and L862-, only one line of Z1853 which is L862+ is covered for now. Anyway, there is nothing at FTDNA to offer me anything, as I already have mine WGS, it is getting analysed right now, and I expect my results any moment.
Frankly, if you are not (or no longer) a paying FTDNA customer, I don't see why you would expect FTDNA to update its haplotree for your benefit. As someone else correctly pointed out, FTDNA's business is to sell tests, not to publish haplotrees in academic journals. If paying customers care about your section of the haplotree, they will propose a SNP pack, and order it when available.

paulgill
11-13-2015, 02:52 AM
Has your haplogroup project administrator proposed a SNP pack appropriate to your part of the tree? Has the SNP pack been released? And has your haplogroup project administrator then made an effort to get as many appropriate project members as possible to order the SNP pack?


Frankly, if you are not (or no longer) a paying FTDNA customer, I don't see why you would expect FTDNA to update its haplotree for your benefit. As someone else correctly pointed out, FTDNA's business is to sell tests, not to publish haplotrees in academic journals. If paying customers care about your section of the haplotree, they will propose a SNP pack, and order it when available.

Your assumptions. How does the FTDNA knows that I am not their potential customer still, why do they still want me to take all their tests if they think they can't make any money on me any more? Geno 2.0 says that there are .3% of its customers who are Z1853* and L862- and that means over 2000+ possible different lines, not just one L862+ one. But we have no way of knowing who these people are, as you very well know that there is no way of communicating, even knowing the ethnicity and whereabouts of Geno 2.0 customers. My results were showing the right subclade before, but now it shows a wrong ones, and I have already paid them for all that, do I have to pay them to correct their own mistakes too?

Cofgene
11-13-2015, 03:47 AM
You are ignoring a colossal issue, indeed the "elephant in the room." Alex, like YFull, trusts NGS sequencing to build a haplotree. This is highly controversial. Current ISOGG rules (http://www.isogg.org/tree/ISOGG_SNP_Requirements.html) say:
---
If the evidence for the SNP is based solely on next generation sequencing, the SNP will appear in italics on the tree.
---
Needless to say, the italics are meant to indicate that the entry is not yet trustworthy.

If I understand correctly, the new ISOGG tree coordinator (Ray Banks) is proposing a way to place SNPs on the tree based on NGS sequencing alone, but only under strict technical conditions that require considerable technical expertise to apply.

FTDNA clearly does not trust NGS sequencing alone to build a haplotree, whereas they do trust their SNP pack technology (highly programmable microarrays?).


Ray Banks proposals need significant revision in wording, and sometimes in concept, to be based on solid sequencing principals. Ray doesn't want to pull it draft criteria and address the scientific gaps in his proposals. ISOGG's quality criteria are conceptually good but poorly represented in their current form. THey also do not appropriately address sequencing method independence and what I call the "time" issue.

The ISOGG y-tree group rolled over and played dead in terms of coming out with a supportable set of quality criteria.

Cofgene
11-13-2015, 03:48 AM
Frankly, if you are not (or no longer) a paying FTDNA customer, I don't see why you would expect FTDNA to update its haplotree for your benefit. As someone else correctly pointed out, FTDNA's business is to sell tests, not to publish haplotrees in academic journals. If paying customers care about your section of the haplotree, they will propose a SNP pack, and order it when available.

FTDNA's tree can be updated if you point to data present at FTDNA. SIGH!

miiser
11-13-2015, 03:51 AM
You are ignoring a colossal issue, indeed the "elephant in the room." Alex, like YFull, trusts NGS sequencing to build a haplotree. This is highly controversial. Current ISOGG rules (http://www.isogg.org/tree/ISOGG_SNP_Requirements.html) say:
---
If the evidence for the SNP is based solely on next generation sequencing, the SNP will appear in italics on the tree.
---
Needless to say, the italics are meant to indicate that the entry is not yet trustworthy.

If I understand correctly, the new ISOGG tree coordinator (Ray Banks) is proposing a way to place SNPs on the tree based on NGS sequencing alone, but only under strict technical conditions that require considerable technical expertise to apply.

FTDNA clearly does not trust NGS sequencing alone to build a haplotree, whereas they do trust their SNP pack technology (highly programmable microarrays?).

This is a silly stance. Confidence of the SNP accuracy should be the criterion for acceptance. Sanger sequencing can give high confidence results. So can NGS. A single Big Y test is not generally high confidence for every SNP. But with a few NGS tests on the same branch, confidence quickly reaches a very high level due to redundant coverage and consistency checks. Unrelated branches are very unlikely to contain the same set of SNPs by mere chance, so when you find the same low confidence SNPs in separate Big Y tests, they become high confidence by nature of the phylogenetic consistency.

ISOGG's standards are unnecessarily burdensome, and the paperwork and bureacracy is a liability, not an asset. Based on whatever confidence threshold Alex relies on before he adds an SNP to the tree, there may be a small number of errors in the terminal branches that aren't heavily covered. But anything above the lowest branches is very high confidence. And, frankly, there's no value in withholding an SNP until its confidence is ~100%. There is value in putting fair confidence SNPs on the tree so that they can be further tested. And, if resources were dedicated to continuously maintaining the tree, it would be a trivial matter to correct them in the rare cases that they turn out wrong.

I suppose ISOGG thinks they have their reasons for holding to the standards that they have. But there's no value in FTDNA adopting a similar standard.

miiser
11-13-2015, 04:33 AM
FTDNA clearly does not trust NGS sequencing alone to build a haplotree, whereas they do trust their SNP pack technology (highly programmable microarrays?).

As a side note, I was thinking about the FTDNA's SNP pack testing method. My initial thought was that these SNP packs were custom chips for each pack. But another possibility is that FTDNA is utilizing their NGS sequencing equipment now that the initial wave of Big Y has passed. I think the process would be: 1) add a molecular ID tag to each kit, 2) combine all the kits' DNA and primer sequences for all the pack's SNPs into one "soup", then amplify everything as a batch, 3) put into the NGS equipment and sort out the results with the ID tags. I think this method would be consistent with the SNP pack features. It would have a limit of the number of SNPs you could target in one run, possibly on the order of 100. Orders of SNP packs would be run in batches, which appears to be the case. And it would be easy to change the set of included SNPs for each run, which appears to be the case.

haleaton
11-13-2015, 05:29 AM
As a side note, I was thinking about the FTDNA's SNP pack testing method. My initial thought was that these SNP packs were custom chips for each pack. But another possibility is that FTDNA is utilizing their NGS sequencing equipment now that the initial wave of Big Y has passed. I think the process would be: 1) add a molecular ID tag to each kit, 2) combine all the kits' DNA and primer sequences for all the pack's SNPs into one "soup", then amplify everything as a batch, 3) put into the NGS equipment and sort out the results with the ID tags. I think this method would be consistent with the SNP pack features. It would have a limit of the number of SNPs you could target in one run, possibly on the order of 100. Orders of SNP packs would be run in batches, which appears to be the case. And it would be easy to change the set of included SNPs for each run, which appears to be the case.

I would think that without defining what the technology of the FTDNA SNP packs is, that ISOGG should not allow any SNP Pack results to be used for inclusion on their three based on the kind of standards they outline. Not that ISOGG's or FTDNA's or YFull's trees matter a whole that much at this moment. But it is important that in a future date, perhaps next Tuesday, they do.

GTC
11-13-2015, 06:51 AM
Has your haplogroup project administrator proposed a SNP pack appropriate to your part of the tree? Has the SNP pack been released? And has your haplogroup project administrator then made an effort to get as many appropriate project members as possible to order the SNP pack?

Yes. Yes. Yes.

@lgmayka: What is your haplogroup"?

lgmayka
11-13-2015, 10:44 AM
How does the FTDNA knows that I am not their potential customer still, why do they still want me to take all their tests if they think they can't make any money on me any more?
A potential customer should follow the procedure I outlined:
- Join a haplogroup project
- Ask your project administrator to submit a proposal for a SNP pack
- When the SNP pack is released, buy it yourself if appropriate, or encourage others to do so


Geno 2.0 says that there are .3% of its customers who are Z1853* and L862- and that means over 2000+ possible different lines, not just one L862+ one. But we have no way of knowing who these people are, as you very well know that there is no way of communicating, even knowing the ethnicity and whereabouts of Geno 2.0 customers.
If you are a customer of Geno 2.0 and have an issue with it, I suggest you contact National Geographic.


My results were showing the right subclade before, but now it shows a wrong ones
If you can point to FTDNA results showing this, please go ahead and submit the inconsistency to FTDNA, preferably through your haplogroup project administrator.

lgmayka
11-13-2015, 11:03 AM
But another possibility is that FTDNA is utilizing their NGS sequencing equipment now that the initial wave of Big Y has passed.
I don't know whether your hypothesis is consistent with this early statement by FTDNA: "They won't be using Sanger sequencing, but it won't be a Big Y-type sequencing either."

I hope FTDNA realizes that eventually, they must describe their method in sufficient detail for outside experts to evaluate its accuracy.

lgmayka
11-13-2015, 11:08 AM
Yes. Yes. Yes.
Then all you need to do is encourage a sufficient number of customers to order the SNP pack, and wait for results to arrive.

What is your haplogroup?
I am encouraging my own haplogroup to propose a SNP pack. I have already participated in SNP pack proposals for another haplogroup.

TigerMW
11-13-2015, 01:04 PM
You are ignoring a colossal issue, indeed the "elephant in the room." Alex, like YFull, trusts NGS sequencing to build a haplotree. This is highly controversial. Current ISOGG rules (http://www.isogg.org/tree/ISOGG_SNP_Requirements.html) say:
---
If the evidence for the SNP is based solely on next generation sequencing, the SNP will appear in italics on the tree.
---
Needless to say, the italics are meant to indicate that the entry is not yet trustworthy.
Does ISOGG really say that italics indicates not yet trustworthy? I submitted a whole bunch based solely on NGS results and they are on ISOGG's tree now. They are in italics. I personally don't mind. I'm like 99.99% sure it is accurate and I just wanted to see things documented somewhere but I look at the italics as indicating a different source for information, not necessarily untrustworthiness. That's why I ask, do they really say that?


FTDNA clearly does not trust NGS sequencing alone to build a haplotree, whereas they do trust their SNP pack technology (highly programmable microarrays?).
Are you sure they don't trust NGS for haplotree placement? I have quite a few in the L513 tree that I don't think anyone has tested for other than by Big Y. It could be they don't trust calls they themselves call as "REJECTED" that some of the NGS experimental trees use. I think FTDNA is way too restrictive and lean towards the experimental tree building on that but if you get some of those SNPs in to a Pack you'll end up with those results too.

TigerMW
11-13-2015, 01:12 PM
I don't know whether your hypothesis is consistent with this early statement by FTDNA: "They won't be using Sanger sequencing, but it won't be a Big Y-type sequencing either."

I hope FTDNA realizes that eventually, they must describe their method in sufficient detail for outside experts to evaluate its accuracy.
I can only go on what I'm told but FTDNA has told me the packs were not Sanger Sequencing and they were not Next Generation Sequencing. About a month ago, several of the YSEQ advocates started making a big deal about this and wanting to know the exact technology. I don't really care as long as they stand behind it, as I know technical people from different labs will always have their disagreements. Still, I was asked to ask FTDNA and so I did. They told me that they are confident in the Pack technology and that it is used in the medical field. I think they will talk about it at their conference this weekend. For heaven's sake, they do lab tours so people can take notes in the lab if nothing else.

GTC
11-13-2015, 01:28 PM
I can only go on what I'm told but FTDNA has told me the packs were not Sanger Sequencing and they were not Next Generation Sequencing.

I reckon Thomas Khran of YSEQ would have a pretty good idea what they are using.

TigerMW
11-13-2015, 01:33 PM
I reckon Thomas Khran of YSEQ would have a pretty good idea what they are using.
He's the one that asked me to ask them. He also at first indicated that FTDNA was sending DNA out of the country. Bennett was absolutely emphatic that no DNA was leaving the Houston lab, let alone the country. I think he thought Thomas was thinking of some project several years ago (when Thomas was there) that got canned. When I told Thomas what Bennett said, Thomas replied (on the forum) that he doesn't believe Bennett.
I don't know, but if they are doing lab tours this weekend certainly this should clear up.

It seems like the Force intervenes in every thread where FTDNA is mentioned and we get into Star Wars competitive discussions that can degenerate into mudslinging. :( I am not blaming anyone but it is nice that YSEQ has their own topic category now so that is a great place for YSEQ stuff.

GTC
11-13-2015, 01:53 PM
I don't know, but if they are doing lab tours this weekend certainly this should clear up.


Anyone here doing the tour?

Cofgene
11-13-2015, 02:28 PM
He's the one that asked me to ask them. He also at first indicated that FTDNA was sending DNA out of the country. Bennett was absolutely emphatic that no DNA was leaving the Houston lab, let alone the country. I think he thought Thomas was thinking of some project several years ago (when Thomas was there) that got canned. When I told Thomas what Bennett said, Thomas replied (on the forum) that he doesn't believe Bennett.
I don't know, but if they are doing lab tours this weekend certainly this should clear up.

It seems like the Force intervenes in every thread where FTDNA is mentioned and we get into Star Wars competitive discussions that can degenerate into mudslinging. :( I am not blaming anyone but it is nice that YSEQ has their own topic category now so that is a great place for YSEQ stuff.


When someone goes on the admin lab tour you will see the instruments that are used for the SNP packs. Remember the name and the shape of the instruments then Google them for all of the details. After you guys have the tour and the word is out - unless FTDNA throws a blanket over the instruments - we can discuss what is being used. FTDNA is using a standard technology for variant identification that is available in the marketplace. FTDNA probably considers it a state secret just to poke at Thomas's abilities.

miiser
11-13-2015, 09:41 PM
I don't know, but if they are doing lab tours this weekend certainly this should clear up.

I would be very surprised if tour guides or Bennett answered any questions on the tour that they're unwilling to answer in other forums.

My bet is, if they operate like any other technology oriented company, they made a point to put away or hide any proprietary information before the tour, and they tidied up the lab to make it look much more organized and professional than it typically is. A person touring the lab is not going to be able to pick up any technical information by a glance around the room. We already know that FTDNA has access to Sanger sequencing, NGS, and chip technology. Details of procedures for specific tests won't be apparent from a lab tour.

Mud on my face if I'm wrong.

miiser
11-13-2015, 09:55 PM
I don't know whether your hypothesis is consistent with this early statement by FTDNA: "They won't be using Sanger sequencing, but it won't be a Big Y-type sequencing either."

I hope FTDNA realizes that eventually, they must describe their method in sufficient detail for outside experts to evaluate its accuracy.

I suppose it depends on what exactly they mean by "a Big Y-type sequencing". This phrase is rather vague. Does this exclude all NGS methods? Or just the specific procedure used for Big Y? It's not clear to me.

TigerMW
11-13-2015, 10:11 PM
Please, may be we go back to the thread topic/title.

Please start another thread if you want to cover something else. Some have been very clear they don't care about FTDNA's haplotree. That's okay, please start another thread to talk about whatever tree or other kind of thing want to. This thread is intended to help provide insight and exchange ideas on how to progress things.

We may not perceive the same things about a documented Y tree so I'd like to go back explain a little more about FTDNA's haplotree. It is not just a paper document or html web page. It actually sits in some kind of database and is used to drive haplogroup labels that are integrated across the myFTDNA haplotree pages (where it does show up as a dynamic web page) and project systems as haplogroup labels. In other words it is an I/T data structure and ripples across multiple I/T applications that we use and probably that they use internally. It is a production tree that affects many things so it they don't update it lightly and have version change controls. Unfortunately, I don't think their change management is as strong as it should be, although that might actually slow things down more.

Their haplotree has links directly into their ordering system (for SNPs and SNP Packs) as well as their email and banner popup applications. As you know, it screws people up if it is not correct. For instance, I am at my terminal known SNP, ZW02 and it is about 400 years old so that may be as far as I will ever get on a public tree due to diversity requirements. I don't see popups for banners for any packs. The reason why is their haplotree knows I'm at their terminal haplogroup for me. However, when the haplotree is not correctly detailed people will see banner ads, etc. that are not correct. The problem isn't their I/T system or some intent, it is their haplotree data structure is not correct and needs updating.

lgmayka
11-13-2015, 10:27 PM
Are you sure they don't trust NGS for haplotree placement?
Here's a very obvious example.

FTDNA's haplotree shows Y35 on the same level as CTS3402 (this is within the R1a haplogroup). However, four different kits clearly show a green Y35 and a red CTS3402. Either FTDNA does not trust its own Big Y results sufficiently to change the haplotree, or FTDNA does not have the staff time to change the tree based on Big Y results. As far as the customer is concerned, those two alternatives are exactly the same!

I am sure this will get corrected if/when a SNP pack result shows Y35+ CTS3402- .

miiser
11-13-2015, 10:31 PM
Please, may be we go back to the thread topic/title.

Please start another thread if you want to cover something else. Some have been very clear they don't care about FTDNA's haplotree. That's okay, please start another thread to talk about whatever tree or other kind of thing want to. This thread is intended to help provide insight and exchange ideas on how to progress things.

We may not perceive the same things about a documented Y tree so I'd like to go back explain a little more about FTDNA's haplotree. It is not just a paper document or html web page. It actually sits in some kind of database and is used to drive haplogroup labels that are integrated across the myFTDNA haplotree pages (where it does show up as a dynamic web page) and project systems as haplogroup labels. In other words it is an I/T data structure and ripples across multiple I/T applications that we use and probably that they use internally. It is a production tree that affects many things so it they don't update it lightly and have version change controls. Unfortunately, I don't think their change management is as strong as it should be, although that might actually slow things down more.

Their haplotree has links directly into their ordering system (for SNPs and SNP Packs) as well as their email and banner popup applications. As you know, it screws people up if it is not correct. For instance, I am at my terminal known SNP, ZW02 and it is about 400 years old so that may be as far as I will ever get on a public tree due to diversity requirements. I don't see popups for banners for any packs. The reason why is their haplotree knows I'm at their terminal haplogroup for me. However, when the haplotree is not correctly detailed people will see banner ads, etc. that are not correct. The problem isn't their I/T system or some intent, it is their haplotree data structure is not correct and needs updating.

I would argue that the missing ingredient preventing FTDNA from keeping their haplotree updated is the financial incentive to do so. For those not in a position to motivate a tree update via an SNP pack development (financial incentive for FTDNA), threatening to take your business to a competitor, and convincing FTDNA that their failure in this service is a real threat to their bottom line, may be the ONLY means of convincing FTDNA to invest the necessary resources.

Diagnosing the disease is a prerequisite for treating it. Getting FTDNA to update their tree first requires understanding why it is not already updated. I think the past few pages of discussion have been very relevant to the topic. And, contrary to your perception, there has been very little mud slinging or Star Wars talk of the evil empire. The discussion has mostly been thoughtful analysis of the problem. Your personal disagreement with the viewpoints does not make it mudslinging by default.

lgmayka
11-13-2015, 10:34 PM
I have quite a few in the L513 tree that I don't think anyone has tested for other than by Big Y.
You may be seeing the effect of experimental runs that are not transferred into the customer account interface.

With every SNP pack proposal, I have been asked to provide, for each SNP, a kit number believed to be positive. The only reason to do this is if FTDNA runs the SNP pack experimentally on at least some of these kits, as a test of the SNP pack itself.

lgmayka
11-13-2015, 10:37 PM
For those not in a position to motivate a tree update via an SNP pack development (financial incentive for FTDNA), threatening to take your business to a competitor, and convincing FTDNA that their failure in this service is a real threat to their bottom line, may be the ONLY means of convincing FTDNA to invest the necessary resources.
I don't understand who this could possibly be. If you have spending money that you can threaten to take to a competitor, you can also "threaten" to order a SNP pack.

The case that I well understood--which we have already discussed--is that of someone who has essentially completed his personal DNA testing, and hence has no further financial leverage, with FTDNA or any other testing company.

miiser
11-13-2015, 11:04 PM
I don't understand who this could possibly be. If you have spending money that you can threaten to take to a competitor, you can also "threaten" to order a SNP pack.

The case that I well understood--which we have already discussed--is that of someone who has essentially completed his personal DNA testing, and hence has no further financial leverage, with FTDNA or any other testing company.

This might include, for example, someone whose current placement in the tree is incorrect, but who happens to be in a haplogroup which is too small to financially justify review for development of an SNP pack. This might include customers whose haplogroup administrator is not on the ball for coordinating SNP pack development. (In anticipation that you'll reply that the customer's recourse in this situation is to harass the haplogroup admin to harass FTDNA to develop an SNP pack, I would counter that it should not fall on the customer to do this, and it is unlikely to produce results anyways. It is reasonable for the customer to expect the tree to be correct and up to date without the need to harass anyone.)

And if SNP pack sales is the current motivation, then where does that leave us after the initial wave of SNP pack sales has passed? It puts us right back into the position of FTDNA having no financial motivation to invest in maintaining the tree.

paulgill
11-13-2015, 11:08 PM
A potential customer should follow the procedure I outlined:
- Join a haplogroup project
- Ask your project administrator to submit a proposal for a SNP pack
- When the SNP pack is released, buy it yourself if appropriate, or encourage others to do so

If you are a customer of Geno 2.0 and have an issue with it, I suggest you contact National Geographic.

If you can point to FTDNA results showing this, please go ahead and submit the inconsistency to FTDNA, preferably through your haplogroup project administrator.

All that been done already, FTDNA will take no action and the project administrator had advised me not to expect anything better from FTDNA as such problems are experienced by many at FTDNA, I am not alone, but luckily our yDNA J1 Tree is the best, updated regularily, have a look http://genogenea.com/J-M267/tree.

lgmayka
11-13-2015, 11:29 PM
This might include, for example, someone whose current placement in the tree is incorrect, but who happens to be in a haplogroup which is too small to financially justify review for development of an SNP pack.
Small (i.e., relatively untested) haplogroups can share a SNP pack with others, or be included in a catch-all "rare gems" SNP pack that includes F, M, S, etc.


This might include customers whose haplogroup administrator is not on the ball for coordinating SNP pack development.
A savvy customer could compose the proposal himself if necessary, merely getting the administrator's approval or even without it.


And if SNP pack sales is the current motivation, then where does that leave us after the initial wave of SNP pack sales has passed? It puts us right back into the position of FTDNA having no financial motivation to invest in maintaining the tree.
Chuckle. All the testing companies, as well as all of us here, hope and assume that this field will continue to attract new entrants. I agree that after the initial burst, the continuing revenue stream will be smaller; but then, the effort required to add a few SNPs to an existing SNP pack is apparently much less than the composition and testing of an entirely new pack from scratch.

GTC
11-13-2015, 11:46 PM
Getting FTDNA to update their tree first requires understanding why it is not already updated.

Hence my request for somebody attending the FTDNA Conference to ask Bennett Greenspan to explain to his customers how his company manages its Y tree.

miiser
11-13-2015, 11:46 PM
A savvy customer could compose the proposal himself if necessary, merely getting the administrator's approval or even without it.

From what I have read on these boards and elsewhere, some customers have tried this and failed. It seems like it's very difficult to get anyone's attention and get any response from customer service or management other than customer service's canned answers.


Chuckle. All the testing companies, as well as all of us here, hope and assume that this field will continue to attract new entrants. I agree that after the initial burst, the continuing revenue stream will be smaller; but then, the effort required to add a few SNPs to an existing SNP pack is apparently much less than the composition and testing of an entirely new pack from scratch.

I don't share your optimism. All indications are that the SNP pack development is associated with a single, one time update to the tree. There is no indication that on-going frequent maintenance will be performed from this point forward.

My hope is that FTDNA is developing an integrated software solution to automatically tie Big Y results into the haplotree. But if this does not happen, FTDNA's past performance gives no reason to expect that there will be a continued maintenance of the tree. I expect their stance would be that, post SNP pack development, the tree is "good enough" to support the future trickle of SNP pack orders without any regular cycle of tree updates.

miiser
11-13-2015, 11:49 PM
Hence my request for somebody attending the FTDNA Conference to ask Bennett Greenspan to explain to his customers how his company manages its Y tree.

I agree that it is a good question to ask. Based on previous experience with FTDNA's customer interface, I don't think it's realistic to expect an honest answer. One of my fellow project admins attended a previous conference, and she was disappointed by it. Her summary was that she didn't learn much, as the entire conference was essentially a commercial ad to promote FTDNA's products.

lgmayka
11-14-2015, 12:12 AM
From what I have read on these boards and elsewhere, some customers have tried this and failed.
This has already succeeded in at least one case. But I suspect that whenever possible, FTDNA would prefer to engage administrators in the process.


All indications are that the SNP pack development is associated with a single, one time update to the tree.
Two SNP packs have already graduated to 2.0, and another is at 1.1. I myself have been told that my proposed substantial additions to a pack should be completed by the end of this month.

We have every reason to think that when results from these revised packs arrive, FTDNA will adjust its tree accordingly.

miiser
11-14-2015, 12:27 AM
This has already succeeded in at least one case. But I suspect that whenever possible, FTDNA would prefer to engage administrators in the process.

Two SNP packs have already graduated to 2.0, and another is at 1.1. I myself have been told that my proposed substantial additions to a pack should be completed by the end of this month.

We have every reason to think that when results from these revised packs arrive, FTDNA will adjust its tree accordingly.

I think you are confusing unfinished development phase with on going tree maintenance. Most of the changes have been due to SNPs that don't work well for the process, whatever it is.

Once they get a fully functioning pack, I don't expect that they will continue tinkering.

lgmayka
11-14-2015, 02:35 AM
I think you are confusing unfinished development phase with on going tree maintenance. Most of the changes have been due to SNPs that don't work well for the process, whatever it is.
...
Once they get a fully functioning pack, I don't expect that they will continue tinkering.
Your statements have already been proven false. You have successfully added yourself to my ignore list.

RCO
11-14-2015, 11:57 AM
I gave up of the mistakes of FTDNA haplotree and since a long time ago the good reference is YFull tree. Unfortunately FTDNA haplotree is incomplete and they arbitrarily choose what they want to place there as in the case of my haplogroup J1.

paulgill
11-14-2015, 12:46 PM
I gave up of the mistakes of FTDNA haplotree and since a long time ago the good reference is YFull tree. Unfortunately FTDNA haplotree is incomplete and they arbitrarily choose what they want to place there as in the case of my haplogroup J1.
ll
Based on the past experience I expect nothing much from FTDNA when it comes to YTree. All those who have got NGS test done should send their files to YFull so that the YFull Tree can be updated, even if they don't want analysis from YFull. But so for as the J1 tree is concerned it is updated regularly, we have a super great administrator Victar Josef Mas taking care of that there.

I think FTDNA should increase the number of SNPs it tests for in each Pack regularly, to ensure that one gets close to his terminal SNP with just a single test, that will make the yDNA tests much affordable for many.

paulgill
11-14-2015, 12:52 PM
Your statements have already been proven false. You have successfully added yourself to my ignore list.

I think that is being a little too harsh, don't you think so too?

miiser
11-16-2015, 12:43 AM
This has already succeeded in at least one case. But I suspect that whenever possible, FTDNA would prefer to engage administrators in the process.

Two SNP packs have already graduated to 2.0, and another is at 1.1. I myself have been told that my proposed substantial additions to a pack should be completed by the end of this month.

We have every reason to think that when results from these revised packs arrive, FTDNA will adjust its tree accordingly.


From Jennifer Zinck's coverage of the FTDNA conference:

"So far they have created about 40 panels, which takes a large amount of vetting. All of the primers have to be able to work together to create single base reactions. They pull out the weaker ones. Once they have the whole mapping down they can’t remove one and add another. They are now trying to develop a system to help them with redesigning all of the panels."

From the horse's mouth - they're not planning to continuously update the SNP panels. Revisions have, in fact, been to adjust for problematic SNPs.



Small (i.e., relatively untested) haplogroups can share a SNP pack with others, or be included in a catch-all "rare gems" SNP pack that includes F, M, S, etc.

. . .

A savvy customer could compose the proposal himself if necessary, merely getting the administrator's approval or even without it.

From Jennifer Zinck's coverage of the FTDNA conference:

"The 40 panels are primarily for the largest haplogroups. They are created by information provided by administrators. The more information they have, the more comfortable they are moving forward. Other haplogroups covered are G, J1, J2, N, and I. Additional panels are slated for release in December. 80-100 panels to cover the existing Y-Tree."

Apparently, SNP pack development IS focused on the largest haplogroups. Those not in the major branches are NOT, in fact, in a position to motivate a tree update via an SNP pack development, but will just have to wait and cross their fingers. And, as SNP packs "are created by information provided by administrators", a "savvy customer" would not, in fact, generally be able to prompt an SNP pack development.

A customer who is stuck in an incorrect or out of date branch is apparently out of options, except to wait and hope, or to focus their attention on FTDNA's competitors.

From Jennifer Zinck's coverage of the FTDNA conference:

"Max noted that FTDNA is the only company that has their own lab. He noted the complexity that goes behind the panels and analyzing the results."

Apparently, Max is indeed worried about the competition, and is making comments to discourage customers from using them. Hopefully, this worry will prompt action regarding tree updates and maintenance, improved Big Y matching, proper distinction of shared versus private SNPs, and appropriate promotional ad targeting.

Although this tap dancing answer doesn't sound too promising:

From Jennifer Zinck's coverage of the FTDNA conference:

"Will the new complexity discovered by NGS be added to the dataset? If they find something that happens all the time, they will incorporate it. They need to do a lot more research into the complexities. If they can find meaning that will help you, then yes."

miiser
11-16-2015, 01:05 AM
"All of the primers have to be able to work together to create single base reactions."

The SNP pack does sound like some sort of multi-target combo SNP soup, rather than a microarray, Sanger, or such, otherwise the primers wouldn't have to "work together".

paulgill
11-16-2015, 01:14 AM
"All of the primers have to be able to work together to create single base reactions."

The SNP pack does sound like some sort of multi-target combo SNP soup, rather than a microarray, Sanger, or such, otherwise the primers wouldn't have to "work together".

How many Max, SNPs can be added on one of those chips?

miiser
11-16-2015, 01:39 AM
How many Max, SNPs can be added on one of those chips?

Well, if it is indeed a "soup" used with NGS technology, then it's not a "chip" and I think there is no clearly defined limit. But the more SNPs you include, the more difficult it will become to design primers that selectively bond only with the targeted segment and not interfere with the other targeted segments. Also, if you amplify everything together as a soup, a larger number of targets makes it more challenging to evenly amplify all of them. Some may tend to dominate and others may tend to get lost in the muddle. Perhaps 100 was FTDNA's initial guesstimate of a limit, and they are now experimenting to push the envelope beyond that.

paulgill
11-16-2015, 01:45 AM
Well, if it is indeed a "soup" used with NGS technology, then it's not a "chip" and I think there is no clearly defined limit. But the more SNPs you include, the more difficult it will become to design primers that selectively bond only with the targeted segment and not interfere with the other targeted segments. Also, if you amplify everything together as a soup, a larger number of targets makes it more challenging to evenly amplify all of them. Some may tend to dominate and others may tend to get lost in the muddle. Perhaps 100 was FTDNA's initial guesstimate of a limit, and they are now experimenting to push the envelope beyond that.

I think they already got 120 on J1 Pack. Changed to 167 SNPs now.

TigerMW
11-16-2015, 03:25 AM
From Jennifer Zinck's coverage of the FTDNA conference:

"So far they have created about 40 panels, which takes a large amount of vetting. All of the primers have to be able to work together to create single base reactions. They pull out the weaker ones. Once they have the whole mapping down they can’t remove one and add another. They are now trying to develop a system to help them with redesigning all of the panels."

From the horse's mouth - they're not planning to continuously update the SNP panels. Revisions have, in fact, been to adjust for problematic SNPs.
"
I'm not sure what your definition of "continuous" is but I don't think anyone has or can keep fixed SNP bundles updated in real-time. That's unrealistic in a fixed SNP bundle system.

FTDNA has added additional SNPs to their packs as new tree branching information comes in (not just for problematic SNPs.) That's a fact. The R1b-M222 refresh is proof positive. How often will they update? I don't know and I suspect it will never be often enough.

I'm not sure what is meant by "whole mapping". I think that may be a reference to the configuration between packs, not within packs. This is important as the bridge/pointer SNPs in the R1b-M343 Backbone SNP Pack and some of the other "top-layer" packs must adjust to whatever downstream packs are available. In other words, there has to be a coverage strategy and they would want to try to minimize overlap but insure complete coverage. Now I can understand why the mapping of subclade targeting between packs is something that could not be changed in wholesale. There are too many inter-connected pieces with potential ripple effects leaving customers out in the cold caught in between ripples.

I'm going to have to agree with Lawrence. I'm not sure how people are coming with some of these interpretations.

miiser
11-16-2015, 03:39 AM
I'm not sure what your definition of "continuous" is but I don't think anyone has or can keep fixed SNP bundles updated in real-time. That's unrealistic in a fixed SNP bundle system.

FTDNA has added additional SNPs to their packs as new tree branching information comes in (not just for problematic SNPs.) That's a fact. The R1b-M222 refresh is proof positive. How often will they update? I don't know and I suspect it will never be often enough.

I'm not sure what is meant by "whole mapping". I think that may be a reference to the configuration between packs, not within packs. This is important as the bridge/pointer SNPs in the R1b-M343 Backbone SNP Pack and some of the other "top-layer" packs must adjust to whatever downstream packs are available. In other words, there has to be a coverage strategy and they would want to try to minimize overlap but insure complete coverage.

My initial comment revolved around the question of whether FTDNA will be financially motivated to keep the tree updated once the initial development phase of SNP packs has passed. In response to my concern, lgmayka had suggested that FTDNA will continue updating the tree regularly, along with the SNP packs, well into the future as additional SNPs are discovered, and that recent SNP pack revisions was proof of this. I argued that the recent revisions were only part of an initial development phase to work out the kinks, and should not be taken as an indication of future behavior - that once the kinks were resolved, FTDNA would not continue updating the packs regularly. lgmayka apparently became engraged at my reply.

The comment I quoted directly from FTDNA's conference clearly shows that, contrary to lgmayka's claim, and regardless of what specifically "whole mapping" is intended to convey, FTDNA does view these revisions as initial development phase, with the expectation that the configuration of the packs will become stabilized and unchanged once the problem SNPs are addressed.

This was all I meant by the term "continuously", the distinction between the initial development phase for the SNP packs, which we are still in the midst of, versus normal on going operations into the future. I think the intent of my comment was fairly clear from the previous comments and the line of argumentation. I have not expressed any hope, demand, or expectation that FTDNA or any other company will update the tree and SNP packs "in real-time". But if you wish to take my comments out of context and debate against arguments that I have not made, I cannot prevent you from doing so, and would not seek to deny you whatever satisfaction it gives you.

TigerMW
11-16-2015, 04:10 AM
Here's a very obvious example.

FTDNA's haplotree shows Y35 on the same level as CTS3402 (this is within the R1a haplogroup). However, four different kits clearly show a green Y35 and a red CTS3402. Either FTDNA does not trust its own Big Y results sufficiently to change the haplotree, or FTDNA does not have the staff time to change the tree based on Big Y results. As far as the customer is concerned, those two alternatives are exactly the same!

I am sure this will get corrected if/when a SNP pack result shows Y35+ CTS3402- .

There must be more going on that we don't understand. I suspect FTDNA's Y tree has just not updated the tree due to being behind or just plain making a mistake.

Here is an example of where they added SNPs strictly from Big Y test results: Z16400, Z16403, Z16403, and Z16409 were all added at the same level along with CTS6621, CTS11744 under R1b-L513 (this is where I sit.) Those four Z SNPs have not been tested by other means at FTDNA although by now (with their L513 pack) that has changed.

There are many in L513 that I'm familiar of like this. I can't say whether or not they insist on Big Y calls they "PASSed" versus "REJECTED". I flooded them, though. We've had a lot of Big Ys in L513 (about 130) so if you have ten or twenty kits that back up your case for a tree change it is kind of hard to refute. I have to admit that I've had a more difficult time when there are only two or three derived SNP individuals in a branch.

TigerMW
11-16-2015, 04:15 AM
My initial comment revolved around the question of whether FTDNA will be financially motivated to keep the tree updated once the initial development phase of SNP packs has passed. In response to my concern, lgmayka had suggested that FTDNA will continue updating the tree regularly, along with the SNP packs, well into the future as additional SNPs are discovered, and that recent SNP pack revisions was proof of this. I argued that the recent revisions were only part of an initial development phase to work out the kinks, and should not be taken as an indication of future behavior - that once the kinks were resolved, FTDNA would not continue updating the packs regularly. lgmayka apparently became engraged at my reply.

The comment I quoted directly from FTDNA's conference clearly shows that, contrary to lgmayka's claim, and regardless of what specifically "whole mapping" is intended to convey, FTDNA does view these revisions as initial development phase, with the expectation that the configuration of the packs will become stabilized and unchanged once the problem SNPs are addressed.

This was all I meant by the term "continuously", the distinction between the initial development phase for the SNP packs, which we are still in the midst of, versus normal on going operations into the future. I think the intent of my comment was fairly clear from the previous comments and the line of argumentation. I have not expressed any hope, demand, or expectation that FTDNA or any other company will update the tree and SNP packs "in real-time". But if you wish to take my comments out of context and debate against arguments that I have not made, I cannot prevent you from doing so, and would not seek to deny you whatever satisfaction it gives you.
I don't think your intent is very clear unless I assume you have some kind of agenda. I won't assume that so I just find your interpretations some times seem confused and hard to understand.

Lgmayka is very reasonable. You can see places where he and I disagree strongly, but he is very reasonable and straightforward, which I greatly appreciate.

paulgill
11-16-2015, 04:20 AM
I don't think your intent is very clear unless I assume you have some kind of agenda. I won't assume that so I just find your interpretations some times seem confused and hard to understand.

Lgmayka is very reasonable. You can see places where he and I disagree strongly, but he is very reasonable and straightforward, which I greatly appreciate.

Agreed.

razyn
05-17-2016, 07:19 AM
The FTDNA haplotree still has DF27 and L194 on the same level (and some people are buying a SNP test for the latter -- about the longest shot there is, now that M153 isn't directly under P312). We have one kit (N2642) that tested L194+ via WTY in 2009; and later DF27+ (SNP test in 2012). His Geno2 results don't show Z195+, and he's had a Sanger sequenced Z196-. So he is at the very least in our big group E, DF27+ and Z195-. (Which I have started calling ZZ12+ because it takes fewer letters, in a limited caption.) I don't know what else should be required, to get L194 off their list of things every P312* guy might want to buy, just in case.

The DF27 SNP pack results are just in for kit N2642, showing that L194 is downstream of the more recently discovered BY3332. I have renamed this group (three guys who share a Rogers ancestor) Pda, to get them below BY3332. Their STR modals shift at several markers from those of the parent BY3332 group. The previous name of Pda was Pg, currently not a subgroup's name.

RobertCasey
05-17-2016, 05:05 PM
I'm not sure what your definition of "continuous" is but I don't think anyone has or can keep fixed SNP bundles updated in real-time. That's unrealistic in a fixed SNP bundle system.

FTDNA has added additional SNPs to their packs as new tree branching information comes in (not just for problematic SNPs.) That's a fact. The R1b-M222 refresh is proof positive. How often will they update? I don't know and I suspect it will never be often enough.

I'm not sure what is meant by "whole mapping". I think that may be a reference to the configuration between packs, not within packs. This is important as the bridge/pointer SNPs in the R1b-M343 Backbone SNP Pack and some of the other "top-layer" packs must adjust to whatever downstream packs are available. In other words, there has to be a coverage strategy and they would want to try to minimize overlap but insure complete coverage. Now I can understand why the mapping of subclade targeting between packs is something that could not be changed in wholesale. There are too many inter-connected pieces with potential ripple effects leaving customers out in the cold caught in between ripples.

I'm going to have to agree with Lawrence. I'm not sure how people are coming with some of these interpretations.

Mike, the equipment that FTDNA uses for their SNP packs was described in detail (Vince T):

You can review the MassARRAY system here (http://agenabio.com/products/massarray-system/). They've recently added a nice sales-pitch oriented video overview.

By my understanding, each target site uses a three-primer set with single base extension per variant target: two primers to amplify the target region, and one primer immediately adjacent to the variant site for the mass spectrometer to lock in on.

The main constraints are the number of multiplexed targets detectable per pad: the limit is 40.

All primers in the multiplex have to behave nicely with each other, and all need to have different lengths so that the mass spectrometer can discern them correctly. So there is significant fine-tuning involved to get each multiplex mix just right, but they have software to help with that. The problem is, that it becomes tricky every time additional targets are added to a multiplex. The multiplex may need to be redesigned to re-balance the primer mix, and this could have a domino effect if primers need to be re-jigged across several multiplexes to avoid discarding previously testable variants.

I think there was a nice technical overview provided at the 2015 Group Administrators Conference in Houston, and slides of the process were made available somewhere, but I've misplaced that link.

This test is somewhere between the Nat Geo chip array and Sanger Sequencing (but is much closer to Sanger sequencing). The FTDNA test requires a lot more fine tuning of primers which are dependent on each other. So, from just a technology point of view, this test has significantly more development costs which would result in fewer updates than the simplistic Sanger Sequencing setup costs. However, the test costs vary in multiples of 40 YSNPs and costs much less per YSNP than Sanger Sequencing. But FTDNA tends to test all branches of each haplogroup where YSEQ uses a three or four tiered approach that keeps their costs down since they effectively test five or ten times the YSNPs actually tested to yield the same results. But with much higher sales, FTDNA may be able to update more than technology alone due to higher volume.

To date, both YSEQ and FTDNA seem to be responding each others offerings but who knows the upper limits for either approach. Currently, both approaches do not make much progress as individual YSNP testing where YSEQ seems to excel by adding YSNPs upon request. To date, major branches for L226 via just Big Y is around $1,400 per branch discovered but individual YSNP testing has yielded new branches at only 10 % of the cost per YSNP. Individual testing discovered some pretty broad branches that were caught by Big Y testers. However, NGS tests do yield new private YSNPs for long term testing - but FTDNA's approach of shutting down individual YSNP testing is not the best thing for discovering branches faster and more economically.

TigerMW
08-30-2016, 09:49 PM
I posted this elsewhere but this is where it belongs.

FTDNA has been much more aggressive the last year and a half in updating their haplotree.

You can submit novel SNPs via the admin GAP tool by going to the genetic reports pull down menu and picking the Y DNA SNP report. On that page there is a big blue button to request SNPs. Make sure to fill in the SNP name if it is already named or they may pick a BY name. The other way to submit novel SNPs is do it in a request for a new SNP Pack product. Dr. Carlos Wolfgang Nossa is the contact for the Y team and heads SNP Pack development. This is the more effective way to get FTDNA's attention. If you can convince them that the pack you are proposing will sell you'll get faster response.

To get a large number of branches added to a subclade requesting an SNP Pack is truly the most effective way to proceed. However, you have to get your request for a pack in as FTDNA has a pretty good queue of packs in the backlog. They can actually produce them within a couple of months once they start work on a particular pack, but it is the wait time in the queue that is problematic.

To get single branches added to the haplotree or repositioned the right contact on the Y team is Michael Sager. He has been very responsive with people getting branches added or corrected within 24-48 hours in many cases. Rory C. pointed him out to me. I think Rory has thoroughly done a great job with his subclade. Few people realize the amount of work a person like Rory has put in.

Michael can look directly at BAM files, x identity%'s, etc. so you don't need to submit that kind of detailed information. You just need to email clear instructions on the parent branch, parallel branches, and child branches as is appropriate. You need to provide kit #s that show the proving results in the FTDNA database so the Y DNA SNP report project screens are immensely helpful. I download them into a spreadsheet so I can do Excel column heading autofiltering to find the row/kit # I need.

They do have Advanced Test Sanger, Big Y NGS and SNP Pack results in their database now. It's been integrated for over a year.

There is no substitute for sound judgement and good analytical work on the part of a project administrator.