PDA

View Full Version : Behind the scenes at a DNA testing company



therrien.joel
12-08-2017, 03:30 AM
Just read an interesting article about someone's experience working at one of the bigger names in DNA testing. http://www.cracked.com/personal-experiences-2522-inside-shady-world-dna-testing-companies.html

The first part about accuracy of the tests is no surprise to anyone on this forum, I'd imagine. But I never thought they'd have to worry about people actually showing up at their doorstep fuming about their results. Very interesting read.

MitchellSince1893
12-08-2017, 03:41 AM
I thought this quote was informative
"If you get a high percentage, it's a safe bet that you have ancestors from there. I'm talking about a 50-60 percent on your test. Anything lower, and take it with a grain of salt."

and this one

"One woman sent the packet out with green shamrocks and a green leprechaun hat on it. She was really proud to be Irish. She even said she was excited to see if she was 100 percent Irish. But the test found no Irish blood. It was half Eastern Europe, then a mix of different places in Germany and Italy, and even Greece." Clearly, this woman's family had either lied to her, been lied to themselves, or she was one of those stealth adoptions that happen every so often. "The consensus was that she would send a huge fit if she was shown not to be Irish at all, so we made her 20 percent Irish and highlighted our disclaimer about results not being accurate."

BalkanKiwi
12-08-2017, 06:02 AM
This has some of the funniest stuff I've heard all week (not including going to the office to threaten). I can't believe someone would take it that seriously.

lgmayka
12-08-2017, 09:36 PM
We can't be sure that the article is factual because Cracked specifically admits that some of its material is fiction or semifiction (http://www.cracked.com/terms-and-conditions.html). But if the article is an accurate account, we need to say the obvious: That specific testing company is committing fraud, by any reasonable definition of the term. People who pay good money for a DNA test advertised as scientific, expect results based on science, not cowardice or spite.

The article makes clear that these swindlers use "spit" (saliva) rather than a cheek swab. Thus, the company is not Family Tree DNA, obviously.

therrien.joel
12-09-2017, 12:18 AM
We can't be sure that the article is factual because Cracked specifically admits that some of its material is fiction or semifiction (http://www.cracked.com/terms-and-conditions.html). But if the article is an accurate account, we need to say the obvious: That specific testing company is committing fraud, by any reasonable definition of the term. People who pay good money for a DNA test advertised as scientific, expect results based on science, not cowardice or spite.

The article makes clear that these swindlers use "spit" (saliva) rather than a cheek swab. Thus, the company is not Family Tree DNA, obviously.

Cracked typically makes it quite clear what is fiction/semifiction, so I would say this is probably a true account.

Donwulff
12-09-2017, 02:40 AM
Under most personal information laws, if you have evidence (Including family stories etc.) that your personal information on a site is wrong, they need to change it. So I don't think that changing people's ethnic break-down by demand is wrong per se, as long as it's not used for research, and the person in question knows exactly what they did. Also this would be understandable, even commendable in case that a biogegraphical analysis showed someone had 10% of British or German, but they had genealogical evidence they had a fully British grandfather. However, big DNA-testing companies *do not have time* to personally customize each customers results individually. Which brings us to the main point, that the article is completely made up and supposed to be funny, I guess.

Case in point, the prospective whistleblower recounts how he was there for their internal tests on the ancestry analysis, but then few paragraphs down recounts how he wasn't yet working there when a customer angry about their ancestry analysis results came in. And even if someone's teeth were really breaking off pieces for months on an end, and any of the major testing companies actually kept re-sending new kits for that long, the teeth are just calcium. It does nothing to DNA analysis, and gets filtered out anyway. Swigging a chug of Listerine, as the supposed DNA-testing company employee says you're *supposed* to do, would just wash out any DNA-containing leukocytes and epithelial cells in the mouth and potentially mess up with the DNA testing chemistry. More importantly, this is not the recommended testing practice at any company I have heard of. Not only isn't the "whistleblower" a DNA-company employee, but he clearly knows nothing about DNA-testing to the point of probably never having taken one.

But then, it's a satire/humor magazine. More insidious is the article's contention that DNA-testing companies are part of a conspiracy to mess up with people who oppose multiculturalism, and make people look more multicultural in general.

Geborgenheit
12-09-2017, 08:43 AM
This article is clearly a joke, but in every joke there is some truth. I wonder myself if DNA testing companies modify results depending on location and names of customers. It would be fraud, but I think it's possible.

RobinBMc
12-09-2017, 05:39 PM
I'll be the first to admit ethnicity reports are only estimates which vary depending on the company/calculator. But this article is very misleading.

"But when Inside Edition had a set of triplets send their spit in to Ancestry.com and 23andMe, they got wildly different results from both services. Neither gave each triplet the same ancestry results -- which, considering they all came from the same womb, is pretty weird."

The fact that they came from the same womb does not mean they should have exact same results. It's the fact that they are identical twins that means they should all have the exact same DNA, and therefore their results from the same company shouldn't vary. But they neglect to say their results from the same company only vary by 1-2% IIRC, which is fairly negligible, if you ask me, and I wonder if it maybe has to do with the lab testing and raw data (maybe one or two of the triplets was missing an SNP here or there during the chip's extraction?).

They claim: "Inside Edition found differences of over 10 percent between the triplets they tested." - maybe between the two companies - but I distinctly remember checking into this and the differences between the triplets from AncestryDNA was only 1-2%. This is misleading at best, and just a lie at worst.

It goes on to say:

"Some companies may use 12, 37, or 67, while others claim to use more than 700,000 different markers."

Those are different types of DNA. 12, 37, and 67 is Y-DNA, 700,000 is autosomal DNA. They are not comparable, and Y-DNA doesn't produce an ethnicity report. Did the author of this even do ANY research before writing it? Internet journalism is such a joke.

I'm not even going to bother reading the rest.

Donwulff
12-09-2017, 09:32 PM
Inside Edition is similar category to Cracked, being the original example of Tabloid Journalism. They manufacture reports for controversy, not for fair and accurate reporting. In their response that other media (But not Inside Edition who naturally left that part out) printed, 23andMe in particular responded that they believed Inside Edition compared the triplets using different confidence levels (23andMe allows you to use one of three confidence levels, which generally change how much of the percentages are assigned into "broadly" instead of specific categories). Due to privacy issues, 23andMe clearly can't state whether or not the triplets did have similar results or not.
At least the Admixture style analysis used by FTDNA and AncestryDNA, I believe, includes a stochastic component where the algorithm is run intentionally with a few different random variations with the intent of finding average results that are robust against small errors in genotyping and population data. This can lead to slightly different results from the same data, though obviously the very intent is to get same data from different runs. Inside Edition quoted FTDNA as saying "Family Tree DNA told Inside Edition they have improved their algorithm and will implement a new method in the next few weeks" so at least according to Inside Edition, FTDNA did admit they at least had a problem. No idea about AncestryDNA's response.

Though again, more to the point, I couldn't find a single verifiable statement in the Cracked article that checked out. "considering they all came from the same womb, is pretty weird" reflects a common misunderstanding between genetic and genealogical ancestry, however, and it seems to be the article writers own contribution rather than the alleged DNA-company whistleblower's, so I'm willing to give it a pass as far as the intent is concerned. Incredibly, the author seems to accidentally get it essentially "right" in this case as well since they were identical twins, not because they came from the same womb. (Interestingly, it would be possible for the algorithms to detect for example that three people are siblings, use them to construct a model for their parents, and give each one the same ancestry breakdown based on that model. I hope to see this in the near future, and 23andMe already uses phasing when parent is available, though it will muddle the line between genetic and genealogical ancestry.)

So if you were a DNA-company employee intending to whistleblow about abuses within the industry, would you pick a failing (they laid off majority of their employees this month) MAD-magazine spin-off satire site like Cracked.com to spill the beans, or would you sell the story to any number of investigative rags willing to pay big money for it instead of the $50 Cracked.com reportedly pays to the author (not the supposed whistleblower)?
The question is rhetorical, of course, since Cracked.com makes no secret it's a comedy site ("America's Only Humor Site: A funny website filled with funny videos, pics, articles, and a whole bunch of other funny stuff." their front-page proudly declares), that publishes mainly lies (https://www.vice.com/en_us/article/5gwen8/crackedcom-is-full-of-lies - yes, yes, I get that article in itself is party satirical). Unfortunately, you've still got the "Okay, so the article is full of lies and incorrect information, but... I want it to be true, so it must be at least partly true!" folks. This is how myths get started, people, and then you've got people still insisting the Earth is flat because they room-mates father's buddy once read this article about it...

Simple reality check: As I said, companies that are testing MILLIONS of people don't have time or manpower to customize reports for everybody. If you used a hour to dig into the (usually unavailable) background of a million people each, you'd need 500 man-work-years. AncestryDNA was at about 5 million and sold 1.5 million additional kits during Black Friday so a rough estimate of 3250 man-work-years *each and every time* they update their ancestry-estimates algorithm. Even if you argue it could be done in half a hour or less, they would be hiring tens of thousands of people to get it done in reasonable time, and somebody else than Cracked.com would take notice. And yet the biggest if not only complaint people have about DNA testing is the ancestry results don't match their expectations. Which gets us to the final point, that since the DNA-companies don't know each individuals expectations (beyond that they expect to get information that helps them identify ancestors, and preferably a Native American or two), it would be pointless for them to change them.

lgmayka
12-10-2017, 12:12 AM
Unfortunately, you've still got the "Okay, so the article is full of lies and incorrect information, but... I want it to be true, so it must be at least partly true!" folks.
You bring up an important point.

Truly professional liars--employed by a satire magazine, a supermarket tabloid, a government's secret agency, etc.--do not merely spit out a pack of 100% lies. That's too obvious. Truly professional prevarication requires an interleaved mixture of truth and falsehood, woven so as to be at least superficially plausible and, more important, difficult to disprove. Anonymous unverifiable sources are a standard technique, as are detailed, somewhat humorous anecdotes to give the reader that I'm-just-an-ordinary-guy feeling.

therrien.joel
12-10-2017, 02:41 AM
Well this certainly has been enlightening to read the comments on this. I guess my takeaway is that I'll stick to reading Cracked for the entertainment value.

Bear in mind, when I read it, I was not of the impression that this was necessarily recounting a recent experience. If you read the way the operation is described it sounds way more like an early startup experience. So,
there would not have been the millions of samples being tested at that time. And I can only speak for myself that I did not come away from that thinking this meant that they changed results routinely. It was fascinating to see many of those who responded to that post having the exact opposite conclusion. But I was more interested by their experience working with people who find that their expectations do not match the facts (i.e. not being Irish for example). It reflects what we see here pretty often. Is there even a day where we don't see someone post something along the lines of "I got a strange result from X"?

But yes, I will agree that Cracked has wannabe journalists that are obviously not digging deeply into the background to find out the facts on their statements. As for the professional liars cleverly crafting a mix of truths and lies... eh, that has been your everyday person's life for millennia. They didn't invent it, they just took advantage of the fact that these days anyone can publish what they think and the vast majority of that is exactly this mix of truth and lies. It just so happens that it is a very good delivery mechanism for more nefarious stuff.

Donwulff
12-10-2017, 05:12 AM
I'm certain changing results at the account-holders request has happened at some of the testing companies at some point. By the converse of "it would take too much time", there's just so many testing companies and so many people tested. The thing is, that's not illegal or even unethical, in fact arguably the companies have a legal obligation to correct the results if the customer can show they're wrong. It also hurts no-one, even though I don't particularly agree with it, since the person receiving the results knows exactly what they did. Just thought to try to nip in the bud the myth/rumor that there's supposedly truthiness to the article because it's "possible" that they're adjusting the results based on the customer's personal information, or whether they're a racist or not or whatever. Trouble is, it's hitting fertile ground because for whatever reason people are already biased against at-home genetic testing or biogeographical ancestry analysis (I suspect not enough Native American on their ancestry reports... haha, I kid;)

Although actually, Cracked.com kind of DID invent the "Made-up listicle told tongue-in-cheek"-style with authors getting paid fixed sum per article they churn out en masse with little thought. I was kind of hedging my bets on this article, because it's generally impossible to prove a negative and it's interesting to see other people's opinions. But the avenue makes no sense for this kind of expose, the source's timeline of happenings isn't logical, the things they cite as messing up with DNA testing don't, but Listerine probably would, and by the categories they cite it sounds like they've never seen an actual DNA ancestry report (Though the image on the story comes from youngandsavvygenealogists.blogspot.com/2014/07/my-ancestrydna-review-cautionary-tale.html so I guess at least they Googled that picture). So yeah, I did call BS on this, the author most likely made up the interview because they don't pay authors enough to conduct actual interview. And scientific & genetic testing communities should probably do more to call out made-up scaretales rather than say there's a kernel of truth to every accusation and hey, you can't positively prove that it isn't going on even if this one is made up.

RobinBMc
12-11-2017, 05:22 PM
I'll be the first to admit ethnicity reports are only estimates which vary depending on the company/calculator. But this article is very misleading.

"But when Inside Edition had a set of triplets send their spit in to Ancestry.com and 23andMe, they got wildly different results from both services. Neither gave each triplet the same ancestry results -- which, considering they all came from the same womb, is pretty weird."

The fact that they came from the same womb does not mean they should have exact same results. It's the fact that they are identical twins that means they should all have the exact same DNA, and therefore their results from the same company shouldn't vary. But they neglect to say their results from the same company only vary by 1-2% IIRC, which is fairly negligible, if you ask me, and I wonder if it maybe has to do with the lab testing and raw data (maybe one or two of the triplets was missing an SNP here or there during the chip's extraction?).

They claim: "Inside Edition found differences of over 10 percent between the triplets they tested." - maybe between the two companies - but I distinctly remember checking into this and the differences between the triplets from AncestryDNA was only 1-2%. This is misleading at best, and just a lie at worst.

It goes on to say:

"Some companies may use 12, 37, or 67, while others claim to use more than 700,000 different markers."

Those are different types of DNA. 12, 37, and 67 is Y-DNA, 700,000 is autosomal DNA. They are not comparable, and Y-DNA doesn't produce an ethnicity report. Did the author of this even do ANY research before writing it? Internet journalism is such a joke.

I'm not even going to bother reading the rest.

Okay, I looked into the details again - the results from 23andMe varied greatly between the triplets, as did the results from FTDNA. But AncestryDNA was only off by 1-2%, that's what I was thinking of. (And mine you, they used different sets of triplets for each company).

23andMe: "Nicole was 11 percent French and German but Erica was 22.3 percent. Their sister Jaclyn was in the middle at 18 percent.

FTDNA (British Isles): "Erin Maynard was 59 percent, Mandy was 66 percent and Melissa was 70 percent." and "The tests also showed that Mandy had six percent Scandinavian ancestry but her identical sisters showed none."

AncestryDNA: "It revealed that their roots are largely from Great Britain, 45%-47% and their Italian and Greek ancestry was exact at 25%."

But then again, the identical quadruplets they tested at 23andMe had identical ethnicity results, so it's not necessary the company as a whole that's unreliable.

As far as I know, all tests confirmed they were identicals, not fraternal, it's only the ethnicity reports that weren't identical. I could excuse a 1-2% difference as negligible, but the larger differences up to 11% are weird. I can still only suggest that it's something to do with the fact that they are not using the full genome, only a portion of it, and the portion that gets extracted isn't always exactly the same every time. I wouldn't have expected it to be as much as 11% different since I believe theoretically, the chip is supposed to extract the exact same portions of DNA every time, but I am certainly no expert on how the chip works.

Saetro
12-11-2017, 08:03 PM
The original program on this was entitled "How reliable are home ethnicity tests?"
The implication of the "home" qualifier is that other tests - from a doctor, maybe, or forensic - are more reliable.
I have read quite a few papers on forensic DNA testing, and many support tests that are far more limited in what they test, so they would probably be more reproducible on identical triplets and quads.
But that says absolutely nothing about their accuracy.
They could be 100% reproducible and 100% wrong.

The main thing here is that the current state of science is not sufficient to produce the sort of absolute result that some may want.
But life is much more interesting than that!

Osiris
12-11-2017, 09:17 PM
Would no reads be the source of some of the differences for the triplets? My mother and I both tested at 23andme and i investigated several oddities that popped up during phasing that came fromher having no reads at certain points.

RobinBMc
12-11-2017, 09:28 PM
The original program on this was entitled "How reliable are home ethnicity tests?"
The implication of the "home" qualifier is that other tests - from a doctor, maybe, or forensic - are more reliable.
I have read quite a few papers on forensic DNA testing, and many support tests that are far more limited in what they test, so they would probably be more reproducible on identical triplets and quads.
But that says absolutely nothing about their accuracy.
They could be 100% reproducible and 100% wrong.

The main thing here is that the current state of science is not sufficient to produce the sort of absolute result that some may want.
But life is much more interesting than that!

Testing your full genome is currently possible, if that's what you mean, just incredibly expensive.

Donwulff
12-12-2017, 02:07 AM
In http://www.healthyway.com/content/these-identical-triplets-were-shocked-to-learn-that-their-dna-didnt-match (this article seems to feature few other inaccuracies/misleading statements in itself, but it's the questions we're interested in!):

'To get to the bottom of this, we reached out to 23andMe, the service that interpreted the genetic test for the Dahm sisters.

"It's a misleading story," a spokesperson told us via email. "Our Ancestry Composition feature allows you to select varying confidence levels, between 50-90 percent."
"In this case I believe they were changing their confidence levels and making comparisons," continues their spokesperson. "If you drop to the 'speculative' selection, which has a confidence level of 50 percent you are going throw off the comparisons. Even when they did that, there wasn't that great of a difference."'

To be sure, I'm not entirely clear on here whether 23andMe is saying that they used different confidence levels for the different twins, or that they just chose a very low confidence level. Due to legal privacy & libel concerns, 23andMe can't afford to be very precise on their statement here, but the overall implication of the article seems to be that they used different confidence levels for the different twins to generate controversy (Which is what Inside Edition does: 'Inside Edition has been characterized by the Pew Project for Excellence in Journalism as "tabloid press" and a "pseudo news program".'). Inside Edition's own article simply quoted the company spokesmen:

"Since the test, Family Tree DNA told Inside Edition they have improved their algorithm and will implement a new method in the next few weeks.

A spokesperson for 23andMe says their results are based on a sliding confidence scale, ranging from 50-90 percent. The higher the confidence level chosen, the less specific the result can be as to the region or country of the person’s ancestry."

So it seems that FTDNA admits their previous method was poor. 23andMe's response quoted by Inside Edition is just a generic "the user can choose the results they get".