PDA

View Full Version : Living DNA raw data downloads are now available



Ann Turner
06-11-2017, 12:22 AM
This has been mentioned in some other threads, but it deserves a thread of its own.

The autosomal results are in VCF format. Has anybody already written a utility to convert them to a form that can be uploaded to GEDmatch?

The mtDNA results come in a CSV file showing only the differences from the CRS and gives the allele, e.g. 263G. The results for my son are very sparse compared even to 23andMe.

The Y results come in a CSV file. My son shows 585 SNPs but they don't give an allele.

I'm traveling right now, so I won't have the time to look at things in more detail for a few days, but others may have some useful observations.

firemonkey
06-11-2017, 12:51 AM
Why couldn't Living dna provide a zip file of the raw data like other companies? Even if there is a way to convert the VCF it may be too difficult for those of us who are not tech savvy to do. Bad move by them.

deadly77
06-11-2017, 01:41 AM
My raw data isn't available yet. I couldn't find anything from LivingDNA, but Eupedia said that LivingDNA test 4700 mtDNA SNPs http://www.eupedia.com/europe/dna_project_faq.shtml

AntG
06-11-2017, 07:40 AM
Gedmatch Genesis can accept VCF files but it's a beta site and doesn't have the amount of people that standard Gedmatch has. Hopefully LivingDNA/Gedmatch can sort this so standard Gedmatch can accept LivingDNA results. Gedmatch Genesis is at https://genesis.gedmatch.com/ (use same logon as per standard Gedmatch).

deadly77
06-11-2017, 08:13 AM
Ok, Mine is available now. It looks like the mtDNA and the YDNA are excel files which only list the positive calls and not the negative ones. My mtDNA lists 20 SNPs but it's enough to give me the same haplogroup designation as a full sequence mtDNA from FTDNA, but not my private mutation. I'd like to see the negative calls too so I can see what the coverage is. YDNA was an excel file with 595 positive SNPs.

ajc347
06-11-2017, 10:05 AM
Gedmatch Genesis can accept VCF files but it's a beta site and doesn't have the amount of people that standard Gedmatch has. Hopefully LivingDNA/Gedmatch can sort this so standard Gedmatch can accept LivingDNA results. Gedmatch Genesis is at https://genesis.gedmatch.com/ (use same logon as per standard Gedmatch).

Thank you for this information. I've just successfully uploaded my raw data onto Genesis without problem. :)

There's no admixture tools available on Genesis as yet, and I'm hopeful that they will be added in due course. It'll be interesting, once this occurs, to see how the admixture results compare to those generated using data from other companies.

ollie444
06-11-2017, 10:44 AM
my mt dna and my grandfather's both have only 5 SNPs listed (are these actually SNPs?).

genesis kit no. YB5825449 (me) RM6757535 (paternal grandfather)

ollie444
06-11-2017, 11:06 AM
Thank you for this information. I've just successfully uploaded my raw data onto Genesis without problem. :)

There's no admixture tools available on Genesis as yet, and I'm hopeful that they will be added in due course. It'll be interesting, once this occurs, to see how the admixture results compare to those generated using data from other companies.

Both my kits match with you: 16783 16784

ajc347
06-11-2017, 11:48 AM
Both my kits match with you: 16783 16784

Awesome. :)

I've sent you over a PM.

AntG
06-11-2017, 12:14 PM
I did notice that my mother matches David Nicholson from LivingDNA!

firemonkey
06-11-2017, 01:02 PM
Just uploaded my VCF. Showing no matches .

ajc347
06-11-2017, 01:16 PM
Give it a couple of hours and they should hopefully show up. :)

firemonkey
06-11-2017, 01:45 PM
my mt dna and my grandfather's both have only 5 SNPs listed (are these actually SNPs?).

genesis kit no. YB5825449 (me) RM6757535 (paternal grandfather)


On my UB1655155 kit

YB5825449 A *ollie444 5.207 5.207 7.7 164252 2017-06-11

However when I click on the A the snp count is very low.

sktibo
06-11-2017, 07:37 PM
Hopefully they're working on making it Gedmatch compatible, not just Genesis.

Mike_G
06-11-2017, 09:39 PM
Both my kits match with you: 16783 16784

He's a match for me too.

16790

ollie444
06-12-2017, 10:31 AM
He's a match for me too.

16790

ha, we're all related ;)

ajc347
06-12-2017, 12:41 PM
It looks like there may be a problem with the matches being shown using Living DNA raw data in Gedmatch Genesis.

On three occasions this morning I've seen a positive match for myself and another person generated on Genesis (using Living DNA data) which has then been shown to be a negative match for the two of us when the standard (i.e. non-Living DNA data) Gedmatch kits are compared.

This would suggest to me that Genesis may potentially reporting false positive results when using Living DNA data. :(

ollie444
06-12-2017, 12:56 PM
ha, we're all related ;)


It looks like there may be a problem with the matches being shown using Living DNA raw data in Gedmatch Genesis.

On three occasions this morning I've seen a positive match for myself and another person generated on Genesis (using Living DNA data) which has then been shown to be a negative match for the two of us when the standard (i.e. non-Living DNA data) Gedmatch kits are compared.

This would suggest to me that Genesis may potentially reporting false positive results when using Living DNA data. :(

Or it could be that my ancestry kit and your ftdna kit don't test for the SNPs we share, but Living DNA does. It does seem staggering that I'm matching so many people on this forum, but it's usually at about 7 generations back - I've found that's normally a fairly optimistic estimate. Something to keep an eye on though.

Amerijoe
06-14-2017, 06:12 PM
Just received this email


Respond to this post by replying above this line
New post on DNAeXplained - Genetic Genealogy


Beware – LivingDNA Requires Customer to Indemnify Company to Download Raw Data
by robertajestes
Say what?

Yep, you read that right.

I discovered that LivingDNA now provides a raw data download for their customers, with a huge, and I mean HUGE, caveat – indemnification of the company (LivingDNA) if they are sued as a result of your download of your DNA data.

Yes, seriously! This is not April 1st.

While this may sound like a trivial complication, it's not, given that one of the reasons genealogists purchase ethnicity tests is to download the data and transfer the results to other companies or services for additional ethnicity results or matches to other testers, a feature not provided by LivingDNA.

Let’s walk through the download steps and take a look.

At LivingDNA, your raw data is now available on the left hand side of your page. Click on “Download Raw Data” to view.



You will then see the following screen, captured in two graphics, below. Please note that you can click to enlarge any graphic.

Do NOT do anything else until you READ and understand the entire contents of the Download Raw Data page.





LivingDNA provides what I would consider typical verbiage and also some appropriate cautions about how you may or may not match people you expect to match, and you may match people you don’t expect to match. In other words, they are gracefully trying to say you may encounter a misattributed parentage, either in your line or the line of someone you expect to match. Given that LivingDNA doesn’t provide matching services, this is especially important for their customers to understand.

But then comes the (thankfully) bolded bombshell. Bolding is theirs, not mine, although I would also add the color red.

By choosing to download your data you agree to indemnify (which broadly means to reimburse) Living DNA and its related companies and their directors and employees for any losses, damages or costs they incur as a result of any claims being made against them which relate to you downloading your data and the use by you of your data, or as a result of you having shared your data with any third party.

Holy cow.

Not only are you indemnifying LivingDNA, you're also indemnifying their directors and employees and related companies. Furthermore, "claims" could mean that someone doesn't even need to file a lawsuit. As I said, I'm not an attorney, but I'm a savvy enough consumer to know this isn't good for me.

You must click the “consent” box in order to proceed.



The consent says very clearly that by downloading, you agree to provide indemnity to LivingDNA. Bolding and red, both mine, below.

I have read the information provided about gaining access to my genetic data, and in particular I understand that my use of my data is my responsibility and that by downloading my data, I am providing an indemnity to Living DNA.

Merriam-Webster says this about indemnify:



I’m not a lawyer, but let me explain one thing further about indemnification. It includes the costs of defense – meaning the lawyers, and the lawyers travel, etc. Lawyer fees alone can run into tens of thousands of dollars, and more. Currently intellectual property attorneys bill at the rate of $450 per hour, or did last year, and defense preparations take hundreds of hours. If this makes you shake in your shoes, it should!

So let me say this in plain English. If you upload your LivingDNA file to any third party site and you match someone who is angry that your match revealed (or helped to reveal) that their father is not their father (for example), but is instead your father, or uncle, or cousin, etc., and they decide to sue LivingDNA for running your test – you have to pay for LivingDNA to defend themselves against the lawsuit – no matter how frivolous and no matter the outcome. I would think this would also extend to someone utilizing numerous matches to discover a link to unknown parentage if someone is unhappy about the outcome. In essence, if anyone sues LivingDNA or make a claim over anything having to do with your test, you have agreed to pay for LivingDNA to defend themselves, affiliated companies, employees and directors, even if the suing party loses. And if the suing party wins, you get to pay for that too.

The bottom line is that you have to agree that you are responsible for whatever after downloading your DNA. "Whatever" means anything that you can think of, and probably several things you can't. The example above is by no means a comprehensive list of what could go wrong and cause you a massive legal headache. If anything goes wrong after you download your DNA, you're responsible.

Period.

Consider yourself warned.

There is absolutely no upside or benefit in this verbiage to you. None. Nada.

So what am I going to do with my LivingDNA results? Not one single solitary thing. The ante is just too large. Thankfully, I’ve already tested with the other vendors so I don’t need to upload my results from LivingDNA anyplace.

That’s exactly what I recommend you do too – nothing. Don't even download. Personally, I would simply test elsewhere, all things considered.

You cannot control how your matches utilize the fact that you match and what they do with that information. It will be interesting to see if LivingDNA will require their customers to indemnify them against the results of matches at their own company if they add the matching feature as they have stated they plan to do.

Given that I'm not an attorney, if you are considering downloading your LivingDNA data and uploading elsewhere, I strongly, STRONGLY, recommend that you contact an attorney and obtain a professional opinion.

Today, as I write this, I know that Family Tree DNA and GedMatch don’t accept LivingDNA files as a standard upload because the chip LivingDNA uses is different than any other vendor.

Even if everyone accepted LivingDNA files, in my opinion, given the LivingDNA indemnity language, if you want to upload your results to any site, you would be far safer to test a second time with one of the three major vendors and avoid the potential indemnity headache.

Click here to read my LivingDNA Product Review that was written a month ago, before the data download become available.

You can see which vendors accept whose transfer files in the article, Autosomal DNA Transfers – Which Companies Accept Which Tests?

Within the next few days, I'll be publishing an article titled, "Which Ethnicity Test is Best?," so stay tuned.

__________________________________________________ ____________________

Standard Disclosure

This standard disclosure will now appear at the bottom of every article in compliance with the FTC Guidelines.

Hot links are provided to Family Tree DNA, where appropriate. If you wish to purchase one of their products, and you click through one of the links in an article to Family Tree DNA, or on the sidebar of this blog, I receive a small contribution if you make a purchase. Clicking through the link does not affect the price you pay. This affiliate relationship helps to keep this publication, with more than 850 articles about all aspects of genetic genealogy, free for everyone.

I do not accept sponsorship for this blog, nor do I write paid articles, nor do I accept contributions of any type from any vendor in order to review any product, etc. In fact, I pay a premium price to prevent ads from appearing on this blog.

When reviewing products, in most cases, I pay the same price and order in the same way as any other consumer. If not, I state very clearly in the article any special consideration received. In other words, you are reading my opinions as a long-time consumer and consultant in the genetic genealogy field.

I will never link to a product about which I have reservations or qualms, either about the product or about the company offering the product. I only recommend products that I use myself and bring value to the genetic genealogy community. If you wonder why there aren’t more links, that’s why and that’s my commitment to you.

Thank you for your readership, your ongoing support and for purchasing through the affiliate link if you are interested in making a purchase at Family Tree DNA.

robertajestes | June 14, 2017 at 5:54 pm | Categories: Legalities, LivingDNA | URL: http://wp.me/p2AwO4-4xh
Comment See all comments
Unsubscribe to no longer receive posts from DNAeXplained - Genetic Genealogy.
Change your email settings at Manage Subscriptions.

Trouble clicking? Copy and paste this URL into your browser:
http://dna-explained.com/2017/06/14/beware-livingdna-requires-customer-to-indemnify-company-to-download-raw-data/
Thanks for flying with WordPress.com

ajc347
06-14-2017, 06:48 PM
Wow. �� That's scary stuff and my kit is definitely going to be deleted from Genesis later tonight.

I was working on the presumption that the indemnification related to my suing them and hadn't given any thought to the involvement of third parties.

Thank you for posting up the above information - it's been really helpful.

jolson
06-15-2017, 09:47 PM
There's no admixture tools available on Genesis as yet, and I'm hopeful that they will be added in due course. It'll be interesting, once this occurs, to see how the admixture results compare to those generated using data from other companies.

Genesis Admixture is almost ready. Probably sometime this weekend.

EDIT: Genesis Admixture is now online.

Saetro
06-16-2017, 02:33 AM
Just received this email

Respond to this post by replying above this line
New post on DNAeXplained - Genetic Genealogy

Beware – LivingDNA Requires Customer to Indemnify Company to Download Raw Data
by robertajestes
Say what?

Yep, you read that right.

I discovered that LivingDNA now provides a raw data download for their customers, with a huge, and I mean HUGE, caveat – indemnification of the company (LivingDNA) if they are sued as a result of your download of your DNA data.


Whoa!!!
Roberta Estes has an update that shows LivingDNA has responded to customer concerns https://dna-explained.com/2017/06/15/livingdna-replaces-download-terms/
Please read this update before acting.

MacUalraig
06-16-2017, 07:41 AM
Back on the technical side: LDNA posted this yesterday

"I have spoken to the team at GedMatch, who we have been working with since late 2016, and I can confirm they are working on making all their tools compatible throughout the Genesis project. Once that is complete, they will merge their databases."

This implies that the current format isn't going to be implemented as an input format on the current version of GEDMatch.

ollie444
06-16-2017, 10:14 AM
Back on the technical side: LDNA posted this yesterday

"I have spoken to the team at GedMatch, who we have been working with since late 2016, and I can confirm they are working on making all their tools compatible throughout the Genesis project. Once that is complete, they will merge their databases."

This implies that the current format isn't going to be implemented as an input format on the current version of GEDMatch.

does it not imply that there will be no gedmatch or genesis, but a new superior version. It will probably be gedmatch with vcf in addition to the normal gedmatch features.

Pylsteen
06-16-2017, 10:56 AM
Interesting.. when the product is ready, I will take a look.

jolson
06-16-2017, 01:25 PM
does it not imply that there will be no gedmatch or genesis, but a new superior version. It will probably be gedmatch with vcf in addition to the normal gedmatch features.

That is correct. After we have completed porting GEDmatch applications over to Genesis, the databases will be merged, and there will only be a single site. Our goal is to complete this transition by Fall 2017, but there is still a lot of work to do, and there is no hard date for completion. Much of the work involves converting over to a new internal format designed to better accommodate testing companies other than "The Big 3", and to fine tune a new comparison algorithm that will provide better accuracy with less overlap between kits being compared.

The provision for accepting .VCF files was made primarily to accommodate large datasets as "variant" files. This includes traditional genealogical raw data as well as "whole genomes", Exomes, etc. The VCF format also has provisions for additional information such as the quality of the read, and the allele frequency in the subject population. We are also likely to be adding capability to upload TSV (Tab separated values) format files to better accommodate uploads from companies such as Complete Genomics.

Xtian
06-16-2017, 01:46 PM
wow its up and working

any advice on Gedmatch admixture? its very complicated

06-16-2017, 01:59 PM
wow its up and working

any advice on Gedmatch admixture? its very complicated

Depends where your from Xtian, if your European, then I recommend Eurogenes K13 calc.

06-16-2017, 02:01 PM
I created a thread which compared both my Gedmatch and Gedmatch Genesis results for K13,
http://www.anthrogenica.com/showthread.php?10954-GedMatch-Genesis-(with-LivingDNA)-v-GedMatch-(23andme)-Eurogenes-K13

Xtian
06-16-2017, 02:06 PM
Southern Russian

I get 14% West Med and 12% West Asian which is unusual.
44% Baltic 22% North Atlantic and some noise.


What does Mixed Mode Population Sharing:

jolson
06-16-2017, 02:13 PM
There are about 50 +/- admixture "calculators" on GEDmatch (main page, not Genesis). There are a couple of strategies you can use to narrow the choices.

1. If you have no idea of the ethnicity of the kit you want to test, I suggest starting with Dodecad / World9 or maybe one of the other "World" calculators.

2. One feature that is probably underused is the link from the GEDmatch home page: "Admixture/Oracle with Population Search". Enter an abreviated name for a single ethnicity. Short substrings work better than full names for ethnicities. eg. "mex" rather than "mexican", or "asia" rather than "asian". You will get a list of all calculators and the matching populations they support. If you have a mix of ethnicities, try each one, and look for calculators that support the most of what you are.

06-16-2017, 02:27 PM
Southern Russian

I get 14% West Med and 12% West Asian which is unusual.
44% Baltic 22% North Atlantic and some noise.


What does Mixed Mode Population Sharing:

I think the West Asian is an ancient marker, which is common for all Europeans. I get about 7%
You now need to click on the Oracle 2 button or Oracle button.

timberwolf
06-18-2017, 01:36 AM
Not sure whether anyone else has seen this?

From the LDNA blog dated 15/6/2017


New Feature: Downloading Your Raw Data
Post date: 15th June 2017
Living DNA are pleased to annouce that we have launched our highly anticipated feature, allowing customers who’ve tested and received their results from Living DNA, to download their raw data, helping to continue research through other platforms.

The purpose of allowing this feature is to open up the industry of DNA and genetic genealogy, giving people the tools to explore their results and analysis through other platforms.

When asked to comment on the progression of the industry during a genealogy community round table, Living DNA company directors David Nicholson & Hannah Modern recently expressed that:

"When it comes to your DNA data we firmly feel this is yours. That’s why we wanted to give customers the option to download it. The process of using the data can sometimes be complex and so Living DNA would love to team up with all the companies in the family history and ancestry markets to form a universal database for DNA matching."

Customers can find the download option through their online portal, on the left-pane. By clicking on this, you’ll be able to start the process of downloading your data (please be aware this is our first stage version of raw results, and will be updated soon).


Before you’re able to access your raw data, you will be presented with some considerations before and after you’ve downloaded your raw data.

Two of the main points are*:

Further analysis of your raw data could tell you that you are, or are not related to another person.

It is your choice to download your genetic data, and what it is used for. As a company, we’re not responsible for how you use your data once you have chosen to download it.

*all points of consideration and terms of consent can be found when downloading your raw data

After all of these considerations have been consented to, you’re given the choice as to which data you would like to download. As we offer 3 tests in 1 package, you can separately download your family ancestry data, mtDNA and/or Y-DNA. Simply click on your chosen data you wish to download, and proceed to download ensuring you have saved the file on a secure system.


Where can you use your raw data?


Your Living DNA raw data can currently be used through GEDMatch Genesis (https://genesis.gedmatch.com/). GEDMatch Genesis is a sneak-preview of what is to come from GEDmatch and provides the ability to accept uploads of testing companies, like ourselves, with SNP set formats and uses a new algorithm that can potentially provide better accuracy, and more flexibility. In the near future, GEDmatch Genesis will merge with GEDmatch standard.

We’re currently working with a number of other platforms where Living DNA data will be accepted for uploads, and we’ll announce these once each one has been finalised and ready to use.

A new data version will be released July 10th 2017.

We have had some great success stories and feedback from the initial download phase. Although the ‘.vcf’ file format is the current format, we will be changing this to a ‘.txt’ format which can be more easily used on platforms. In terms of formatting of the data alleles in the new version will be displayed as A, C, G or T instead of 0 and 1. We will also be including SNPs from the x-chromosome markers which are used in the family ancestors test.

sktibo
06-18-2017, 01:39 AM
Not sure whether anyone else has seen this?

From the LDNA blog dated 15/6/2017


New Feature: Downloading Your Raw Data
Post date: 15th June 2017
Living DNA are pleased to annouce that we have launched our highly anticipated feature, allowing customers who’ve tested and received their results from Living DNA, to download their raw data, helping to continue research through other platforms.

The purpose of allowing this feature is to open up the industry of DNA and genetic genealogy, giving people the tools to explore their results and analysis through other platforms.

When asked to comment on the progression of the industry during a genealogy community round table, Living DNA company directors David Nicholson & Hannah Modern recently expressed that:

"When it comes to your DNA data we firmly feel this is yours. That’s why we wanted to give customers the option to download it. The process of using the data can sometimes be complex and so Living DNA would love to team up with all the companies in the family history and ancestry markets to form a universal database for DNA matching."

Customers can find the download option through their online portal, on the left-pane. By clicking on this, you’ll be able to start the process of downloading your data (please be aware this is our first stage version of raw results, and will be updated soon).


Before you’re able to access your raw data, you will be presented with some considerations before and after you’ve downloaded your raw data.

Two of the main points are*:

Further analysis of your raw data could tell you that you are, or are not related to another person.

It is your choice to download your genetic data, and what it is used for. As a company, we’re not responsible for how you use your data once you have chosen to download it.

*all points of consideration and terms of consent can be found when downloading your raw data

After all of these considerations have been consented to, you’re given the choice as to which data you would like to download. As we offer 3 tests in 1 package, you can separately download your family ancestry data, mtDNA and/or Y-DNA. Simply click on your chosen data you wish to download, and proceed to download ensuring you have saved the file on a secure system.


Where can you use your raw data?


Your Living DNA raw data can currently be used through GEDMatch Genesis (https://genesis.gedmatch.com/). GEDMatch Genesis is a sneak-preview of what is to come from GEDmatch and provides the ability to accept uploads of testing companies, like ourselves, with SNP set formats and uses a new algorithm that can potentially provide better accuracy, and more flexibility. In the near future, GEDmatch Genesis will merge with GEDmatch standard.

We’re currently working with a number of other platforms where Living DNA data will be accepted for uploads, and we’ll announce these once each one has been finalised and ready to use.

A new data version will be released July 10th 2017.

We have had some great success stories and feedback from the initial download phase. Although the ‘.vcf’ file format is the current format, we will be changing this to a ‘.txt’ format which can be more easily used on platforms. In terms of formatting of the data alleles in the new version will be displayed as A, C, G or T instead of 0 and 1. We will also be including SNPs from the x-chromosome markers which are used in the family ancestors test.

Fantastic. Hopefully the new July 10 version will fix the low SNP problem it currently has.

MacEochaidh
06-18-2017, 01:52 AM
Whoops! Wrong thread...

06-21-2017, 11:03 AM
So now we can download our Y data from LivingDNA, can anybody recommend anything useful to do with the Y download file?

genealogybugtoo
06-22-2017, 08:24 PM
I noticed today that the admixture calculators are now available on Genesis.