'mergeit' problem

[Telfermagne]

Reckoned to try my hand at qpAdm n junk. I was able to use qpAdm with no issues when only Reich Lab datasets were used. I have stalled when trying to use 'mergeit' to merge my raw data with a Reich Lab dataset (Patterson 2021) so that I can include myself in qpAdm runs. I get the following error:

fatalx:
snps out of order and packed format. Run convertf with pordercheck: NO

Process so far:

 

plink --23file A01.txt --make-bed --out me
plink --file ped_Me --maf 0.05 --make-bed --out binary_fileset_Me
plink --bfile binary_fileset_Me --recode --out ped_Me_again
convertf -p par.PED.EIGENSTRAT

Now my genomic data should be formatted to match the of the downloads at Reich Lab.

I want to merge my genomic data with a dataset downloaded from Reich Lab.

So, 'mergeit'.

Make a par file:

geno1: brit.geno
snp1: brit.snp
ind1: brit.ind
geno2: eigen_Me.geno
snp2: eigen_Me.snp
ind2: eigen_Me.ind
indoutfilename: MeBritMerge.ind
snpoutfilename: MeBritMerge.snp
genooutfilename: MeBritMerge.geno
outputformat: EIGENSTRAT

Then, 'mergeit' with command:
mergeit -p mergeBrit.EIGENSTRAT

EDIT:

Also tried to convert the v54 data and got the following:

fatalx:
Data sets with more than 8 billion genotypes are not permitted


EDIT:

Tried the Classical Greek Army dataset and managed to get a bit farther but got stuck again. When converting the merged dataset back into EIGENSTRAT format for some reason my sample gets 'ignored', e.g.) ME_ID001 ignored.

Current scratch sheet (utilizing information from posts by anglesqueville, Nganasakhan, PLogan, etc.):

 

My 23andMe Raw Data:
plink --23file A01.txt --make-bed --out me
plink --file ped_Me --maf 0.05 --make-bed --out binary_fileset_Me
plink --bfile binary_fileset_Me --recode --out ped_Me_again
convertf -p par.PED.EIGENSTRAT

Ancient DNA Dataset:
1) Convert the 2 files into bed/bim/fam with eigentools/convertf
convertf -p Greek_Army2.PED

2)Exclude unique SNPs
plink --bfile Reitsema2022PNAS_Ancient_1240K --write-snplist
plink --file Reitsema2022PNAS_Ancient_1240K --extract plink.snplist --make-bed --out B1

3) Merge
plink --bfile binary_fileset_Me --bmerge B1.bed B1.bim B1.fam --indiv-sort 0 --make-bed --out GRK1
If multiallelic or multichromosomic snps, then:

4) List 'badsnps' as a file, .missnp format. Run: plink --bfile B1 --exclude badsnps.MISSNP --make-bed --out B2

5) Merge B2 and your data. Run: plink --bfile binary_fileset_Me --bmerge B2.bed B2.bim B2.fam --indiv-sort 0 --make-bed --out GRK2

6) Back to EIGENSTRAT
plink --bfile essai2 --recode --out ped_essai2
then,
convertf -p par.PED.EIGENSTRAT

[Telfermagne]

I think I got it working, sort of. This page reminded me of something I read elsewhere but forgot to do: https://www.biostars.org/p/165210/

Question: convertf decides to “ignore” all my samples. Why?
Answer: A likely reason is that you are using a “fam” or “ped” file with a funny value (0, 9 or -9) in column 6. Try setting column 6 to 1.

Got my data merged with the Greek Army data, haven't tried a larger dataset yet. That is I/P.

EDIT:

Shot down out of the sky. Downloaded the v54 dataset and tried to 'convertf'. Got the following error:

fatalx:
Data sets with more than 8 billion genotypes are not permitted

Didn't have this problem with the Greek Army one.

[Telfermagne]

Finally! Got the datasets to merge:

command:
convertf -p par.convertf.EIGENSTRAT

par file:
genotypename: B1_binary_fileset_Me.bed
snpname: B1_binary_fileset_Me.bim
indivname: B1_binary_fileset_Me.fam
pordercheck: NO
outputformat: EIGENSTRAT
genotypeoutname: B1_Me_EIGEN.geno
snpoutname: B1_Me_EIGEN.snp
indivoutname: B1_Me_EIGEN.ind

command:
mergeit -p par.mergeit.EIGENSTRAT

par file:
geno1: brit.geno
snp1: brit.snp
ind1: brit.ind
snplist: badsnps.txt
docheck: YES
strandcheck: YES
allowdups: NO
geno2: B1_Me_EIGEN.geno
snp2: B1_Me_EIGEN.snp
ind2: B1_Me_EIGEN.ind
genooutfilename: Mebritmerged.geno
snpoutfilename: Mebritmerged.snp
indoutfilename: Mebritmerged.ind

Seems that I can't merge large datasets. Oh well, I'll play with the Patterson data for now I guess.