Updated DNA gramplet ready for review

Comments on a new view of the graph. I changed the maternal/paternal background colors and removed border, added the ID to the tooltip and added a legend with the same name text as the tooltip.

I also removed the M/F flag in the input and added the segment length (altho that is not used for anything yet). One suggestion was to add segment length to the tooltip after the cMs.

3 Likes

Thatā€™s a lot less busy. :tada: But why are Robertā€™s blocks spanning both arms of the chromosome? Iā€™m glad you removed the M/P/U is probably a good idea too. The U (unknown) was crashing the Gramplet. And if parsing fails, it bomb gracelessly too. (I had an extra CSV value per row from my FTdna import.) If the ā€œAssociation Noteā€ parse fails, shouldnā€™t it just ignore the Note?

I was looking at the overall lengths. According to the karyotype idiogram spec, the chromosomes are always charted in deceasing lengthā€¦ except for 21 & 22 and the XY (or XX) at 23. So that means that at least 10 and 19 are rendering incorrectly.

Could you check the program defaults for the chomosome lengths?

I was thinking plotting the Centromere might make a nice overlay that implies chromosomes better than just parallel bars. I finally found docs that list the overall standard length, the centromere loci and the difference between those same values for the XY and XX The graph for male has Y (instead of X) as the 23rd and a short arm.

1 Like

Wikipedia gives a different length for the chromosomes, and that is what is used.

Robert has a step-relationship, not a blood relationship so it is mapped as Unknown and is shown in both the maternal and paternal row. In the PR version, the data used the P flag for each line for Robert. So even tho no match was found, it used the value in the data. Now it only uses the MRCA that it can determine.

If you use an inappropriate chromosome number, it ignores the row. I need to add more bullet-proof.

1 Like

Heyā€¦ that centromere aligned diagram on the WikiPedia page you referenced makes a LOT more sense than table data for the XY.


Curious that the length in mm and Mbp donā€™t directly correspond. Then again, density or the loci probably varies and the length is stretched out rather than the normal spirals.

1 Like

It looks good so far. The only problem I encountered was the speed, but Iā€™m unsure what was causing it.

Yes that would be a good additional information, but it should be optional since some companies do not provide that information.

Centomeres have no function except looking nicer. I wonder if should even add them.

In the chart usually only the autosomal chromosomes (1-22) and the X-Chromosom (=male) two X-chromosomes (=female) are painted. They all are affected from recombination while the Y-chromosom (=males) and mtDNA (=male and females) are not affected and therefore only have mutations (=haplogroups) but no segments to paint.

I think what you describe is currently probably the best way to indicate the behaviour seen in nature, but also more difficult to code than a list of between two matching persons. Another problem is that we donā€™t know from which ancestor a segment originated. Usually you have a pair of common ancestor from which a segment was inherited, but only one of them was carrying it.

Would you consider renaming this Gramplet before going public?

Thankfully, this gramplet isnā€™t actually dealing with raw DNA data, it is charting a very abstract representation of chromosome loci. Sort of like the difference between a histogram of an image and the actual image.

It abstractly charts Chromosomal Segment Matches. How about adapting the name in that direction?

Also, the Help button on the undocked gramplet currently points at the built-in Gramplets page. You might have to rename/move the current documentation. But could you add a gpy.py link to the documentation? I added the following 2 lines below the Gramps version line:

 gramps_target_version='5.1',
 help_url="Addon:DNAgramplet",
 navtypes=["Person"],
 include_in_listing = True,

In my tests, that Navtypes insertion isnā€™t restricting the Gramples to the People View modes. Not sure whatā€™s wrong there.

Although, come to think of it, this gramplet suits the Relationship Views too. I hope that eventually someday Charts will have an option to graph the direct pedigree (+ spouse) of 2 cousins or create (with unknowns as blank boxes) worksheet charts for something like a ā€œ3 cousin 2x removed (up)ā€. This DNA gramplet would be a great adjunct to such a Chart mode.

A new version of the DNA gramplet PR is ready for review

This incorporates the user feedback to: remove M/F/U flag and add SNP (optional) in input file, add background color to graph, add legend, add SNP and ID to tooltip, and add more data checks.

Documentation has been updated Addon:DNAgramplet

NOTE: the input data format has changed from the last version.

New graph (with tooltip) for example dataset looks like:

Please review and provide suggestions/comments.

3 Likes

It is REALLY sensitive to input format problems now.

If 1 Personā€™s DNA Association fails the Gramplet fails to map ANY Associations for that person.

Chokes on an extra linefeed after the last row.
I was hoping to use the item you mentioned before about it ignoring lines that didnā€™t have 1-22 or X as the first term to add a comment header line or source ID comment

Once I had 4 Notes with Match Sequence data it was too hard to keep track of which Note to freshen from the Source. It required opening the Association Note & looking at the References tab. (I have a CSV from FamilyTreeDNA with 60 people on that worksheet.) That imported data is set to a minimum of 20 centimorgan. (The default is 5.)

I was hoping to include the 1st two raw rows (header & the FTDNA rows start with the 2 names being compared.)

I think the chromosome bars are a bit too light. Itā€™s hard to see them.
The rest is working well.

Any chance you can hotlink the ID in the Color Key to open the Person Editor? It would make switching Active Person focus between those people MUCH quicker.

Yes, but that is where triangulation helps. I share a segment with my first-cousin, so I know that we inherited from our shared grandfather or grandmother, but I donā€™t which. But if we also have a second-cousin who shares that segment (or subset of it), now we know which grandparent we got it from, because we know which great-grandparent couple we have in common with that second-cousin.

Anyway, I am just saying that these association notes might be useful for other Gramps enhancements beyond this particular gramplet. Eventually maybe the Gramps data model will evolve to include specific places to store DNA information, but I think itā€™s good to start out this way (using Notes) and try creating a variety of gramplets, reports, etc. before designing specific data model changes.

Yes triangulation helps, but requires having the connections found which is hard for distant relatives. Right now for the majority of my matches I just know the side, but not the common ancestors.

That could make many things easier, but it was rejected for now in the previous discussion, since this data wouldnā€™t be able to export e.g. to GEDCOM, in contrast to using notes and associations.

As mentioned in the wiki:

ā€œIf there are overlapping segments within a maternal/paternal view of a chromosome, only the front (last drawn) will be pickable. If it is completely obscured, you will not be able to see the details of the hidden segment.ā€

It seems that they are drawn in the order that the associations appear under the person, is that correct? If so, users could be advised that they can easily change this order (using the up/down arrows in the Associations tab of the person editor) in order to see the segments differently ā€“ specifically, to see all of the segments for a particular match, just move that association to the bottom of the list.

More generally, users might want to order the DNA associations of a given person so that the closest relatives are at the top and the most distant relatives are at the bottom, in order to minimize the amount of hidden segments. For example, when a distant relative shares a subset of the same segment as a closer relative, the distant relativeā€™s portion will be hidden if the closer relativeā€™s association is further down in the list. By moving the distant relative further down, their subset will be drawn on top.

1 Like

Is this better? I worry about cases where there are 20+ Associations and it may be difficult to separate.

Updated documentation to recommend closest to furthest order for Associations.

I added more error checking. I used the following as a sample and every line is ignored. Plz provide more samples if you want me to test. It ignores lines that have errors. Checks are: no non-numerics, must have at least 4 fields.

Test of Header
12,1000,100000,M,23.2

a,b,c,d,e
X



asd

If you put 60 Associations, the Legend will crop. I think about 30 would fit as it is formatted currently.

Interesting idea. Let me ponder viability.

Changed the name to DNA Segment Map.

Any more thoughts?

It is an ugly approach butā€¦ is there an unacceptable downside to the following process for refining the selection of mapped segments?:

  • run Tools => Family Tree Processing => Sync Associationsā€¦
    (Note the ellipsis is inappropriate for this command. There is no dialog.)
  • Select the reference person
  • navigate to Relationship view
  • Add the DNA gramplet to the Relationship view Bottom bar
  • Open the Associations tab for the reference Person Editor
  • delete any excess Associations

When complete,
run Tools => Family Tree Processing => Sync Associationsā€¦

The associations will be restored

Just a thought ā€¦ it would be nice if the tool DID have a dialog, in which the user could select (or enter) the associations to be processed and reciprocal associations to be created. Being able to enter them would mean not having to customize the hard-coded list, and would also let them use their own language.

Another thought ā€“ would it be good if a Tool could have its own tab in Preferences as a place to store such things? Or maybe there could be a separate place for Tool Preferences?

Trying to understand this request. If there was a hotlink from the Color Key (or the segment itself, which probably makes more sense), and it brings up the Associated person in a Person Editor, how does that help? The Association is in the active person, not the associated person. I dont think this action should change the active person.

That is, if person A is active and has an Association with person B and person C, if you click on the segment map of person A in the rectangle of person B, it would bring up person B editor (but A would still be the active person). What would you do in that editor?

This is easy to implement, but I am trying to understand the value of it. You cannot change anything about the Association as that is within person A, not person B. Are you just looking for another navigation tool based on the segment map that is similar to the Combined Relationship navigation (where you can edit parents or children but keep the active person)?

And what action would you want to trigger this navigation? double left click? single left click, ā€¦ This becomes a usability and consistency question that could be quite contentious. Is there any other similar navigation in the system today that we can replicate so as not to invent a new use model?

If the hotlinks worked the same as in the Pedigree gramplet, then single-left-clicking would shift the Active Person focus on a click. While a single-right-click would open the Person Editor.

My 1st assumption is that the Sync Associations has been run. So all the Reciprocal Associations will be in place.

The benefit of having Legend/ColorKey IDs hotlinked like this would be: clicking a hotlink moves the View to the other person, which refreshes the DNA. Since most of the individuals will have only the 1 person Reciprocal Association DNA Match, the multiple person chart interpretation isolates to JUST those 2 people. The new legend will only have 1 person. So clicking that returns you back to the more heavily populated chart.

also, you could have the Pedigree or Fan Chart in the Side bar. So quickly swapping between two people that you KNOW are related can let you spot the common ancestor. (Very useful if your newly discovered cousin sent you a Pedigree GEDCOM to import and they havenā€™t been merged yet.)

Hello, I am trying to make this grmaplet work, but I canā€™t find what I am doing wrong
When I created the association and note, nothing was shown in the DNA tab.
After a few restarts of Gramps, the DNA graph appeared, but when hovering over it, it doesnā€™t show the name of the person who I share DNA with, the amount of cMs shown is the SNP number, and the same graph is shown for every person

Gary has been revising this Gramplet heavily for the last couple days and is about ready for release.

Heā€™s bringing the new wiki page into sync with the code now. The Gramplet has been renamed since no DNA data is actually being storedā€¦ rather it maps abstractions of the comparison between 2 samples. We may need a DNA gramplet name for when Gramps has something that compares the actual Deoxyribonucleic Acids.

The ā€˜sameā€™ graph youā€™re seeing is the backdrop and there probably werenā€™t any valid ā€˜segment matchā€™ lines to be plotted.