GEDCOM related reports on Mantis

[ennoborg]

Hello Devs,

Last week, I found two complaints about GEDCOM imports on Mantis. I reacted to both, using my own experince with the exporting programs, PAF and RootsMagic. And based on what I know, I see no reason for a follow-up there, but I want to discuss a few things here.

The first issue is, that the RootsMagic user complained about ‘garbage’ caused by Gramps, and did not realize the obvious, which is that RootsMagic has several options which can be used to suppress the export of things like tasks, note formatting, and source templates. And I know that when you suppress those, the GEDCOM export is quite clean, except when you’ve downloaded trees from Ancestry, in which case custom Ancestry tags will show up in the GEDCOM. And you can’t suppress those.

In the other ‘complaint’, about import from PAF, the user does not seem able to make himself clear, and it looks like he’s partly confused by our file selector. I can understand that, but I prefer not to engage in that, because I don’t see much positive effort on his side. He claims that he’s not a beginner, but that doesn’t really show. He also complains about a few custom tags exported by PAF, which I don’t see as very important. They’re all caught in notes, which can be easily removed after import.

I’m writing about this, because I feel that there’s a need to educate some users a bit, about choosing the right options for export, for which I gave an example on Mantis. I’m also writing, because I think that we CAN do a few things for some custom tags.

And I write that, because I have noticed that some of these are imported as events, and I find that quite annoying, especially for tags that have a single value, which can be easily imported as attributes, just like we already do for standard tags like _UID and _FSFTID. They’re both GEDCOM extensions, officially speaking, but in reality they’re standard, in the sense that many competing programs use them, including PAF and RootsMagic.

What do you think about this? Can we make an inventory of the custom tags that can be stored in attributes? I can generate some examples with Ancestry, PAF, and RootsMagic.

Regards,

Enno

[anon9092048]

Also maybe entries with advice should be added to the wiki for those programs

• Import from another genealogy program > Import instructions for different programs

[Davesellers]

I agree that the GEDCOM import should/could include some customization for popular imports.
Like the tags from Ancestry being recognized in GRAMPS. E.G. “Competed” being created as “Completed”. Sorry, don’t remember what the tag ID# is.

[ennoborg]

OK, great, thank you both.

Can we discuss a strategy for this, especially referring to these bug reports?

https://gramps-project.org/bugs/view.php?id=12858

https://gramps-project.org/bugs/view.php?id=12611

My personal position is, that both are not really a Gramps problem, and that is because for the first, most of the so called garbage disappears when the users switches off the options to export note formatting and RootsMagic extension. The first option speaks for itself, and the second is mostly about the use of source templates inspired by Evidence Explained. We don’t support those, and when suppressed the GEDCOM still has the usual author, title, volume/page, and date fields, which are exported as formatted by the templates, meaning that the citations are not really lost. We simply don’t import the elements, so that the user can’t change styles, except by reformatting the titles manually.
RootsMagic can also export tasks, but I haven’t tested those.

When you use the settings that I recommended, most of the ‘garbage’ disappears, and the only import error that I see comes from an illegal WWW tag, for the manufacturer, I think, so that’s not a big loss either. Also, when a user imported from Ancestry to RootsMagic, some custom tags from Ancestry are passed on to the exported GEDCOM. Most of these are flags and ID’s used by Ancestry, which don’t contain any real data on persons or events.

For PAF, it’s pretty much the same. There are some custom tags attached to media, and most of those are flags. PAF can also export phrases to be used for user defined events, which may be more important for that user.

Today I found some annoying things in a GEDCOM produced by the Family Tree Builder program that you can download from My Heritage. My Heritage is quite popular here, and the GEDCOM has _UPD tags that contain a time stamp string, and RIN tags, which are internal reference numbers that can be attached to persons ane events. And to my surprise, the _UPD lines are imported as events, and I really see no point in that. They could be easily imported as attributes, or ignored. The RIN tags are reported as errors, which is right by itself, because they’re not in the standard, but it might be easier to treat those the same as the REFN tag, which is legal, and stored in attributes, just like the _UID and _FSFTID.

I found @anon9092048 in the notes, and also a person known as CallMeDave. Is that you @Davesellers ?

Do you have suggestions on how to move on from here? Should we create a new issue for those custom tags? And how can we follow up on tasks exported by RootsMagic (not tested by me) and the event definitions exported by PAF?

[S.Mackay]

You didn’t seem to realize that I was referring to Rootsmagic 7, which did not suppress the export of Rootsmagic specific attributes as well as Rootsmagic 8, which you were using. HTML tagging and the FamilySearch attribute tag problems in Rootsmagic 7 (but not 8 as you mentioned) aside, both Rootsmagic 7 and 8 gedcoms when imported into Gramps (if you had looked at my examples) both lump all author and publisher information into the title. Also citation dates get stripped from the title and put into the citation as they should, but there are still vestiges of the citation left in the source title. So the Gramps import of gedcoms from Rootsmagic 7 and 8 are both inadequate for many potential users that I know of personally. You are actually the first I have seen who said it is ok, so maybe there are others too who are ok with it.

I did not demand the problem to be fixed as some seemed to think in the mantis report. I merely mentioned that it exists. It is one thing for people to recognize faults in a work, and I did mention in the mantis report that I recognize we are all volunteers. However, misrepresenting the problem like this and bringing up an issue that was already dismissed, is disappointing.

[ennoborg]

I’ve been a RootsMagic user since version 7, or earlier, and I am sure that the options to suppress non standard GEDCOM tags was already in that version. And I am, because I remember that, and also, because I know that the change of version 7 to 8 was primarily a migration to a new user interface toolkit, without any changes underneath. I took part in the public beta, so I know what I did. Most functional changes were moved to version 9, which works perfectly well under Wine, meaning that you can migrate to that version too. The new UI takes some getting used to, but that’s how the world works.

And if you don’t believe me, just take a look at this screenshot of RootsMagic 7, taken in Wine.

When you look at the options in the bottom left, it’s clear that the options to suppress the so called garbage are there, and the only issue that I see is that you didn’t use them, maybe because you were not aware of what they do.

This screenshot was taken with a RootsMagic 7 version that I downloaded just a few minutes ago, so you still have the chance to download that, and do a new export.

I have no idea why you accuse me of not looking at your examples, but I can tell you that you are wrong. I have a good eye for detail, and I rarely miss a thing.

And all in all, I honestly think that you’re blaming the wrong people, because all the information that Gramps reads from author, title, publishing information, etc., is read exactly as it is exported by RootsMagic, 7, 8, 9, or whatever. And that means, that if you don’t like it, you better talk to them, and not blame us for importing ‘garbage’. You are the one that chose to use a program that creates it, and expect wonders, not me.

I like RootsMagic a lot. I think it’s one of the best programs around if you want support for Evidence Explained, but my primary reason for using it is because it can sync data with Ancestry and FamilySearch.

The truth is, that there is no real problem in Gramps. The program imports what’s sent to it, so if you think about garbage, remind yourself that the main principle is garbare in, garbage out.

Moreover, if you look at how FamilySearch presents its citations on-line, there is another factor here, and that’s what they send to RootsMagic. And in their communications with RootsMagic, or Legacy, or Ancestral Quest for that matter, there are no citation elements a la Mills, and not even the fields that we know from GEDCOM 5.5.1, which are all supported by Gramps. It’s a format that I don’t like at all, but I have no other choice than to live with it.

[S.Mackay]

https://gramps-project.org/bugs/file_download.php?file_id=11038&type=bug
Many people would not be happy with this example, that I posted in the mantis. This has nothing to do with Rootsmagic specific settings. I have repeated the problem repeatedly, but continue to be misrepresented and I even had my response flagged as inappropriate. I will leave you alone to slander me all you want, you can flag this too if you want. I did my part by demonstrating there is a problem, you can shoot me as an unwanted messenger all you want. I am out.

[ennoborg]

Well, I just ran another test with RM 7 and the settings that I mentioned earlier, and when I do that, I see only 3 errors, all related to the address of a few repositories. There is no template stuff then, nor formatted notes, and yes, the _UID and _FSFTID tags are imported, largely because I wanted them to be imported, and fellow developers approved that change. And this means that it has to do with RootsMagic settings, which is something that I’ve told you from the very beginning.

This address thing may be something that we can repair, but I don’t think that we will produce better support for templated citations anytime soon, partly because there is no real standard for that.

I also found a lot more garbage in RM 9 GEDCOM’s, even with the recommended options, and those are clear violations of the GEDCOM 5.5.1 standard.

[ennoborg]

Update: I added instructions for RootsMagic, because they are the easiest. In that, you can suppress all custom tags with two or three clicks. I think that I also added a comment about RootsMagic 9, which exports a standard tag (NAME) in the wrong place.

I also closed both reports, because there is no error in Gramps in the sense of GEDCOM support, for version 5.5.1. One report also had a complaint about invisible drives in the file selector, on Windows 7, but its author (a fellow Dutchman) didn’t respond to my questions, and we have other reports about the same issue from other users, who are more responsive.

This leaves some issues for tags that look so useless, at least to me, that it would be nicer to ignore them altogether, and not put them in notes. And that’s a bit difficult, because in that case, useless is an opinion. There are also some issues for tags like UID, exported by Ancestry, which have the same contents as the de facto standard _UID, and are not the same as the UID as it is defined in GEDCOM 7.