Gramps 6.0 and gedcom 7

Development Beta Testing
,
1

I know @Nick-Hall said gedcom 7 won’t be fully supported right away. But I want to post what I notice.

I read in a gedcom7 file and noticed that the character set encoding of the gedcom file is not recognized.
In gedcom 5.5.1 it is displayed like this:

1 CHAR UTF-8

As far as I know, UTF-8 is always required in gedcom7. So it should automatically recognize the character set as UTF-8.

2

From the Gedcom 7.0 specification:

“Previous versions allowed multiple character encodings, defaulting to ANSEL. 7.0 only uses the UTF-8 character encoding.”

1 Like
3

Yes, OK. So Gramps 6.0 isn’t reading correctly yet?

4

Gramps will read UTF-8 encoded files.

5

You’re right.
I’ve now imported gedcom7 files from three different programs into Gramps.
The character set in the file from the first program wasn’t converted correctly.
The other two worked.
Is the import of the ASSO tag already working?

6

Haver you looked at the character encoding for the files not working in e.g. Notepad++ or similar tools that shows the metadata of the file?

It can be that it is encoded as something else than UTF-8 when saved…

I know Legacy had big problems with this when it come to the UTF-8 encoded 5.5.1, it was not encoded correctly even though it had the UTF-8 in the header.

I had to open these files in Notepad++ and set the correct character encoding and then save the file…

All characters was displayed corrctly in NP++ before converting it, but not in any other genealogy software, including Gramps when importing it.
After the change, the files got imported correctly.

This can also be a problem when working with CSV and similar files…

1 Like
7

Only for people, not for families or events.

8

But support for families or events will come later?

9

I would welcome someone implementing support for the ASSO tag in families and events for the Gedcom import/export.

2 Likes
10

It would be nice if this was done VERY publicly.

So people could understand the process of proposing and developing new 7.0 Tag support in the Gramps data model and GEDCOM import/export plugin support.

References:

2 Likes
11

It is a gedcom7 file that I made from a program that promised a lot.
I then made a gedcom 5.5.1 export of my 75,000 people (took over 4 hours) and then fled to Gramps.
I have had my data in Gramps for over a year now and still need to do some corrections.
I thought that I could use this gedcom7 file for testing. But this file was just as rubbish as the whole program…

1 Like
12

Can this be done as an Import Addon? As an experiment in superseding the builtin 5.5.1 GEDCOM modules when detecting a 7.0 file for import?

The adding of an export GEDCOM7 (tree and/or media zip) as an addon seems well defined.

13

The ASSO tag in events could be implemented by just updating our existing Gedcom import/export. It is the way that Gedcom handles shared events.

The ASSO tag in families would need a schema change.

14

This is a hack but… could a transitional data model representation be done with a Family object attribute?

Attribute GEDCOM7.ASSOCIATION_STRUCTURE storing the full structure of the not-yet-supported data

So when converting schemas (or with a tool), the attribute structure is parsed and folded into the data model properly?

(We have a need for such a tool already. As an example, grandparent was added to 5.2 as a standard Association. But 5.1 and earlier users have Grandparent stored in a variety of workarounds… all with Grandfather/Grandmother variants. So a flexible migration/harmonization tool is helpful for the users who cannot SuperTool script.)

15

Yeh, but “gedcom 7” doesn’t mean a lot if the file itself is character encoded as ANSI or ANSEL or if the file is encoded correctly with or without the BOM marker or if the BOM marker is at a wrong place or missing…

I have experienced this with Legacy multiple times, even when trying to export to gedcom 5.5. that do not support UTF-8.

that was why I asked if you have tried to open the gedcom file as a text file in e.g. Notepad++…
Because Gramps can’t read/understand a file that is encoded wrong…

Especially because it worked for the files from two other programs, it can be as simple that the exported file is not encoded correctly, and that can easily be fixed in NP++ or similar software.

But it can also be that the software you used doesn’t write the content of the file correct… that will be a bigger problem…

I mentioned it just to give you an option that is easy to do and that might be an easy fix for atleast one of your problems transferring data…

16

It looks good in Notepad++.
But i can’t test anything from my previous program anymore because it keeps crashing when exporting gedcom since the last update.
The programmer isn’t particularly interested in working on the export module either. He said “Which programmer would be interested in exporting the data from their program again in order to switch to another program?”
That says it all…
I’ve already entered so much new data in Gramps that I don’t want to transfer the data from the old program to Gramps anymore. I only use the program to check the data in Gramps.

17

The GEDCOM7 decision to drop the character encoding and assume UTF-8 seems a bit myopic. Particularly when the existence of UTF-16 and UTF-32 imply that there is current insufficiency. And so UTF-8 is certainly not future‐proof.

The GEDCOM7 .ged reference file that everyone is supposed to test against:
https://gedcom.io/testfiles/gedcom70/maximal70.ged

1 Like
18

yeh, that’s why the actual character encoding is so important, and that the BOM marker is set correctly or maybe totally ignored (it might work if it is in some software)

1 Like
19

thats what I figured out when I exported from Legacy too… but even though I didn’t have anywhere near your amount of people, it has been a huge job… and I ended up register everything manually… I had more than 10000 notes with 4 to 10 characters in Gramps for approx. 5000 entities.
And nearly every Scandinavian name with the ÆØÅ characters was wrong because of the char. encoding.

But, I will stop bloating this post now… good luck getting it right…

1 Like
20

A Person object has a person_ref_list that is used to store the associations. A Family object doesn’t have such a list, so one would have to be added.

Event associations are just links from an event back to a person, that include a role. In Gramps we use event reference objects for this purpose, so no schema changes would be required.

1 Like