Format problems in Gramps XML

Help
1

Gramps version 5.1.6 on Debian Linux 6.1.76-1 (2024-02-01).
I have been using Gramps for 12 years now, and have a fairly large tree (about 20.000 persons and 30.000 events). In order to correct some mistakes I made years ago, to simplify things (e.g. merge identical sources) and to eventually split it up in separate trees, I’m writing a script (in perl, don’t know Python, sorry) which manipulates the export XML.

Up to now that went very well, but with my latest development Gramps refuses to import the new XML, complaining about missing relations.

Is there any way I can get more information out of it: where in the XML the problem occurs, what items are not correctly linked etc. ? That would enormously facilitate my debugging (the XML is 20 MB!).

2

You can merge sources with the Isotammi tools, and relations are bidirectional. Can it be that you forgot to change some reference handles?

3

Have you tried running gramps from Linux’s terminal? There might be additional messages that will be routed there.

There are also command line options for additional debugging features.

4

The sources are already merged, including the re-referencing to the new single source. The import went well at that stage. What I’m now doing is creating new “Baptism” events where I added a note to Birth (“baptized on the same day”). The new events get new ID’s (E…) counting up from the last E number found, a new handle and change time, and I insert an eventref to the new handle in the where the eventref to the Birth already exists. But somehow, somewhere that went wrong apparently… How to find out where/what?? The error message is rather terse.

5

You might also explore the import xml module (around where the message occurs) and insert some debugging “print” statements.

(Had to do that for some thumbnailer troubleshooting. Described in another thread)

1 Like
6

@Nick-Hall

A common suggestion has been to tweak the XML (with Perl or Grep).

Do the Import modules track the line number of the data file being imported?

If so, can you recommend a standard Gramps debugging line to insert to print the line number of the XML/gpkg file that caused a parsing error?

(Are there any ‘smart’ debugging options? Where the print line is skipped if Gramps isn’t in a debug mode? Or where only the 1st x number of instances are reported to avoid a 30,000 line bebug report.)

7

The only additional information I received from the command line import is that the problem occurs at 19% of the import file. No idea if that refers to bytes or lines.
I couldn’t get debug to work (what is a LOGGER_NAME ?).
Tweaking the import script is a bit problematic as I don’t speak Python.
I think I’ll write a perl script just to check if every handle has one or more corresponding hlinks, and vice versa. Can’t be that difficult with what I already have.
Thanks for the suggestions so far.

8

Inserting a simple print line in the Python outputs to the Terminal. And, I think, to the default log file.

(Note that the biggest problem I have with Python is how sensitive it is to white space. If white space is SO critical, why isn’t there an easy option to show placeholder characters for spaces & tabs?)

9

You can use Visual Studio Code for Python debugging in Gramps. That’s probably the easiest when you run Gramps from source, from the maintance/gramps51 branch. You will need to be able to read enough Python to figure out where you can set a breakpoint. You can tell Visual Studio Code to start Gramps.py in the gramps directory.

You can download the program here

and when you open a Python file, it will automatically download the Python extensions, so you can be up and running quite fast.

Visual Studio Code can run git too.

1 Like
10

Writing the check script was indeed a peace of cake, and it revealed the errors (all my mistakes, no problem with Gramps!).
Thanks!

2 Likes
11

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.