Error when importing a Gramps XML file

Help
21

Ok, I guess I understand what you’re trying to say. I agree that my problem is (most probably) not a formal XML syntax error that could be identified by feeding the file to a syntax checker or schema validator, but nonsensical content of a XML element. Or to use a picture: all my boxes are ok, none is missing and each box is exactly at the place it is supposed to be, but at least one box contains something that doesn’t make sense for Gramps. Right?

I have been trying to implement a new functionality for the XML file in a process that takes an existing one (produced by Gramps) and produces a new one with appropriate changes. So it’s not a problem of going back to an older version of the XML file, but if I’m stuck in the dark what the problem is, I simply do not know how to adapt my code to implement the new functionality.

Since I’m no Python guy, I don’t have any clue about debugging Python code. Would it be possible that you try to debug it?

22

Right. There is something missing, or pointing to something that’s missing, and without the XML, it’s hard to figure out what that is, so I’m interested in seeing that XML in my mail.

Also, if you modified a Gramps XML, you must have an original too, right? If so, you should be able to use a program like WinMerge to reveal what you did, right? Does that help, or does it simply look OK to you?

I like to have that original too, so that I can see the exact changes that you made.

I have no debugging environment ready, because I just set up a new Linux environment, so it will take some time.

Another thing is, that because the error gives no details about the offending data, it may be worthwhile to file an enhancement proposal for the importer itself.

23

I’m not sure if that will help since I made some 500 changes but I’ll give it a try. At least 500 changed lines are a much smaller haystack to look for a needle compared with 3 M lines :wink:

I’ll keep you updated. Thanks for your help!

24

With such a number, I am assuming that you generate these with a script. Can you change that in such a manner that it stops halfway?

I’m saying this, because when you’re in the dark, and you can control what you create, you can use a bisection method to find which of the 500 changes is the bad one. I had to do that once, when I created a C program that crashed the compiler in such a way that it just crashed, so I had no idea which line caused the problem.

With 500 lines, it means that you can find the one that’s wrong in 9 steps.

25

Gotcha! Ok, kind of :thinking:

image
image776×193 15.9 KB

Notepad++ has a plugin to compare text files which appears to be much faster than WinMerge. I found three of the fire breathing dragons shown above in the XML file. Those three segments are clearly not compatible with any XML syntax known to mankind so I wonder how those lines were able to avoid detection by the syntax checker and the schema validator. I have to check now how my code was able to produce those monsters.

The plugin didn’t identify anything else that looked suspicious. The next step was obvious: correct the lines manually and try to import into Gramps. The result:

Error Details: 
===================

96197: ERROR: dbloader.py: line 569: Failed to import database.
Traceback (most recent call last):
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\gui\dbloader.py", line 558, in do_import
    dbstate=self.dbstate))
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 149, in importData
    info = parser.parse(xml_file, line_cnt, person_cnt)
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 936, in parse
    self.p.ParseFile(ifile)
  File "C:/repo/mingw-w64-python3/src/Python-3.6.4/Modules/pyexpat.c", line 468, in EndElement
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 3146, in endElement
    self.func(''.join(self.tlist))
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 2678, in stop_event
    if self.event.get_description() == "" and \
AttributeError: 'NoneType' object has no attribute 'get_description'
17778277: WARNING: uimanager.py: line 386: **Duplicate Accelerator <PRIMARY>i
17778278: WARNING: uimanager.py: line 386: **Duplicate Accelerator <PRIMARY>b
17778278: WARNING: uimanager.py: line 386: **Duplicate Accelerator <PRIMARY>u
17778278: WARNING: uimanager.py: line 386: **Duplicate Accelerator <primary>z
17778278: WARNING: uimanager.py: line 386: **Duplicate Accelerator <primary><shift>z
24890668: ERROR: dbloader.py: line 569: Failed to import database.
Traceback (most recent call last):
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\gui\dbloader.py", line 558, in do_import
    dbstate=self.dbstate))
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 149, in importData
    info = parser.parse(xml_file, line_cnt, person_cnt)
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 936, in parse
    self.p.ParseFile(ifile)
  File "C:/repo/mingw-w64-python3/src/Python-3.6.4/Modules/pyexpat.c", line 468, in EndElement
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 3146, in endElement
    self.func(''.join(self.tlist))
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 2678, in stop_event
    if self.event.get_description() == "" and \
AttributeError: 'NoneType' object has no attribute 'get_description'
32942279: ERROR: dbloader.py: line 569: Failed to import database.
Traceback (most recent call last):
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\gui\dbloader.py", line 558, in do_import
    dbstate=self.dbstate))
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 149, in importData
    info = parser.parse(xml_file, line_cnt, person_cnt)
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 936, in parse
    self.p.ParseFile(ifile)
  File "C:/repo/mingw-w64-python3/src/Python-3.6.4/Modules/pyexpat.c", line 468, in EndElement
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 3146, in endElement
    self.func(''.join(self.tlist))
  File "C:\Program Files\GrampsAIO64-5.1.6\gramps\plugins\importer\importxml.py", line 2678, in stop_event
    if self.event.get_description() == "" and \
AttributeError: 'NoneType' object has no attribute 'get_description'

System Information: 
===================

Gramps version: AIO64-5.1.6-1 
Python version: 3.6.4 (default, Jan 23 2018, 13:17:37)  [GCC 7.2.0 64 bit (AMD64)] 
BSDDB version: 6.1.0 (6, 0, 30) 
sqlite version: 3.21.0 (2.6.0) 
LANG: de_DE.UTF-8
OS: Windows

GTK version    : 3.18.9
gobject version: 3.26.1
cairo version  : (1, 16, 1)

Still the same type of error AttributeError: 'NoneType' object has no attribute 'get_description' with a few new warnings:

17778277: WARNING: uimanager.py: line 386: **Duplicate Accelerator <PRIMARY>i
17778278: WARNING: uimanager.py: line 386: **Duplicate Accelerator <PRIMARY>b
17778278: WARNING: uimanager.py: line 386: **Duplicate Accelerator <PRIMARY>u
17778278: WARNING: uimanager.py: line 386: **Duplicate Accelerator <primary>z
17778278: WARNING: uimanager.py: line 386: **Duplicate Accelerator <primary><shift>z

So my progress appears to be quite limited …

26

I just looked at the code, and when I read that, it looks like you may have an event that’s not an event, and that’s quite weird, so I suggest that you send the file, so that I can add some code to see what’s happening.

1 Like
27

I compared your files with diff -w, and that showed the exact same event as you showed here, and when I correct that, the import works.

Correcting means that I removed all duplicate lines from this event, so that it’s not an event in an event anymore, and there is no duplicate handle and id either.

1 Like
28

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.