Interest in enhancing verify.py

Development
41

Thanks, that’s it. They were all role problems indeed.

Another one is, that for a gay couple, I get two warnings. One says, that it’s a gay couple, and the other says that the wife is a man. Would the latter go away when I change his role from Primary to Groom?

And a wish: When I download persons from FamilySearch, some persons have birth dates with a range with from and to, instead of between … and …, and in most cases the latter is what people meant, that they are not sure, not that the birth took more than a day, so I’d love to see a warning for this too, in case that I forgot to intercept it beforehand.

42

OK, I generated a tree with 262,143 people, 131,071 families and 655,357 events (2 per person, 1 per family). The tests I ran where really educational :smile:

  • I am back with LRU caching, limited to 20k people and 10k families. Everything else is suicide with very big trees
  • whenever I process a person, i process its families first (if it wasn’t processed already)
  • i learned that looking up a value in a set is much more faster than in a list when the amount of values is huge :smile:
  • the left over unprocessed families (usually none are, but there might be some families without any spouses) are processed afterwards
  • i preload caches only if the whole tree fits into the limited caches (otherwise there is no preloading done)

Running the Verification Tool (CLI mode) on that mentioned generated tree:

Stock Gramps 5.1.5:

  • runtime: 2:41 minutes
  • peak memory: 180 MiB

My performance enhanced code with no additional rules:

  • runtime: 1:20 minutes
  • peak memory: 217 MiB

My performance enhanced code with my additional 8 new rules:

  • runtime: 1:41 minutes
  • peak memory: 217 MiB

I also successfuly tested my reallife tree with ~15k people and Royal92.

I’d say the code is ready now.

43

Before pushing the commit, please add a line 8… include yourself in the list of contributor dated copyrights.

Another future feature to manage your workflow would be a ‘distance’ sorting option.
Calculate the degree of distance from the Home Person and sort the closest Alerts to the top.

44

Done :smile:
Not sure what to do with my pending Pull request (which only covers the Tree View) and all the additional changes I made in the meantime though

1 Like
45

@Nick-Hall ? What is the next step if this is to roll into 5.2 core?

46

It looks like you have 3 branches in your cloned Gramps repository:

  • a master (currently synced with the main Gramps)
  • a “verify_tool_treeview” with 2 commits where you already have a PR#1550
  • a “verify_tool_more_rules” with 17 commits (that include the 2 commits in PR#1550) but has not had a Pull Request submitted

I looks like you just need to make a 5.2 Pull Request for “Verify tool more rules” and GitHub will deal with the overlap.

@Nick-Hall - does Oliver’s PR need to be against the Gramps-Project/gramps Master or maintenance/Gramps52 ? Should he cancel PR1550 first? (1550 has lots of good commenting)

1 Like
47

I have rebased PR #1150 onto the gramps52 branch and merged it.

Another pull request against the gramps52 branch can be created for the remaining changes. I suggest that we squash the commits down to just two: new rules and performance enhancements.

1 Like
48

Ok, I created Verification Tool - 8 more rules + performance enhancements by OlliL · Pull Request #1574 · gramps-project/gramps · GitHub @Nick-Hall - I hope thats sufficient and reflects what you had in mind :slight_smile:

2 Likes
49

@OlliL Yes, that looks good. I’ll review it tomorrow.

2 Likes
50

I did a quick review of the newest message strings.

Could you do some minor re-phrasing?:

return _(“Birth date equals death date”)

return _(“Death date equals marriage date”)

return _(“Children are not in chronological order”)

return _(“Families are not in chronological order”)

return _(“Family events are not in chronological order”)

return _(“Person events are not in chronological order”)

1 Like
51

The first two are existing messages from old rules, so no idea what might be triggered when I change them (all existing translations for those two must be adjusted? No idea)
But yes - I can fix the phrasing later this day.

2 Likes
52

OK, I rephrased those messages.

2 Likes
53

I noticed that after closing that verification tool quite some data remains referenced and can’t be garbage collected. I’m now adding those to utilize the given feature for explicitly destroying the objects on close.

grafik

I wonder if there isn’t a more general way to destroy the whole ManagedWindow instance after close because there are still leftovers on the heap which can’t be collected because they are still referenced.

We could add all those properties like i did with the 5 to get them deleted - but is that the way to do?

grafik

54

We had a problem in the past with garbage collection of Gtk objects. The track_ref_for_deletion method was created to overcome this and it is perfectly acceptable to use it.

The Uncollected Objects should always be empty after a reset. I have fixed a few such bugs recently.

55

@Nick-Hall ok, there are numerous self.something variables registerd in verify.py’s main window Class which are abviously not freed after the window is closed. Just that I understand you correctly - are you suggesting to hunt them down and call track_ref_for_deletion for each of them? It’s fine for me - I just want to do it “the right way”.

56

Often deleting an object with del will work to break a dependency cycle.

I usually try to do as little as possible, but sometimes it is difficult to track down the cause of the problem.

57

OK, I managed to find some troubling variables which I registered with track_ref_for_deletion. I’m now down to 8 uncollected objects - with not much further luck getting rid of them so far. I guess its already way better than it is in master right now (down from over 30 Objects to now 8)

grafik

58

Do you get the same results with the Uncollected Objects gramplet?

Or is that Debug feature more for a view than for a tool?

59

That is the Uncollected Objects gramplet I’m using :smile:

1 Like
60

Thanks. I am in the process of revising the 5.2 gramplets page (so Help Buttons work) and have never used it … since I’m not a developer.

Any feedback about what was not intuitive about the Gramplet would be helpful. It can be rolled into the wiki docs.