Threading (or multiprocessing), performance and refactoring

[romjerome]

@dsblank

But you call that again on line 3. I don’t think this code is doing what you think it is doing. And this doesn’t look like proper thread management. But even if it were done properly, Gramps makes no guarantees that you can handle database actions in a thread.

Yes, it was the plan! I just call it twice to keep a trace of the incompatibilities you warned me and also listed on:

Threading (or multiprocessing), performance and refactoring

The primary idea was to move some basic calculations to functions and their related threads. I made a mistake on one of the function and the test for limiting entries was broken into the loop, generating extra calculations and increasing time process. So, testing my primary idea around threading should be now complete. After looking at some samples, I just understood that dealing with lock() and playing with location of join() might not provide the expected result. The thread on filter could be…

Something like a testing without crash because the thread section does not really generate a useful data. The second one does. Sure, a final version (polish outside draft PR) should remove the thread “lines”.

I remember that one script on experimental Gram.pycould generate something very close (at least the primary idea).

New Gram.py Script for 6.0

Relationship list: from gramps.gen.relationship import get_relationship_calculator rc = get_relationship_calculator() columns("Relative", "Relationship, of", "Active Person", "Dist org", "Dist other") for person in people(): (relationship, _ga, _gb) = rc.get_one_relationship( self.db, active_person, person, extra_info=True, ) if relationship: row(person, relationship, active_person, _ga, _gb) [image]

By looking at some comments on code, history and documentation, there was some performance issues in the past (e.g., SQLite backend 30% slower than bsddb3 backend). As it was some years ago, I just try to improve a little bit any possible way to limit any slow down or extra time process with large database.