Gramps v6 regex search much slower than Gramps v5.1?

I just installed Gramps v6.0 on my Win10 machine from the AIO package. After importing my data from a Gramps XML file (approx 90 k individuals, 300 k events, SQLite on a SSD drive), I did some quick and preliminary performance tests and found, that a simple regex search for a name in the person view now needs 47 seconds compared with 19 seconds in v5.1 (BSSDB on the same SSD drive). This is quite similar to v5.2 where the same regex name search also needed nearly one minute and forcing me back to v5.1.

Is this result realistic or did I miss something when installing v6.0 (e g creating indexes)?

PS: To avoid misinterpretations: the performance problem exists with both regex and non-regex searches and there is no noticable difference. I also created indexes on all columns in the SQLite database that had none (in particular the JSON columns), but this makes no difference.

Searched the old mailing list and new discourse forum and also could not see any raised issues on the bugtracker for Gramps 5.2 from yourself or anybody else about other people experiencing slow regex searching can you please raise an issue so the developers know that it slowed down for you between the 5.1 and 5.2 releases?

Isn’t this the slowdown related to Name searches changing from Phrases to Words? @SNoiraud had some good insights about that and RegEx

This performance issue is something that should also be brought to the attention of @dsblank

There has been a lot of discussion on this forum about the performance issues, e g

Performance Issues with Gramps, in particular Performance Issues with Gramps - #16 by UlrichDemlehner

Error using PostgreSQL as database backend with Gramps v5.2.2 - #14 by UlrichDemlehner and below

Since it was clear that those performance issues are not the consequence of a specific bug but the consequence of an old software design decision (using Python blobs), it made no sense to file a bug report.

An executive summary of my experiences (all statements refer to Win10, database on SSD drive, approx 90 k individuals with 350 k events):

(1) When migrating from v5.1.6 to v5.2.2 and from BSSDB to PostgreSQL and SQLite, I found that the performance of searches went south. Regex name searches in the person view that used to run for 15 - 20 sec (which by itself is clearly not very impressive) now needed 45 - 60 sec, searches in the event view (which have always been the slowest searches by far) went up from 45 - 60 sec to 100 - 150 sec. Even searches in the other views (e g citiation or place view) which used to be quite fast (around 1 - 3 sec) were now significantly slower.

(2) I haven’t noticed any significant differences between PostgreSQL and SQLite but I have not done any sophisticated speed tests comparing both database backends. Based on the discussion I refered to above, this was exactly what to expect so tests did not make much sense for me.

(3) So at the end of the day, I had to go back to v5.1.6 on BSSDB where I’m still. And to deal with those performance issues, I wrote some apps by myself which basically “import” a Gramps XML file into a PostgreSQL database where I do all my searches.

Based on my understanding of all those discussions, the answer to your question is very propably no. And BTW: I haven’t seen any differences between regex and non-regex searches. They all slow down to a crawl when migrating from BSSDB to PostgreSQL or SQLite.

1 Like