Performance problems with large datasets

I’m new to Gramps and recently imported my wife’s GEDCOM data. She has about 186,000 names and the system is very sluggish and takes 40-90 seconds for a simple name search. With my own development projects I would look at the database and start making indexes, materialized views, and other creative ways to improve performance. I know this tree is larger than most but from a development perspective, it isn’t a huge dataset. I searched through this forum and didn’t see any recent discussions about performance. Is that something you’re working on? Is there anything already in the roadmap? I’m fine getting set up and helping with development but I didn’t want to start something that’s already in progress.

Thanks! Ben

Hey, Ben; we do have some things on the roadmap!

Can you say exactly which search you were using? What screen? There are a few things we’re working on, and more that we could.

Thanks for the quick response. I was using the People Filter. I attached an image to show it.

Making such search on names from People View instead of the Grouped People View (see toolbar, category navigator or the menu) might provide better performances.

There is also a dedicated wiki page for tips with large databases. This will not be a solution, but this could help to understand current limits (some of them might be outdated on recent configs too).

e.g., Fast search can be done by enabling column search (disable Sidebar on View menu), but this will search all letters on words into the column, not only the first letters.

For a surname search:

  • Click on a person in the People Grouped Category view. Now start typing the family name of the person you need. The view will automatically go to the family name as you complete your typing.
  • For a more detailed search, use the keyboard shortcut CTRL+F. This will expand all the nodes and allows to search more specifically.

Flat views are faster than Tree views

Using a large family tree database and the People grouped tree view needs to often rebuild the view especially after adding or deleting an object eg: Flat People (list) view is faster than grouped view.

https://gramps-project.org/wiki/index.php?title=Tips_for_large_databases

@bhofmann Thanks for sharing your data! @romjerome is correct that using a flat view is faster, but not significantly. There are two significant factors:

  1. A bug hidden in some innocuous-looking code that is now fixed. That alone reduces your 77 seconds to probably around 10 seconds.
  2. Being able to use SQL. With a solution planned for Gramps 6.1, that reduces the time to 0.41 seconds (my test database has 100k people).

CC: @UlrichDemlehner

Thank you both! I figured you’d be on top of it. I’ll try the flat view as I wait for Gramps 6.1 to be released. Thanks!

Thank you! I’ll check it out.