I usually say that speed is not my greatest issue and it isn’t, however my database (not my tree) recently exceeded 20000 names and there is a definite slowing down of the searches in the likes of Graphview so was wondering is there some sort of tipping point in terms of numbers and is this related to the speeding up I know is currently being looked at.
phil
It is hard to say specifically without more details because there are many different issues that could slow down the system. If you could narrow down the slow functionality that would help. Try closing all of the sidebars (left an bottom) and note exactly which items seem too slow.
Many Gramps users don’t seem to worried about performance, until they do. I’d be happy to see more interest in addressing the performance PRs that I have raised.
Many users are interested in performance issues, but most of us also realize that it will take time to make the necessary changes, like moving logic from the frontend to the backend. Sadly, not all of us are developers.
All I can do is support the suggestions I see here on Discourse that look promising, and share my own thoughts on what might improve performance. On a more personal note, I am just waiting for the day I can start asking about Main/Sub-events and Events on Places again. Until then, I hope more performance issues can be resolved if the developers get enough peace and quiet to actually work on the code.
Regarding this specific issue, Nick mentioned something about a memory/cache hack in another post here on Discourse, that might help. I just can’t seem to find it right now.
In the Tips for large databases wiki page I suggested increasing the read cache size. We did this by default on the BSDDB backend.
A little background may help here
My hardware all at least 5years old consists of i5 NUC 16GB ram PC
linked via ethernet to a NAS where all Media Files are stored, during
that time I have moved from Ubuntu to LMDE6 and then LMDE7, GRAMPS 5 to
6 with the database growing slowly with little evidence of slow or
slowing response. For comparison GRAMPS media 15GB, Photos 150GB, music
files 700GB and no evidence of slowing with either DigiKam for photo’s
or Lyrion Media Server for the music.
Now to GRAMPS, Sidebars have been the same for as long as I can remember
for regular use turning off might have made slight difference to the
stutter which is all it is no chance of making a coffee whilst it happens.
So I notice the issue mainly when using the search box in Graphview and
elsewhere when searching for a name. Also if I add a new media item to
my files and the go to the event to attach it (Gallery +) I get the
whirring dervish for a few seconds before it finds the new file which
never used to happen.
Hope this helps someone but please do not spend any time on it.
Noted Nick’s comment on cache size will look at this.
phil
Made the mod to sqlite.py will see what happens
phil
The cache mod appears to have made no difference to the speed.
However looking at one of my older smaller databases the improvement is
noticeable.
So I have either got a problem of my own creation within the database or
there is indeed a tipping point when the quantity of data goes above a
certain point.
So will start verifying all elements of my database may take a while.
phil
Looking at this in more detail
The delay occurs if you have to change media folder/directory and when
you select the correct folder, not in finding the actual file if you are
already in the correct folder.
phil
Continuing the Slow Filtering/Searching
I have been investigating this further, My main interface being GraphView
The GraphView Search box will allow either Smith John or John Smith as valid search parameters. John Smith being slower than Smith John some what expected and definitely noticeable on larger databases.
People Selectors (Example Child Selector)will only allow full or partial Surname in the Search box and the Press Enter. This is fine until have a surname with hundreds of people with different forenames (given) then we end up scrolling.
Relationship Calculator works differently again either you Scroll through potentially thousands of Surnames or you use CTRL+F Search box (which is not available in the above 2) which scrolls the surnames as you type but then you hit the same problem where you have hundreds with the same surname.
So my question is why can they not all have the same functionality and ideally work in the same way preferably as the Relationship Calculator CTRL+F extended to include the forename.
phil
One factor that also might be causing a sudden drop in performance could be simply running out of physical memory. The OS will start using virtual memory, and the continuous swapping of data between physical RAM and disk storage will slow down the whole computer.
This isn’t specific for Gramps and is not specifically linked to a certain size of the database, but the effect will show sooner with a larger database. Having many applications and many browser tabs open (especially streaming ones) will increase the total demand for memory (“memory pressure”).
If the memory pressure gets high enough, the whole system will slow down, including Gramps. Operations that require a lot of disk I/O will be especially affected if the data is stored on the same device as swap space for the virtual memory is (which nowadays is usually the case, if you run everything on one computer). The use of disk I/O by the OS for virtual memory page swaps then clashes with disk I/O for the application data. If the data is on a remote disk, the problem is not as bad, but the general slow-down will still be there.
On macOS you can check the memory pressure by running the Activity Monitor and showing the Memory tab. It has a running graph for the Memory Pressure. When the graph is green, there is no problem. When it turns yellow, the system slows down, and it’s time to close some applications or windows. When it turns red, the system becomes nearly unusable. Don’t panic, patiently save any essential data and start closing applications one by one until the graph turns green again.
I expect other operating systems will have something similar to show memory pressure.
Apologies for delayed response
Yes indeed hardware can be a problem and yes memory usage can be monitored on Linux.
However when doing speed comparisons I would normally only have the program concerned running along with the operating system itself if you do not do this you can be led astray and deceive yourself.
The issue I am describing is around sorting and filtering and is a minor annoyance to me no more than that, however given previous conversations on this forum suspect it has a very complex solution if any.
phil
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.