I have many entries for towns and villages in England.
If I add a new Place such as “Banbury, Oxfordshire, England” with an Enclosed By of “United Kingdom” (which already exists) and the Date: “after 1800-12-31” it takes 10 seconds to close the first window.
This is in English on a Win10 machine with a SSD. No other add acts like this. Adding a Place without the Encloded By does not cause this problem.
H’m, weird. When I right click United Kingdom in my location list, choose Add, and then enter the place name, and add the date to the enclosed by line, it takes less than a second, even on Windows 10 running inside Virtual Box.
@emyoulation how does this work on your sleepy Windows?
Try this on your machine:
- in People view, select a person and open an event
- click the + in the Place field
- type a new name of a place in the name field
- go to the enclosed by tab and select the page icon
- in the search field type in “kingdom” and search
- select “United Kingdom”
- in the date field add “after 1800-12-31”
- select OK
Now wait 10 seconds for it to complete.
Edited to correct process.
As you stated, doing a simple Add of a new location (with or without date) is fast. As is adding a new alternative name to an existing Place
However, inserting a new enclosing Civil Division level into an existing hierarchy is slow as Mudd.
As an example, I have a Place Hierarchy created for England currently enclosing about 160 (out of 4,011 total) places applied to 1,558 (out of 97,038 total) events. Not long ago, I inserted the eight traditional geographic regions between the Country and Counties levels. This was painfully slow.
@Davesellers I tried Linux and Windows, on my desktop, laptop, and in two different VMs inside two different Linux versions, and for me there is no way to reproduce anything that you describe here, meaning that each OK takes less than a second to be processed.
@emyoulation When I try this on the same environments, not using the traditional regions, but moving England inside the United Kingdom, with a similar number of enclosed places, the largest delay that I see here is about 5 seconds, and that’s on Windows 10 Pro running in Virtual Box, on an i7 with 8 GB and a HDD, with 4 GB and 4 of 8 cores assigned to the VM. This VM has a Windows experience index of 4.3. What’s yours?
SystemScore: 5.8
AMD A6-3420M APU with Radeon™ HD Graphics 1.50 GHz (4 cores)
8.00 GB (7.48 GB usable)
My scores are:
CPUScore: 8.4
D3DScore: 9.9
DiskScore: 7.35
GraphicsScore: 8.2
MemoryScore: 8.4
My records are:
The 1059 records are like this: Alcester, Warwickshire, England
In addition: Scotland 139 records, Wales 79 records. There are a total of 1627 records in Places
These are enclosed by “United Kingdom”.
I don’t break out the regions or districts or parishes.
Windows is always started fresh each day and I have never seen a memory issue.
Did a further test as described in a previous post, it took 15 seconds to save.
I did a like test from Places. (which works differently than from an Event)
- highlighted United Kingdom in the list.
- selected New (the screen was populated with United Kingdom in the Encloded By)
- added a name and saved
- less than a second to save.
If you don’t know how to use a relational database, yes that is true…
But this type of combined data from different tables is what Views are for… the Query is done on the database level, and it is lightning fast… only thing the software needs to do is a simple query from the view.
Same when writing or updating the data, all the software do is commit the data to the view, the database engine does the rest…
And SQLite, Postgresql etc. are made to work on millions of records, so a few 100k records from Gramps wouldn’t be much…
Just as an example, I have a Postgresql database with a little over 1 million records (different places)
I have tables for Country, State, County, Municipalities, Cities, Street Names, House Name, Farm Names, House/Street Number, Postal Codes with an reference to it’s place name in another table (two ways) in any of the other tables, in addition there is a few other tables (something similar to geonames)…
To run a “join select” query on that database and get an output as a structured table sorted on Country, then State, then county, with a total of 15 fields (some empty) takes approx. 3.5 sec without a view, with a query on a view of the same data it is nearly instant…
Edit: had to add half a sentence that I forgot, and some typos.
@Davesellers I don’t get it. My place table is larger than yours, and my hardware is slower, and yet, the whole operation is 10 times faster on mine. And that suggests that we’re missing something, like an active Gramplet that’s triggered and gets in the way, because there is no way that the database, or the data model could take so long for a simple operation that works on 3 records, meaning the person, event, and place. And this has nothing to do with the fact that we don’t have a relational database, because if it had, the effect would be the same here. That’s just a red herring.
What Gramplets do you have in the person view?
The only Gramplets I have loaded in the People view is Filter and TODO.
As a test, I went to Events and created an event with a new Place. Same thing, 17 seconds to save.
I wonder if the Hierarchy has loops that have to be resolved. (I’ve wondered if my Place Hierarchy could be optimized. Some time ago, I tried a few experiments that were suggested to improve my ability to select commonly used places. One experiment involved parallel branches of greater metropolitan areas that could span state lines. I also had western Pennsylvania isolated. And another that had regions of the the US: North East, South East, Central, etc.
1 Like
Do you mean a place with an enclosed by? How big are your person and event tables? Do you have any fancy virus scanners, or file history, or anything like that? Do you have another machine that you can try this on?
I’m quite convinced that it’s not the database technology, because the operation is so simple, and involves such a small amount of records, that it can’t be spoiled by code in our data model, even if that would be 10 times slower than the database engine. I know that, because if that code were really bad, I would see the same problem here.
The proof of the pudding might be to import the tree into another Gramps installation, on another machine, but I don’t know whether you have that. Another option might be to share a backup some place, so that I or @emyoulation can import it. I have two desktops, one laptop, and 2 VMs, that can all run Windows 10, or 11, and 2 Linux setups on my oldest desktop.
The database that I test with is my own tree, with 10,440 persons, 24,267 events, and 4870 places in Gramps 3.4.9. When I import that into 5.1.5, and generate a place hierarchy using the Place Update Gramplet, the number of places increases to a little over 7,000, without loops, or time ranges, and the depth of the hierarchy is mostly 3 or 4, with country, state or province, county or municiplality, and town. Streets are rare. I test by doing things that are quite similar to what you do, like moving the whole England tree inside the United Kingdom, or London inside Greater London, that kind of things.
I have Latitude and Longitude in each record. It there any tie into the mapping feature that could affect this?
In what way? You’re adding ONE place to the hierachy, and the map is not visible when you work in the person or event category, right?
@emyoulation we have an add-on that can detect loops. Have you tried that?
I got a warning today, when I was about to create a loop in the place hierarchy, but I have no idea how new that warning code is, and whether it’s reliable.
1 Like
No map visible. Just speculating that something may be there under the covers. Didn’t know if your records also had Latitude and Longitude. Looking for differences.
Don’t have the add on. Where do I find it?
I have used the “Find database loop” utility. But that probably focuses on Relationship loops… like if I am my own grandfather.
Ok, got it. No coordinates here, just names, and some titles that are different from those names. But the puzzling thing is, that creating one place enclosed by another, and referenced by an event, means looking up the handle of the enclosing place, which is already in memory, creating one new place object and writing that to the database, and storing its handle in the place field of the event, and updating that. And these are all basic database operations, no matter whether the database is relational, or it’s not.
And that’s why I keep thinking that the problem is in the business logic above that, or in the UI.
You’re right. It says so on the wiki.
@Davesellers would you like to share a copy of your database with me, via OneDrive, or any other means that we can agree on? I can give you my email in a personal message, and it allows me to see what it does on my hardware, in Linux and Windows.
I hope that a backup without media will do, assuming that the backup is an accurate export of all relations that are present in your database. I will delete it after testing, but chances are that I will scan it for familiar names.