Merge > 2 Places?

I’ve been using Gramps now (v5.1.3, Ubuntu) for about a year. After importing lots of records via various GEDCOM files (probably a common noob issue), I’ve now aggregated over 27k place entries, much to my chagrin. As many are duplicates, some with identical names and others with slight variations, I am now faced with the task of using the Merge tool, 2 places at a time, to normalize my place data. I despaired at this prospect, so decided to try my hand at modifying the merge interface to accept, and properly handle, more than 2 entries.
Through deft programming skillz (ahem, trial-and-error), I’ve managed to create a local mod that handles this case.
Looking at the discussion here (Merging places - is this possible?) I’m wondering if there’s a fundamental, or stylistic, reason that Gramps doesn’t have a utility already to handle this? Or (being a noob), perhaps I just overlooked it?
Looking forward to flames…

No flames here! Anybody willing to take on merge issues has our attention. The merging of Alternate Names for People was ugly enough. I shudder to think of the issues related to merging Places … resolving their differing dated hierarchies and alternate names.

Note there is a Command Line Merge developer tool download on the add-ons list. (No documentation page but it refers to a usage example on GitHub.) If you wanted to build a script by exporting a place list and using external analysis, you might be able to automate some of the drudgery.

And I know I’ve used a Multiple Object Merge Gramplet… but it STLL required a huge amount of manual selection. (As does the Find Possible Duplicate People tool.)

I’ve taken to merging them gradually with a massive batch recently to tidy up and add the “enclosed by” bits to tidy stuff up. Some odd places that come under a city when using Google Maps that I’d not link to the location as way too far away but that is just for me to decide what goes where.

It is a pain with the 2 items at a time merge but still useful. Gramps still handles duplicates far better than other tools. Roots Magic’s auto resolution is very hit and miss.

Thanks for the suggestion! I’ll check out the Command Line Merge.
My trials with my very fresh code seem to be working, in the mean time.
I’m cleaning up my debug comments and checking edge cases. Then will give it a crack at my primary db. After a backup, of course!

kjvincent1979: “gradually” and “massive batch” seem a little contradictory. How do you mean?

I should think that he generated a massive batch. But it would take a long time to do that many merges in series.

So he’s executing graduations (say, 50 commands at a time) of the batch & doing backups between each.

That was bad phrasing on my part.

I’ve been trying to correct things as I go and enter places in a consistent format. So, little and often has helped, especially when dealing with a specific part of the family.

I sat and worked though the whole lot in one sitting, so my batch, the other night. It took a while but I created a master location for each that was correct and then merged or edited others to the same format.

Just doing 2 at a time is a pain but did work ok.

My biggest problem has been similar place names and avoiding merging those. Though, in my case I didn’t even realise some born in the same church name were actually counties apart.

I’ve not touched commandline yet but I will have a look at some point.

There is a very useful tool if you want to clean and normalise places, called Place Cleanup (Addon:PlaceCleanupGramplet - Gramps). It uses Geonames to find places and their hierarchy. I used it to clean my places database, and if you use it, you’ll see that the further you go, the faster it goes. And if you do the same cleaning on 2 different places, they are automatically combined.

I have also developed a gramplet that can merge multiple places (or other objects) in one step. See

There is also an “automerge” feature: the gramplet can automatically merge all places with the same name or title.

Maybe this is useful for somebody.

4 Likes

Thanks, kku! I’ll check that out!

Now that I have tried OpenRefine, definitely for such a large number of places, I will try to reconcile these places (between them and eventually against wikidata) with this external tool by exporting the gramps database to sqlite and cleaning and selecting my data with it then exporting a sql file to replace the sqlite table of places. Finally by re-importing the sqlite database into a new Gramps database to see if it is good as I want it

Edit: place table and link table to make place use unique (to do as merging do in gramps)

Interesting…I know nothing about OpenRefine. Have looked at the sqlite tables via a frontend, but was a little daunted trying to manage the data from there.

1 Like

@kku wow…what a powerful set of tools you created! I just gave the MultiMergeGramplet a spin and it works like a charm! Thank you!

@Bouscram yes, I thoroughly appreciate the PlaceCleanupGramplet. Thanks for the suggestion!