I’ve been using Gramps now (v5.1.3, Ubuntu) for about a year. After importing lots of records via various GEDCOM files (probably a common noob issue), I’ve now aggregated over 27k place entries, much to my chagrin. As many are duplicates, some with identical names and others with slight variations, I am now faced with the task of using the Merge tool, 2 places at a time, to normalize my place data. I despaired at this prospect, so decided to try my hand at modifying the merge interface to accept, and properly handle, more than 2 entries.
Through deft programming skillz (ahem, trial-and-error), I’ve managed to create a local mod that handles this case.
Looking at the discussion here (Merging places - is this possible?) I’m wondering if there’s a fundamental, or stylistic, reason that Gramps doesn’t have a utility already to handle this? Or (being a noob), perhaps I just overlooked it?
Looking forward to flames…
No flames here! Anybody willing to take on merge issues has our attention. The merging of Alternate Names for People was ugly enough. I shudder to think of the issues related to merging Places … resolving their differing dated hierarchies and alternate names.
Note there is a Command Line Merge developer tool download on the add-ons list. (No documentation page but it refers to a usage example on GitHub.) If you wanted to build a script by exporting a place list and using external analysis, you might be able to automate some of the drudgery.
And I know I’ve used a Multiple Object Merge Gramplet… but it STLL required a huge amount of manual selection. (As does the Find Possible Duplicate People tool.)
I’ve taken to merging them gradually with a massive batch recently to tidy up and add the “enclosed by” bits to tidy stuff up. Some odd places that come under a city when using Google Maps that I’d not link to the location as way too far away but that is just for me to decide what goes where.
It is a pain with the 2 items at a time merge but still useful. Gramps still handles duplicates far better than other tools. Roots Magic’s auto resolution is very hit and miss.
Thanks for the suggestion! I’ll check out the Command Line Merge.
My trials with my very fresh code seem to be working, in the mean time.
I’m cleaning up my debug comments and checking edge cases. Then will give it a crack at my primary db. After a backup, of course!
kjvincent1979: “gradually” and “massive batch” seem a little contradictory. How do you mean?
I should think that he generated a massive batch. But it would take a long time to do that many merges in series.
So he’s executing graduations (say, 50 commands at a time) of the batch & doing backups between each.
That was bad phrasing on my part.
I’ve been trying to correct things as I go and enter places in a consistent format. So, little and often has helped, especially when dealing with a specific part of the family.
I sat and worked though the whole lot in one sitting, so my batch, the other night. It took a while but I created a master location for each that was correct and then merged or edited others to the same format.
Just doing 2 at a time is a pain but did work ok.
My biggest problem has been similar place names and avoiding merging those. Though, in my case I didn’t even realise some born in the same church name were actually counties apart.
I’ve not touched commandline yet but I will have a look at some point.
There is a very useful tool if you want to clean and normalise places, called Place Cleanup (Addon:PlaceCleanupGramplet - Gramps). It uses Geonames to find places and their hierarchy. I used it to clean my places database, and if you use it, you’ll see that the further you go, the faster it goes. And if you do the same cleaning on 2 different places, they are automatically combined.
I have also developed a gramplet that can merge multiple places (or other objects) in one step. See
There is also an “automerge” feature: the gramplet can automatically merge all places with the same name or title.
Maybe this is useful for somebody.
Thanks, kku! I’ll check that out!
Now that I have tried OpenRefine, definitely for such a large number of places, I will try to reconcile these places (between them and eventually against wikidata) with this external tool by exporting the gramps database to sqlite and cleaning and selecting my data with it then exporting a sql file to replace the sqlite table of places. Finally by re-importing the sqlite database into a new Gramps database to see if it is good as I want it
Edit: place table and link table to make place use unique (to do as merging do in gramps)
Interesting…I know nothing about OpenRefine. Have looked at the sqlite tables via a frontend, but was a little daunted trying to manage the data from there.
@kku wow…what a powerful set of tools you created! I just gave the MultiMergeGramplet a spin and it works like a charm! Thank you!
This is the second time that it has seemed like running PlaceCleanupGramplet would improve my places. The first time I requested a login and got distracted before it got approved. This time I got it configured. 
I started with the United States of America and then started into states enclosing it.
Not sure where I got a warning about a location no longer in use, but ok.
I am disappointed that as advertised not all of them came up with type state. ok.
It certainly did not merge Texas, United States of America with TX. Dissappointing.
Upon further examination this tool does not address the historical nature of places either. This is absent in my current endeavors, although it has come up in the pre US times.
Also under United States of America, I had Washington DC.
I am now wondering if this rates an enclosure and what the types are.
The next thing I tried was to add Wheaton College. I received a nice result with lots of commas. I did NOT end up with the appropriate enclosures.
What did I get from this experiment? an awareness of aliases.
So I don’t feel like it cleaned up my places. Am I missing something?
Thanks for your time and patience, Karen
One site I find helpful (for the U.S.) is the Atlas of Historical County Boundaries from Newberry.org. It is manual entry but a great source of the coming and going of States, Territories and Counties. There are some mapping tools that may help to determine how these changes may have affected individual cities and towns.
I have “District of Columbia” with the Type “District” and at the same level as a State with Washington as only city.
Arlington Co, Virginia and the city of Alexandria also show up for a period of time (if I have correctly mapped them through time) so splitting the City and the District as separate entries may be desirable.
As to the PlaceCleanupGramplet, I have dabbled with it but since most of my places structure was present when I migrated to Gramps, I tend to do the fine tuning editing manually. If I was just starting out, the tool would probably be a huge benefit.
There is an addon for Openrefine that add a Local Reconcile Server for CSV files.
You find it a at this page: Reconcilable Data Sources · OpenRefine/OpenRefine Wiki (github.com):
Githup Project is here: okfn/reconcile-csv: A simple OpenRefine reconciliation service that runs on top of a CSV file (github.com)