Merging data from different sources

Help
1

Gramps version 5.1.4. on Manjaro Linux, community version

Dear all, I tried to find a solution in here already but either my search skills are bad or it isn’t in here (I doubt that).

Together with another family member I am working on our combined trees and we somehow get lost on how to import and more important, merge data. I can import CSV sheets but it is seen as a different instance, it will not be merged. Could anyone try to explain what it needs to fix this?

Kind regards, I admire all the effort that has already been made

2

Welcome to the discussion group @hansw-nlo

Am I understanding your situation correctly, that you and a family member both have a Gramps family tree which you modified individually, and you want to merge it now?

Did you both start from the same tree at one point?

It would help to know a bit more as to what you’ve tried so far, what worked, and what isn’t working.

3

Thank you,

He already had a different solution but switched to Gramps since it is better to keep an eye on what to do. I also started a tree a few years ago.

Partly there are the same family members in both trees

4

There are people collectively doing genealogy with Gramps as one of their tools… even though it wasn’t designed for collaborative work.

The Isotammi project is building Gramps tools for collective genealogy of Finland. Their Multimerge Gramplet is a popular addition.

I had a thought if you are doing a 2 person tag-team approach with your genealogy. There a discussion about an experimental TreeMerge addon. And there’s the Import Merge add-on tool.

Neither considers the Gramps ID as a reliable hint for merging. But what if you were able to work out a schema for helping the Merge process? Maybe the developers could be encouraged to leverage it as an option?

In combination with the datestamps, merged Gramps IDs and some sort of audit trail, a smarter merge might be possible.

Let’s say that you and your partner decide to loosely coordinate your work. You’ve started with a common base Tree in Gramps but each works independently. But you periodically harmonize the 2 Trees.

If you were to change your Gramps “ID tab” Preferences so that each researcher’s new object were in different ranges, then neither the Gramps ID nor the seeded handle are likely to overlap. In combination, they’d be very strong indicators.

By default, the F%04d range for Families allows for 10,000 families, ranging from F0000 to F9999. So instead, I might use a Fa%06d ID, ranging from Fa000000 to Fa999999. Meanwhile, my partner uses Fb%06d and the master merge Gramps uses F%06d (without a collaborator modifier).

Since ASCII has 52 single-keystroke non-symbol & non-numeric characters, you could support up to a team of 52 partners. (52 squared or 2,704 with a 2 character code.)

Any object with a Partner’s ID would not yet have been validated & merged into the Master tree. And, as part of a 2-person validation process, you should only merge your partner’s data into the master, not your own.

There’s already a loose form of coordinating the Place objects using the GeoCodes as IDs via the Place Cleanup addon. Any of the Place record IDs beginning with a “GEO” is a part of a master tree of Places.

2 Likes
5

Can you give us an idea about their number?

If it’s small, like a few dozen, the existing tools to find duplicates will work quite well. I only get in trouble when I know that I have hundreds or even thousands of the same people, like when I want to compare my Gramps tree with the version that I have on Ancestry.

6

If you both use Gramps, there’s no need to mess with CSV’s, or GEDCOM files, or anything like that. You can both make exports, like for your respective ancestors, and send the resulting data.gramps files, through mail, or Dropbox, or anything, and import those into a new tree that you create by importing a backup of your own into a new database, followed by an import of the data.gramps file sent by the other party. And you can also use full backups to eachother, and import those to see what you have.

What works best really depends on how tight you are, both as relatives, and by the number of common persons that you both have in your trees.

7

In my gramps database there are some 70 people, 10 families. His old system is holding a lot more.

What I wonder is

  1. Can I import his data into my instance and will links between people, places, etc… only be double of will it be mixed up due to id’s?

I know mysql can make unique identity numbers but indexes are not possible in here. Nor remote constraints

8

Hmm, interesting, we might try to do that. I did notice it was imported as a seperate instance (8 places hex) next to my own instance. The wish is to get a mixed instance with both his and my data

9

When you do an import, Gramps will always create new ID’s, so all persons, places, and other objects that you both have will appear as duplicates in the merged database, which will then sort of have two trees, when you see a tree as a cloud of connected people.

Now, if you have an idea about the duplicates, you can do a manual merge from the person view, where it’s easy to see that after import, you have your common ancestors twice. You can then merge them one by one, so that, after a while, you have one tree without duplicates. And from my point of view, your tree is small enough to do that. You may also need to merge places, and that too should not be that difficult. You can also try the tools mentioned by @emyoulation , but I don’t know much about them.

Please note, that real imports never appear as a new instance in the grampdb folder. Those instances are only created when you double click a .gramps file. Gramps will then create a new database, with a new magic number (your 8 places hex) and that’s not what I mean. Real imports happen when you open your database in Gramps, and use the menu to import another .gramps file into that, or a GEDCOM file for that matter. But if you both work with Gramps, exchanging .gramps files is the best choice.

1 Like
10

Thanks all,

I will do some additional tests based on what I read here.

11

The Import Merge actually allows IDs to be kept during import of a revised .gramps/.gpkg file. (Can’t say what it does with Internal Gramps Handles.)

It seems like it might be worth experimenting with the scheme of separate collaborator ID ranges.

12

Well, If I look at the import data then it is not in my tree but imported as a different tree (import using csv). That is also one of my concerns. I simply want to merge the trees.

13

I understand what you’re saying, but I was referring to the standard import, which will re-arrange IDs if the receiving database is not empty.

Also, when people have shared ancestors, I don’t see much sense in having separate ranges, because they will be messed up when duplicate persons are merged.

As far as I’m concerned, it’s more important to think about the process after the fork.

P.S. I’m replying by email now, hoping that this will work.

14

What do you mean by a different tree? Another database, or a group of persons that is not connected to the other ones in that database?

Importing CSVs is nice when you worked with a spreadsheet earlier, and because it can be used to overwrite existing data, if you know what to do, but it is rather incomplete, so if you have a GEDCOM or a .gramps file, it’s much better to import that, because there is much less data loss.

We have a wiki page with a lot of information about imports here, and I recommend that you read that to understand the process:

https://gramps-project.org/wiki/index.php/Import_from_another_genealogy_program

This article also has links to manual procedures for merging persons, places, etc., but it does not mention the tool to find duplicate persons, named “Mogelijke Dubbele Personen zoeken …”. You can find that in Hulpmiddelen/Stamboombewerking.

The procedure that I recommend is that you import the .gramps file (or GEDCOM) that you get from your relative into a new tree (database), and then import a backup of your own tree into the tree that you just created. You can then use the find duplicate persons tool to start merging, or start merging from the person view. You may need both, because the tool may give low scores to persons that don’t have much data, for whom you know that they are the same.

Another useful tool is the “Verwanten” (Related Persons?) Gramplet. If you add that to the side bar of the Person view, you can immediately see whether a selected person has multiple parents, spouses, or children, that you forgot to merge. And if you see some, a click on their name in that Gramplet wlll move the Person view to that person, so that you can perform a manual merge.

15

Thanks all, I will wait for the next export he makes and see what is that best method for us.

1 Like
16

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.