How can people be deleted according to a list?

Help
1

Gramps 5.1.3 MX Linux

All people were exported from the existing database into a CSV list. Which ones are to be deleted were determined according to various criteria and now a list of GRAMPS_IDs remains which are to be deleted from the database.
Trying to do a direct query with the DB Browser for SQLite "DELETE from person where gramps_id in( ‘I1234’,…) " works, but I think it’s not clean because Gramps gives error messages afterwards. I ask for help, for example if it would be possible with the Super Tool or something similar.

2

Are there a lot of people in the list of IDs to be deleted?

SuperTool works in frames of 1000 objects at a time (due to a Gtk limit). And Gramps 5.1.x is VERY slow in deleting a large number of objects. (It refreshes after each object deletion. That’s fixed in the 5.2 beta.)

But it is certainly viable to do this in SuperTool.

The standard caveat: make a backup before any bulk operations. And name that backup so you know what data it has. (It might even add a ToDo Note describing why you doing the Deletion and another Note with a CSV list of those about to be deleted. This would simplify exporting the same people … their family objects… to reverse the action in the future.)

Personally, I’d use SuperTool to Tag the list by processing the CSV list of IDs in that Note. Since Gramps 5.1.x is glacially slow for deletions en masse, my workaround would be to filter out those tags on an export.

(Well, actually, I’d use the unreleased beta of 5.2 where the large deletions are fast!)

3

First - Are the persons to be deleted without any events, notes, or any other links to other objects within the database. All such links should be removed before deleting any persons.

4

This is important when removing people by directly touching the database instead of using the Gramps User Interface. (Since there are provisions in the UI for resolving those references from other objects.) However, removing a Person does not remove its secondary objects. (Since those objects could be shared or be re-usable for other purposes.)

6

Thanks for the feedback.
The entire database consists of 68000 people.
45,000 people are to be deleted.
For the linked notes & events, I would use the tool to delete unused objects afterwards. - I have already tried this procedure with the DEMO database and it worked.

How would the command work in SuperTool?
db.delete_person or something like that?
I would like to try it and report back. Of course I have corresponding backups of my database.

Thanks

7

Can you create a filter that selects the people to be kept?

Create the filter and then export the results to a new file that you then import into a new empty tree/database.

1 Like
8

How can I create a filter that contains 45000 Gramps_IDs?

9

Just did a test. This is ugly, but it works.

Create a CSV file with the IDs of the People you want to delete in Square Brackets and a dummy surname. Then Import that CSV file to update the tree. (Data loss! Backup immediately before the import.)

Follow the format below. I use “DeleteMe” as the surname. Importing it will overwrite the names of all those people. (Note that the Given name is lost too… even though that isn’t one of the import columns.)

Now these deletable persons will be easy to filter out of the export.


Person, Surname
[I0008], DeleteMe
[I0001], DeleteMe
[I0010], DeleteMe
⋮
[I43115], DeleteMe
[I43116], DeleteMe

An export filter might look like:
(The “Return values that do not match the filter rules” is critical.)

image
image676×559 34.6 KB

1 Like
10

You are looking for a filter that selects the 23000 people you want to KEEP!

Then you create an export file of these people which you then import into a new empty tree/database.

1 Like
11

But his message says that he has a list of the People he does NOT want to keep. So the workflow has to be building the opposite of that filter. (Add to that understanding … that the CSV import does not currently have the ability to set a Tag or add an Attribute or some other non-destructive property.) So we have to fallback to a destructive method: mangling the name of the People to be excluded from the export.

Thus, the custom filter finds the People WHO DO NOT MATCH the people to be removed. And export just those 23,000 people.

12

I tried the scenario with the DEMO database (2157 people in total), and removing unused objects afterwards also worked.
This solution meets the requirement 100%.
No extra add-ons are needed, you don’t need any programming knowledge and it’s easy to remember.
Thank you very much! I am very happy with this solution.

1 Like
13

Nice use of the CSV Import! I was thinking of uses just like this when I wrote it. Your solution suggests two follow ups:

  1. Fix the bug where Given name is removed
  2. Add the ability to add tags to a Person (etc). That would make your solution less “ugly” :smile:

Patch created 28 Dec 2023: Update and fix CSV import by dsblank · Pull Request #1629 · gramps-project/gramps · GitHub

1 Like
14

Thanks!

If you get into it, Serge added the option to use tab delimited (TSV) instead of comma delimited (CSV). This makes is possible to paste into the Import Text gramplet from copied Excel cells rather than the extra steps of saving in CSV file format, and importing. The interface to enable is a bit of a hidden Easter Egg… you have to use the Configure for a tabular View where the View can export to CSV. But that enables TSV in the Import Text gramplet too. Very cut’n’paste friendly.

I am also confused by the Source options in the CSV import. I don’t see how to specify the Citation Page/Vol or Date. But Sources seem to generate a Citation. Is it an assumption?

Likewise, it looks like Marriages import seems to assume the Relationship type is “Married” only if both Spouses are specified. If both are not specified, the type is “Unknown” I would have assumed that type if there was a Marriage Date or Marriage Place specified. Or maybe some reliance on the Preferences for default relationship type.

Not asking for changes, just information to fill out the docs.

An interesting thing discovered during experiments is: while the export to csv from a view may not output Headers that are recognizable by the Import Text gramplet, the Family Trees → Export… to CSV of the whole tree does write recognizable headers. Or maybe I had to enable the “Translate headers” option for that? More experimenting to do!

1 Like
15

TSV is an interesting addition; nice!

It could be that the CSV Import was written before there was a default family relationship (there is one now). I’ll look on adding that if not already in there. (“Unknown” is the default, but I change it to “Married” in the config whenever I setup a new gramps install).

The CSV Import/Export was one of my first contributions to gramps 17 years ago as I hate typing and cut and paste in many fields is painful. Glad it is still useful!

1 Like