I’ve recently been given ownership of a family tree that has been thru multiple different software packages over the years. All of the people that were entered from the late 1990’s into the early 2000’s have an event titled “Record Change”. This has led me to identify 2 issues.
First is there any way to programmatically remove all of these events? There are over 23K and removing them individually is painful. I’ve tried to remove the entries from an uncompressed export of the database. That produced a new Unknown event… not quite what I was looking for.
From scanning the documentation it appears that the database is never cleaned up. How do I change the DB from an external script to remove these entries?
Second, it appears that the sort order is sort by alpha or dates. The challenge that I’m running into here is that I can change the order manually, but that means I need to review every person in the DB manually 34K+ and put them in the right order. For me, the right order is Birth, Death, and Burial. I don’t have dates on everything. In some cases it’s just a location. The only way that I can see to fix this is to change the event names to include a leading number… something like 1-Birth, 2-Death, etc… Is there a place that I can make this change again with a script? IF there are other ways to solve this please suggest any options that you have.
Before following this suggestion, Back up the Tree to an external device & label to identify the stage of cleanup. People often find they go that “one step too far” during cleanups and have to revert. If you do this backup at each step, you can jump back to the appropriate stage.
To clean out those “Record Change” events, change to the Events category view, sort on the Type column (it is faster to change the sort order BEFORE filtering), use the “Type” field of the Filter Gramplet or the “Type contains” Searchbar to look for “change”. All those 23K events should appear in the list.
Check the Type at the top of the List, then scroll to the bottom to make certain they are ALL of the same Type.
If you are SURE they all need to go, select all (ctrl-A hotkeys on Windows) & delete. Deleting thousands of records might take a while. Do this when there will be some down time. Maybe overnight?
Also, importing into Gramps Tags EVERY record imported. (It is the default option in Preferences.) For an import that creates a Tree, this is a waste of storage and makes the Tree bigger for no useful reason.
Delete the Import tag via the Organize Tags dialog. This also flushes the References to the deleted Tag.
It should not be hard to remove multiple events. It is probably going to take a lot of machine time (hours/days) with that many events, but not hard. We may have a real improvement here scheduled for the next major version of Gramps that would make this a lot faster.
In the Events view, it is possible to sort by event type, select the first of the type of interest, then find the last via scrolling, and select the last to get all the intervening items. Then use the delete function (Edit/Delete).
Another way is to use the Events view Filter sidebar, and find with the ‘Type’ set to the type of interest. Then you can use a to select all the events shown. And delete them.
Have you looked at Tools/Family Tree Processing/Sorts Events"? Please backup your db before trying this in case it doesn’t do what you want.
This is not easy. In the export function, you can export and exclude items that are marked “Private” . These records have the locked padlock icon. The challenge is to mark all of these events as Private.
Manipulating the uncompressed export is the way to go but you do not want to delete the exents, you want to make them Private.
The standard format for an event record in the file is
Obviously the Event type will be the offending events you want removed and the Handle and ID are unique for each event. You just want the priv=“1” added to the line. Not every editor will allow the two line search and replace. I could do it with Windows Notepad.
Once the export file is altered and saved, you can import this file into a new empty database. After confirming that the events are indeed made Private, you will make a new export file. When doing this export, in the export options, check the box Privacy Filter; Do not include records marked private.
Once again create a new empty database, and import this last export. The event records should be removed from this tree.
Some caveats, the last export excluding Private records will exclude any record marked private. There are search parameters in each area of the database to search for records marked Private if you are not sure if this has been set on other records. And as always practice safe file management. Always know which file you are utilizing/manipulating at each step along the way.
The mass delete posted by @emyoulation will also work but will take a while to process.
There is no Pack & Reindex in Gramps as exists in so many database driven applications.
But you can create a new Tree & import the most recent backup. That will do the same thing.
Be certain to go in the Edit : Preferences… : General tab to deselect the “Add tag on import’” checkbox. The import will be much faster and generate a smaller Tree. Importing backup is a bit slower than imported an exported XML because the XML is compressed… but you DO get to skip the export step.
Warning. Do not use the tool Sort Events tool under Family Tree Processing. This will move events without dates to the top of the list. Most of my Burial records did not have dates and after using the tool, these were made the person’s first event.
At the time, I had 5X the number of records and the only way was to Clean each record manually.
I first tagged each person with a Needs Sorting" tag. As I cleaned a record, remove the tag.
Maybe others have other tricks to sort the events. My main reason to post was to warn against the Sort tool where you already know you have events without dates.
The natural sort order is by the record creation timestamp. If you let Gramps define the ID automatically, they’ll follow that order. However, you can override the natural sort criteria of any table of records by clicking on the Title of that column.
I agree with @DaveSch I used the “Tools Family Tree Processing Sort events…” once and have been regretting it ever since.
As Dave said, the problem lies in Undated Events.
There is a “Tools Family Tree Processing Calculate Estated Dates…” 3rd party add-on. But it only addresses Birth & Death events without dates.
But the rest are not considered even if they have an implicit order in a lifetime.
Implicitly, Burial & Cremation should be after Death (whether dated or not) and Cause of Death & Probate after all others. Likewise, Birth events are expected before all others. And you’d expect Bar/Bat Mitzvah at (or after) age 13. Then marriage & vocational after that. A few other have implicit order. The Family events seem to have a natural order only relative to one another. But adding custom Types into the mix really contaminates the recipe.
It’d take some serious commitment to make an automated sort consider all the implications. But other enhancements just seemed more practical. So I’ve just resigned myself to manual sorting.
Thanks for the pointer. I didn’t know that I could use the Events Category to get a list of all of the events… That was exactly what I was looking for… Simple and fairly quick. Looks like it can turn 3-4 events/sec and that speed is good enough for now.
I’ll have to look at the Sorts Events further. It seemed to work fine with children. I didn’t snap to the fact it could do the primary personal events as well…
Thanks for the warning… Did you ever explore putting in estimated dates for burial? I don’t have a of other types of event records that I’ve seen so far.I think I’m going to see if I can automatically do an estimate 3-7 day window for the burial. That seems to be about right for the time period. It’s not ideal, but since I’m marking it estimated, at least it will be a consistent methodology.
I put in just the year for unknown burial (month-&-year without day-of-month when the death was in the 1st half the month). It autosorts fairly well that way.
The lack of the day (or month) implies enough uncertainty that is apparent I haven’t found the funeral date yet.
The aggravating case for autosorts is for the religions that pushed for burial on same day as death. ‘burial’ alpha sorts before ‘death’, so naturally burial would float to the wrong order.
Thanks for the warning… Did you ever explore putting in estimated
dates for burial? I don’t have a of other records that I’ve seen so
far.I think I’m going to see if I can automatically do an estimate 3
I use “After…” for a burial/cremation date. This is when I know the
date of death. Even then, I have to have evidence of an interment.
From Find A Gave, or Billion Graves, for example. Otherwise, no
burial/cremation event gets entered.
You solution is the one that I finally settled on for dating the burial. Using After + Death Date, Gives me a standard answer. If I have a specific data from an Obit, then I use that. I thought a lot about using “About” or “Estimated” but that didn’t give the right picture.
The evidence that I have for burial is typically a graveyard visit or now using Find A Grave. The database that I have many times doesn’t have full locations or they have farms listed that makes documentation difficult.
One word of caution that I did run into with Find a Grave, There are graves that I know for a fact exist in specific grave yards that are not listed on the site, I’m having to figure out where the graves are to get them listed. Just a thought.
I have to agree with you on the auto sorts. After the warnings here and a little testing, I finally decided that it’s something I can fix when opening a record if I need to. That gives me exactly what I want ven though it’s not complete…
One word of caution that I did run into with Find a Grave, There are
graves that I know for a fact exist in specific grave yards that are
not listed on the site, I’m having to figure out where the graves are
to get them listed. Just a thought.
Find A Grave is far from complete, I know. Same applies to Billion
Graves and, for that matter, Deceased Online.
Why do you want to add dates or information at all to an Event where you didn’t find the information in the first place?
I never like to find any research where I find that some dates or other information is calculated or “guessed”, especially if there are no documentation that back up the information (its a lot of that stuff going on at all of the large sites)…
I always thought it was better to just leave the fields open, I never add an Event without documentation and never add a date if I dont have it in a document…
Why am I adding data that I don’t have proof for?? Basically, I need to add some structure to the data. I don’t work well without any structure, so this is to help me understand the data. My goal is to add the documentation to the work that was previously done.
What I’m doing is building in burial dates that are AFTER {Burial Date}. Since it’s a reasonable assumption that the burial date will be the day after the death date or sometime thereafter, I don’t feel like the information is inaccurate. It’s not perfect, but it gives me a way to put all of the data into buckets. 1 bucket where I have good death dates that just need to have those dates documented and a 2nd bucket of everything else.
Once I finish processing the initial list, then I’m intending to go cemetery by cemetery to pull pictures of the gravestones. That allows me to verify the data in the database and document the dates. This process also allows me to find the the places where the names of the cemeteries are shortened or maybe a little inconsistent. Again, I’m adding structure.
Eventually, several bookcases of books and binders should arrive and then I get to start reading everything to migrate the documentation into the database. I’ve got years of work in front of me….
Thanks for explaining,
I do understand the need of structuring in that case :-), I probably would have done something similar if I used CSV from a spreadsheet to import the data to Gramps, to be absolute sure I got it all, in addition to some tag’s…
I wish you luck with the all that paperwork, are you going to scan it all?
Scanning everything??? I’m not sure yet. I need to get a handle on how much can go thru an ADF to see how fast it would be. I also want to see if the text can be recognized sufficiently well to be indexed.
One thing that I have found out is that many of the pictures have EXIF notes listing the people in the pictures. That is a piece I really want to explore to add pictures to the various people.
Family Tree Processing 
.