I seem to have run Check and Repair a few times over the years and each time where some kind of error has been found by the tool it seems to have dumped all instances into a General Note. The details recorded in the Notes are all of type Event and all but one Birth (I have one Christening). My most recent such note only has 1 entry but an earlier one about 60.
Since the note says “Objects referenced by this note …” I’m guessing that Media is involved (so in my case static images, all Type image/jpeg or image/png, apart from a few image/gif.
Such notes seem surplus to my requirements, hence I would like to delete them. However in attempting to do so I am presented with the threat:
This item is currently being used. Deleting it will remove it from the database and from all other items that reference it. The data can only be recovered by Undo operation or by quitting with abandoning changes.
“Objects referenced by this note were referenced but missing so that is why they have been created when you ran Check and Repair”. If an image was missing at the time of running Check and Repair for that tool to create the image seems to me a tall order (though perhaps it has found files with the expected checksums?).
I suspect the underlying cause may be my merging of duplicates. If say Person A had two records, one with an associated image and one without, and these were merged with the “without” favoured then perhaps the system would know where to find the relevant image. But would it still be able to if I deleted the Notes?
Put another way, what may be the consequences of deleting such Notes and how to deal with them?
I consider those markers to be ”ToDo" placeholders … indicating datapoints that need to be manually compared again against the original data source.
(It is worth running an Event custom filter for events with no references. There’s a possibility that the event had been created but Gramps was interrupted before linking it. That can happen with an abnormal exit from the application; or, if you close the parent {Person or Family} dialog before the spawned child {Event} dialog. Gramps WARNS when that happens and offers to save the spawned data… but forgets to actually link that saved data.)
But the missing media objects might have simply been moved in the OS, outside of Gramps. There are tools specifically for Gramps that locate moved files and fix the link.
I have a lot of Notes (created by me) as “ToDo” placeholders, so I have no problem with ‘the system’ doing much the same. At present however I am unaware of what action is required from me. I’ve checked one run of Check and Repair that seems to have thrown three warnings. In one case I have an image and it is the one I expected. In the other two cases I do not have an image – but wasn’t expecting one. Everything seems in order.
If I go: Edit > Event Filter Editor > + > add “EFE” for Name > + > General filters > + > Events with a reference count of > Reference “0” (default?) then to Events and EFE in Custom filter and Find, nothing is selected.
These added notes, “Objects referenced by this note …”, will not show up when Reference=0 because they ARE attached to other objects.
Open the note and check its References tab. This will tell you what the note is attached to. This is the object that the C&R could not find so created it anew and attached the note. You need to open this object to find what it is attached to in its References tab.
I suspect that the user deletes an object where a tidbit of the code is left behind. The C&R finds the tidbit of code but cannot find the object that created it. You need to navigate through the References tabs to find the Person (as an example) to investigate what and why it was missing.
Once you resolve the issue you can delete the note. Once done you can use the Delete Unused Object tool or do the filter for References = 0. And to confirm the issues are solved, run the C&R again.
Note1: After running the C&R, the ultimate offending objects will be newly recreated so can be found by their creation dates. Remember to search all areas of the database.
Note 2: Valid notes attached to the ToDo gramplet only (no other object) will not have a Reference count so will return in a filter References = 0. This also applies to notes created to be used by the NarrWeb report. So be careful.
The reference count rule is to look at something different than the Notes generated by the tool.
What the Custom Filter is seeking are the records that might have been orphaned by a partially completed transaction. The assumption is that the Check and Repair tool finds the 1st incomplete half. The custom filter might find the OTHER half.
Here’s the rule that I suggested (displayed in the Isotammi experimental FilterParams tool). It looks for count of less than 1. (Which SHOULD catch corrupt values like negative and NULLs, not JUST zeros.)
For my Tree of nearly 100,000 events, this filter finds a little over 300 orphaned events. Many of these are events that were unlinked (inappropriately described as ‘deleted’) from People or Families but were NOT truly deleted … as in removed from the Events table.
I have clicked on one of these Notes (same one as above with three ‘errors’ – it is not the most recent) and then on References at the top of the pop-up window. This shows me three Events (E series, not O series). I can click on each of these individually and they all call up a Birth Event. Clicking on References (near the middle of the new window) brings up (at the bottom of this window) one entry each for a Type (Person) with ID (I series) and Name. I can find each of the three IDs in a search of People, and for each the chart for them is as I would expect (i.e. they exist, are displayed, and are linked to others as I would expect).
I’m wondering. Should perhaps the C&R process include removing any of its Notes generated by a previous C&R run? That is, perhaps only the latest such Note is relevant (because everything else has been fixed or will show up on the most recent run?)
I’m also wondering whether “Objects referenced by this note…” should really be “Objects” (as in the O series) – perhaps “Items” or “Data” would be clearer, if seemingly not necessarily Media? I’m fairly certain two of my three example ‘errors’ never had any associated Media.
Also, I’ve run most of the Processing and Repair Tools since some of these “ToDo” Notes were created by the system. Might that be confusing matters?
I hope I don’t seem ungrateful, but I have not installed the Isotammi experimental FilterParams tool because about half of the built-in features still don’t make sense to me and I have mostly given up with the documentation (for those I manage without) so hesitate to take on what would just lead to more bewilderment. However, I am now curious to see what happens for me when running your filter and I do plan to give it a go at some point, though preferably after it has lost the “experimental”.
Ah! Progress. I changed my “EFE” spec to “less than 1” (rather than “0”) and three Events have appeared. These seem to be totally unrelated to the three “errors” found in my C&R example. I plan to investigate them, but may need some time to do so.
A matter on nomenclature. A macro Object in Gramps are the records that can be shared with other objects. People, Families, Places, Events, etc. The components of the Gramps database. Not to be confused with Media objects, although Media records are Gramps Objects. One of my first changes I made was to change the leading letter from O to M for the Media category.
I try to run C&R at least once a month. If any errors show up, hopefully there are only one or two and I try to resolve them at that time. Frequent use of the tool allows me to have a chance to remember what I was doing with those records.
Likewise, I run the Remove Unused Objects tool (macro Objects) at least once a week first for Events, then for Citations and finally for Notes. When items show up in the list to delete, I again have a chance to remember why I am deleting them.
The only time the “Objects referenced by this note…” should be removed from the offending object is when the user has resolved the issue. Remember, it is the record the has the note that is the problem, not the note Gramps created.
My most frequent occurrence are for events that I have deleted that the C&R finds as a problem. A new event is created with the attached note. The only part of the event that gets created is the event’s Type. No date, no place. This does not happen often but when the C&R finds a problem it usually is an event. Gramps has resolved the issue having ‘fixed’ the database with the C&R. But it takes action by the user to resolve the issue in their genealogy.
Once I have resolved the issues, I immediately run the C&R again to confirm no ongoing issues. A clean bill of health.
Excellent idea. I wish I had done so before today.
Sounds somewhat dismissive. However in documentation I think nomenclature is very important, for clarity. My confusion would certainly have been reduced if the list in the Preferences > Gramps ID format settings had either included “Object” after ‘Person’, ‘Family’, ‘Place’, etc. or, preferably, excluded “Object” from ‘Media Object’.
Excellent advice.
I now run it often enough that I can have all the search options ticked at the same time. Though before having a feel for what it was doing I ran them individually because to begin with I had a lot of hits. Just now I only had five, three of which are the three that showed up with my “EFE” filter spec set at “less than 1”. I deleted all five Unused Objects and reran C&R. Still one hit from that but I have deleted all the other system “ToDo” General Notes and hopefully can deal with my last remaining current anomaly.
Still don’t now what I am doing, or why, but the results seem satisfactory.
… until I deleted that one remaining Unused Object and reran C&R. I now have 97 Events to puzzle over.
… or rather, to delete. Looks as though all 97 are Events with a Type but no other details. For example, I tried to force the “/” in the top left in some boxes in Charts (for some people I knew were dead but for whom I had no date or place of death), by adding just an Event Death for them. Bad move. I will now go through and delete all these ‘Type only’ entries.
I deleted about 90 of the 97 by finding the associated Reference (only ever one per event, or perhaps none for the remaining 7 or so), double clicking on it, then deleting whichever Event showed up (usually Birth) without any details added by me. Took me three or four attempts as Gramps crashed three or four times.
Then I ran Remove Unused Objects. About half a dozen showed up (the ‘remaining 7 or so’ mentioned above perhaps?) which I deleted. Then reran C&R. This time 92 showed up.
In ‘deleting’ (with the “-” icon) the Referenced Events I had been observing that the entry disappeared from the screen and then gone straight on to delete the next. It seems I should have been clicking “OK” twice before doing so. At least similar to emyoulation’s mention:
“deletion” as I was doing just added the Events to the Unused Objects list. So I also deleted what showed up there and it seems this has worked for two sample batches of ten each. Also rerunning C&R (which returns “No errors were found”) after a couple of batches of ten seems to have shortened the list (without changing the timestamp in the system generated “ToDo” General Note).
Next I ‘deleted’ 20 (with “OK” twice), deleted Unused Objects (there were 40), deleted the last remaining system generated “ToDo” General Note (I’d deleted the rest earlier) and reran C&R. This time Integrity Check Results came up with 52 “events were references, but not found”. So I deleted those, deleted Unused Events, deleted the “ToDo” General Note and reran C&R.
It has taken thousands of clicks and has been very much less intuitive for me that the rest of Gramps that I use, but the result is what I was after.
Many thanks to both DaveSch and emyoulation who helped me on my way.
I hesitate to mention this, after you did so much manual clicking…
But if you had a custom filter that already showed all the events/notes to be deleted in their Category View, an extended selection in that view would allow those items to be deleted en masse. This extended deletion would’ve popped a confirmation dialog that offers to apply the action to the rest of the selection.
Deleting from a Category view (for that type of record) is a true delete, not just the unlink of the Object Editors (or Relationship category view… since relationships are the complex pattern of inter-linkings between records). Deleting from the object’s category simultaneously unlinks as it deletes.
”en masse" operations are often a mistake that you will regret almost immediately. Make a backup so that you can undo.
Note: There IS an extraneous refresh (removed for the forthcoming 5.2 version) between each deletion. So it takes about a second per delete instead about a second to do them all.
That could be very useful to know for another occasion, either for someone else or for me if I have not learned my lesson. But for the moment I’ve had enough of merely trying to remove four or five system-generated Notes - and of course since I think I was successful, at present I have no Objects requiring deletion to test with.
Also, now that I have some idea of the consequences, I’ll cease to use a Birth Event with no detail as indication that I have failed, but have already tried, finding the relevant place and date of birth.
Separately, I guess I have auto-updates turned off and am due a newer version. Thanks again!
The Gramps Check and Repair tool is intended to clean up databases that have accumulated errors caused by actual BUGs in Gramps. In this case, some early versions of Gramps did not do a very good job of removing or repairing internal cross references to objects during delete or merge operations. When the bug was recognized, the tool was updated to ‘fix’ databases where the bug left corrupted/dangling references.
We think this class of bugs has been corrected in current versions of Gramps, so it is generally ok to simply delete the objects that these notes refer to. The references will be removed and your database should be fine. This presumes that the original delete/merge really did intend to remove the original missing object…
I think that is what I found (i.e. all I did was delete Events that I seem to be managing perfectly without). But the warning message was scary and my long-winded (though straightforward) approach to deletion took thousands of clicks. I suggest there is scope for some clarification (of what C&R aims for and how to achieve ‘tidiness’ efficiently) Perhaps it would help to suggest that Objects without references are dealt with before running C&R?
… or Maybe make Remove Objects without References a separate Tool?
Perhaps it would help to suggest that Objects without references are dealt with before running C&R?