Database cannot be repaired after system crash

Help
1

My laptop shut down on its own overnight and now I can’t open my family tree db in Gramps (5.1.2, Linux).
If I try to open it, gramps sees the db as locked. I break the lock, and then when I try to open it the error I see is (apologies in advance for any typos in this, I can’t seem to copy the text from the error window)


Cannot open database
Gramps has detected a problem in opening the 'environment' of the underlying Berkeley database used to store this Family Tree.  The most likely cause is that the database was created with an old version of the Berkeley database program, and you are now using a new version.  It is quite likely that your database has not been changed by Gramps.
If possible you should revert to your old version of Gramps and its support software; export your database to XML; close the database; then upgrade again to this version of Gramps and import the XML file in an empty Family Tree.  Alternatively, it may be possible to use the Berkeley database recovery tools.

(-30973, 'BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery -- BDB1544 process-private: unable to find environment')

Notably, when I try to open my databse, the “repair” button is greyed out, so that is apparently not an option for me. I’ve tried using db5.3_recover from the command line. If I run that without “-c” the return looks like


db5.3_recover: BDB0110 Log sequence error: page LSN 1 375875; previous LSN 1 1778303
db5.3_recover: BDB1520 Recovery function for LSN 1 1778381 failed on forward pass
db5.3_recover: BDB0061 PANIC: Invalid argument
db5.3_recover: BDB0060 PANIC: fatal region error detected; run recovery
db5.3_recover: BDB3015 ~/.gramps/grampsdb/5e495d95/referenced_map.db: write failed for page 2
db5.3_recover: BDB3027 ~/.gramps/grampsdb/5e495d95/referenced_map.db: unable to flush page: 2
db5.3_recover: BDB1544 process-private: unable to find environment
db5.3_recover: DB_ENV->open: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

I also tried exporting the database using by running gramps from the command line:


$ gramps -u --open="Family Tree 1" --export=full-dump.gramps
/usr/lib/python3/dist-packages/gramps/gen/plug/_pluginreg.py:1138: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if PluginRegister.__instance is not 1:
/usr/lib/python3/dist-packages/gramps/gen/plug/_manager.py:86: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if BasePluginManager.__instance is not 1:
/usr/lib/python3/dist-packages/gramps/gui/pluginmanager.py:74: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if GuiPluginManager.__instance is not 1:
/usr/lib/python3/dist-packages/gramps/plugins/export/exportvcalendar.py:205: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if date_string is not "":
2022-10-12 20:37:27.269: WARNING: write.py: line 634: Error opening db environment: (-30973, 'BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery -- BDB1544 process-private: unable to find environment')
ERROR: Gramps has detected a problem in opening the 'environment' of the underlying Berkeley database used to store this Family Tree. The most likely cause is that the database was created with an old version of the Berkeley database program, and you are now using a new version. It is quite likely that your database has not been changed by Gramps.
If possible, you should revert to your old version of Gramps and its support software; export your database to XML; close the database; then upgrade again to this version of Gramps and import the XML file in an empty Family Tree. Alternatively, it may be possible to use the Berkeley database recovery tools.

(-30973, 'BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery -- BDB1544 process-private: unable to find environment')
Error opening the file.
Exiting...

It seems that I can’t get through this “environment” error, but I have no idea where it came from. Any suggestions would be appreciated.

2

1st things first. It is important to be aware that trying to open a database can MODIFY the database. That’s just the nature of database engines. (You’re probably aware of that already … but it has to be said.)

So, at the earliest stage that you realize you might have a problem, make a duplicate of the database files (preferably 2 copies, one on a thumb drive or other external storage). Then focus your recovery attempts on the duplicate. Leave the original alone as much as possible.

In an ideal world, you already had Gramps set to make backup on Exit. So the corruption of the database is just a prompt to create a fresh tree & import the latest backup.

3

If the computer had crashed it doesn’t have done any backup. That’s why I use automatic backups every hour too (and because I don’t exit often from Gramps)

1 Like
4

So, at the earliest stage that you realize you might have a problem, make a duplicate of the database files (preferably 2 copies, one on a thumb drive or other external storage). Then focus your recovery attempts on the duplicate. Leave the original alone as much as possible.

I did make a duplicate of the corrupted database, and I’ve been working on that duplicate. Unfortunately nothing I do with it results in a database I can load back in to Gramps.

In an ideal world, you already had Gramps set to make backup on Exit. So the corruption of the database is just a prompt to create a fresh tree & import the latest backup.

Unless that was a default setting in Gramps to do that, I did not have it set to do that. Being as the computer crashed (actually from /var/log it looks like it exceeded a temperature threshold and shut itself down without warning) with Gramps open there wouldn’t have been an exit point to make a backup from.

5

Backup upon exit is not the default. (That’s probably a good refinement for a future version.)

My recovery suggestion is somewhat different that what’s been assumed. A crash in the middle of a session would mean losing the day’s work if you restore the backup. Unpleasant but survivable.

Note that interval backup (Autobackup) is a recent addition to Gramps. But the intervals are too short (all an hour or less) for users with big trees. There’s an enhancement patch to add 12hr & 24hr to the interval list. That will be rolled in for the 5.2 version.

6

If you don’t have a recent backup; I suggest following the rest of the advice from the wiki and start a ticket on the Gramps bug tracker. make it private and upload your files and hopefully one of the developers can see if they can recover anything?

After recovery you might want to migrate to SQLite instead of using the old BSDDB format databases.

1 Like
7

I don’t want to migrate to a SQlite database, so
I often use the following when this problem occurs.

make a copy of your database: (5af54488 is your database directory to recover)
mv 5af54488 5af54488sav # Save your database
mkdir 5af54488 # Create a new database directory
cd 5af54488
cp …/5af54488sav/*.db . # copy all your db files in the new directory
for f in *.db; do db_dump -k $f >$f.dump.txt; done
for f in *.db; do db_load -n $f <$f.dump.txt; done
for f in *.db; do db_load -r lsn $f <$f.dump.txt; done

It should work.

1 Like
8

I didn’t notice this before but there are a few “.gramps” files in my home directory! I just loaded the youngest of them using the command prompt and that got most of my data back. Much better than having to rebuild it all.

1 Like
9

I often use the following when this problem occurs.

Thank you for the suggestion; I was hoping there would be a command line option to recover the database. Unfortunately it didn’t work for me. The db_dump command produced quite a few “unknown type” and “invalid argument” responses, and the db_load of after that of course didn’t do much either. The txt files had some meaningful data in them but apparently not enough for db_load to do anything useful.

10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.