Corrupted database?

Gramps Web
1

I haven’t touched my gramps installation for about a year. Yesterday I started looking back at both the Docker installation of Gramps Web, and Gramps Desktop, and noticed that Gramps Web had updated to v5.2.2 while my Gramps Desktop was still at v5.1. So I tried reinstalling Gramps Desktop to the newer v5.2.3, and I had to reinstall the plugins. But I’m not able to access any of the Family Trees, I have no Tools menu and I can’t seem to figure out how to synch from remote to local.

So I thought I would try to export data from Gramps Web, but the export wasn’t working and looking at Chrome Developer Tools I saw that all API endpoints seemed to be erroring out. So I thought, maybe I should restart the docker container. But on restarting the container it seems to not find the database anymore, and launched the firstrun screen. Only way I could get in was to recreate my user account, but on entering with the new account it looks empty.

I inspected the docker container with docker exec -it grampweb sh, and it seems like the database is still there, in /root/.gramps I see:

drwxr-xr-x 1 root root 4096 Apr 28 11:39 gramps52
drwxr-xr-x 3 root root 4096 Aug 28 19:10 grampsdb
-rw-r--r-- 1 root root  266 Aug 28 19:20 recent-files-gramps.xml
And `recent-files-gramps.xml` has: 
<?xml version="1.0" encoding="utf-8"?>
<RecentFiles>
  <RecentItem>
    <Path><![CDATA[/root/.gramps/grampsdb/257944e6-d772-424c-b37d-e87d34935d59]]></Path>
    <Name><![CDATA[Gramps Web]]></Name>
    <Timestamp>1724872800</Timestamp>
  </RecentItem>
</RecentFiles>

The grampsdb/257944e6-d772-424c-b37d-e87d34935d59 directory has:

-rw-r--r-- 1 root root      6 Aug 28 19:10 database.txt
-rw-r--r-- 1 root root     10 Aug 28 19:10 name.txt
-rw-r--r-- 1 root root 196608 Aug 28 19:10 sqlite.db
-rw-r--r-- 1 root root  49152 Aug 28 19:10 undo.db
Which seems to me like there is quite a bit of information in the database.

If I inspect the docker logs I see:

INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> c89728e71264, empty message
INFO  [alembic.runtime.migration] Running upgrade c89728e71264 -> e5e738d09fa7, Added configuration table
INFO  [alembic.runtime.migration] Running upgrade e5e738d09fa7 -> e176543c72a8, Add tree column to User
INFO  [alembic.runtime.migration] Running upgrade e176543c72a8 -> 66e56620891a, Add trees table
INFO  [alembic.runtime.migration] Running upgrade 66e56620891a -> 22c8d1fba959, Add trees.enabled
INFO  [alembic.runtime.migration] Running upgrade 22c8d1fba959 -> b0582f54029c, Use BigInt for usage_media
INFO  [alembic.runtime.migration] Running upgrade b0582f54029c -> 84960b7d968c, Use BigInt for quota_media
[2024-08-28 19:38:10 +0000] [35] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 134, in handle
    req = next(parser)
          ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/http/parser.py", line 42, in __next__
    self.mesg = self.mesg_class(self.cfg, self.unreader, self.source_addr, self.req_count)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/http/message.py", line 257, in __init__
    super().__init__(cfg, unreader, peer_addr)
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/http/message.py", line 60, in __init__
    unused = self.parse(self.unreader)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/http/message.py", line 281, in parse
    self.parse_request_line(line)
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/http/message.py", line 445, in parse_request_line
    raise InvalidHTTPVersion(self.version)
gunicorn.http.errors.InvalidHTTPVersion: <exception str() failed>
During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/local/lib/python3.11/dist-packages/gunicorn/arbiter.py", line 609, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/base.py", line 142, in init_process
    self.run()
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 126, in run
    self.run_for_one(timeout)
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 70, in run_for_one
    self.accept(listener)
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 32, in accept
    self.handle(listener, client, addr)
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 158, in handle
    self.handle_error(req, client, addr, e)
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/base.py", line 225, in handle_error
    mesg = "Invalid HTTP Version '%s'" % str(exc)
                                         ^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gunicorn/http/errors.py", line 56, in __str__
    return "Invalid HTTP Version: %r" % self.version
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
TypeError: not all arguments converted during string formatting
[2024-08-28 19:38:10 +0000] [35] [INFO] Worker exiting (pid: 35)
[2024-08-28 19:38:10 +0000] [18] [ERROR] Worker (pid:35) exited with code 255
[2024-08-28 19:38:10 +0000] [18] [ERROR] Worker (pid:35) exited with code 255.
Where can I go from here to get my database restored?
2

On the new version installation on the Desktop side with no Trees or Toolbar:

Start by searching your system for the grampsdb folder.

image
image898×648 68.8 KB

Your Edit → Preferences might have had the Database path changed in the Family Tree tab.

image
image970×754 54.4 KB

1 Like
3

Hi,

that gunicorn error is weird, it seems to be a bug in gunicorn itself that was fixed recently: InvalidHTTPVersion / TypeError: not all arguments converted during string formatting · Issue #3195 · benoitc/gunicorn · GitHub

I’m wondering what you mean by “Gramps Web had updated” - did you configure it to somehow auto-update? The upgrade of Gramps Web from Gramps 5.1 to 5.2 is not an automated process, see 2.0 upgrade guide - Gramps Web.

In any case, no data is lost, so please don’t panic and let’s get it sorted step by step :slight_smile:

4

Every now and then when I open Gramps Web I get a notification along the lines of “there is an update available, would you like to update now?”, so it would seem that there is some kind of auto-update mechanism?

5

I have a local gramps file on my Desktop that I was able to right-click → open in gramps. So now I seem to have the local Desktop data back again, and I see the Tools menu item. When I go to Gramps Web sync, I enter the credentials that I just re-created for Gramps Web, and it continues to “comparing local and remote”. After a few minutes I get the result that all data is local (some 10,000 records or so) and there is no remote data. But when I try to continue the synchronization I get the error that I am unauthenticated?

I retried the process, and now it says Remote changes: 12000 deleted. Continue?

Of course I am quite hesitant about continuing now. Is this going to delete everything locally and remotely?

1 Like
6

I find it easier to manage docker images by version number rather than just using “latest”, so I downed the docker container again from my Plesk interface, I pulled v24.8.0 from the command line and then upped the container again from the Plesk interface. I again get the firstrun screen but I also got the notification about an updated app, here is a screenshot:

image
image1532×847 47.3 KB

See the notification in the bottom left corner “There is a new version of the app available. REFRESH”.

7

In any case I proceeded with Gramps Web Sync and chose “reset remote to local”. That way I can be sure that no local data is deleted by the remote. Am now attempting to proceed to sync media files… But I got a 500 error argh. And when opening the website I still don’t see any data.

8

I found the folder on my Windows system at C:\Users\username\AppData\Roaming\gramps\grampsdb. I set this in the Gramps interface and now I have the previous databases back again. The one that was sync’d to Gramps Web had a lock on it, I unlocked it and was able to load it again.

From there I went back to Gramps Web Sync, but I still don’t like the looks of the results:
image
It would seem like it prefers the remote data over the local data, and the remote data just wants to delete everything.

Any pointers on how I should proceed from here?

9

Gramps Web Sync is black magic to me. David will have help you with that. But I’m VERY glad to see your database path was recoverable on the Windows desktop box!

There was a recent change to support XDG base directory specification. For MS Windows users, this included moving the Gramps User Directory from %AppData% to %LocalAppData%.

Legacy installations will try to preserve the old path rather than move everything to the new target. I suspect that that policy of supporting 2 locations will develop into a support nightmare with endless quirks. But I’m a pessimist.

See

.

10

I tried resetting Gramps Web to a clean slate once again. Then I exported a new Gramps file from my recovered Desktop database. Now I’m trying to import that Gramps file to Gramps web, but it doesn’t seem to be making any progress… It just keeps spinning. I checked the browser localstorage and found a few keys with tasks that were probably from a previous attempt, so I deleted all of the localstorage keys, logged in again, and attempted again to import the Gramps file, but still no luck, it just keeps spinning next to the “import” button.

I don’t see any messages in the GrampsWeb docker logs. I tried restarting the Redis container also just for good measure, but it seems like the last messages in those logs go back to June 2024? Something seems not quite right with Gramps Web…

11

Ok well sorry for the trouble, apparently the Celery container hadn’t restarted with the Gramps Web and Redis containers. Once I got the Celery container up and running I was able to correctly import the last Gramps Web desktop export. I believe I’m up and running again.

1 Like
12

No, that’s just the browser prompting you to reload the frontend when an updated version is available on your server, so that happens after updating the image.

1 Like
13

Great to hear you’re back up and running again. As a general rule, if the sync addon shows thousands of changes while you expect the databases to be similar, never click apply!

After a clean new start, it’s also safer/better to start with an XML import rather than a “reset to local” sync.

14

I have submitted a PR to the gramps-web-api repository, because I found out that every time I issue docker compose down all volumes are removed. So they aren’t really persistent, they only persist between stopping and restarting. The PR gives an alternative solution of creating the volumes ahead of time and setting the volumes in the docker-compose.yml to external so that they will not be removed with docker compose down. I think now that I have done this I will have a much more stable situation with my production instance.