Can't import big family tree (over 24.000 people). Error 504 in GrampsWeb

Dentistix · December 5, 2025, 6:38pm

Hello, my 94-year-old uncle gave me his “Brother’s Keeper” database. I imported it into Gramps via a GEDCOM file and repaired it. Then I exported it to a Gramps XML file (53MB file size without images). Importing this file into my self-hosted GrampsWeb repeatedly fails: Error 504.

GrampsWeb in a docker-container on my Synology NAS 218+ upgraded to 10 GB RAM

Gramps 6.0.6
Gramps Web API 3.5.0
Gramps Web Frontend 25.11.2
Gramps QL 0.4.0
Sifts 1.1.0
locale: en
multi-tree: false
task queue: false
OCR: true
chat: false

I hope someone can help me. Thank you very much!

Matthias

Sorry, as a new user, I’m not allowed to upload attachments. Here are the log data:

Gramps-Web
date,stream,content
2025/12/05 00:10:53,stderr,  warnings.warn(

2025/12/05 00:10:53,stderr,/usr/local/lib/python3.11/dist-packages/flask_limiter/\_extension.py:364: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.

2025/12/05 00:10:52,stderr,(gunicorn:29954): Gtk-CRITICAL \*\*: 23:10:52.243: gtk_icon_theme_get_for_screen: assertion ‘GDK_IS_SCREEN (screen)’ failed

2025/12/05 00:10:52,stderr,

2025/12/05 00:10:48,stderr,\[2025-12-04 23:10:48 +0000\] \[29954\] \[INFO\] Booting worker with pid: 29954

2025/12/05 00:10:48,stderr,\[2025-12-04 23:10:48 +0000\] \[27\] \[ERROR\] Worker (pid:32) was sent SIGKILL! Perhaps out of memory?

2025/12/05 00:10:47,stderr,\[2025-12-04 23:10:47 +0000\] \[32\] \[INFO\] Worker exiting (pid: 32)

2025/12/05 00:10:47,stderr,SystemExit: 1

2025/12/05 00:10:47,stderr,    sys.exit(1)

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/base.py", line 204, in handle_abort
"
2025/12/05 00:10:47,stderr,"    self.\__cursor.execute(\*args, \*\*kwargs)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/sqlite.py", line 138, in execute
"
2025/12/05 00:10:47,stderr,    self.dbapi.execute(

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/dbapi.py", line 1199, in \_update_secondary_values
"
2025/12/05 00:10:47,stderr,    self.\_update_secondary_values(obj)

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/dbapi.py", line 695, in \_commit_base
"
2025/12/05 00:10:47,stderr,               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    old_data = self.\_commit_base(person, PERSON_KEY, transaction, change_time)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/generic.py", line 2048, in commit_person
"
2025/12/05 00:10:47,stderr,"    commit_func(obj, trans)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/generic.py", line 1944, in \_add_base
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,    return self.\_add_base(

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/generic.py", line 1948, in add_person
"
2025/12/05 00:10:47,stderr,"    add_func(prim_obj, self.trans, set_gid=False)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/importer/importxml.py", line 897, in inaugurate
"
2025/12/05 00:10:47,stderr,             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    handle = self.inaugurate(attrs\["hlink"\], "person", Person)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/importer/importxml.py", line 1733, in start_personref
"
2025/12/05 00:10:47,stderr,    f(attrs)

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/importer/importxml.py", line 3364, in startElement
"
2025/12/05 00:10:47,stderr,"  File "../Modules/pyexpat.c", line 416, in StartElement
"
2025/12/05 00:10:47,stderr,    self.p.ParseFile(ifile)

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/importer/importxml.py", line 1064, in parse
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    info = parser.parse(xml_file, line_cnt, person_cnt)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/importer/importxml.py", line 204, in importData
"
2025/12/05 00:10:47,stderr,              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    results = import_function(db, filename, user)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/utils.py", line 179, in import_from_filename
"
2025/12/05 00:10:47,stderr,             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    status = import_from_filename(db, filename, user)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/utils.py", line 161, in import_as_dict
"
2025/12/05 00:10:47,stderr,                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    db_handle: DbReadBase = import_as_dict(filename=file_name, user=User())
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 1431, in dry_run_import
"
2025/12/05 00:10:47,stderr,                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,    object_counts = dry_run_import(file_name=file_name)

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 239, in import_file
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    return self.run(\*args, \*\*kwargs)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/util/celery.py", line 18, in **call**
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    return self.\_get_current_object()(\*a, \*\*kw)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/celery/local.py", line 182, in **call**
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,    return task(\*\*kwargs)

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 80, in run_task
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^

2025/12/05 00:10:47,stderr,    task = run_task(

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/importers.py", line 100, in post
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    return func(\*args, \*\*kwargs)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/webargs/core.py", line 652, in wrapper
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,    return current_app.ensure_sync(meth)(\*\*kwargs)  # type: ignore\[no-any-return\]

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 191, in dispatch_request
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,    return current_app.ensure_sync(self.dispatch_request)(\*\*kwargs)  # type: ignore\[no-any-return\]

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 110, in view
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    return func(\*args, \*\*kwargs)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/auth.py", line 44, in wrapper
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,    return self.ensure_sync(self.view_functions\[rule.endpoint\])(\*\*view_args)  # type: ignore\[no-any-return\]

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 902, in dispatch_request
"
2025/12/05 00:10:47,stderr,         ^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,    rv = self.dispatch_request()

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 917, in full_dispatch_request
"
2025/12/05 00:10:47,stderr,               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,    response = self.full_dispatch_request()

2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1511, in wsgi_app
"
2025/12/05 00:10:47,stderr,           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    return self.wsgi_app(environ, start_response)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1536, in **call**
"
2025/12/05 00:10:47,stderr,               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2025/12/05 00:10:47,stderr,"    respiter = self.wsgi(environ, resp.start_response)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 177, in handle_request
"
2025/12/05 00:10:47,stderr,"    self.handle_request(listener, req, client, addr)
"
2025/12/05 00:10:47,stderr,"  File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 134, in handle
"
2025/12/05 00:10:47,stderr,Traceback (most recent call last):

2025/12/05 00:10:47,stderr,\[2025-12-04 23:10:46 +0000\] \[32\] \[ERROR\] Error handling request /api/importers/gramps/file

2025/12/05 00:10:46,stderr,\[2025-12-04 23:10:46 +0000\] \[27\] \[CRITICAL\] WORKER TIMEOUT (pid:32)

emyoulation · December 5, 2025, 6:45pm

Your “trust level” has been increased from new to basic.

dsblank · December 5, 2025, 7:46pm

Perhaps you can increase the Worker Timeout value? [I don’t know how]

If that fails, maybe try exporting different segments (ancestors of X, descendants of X) of the tree to get a few smaller XML files, and then import each of those.

DavidMStraub · December 5, 2025, 8:44pm

Hi!

that’s the problem. Long-running tasks like imports don’t work without a task queue configured. How did you install Gramps Web? All our examples in the official documentation have a task queue.

Please make sure you set up Gramps Web according to the official docs and if the problem persists, please share

your Docker Compose file (with secrets redacted)
your Celery log

Good luck!

emyoulation · December 5, 2025, 9:30pm

When reading this thread and the experiment that @jj0192 is doing (writing an external vb tool to parsing Gramps XML and directly write a SQLite db), the thought occurs that maybe doing this import on the server might not be the best use of server time. For large trees, maybe the PostgreSQL database should be built offline and ftp uploaded, then swapped out.

Key points of Jeff’s experiments:

4 days to import a a 2 million person .ged GEDCOM in Gramps for desktops.
7 hours with Python to import a 2 million person .gramps XML in Gramps for desktops.
16 minutes for his vb tool to parse the XML and wite the database.

Jeff mentioned how long his GECOM vb parser took took when converting to XML. But cannot seem to find that posting. But it was a whole lot less than 4 days.

DavidMStraub · December 6, 2025, 8:19am

Ok but this is off topic.

There are many ways to manually get the data to the server. Upload the SQLite, import XML from the terminal, etc. etc. - but all of those are more complicated. The server also needs to update the search index, which would have to be done manually as well.

So, let’s please stick to the officially documented and easiest solution.

Dentistix · December 6, 2025, 10:22am

Thank you so much for your quick replies! One possible problem is that I didn’t follow your installation instructions exactly. Sorry about that! I’m getting more comfortable with the command line, but I installed Grampsweb via Portainer following the instructions from mariushosting: How to Install Gramps Web on Your Synology NAS – Marius Hosting
See the attached configuration file:

grampsweb-yml.pdf (42,8 KB)

That’s why I don’t have a “grampsweb_celery” container.

The following command doesn’t work on my Synology NAS:

sudo docker compose logs --tail 100 Gramps-Web
no configuration file provided: not found

Only this command works:

sudo docker logs --tail 100 Gramps-Web

“lscpu” is not installed, but my CPU has two cores.

lscpu
sh: lscpu: command not found

Should I add “GUNICORN_NUM_WORKERS: 2” to the configuration file?

free -h

Total used free shared buff/cache available
Mem: 9.6Gi 3.5Gi 241Mi 180Mi 5.9Gi 5.6Gi
Swap: 7.7Gi 302Mi 7.5Gi

I also found the following line:

CMD /bin/sh -c gunicorn -w ${GUNICORN_NUM_WORKERS:-8} -b 0.0.0.0:5000 gramps_webapi.wsgi:app --timeout ${GUNICORN_TIMEOUT:-120} --limit-request-line 8190

Could the timeout be increased here?

You wrote me: “task queue: false”

How can I set this to “true”?

What should I do now?

Thank you very much!

Dentistix · December 20, 2025, 4:36pm

Update

I updated the YAML file with “celery” (see attachment) and reinstalled the server. Importing my database took a while, but it was successful. After that, I uploaded the media files in the ZIP file. The database check was also successful. - Great!

Unfortunately, synchronizing the identical databases with “Gramps Web Sync” didn’t work. I suspect another timeout error: 504 (see attachment). I’ve attached logs from grampsweb and grampsweb-celery again.

Can anyone help? - Thank you.

my_grampsweb_with_celery-YML.pdf (49,3 KB)

grampsweb_logs.pdf (48,4 KB)

grampsweb_celery_logs.pdf (57,2 KB)

Topic		Replies	Views
Gramps-Web freezing and giving errors 502 and 404 Gramps Web	2	37	December 19, 2025
Failure by Import Gramps Web	2	106	May 1, 2025
System Requirements / Options to handle big family tree Help third-party-addon , gedcom , data-import	4	88	March 20, 2025
Gramps Web & Gramps 6.0: Beta Testers Wanted Gramps Web	62	571	May 4, 2025
Gramps Web Docker's First Run wizard Gramps Web	10	1178	June 22, 2023

Can't import big family tree (over 24.000 people). Error 504 in GrampsWeb

Related topics