Having trouble with exporting data

Zelion.exe · September 29, 2024, 9:29pm

I have built my family tree, but since I put a lot of work in it, I wanted to take a backup. So I went to “Export” and then “Generate” and a scrolling wheel appeared next to it, and it just kept loading and loading. Same happens with exporting media files fyi.

So now I’m sitting here waiting for 10-15min already. I’ve tried several browsers, restarted the docker container. But nothing seems to work. Here is my setup: I’m running Gramps 5.2.2 in docker. Here is the docker compose file I use (i use redis in another stack on the same network, i just left the network portion out of the compose file):

services:
  grampsweb: &grampsweb
    image: ghcr.io/gramps-project/grampsweb:latest
    container_name: grampsweb
    restart: unless-stopped
    environment:
      GRAMPSWEB_TREE: <my last name>
      GRAMPSWEB_CELERY_CONFIG__broker_url: "redis://redis:6379/0"
      GRAMPSWEB_CELERY_CONFIG__result_backend: "redis://redis:6379/0"
      GRAMPSWEB_RATELIMIT_STORAGE_URI: redis://redis:6379/1
    volumes:
      - grampsweb-users:/app/users
      - grampsweb-index:/app/indexdir
      - grampsweb-thumbnail-cache:/app/thumbnail_cache
      - grampsweb-cache:/app/cache
      - grampsweb-secret:/app/secret
      - grampsweb-db:/root/.gramps/grampsdb
      - grampsweb-media:/app/media
      - grampsweb-temp:/tmp

  grampsweb_celery:
    <<: *grampsweb  # YAML merge key copying the entire grampsweb service config
    ports: []
    container_name: grampsweb_celery
    command: celery -A gramps_webapi.celery worker --loglevel=INFO

Thank you in advance!

DavidMStraub · September 30, 2024, 6:05am

Hi, the first thing to check is the log of the celery container after you click export. Please post it here. Usually, Gramps XML export shouldn’t take longer than a couple of seconds.

Zelion.exe · September 30, 2024, 6:57am

Hi David, thank you for your response. Here is the log of the celery container:

  . gramps_webapi.api.tasks.export_db
  . gramps_webapi.api.tasks.export_media
  . gramps_webapi.api.tasks.generate_report
  . gramps_webapi.api.tasks.import_file
  . gramps_webapi.api.tasks.import_media_archive
  . gramps_webapi.api.tasks.media_ocr
  . gramps_webapi.api.tasks.search_reindex_full
  . gramps_webapi.api.tasks.search_reindex_incremental
  . gramps_webapi.api.tasks.send_email_confirm_email
  . gramps_webapi.api.tasks.send_email_new_user
  . gramps_webapi.api.tasks.send_email_reset_password
  . gramps_webapi.api.tasks.upgrade_database_schema
[2024-09-30 06:55:21,690: WARNING/MainProcess] /usr/local/lib/python3.11/dist-packages/celery/worker/consumer/consumer.py:508: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
  warnings.warn(
[2024-09-30 06:55:21,709: INFO/MainProcess] Connected to redis://redis:6379/0
[2024-09-30 06:55:21,711: WARNING/MainProcess] /usr/local/lib/python3.11/dist-packages/celery/worker/consumer/consumer.py:508: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
  warnings.warn(
[2024-09-30 06:55:21,717: INFO/MainProcess] mingle: searching for neighbors
[2024-09-30 06:55:21,783: CRITICAL/MainProcess] Unrecoverable error: ContentDisallowed('Refusing to deserialize untrusted content of type pickle (application/x-python-serialize)')
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/celery/worker/worker.py", line 202, in start
    self.blueprint.start(self)
  File "/usr/local/lib/python3.11/dist-packages/celery/bootsteps.py", line 116, in start
    step.start(parent)
  File "/usr/local/lib/python3.11/dist-packages/celery/bootsteps.py", line 365, in start
    return self.obj.start()
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/celery/worker/consumer/consumer.py", line 340, in start
    blueprint.start(self)
  File "/usr/local/lib/python3.11/dist-packages/celery/bootsteps.py", line 116, in start
    step.start(parent)
  File "/usr/local/lib/python3.11/dist-packages/celery/worker/consumer/mingle.py", line 37, in start
    self.sync(c)
  File "/usr/local/lib/python3.11/dist-packages/celery/worker/consumer/mingle.py", line 41, in sync
    replies = self.send_hello(c)
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/celery/worker/consumer/mingle.py", line 54, in send_hello
    replies = inspect.hello(c.hostname, our_revoked._data) or {}
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/celery/app/control.py", line 389, in hello
    return self._request('hello', from_node=from_node, revoked=revoked)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/celery/app/control.py", line 106, in _request
    return self._prepare(self.app.control.broadcast(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/celery/app/control.py", line 776, in broadcast
    return self.mailbox(conn)._broadcast(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/pidbox.py", line 346, in _broadcast
    return self._collect(reply_ticket, limit=limit,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/pidbox.py", line 388, in _collect
    self.connection.drain_events(timeout=timeout)
  File "/usr/local/lib/python3.11/dist-packages/kombu/connection.py", line 341, in drain_events
    return self.transport.drain_events(self.connection, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/transport/virtual/base.py", line 997, in drain_events
    get(self._deliver, timeout=timeout)
  File "/usr/local/lib/python3.11/dist-packages/kombu/transport/redis.py", line 591, in get
    ret = self.handle_event(fileno, event)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/transport/redis.py", line 573, in handle_event
    return self.on_readable(fileno), self
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/transport/redis.py", line 569, in on_readable
    chan.handlers[type]()
  File "/usr/local/lib/python3.11/dist-packages/kombu/transport/redis.py", line 979, in _brpop_read
    self.connection._deliver(loads(bytes_to_str(item)), dest)
  File "/usr/local/lib/python3.11/dist-packages/kombu/transport/virtual/base.py", line 1017, in _deliver
    callback(message)
  File "/usr/local/lib/python3.11/dist-packages/kombu/transport/virtual/base.py", line 639, in _callback
    return callback(message)
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/messaging.py", line 650, in _receive_callback
    decoded = None if on_m else message.decode()
                                ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/message.py", line 201, in decode
    self._decoded_cache = self._decode()
                          ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/message.py", line 205, in _decode
    return loads(self.body, self.content_type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/kombu/serialization.py", line 255, in loads
    raise self._for_untrusted_content(content_type, 'untrusted')
kombu.exceptions.ContentDisallowed: Refusing to deserialize untrusted content of type pickle (application/x-python-serialize)
/usr/local/lib/python3.11/dist-packages/gramps/plugins/tool/check.py:51: PyGIWarning: Gtk was imported without specifying a version first. Use gi.require_version('Gtk', '3.0') before import to ensure that the right version gets loaded.
  from gi.repository import Gtk
(__main__.py:9): Gtk-CRITICAL **: 06:55:25.949: gtk_icon_theme_get_for_screen: assertion 'GDK_IS_SCREEN (screen)' failed
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.

DavidMStraub · October 7, 2024, 6:02am

Hi, the kombu error is weird, but I don’t think it’s related; it looks like it happens during startup and not while you synced.

Since there is nothing else in the log, it looks to me like the export task never reaches the worker, so it’s likely a wrong configuration.

Do you have a redis container?

system · November 6, 2024, 6:02am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Gramps Web Docker version doesn't work with Let's Encrypt Gramps Web	3	294	January 12, 2024
Setting up Gramps Web questions (Unraid server) Gramps Web	6	141	July 17, 2024
Since Gramps Web does not support importing gpkg file, how to import media files? Help websolutions	2	501	June 7, 2023
Deploying Gramps-Web on a Raspberry Pi 3 Model B+ Gramps Web	5	59	December 4, 2024
Difficulty with Gramps Web docker Gramps Web websolutions	1	1067	May 19, 2023

Having trouble with exporting data

Related topics