Importing a .gramps file into a Postgres database with GrampsWeb

Hi!

I’ve successfully managed to deploy a GrampsWeb app on my Ubuntu server.
However, since my tree is quite large, I’d like to use PostgreSQL instead of SQLite.

I’m following the steps from the documentation.

I’m using this docker-compose.yml :

services:
  grampsweb: &grampsweb
    container_name: grampsweb
    image: ghcr.io/gramps-project/grampsweb:latest
    restart: always
    environment: &grampsweb-env
      GRAMPSWEB_TREE: "Gramps Web"  # will create a new tree if not exists
      VIRTUAL_PORT: "5000"
      VIRTUAL_HOST: mydomain.com  # e.g. gramps.mydomain.com
      LETSENCRYPT_HOST: mydomain.com  # e.g. gramps.mydomain.com
      LETSENCRYPT_EMAIL: myemail.com  # your email
      GRAMPSWEB_CELERY_CONFIG__broker_url: "redis://grampsweb_redis:6379/0"
      GRAMPSWEB_CELERY_CONFIG__result_backend: "redis://grampsweb_redis:6379/0"
      GRAMPSWEB_RATELIMIT_STORAGE_URI: redis://grampsweb_redis:6379/1
      # the PostgreSQL addon assumes the tree name to be
      # equal to the database name and here the default
      # database name of the PostgreSQL image is used
      TREE: postgres
      # The credentials must agree with the ones used for
      # the PostgreSQL container
      POSTGRES_USER: gramps
      POSTGRES_PASSWORD: postgres_password_gramps
    volumes:
      - gramps_users:/app/users
      - gramps_index:/app/indexdir
      - gramps_thumb_cache:/app/thumbnail_cache
      - gramps_cache:/app/cache
      - gramps_secret:/app/secret
      - gramps_db:/root/.gramps/grampsdb
      - gramps_media:/app/media
      - gramps_tmp:/tmp
    networks:
      - proxy-tier
      - default

  grampsweb_celery:
    <<: *grampsweb  # YAML merge key copying the entire grampsweb service config
    container_name: grampsweb_celery
    depends_on:
      - grampsweb_redis
    environment:
      <<: *grampsweb-env  # YAML merge key copying the grampsweb environment config
      # overriding let's encrypt variables since celery is not exposed
      VIRTUAL_PORT: ""
      VIRTUAL_HOST: ""
      LETSENCRYPT_HOST: ""
      LETSENCRYPT_EMAIL: ""
    command: celery -A gramps_webapi.celery worker --loglevel=INFO --concurrency=2

  grampsweb_redis:
    image: docker.io/library/redis:7.2.4-alpine
    container_name: grampsweb_redis
    restart: always

  proxy:
    image: docker.io/nginxproxy/nginx-proxy
    container_name: nginx-proxy
    restart: always
    ports:
      - 80:80
      - 443:443
    environment:
      ENABLE_IPV6: "true"
    volumes:
      - ./nginx_proxy.conf:/etc/nginx/conf.d/my_proxy.conf:ro
      - conf:/etc/nginx/conf.d
      - dhparam:/etc/nginx/dhparam
      - certs:/etc/nginx/certs:ro
      - vhost.d:/etc/nginx/vhost.d
      - html:/usr/share/nginx/html
      - /var/run/docker.sock:/tmp/docker.sock:ro
    networks:
      - proxy-tier

  acme-companion:
    image: docker.io/nginxproxy/acme-companion
    container_name: nginx-proxy-acme
    restart: always
    environment:
      NGINX_PROXY_CONTAINER: nginx-proxy
    volumes:
      - certs:/etc/nginx/certs:rw
      - vhost.d:/etc/nginx/vhost.d
      - html:/usr/share/nginx/html
      - acme:/etc/acme.sh
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks:
      - proxy-tier
    depends_on:
      - proxy

  postgres_gramps:
    image: ghcr.io/davidmstraub/gramps-postgres:latest
    restart: unless-stopped
    environment:
      POSTGRES_PASSWORD: postgres_password_admin
      POSTGRES_PASSWORD_GRAMPS: postgres_password_gramps
      POSTGRES_PASSWORD_GRAMPS_USER: postgres_password_gramps_user
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  acme:
  certs:
  conf:
  dhparam:
  vhost.d:
  html:
  gramps_users:
  gramps_index:
  gramps_thumb_cache:
  gramps_cache:
  gramps_secret:
  gramps_db:
  gramps_media:
  gramps_tmp:
  postgres_data:

networks:
  proxy-tier:

and this nginx_proxy.conf :

client_max_body_size 500m;

With my_tree.gramps, these are the 3 files in my home dir on the server.

I installed the required libraries:

  1. sudo apt update && sudo apt upgrade -y
    
  2. sudo apt install -y ca-certificates curl gnupg lsb-release
    
  3. curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
    
  4. echo “deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg]  $(lsb_release -cs) stable” | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
    
  5. sudo apt update
    
  6. sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
    
  7. sudo usermod -aG docker $USER
    

Then, I ran:

  1. docker compose up

I’m not sure whether my docker-compose.yml is correct, and I’m confused by the next step.

I can see the onboarding page at mydomain .
Then, I can import the .gramps file through the Web UI and everything works fine, but is that it? Is my_tree imported in Postgres, or is it still imported in SQLite?

I’m asking since I’m reading the documentation:

Importing a Gramps family tree

Again, if you have set up the PostgreSQL server yourself, you can follow the instructions in the Gramps Wiki to import a family tree into the database.

Alternatively, if you have followed the Docker Compose instructions above, you can use the following command to import a Gramps XML file located on your docker host:

docker compose run --entrypoint "" grampsweb \
    gramps -C postgres \
    -i /root/.gramps/grampsdb/my_tree.gramps \
    --config=database.path:/root/.gramps/grampsdb \
    --config=database.backend:postgresql \
    --config=database.host:postgres_gramps \
    --config=database.port:5432 \
    --username=gramps --password=postgres_password_gramps

By reading this, I’m not sure whether I can simply import my .gramps file to Postgres through the UI, or if it must be through the command (since I have followed the Docker Compose instructions from the documentation).

Thanks in advance for the help.

If everything works fine importing through the UI → happy days!

Probably the Postgres docs need updating. It might even be from a time where importing from the UI was not possible :grimacing:

Doc contributions welcome btw :slightly_smiling_face:

1 Like

Thank you for the clarification, David!

Hi @DavidMStraub ,

I’m getting back to you since I’m not sure that my .gramps file was imported in Postgres.

  1. I haven’t noticed any speed improvement (my tree has 123,000 people, and displaying an ancestor tree takes 30 sec, which was also the case when I didn’t attempt to import the tree into Postgres)
  2. docker exec -it ubuntu-postgres_gramps-1 psql -U postgres -d gramps and then \dt returns Did not find any relations.
  3. In this blog post, the person says: “We can verify by using the Grampsweb web-interface and going to “Profile”/”Settings” → “System Information” → “Tree Information”. We should see that the ID is a short sequence on the form 0000x000, not a longer hash like sequence e.g. b2a81742-fc73-4b31-9f4a-f561850945ae.” Though I still see a hash like sequence like “b2a81742-fc73-4b31-9f4a-f561850945ae”.

I’m not sure that my docker-compose.yml is correct, especially the environment variables:

services:
  grampsweb: &grampsweb
    container_name: grampsweb
    image: ghcr.io/gramps-project/grampsweb:latest
    restart: always
    environment: &grampsweb-env
      GRAMPSWEB_TREE: "Gramps Web"  # will create a new tree if not exists
      VIRTUAL_PORT: "5000"
      VIRTUAL_HOST: mydomain.com  # e.g. gramps.mydomain.com
      LETSENCRYPT_HOST: mydomain.com  # e.g. gramps.mydomain.com
      LETSENCRYPT_EMAIL: myemail.com  # your email
      GRAMPSWEB_CELERY_CONFIG__broker_url: "redis://grampsweb_redis:6379/0"
      GRAMPSWEB_CELERY_CONFIG__result_backend: "redis://grampsweb_redis:6379/0"
      GRAMPSWEB_RATELIMIT_STORAGE_URI: redis://grampsweb_redis:6379/1
      # the PostgreSQL addon assumes the tree name to be
      # equal to the database name and here the default
      # database name of the PostgreSQL image is used
      TREE: postgres
      # The credentials must agree with the ones used for
      # the PostgreSQL container
      POSTGRES_USER: gramps
      POSTGRES_PASSWORD: postgres_password_gramps

If you spot something I’m missing in the docker-compose.yml, I’d be grateful!

Hi,

Nothing wrong with the long hash.

Concerning performance, various things get slow when trees are that large. But if you thought postgres helps - it doesn’t. It makes some things slower actually.

Thanks again for clarifying :slight_smile:

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.