Using Gramps Web API for adding new information

Gramps Web
1

Hello,

I’m struggeling a littel bit on how to use the API:

A person object contains the field “parent_family list”. So I thought that i could easily link a family (e.g. F0001) to a person (e.g. I0002):

familydata = GET: “api/families?GrampsID=F0001”
persondata = GET: "api/people?GrampsID=I0002
persondata[“parent_family_list”].push(familidata[“handle]”)
PUT: “api/people/”+persondata[“handle”] (body=persondata)

Therafter, the family is shown in the person view within Gramps Web, but in the family view the child is not visible.

From Gramps I know that the relation is Family→Children. But from the API documentation I thought that I can also do Child →Family and Gramps Web API does the linking.

For the future, how do I know what information of an object can be changed within that object (e.g. surname) and what information is just derived from other objects (e.g. parent_family_list) and considered as static and need to be changed elsewhere (family[“child_ref_list])?

Background

I’m creating a special web view to quickly add the information from a large paper-based family tree into Gramps Web. I want to reduce the amount of clicks from

  • Creating father
    • Creating birth event
    • Creating death event
  • Creating mother
    • Creating birth event
    • Creating death event
  • Creating child
    • Creating birth event
    • Creating death event
  • Creating family
    • Attach father
    • Attach mother
    • Create wedding event
    • Attach child

to a single form, where every data can be entered and on submit all the different objects are created and linked under the hood.

2

Hi,

this is an excellent application for Gramps Web API, looking forward to it!

Yes, child-parent relationships are a painful part of the Gramps data model because adding a child to a family affects both the child and the family object.

In Gramps core, it happens to be like that: when you update a family, the method at the database level will make sure to update the person as well.

Indeed you have to know something about the inner workings of Gramps core to know which one of the two options is the right one.

But at least I think this is the only case of such a problematic bi-directional linking.

Code:

3

So if I update the family with the reference to the child, Gramps takes care of the backlink?

How about family_list? Needs also changed in the family instead of within the person?

4

Yes.

Family doesn’t have a family_list. You only need to update the child_ref_list, father_handle, or mother_handle of the family.

5

How about adding a birthday?

Is it enough to create an event with type birth and refernce it in event_ref_list of that person and Gramps is doing its magic, or do I also have to set birth_ref_index with the corresponding array index?

And is the backlinks[“person”] array within the event automatically filled or do I have to to something?

Sorry for all these questions, but I’m afraid that I get an inconsistend database. I really wonder why redundant information is stored.

6

Is it enough to create an event with type birth and refernce it in event_ref_list of that person and Gramps is doing its magic, or do I also have to set birth_ref_index with the corresponding array index?

Yep, that’s enough.

Also backlinks is filled automatically, that’s a Web API specific thing. (In Gramps core, it corresponds to the “references” table in the database, which is also populated automatically.)

Sorry for all these questions, but I’m afraid that I get an inconsistend database.

Don’t be sorry, your confusion is understandable, and if you have a good idea where we should document such things, it would be awesome if you could do so as you bump into them.

I really wonder why redundant information is stored.

I don’t have an answer because the database model predates my involvement in the project, but until a few years ago, Gramps didn’t even use SQL, but an obscure (sorry BSDDB fans) database format called BSDDB; the Gramps core team is doing a great job carefully modernizing the database backend without breaking anything for users, but it’s true that if someone reimplemented it from scratch today, the architecture would probably look a bit different.

7

BSDDB was shipped with early python versions and was a default database choice.

The double-linking was inherited from GEDCOM.

2 Likes
8

The Gramps Web API documentation is a very good starting point. There we could mention what parts of the payload should not be changed in PUT requests.

grafik
grafik1072×972 77 KB

Shall I go for a pull request of the swagger docu?

I don’t know hof tight the web API is bound to Gramps core. If there is some kind of freedom I would propose that object details and derived object information are split in different REST endpoints:

  • GET/PUT: api/persons/{handle}
    • Detail like name, references to events, etc.
  • GET: api/persons/{handle}
    • Additional information like back-references to families
9

Shall I go for a pull request of the swagger docu?

Yes please! :folded_hands:

I don’t know hof tight the web API is bound to Gramps core. If there is some kind of freedom I would propose that object details and derived object information are split in different REST endpoints:

That would have been a reasonable choice indeed, but now I think it would be a bit painful to disentangle and rewrite all the logic in the frontend. The additional information is all under a small number of top level object properties. See e.g. here where they are stripped in the frontend before a PUT request:

1 Like
10

While digging deeper, I’m curios how the handle is generated. Before I thought that the backend creates it on POST requests. But now I saw that the frontend creates it (while adding a new person with birth event). Are there any special rules (besides max 50 characters)?

11

The backend generates it where needed, but when the frontend generates multiple objects with links between them - such as a person and a birth event - it needs to generate the handle in advance.

There are no other rules in theory, but in practice the UUIDs with hyphens that the frontend generates caused some issues with addons that made stricter assumptions than the data model allows. So UUID4s without hyphens (e.g. its hex digest) is what I would recommend.

12

Gramps (for desktops) encodes a creation timestamp to ensure unique handles.

There is a discussion about decoding handles in another thread.

I do not know if Gramps Web follows the same approach. So this may not apply if the Gramps Web frontend is determining the handle.

13

Gramps Web does not use timestamps in handles because in a multi-threaded server application, this is not unique enough. It uses UUIDv4s, which are unique for all practical purposes.

1 Like
14

Thanks for all the background information. I thought the handle is Gramps Web specific and not known in Gramps.
The IDs like I0001 are also not used in Gramps and just for GEDCOM compatibility?

15

The IDs are convenience features. They make it easier for humans to remember and reference in the interface. The Gramps engine uses ‘handles’ for every object internally

1 Like
16

Is there a good documentation of all these fields?
Currently, I’m scratching my head why an Event contains the year two times (Event.year and Event.date.dateval) and how to set sortval.

17

Dates are problematic for sorting because the formatted date will be a string… which could be a span or a range or an oddly ordered set of date components. And unless the date format is yyyy.mm.dd, the formatted display date value will sort alphabetically and that is unlikely to follow the order of the dates. (e.g., 1999.08.01 will come before 1970.01.20 when displayed in formats like d mmm yyyy, mmm dd yyyy or d m yyyy)

yyyy.mm.dd d m yyyy d mmm yyyy mmm dd yyyy
1970.01.20 1 8 1999 1 AUG 1999 AUG 01 1999
1999.08.1 20 1 1970 20 JAN 1970 JAN 20 1970

See the Sphinx Developer Documentation for information about field values

And there is a feature request requesting a less fussy method for handling date sorting. It references previous bug reports and Pull Requests related to Gramps incorrectly sorting date columns in tables: 0013317: Create a standard "Date" column routine that simplifies displaying and sorting - Gramps - Bugtracker – Free Genealogy Software

18

I’m now able to add a person (POST api/people) and also to add an event (POST api/events).
However, when I start to to POST request quickly after each other, I get a 500 “Internal server error”. The docker log output shows sqlite3.OperationalError: database is locked:

grampsweb-1       | [2025-10-19 05:12:16 +0000] [11] [ERROR] Error handling request /api/people/
grampsweb-1       | Traceback (most recent call last):
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1514, in wsgi_app
grampsweb-1       |     response = self.handle_exception(e)
grampsweb-1       |                ^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask_cors/extension.py", line 176, in wrapped_function
grampsweb-1       |     return cors_after_request(app.make_response(f(*args, **kwargs)))
grampsweb-1       |                                                 ^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1511, in wsgi_app
grampsweb-1       |     response = self.full_dispatch_request()
grampsweb-1       |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 919, in full_dispatch_request
grampsweb-1       |     rv = self.handle_user_exception(e)
grampsweb-1       |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask_cors/extension.py", line 176, in wrapped_function
grampsweb-1       |     return cors_after_request(app.make_response(f(*args, **kwargs)))
grampsweb-1       |                                                 ^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 917, in full_dispatch_request
grampsweb-1       |     rv = self.dispatch_request()
grampsweb-1       |          ^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 902, in dispatch_request
grampsweb-1       |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/auth.py", line 44, in wrapper
grampsweb-1       |     return func(*args, **kwargs)
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 110, in view
grampsweb-1       |     return current_app.ensure_sync(self.dispatch_request)(**kwargs)  # type: ignore[no-any-return]
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 191, in dispatch_request
grampsweb-1       |     return current_app.ensure_sync(meth)(**kwargs)  # type: ignore[no-any-return]
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 481, in post
grampsweb-1       |     add_object(db_handle, obj, trans, fail_if_exists=True)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/util.py", line 1002, in add_object
grampsweb-1       |     return add_method(obj, trans)
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/generic.py", line 1949, in add_person
grampsweb-1       |     return self._add_base(
grampsweb-1       |            ^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/generic.py", line 1945, in _add_base
grampsweb-1       |     commit_func(obj, trans)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/generic.py", line 2049, in commit_person
grampsweb-1       |     old_data = self._commit_base(person, PERSON_KEY, transaction, change_time)
grampsweb-1       |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/dbapi.py", line 692, in _commit_base
grampsweb-1       |     self.dbapi.execute(
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/sqlite.py", line 139, in execute
grampsweb-1       |     self.__cursor.execute(*args, **kwargs)
grampsweb-1       | sqlite3.OperationalError: database is locked
grampsweb-1       |
grampsweb-1       | During handling of the above exception, another exception occurred:
grampsweb-1       |
grampsweb-1       | Traceback (most recent call last):
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 134, in handle
grampsweb-1       |     self.handle_request(listener, req, client, addr)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 177, in handle_request
grampsweb-1       |     respiter = self.wsgi(environ, resp.start_response)
grampsweb-1       |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1536, in __call__
grampsweb-1       |     return self.wsgi_app(environ, start_response)
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1527, in wsgi_app
grampsweb-1       |     ctx.pop(error)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/ctx.py", line 426, in pop
grampsweb-1       |     app_ctx.pop(exc)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/ctx.py", line 262, in pop
grampsweb-1       |     self.app.do_teardown_appcontext(exc)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1382, in do_teardown_appcontext
grampsweb-1       |     self.ensure_sync(func)(exc)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/app.py", line 212, in close_db_connection
grampsweb-1       |     close_db(db_write)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 430, in close_db
grampsweb-1       |     db_handle.close()
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/generic.py", line 862, in close
grampsweb-1       |     self._set_all_metadata()
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/generic.py", line 882, in _set_all_metadata
grampsweb-1       |     self._set_metadata("version", str(self.VERSION[0]))
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/dbapi.py", line 416, in _set_metadata
grampsweb-1       |     self.dbapi.execute(
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/sqlite.py", line 139, in execute
grampsweb-1       |     self.__cursor.execute(*args, **kwargs)
grampsweb-1       | sqlite3.OperationalError: database is locked
grampsweb-1       | [2025-10-19 05:12:21 +0000] [12] [ERROR] Error handling request /api/people/
grampsweb-1       | Traceback (most recent call last):
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 134, in handle
grampsweb-1       |     self.handle_request(listener, req, client, addr)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gunicorn/workers/sync.py", line 177, in handle_request
grampsweb-1       |     respiter = self.wsgi(environ, resp.start_response)
grampsweb-1       |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1536, in __call__
grampsweb-1       |     return self.wsgi_app(environ, start_response)
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1514, in wsgi_app
grampsweb-1       |     response = self.handle_exception(e)
grampsweb-1       |                ^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask_cors/extension.py", line 176, in wrapped_function
grampsweb-1       |     return cors_after_request(app.make_response(f(*args, **kwargs)))
grampsweb-1       |                                                 ^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1511, in wsgi_app
grampsweb-1       |     response = self.full_dispatch_request()
grampsweb-1       |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 919, in full_dispatch_request
grampsweb-1       |     rv = self.handle_user_exception(e)
grampsweb-1       |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask_cors/extension.py", line 176, in wrapped_function
grampsweb-1       |     return cors_after_request(app.make_response(f(*args, **kwargs)))
grampsweb-1       |                                                 ^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 917, in full_dispatch_request
grampsweb-1       |     rv = self.dispatch_request()
grampsweb-1       |          ^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 902, in dispatch_request
grampsweb-1       |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/auth.py", line 44, in wrapper
grampsweb-1       |     return func(*args, **kwargs)
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 110, in view
grampsweb-1       |     return current_app.ensure_sync(self.dispatch_request)(**kwargs)  # type: ignore[no-any-return]
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/flask/views.py", line 191, in dispatch_request
grampsweb-1       |     return current_app.ensure_sync(meth)(**kwargs)  # type: ignore[no-any-return]
grampsweb-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/resources/base.py", line 479, in post
grampsweb-1       |     with DbTxn(f"New {self.gramps_class_name}", db_handle) as trans:
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/gen/db/txn.py", line 83, in __exit__
grampsweb-1       |     self.db.transaction_commit(self)
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/dbapi.py", line 332, in transaction_commit
grampsweb-1       |     self.dbapi.commit()
grampsweb-1       |   File "/usr/local/lib/python3.11/dist-packages/gramps/plugins/db/dbapi/sqlite.py", line 168, in commit
grampsweb-1       |     self.__connection.commit()
grampsweb-1       | sqlite3.OperationalError: database is locked

I didn’t found a way to unlock the database. I can only whipe all docker volumes and start over again.
While doing this (docker compose down --volumes) I have the next problem is that the welcome screen where I can create the first user is not shown, but instead the login screen. Only after whiping and restarting everything two to three times, the welcome screen shows again.

As this makes it hard to investigate further for the first problem, I wonder how GramosWeb decides when to show the welcome screen. I thought whiping the volumes should be sufficient.

Any suggestion where to look deeper is welcome.

19

I had to clear the Local Storage of the browser to have the welcome screen back again.

When I add a sleep of 2 seconds between POST requests, I don’t get the database locked problem. The database is blocked only when I do the POST request immediately after each other.

1 Like
20

Concerning the database locked problem: you are using SQLite, right? This kind of problem can happen with SQLite as it’s not designed for concurrent editing. If you look at how the Gramps Web frontend avoids this kind of problem, it never dispatches edit actions simultaneously. You can add a person and an event simultaneously (and linked to each other) by posting to /api/objects.