The Gramps ecosystem

I thought I’d make a post giving an overview of the Gramps ecosystem since a couple of new projects have opened up new workflows. I didn’t include any of the website generator projects, nor any of the addons, and addon “app stores”. The following list may not be complete. Let me know if I left out things, and I will update the diagram.

First, let’s take a look at an overall picture of the main components:

In the bottom, left-hand corner is the gramps desktop that most of us use. Of course, you can get data in and out by doing imports/exports through the Gramps file formats (3rd row, righ-hand side). Gramps supplies the core code, database abstractions, and a robust addon community. All of this code is written in Python.

On the second row from the top is a key component: the gramps-web-api. Usually, you might just consider it as the “backend” of Gramps Web, but it is actually much more. It too is written in Python, and uses much of the infrastructure provided by the gramps core code. But it is designed to run as a host on the internet, managing permissions and access to different family trees. It is a production-ready web server tested over the last few years.

On the top row sits the official Gramps Web, and a newcomer Gramps Connect. These are both the “frontends” for interacting with gramps-web-api server. Both frontens are written in JavaScript, or a variant. They talk to gramps-web-api through a REST interface, a standard method of communication between systems on the internet. There is no direct database access in the frontends: they can only do what the backend allows, when it comes to accessing the data.

On the second to the bottom row lives the addons to Gramps desktop:

The GrampsWebSync addon tool allows you to do bi-directional merging between gramps-web-api, and a local Gramps database. Every once in a while, you run the tool to sync the local gramps database and the server. Useful for one user to keep in sync with the server. Handles conflicts interactively with you.

The GrampsWebApiDb addon database allows you to dynamically stay in sync between a gramps-web-api server and a local mirror of the hosted data. In this addon, the server handles the dynamic updates. Handles conflicts automatically. [More about this soon]

Notice that when describing the two addons that I didn’t mention the frontends, ie the top row. That is because the addons don’t interact at all with Gramps Web or Gramps Connect, they only talk to the backend server, gramps-web-api.

But the new GrampsWebApiDb suggests a novel method of working together: you don’t necessarily need a frontend at all; you (and your collaborators) could all use standard Gramps desktop apps, and each user could keep their mirror in sync using the addon. Of course, you can also use the web apps, but they aren’t required.

Finally, I will briefly mention the new kids on the block (in development): gramps-connect, and gramps-connect-desktop. The first is a hosted frontend, and the second is a stand-alone version that acts like a desktop app. Thus it gets to sit on the top row as a web app, and also on the bottom row as a desktop app. The desktop version has a gramps-web-api server embedded in it. Both are in development, and ready for beta testing.

Overall, this is a rich ecosystem—do you have questions, or ideas?

Very interesting! But in that case, everyone would have equal access (not distinct “roles”)?

Thanks for the chart, very helpful.

Thanks for the question! No, the roles for GrampsWebApiDb are the same exact roles provided by gramps-web-api (and seen in Gramps Web). I’ll describe this in detail soon, but the secret is that the db addon uses an API key that associates to the server user role on a given host.

So, in gramps desktop, a non-edit user could see the data (and even watch it change in real time) but editing data would be prevented, effectively making a read-only database.

Thanks, and In ways useful for me too! Keeping this straight is important for understanding how these components work. I hate the name GrampsWebApiDb but I felt it was important to be clear in its position in the chart.

The problem is that “web” means multiple things: runs in a web browser (but so does Gramps Connect), or runs in the cloud (but even Gramps desktop can use a remote PostrgeSQL database).

So it might be helpful to have versions of the chart showing where the components are running in each scenario.

Yes, that is another dimension. It gets messy on that bottom row where they are running on your computer, but also in the browser. There are also complexities under the hood that I glossed over as Gramps Connect uses an internal SQLite inside the browser for caching.

The source for the graph is here if you want to tweak it:

Thanks for the graph source, but I fear I would make a mess of it.

One thing I’m confused about is where all Gramps “core” exists (and what all it includes). Obviously it’s part of the desktop app. Is it also embedded (duplicated) within gramps-web-api, or a separate piece of the gramps-web backend?

Also, you must already know this, but there is 10-year-old (retired) “gramps_connect” project which might cause further confusion.

The gramps core code is a Python package that is used by gramps-web-api. That’s what make the web-apps 100% compatible (database-wise). It uses it, but gramps core is not duplicated. The web-apps also reuse Gramps filters and reports.

Yes, I am aware :smiley: . That was my first (or second) attempt at making a web version of gramps. I couldn’t come up with a better name this time around, so I re-used. We can rename the old archive to help avoid confusion.

Ah, now I see it in pypi.org. I had been trying to make sense of things only from the repos in the gramps-project in github.

But I also notice that’s only version 5.2.0rc1, so maybe I’m looking in the wrong place.

If you look in the release history you will see that the latest version of the gramps package is 6.0.8.

Yes, but I was referring to the gramps-core package.

Looks like a test package (rc = release candidate) that didn’t lead anywhere… yet. The real package is called gramps

The gramps-core package is just a placeholder. I reserved a few package names in case we wanted to use them in the future.

Thanks. But the gramps package also includes the desktop GUI. So when you said:

You meant that it’s currently using the gramps package, but ideally it could use the gramps-core package (when it becomes a real thing)?

Yes. That is why I reserved gramps-core, gramps-desktop, gramps-cli and gramps-gtk.