Posted a query on an R & Python development forum asking for examples of clean entry points for developers and newbie coders. (The Gramps developer docs are spread out & require a good bit of exploration before someone can get started. So I was wondering if there was a cleaner framework we could mirror.)
One reaponse cited the example of the Sphinx based GeoPandas docs page:
With R, I extracted basic family tree data from Gramps’ SQLite export file. The task was to compose a GEDCOM file for export to MyHeritage with proper handling of the patronymic name. From the SQLite file, I extracted names, relations, and key events with dates and locations.
Thanks! That is good example of creating an external entry point to the Gramps database using a tool coded in R.
If this is a stable GitHub repository, do you mind if I create a link to it from a wiki page? If it is a temporary section, maybe we could fork it as sample code in the main project Repository?
You reply also showed that my original posting was ambiguous. Thanks! I edited to clarify it. I hoping people will point us at Developer Documentation examples that simplified learning code with a new API or open source project. Our project has LOTS of developer documentation but (like the user Wiki) it is mostly reference style. There are tutorial articles but they are not sequenced. It’s like a house of a thousand doors. You can enter from anywhere but you are likely to wander around, hopelessly lost. A grand entrance with a guided tour would streamline getting started.
When I read this right, @malexan mentioned the SQLite export file, which means that there is no direct access to the actual Gramps database.
I like the idea of an API, but that can probably only work if it can unpack the pickled blobs that we have in our database, and make them available as Python, JSON, or XML objects.
If this is a stable GitHub repository, do you mind if I create a link to it from a wiki page? If it is a temporary section, maybe we could fork it as sample code in the main project Repository?
Maybe let me clean it up and provide some comments.
which means that there is no direct access to the actual Gramps database.
You can access the database directly: its location is known. I don’t remember exactly why I chose to work with the exported variant. Probably, it was something with the structure of the database. And I am not sure how the internal DB and the exported one differ.
I didn’t find detailed documentation, so I spent some time guessing and exploring. For example, I didn’t find a way of retrieving the order of children in the family.
One of the challenges with the Gramps database is that it doesn’t use a typical scheme with obvious links like a table with families, a table with persons, and a table with connections of persons to families as children. Instead, there is an additional layer (a set of tables) with links. And a simple question like Give me all the children of this family becomes something like Give me all the persons who have links of type “Child” and that links point to other links of type “Family” where Family = F0001.
There was a discussion on this topic among the developers: the idea was that the database is not suitable for direct SQL querying because they designed it with another purpose in mind. I should have the link to this discussion in the code, I believe 
Yes, I was hoping @malexan would read what I wrote and correct any misinterpretation.
Since database formatted trees are the working file format, a SQLite file probably is not an “export” file. The normal export file format would be a .gramps file in XML (in zip compressed or plain text).
There were a bunch of files in his Repository so I did not slog through to determine which file format they opened.
Since database formatted trees are the working file format, a SQLite file probably is not an “export” file. The normal export file format would be a
.grampsfile in XML (in zip compressed or plain text).
There are many possible formats for export from Gramps, including XML, JSON, SQLite, and GEDCOM. I worked with SQLite.
You may find the following Gramps documentation useful:
The default database engine was formerly BSDDB. With version 5.1 of Gramps, the default db engine was changed to SQLite.
That was so recently changed that most documentation (like SQL Gramps tables) and SQL schema still needs validation by more developers.
You’ll also want to look at the block diagram of the Gramps Data Model.
In addition, the Isotammi SuperTool helps when trying discover field names.
The unexported database stores nearly all data in pickled blobs, which makes it difficult to access the data. I guess that was the reason. 
I know where it is, and if you like to work with SQL, you made the right decision to use the SQLite export, because the database itself has most of the data in blobs, and isn’t normalized.
These blobs are pickled Python objects, the contents of which are described on our Wiki. And as far as I know, there are no libraries that you can use to access these objects from another language, except when you find a way to hack into the Python source itself, which is a moving target. You can reverse engineer it, if you want, and write some independent C code to access them, but I don’t know whether that’s worth investing time in.
And I guess that in the old days, using an object database was a pretty good idea, because by then SQL was slow, and depended on a server process, whereas BSDDB was nothing more than a library. And in fact it sort of reminds me of Brother’s Keeper, which has a database built with B-Trieve, or something.
For our developers, migrating to an SQL based solution, based on standard transactions, was a bit more than we could handle, so the only thing that we did was adding a custom DB-API layer that enabled us to store our objects in all sorts of different databases, while keeping all relations and transaction logic in code. And that made a lot of sense at the time, because the transition to SQLite has reduced the chance of database corruption to a number close to zero.
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
