at work I’m using Jira quite a bit and I like it’s JQL (Jira Query Language) advanced search feature. Thinking about how to allow more complex filters in Gramps Web, I realized it would be nice to have something similar for Gramps objects. It would be more suited for tech savvy users, but would allow writing complex filters much faster than with a complex user interface.
I implemented a first version of this idea and released it as gramps-ql package on PyPI.
Here are some examples to make you curious:
type = note and private and text.string ~ David
media_list.length >= 10
type != person and media_list.any.rect
type = family and child_ref_list.length > 10
type = event and date.modifier = 0 and date.dateval[2] > 2020
See the Readme at the Github repository for more details.
Any volunteers writing a simple Gramplet for this? I myself would like to add a GQL filter option to Gramps Web API, obviously.
Yes it would be cool to add the existing Gramps filter functions, something like type = person and HasCommonAncestorWith("I0123"). Definitely possible, just a bit more work.
Have you written some form of specification? I’m looking for such a feature in a different context.
I’d like to add “auxiliary ‘dynamic’ notes” to objects to synthesise global information which could then be emited into Narrative Web. An example of an “auxiliary note” is generation of a link for citations pointing to the original data.
The repository record provides the hostname, the source record the path to the document and the citation record the final bit of data (page number or else).
The repository record has an “Internet” tab filled with urls entry. Nothing needs to be done here.
The source record would be augmented by a Note (with a suitable type) containing @reporef_list[0].urls['Web home'] & '/' & note_list['path'] & '/'
wher @ is “indirect access” operator, [ string ] selects the array item with the requested “type” (notion to be refined), & is concatenation operator.
The citation record in its turn receives a Note with contents @source_handle.note_list[ “suitable type” ] & '/daogrp/0/' & note_list['Page number']
These additional notes are generic enough to be shared across citations and sources. They don’t modify the record itself. They only display synthetic information and emit this information in Narrative Web pages. My goal is to have a link on which I can click to jump to the original data.
I have not yet thought in depth about it but it looks like a Gramps Query Language based on fields declared in the various JSON record descriptions. What puzzles me is how to disambiguate references to array elements (like note_list). I can’t dedicate enough time to it presently because I’m busy with other things. Any ideas?
@pgelier interesting, that seems quite ambitious, are you sure a mini-language is the right approach in this case rather than directly implementing it in Python in an addon?
Regarding array elements: since GQL is meant for filtering/searching, currently it only supports array access by numeric index (which is not super useful) or using the pseudo-properties any or all.
I briefly looked into implementing a Gramps addon to filter by GQL but wasn’t sure what is the best way.
Gramplet: it would be trivial to implement the input field, but how to display the results?
Rule: that would be straightforward, but to use it a user would have to create a custom filter, save it, and then apply it, which seems too clumsy.
Word “language” may be presumptuous. It is my inclination to abstract things. There are already such “micro-languages” in Gramps. For example, take the %x descriptors in the name formatter. This is limited to “dumb” substitution.
My idea is to extend slightly the concept so that data can be fetched from related objects (not only from the single object under scrutiny). Parsing must be kept easy in order to avoid implementing (and maintaining !) a real parser. A priori, expressions will be limited to concatenation. If there is a single “generating” operator, it could be made implicit but having it explicit allows to better control spaces between operands because they become strings under user control).
How do existing filters display their result? I thought the handled the selection to some library function.
Isn’t it the present approach?
Not related, but who knows? I have always been puzzled by the inability to apply some filter on the result of a previous filter. I don’t mean I want to combine two filters with and “and” connector. What I’d like to do is to use a filter on the DB. Then try another filter on what was returned, just to see. Presently, applying a second filter operates from scratch on the full DB.
I like the JQL idea. Native Filters (filter editor) are very powerfull but they are also inconvenient and slow. I would prefer use JQL for simple-middle searches and sometimes use Filters for more difficult searches.
Hi @kku, thanks for bringing that up, you’re right that there is definitely overlap and I should have looked more closely at SuperTool. It is definitely an awesome tool! However what I thought (but admittedly haven’t investigated in detail) is that it relies on exec(), which makes it a great tool for power users on desktop, but is not something one can do on a web server (and GQL was developed with web use obviously).
Let me have another look to check if there are synergies.
You need to look beyond the SuperTool interpreter shell itself. Kari has been building an entire ecology of tools.
Inside the SuperTool, there is a Help system that is user expandable. There are Query Rules that let SuperTool built queries be used in Custom Filters. Then outside, there is a Filter+ gramplet that will scrape parameters into a custom filter which can then be refined. The FilterParams allows overview of cascades of custom filters and the parameters to be manipulated without recursive Edit dialogs.
And scripts can be saved as Notes and shared as importable Trees. (Those Trees can be shared in Postings here on the Discourse forum.)
It also allows use of the Simple Access framework.
Although I wonder if the Gramps tree database file having so many ways of being touched might not be counter productive. The Gramps engine has the Gramps gen module, the Classes: date handler, relationshipcalculator, Simple Access. SuperTool has its own objects, the GQL, the database engine’s query language… are there others?
My biggest worry with SuperTool is that using it with the GUI interpreter version has a 1,000 record hard limit. You have to use the Command Line mode to get around that.
All, if you’re wondering why the GQL repo is not reachable - Github’s AI has suspended my account because of a spam comment made with my account (with a leaked access token) and I’m waiting for the appeal to be processed by a human being, which can take days or even weeks I heard
