I have a new proposal about how to move forward in allowing filters to be SQL optimizable.
I developed a method in Dec 2024 that allowed Python syntax to be converted into SQL. But this is a bit complicated, and there aren’t very many people in the Gramps development community today that can appropriately review it. And it is opinionated in its approach. One might argue that the syntax shouldn’t be the one and only for core Gramps, and I wouldn’t argue. (There are lots of options for Object-Relational Mappers (ORM) for Python, like SQLAlchemy).
My proposal: create a well-defined method of allowing database plugins to override filter implementations. This keeps Gramps operating exactly as it is, but also allows a plugin to be able to use whatever method to implement direct SQL (or other backend) commands.
I’ll move my select_from_table method (from above) to a plugin. And people can decide to use that or not, and development can happen outside the core Gramps development release time line.
I’ve created a PR (work in progress, but almost done): Add register_override API so DB backends can plug in optimized implementations. This has the bare minimum to allow such plugins to be developed. If a database has registered an Rule override, then it will be used instead. This can also take advantage of the Filter Optimizer added in Gramps 6.0.
I look forward to getting the Gramps 6.1 PRs reviewed and merged. Questions and comments of course welcomed!