There is a filter called “Probably Alive” that allows filtering and exporting of just those people that were alive on a given date.
Do you use that feature?
This filter is one that can take a long time (25 seconds on a tree of 100k people). So, we are looking to see if it is worth making changes to make it faster (reduce time to less than a second).
Could really use a Family filter that finds family records that do or do not have children attached. It would be nice to be able to select on how many children. E.g. >1 or <3
Making a list, and checking it twice. Added to wish list. (It occurs to me we could expose the proposed enhancements for next Gramps so you could do your own filters like that. Do you think “len(family.child_ref_list) == 1” would be too hard for user to understand and write?)
There’s already another (stagnant) rule request list thread. Unfortunately, you could could get buried in infinite rule request work. There’s another request that asks for a tutorial on creating Rules. But that might be sidestepped with the SQL supprt … since there are so many tutorial in existence on building those queries. (Although something intoductory about the Gramps database model might help.)
However, @Davesellers has a point about the existing filter rules being a bit thin in the Families category. Both in finding families that match criteria and in finding secondary objects of specific families.
Merging interactively could be greatly steamlined if there was an (efficient!) winnower of objects related to a family.
Say that there was a rule to find the Families (both as a child and parent) of the Active Person.
Then a matched set of Family Rules for each object type:
find <object type> under [family] (where [family] could be a list of IDs)
find <object type> under <family filter] (which be an Active Family filter)
And where these rules had options to include:
Family level objects and/or Family Member objects
full drill-down of secondary objects. (So a search for Citations would even find the Citation for the Media object attached to marriage Family event of the 3rd child.)
Essentially, the (full drill-down including Family and Personal secondary objects) Rules would work with the same subset of objects that are already exported for a selected person. So the winnowing functionality already exists elsewhere.
So… with the families of active Person filtered in the Families, you could walk through selecting the merging all the duplicate families within 1 degree of separation. Then step through each category, merging the duplicate people of the active family, then all the duplicate Events, Places, Repositories, Sources, Citations, Media, Notes.
Doing this step through Person-by-Person would be massively more cycles (and error prone) than Family-by-Family.
A complication about an ever expanding set of custom filters is discovering whether the Filter already exists.
This may be one of the situations where an AI librarian (overseeing a catalog of queries) could become necessary.
I’ve always wondered what methods programmers, data scientists and mathematicians use for keeping their code snippets, queries, and formulae “findable”.
Thanks for the list of filter requests. That will be useful.
[There are over 50 in that list, and there are already 250 in Gramps. One problem is that these have to be added in a new version as they would be enhancements. If a user can construct their own dynamically, then that would solve three issues: 1) You need a filter, 2) someone has to write it, and 3) it won’t be available until next version.]
[I think you are describing a new way of using/handling filters. If important, please make a ticket and explain as best you can what you are describing. But this thread is a survey on Probably Alive].
I’m not a programmer and not the best at putting together query strings. However, before I retired, I had an application (can’t remember the name) that allowed me to build SQL queries just by dragging and dropping items onto a query edit page. Once you connected to the DB you could see the tables and elements and then form the query you wanted and then build the report you wanted.
That is the kind of enhancement that would get rid of many of the 250 queries and allow users to create their own. It would take some work. Is there something like that already in the public domain?
ps. It built the complex sql in the background which you could also see if needed.
Perhaps command-completion will be a good start? It certainly helps me remember what is available. [@DavidMStraub Another need for type hints on primary objects and their attributes.]
I had to mull over why this might be anything new to the use or handling of filters.
My original thought was just additional rules that would be more viable with multi-stage filters with the increased efficiency. (If the view re-filtering is applied between every merge, then a filter that takes 45 seconds to run is not viable if there with be 80 separate merge operations. It would add an hour of overhead. But if the filters only took 1/10 as long, then it only adds 6 minutes. That’s tolerable.)
But are you thinking that a whole new level of efficiency being achieved by only applying the filter to the changed records? So the deleted record(s) would be removed from the view and only the surviving record(s) needs to be retested against the filter criteria. So the “partial” view filter refresh might change to a fraction of a second?
I hadn’t even considered that possibility! I was still thinking of the brute force approach simply being faster.
The original survey question was to help prioritize work on the Probably Alive filter. However, in looking at it further, I don’t think it can be sped-up much.
However, in working with a 100k person test tree, my recommendation is the following flow:
select all people with no filters (Control+A)
remove the tag “Probably Alive”
select the filter with rule “Probably Alive” and apply it
select all matching people (Control+A)
add the tag “Probably Alive”
Using the tag is much faster (reduced from 25 seconds down to about 3 seconds in Gramps 6.0).
NOTE: if you edit people that could change the estimates of whether someone is alive or not, you’ll have to repeat the above.
Could I ask why there is all this discussion about a totally flawed
concept, if this is intended for use when you wish to publish a tree
then the concept should be Provably Deceased.
My Rules for this are
Provably born 125 years before current date (Statistically very few
have reached this age)
Born before 1939 and Provably Deceased for 10 years. (Any 126+ year
old who died within the last 10 years would be excluded)
Everything that does not comply with this to me is either speculation or
published at the publishers risk.
Or am I missing something useful with the Probably Alive Filter
phil
I think “probably alive” would be better named “possibly alive”, if its purpose (e.g. when exporting) is to conservatively treat people as still living (for privacy reasons) if they are not provably deceased. A person born 100 years ago is possibly alive, but probably not alive.