Auxiliary XML format documentation?

Development
1

The Backup Omissions wiki page list 4 auxiliary XML documents that Gramps generates:

The primary .gramps XML format is documented in https://www.gramps-project.org/xml/. Where are the docs for the auxiliary XML files?

This question was prompted while collating a list of Feature Requests for add-on custom filter Rules in MantisBT. The long-term goal being productive exercises of trying to create the rules in SuperTool … while also clearing some outstanding requests. And to use the experience to improve the wiki on filters and a writing rules workflow. (Particularly, using SuperTool, FilterParams, & the Query Gramplet in concert. With an eye towards a workflow on optimizing & converting a privately created rule into a public add-on.)

I was wondering if the XML formats included chunks to make the file more failsafe?

Specifically:

  • Since reports in Books.xml tend to reference custom filters, does the format embed a trimmed copy of the custom_filters.xml as a failback?
  • Since custom_filters.xml may reference add-on Rules, is a chunk listing requisite add-on Rules included for validation purposes?
2

Hum… I’m renaming my filters. My books will probably not work anymore…

Thanks for this post.

1 Like
3

Do you know how filters are stored in books.xml ?

In that entry:

  <book name="Legoux, André" database="G:\8. Données techniques\Données\Gramps\Gramps 5\62d2ad71">
    <item name="simple_book_title" trans_name="Page de garde">
      <option name="footer" type="unicode" value=""/>
      <option name="imgid" type="unicode" value="O00002"/>
      <option name="imgsize" type="int" value="0"/>
      <option name="subtitle" type="unicode" value=""/>
      <option name="title" type="unicode" value="André Legoux"/>
      <style name="default"/>
    </item>
    <item name="indiv_complete" trans_name="Fiche individuelle complète">
      <option name="cites" type="bool" value="True"/>
      <option name="date_format" type="int" value="1"/>
      <option name="filter" type="int" value="0"/>
      <option name="images" type="bool" value="True"/>
      <option name="inc_id" type="int" value="1"/>
      <option name="incl_attrs" type="bool" value="True"/>
      <option name="incl_census" type="bool" value="True"/>
      <option name="incl_notes" type="bool" value="True"/>
      <option name="incl_private" type="bool" value="False"/>
      <option name="incl_relname" type="bool" value="True"/>
      <option name="incl_tags" type="bool" value="False"/>
      <option name="incsrcnotes" type="bool" value="True"/>
      <option name="living_people" type="int" value="99"/>
      <option name="name_format" type="int" value="1"/>
      <option name="pageben" type="bool" value="True"/>
      <option name="pid" type="unicode" value="Bk00004__"/>
      <option name="place_format" type="int" value="-1"/>
      <option name="sections" type="unicode" value="True,True,True,True,True,True,True,True,True"/>
      <option name="sort" type="bool" value="True"/>
      <option name="trans" type="unicode" value="default"/>
      <option name="years_past_death" type="int" value="0"/>
      <style name="default"/>
    </item>
    <item name="SourcesCitationsReport" trans_name="Rapport de sources et citations">
      <option name="date_format" type="int" value="1"/>
      <option name="filter" type="int" value="2"/>
      <option name="footer" type="unicode" value="2021, Généalogie de Patrice Legoux - https://bit.ly/PlxNotionHome"/>
      <option name="showperson" type="bool" value="True"/>
      <option name="subtitle" type="unicode" value="Source et citations"/>
      <option name="title" type="unicode" value="Sources ayant trait à Legoux, André"/>
      <option name="trans" type="unicode" value="default"/>
      <style name="default"/>
    </item>
    <item name="table_of_contents" trans_name="Table des matières">
      <option name="trans" type="unicode" value="default"/>
      <style name="default"/>
    </item>
    <paper name="A3"/>
    <orientation value="0"/>
    <metric value="0"/>
    <size value="29.692600 42.011600"/>
    <margin number="0" value="2.540000"/>
    <margin number="1" value="2.540000"/>
    <margin number="2" value="2.540000"/>
    <margin number="3" value="2.540000"/>
    <format name="pdf"/>
    <output name="G:\0. Données généalogiques\2. Individus\LEGOUX\Legoux, André\20220820, Livre.pdf"/>
  </book>

the only one filter entry I find is that one:

      <option name="filter" type="int" value="2"/>

I don’t have any filter named 2 so I think it’s a kind of index but an index in what list?

When I open another book with filters on Places (image bellow), I can see filters used aren’t those needed for that book dedicated to a specific place. So if it’s an index it doesn’t match filters when filter lists change.

This seems really to be an issue.

image
image1103Ă—1375 126 KB

1 Like
4

books.xml saves the report options for each report. In the report options filters are stored as enumerated lists (index, filter_name). In books.xml only the value of a filter is saved and not its name. The number 2 means the filter is 2nd or 3rd item in the filter list (depending if the first filter/item starts with 0 or 1) used in the SourcesCitationsReport.

1 Like
5

The ability to do Add-on Rules is relatively new. It introduces a new way that stored files can fail and where Gramps needs to handle the failure gracefully.

I can think of 2 specific ways that Custom Rules can decay. Others can probably think of other interactions need to be submitted for enhanced error handling?

  • a Custom Filter references an add-on query rule that is not installed
  • a multi-stage (or multi-rule) Custom Filter references another Custom Filter that has been deleted or renamed.

Besides being referenced by name in the custom_filters.xml, bad Rules can be referenced in:

  • books.xml and report_options.xml (by an option name="filter", a VERY fragile index)
  • tool_options.xml (by type and name)

This problem is similar to link rot for URLs.

The project probably needs to:

  • define the methods for gracefully handling the 2 variants of degraded Custom Filters
  • harmonize the XML reference to filters

Example reference by index:
<option name="filter" value="5"/>

Example reference by type and index:

  <option name="media" value="True"/>
  <option name="media_filter" value="4"/>

Example of Custom Filter referenced by name from within another Custom Filter:

     <rule class="MatchesFilter" use_regex="False">
       <arg value="Clipboard"/>
     </rule>
6

Why not assigning a handle to filters and saving it in custom_filters.xml, handle which could be used in reports or books, replacing that index

1 Like
9

Note that one of the 5.2 enhancements is to “Add header to XML fragments file. A header containing a charset is required by xgettext.”

  • Fix Language update_po.py to scan for XML fragments in Python files.
    PR#826 MantisBT#11151

This means that XML fragments embedded in Python files will be more recognizable as being for translation.

Another 5.2 enhancement was added for adding a comment that could help identify .ini files:

  • Improvement: Add optional support for comments section at top of ini in the save method
    PR#1378
10

This was just a bug fix for the translation of embedded XML.

1 Like
11

I was hoping to find headers for all XML files in the future. If the type of Gramps XML were identifiable, tools like the Text Import might be able to be enhanced to import infrastructure data (like filtersets in custom_filters.xml files) in addition to Tree data from .gramps files.