# Auxiliary XML format documentation?

**URL:** <https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800>\
**Category:** Development\
**Created:** [September 22, 2022, 2:44pm UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800 "2022-09-22T14:44:38Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![emyoulation](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/emyoulation/32/67_2.png) [@emyoulation](https://gramps.discourse.group/u/emyoulation)\
**Post date:** [September 22, 2022, 2:44pm UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/1 "2022-09-22T14:44:38Z")

</div>

The [Backup Omissions](https://www.gramps-project.org/wiki/index.php/Template:Backup_Omissions) wiki page list 4 auxiliary XML documents that Gramps generates:

- filters (`custom_filters.xml`)
- report settings (`report_options.xml`)
- Books (`books.xml` [`gramps/gen/plug/report/_book.py`](https://github.com/gramps-project/gramps/blob/master/gramps/gen/plug/report/_book.py)?)
- holiday sets or country calendars (`holidays.xml`) [[wiki page includes format docs](https://www.gramps-project.org/wiki/index.php/Calendar_tools_holidays#format_of_holidays.xml)]

The primary `.gramps` XML format is documented in [`https://www.gramps-project.org/xml/`](https://www.gramps-project.org/xml/). **Where are the docs for the auxiliary XML files?**

_This question was prompted while collating a list of Feature Requests for add-on custom filter Rules in MantisBT. The long-term goal being productive exercises of trying to create the rules in SuperTool … while also clearing some outstanding requests. And to use the experience to improve the wiki on [filters](https://gramps-project.org/wiki/index.php/Gramps_5.1_Wiki_Manual_-_Filters) and a writing rules workflow. (Particularly, using [SuperTool](https://gramps-project.org/wiki/index.php/Addon:Isotammi_addons#SuperTool), [FilterParams](https://gramps-project.org/wiki/index.php/Addon:Isotammi_addons#FilterParams_tool), & the [Query Gramplet](https://www.gramps-project.org/wiki/index.php/Addon:QueryGramplet) in concert. With an eye towards a workflow on optimizing & converting a privately created rule into a public add-on.)_

I was wondering if the XML formats included chunks to make the file more failsafe?

Specifically:

- Since reports in `Books.xml` tend to reference custom filters, does the format embed a trimmed copy of the `custom_filters.xml` as a failback?
- Since `custom_filters.xml` may reference add-on Rules, is a [chunk listing requisite add-on Rules](https://gramps.discourse.group/t/how-to-save-a-list-of-plugins/1992/14) included for validation purposes?

---

<div class="post-metadata">

**Author:** ![PLegoux](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/plegoux/32/413_2.png) [@PLegoux](https://gramps.discourse.group/u/PLegoux)\
**Post date:** [September 22, 2022, 3:14pm UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/2 "2022-09-22T15:14:07Z")

</div>

> [@emyoulation](#):
>
> Since reports in `Books.xml` tend to reference custom filters, does the format embed a trimmed copy of the `custom_filters.xml` as a failback?

Hum… I’m renaming my filters. My books will probably not work anymore…

Thanks for this post.

---

<div class="post-metadata">

**Author:** ![PLegoux](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/plegoux/32/413_2.png) [@PLegoux](https://gramps.discourse.group/u/PLegoux)\
**Post date:** [September 30, 2022, 7:38am UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/3 "2022-09-30T07:38:55Z")

</div>

Do you know how filters are stored in books.xml ?

In that entry:

```xml
  <book name="Legoux, André" database="G:\8. Données techniques\Données\Gramps\Gramps 5\62d2ad71">
    <item name="simple_book_title" trans_name="Page de garde">
      <option name="footer" type="unicode" value=""/>
      <option name="imgid" type="unicode" value="O00002"/>
      <option name="imgsize" type="int" value="0"/>
      <option name="subtitle" type="unicode" value=""/>
      <option name="title" type="unicode" value="André Legoux"/>
      <style name="default"/>
    </item>
    <item name="indiv_complete" trans_name="Fiche individuelle complète">
      <option name="cites" type="bool" value="True"/>
      <option name="date_format" type="int" value="1"/>
      <option name="filter" type="int" value="0"/>
      <option name="images" type="bool" value="True"/>
      <option name="inc_id" type="int" value="1"/>
      <option name="incl_attrs" type="bool" value="True"/>
      <option name="incl_census" type="bool" value="True"/>
      <option name="incl_notes" type="bool" value="True"/>
      <option name="incl_private" type="bool" value="False"/>
      <option name="incl_relname" type="bool" value="True"/>
      <option name="incl_tags" type="bool" value="False"/>
      <option name="incsrcnotes" type="bool" value="True"/>
      <option name="living_people" type="int" value="99"/>
      <option name="name_format" type="int" value="1"/>
      <option name="pageben" type="bool" value="True"/>
      <option name="pid" type="unicode" value="Bk00004__"/>
      <option name="place_format" type="int" value="-1"/>
      <option name="sections" type="unicode" value="True,True,True,True,True,True,True,True,True"/>
      <option name="sort" type="bool" value="True"/>
      <option name="trans" type="unicode" value="default"/>
      <option name="years_past_death" type="int" value="0"/>
      <style name="default"/>
    </item>
    <item name="SourcesCitationsReport" trans_name="Rapport de sources et citations">
      <option name="date_format" type="int" value="1"/>
      <option name="filter" type="int" value="2"/>
      <option name="footer" type="unicode" value="2021, Généalogie de Patrice Legoux - https://bit.ly/PlxNotionHome"/>
      <option name="showperson" type="bool" value="True"/>
      <option name="subtitle" type="unicode" value="Source et citations"/>
      <option name="title" type="unicode" value="Sources ayant trait à Legoux, André"/>
      <option name="trans" type="unicode" value="default"/>
      <style name="default"/>
    </item>
    <item name="table_of_contents" trans_name="Table des matières">
      <option name="trans" type="unicode" value="default"/>
      <style name="default"/>
    </item>
    <paper name="A3"/>
    <orientation value="0"/>
    <metric value="0"/>
    <size value="29.692600 42.011600"/>
    <margin number="0" value="2.540000"/>
    <margin number="1" value="2.540000"/>
    <margin number="2" value="2.540000"/>
    <margin number="3" value="2.540000"/>
    <format name="pdf"/>
    <output name="G:\0. Données généalogiques\2. Individus\LEGOUX\Legoux, André\20220820, Livre.pdf"/>
  </book>

```

the only one filter entry I find is that one:

```xml
      <option name="filter" type="int" value="2"/>

```

I don’t have any filter named 2 so I think it’s a kind of index but an index in what list?

When I open another book with filters on Places (image bellow), I can see filters used aren’t those needed for that book dedicated to a specific place. So if it’s an index it doesn’t match filters when filter lists change.

This seems really to be an issue.

 ![image](https://global.discourse-cdn.com/free1/uploads/gramps/original/2X/5/5774b5f3d036884c3d9f7dd608b0d3f458cfbb1e.png)

---

<div class="post-metadata">

**Author:** ![Mattkmmr](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/mattkmmr/32/410_2.png) [@Mattkmmr](https://gramps.discourse.group/u/Mattkmmr)\
**Post date:** [October 1, 2022, 2:00pm UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/4 "2022-10-01T14:00:57Z")

</div>

> [@PLegoux](#):
>
> Do you know how filters are stored in books.xml ?

`books.xml` saves the report options for each report. In the report options filters are stored as enumerated lists (index, filter\_name). In books.xml only the value of a filter is saved and not its name. The number 2 means the filter is 2nd or 3rd item in the filter list (depending if the first filter/item starts with 0 or 1) used in the `SourcesCitationsReport`.

---

<div class="post-metadata">

**Author:** ![emyoulation](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/emyoulation/32/67_2.png) [@emyoulation](https://gramps.discourse.group/u/emyoulation)\
**Post date:** [October 8, 2022, 6:25pm UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/5 "2022-10-08T18:25:18Z")

</div>

The ability to do Add-on Rules is relatively new. It introduces a new way that stored files can fail and where Gramps needs to handle the failure gracefully.

I can think of 2 specific ways that Custom Rules can decay. Others can probably think of other interactions need to be submitted for enhanced error handling?

- a Custom Filter references an add-on query rule that is not installed
- a multi-stage (or multi-rule) Custom Filter references another Custom Filter that has been deleted or renamed.

Besides being referenced by name in the `custom_filters.xml`, bad Rules can be referenced in:

- `books.xml` and `report_options.xml` (by an `option name="filter"`, a VERY fragile index)
- `tool_options.xml` (by type and name)

This problem is similar to link rot for URLs.

The project probably needs to:

- define the methods for gracefully handling the 2 variants of degraded Custom Filters
- harmonize the XML reference to filters

Example reference by index:  
` <option name="filter" value="5"/>`

Example reference by type and index:

```auto
  <option name="media" value="True"/>
  <option name="media_filter" value="4"/>

```

Example of Custom Filter referenced by name from within another Custom Filter:

```auto
     <rule class="MatchesFilter" use_regex="False">
       <arg value="Clipboard"/>
     </rule>

```

---

<div class="post-metadata">

**Author:** ![PLegoux](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/plegoux/32/413_2.png) [@PLegoux](https://gramps.discourse.group/u/PLegoux)\
**Post date:** [October 9, 2022, 9:36am UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/6 "2022-10-09T09:36:36Z")

</div>

> [@emyoulation](#):
>
> define the methods for gracefully handling the 2 variants of degraded Custom Filters

Why not assigning a handle to filters and saving it in custom\_filters.xml, handle which could be used in reports or books, replacing that index

---

<div class="post-metadata">

**Author:** ![emyoulation](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/emyoulation/32/67_2.png) [@emyoulation](https://gramps.discourse.group/u/emyoulation)\
**Post date:** [August 5, 2023, 3:37pm UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/9 "2023-08-05T15:37:00Z")

</div>

Note that one of the 5.2 enhancements is to “Add header to XML fragments file. A header containing a charset is required by xgettext.”

- _Fix Language update\_po.py to scan for XML fragments in Python files._   
PR#[826](https://github.com/gramps-project/gramps/pull/826) MantisBT#[11151](https://gramps-project.org/bugs/view.php?id=11151) 

This means that XML fragments embedded in Python files will be more recognizable as being for translation.

Another 5.2 enhancement was added for adding a comment that could help identify `.ini` files:

- _Improvement: Add optional support for comments section at top of ini in the save method_   
PR#[1378](https://github.com/gramps-project/gramps/pull/1378)

---

<div class="post-metadata">

**Author:** ![Nick-Hall](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/nick-hall/32/95_2.png) [@Nick-Hall](https://gramps.discourse.group/u/Nick-Hall)\
**Post date:** [August 5, 2023, 3:40pm UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/10 "2023-08-05T15:40:18Z")

</div>

> [@emyoulation](#):
>
> Note that one of the 5.2 enhancements is to “Add header to XML fragments file. A header containing a charset is required by xgettext.”

This was just a bug fix for the translation of embedded XML.

---

<div class="post-metadata">

**Author:** ![emyoulation](https://yyz2.discourse-cdn.com/free1/user_avatar/gramps.discourse.group/emyoulation/32/67_2.png) [@emyoulation](https://gramps.discourse.group/u/emyoulation)\
**Post date:** [August 5, 2023, 3:52pm UTC](https://gramps.discourse.group/t/auxiliary-xml-format-documentation/2800/11 "2023-08-05T15:52:47Z")

</div>

> [@Nick-Hall](#):
>
> This was just a bug fix for the translation of embedded XML.

I was hoping to find headers for all XML files in the future. If the type of Gramps XML were identifiable, tools like the Text Import might be able to be enhanced to import infrastructure data (like filtersets in `custom_filters.xml` files) in addition to Tree data from `.gramps` files.