Gramps forgot my custom roles and types after re-import

I’ve made export to .gramps format, renamed some my custom surname types, custom roles, custom association names. And then made import. It works good except one thing. Gramps shows the new custom types, custom association names, custom roles, but it forgot them - it does not propose them in dropdown when I type several first letters. How to fix it? Thanks.

There are 3 likely possibilities:
2 Built-in tools:

Restart Gramps after each change to test whether the Custom Type list is rebuilt for the GUI popups.

If that doesn’t help, try the Types Cleanup Tool addon. It might require renaming a Type to make it freshen the list. You can rename it back again after testing.

Let us know which resolves the problem.

1 Like

@emyoulation nothing.

  1. restart - no changes
    2-3. the tools - no changes
  2. Types Cleanup Tool. I created another event with new custom type Birth2 and then used this tool to rename Birth2 → Birth. Then checked the Custom list of my events. This is empty. But anyway I know, that I use custom event types.

And one more thing. Now Gramps asks again each name - is it male or female. It doesn’t use database data as earlier.

You can use the Family Tree RepairRebuild Gender Statistics built-in Tool for that one.

1 Like

@emyoulation yeah, the names are indexed now, it no asks gender each time. Thanks!
As for other missing indexes - this is not fixed yet. I mean: custom event types and custom surname types

I’m looking to see if there is a way to determine what’s happening. While unlikely to be able to find a resolution, I hope to help isolate the problem. It sounds like a list is saved somewhere but has become corrupted.

Currently looking at the master/gramps/gui/editors/editeventref.py (Event Reference Editor) for where it gathers its list of Roles and Events.

            self.db.get_event_roles(),
            custom_values=self.get_custom_events(),

(By the way, I’ve never experienced the Edit Associations dialog showing a list of Association types. I’m not certain it ever had that ability. There is actually an Enhancement Request 11102 to improve that Editor dialog. It requests that the text box be upgraded to an editable pop-up list.)

1 Like

You are right about Association types. I made renaming but it did not propose custom association names earlier. So, this is definitely not a problem currently.

Here is the gramps/gui/editors/editeventref.py code

yurii@yurii-desktop:/usr/lib/python3/dist-packages/gramps/gui/editors$ cat editeventref.py
#
# Gramps - a GTK+/GNOME based genealogy program
#
# Copyright (C) 2000-2006  Donald N. Allingham
#               2009       Gary Burton
#               2011  Michiel D. Nauta
# Copyright (C) 2011       Tim G L Lyons
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#

#-------------------------------------------------------------------------
#
# Python modules
#
#-------------------------------------------------------------------------

#-------------------------------------------------------------------------
#
# gramps modules
#
#-------------------------------------------------------------------------
from gramps.gen.const import GRAMPS_LOCALE as glocale
_ = glocale.translation.sgettext
from gramps.gen.lib import EventType, NoteType
from gramps.gen.db import DbTxn
from ..glade import Glade
from .displaytabs import (CitationEmbedList, NoteTab, GalleryTab,
                         EventBackRefList, EventAttrEmbedList)
from ..widgets import (PrivacyButton, MonitoredEntry,
                     MonitoredDate, MonitoredDataType, MonitoredTagList)
from .editreference import RefTab, EditReference

from .objectentries import PlaceEntry

from gramps.gen.const import URL_MANUAL_SECT2

#-------------------------------------------------------------------------
#
# Constants
#
#-------------------------------------------------------------------------

WIKI_HELP_PAGE = URL_MANUAL_SECT2
WIKI_HELP_SEC = _('manual|Event_Reference_Editor_dialog')

#-------------------------------------------------------------------------
#
# EditEventRef class
#
#-------------------------------------------------------------------------
class EditEventRef(EditReference):

    def __init__(self, state, uistate, track, event, event_ref, update):
        EditReference.__init__(self, state, uistate, track, event, event_ref,
                               update)
        self._init_event()

    def _local_init(self):

        self.top = Glade()
        self.set_window(self.top.toplevel,
                        self.top.get_object('eer_title'),
                        _('Event Reference Editor'))
        self.setup_configs('interface.event-ref', 600, 450)

        self.define_warn_box(self.top.get_object("eer_warning"))
        self.define_expander(self.top.get_object("eer_expander"))
        self.share_btn = self.top.get_object('share_place')
        self.add_del_btn = self.top.get_object('add_del_place')

        tblref = self.top.get_object('table64')
        notebook = self.top.get_object('notebook_ref')
        #recreate start page as GrampsTab
        notebook.remove_page(0)
        self.reftab = RefTab(self.dbstate, self.uistate, self.track,
                              _('General'), tblref)
        tblref = self.top.get_object('table62')
        notebook = self.top.get_object('notebook')
        #recreate start page as GrampsTab
        notebook.remove_page(0)
        self.primtab = RefTab(self.dbstate, self.uistate, self.track,
                              _('_General'), tblref)

    def _post_init(self):
        date = self.top.get_object('eer_date_entry')
        if not date.get_text_length():
            date.grab_focus();

    def _init_event(self):
        if not self.db.readonly:
            self.commit_event = self.db.commit_event
            self.add_event = self.db.add_event

    def get_custom_events(self):
        return sorted(self.db.get_event_types(), key=lambda s: s.lower())

    def _connect_signals(self):
        self.define_ok_button(self.top.get_object('ok'),self.ok_clicked)
        self.define_cancel_button(self.top.get_object('cancel'))
        self.define_help_button(self.top.get_object('help'),
                WIKI_HELP_PAGE, WIKI_HELP_SEC)

    def _connect_db_signals(self):
        """
        Connect any signals that need to be connected.
        Called by the init routine of the base class (_EditPrimary).
        """
        self._add_db_signal('event-rebuild', self.close)
        self._add_db_signal('event-delete', self.check_for_close)
        self._add_db_signal('place-delete', self.place_delete)
        self._add_db_signal('place-update', self.place_update)

    def _setup_fields(self):

        self.ref_privacy = PrivacyButton(
            self.top.get_object('eer_ref_priv'),
            self.source_ref, self.db.readonly)

        self.descr_field = MonitoredEntry(
            self.top.get_object("eer_description"),
            self.source.set_description,
            self.source.get_description,
            self.db.readonly)

        self.gid = MonitoredEntry(
            self.top.get_object("gid"),
            self.source.set_gramps_id,
            self.source.get_gramps_id,
            self.db.readonly)

        self.tags = MonitoredTagList(
            self.top.get_object("tag_label"),
            self.top.get_object("tag_button"),
            self.source.set_tag_list,
            self.source.get_tag_list,
            self.db,
            self.uistate, self.track,
            self.db.readonly)

        self.place_field = PlaceEntry(
            self.dbstate,
            self.uistate,
            self.track,
            self.top.get_object("eer_place"),
            self.top.get_object("eer_place_event_box"),
            self.source.set_place_handle,
            self.source.get_place_handle,
            self.share_btn,
            self.add_del_btn)

        self.ev_privacy = PrivacyButton(
            self.top.get_object("eer_ev_priv"),
            self.source, self.db.readonly)

        self.role_selector = MonitoredDataType(
            self.top.get_object('eer_role_combo'),
            self.source_ref.set_role,
            self.source_ref.get_role,
            self.db.readonly,
            self.db.get_event_roles()
            )

        self.event_menu = MonitoredDataType(
            self.top.get_object("eer_type_combo"),
            self.source.set_type,
            self.source.get_type,
            self.db.readonly,
            custom_values=self.get_custom_events())

        self.date_check = MonitoredDate(
            self.top.get_object("eer_date_entry"),
            self.top.get_object("eer_date_stat"),
            self.source.get_date_object(),
            self.uistate,
            self.track,
            self.db.readonly)

    def _create_tabbed_pages(self):
        """
        Create the notebook tabs and inserts them into the main
        window.
        """

        notebook = self.top.get_object('notebook')
        notebook_ref = self.top.get_object('notebook_ref')

        self._add_tab(notebook, self.primtab)
        self._add_tab(notebook_ref, self.reftab)
        self.track_ref_for_deletion("primtab")
        self.track_ref_for_deletion("reftab")

        self.srcref_list = CitationEmbedList(self.dbstate,
                                             self.uistate,
                                             self.track,
                                             self.source.get_citation_list())
        self._add_tab(notebook, self.srcref_list)
        self.track_ref_for_deletion("srcref_list")

        self.attr_list = EventAttrEmbedList(self.dbstate,
                                            self.uistate,
                                            self.track,
                                            self.source.get_attribute_list())
        self._add_tab(notebook, self.attr_list)
        self.track_ref_for_deletion("attr_list")

        self.note_tab = NoteTab(self.dbstate,
                                self.uistate,
                                self.track,
                                self.source.get_note_list(),
                                notetype=NoteType.EVENT)
        self._add_tab(notebook, self.note_tab)
        self.track_ref_for_deletion("note_tab")

        self.note_ref_tab = NoteTab(self.dbstate,
                                    self.uistate,
                                    self.track,
                                    self.source_ref.get_note_list(),
                                    notetype=NoteType.EVENTREF)
        self._add_tab(notebook_ref, self.note_ref_tab)
        self.track_ref_for_deletion("note_ref_tab")

        self.gallery_tab = GalleryTab(self.dbstate,
                                      self.uistate,
                                      self.track,
                                      self.source.get_media_list())
        self._add_tab(notebook, self.gallery_tab)
        self.track_ref_for_deletion("gallery_tab")

        self.backref_tab = EventBackRefList(self.dbstate,
                             self.uistate,
                             self.track,
                             self.db.find_backlink_handles(self.source.handle),
                             self.enable_warnbox)
        self._add_tab(notebook, self.backref_tab)
        self.track_ref_for_deletion("backref_tab")

        self.attr_ref_list = EventAttrEmbedList(
            self.dbstate,
            self.uistate,
            self.track,
            self.source_ref.get_attribute_list())
        self._add_tab(notebook_ref, self.attr_ref_list)
        self.track_ref_for_deletion("attr_ref_list")

        self._setup_notebook_tabs( notebook)
        self._setup_notebook_tabs( notebook_ref)

    def build_menu_names(self,eventref):
        if self.source:
            event_name = str(self.source.get_type())
            submenu_label = _('Event: %s')  % event_name
        else:
            submenu_label = _('New Event')
        return (_('Event Reference Editor'),submenu_label)

    def ok_clicked(self, obj):

        if self.source.handle:
            with DbTxn(_("Modify Event"), self.db) as trans:
                self.commit_event(self.source,trans)
        else:
            if self.check_for_duplicate_id('Event'):
                return
            with DbTxn(_("Add Event"), self.db) as trans:
                self.add_event(self.source,trans)
            self.source_ref.ref = self.source.handle

        if self.update:
            self.update(self.source_ref,self.source)

        self.close()

    def place_update(self, hndls):
        ''' Place changed outside of dialog, update text if its ours '''
        handle = self.source.get_place_handle()
        if handle and handle in hndls:
            place = self.db.get_place_from_handle(handle)
            p_lbl = "%s [%s]" % (place.get_title(), place.gramps_id)
            self.top.get_object("eer_place").set_text(p_lbl)

    def place_delete(self, hndls):
        ''' Place deleted outside of dialog, remove it if its ours'''
        handle = self.source.get_place_handle()
        if handle and handle in hndls:
            self.source.set_place_handle(None)
            self.top.get_object("eer_place").set_markup(
                self.place_field.EMPTY_TEXT)
            self.place_field.set_button(False)
yurii@yurii-desktop:/usr/lib/python3/dist-packages/gramps/gui/editors$ 

I have a question and enhancement.
Why do we use a lot of rebuild tools for everything? It would be better load all custom types, custom roles, etc… to memory or to a separate indexing table or get each time these lists directly from the DB. Databases are very fast now, even if it will be 1.000.000 records it should work fast.

I think it is an artifact of the organic growth of Gramps.

Just like the new 5.2 feature to allow Column widths of Object Editor dialogs and views to be adjusted and saved. The new data is stored in a new place: a [spacing] chunk in the Gramps.ini file. (A file that already exhibits occasional corruptions.) But it is likely that the storage method will get corrupted somehow in the future. (For instance, moving between hardware configurations… large virtual monitor to small… might result in an invalid width/position.)

But I doubt that any ability to recover from such corruptions was added at the same time. Particularly since there still isn’t a tool to clean the Gramps.ini file of corrupt data. You have to do that with a manual editor.

1 Like

They are stored in the metadata table.

2 Likes

Is there any tool that will validate or repair the metadata table? Or can we delete something to force a rebuild?

I’ve intermittently had the custom type lists not do the suggest-as-you-type on the Linux 5.1.6 box. (Not so much on the Windoze test box.) And it really slows down the data-entry.

1 Like

The metadata together with all the indexes are rebuilt when the restore a backup. Just export your tree and then import it into an empty database.

This was the recommended method before the tool was written.

2 Likes

That recovered the suggestions on my test files on the Windows 5.2 beta 3 box. Thanks! @Urchello You started this thread after an import. Were you importing into an existing Tree or a fresh, empty one? And does saving a backup (no media) and importing into a fresh, empty Tree resolve the problem? (I definitely recommend backup over export.)

Why are there Zero filesize meta_data.db files in all the 5.1.x and 5.2b3 database folder? Particularly since there are meta data tables are inside the sqlite.db files

1 Like

One factor that has not been discussed is the manual edit of the xml file. Was a mistake made in the edits that when imported caused the issue? The fields @Urchello edited are the ones causing the problems. What was the need to do a manual edit that the Type Cleanup could not handle?

Whenever I have edited the XML file I have always used the editor’s search and replace function to minimize the potential for errors.

1 Like

@Nick-Hall i exported to gramps xml again and imported it again. And now all custom types appeared. Works good!
I think it was any typo in the xml-file and something failed after the first import.

I needed make different renames and merge some types and specific case about “Witness” - “Witness for” pairs:

1 Like

This is the way it is done.

Custom types are saved in metadata when you terminate a session. Metadata is reloaded when you open a database. Depending on metadata, it goes in various Python objects.

But, back on your initial problem of renaming custom types, it won’t work. I mean it won’t change the name in existing records because there is a design flaw IMHO. A custom type is implemented as a pair made of built-in CUSTOM code (integer) and a string to particularise CUSTOM. The string is not kept in the type definition object (there is a single slot per built-in type and CUSTOM is multi-valued). The particular string is stored in the type instance of the record (event, place, name, …). This makes it impossible to rename globally a custom name, e.g. when you misspelled it, because there is no function for it.

If you change the string in the XML only where the set of custom types is enumerated, this has no effect on the uses of this custom type. You must change all occurrences of the string (hoping it is not also used in notes or other textual “comments”.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.