Extracting constants from python code

Help
1

I’m experimenting with an SQLite export of my database, creating my own views and such. It would be nice if the export included a table of the various constant strings that Gramps uses, so that I could join to it in order to decode the integer values contained in the other tables. So, I’m going to create my own table of constants, but would like to minimize the copying and pasting and reformatting involved.

To take just one example, in nametype.py I see:

UNKNOWN = -1
CUSTOM = 0
AKA = 1
BIRTH = 2
MARRIED = 3

_CUSTOM = CUSTOM
_DEFAULT = BIRTH

_DATAMAP = [
    (UNKNOWN, _("Unknown"), "Unknown"),
    (CUSTOM, _("Custom"), "Custom"),
    (AKA, _("Also Known As"), "Also Known As"),
    (BIRTH, _("Birth Name"), "Birth Name"),
    (MARRIED, _("Married Name"), "Married Name"),
    ]

What I need to create is a set of rows something like this:

“NameType”,-1,“Unknown”
“NameType”,0,“Custom”
“NameType”,1,“Also Known As”
“NameType”,2,“Birth”
“NameType”,3,“Married”

I don’t know enough python to do it unsupervised, so I’m hoping that someone can suggest a snippet of code that I could run in the Python Shell gramplet, that would reference each GrampsType object in turn and magically create the desired output.

Or maybe there’s another way?

2

Just to clarify, I’m aware that the tables in the export contain columns such as “name_type0” and “name_type1” but the type1 columns are empty. So maybe this is a bug in the export?

3

The SQLITE addon was an experimental test of a way to store data in SQLite. It is what it is, so I don’t think we can really call anything a bug. The various ‘types’ that are exported are stored in Gramps as GrampsType Python Classes. For standard types, the integer part “_type1" is all that is present. For custom types the string part "_type2” is used instead with the integer value set to that particular types CUSTOM.

If you want to experiment with making this export a bit better for your purposes, you could modify the Python code wherever the types are exported. For example
in the ExportSQL.py file at 634 we have

def export_event(db, data):
    (handle, gid, the_type, date, description, place_handle,
     citation_list, note_list, media_list, attribute_list,
     change, tag_list, private) = data
    db.query("""INSERT INTO event (
                 handle,
                 gid,
                 the_type0,
                 the_type1,
                 description,
                 change,
                 private) VALUES (?,?,?,?,?,?,?);""",
             handle,
             gid,
             the_type[0],
             the_type[1],
             description,
             change,
             private)

If you change the “the_type[1],” at 649 to “str(the_type),” it would insert the translated string for the type into the exported SQL for “the_type1” column. Or you could use “the_type.xml_str(),” instead to always use the English string.

You would have to do this for all the various types that you want to work with in the exported SQLITE db.

4

Thanks, Paul.

When I use str(the_type), the export contains values like “(1, ‘’(” for the_type1 (that’s actually two single quotes after the 1-comma-space), whereas previously it was just an empty string “”.

When I use the_type.xml_str(), I get AttributeError: ‘tuple’ object has no attribute ‘xml_str’

5

Oops… I missed the fact that we had ‘serialized’ data at that point. So the problem is a bit more difficult. In that particular situation, the fix to obtain the string would be to change “the_type[1],” to “str(EventType(the_type)),”. And you would have to have imported EventType at the top of the module with the other imports. “from gramps.gen.lib import EventType, AttrType, ChildrefType” and all the other types of interest. The more difficult problem is just what type is needed at each point in the code. For that https://gramps-project.org/wiki/index.php/Gramps_Data_Model should help you figure out what ‘type’ to use. Sorry that this is getting more involved…

6

Thanks again, Paul. That works fine now for EventType. I should be able to figure out the other ones.

FYI, in case anyone else wants this, here are the changes I made in my copy of ExportSQL.py.

Added these Gramps modules:

from gramps.gen.lib.attrtype import AttributeType
from gramps.gen.lib.eventroletype import EventRoleType
from gramps.gen.lib.eventtype import EventType
from gramps.gen.lib.familyreltype import FamilyRelType
from gramps.gen.lib.childreftype import ChildRefType
from gramps.gen.lib.nameorigintype import NameOriginType
from gramps.gen.lib.nametype import NameType
from gramps.gen.lib.notetype import NoteType
from gramps.gen.lib.placetype import PlaceType
from gramps.gen.lib.repotype import RepositoryType
from gramps.gen.lib.srcmediatype import SourceMediaType
from gramps.gen.lib.srcattrtype import SrcAttributeType
from gramps.gen.lib.styledtexttagtype import StyledTextTagType
from gramps.gen.lib.urltype import UrlType

Changed this line in export_url_list():

# type_[1],
str(UrlType(type_)),

Changed these lines in export_note():

# note_type[1], change, private)
str(NoteType(note_type)), change, private)

# export_markup(db, “note”, handle, markup_code[0], markup_code[1],
export_markup(db, “note”, handle, markup_code[0], str(StyledTextTagType(markup_code)),

(Note: I haven’t used markup in my Notes, so I had no data of my own for testing this. Using the Gramps “example” database, I got results like this:

handle,markup0,markup1,value,start_stop_list
e8b4d90b33ad9a8eb8e741a6a9,0,Bold,"[(0, 10)]"

I don’t know if that’s how the start_stop_list should be formatted. The following line of code, which I did not change, looks like this:

value, str(start_stop_list)) # Not normal form; use eval

I don’t know what that comment means.)

Changed this line in export_event():

# the_type[1],
str(EventType(the_type)),

Changed this line in export_event_ref():

# role[1],
str(EventRoleType(role)),

Changed this line in export_surname():

# origin_type[1], connector)
str(NameOriginType(origin_type)), connector)

Changed this line in export_name():

# name_type[0], name_type[1], group_as,
name_type[0], str(NameType(name_type)), group_as,

Changed this line in export_attribute():

# handle, the_type[0], the_type[1], value, private)
handle, the_type[0], str(AttributeType(the_type)), value, private)

Changed these lines in export_child_ref_list:

# handle, ref, frel[0], frel[1],
handle, ref, frel[0], str(ChildRefType(frel)),
# mrel[0], mrel[1], private)
mrel[0], str(ChildRefType(mrel)), private)

Changed this line in export_datamap_list():

# from_handle, data_type[0], data_type[1],
from_handle, data_type[0], str(SrcAttributeType(data_type)),

Changed this line in export_repository_ref():

# source_media_type[1],
str(SourceMediaType(source_media_type)),

Changed these lines in export_data():

# the_type[0], the_type[1], change,
the_type[0], str(FamilyRelType(the_type)), change,

# handle, gid, the_type[0], the_type[1],
handle, gid, the_type[0], str(RepositoryType(the_type)),

# place_type[0], place_type[1],
place_type[0], str(PlaceType(place_type)),

2 Likes
9

Please note, in the previous message above, “export_data()” should be “exportData()”.

10

There are a few other integer values for which you’ll need to create your own descriptions, if you want them. I created a lookup table like this:

DROP TABLE constants;
CREATE TABLE constants (
table_name CHARACTER(25),
column_name CHARACTER(25),
code INTEGER,
value TEXT);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘quality’, 0, ‘Regular’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘quality’, 1, ‘Estimated’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘quality’, 2, ‘Calculated’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘modifier’, 0, ‘Regular’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘modifier’, 1, ‘Before’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘modifier’, 2, ‘After’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘modifier’, 3, ‘About’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘modifier’, 4, ‘Range’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘modifier’, 5, ‘Span’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘modifier’, 6, ‘Text only’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘newyear’, 0, ‘January 1’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘newyear’, 1, ‘March 1’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘newyear’, 2, ‘March 25’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘newyear’, 3, ‘September 1’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘calendar’, 0, ‘Gregorian’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘calendar’, 1, ‘Julian’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘calendar’, 2, ‘Hebrew’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘calendar’, 3, ‘French’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘calendar’, 4, ‘Persian’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘calendar’, 5, ‘Islamic’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘date’, ‘calendar’, 6, ‘Swedish’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘citation’, ‘confidence’, 0, ‘Very low’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘citation’, ‘confidence’, 1, ‘Low’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘citation’, ‘confidence’, 2, ‘Normal’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘citation’, ‘confidence’, 3, ‘High’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘citation’, ‘confidence’, 4, ‘Very high’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘display_as’, 0, ‘Default’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘display_as’, 1, ‘Surname, Given Suffix’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘display_as’, 2, ‘Given Surname Suffix’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘display_as’, 4, ‘Given’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘display_as’, 5, ‘Main Surnames, Given Patronymic Suffix Prefix’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘sort_as’, 0, ‘Default’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘sort_as’, 1, ‘Surname, Given Suffix’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘sort_as’, 2, ‘Given Surname Suffix’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘sort_as’, 4, ‘Given’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘name’, ‘sort_as’, 5, ‘Main Surnames, Given Patronymic Suffix Prefix’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘person’, ‘gender’, 0, ‘Female’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘person’, ‘gender’, 1, ‘Male’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘person’, ‘gender’, 2, ‘Unknown’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘note’, ‘format’, 0, ‘Unformatted’);
INSERT INTO constants (table_name, column_name, code, value) VALUES (‘note’, ‘format’, 1, ‘Preformatted’);

1 Like