Is there any way to bulk import events?

Help
1

Data entry in Gramps is rough for me. So many windows and clicks to get everything attached. I am hoping that there is easier way. Inputting 32 events so far for this one person has taken HOURS. I have like 27 more probate events this person, and excel is so much faster!

Data in my spreadsheet: An event date, event type (including custom, that have already been made in Gramps), a place (in gramps), its description, a event note, a transcription note, a source (in gramps), a citation, a citation media url, a citation note.

I’ve been looking at the CSV import and there has been a lot of requests to enhance the import and even a request, but it doesn’t seem like there is any movement on it since 2024. The ExceltoGED is too limited, and the SuperTool is a little too advanced for me without examples, I am not a programmer. I also couldn’t find a updated data model for 6.x.

Is there any way to do this? Is this something that other people are interested in?

6.0.8 - Windows

2

Actually, I just fixed a bug this week in the CSV import. What limitation is currently holding you back? If you have a sample (even fake) of what doesn’t work, or what you are requesting, please attach that too.

On a related note, Gramps 6.2 will also have a PDF import. It uses the same code under the hood, but you can fill in bulk data in a PDF file. [Thanks to @emyoulation for the idea!]

3

So it’s my (mis)understanding that the bulk import doesn’t support anything beyond:

Place Title Name Type Latitude Longitude Code Enclosed_by Date
Person Surname Given Call Nickname Suffix Prefix Title Gender Birth date Birth place Birth source Baptism date Baptism place Baptism source Death date Death place Death source Burial date Burial place Burial source Note
Marriage Husband Wife Date Place Source Note

I’ve read all the forum threads, the addons pages, and have only seen that the import is limited. Am I missing a way to import events beyond those listed in the person section above?

Data Example:
Import for Joel Daniels:

  • Event date
  • Event type (including custom)
  • Event Place
  • Event Description
  • Event Note (type: event)
  • Event Note (type: Transcription)
  • Citation
  • Citation Source
  • Citation Note
  • Citation Gallery Media

Example

4

I guess if we add media to Citations, we mights as well add it to all? What do you think of the following?

Event table — all columns:

event,    eventtype,  date,        place,    description,        source,          note,                           note_type,  tag,      person,   family,   role,    media,                         media_description,    media_date
E1,       Census,     1901-04-01,  London,   Household of five,  1901 Census,     Transcribed from microfilm,     Transcript, Verified, [I0001],  ,         Primary, /scans/1901_pg12b.jpg,         Census page 12b,      2024-05-01
E2,       Emigration, 1902-03-15,  Hamburg,  ,                   ,                Cross-ref with arrival record,  Research,   ,         [I0001],  ,         Primary, ,                              ,
E3,       Marriage,   1885-06-20,  [P0012],  ,                   Parish register, ,                               ,           ,         ,         [F0003],  Family,  /scans/marriage_cert_1885.tif, Mueller marriage cert,
E4,       Burial,     1932,        ,         ,                   ,                ,                               ,           ,         [I0002],  ,         Primary, /photos/gravestone.jpg,        Mueller gravestone,   2023-09-14
[E0042],  Census,     1851-03-30,  Bristol,  ,                   1851 Census,     ,                               ,           ,         [I0003],  ,         Primary, ,                              ,

Citation table — all columns:

citation, source,          page,  date,       confidence, note,                                  note_type,  tag,      person,   family,   event,    place,    media,                         media_description,    media_date
C1,       1901 Census,     12b,   1901-04-01, 3,          Original held at TNA; ref RG13/1234,  General,    Verified, [I0001],  ,         E1,       ,         /scans/1901_pg12b.jpg,         Census page 12b,      2024-05-01
C2,       Passenger list,  ,      1902-03-15, 2,          Verbatim: "Johann Müller, age 34",    Transcript, ,         ,         ,         E2,       ,         ,                              ,
C3,       Parish register, p.47,  ,           4,          ,                                      ,           ,         ,         [F0003],  E3,       [P0012],  /scans/marriage_cert_1885.tif, Mueller marriage cert,
C4,       Cemetery survey, ,      ,           1,          Not personally verified,               Research,   ,         [I0002],  ,         E4,       ,         ,                              ,
[C0010],  1851 Census,     8a,    1851-03-30, 3,          ,                                      ,           ,         ,         ,         [E0042],  ,         ,                              ,

Notes on the columns:

  • confidence accepts integers 0–4 or strings: Very Low, Low, Normal, High, Very High.
  • role strings match EventRoleType names: Primary, Witness, Clergy, Celebrant, Informant, Family, etc.
  • note_type strings match NoteType names: General, Research, Transcript, Source text, Event Note, Citation, etc. Defaults to Event Note / Citation when omitted.
  • media and media_description / media_date can appear on the same row as all other columns — they attach one media object per row. Multiple media on the same event or citation requires multiple rows referencing the same event/citation ref.
5

I think the limitations that hit people are:

Only a limited set of Event types are supported:

  • Birth
  • Baptism
  • Death (one of 2 event types that support Description data, as deathcause)
  • Burial
  • Occupation (one of 2 event types that support Description data, as occupationdescr)
  • Residence

And that, while assign a Source is possible in CSV objects, Citations cannot be assigned.

The single most frequent request is for: mass additions of a “Military Service” record for all the members of a unit. Where rank, unit information is flowed into the description. And they want to cite the Source page for the soldiers.

It would be useful to have an infinitely re-usable Event csv section where the options were:
PersonID/FamilyID, EventID, EventType, EventDate, EventDescr, EventPlace, EventPlaceID, Source, CitationID, CitationVolPage, CitationDate, Note, attributetype, attributevalue, attributesource, attributecitationID, role

And/or maybe a separate Citation section where you can fill out a Citation for re-use throughout the CSV. (When I’m doing CSV, I am usually working from a page in a sources, or maybe several successive pages. So I’d really, REALLY like to start the CSV with creating the Citation(s) and use that [citationID] for virtually every element in the CSV.

6

I suspect that a Media attachment has the more potential to be fragile and break an import. The paths, delimiters, case and space sensitivity are likely to choke any validation on import.

So maybe Media should be a separate section with special fault-tolerance. Where the path can safely fail and not cause the whole import to stop.

7

Yes, that looks like an oversite. Other types (and custom types) will be added for all types.

Can you make a specific suggestion of what the a sample row would look like in CSV?

Can you show a specific example of what you mean?

Are you fixing bugs before I have created them?!

8

I think that I am not a good example of how Military Service is recorded. Gramps does not support that data very well yet.

(Before any decided that its deficiency is a reason to bring up main-subevents again, that is a discussion for another time. It would require a major redesign affecting a broad swatch of Gramps. And this should remain a discussion about patching a single import functionality.)

I know that somewhere down the I will have to figure out a way to have the Description data sortable, filterable but remain highly visible. So I’m trying to make the descriptions somewhat structured but still readable. Having done entries over a decade, it hasn’t been as consistent as I’d like.

I used a workaround in bulk imports for military that I’ve done. I imported roll call in a County History book for a local Civil War regiment : the companies A-F of the 100th Regiment (nicknamed the “Roundheads”) of the Pennsylvania Volunteer Infantry. And all the servicemen (and spouses) in a particular section’s row of gravestones of a US National Cemetery. (Neighbors in my parent’s row.) For both of these, I was able to use the Occupation Event because it had the necessary “Description”. Then coverted the type from “Occupation” to “Military Service” or “Burial”

image
image2440×832 288 KB

Here’s a posting from another user wanting to do Military Service spreadsheet import:

9

So I spent some time refining the CSV import/export. It now handles: Place, Person, Marriage, Family, Event, Citation, and allows full (and custom) type entries.

The two new row types are:

Event table columns:

event, eventtype, date, place, description, source, note, note_type, role, tag, person, family, media, media_description, media_date

Citation table columns:

citation, source, page, date, confidence, note, note_type, tag, person, family, event, place, media, media_description, media_date

The Gramps primary object types that are not handled after the update:

  • Source — citations reference sources by title string, but sources themselves aren’t exported/imported as a table
  • Repository — no table at all
  • Media — no table at all
  • Note — only inline note text on persons/families, not standalone Note objects
  • Tag — exporter writes tag names on some objects, but there’s no Tag table for standalone tag management

@emyoulation the importer stores the media path as-is without checking if the file exists — no error is raised. mimetypes.guess_type() infers the MIME type from the extension alone (no file access). The Media object is committed to the database with whatever path was given, and Gramps will show it as missing when it tries to display it later. This is consistent with how all other Gramps importers behave: path validation is deferred to display time, not import time.

Does this upgrade handle most of the issues with CSV import?

As this is a new feature, it will appear in Gramps 6.2 slated to be released in Oct 2026.

10

Check me on this… PR#2372 looks like users could patch it in as far back as 5.2 if the gramps/plugins/importer/importcsv.py importer and gramps/plugins/export/exportcsv.py exporter were made into addons and the pertinent gramps/plugins/importer/import.gpr.py and gramps/plugins/export/export.gpr.py .gpr.py entries were extracted/adapted from the combined core registration files.

Users would need to ‘hide’ the built-in versions to use them as addons though. (Which is still easier than swapping out protected and sometimes hidden files in the core.)

This would allow some beta testing.

11

This would create a lot of confusion and extra work for the team. And Gramps is already complicated enough! My advice would be to push for faster release cycles.

There could also be a gramps-nightly version that has the latest master changes. But because the PRs aren’t merged until the last seconds before a release, that won’t help either.

12

This is why I love this community! Happy to beta test if needed. Am I misunderstanding about some of the data not having tables? (only experience with relational databases is Access)

13

I was just indicating that currently the CSV import does not cover 100% of the items that possibly could be added. If those items aren’t needed, then they won’t be added.