Gramps workflow (roughing in a tree)

Gramps 5.1.6 on Ubuntu 20.04 (hosted by Digital Ocean).

I have Gramps Web running, and I’ve been able to enter some data. I’m learning my way around, and starting to plan my workflow.

I have a Word document with a couple hundred people (name, birthdate, marriages, places) that I would like to enter, and I’m trying to figure out my workflow. From playing around, I can tell that I really don’t want to enter a couple hundred people and families using the web UI. I am very comfortable working with text data (CSV or other formats). I have an extensive background in programming, databases and data modeling, so I am leaning to creating the bulk of data manually, e.g. as a CSV file and importing it. The structuring using identifiers like [I0001] and [F00000] is pretty obvious.

Questions about this approach:

  • General advice? Is this sane?

  • Is there any reason to prefer .ged to .csv? The .csv format looks simpler to me.

  • I’ve only entered information on four people so far. Before importing the entire data set (as .csv or .ged) is there some way to wipe data and start clean? In the worst case, I will just delete the four people (and one family) I’ve created manually. Alternatively, is it okay to import data into a non-empty database?

CSV is a good ‘roughing in’ approach using the Desktop version. Use the Gramps Web Sync to start with a Desktop and Web database in sync.

Then, on Gramps Desktop, begin experimenting with the CSV import. (Make a backup before each import.)

I like using the Gramps 5.2 beta because @SNoiraud added the option to select the delimiter. (Changing delimiters is a bit of an Easter Egg – the GUI is hidden in the Edit → Configure… of views with tabular data. ) This means that you do not have to Save a spread sheet to a CSV file, then import the CSV file. With a ‘tab’ delimiter instead of ‘comma’ delimiter, you can simply copy the spreadsheet Cells to the OS clipboard and paste them into the Import Text gramplet.

Note, @dsblank recently posted a patch to the CSV importer plug-in to support Tagging and eliminate unintentional Given Name overwrites.

Take a look at:

You can create a new tree for experimenting or testing imports. The easiest way to start clean is to delete the tree and create a new one. (Or to be safe, start a new tree first and only delete the old tree when you’re confident that will not be a mistake. Rename the old tree so you do not confuse it with what is current.)

Thank you for those ideas.

Pardon a newbie question: What do you mean by “Gramps Desktop”? Is that Gramps running locally in Docker?

Fixing data in Excel sounds like a nightmare. I am proficient in Python, and have homegrown tools for operating on CSV and other forms of data. These have proven very effective in a variety of projects.

So what I’d like to do is to treat my CSV files as my primary source that I operate on while cleaning the data. Then periodically, I would wipe the Gramps database and import all the CSVs.

  • How can I wipe the Gramps database? I would think that there are Postgres tables like Place, Person, Family, etc., and simply writing a psql script to truncate those tables (leaving admin tables in place) would do the trick.

  • Or perhaps wipe through the API? That seems to be oriented exclusively to working on one object (Person, Family, etc.) at a time. So wiping that way seems like it would be quite slow.

(Just saw your last response)

Wiping the tree sounds like it does what I need. How do I go about doing that? I looked around the web UI and didn’t see anything that looks appropriate. Is a Tree the same thing as a Family Tree? If my CSVs are my primary source of data, deleting a tree is a low-risk operation.

Any pointers to documentation on these basic concepts would be appreciated. I haven’t been able to find such documentation so far.

Gramps Web is a young (July 2021) project created for collaborative work on web servers. Since 2011, Gramps been a single-user database python application which runs on Linux, macOS and Windows desktop/laptop computers. Having to distinguish between the local desktop-based and hosted/online server-based variants of Gramps is a new problem.

This question was submitted in the general “Help” category, not the “Gramps Web” category. So I assumed that you wanted to take advantage of the more mature and feature-complete Desktop version. Then use Gramps Web Sync to push the data up to the Gramps Web instance being hosted on Digital Ocean.

I only mentioned a spreadsheet be cause it is an easy way of manipulating CSV data without hassling with formatting problems: strings that include commas, dates. And to take advantage of tabbing between cells, fill-down, pasting blocks of cells, etc.

This data (4 people: the home person of the example.gramps tree, his spouse and parents) was exported to CSV and imported (as Text fields) into LibreOffice calc. Then pasted here into Discourse. But it would be just as easy to have pasted it into the Text Import gramplet with TSV delimiting.

Place Title Name Type Latitude Longitude Code Enclosed_by Date
[P0957] USA USA Country
[P0000] OH, USA OH State [P0957]
[P0007] CA, USA CA State [P0957]
[P0022] WA, USA WA State [P0957]
[P0029] TN, USA TN State [P0957]
[P0032] SC, USA SC State [P0957]
[P0036] AR, USA AR State [P0957]
[P0039] IN, USA IN State [P0957]
[P0041] MS, USA MS State [P0957]
[P0042] NM, USA NM State [P0957]
[P0059] SD, USA SD State [P0957]
[P0062] LA, USA LA State [P0957]
[P0066] MT, USA MT State [P0957]
[P0070] ID, USA ID State [P0957]
[P0239] Humboldt, CA, USA Humboldt County [P0007]
[P0266] Greene, AR, USA Greene County [P0036]
[P0316] LaPorte, IN, USA LaPorte County [P0039]
[P0409] Twin Falls, ID, USA Twin Falls County [P0070]
[P0433] Washington, LA, USA Washington Parish [P0062]
[P1131] Aberdeen, WA Aberdeen City [P0022]
[P1132] Bogalusa, LA Bogalusa City 30.7910204 -89.8486858 [P0433]
[P1388] Portsmouth, OH Portsmouth City [P0000]
[P1398] Albuquerque, NM Albuquerque City [P0042]
[P1414] Eureka, CA Eureka City 40.8020712 -124.1636729 [P0239]
[P1435] Great Falls, MT Great Falls City [P0066]
[P1494] Knoxville, TN Knoxville City [P0029]
[P1498] Paragould, AR Paragould City 36.0584021 -90.4973286 [P0266]
[P1623] Michigan City, IN Michigan City City 41.7075394 -86.8950297 [P0316]
[P1643] Aberdeen, SD Aberdeen City [P0059]
[P1661] Myrtle Beach, SC Myrtle Beach City [P0032]
[P1678] Twin Falls, ID Twin Falls City 42.5629668 -114.4608711 [P0409]
[P1691] Picayune, MS Picayune City [P0041]
Person Surname Given Call Suffix Prefix Title Gender Birth date Birth place Birth source Baptism date Baptism place Baptism source Death date Death place Death source Burial date Burial place Burial source Note
[I0044] Garner Lewis Anderson Anderson Sr Dr. male 21 Jun 1855 [P1435] Baptize registry 1850 - 1867 Great Falls Church 28 Jun 1911 [P1678] 1 Jul 1911 [P1678]
[I0106] Garner Robert W. male 24 Apr 1826/7 (Julian) [P1131] 3 Feb 1916 [P1388] 5 Feb 1916 (Mar25) [P1494]
[I0045] Martel Luella Jacques female 23 Jan 1852 [P1414] 28 Apr 1921 [P1661] 30 Apr 1921 [P1661]
[I0107] Zieliński Phoebe Emily female 12 Apr 1827 [P1132] 7 Mar 1882 [P1398] 9 Mar 1882 [P1623]
Marriage Husband Wife Date Place Source Note
[F0017] [I0044] [I0045] 1 Apr 1875 [P1498]
[F0018] [I0106] [I0107] 4 Oct 1849 [P1498]
[F0019] 3 Jan 1823 [P1691]
[F0306]
[F0307] 4 Apr 1840
Family Child
[F0018] [I0044]
[F0019] [I0106]
[F0306] [I0107]
[F0307] [I0045]

Is this a Gramps Web question or not? If yes, we should move it to the appropriate forum.

Gramps Web currently indeed doesn’t support “wiping” the tree through the API. If you want to start over, just delete the database.

Since you say you have extensive programming experience, you could have a look at a new tool I built a few days ago to do batch operations with Gramps Web API - it is just a thin wrapper around requests but makes the authorization part a bit less painful: GitHub - DavidMStraub/gramps-web-api-client: A simple Python client to interact with a Gramps Web API instance.

Be warned though - this is definitely experimental and not suited for average users.

But you could use it to walk through your CSV in a Python script and create the people programmatically using the format defined in the API docs.

1 Like

Well, it is an ignorant question, as the difference between Gramps and Gramps Web is just now becoming apparent. Let me take a look at the desktop version and revise my thoughts about my workflow.

Thanks for your code, that may come in handy.

BTW: I’ve worked with many open source projects over the years, and I have to say that the support I’m finding here is amazing, on a par only with Postgres.

4 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.