Web Browsing Puppetmaster

I often have external genealogy websites as Citations. (e.g. WikiTree, FamilySearch, Ancestry, my personally generated Gramps site) I’d like to be able to use Gramps interactively as my Research browsing overlord.

Normally, I create a Source for the website and Citation where the “Page” is the unique ID for that record. (I originally put their full Query based URL. But those change too frequently.)
For my database driven sources, I have created a ‘Query’ Attribute that has an example query.
e.g. I have separate Sources for “FindAGrave (memorial)”, “FindAGrave (graver)” & “FindAGrave (cemetery)”. That have “https://www.findagrave.com/memorial/<Citation[ID]:Page>”, “https://www.findagriave/user/profile/<Citation[ID]:Page>” & “https://www.findagrave.com/cemetery/<Citation[ID]:Page>” for their respective Query atttributes.

It would be very useful to have a gramplet that synchronizes browsing those websites with my Active Person navigation in Gramps. This would allow me to quickly harvest sources, identify missing cross-references to external sites, and resolve duplicates

In particular, it’d be helpful to have a gramplet that spawns several ‘named’ Browser instances and pushes a URL to each. The URL is pushed to the named instance if a Citation for a specific Source exists as the Active Person changes. (If no Citation exists for that Source, a “about:blank” URL is pushed so that the websites with advertising updates don’t eat bandwidth)

As an example, the Puppetmaster session might have spawned 8 windows: a [1]&[2] named for the FindAGrave, WikiTree, FamilySearch, & Ancestry sources.
When I switch to an Active Person where the Citations gramplet lists: 1 citation under the FindAGrave source, 2 citations for WikiTree, and 1 citation for FamilySearch; the 1st citation updates the [1] instance for the three named sources, and 2nd WikiTree citation is fed to the [2] instance for WikiTree and all others are sent a “about:blank” URL.

It would be expected that users would have to authenticate a session for each spawned browser window for sites that require a sign-in. But updates should be able to leverage the session authentication.

This feature would probably need an option to NOT push URLs as the Active Person changes. Also, a Manual URL update/sync button that bypassed cached page data. (For when the Source site content was known to have been updated or the user Browsed away from the Active Person)

Future expansions might include:

  • color faux tags (like the Relationship view combined mode) associated with each the Sources driving each Spawned browser window. This would give a quick overview of Parents, Sibs, Spouses, Offspring who are missing a particular source. Most of the Websites have a linked view showing similar relations ships. So, if FindAGrave shows 2 parents, 3 spouses & 3 sibs but your Relationship view only has 4 tags, you could quickly ID & add the missing 2 cross-references.
  • Drag’n’drop website citation - drag a Puppetmaster spawned window indicator onto a person’s link in the Relationship view populates a new Citation Editor for that Source with the current URL in the Page referencing the target Person or Event. (User would have to trim the URL manually.)
  • expansion to Sources - Add quick access to a GoogleBooks or InternetArchive scan of a book. Or the ability to look up an ISBN or Library of Congress call number. Also, document management tools (like Zotero) have a web interface option. This would allow synchronized source browsing.

The WebConnect does a little of this. But it could use some expansion.

  • instead of Pop-up menus added to the People View, it would be nice to have a Gramplet with checkboxes for all the sites I’d like Gramps to browse. Then have a Refresh button that updates the query.

  • And It would be more powerful to be able to associate an attribute with a variable.
    So, if I created a LCCN (Library of Congress catalog number) with a book source, I could feed that into a PermaLink search string.

  • And to have named Target windows so it wouldn’t open endless new tabs.

example:
the source “The life of Col. John Charles Fremont, and his narrative of explorations and adventures, in Kansas, Nebraska, Oregon and California.; The memoir by Samuel M. Smucker” has a custom attribute called LCCN which that record sets = “rc 01001190”. The LCCN permalink is “https://lccn.loc.gov/rc01001190”. So the generic search would be of “https://lccn.loc.gov/”&LCCN
This one may not be a good example. It also needs a Python routine to strip whitespace from the LCCN.

I use Zotero and the Web Clipper for that purpose, and I always take a snippet of the webpage as it is a given time and use it as a timestamped source.

Zotero do not update the query automatically, but at least I can with a little addition of metadata and cleaning up, create a more or less “timeless” citation, with an offline media copy.

The usage of Zotero has reduced my source registration time in Gramps with something like 10-15 steps per. source.


I Agree that it would be great to have better source handling, both like you describe, but also in general, but some times we just have to live with the fact that we need to use multiple software, I think…

The best of two worlds would be if we could get some interactions between software, i.e. Zotero (or similar) for bibliography, Cytoscape, Constellation, Gephi, Tulip, yEd etc. for advanced graph and network research, json-ld, or RDS XML for advanced research and presentation tools online…


One tips if you would like to create MD Notes with advanded functionality like network graph view, integration with Zotero possibility to use a general markdown web clipper, and to add the markdown files to Graps, is to use a combination of Zettlr, Obsidian desktop, Zotero and Tropy… and you could add in the use of Freeplane for some research, event though it is superfluous with the use of Obsidian desktop.


Just a last note: If you really want to have more research tools on hand, you should look at Palladio and InfraNodus, and i.e. Headstart from openknowledgemaps, or Datashare from ICIJ.

1 Like

I was actually going to add my writing about using those tools to the wiki, and an extended description on how they could be used, but then I started to think about how extremely difficult it is to transfer data from and to Gramps in an interchangeable format that kept all data intact… and I figured out that it wouldn’t benefit anyone but those with extended knowledge of programming, scripting or a lot of knowledge about using Excel with Powerquery or Openrefine… and those users have no problem figuring those things themselves, for most “average” users it would just be confusing and add more question and errors.
Hopefully someone will one day see the benefit of a few fully interchangeable formats between research tools.

1 Like