Research Journal

Following the reading of this video on Youtube:
“Create a research journal with a spreadsheet”
[Créer un journal de recherches avec un tableur - YouTube]
and this blog post
“How and why to create a list of missing records in your genealogy”
[Comment et pourquoi créer une liste des actes manquants dans votre généalogie]

I am looking for tools (Gramplets, Quickview, filters, …) to create “To Do” lists concerning, for example, the completeness of individual files, events, dates, places, sources and citation to log the research carried out, in progress and to do in order to plan a possible break.
Two gramplets approximate the expressed needs:
**** “And now”
But is it possible?
1- make it dynamic so that it can react on the active individual and not the strain
2 - copy / paste the return of the request in a Gramps note, keeping the individual “links”
**** “To do” of the dashboard which collects all the other notes distributed on Gramps objects but which external tool to store and use this information
*** Filters?
Is there a library of filters that might be suitable?
*** Quick view?
Maybe this is impossible in the current state of affairs?
Thank you
Ps: Thanks to GoogleTranslate because I’m a French Gramps user

GRAMPS : 5.1.3
Python : 3.5.2 (default, Oct 7 2020, 17:19:02) […
BSDDB : 6.1.0 (5, 3, 28)
sqlite : 3.11.0 (2.6.0)
LANG : fr_FR.UTF-8
Système : Linux
Distribution : 4.4.0-201-generic

Distributor ID: LinuxMint
Description:    Linux Mint 18 Sarah
Release:        18
Codename:       sarah
1 Like

There is a To Do gramplet and Report available.

Anywhere within Gramps, you can add a note with the Type “To Do” with your notes about things that need confirmation, resources to check, etc. Then you can activate the To Do gramplet (Dashboard is a good place) that will aggregate these To Do notes in the one view. You can also add To Do notes directly within the gramplet.

The To Do text report will print out a set of these (or any) Note record with a common Tag. “PRINT” works for me. Tag the To Do notes you may want to print to take to the library. Remember to clear the PRINT tag when no longer needed.

NOTE: The “To Do” note type will not be added to the Remove Unused Object tool for deletion. If a To Do note is no longer needed, change its type to another option, and detach it from its object so the Remove (delete) tool can delete the unneeded note.

Google Translate

Un gramplet de tâches et un rapport sont disponibles.

N’importe où dans Gramps, vous pouvez ajouter une note avec le type “To Do” avec vos notes sur les choses qui nécessitent une confirmation, des ressources à vérifier, etc. Ensuite, vous pouvez activer le gramplet To Do (le tableau de bord est un bon endroit) qui les agrégera Pour faire des notes dans une seule vue. Vous pouvez également ajouter des notes de tâches directement dans le gramplet.

Le rapport texte des tâches imprimera un ensemble de ces enregistrements de notes (ou n’importe quel autre) avec une étiquette commune. “PRINT” fonctionne pour moi. Marquez les notes de tâches que vous voudrez peut-être imprimer pour les apporter à la bibliothèque. N’oubliez pas d’effacer la balise PRINT lorsqu’elle n’est plus nécessaire.

REMARQUE: le type de note «À faire» ne sera pas ajouté à l’outil Supprimer l’objet inutilisé pour suppression. Si une note à faire n’est plus nécessaire, remplacez son type par une autre option et détachez-la de son objet afin que l’outil Supprimer (supprimer) puisse supprimer la note inutile.

I understood: you absolutely have to go through the attribution of a label to be able to generate a report.
Unfortunately, the objects which are the subject of notes are only too little identified and described to constitute a database in a spreadsheet for example for in-depth analyzes.
I was hoping for “links” to individuals, sources, quotes …
Finally very close to the “Label report”
Perhaps more complex Gramps functions should be used to generate this famous “Research Journal” Thanks for your help
Jean Claude

In Practice there are no research journal or research log feature in Gramps, there are no way to analyze data and easily add objects to a research journal/log…
And there are not many possibilities to creates tables with lists or any other “advanced” research notes.

I have started to use external software for this, using Zettlr, Obsidian (Because of the Advanced Graph View plugin, and now the last days started to test out VSCoduim with the Foam extension and a few other markdown extensions…

The markdown(MD) files can be added to Gramps as media files and linked to the object that it belong to.

It would be great if Gramps got more research and analyzing feature in future version, absolutely needed…

Research and Analyzing tools like:

  • A “real” research log like the one in Clooz 4, that can be converted to “live” data when your research for those objects are finalized…
  • A network graph view of any relation and connection in the database with analyzing feature like Cytoscape, not only different layouts of the graph, but possibilities to set different graph parameters based on values and attributes in the Gramps database… i.e. Analyzing based on DNA values or matches… or set a color or cluster or compound nodes, based on where or when or any other attribute…

Again, easiest way to support advanced research would be to have 3 interchangeable formats for the whole database…

  • one Network Graph format
  • One OWL/RDS format
  • and a full export to CSV.

If I had any developer knowledge, this things would be the first things I would make gramplets for…

Thank you for this wealth of information which shows the richness of third-party software.
But not being a computer scientist, I try to use any tools that already exist in Gramps
A priori that does not exist? Unless you build them yourself (Filters, Tags, …?) If I understood correctly?
I will have to look at these filters and basically succeed in exporting the return of these in a spreadsheet if possible.
I’m going to try …
thanks again
Jean Claude

Taking my research notes and managing my genealogical projects in Notion I’ve tried to copy-paste Notion page content in Notepad++ and save that text in a .md file. I have installed a Notepad++ markdown plugin too. Adding that file to Gramps medias and double clicking on it, it opens perfectly in Notepad++ viewing its formatted content looking like in Notion in the markdown panel, except images saved in Notion. I’ll continue like that (I’ve seen the md plugin can generate html content, so md can be converted into html wich can be copied in a Gramps html note, but it’s much less readable)

Notion have now a backlinks feature to show wich pages are linked to the one you’re working on but there is no feature like this.

I’ve take a look at that feature. I don’t know how yet but I think I’ll try it.

In the past I’d used TheBrain (aka PersonalBrain)for years to dump everything in it, even my genealogy researches. You could share father- son (mother-daughter?) links between ideas as in regular mindmap software but you can add transversal links too, you could name and categorize links make them go forward, backward or both directions, takes idea notes on them… add notes and files to ideas and more.

1 Like

Yes, I have looked at all of them, problem with all of them is that they either have some kind of proprietary dataformat or are paid…

I have used Freeplane (Open Source mindmap, not as dynamic as TheBrain) for a lot of my research to find connections between people, companies, documents and places, the problem is that it has limited export formats that is interchangeable, so to be able to use the data in other software you need to know how to programming for transform to other dataformats, even their csv format is totally useless for anything but freeplane…

The Graph View that you have looked at is the “built-in” plugin, the “Advanced Graph View” Community Plugin, is a lot more powerful, but still under development…
Neo4j Graph View 0.2.5 Advanced Graph View in Obsidian using Neo4j - Share & showcase - Obsidian Forum.
The new version of this plugin do not use neo4j, he will make an Add-on to the plugin that will enable that functionality for advanced users.
The new plugin also has a few more layout algorithms…

I think the best usage of those plugins and views are if you write a lot of research logs/notes, have your sources in i.e. Zotero (there is a zotero to markdown plugin for Zotero.), clip a lot from internet, and in addition add documents to your research (there are plugins for pdf and annotation for Obsidian and Zettlr…

The thing with this is that you can read all the notes, clips and annotations you have, as free text and create links for any name, place or other information as wiki-links, and as you read, you will start to connect information in unstructured text in a way that would be difficult in any other software…

Just one simple example:
I have 1800 Norwegian Newspapers where my GG-Grandfathers company is mentioned, both in some articles and in ad’s, but there are also a lot of other news articles in those that can tell something about events… in one of those newspapers I also found one reference to Sweden, so I started to search in Swedish Newspapers and found multiple patents that he had in Sweden, that led me to search more and I found a total of 28 international patents in 14 countries,
I also found that his sons was arrested for burglary in Stockholm, Sweden, and that they was arrested with more than 200 stolen keys that could be used to pick locks… the funny thing in this story is that their father was one of the best locksmiths in Norway at that time… His company is one of the stating companies of what we to day know as - ASSA ABLOY Opening Solutions.

Another example would be if you are looking for a person with the name of i.e. Paul Steffansson…
You create a note with his name and create a YAML heading with an “aliases” key, in that key you make an array of all known aliases for this name, i.e. Paal Steffanson, Paul Stefanson etc.
You create notes for any other important information, i.e. places, family members, dates and so on…

In Phase one of your research; You just start to clip and download all the articles and websites you find that can relate to this person using i.e. Markdownloader or other webclippers or preffered tools.

When you you have done your “Search and Collect Phase”, you might have 500 notes with information, where you might be unsure for maybe 30-70%.
So you start “Phase 2”; You start going through all the information and creates wikilinks on all key information, it can be places, names, dates…

Then you start to see that more than 20 of those individual Notes has connections to a person with an unknown name for “S. E. Rolvsson”, and a place name you have never seen before in addition to the first mentioned person… 5 of those documents also have links to multiple known place names, and a farm name you didn’t know…
30% of those documents has no additional information, but all of them are linked to both the date of birth and to one of the aliases of the PoI.
Suddenly you have at least one name and a few more places to research…

Much of that information could have been lost if you didn’t use a network graph analyzing approach…

I started to use Cytoscape in addition to Freeplane and Aeon Timeline for this work, but all of them need structured data to work…
Then I started to test a storytelling software called Twine2, that use wiki links and has a manual network graph view, I found two new connections with that software, and then I found Obsidian, and later Foam for VS Code or VS Codium if you don’t like the telemetry of VS Code…

The thing about just be able to add unstructured text and start create links for the research part is just extremely easy… and then, when you find the links you can just add the information, notes and documents to Gramps (everything is markdown files, so you can add the files directly to Gramps as media, and have them linked in your folders as well…

This is a really easy way to find connections and relations in large plain text documents without the need of adding it to any form of tables or structured text format, specially when you have text where you don’t know if you actually have any connections or not

I have not found a good and easy way to get the markdown notes in to Gramps as Gramps Notes, but it should be possible to use python to write a Gramps XML where you create the Notes in the XML file if you want to do that…

I am not a developer so for me I need to look at other solutions…

I am working on a Gramps XML import to Excel that create one table for Each Gramps Object Type and then a VBA script that write each line of each table as a Markdown File with YAML keys for all attributes, and the same script also creates 2 CSV files, one CSV for all Nodes (Objects in Gramps) and one CSV for all Edges (all relations and connections in Gramps, including connections between places, and internal links in Notes), those two files I can import to Cytoscape or neo4j or any other network graph software and do analyzes on all my data… same can be done with all my notes, when the next version of the AGV for Obsidian is finished, because it will have a feature to save your graph data to Cytoscape JSON.

I have hoped for more than 2 years now that some Gramps developer did understand the benefit of having interchangeable Open Data exports and imports in Gramps, but I have given up on that.
For me now, the most important thing is to be able to find workable solutions to be able tocollect and consolidate all my research to a few interchangeable formats where I can analyze data in different software in a easy way… since there are no analyzing functionality at all in Gramps…
At the moment I use Excel and Markdown as hubs for this… hopefully I will be able to learn enough Python to make my self a script that can do all the transforming and transcribing between formats for me…

If Gramps had support for JSON-LD and GraphML instead of a broken CSV and a broken JSON export, this extremely long workflow would have been solved in two easy steps…
And everyone that used Zotero for their sources, would also been able to import there sources directly to Gramps via JSON-LD… And they would be able to export to GraphML, do analyzes and and reimport the result to Gramps…

The Graph database and Network Graph formats is extremely suited for genealogy research… but someone that actually do development need to adopt it and see the benefits…
It is also the absolute best way to do research and analyzes on DNA data (both Cytoscape and Gephi was creates as open source software alternatives for Bio-data analyzes.