A new WebSearch gramplet is ready

[Urchello]

@hamkg

Yes, renaming ‘sv-links.csv’ to ‘se-links.csv’ works fine, although I also had to edit the /.local/share/gramps/gramps60/plugins/WebSearch/configs/config.ini file:
enabled_files=[‘sv-links.csv’, ‘common-links.csv’] > enabled_files=[‘se-links.csv’, ‘common-links.csv’]

great!
I think you no need manually edit ini file. You need just turn on the se-links.csv in the UI settings (it should appear there automatically when a new csv are adding).

Okay, I will rename it, thank you!

[hamkg]

Completely forgot the UI settings, of course it can be changed there.
The se (country) vs. sv (language) can be confusing. Don’t know why they aren’t the same (language code se is used for Northern Sami which of course needs a language code, but maybe another one would have been a better choice).

Another suggestion - maybe the following could be added to se-links.csv:
People,Svenska gravar,1,https://svenskagravar.se/search?query=%(given)s+%(surname)s,
(a grave register, partly overlapping but partly completing https://gravar.se already in the file)

[Urchello]

@hamkg

Another suggestion - maybe the following could be added to se-links.csv:
People,Svenska gravar,1,https://svenskagravar.se/search?query=%(given)s+%(surname)s,
(a grave register, partly overlapping but partly completing https://gravar.se already in the file)

OK, I will add. Thank you!

Add https://svenskagravar.se link to se-links.csv

[emyoulation]

@Urchello

Got it. So, we can add it to the README, right? And I think I can insert xml as a separate .gramps file into the data directory of the Websearch project. So, users can download ready .gramps file instead of creating a new one with copy/paste xml from README

You are welcome to add the .gramps file to the support files of the add-on. The reason that I tried an XML file listing was for ‘transparency’.

People are often too suspicious to blindly import something with a bunch of potentially malicious hotlinks.

(Did you import the sample file that I sent previously by Private Messaging? Probably not.

And that might be because you already felt confident in your expectations when entering data for the gramplet.

But when reading the XML above, your curiosity might have been been piqued by external databases that you had not explored.)

Also, with the plain text XML exposing the guts, readers can verify that it contains nothing bad.

And while one must blunder around exploring the hierarchical twigs of a tree, you can read a brief XML from top to bottom and KNOW exactly what key data is used. This improves reader confidence that they have not missed a critical hidden element.

(I experiment with a “To Do” note in this sample Tree to encourage a Person importing data to explore specific areas. Expecting people to intuitively discover secondary objects by blundering around is not reasonable. That is a shortcoming in Gramps.)

[Urchello]

Now a new Websearch feature is implemented.
Users will see links from notes. For example a person has such note:

It has 3 types of links:

1. external link to wikipedia. It will be parsed from the note text
2. external link which is added via the Link button
3. internal gramps link which is added via the Link button. It will open the appropriate Gramps edit window.

In WebSearch you can see (3 last rows):

Also several icons are changed a bit:

Change icons for links from the tabs: Attributes, Notes, Internet

@hamkg your two tasks are also done:

Zip: https://github.com/jurchello/WebSearch/archive/refs/heads/main.zip

@emyoulation your task will be closed later " Add a .gramps import file for WebSearch demonstration #93
New issue". I think we also need add some notes and Internet addresses to the demonstration tree.

[Urchello]

The new blue icons for Attributes, Notes and Internet tabs:

[emyoulation]

@Urchello

@emyoulation your task will be closed later " Add a .gramps import file for WebSearch demonstration #93
New issue". I think we also need add some notes and Internet addresses to the demonstration tree

Yes. Although my initial sample was just Attributes for the GeoWashington person object, I realized that there were no tests files for the other search categories. So that lead to the addition of a couple Places, a Family, a Citation/Source/Repository.

It is growing too large and losing the focus. I’m thinking that separating them might make more sense. So users can import a sample and adapt it for actual use … without having to track down and eradicate all the other extraneous bits.

[hartenthaler]

@emyoulation

Here is a sample file with a variety of UID custom attributes to use while experimenting with the WebSearch gramplet.

… only short remarks to this good idea:

• the Roglo ID is wrong (Roglo changes those internal Id numbers frequently); you have to generate a permanent link like: George.0 Washington; there is a button at the end of each Roglo page
• the GNDID does not work

[hartenthaler]

@emyoulation

It is growing too large and losing the focus.

Maybe you can provide a simple test file containing some important examples. And a full fledged example for users like me, interested in testing all the details.

[hartenthaler]

Regarding the Gramps attribute _UID: you can link based on that information to the GEDBAS database containing about 38 million persons by a link like https://gedbas.de/uid/1685d032-876a-40af-b929-d11a0e23e67au
It would be helpful if you add that template to the UID section as a template.

[Urchello]

@hartenthaler

Regarding the Gramps attribute _UID: you can link based on that information to the GEDBAS database containing about 38 million persons by a link like https://gedbas.de/uid/1685d032-876a-40af-b929-d11a0e23e67au
It would be helpful if you add that template to the UID section as a template.

Thank you!
Added to the uids list. It will be available in v.0.67.45 after the next release.

[hartenthaler]

First: sorry my fault - the correct example link is GEDBAS - Alfred Franz Hermann ZEDLITZ
(I added an “u” at the end by mis-typing).

But it is not a GEDBAS ID, it is the global unique UID defined in GEDCOM, using the GEDCOM tags _UID or UID. In Gramps that is an attribute, if I interpret it correctly. So it should be named UID.

I know only one database where you can use the UID as a link to a person, that is GEDBAS with thousands of trees and 38 million persons. But Ancestry is using UID, too. But I don’t know if you can link to their records by using UID.

[Urchello]

@hartenthaler

First: sorry my fault - the correct example link is GEDBAS - Alfred Franz Hermann ZEDLITZ
(I added an “u” at the end by mis-typing).

No worries, I’ve found another one GEDBAS - 58fcbf61b35a21e79bf54437e60f3d65

@hartenthaler

But it is not a GEDBAS ID, it is the global unique UID defined in GEDCOM, using the GEDCOM tags _UID or UID. In Gramps that is an attribute, if I interpret it correctly. So it should be named UID.

I know only one database where you can use the UID as a link to a person, that is GEDBAS with thousands of trees and 38 million persons. But Ancestry is using UID, too. But I don’t know if you can link to their records by using UID.

I’m not sure we need to rename the attribute to UID or _UID right now.
The template currently uses the attribute name GedbasID, and users can change it to whatever they prefer.
If we switch to using UID or _UID, it could create confusion because any other resource’s UID might be incorrectly interpreted as a GEDBAS ID.
To avoid this, I think it’s safer to use more specific attribute names like GedbasID in the JSON templates.

[Urchello]

A new WebSearch update. Version 0.68.45 will support a new category type: Notes. Links will be extracted and listed in the treeview:

The same for Repositories. Version 0.69.45.
Now all 9 Gramps categories are supported in WebSearch:

    PEOPLE = "People"
    PLACES = "Places"
    SOURCES = "Sources"
    FAMILIES = "Families"
    EVENTS = "Events"
    CITATIONS = "Citations"
    MEDIA = "Media"
    NOTES = "Notes"
    REPOSITORIES = "Repositories"

[Urchello]

In the next WebSearch release, a new feature will be available that automatically recognizes archive reference codes specified in Sources and generates search links to corresponding online resources. If an archive reference is stored in the Abbreviation field — for example, 142-1-15 or ДААРК ф. 142 оп. 1 сп. 15 — the Gramplet will be able to parse it into the following components:

• archive_code,
• collection_number,
• series_number,
• file_number,

and then substitute these values into the corresponding link templates.

Examples of recognized formats (at the moment these are all supported patterns; for additional formats, more regex patterns can be added — feel free to request them):

Two new links have been added that use the newly introduced keys:
In the file common-links.csv:
https://www.familysearch.org/en/records/images/search-results?keyword="%(collection_number)s-%(series_number)s-%(file_number)s"

In the file ua-links.csv:
https://inspector.duckarchive.com/search?q=%(archive_code)s-%(collection_number)s-%(series_number)s-%(file_number)s

Demonstration in Gramps:

The full list of keys now supported in Sources:

• source_title
• full_abbreviation
• archive_code
• collection_number
• series_number
• file_number
• locale

[Urchello]

All planned features for WebSearch have now been implemented — with the exception of one remaining issue:

Add a .gramps import file for WebSearch demonstration

@emyoulation, I’d like to ask your opinion: should we split the demonstration .gramps file into several smaller ones? I could use your help with this, especially since I already use Gramps 6.0. However, I believe using version 5.2 would be more universal, as it’s compatible with both 5.2 and 6.0 users.

Currently, all 9 Navigation Types are supported. WebSearch links can now be saved to Notes and Attributes for every navigation type that has a Notes or Attributes tab. Links from the Internet tab are also displayed for all types that support it. So, in effect, everything supports everything as much as possible.

This means we now need .gramps files that demonstrate all these capabilities.

We can also postpone this to a future release if needed. What do you think?

[Urchello]

Upcoming WebSearch Release Notes

Features

1. Repositories and Notes. Added support for Repositories and Notes as new navigation types.
2. Sources. Implemented archive reference parsing with automatic key extraction (full_abbreviation, archive_code, collection_number, series_number, file_number).
3. Added more link templates to common-links.csv, ua-links.csv files
4. Enabled saving links to Notes and Attributes via context menu for all relevant navigation types.
5. Extracted and displayed links from the Internet tab for all relevant navigation types.
6. Added new UID templates in json for gedbas.de using UID key.
7. Updated link set for Sweden (renamed sv-links.csv → se-links.csv, added svenskagravar.se).

Fixes

1. Fixed crash: gramps.gen.errors.HandleError: Handle is empty.
2. Fixed icon sorting and enable/disable logic in settings.
3. Fixed visited icon display.
4. Improved icon differentiation for Attribute / Note / Internet links.
5. Corrected user data icon display for direct links from tabs.
6. Fixed issues with stripping extra characters from parsed links.

Pre-release Testing Needed
I would be grateful for any help testing the Gramplet before release — especially the 5.2 version. The features that need the most attention are those related to adding links to attributes and notes, as they interact directly with the database and require extra care. Also, it’s unclear whether internal Gramps links extracted from notes will work correctly in version 5.2 — they function properly in 6.0, but their behavior in 5.2 still needs to be verified.
Webserch 6.0: https://github.com/jurchello/WebSearch/archive/refs/heads/main.zip
Webserch 5.2: https://github.com/jurchello/WebSearch/archive/refs/heads/5.2.zip

[Urchello]

Several new features:

1. Implemented possibility list navigation types for any link in csv files. The “*” char means that the link will be shown for all navigation types: for People, Places, etc…

Navigation type,Title,Is enabled,URL,Comment
"People,Places",Static example 1,1,https://www.google.com/search?q=I+like+this+link1,Shown only in People and Places tabs
*,Static example 2,1,https://www.google.com/search?q=I+like+this+link2,Shown in all tabs except People and Places

2. Added 3 new csv types:

• {country_code}-community-links.csv
• {country_code}-forum-links.csv
• {country_code}-archive-links.csv

An example:

The last updates are here: GitHub - jurchello/WebSearch: Gramps addon

[Urchello]

Now these two links are tested and working. You can use them im the addons manager:

• https://raw.githubusercontent.com/jurchello/WebSearch/dist/gramps60
• https://raw.githubusercontent.com/jurchello/WebSearch/dist/gramps52

[emyoulation]

@Urchello

Now these two links are tested and working. You can use them im the addons manager:

Added to :

• Curated addon collections: Urchello’s experimental WebSearch gramplet
• Addon:WebSearch - Gramps (installation section)