FamilySearch Gramplet

Help
,
41

As far as I can check, this is not completely true when the program only reads from FS, like getmyancestors. I see no client key in that, nor in the source code of the gedcomx module used by this Gramplet. And that suggests that we may never be able to write to the FS tree with it, like you can do with an approved client like Ancestral Quest or RootsMagic.

I looked at the source code, and both getcomx and getmyancestors log on to FamilySearch using a fake User-Agent, meaning that they pretend to be Chrome or Firefox. You can see that in the source code, if you want. And I see nothing that looks like an encrypted application key or signature that you would most likely need for write access. The source code has entries that pretend to write to FS, but I really doubt if they work, and my first tests on close ancestors clearly suggest that they donā€™t, and that is fine with me.

There is another reason for not supporting it though, and thatā€™s the language used by and in the source code. Variable names and text strings are in Esperanto, and many log or debug texts are simply printed to the terminal output without using gettext for translation. And that will make maintenance quite difficult for core developers. And in that respect, itā€™s much the same as the isotammi tools, which also have text strings in another language than English.

42

The app has taken a different approach than having a key & token associated with the app. Instead, access is controlled through each user entering their FS Username & Password and whatever limits FS sets for that account.

(The Username is stored as un-encrypted text in the .ini but the Password is NOT saved. You can un-comment the .ini password line and save it there as plain text. But saving usernames and password in plain, un-encrypted text in a predictable file location is a REALLY bad idea.)

When I read the FS license a few years ago, one reason for it was to make certain Developers knew there were rules about how much load they could put on the FamilySearch servers. (A common FS community complaint at the time was that servers were unresponsive.) And that grabbing huge chunks was STRICTLY verboten. Users were only supposed download section reasonably closely connected to themselves. And that they actively tracked usage for apps violating these guidelines so that they could be disabled.

I suspect this gramplet WILL have to be changed to a key & token approach.

43

If this were still true, I would think that FamilySearch would have tried to take down the getmyancestors repository. I would think that such an action would be quite easy for them, because the only thing that takes is a call to Microsoft, which owns GitHub.

It is also not true that I can only download sections close to myself. In all applications that I know, and have tries, meaning Windows applications like Ancestral Quest, Legacy Family Tree, and RootsMagic, it is possible to enter any FamilySearch ID, and download a number of generations up or down from that. And for RootsMagic the defaults are 10 generations up, and 4 down, and since that program automatically includes spouses and siblings, downloads often leave more than a thousand new persons on my system, especially when I forget to be smart, and limit the generations to a more practical level.

With getmyancestors, you can also specify any ID to start with, and when you let that program include all sources and notes, itā€™s even heavier than the commercial programs, since none of those can do that. And I bet that if FS doesnā€™t like getmyancestors, they can not only have it removed from GitHub, but they can also put a limit on the downloads done by this fake Firefox User-Agent.

I am also quite sure that there is such a thing as an application key, the thing that I called a client key earlier, which is indepent from the session key used when a user has logged on, using whatever protocol. Ancestral Quest uses a browser session to log on, but RootsMagic doesnā€™t do that, and both applications can read from and write to the FS database. And I am pretty sure that all authorized applications have a unique application key, because that is the only way for FS to distinguish authorized apps from getmyancestors etc.

And if the latter had an application key, I would have found it already, so it looks like reading can be done without one now.

2 Likes
44

Oh, it absolutely is not accessing with an app key. And FamilySearch may never object unless they determine something is creating an unreasonable burden on their system.

So the more efficient this Gramplet becomes and the more people that start using it, the more flags it might raise. (It could become a victim of its own success.)

45

This is interesting as thought you needed an app key. Wonder if the places registry can be accessed this way, might make a nice add on to be able to import from it.

In case anyone overlooked it take note that this creates an additional table to track some state in that wonā€™t be included in a Gramps XML export only in actual backups of the database.

    db.dbapi.execute('CREATE TABLE personfs_stato '
                         '('
                         'p_handle VARCHAR(50) PRIMARY KEY NOT NULL, '
                         'fsid CHAR(8), '
                         'estas_radiko CHAR(1), '
                         'stat_dato integer, '
                         'konf_dato integer, '
                         'gramps_datomod integer, '
                         'fs_datomod integer,'
                         'konf_esenco CHAR(1),'
                         'konf CHAR(1) '
                         ')')
1 Like
46

Thatā€™s true, and we have some influence on that, in the sense that it will probably not be used much if itā€™s only available in French and Esperanto.

1 Like
47

I saw that in the code indeed, and there are a couple of other surprises too, like a handful of tags that are added to all persons that I synchronized.

48

I actually liked the (color-coded) tags that were created. It helped me identify duplicates (very convenient) or context on why the individual was imported. All of my imports were tagged red, so I know I would have to fix them. But that was expected.

I think the new table is for timestamps on the FamilySearch and gramps data that was imported to know whether to upload again if there is more recent data.

2 Likes
49

I think that it can. Check this out:

50

I tried a big import on a new database yesterday, to download 20 generations of ancestors, and hoping that I would find King Edward III among them, but Iā€™m not sure whether they were all downloaded. I found about 2,400 persons afterwards, and the download took more than an hour. There was a progress bar visible, but Iā€™m used to counters that show progress, like in Ancestral Quest and RootsMagic.

I also found that it created a French place hierarchy, meaning that I was born in Pays Bas.

The imported data triggered a new error in Gramps, which should have been caught by pylint:

https://gramps-project.org/bugs/view.php?id=12865

So, although the import works, Iā€™d rather use getmyancestors, unless you prefer French, of course.

51

Same with any ChatGPT grampletsā€¦
ā€¦ Or reports utilizing ChatGPT.

52

Yes, the project will have some issues with any API that requires executing a signature for licensing.

But the possibilities of: NLP pre-processing of some genealogical source documents, a NLP help system (not as tempting, since it is potentially as annoying as the thrice-damned Clippy was), and AI generated report customizations in Python; are all probably in our future.

And a NLP proof-reader & copy editor would improve the coherence of many published works. (Just as spell and grammar checkers helped.)

53

I was curious and emailed the FamilySearch Developer support address inquiring about access for Open Source projects.

The response received a short while ago was ā€œAPI access is only for organizational and business use. We do not have API access for personal use.ā€ So I guess that is that.

1 Like
54

OK, but what does that actually mean, knowing that applications like getmyancestors are available and can be used, almost without restrictions? That is a read-only program, which does its work quite well, except for exporting long month names, and that can be cured.

To me it suggests, that we will never get write access, and a Gramplet can still be used to link persons by searching and adding the FS ID to our database, which is exactly what Iā€™ve been able to do up till now, with this one.

Sending data to the tree, on a one-by-one basis then means that I export a GEDCOM to a licensed program like Ancestral Quest, Legacy, or RootsMagic, and sync things from there, which is exactly what Iā€™ve been doing for years.

And that implies that I can live with that. :slight_smile:

55

Yes, likely no write access, which perhaps would be nice someday but I agree isnā€™t a big deal, especially since the bulk of the functionality and benefit is in read access.

It is kind of ironic though, if people wanted to share their work in Gramps they would have to go out of their way to enter their data in the Family Tree manually, which most probably wonā€™t take the time to do. So it seems to run counter to the self interest of the LDS church unless Iā€™m missing something.

56

Well, from their position, I can understand that they put up a few hurdles for write access, like demanding an entity that can take (legal) responsibility for what its software does to the tree. And now that I know how to run RootsMagic on Wine, I have enough means to upload stuff, one-by-one, and Iā€™m glad that they stopped most of the bulk upload too, because there is a lot of carbage already. I merged a couple of duplicate cousins today, who lived in Iowa, and I often see lots of duplicates for notable ancestors, like King Edward III.

And if you want, you can still upload a GEDCOM to the genealogies section, and let the site compare that to the tree. I donā€™t recommend that, because its duplicate check is so strict that itā€™s quite easy to conclude that a person does not exist in the tree, and then find out that when you visit the tree, your ancestor has a duplicate, created by an anonymous user, with a source named GEDCOM data. And guess who that was?

From my point of view, there are enough applications that can write to the tree, and I can live with the fact that Gramps is not one of those, because I can always export a GEDCOM file and import that into one of the programs listed here:

https://partners.familysearch.org/solutionsgallery/s/list?fswrite=1&category=family_tree_management

And for Windows, Ancestral Quest, Legacy, and RootsMagic all have free versions that have enough capabilities to suit my FamilySearch needs.

1 Like
57

I installed this gramplet on my Mac and ran into an issue (reported issue and resolution to author). I also wrote a few scripts that translate the text strings to English. I decided since this gramplet is still in development, I wrote scripts rather than change code itself. These scripts use unix utilities: make, sed, and patch. They translate the UI as well as the debug prints. And the tags that are used (which I found useful).

I can share these if there is interest. But you will need to have these developer tools on your system to use. I have run the tools against both the main and dev branches of the source and they seem to work.

make : will convert to English
make reset : will revert to original Esperanto

58

jmichault posted v1.4.1 with an updated POT, French and English PO files to the repository 16:02 GMT 8 Apr 2023

1 Like
59

I saw it, and found an error, so I couldnā€™t use it without hacking. I will inform the author.

For me, itā€™s not very useful, because Gramps looks for a Dutch translation, and if thatā€™s not found, it will default to the original text, which is Esperanto.

1 Like
60

Iā€™d love to try them, because they make things a lot easier, and Iā€™m quite familiar with make, if itā€™s still the same make as I used in the 80ā€™s, or for Gramps 3.4.

The most important advantage is that with English text strings, I donā€™t need any PO to figure out what the program is doing.

So, @GaryGriffin , do you have a fork with these scripts added? Or another public place where I can find them? You may also send a private message of course.

1 Like