GetGOV Import Failure

Help
1

GetGOV place imports are failing for me. It looks like the https://gov.genealogy.net website has been updated. Is anyone else seeing this issue?

2

Can you submit a Mantis bug report with the traceback.

On my Mac, it fails, but not sure it is the same failure you are seeing. I have submitted a report to the GOV admins to address one issue (some of their files are returning 404 errors). These files are critical for the gramplet.

I will update the gramplet to trap for this error, but that will just stop the crash, won’t actually make the gramplet useable.

1 Like
3

Do you really want to use the GetGov? There are a lot of options. (And another Discourse forum thread.)

@kku has noted that there was a huge surge of Place cleaning tools and gramplets. That related in a lot of overlap. So he believes there should be a shakeout and possibly some consolidaton:

From the Gramps User maillist:

On Tue, Jan 29, 2019 at 4:19 AM Sebastian Schubert wrote:

I am thinking about migrating my places to the GOV structure with the GetGOV addon. However, I have quite a big number of places already, which are somewhat based on the GOV database, so this seems like a lot of work.
In particular, it seems to require a lot of manual merging of double places.

On Tue, Jan 29, 2019 at 10:02 AM Paul Culley wrote:

I agree you have some work ahead of you. I would not attempt to “just change some of the IDs to the GOV ID”. As you say that would introduce inconsistencies in the data compared to the GOV data.

I don’t care for the GOV system myself, as I find searching for a place to be rather difficult, I think they only return a match if you happen to know the exact local place name. (Try a search for “Berlin, Germany”, as an English user I expected something, but get no results). But I recognize it is better if you want the hierarchy of places to reflect changes over time.

If I had to do this, I would start with the biggest places and then work down (Countries, then states etc.). Use GetGOV to load a place, and then merge with your preexisting manually added version. Then look at the merged result and see if it needs to be changed, removing extra alternative names, and extra enclosed-by data. You would generally want to keep the GOV data, and drop conflicting manually entered data.

I created the Place Cleanup tool gramplet which uses the GeoNames data. That data is very easy to search, but contains almost no historical information. I originally thought I might do a similar tool for GOV data, or integrate into a single tool, however the lack of a GOV search API that works well made me lose interest.

The Place Cleanup tool has as its first level search, a search in the local data (NOT in the GeoNames data). That can greatly assist in finding duplicate places and merging them.

4

As mentioned further down in your quote, the historical data that the GOV system has is nice, particularly for the London boroughs.

5

0013496: GetGOV Import Fails

Done, thank you :slight_smile:

1 Like
6

Can someone not on a Mac test the GetGOV gramplet. The GOV team migrated their software and just fixed one of their issues. But it still fails on a Mac with an SSL problem. I think they still have another issue with their migration, but need to have it tested on Linux or Windows to verify.

Can you try the GetGOV gramplet for Grosselfingen, Zollernalbkreis, Germany? That place has the GOV-id of :

GROGENJN48KI

Then report whether it crashed the gramplet or loaded a set of places.

7

image
image451Ă—256 14.6 KB

The search form is at:
    https://gov.genealogy.net/search/extended?lang=en

Fedora
Plugin name: GetGOV [Gramplet] Id: GetGOV Gramplet Version: 1.0.21

1017353: ERROR: grampsapp.py: line 188: Unhandled exception
Traceback (most recent call last):
  File "/home/districtsupport/.gramps/gramps52/plugins/GetGOV/getgov.py", line 341, in __get_places
    self.__get_types()
  File "/home/districtsupport/.gramps/gramps52/plugins/GetGOV/getgov.py", line 371, in __get_types
    response = urlopen(type_url)
               ^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 557, in error
    result = self._call_chain(*args)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 749, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/lib64/python3.11/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
8

It looks like there have been some changes to the gov.genealogy.net site. The URL http://gov.genealogy.net/types.owl/ ​​now throws a 404 error. The URL http://gov.genealogy.net/types.owl works.
Changing line 370 from :

        type_url = 'http://gov.genealogy.net/types.owl/'

to

        type_url = 'http://gov.genealogy.net/types.owl'

fixes the bug.

9

Gary had me do some experiments in a backchannel with the Linux version where that allowed it to work. (Although we found changing to https also eliminate some delays.)

But it did not resolve problems for the macOS and had become unreasonably slow. Also, his example GetGov Place creates 41 Gramps Places and builds a defective hierarchy.

So he’s working with the maintainers of the site. Changes are likely to happen soon.

10

Modifying the type_url at line 370 worked for me. Running 5.2.3-2 on Win10

1 Like
11

The GOV site had a major update. The GetGOV gramplet fails with this update.

For non-Macs, the changes described fixes things. For the Mac (Intel and ARM), I am still working on a fix.

I will merge this change for now and make a separate PR to fix Macs, given it may take some time.