Is this being done manually on every release? Who is taking care of that?
Also, just out of curiosity, why are we not using a more standard tool like PyInstaller? Sorry, being very naive here, I’m not even a Windows user. But I imagine it would be great if we could automate builds using Github CI/CD.
There have been 1,200 downloads of the current Win64 installer from SourceForge, over 6,000 for the two 5.1.3 revisions. (For all other installers combined: 180 for 5.1.4; 1,800 for 5.1.3)
This Gramps for Windows with MSYS2 - Gramps is probably a better description of what I’m doing to build the Windows AIO. Note that I have been NOT updating the MINGW libraries for quite a while, as the libraries used pretty much work, and there have definitely been problems associated with using the most recent libraries.
I expect to try the most recent libraries again if/when we ever start the 5.2.x release cycles. I recall in particular that newer versions of Python end up causing bsddb dbs to get their pickled blobs updated in a way older versions of Gramps cannot read. Also I have some patches compiled in to GTK that make the many Gramps windows stack properly; the last time I looked those patches had not yet been committed to the GTK builds.
So be aware of these if you try to build an AIO (or just use the MSYS libs with Gramps Python files) for your own use.
Thanks @prculley! Do you have a suspicion why Python 3.6.4 in AIO might be complaining about the missing typing module even though that’s part of the standard library?
I recall in particular that newer versions of Python end up causing bsddb dbs to get their pickled blobs updated in a way older versions of Gramps cannot read.
If I’m understanding correctly what you mean, it’s an issue that affects not only bsddb but also SQLite and not only Windows but also Linux and Mac. I pointed that out already some time ago:
But I don’t think it’s a reason not to use a more recent Python version. Most people are likely already using Python 3.8, 3.9, or 3.10, so already have some of the blobs in their database in the newer format. It’s not an issue unless they go back to a pre-3.8 Python with the same database.
The build process doesn’t include Python modules unless they are actually used by Gramps core features. So lib/typing.py doesn’t get included. There are a number of items in the standard library that don’t get included, it is fortunate that Gramps is so large that the majority of common items DO get included. Also, Gramps should run on older systems, so we are supposed to design to run on Python v3.3.0 or newer. The ‘typing’ lib was not introduced until 3.5. If/when 5.2.x get introduced, we might be upgrading our minimum Python version supported rules.
As for the blob issue, at the time I released a ‘new-libs’ version of the AIO I had promised that users could revert back to the original version if there were problems with the newer libs (there were some others). Since the older AIO didn’t know how to read newer blobs, it became an issue for bsddb. I don’t recall why it was not also an issue for the SQLite dbs, maybe our code was written differently there…
Can I suggest that the entire Python standard library be included in future Windows builds? Developing addons for Gramps is already quite restrictive as we cannot depend on external libraries, but not even being able to use the entire standard library is too restrictive in my opinion (regardless of Python version). Since Gramps on Linux always has the full standard library available, this will lead to some addons not working on Windows due to missing standard lib modules.
That would take some work. The current cx_freeze systems deliberately scans through the code looking for import statements, creating a list of the required imports, and copying only those that are referenced.
I’m not aware if this behavior can be modified. I suspect to do this we would have to create a dummy Python module with all the standard library imported. I’m not sure where I could find a list of all modules in the standard library, although I haven’t tried yet…
Another question is: could the installer trigger a pre-compile of the Python source?
This would move the long initial delay in starting Gramps from the first-time use (where it is an blind latency liability) to the install (where slowness is expected and indicated with progress bars.)