Make the AIO installer work again

Development
1

Last week @codefarmer and I had a private discussion triggered by an earlier question about developing Gramps on Windows without MSYS2. And in the public part, I found that there is no easy way to do that, because that would mean that you’d need to download all dependencies yourself, of try to make Visual Studio Code do that, and that does not seem to have much of an idea on where to find those. And all these dependencies, including Python, are installed by the build.sh script in gramps/aio. And once you’ve run that, you can even build and run Gramps from source in the same way as I normally do on Linux.

There is a big problem remaning however, and that’s with the same build.sh script, which is not able to build a working installer. And in fact, in its current incarnation, it can’t build an installer at all, not even when I checkout the AIO 5.2.3 branch. And that’s probably because MSYS2 or the dependencies changed too much in the mean time.

And today the status here is, that I can buld an installer, but that does not create a working Gramps 5.2.4, and that’s what I want to be able to, even though there is no official decision about such a version. I got the same problems with master, so the problems seem to be sort of independent of the target version.

I do all my work on Linux, so the problem is not a real obstacle for me personally, but I do like to have a working installer for Windows, just in case, and for those who can’t move to Linux as easily as I can.

This is why I call for help. Is there anyone who knows how to make this work again, on Windows 10 or 11? And if you know, can you prove it by sharing updated build and setup scripts, so that I can test whether they work here too, on Windows 10. I had to quit using 11, because my new HP machine broke down, and I’m working on an old Medion now.

1 Like
2

See WIP: AIO build updates by hgohel · Pull Request #1821 · gramps-project/gramps which looks like it is related.

1 Like
3

It is, and that GitHub user is codefarmer. I did not push my own changes to my repo, but I am working with similar ones, and still get an installer that creates a broken Gramps.

Can you try a build with his scripts? Or some of your own?

1 Like
4

Unsurprisingly, WIP: AIO build updates by hgohel · Pull Request #1821 · gramps-project/gramps also fails on my Windows PC.
The problem is the call to cx_Freeze. It appears to fail to find 3 packages:

  • “pydotplus”,
  • “pygraphviz”,
  • “pydot”

If they are commented out, python setup.py bdist_wheel gets much further.

pacman -S --needed --noconfirm mingw-w64-x86_64-python-graphvizis called at the start of the script.
pip install --upgrade pydot pydotplus is also run but gives an error

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
    $MINGW_PACKAGE_PREFIX-python-xyz', where xyz is the package you
    are trying to install.

    If you wish to install a non-MSYS2-packaged Python package,
    create a virtual environment using 'python -m venv path/to/venv'.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.

    If you wish to install a non-MSYS2 packaged Python application,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. Make sure you have $MINGW_PACKAGE_PREFIX-python-pipx
    installed via pacman.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.```


`pacman -S --needed --noconfirm mingw-w64-x86_64-python-pydotplus` fails with `error: target not found: mingw-w64-x86_64-python-pydotplus` but I've not looked any deeper into the problem.
5

The pip install error shows a couple of remedies, but it’s so long, and it’s hard to decide which is the best. I got the same error on LMDE 6. and there I opted for the one mentioned at the end, which is the --break-system-packages option. It was safe to try that there, because I have timeshift that creates differential backups every day.

And since the MSYS2 environment is not an essential part of my Windows setup, I added that option in build.sh too, at the end of all pip install lines.

6

There was a problem with the first 5.2.3 AIO build resulting in the 5.2.3-2 release for AIO.

https://gramps-project.org/bugs/view.php?id=13360

@Nick-Hall determined the current version of cx_freeze was causing the problem.

https://gramps-project.org/bugs/view.php?id=13360#c67026

7

I know that fix, but with that in place, the build itself fails, even when I checkout the branch tagged as 5.2.3.

8

I made some progress. See Msys2 update changes for aio by stevenyoungs · Pull Request #1826 · gramps-project/gramps
Hopefully it’s just a question of finding out where the paths are not correctly adjusted.

2 Likes
9

I can now build an AIO installer which then leads to a working gramps installation.
I hesitate to say that everything works, but the basics appear OK, in English at least.

3 Likes
10

Very nice, thanks! Looks good in Dutch too.

2 Likes
11

Some more observations now I’ve gone down this rabbit hole!

  1. the UPX compression does not seem worth the additional complexity.
    The installer was marginally larger with UPX compression, whilst the reduction in installed program size was only around ~1.5% (< 4MB)
    I’ve included a commit in my PR to remove UPX compression. I’d be interested if anyone has a reason to preserve UPX compression of the 3 binaries that are compressed.

  2. at the end of the installer, by default, it will run gramps.
    This is fine, but it runs gramps with elevated permissions. In my view this is not sensible, and we should run gramps with normal user permissions

    image
    image841×101 5.05 KB

  3. A nice side effect of running with elevated permissions is that the python interpreter is able to populate the __pycache__ directories. Having the __pycache__ directories improves gramps startup time by approximately 25% (measured by hand). If we want this, maybe we should silently start and stop gramps from the installer (or include the pycache data in the installer?)

  4. The output of aio\build.sh is not deterministic.
    That is, each time it is run, the output seems to vary slightly. I compiled the non UPX version 4 times and ended up with the following installer sizes
    107,554,984 bytes
    107,553,105 bytes
    107,557,273 bytes
    107,562,765 bytes

  5. The official gramps 5.2.3 AIO installer is smaller: 103,390,942 bytes
    There seem to be several factors attributing to this, all expected:
    a. changes to gramps source since 5.2.3 was released
    b. changes to python packages \ dependencies
    c. changes to line endings (I checked out with CR-LF endings, the official release uses LF)

1 Like
12

I plan on a maintenance release this weekend. If this is working, then I’ll use it to make the Windows AIO.

3 Likes
13

It would be nice if we can include a fix for the version numbers not shown in Windows apps:

14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.