Can someone explain why it is so difficult to allow gramplets to use the native Python installation on Windows instead of the AIO version?
It’s often easier to install Pyton on Windows than it is to install and get Gramps to work on Windows, we already need to install graphviz and some other software to get all up running…
Python can be installed from Microsoft Store, Powerscript or by using Chocolatey…
For some reasons I find more version problems with Python (and java/js) today, than I ever found DLL-hell problems on a Windows 95 machine… 20-30 years ago, even when I add the ActiveX components to the list…
PS. this is a genuine question, because I actually do not understand why it is so difficult…
This gramplets solves an issue for many people, but it is still impossible for “normal” Windows users of Gramps to use this and a few other useable addons, because they are not super-users, they can click a button and install a software with some help, but… well… the rest may be what stop them from actually using Gramps…
The issue that I encounter does not come so much from the OS as from the limitations imposed by the developers of Gramps because of the packaging of Gramps for Windows.
This packaging is super well done, the proof is that everyone can install it without issue, without having to add a lot of prerequisites that you have to look for on the net. There we have a bundle and after installation everything works.
But everything works within the framework of this bundle and no more. The FS package works as Gramps should, being able to use any Python library it needs. No matter the OS.
There, because of this limiting Gramps AIO packaging, with this integrated and non-extensible Python, I try like an unfortunate to find the missing libraries, add them by hand in the gramplet directory, and fix them when I can’t. I must have made a good half-dozen corrections of this type and I haven’t yet arrived at the end.
So, no problem with OS or virtual machine or dual boot -dated it must be said-, but the limitations of an aging integration.
The solution in my opinion would be to chain the pre-requisite installations like Python when installing Gramps, if they are not already available on the machine on which it is installed, even if it means carrying them around with it so as not to download them from the net in order to be autonomous, then add the necessary settings for Gramps to work. Then, I would be unable to implement the solution that I recommend. This packaging for Windows also seems to be a bore from what I can sometimes read about it on the Discourse Gramps.
This solution would make it possible to pip add any library necessary for the addon and benefit from all the richness of the Python ecosystem (perhaps GTK too but I don’t know). There it is impossible
I’m genuinely curious about that too. (Of course, I’m curious about almost everything.) Is there some project documentation that talks about what complications existed when porting to Windoze and their resolution?
I just re-read Tamura Jones’ 2008 article Some Thoughts on GRAMPS critiquing the decision to support Windoze and the complexity of early installs. It has some basic misconceptions about how this open source project typically evolves – Gramps grows organically rather than having board of directors making design & marketing decisions then coordinating everyone toward a common goal. But it voices a lot of valid concerns.
The AIO installer seems to have been a HUGE achievement. It was probably the biggest one that made Gramps a viable choice for me. The macOS drag’n’drop install looks like the easiest. But I cannot follow the pre-AIO install instructions… nor the Linux install instructions from the same period. They were just too convoluted for normal humans.
I suspect that new add-on might mean the AIO installer needs tweaks or that a Prerequisites installer might be needed for the non-technical community. The complaints about pre-requisites negatively affecting the 1st time user of add-ons is quite valid.
So @Nick-Hall comments about enhancing the Plug-in Manager and Gramps Plug-in Registration are very encouraging. They should help novice users install only add-ons with no additional pre-requisites.
(Unfortunately, the complexity of installing the beta 5.2 makes it unlikely that any uninitiated users will be have ANY opportunity to give feedback on the new Plugin Registration features. So if the changes miss the UX mark for that audience, that means no improvements will be possible until the next major release… probably 2 years. By then, the outcry from critics will be so nasty for so long that everyone will be deterred from working on it.)
Installing Gtk is actually the problem. It needs to be compiled with python and other packages in a consistent build environment. Fortunately, there is a project called MSYS2 which provides a library of packages which are pre-compiled to run on Windows. This includes python and Gtk.
Gramps just bundles a selection of these packages into the AIO.
If a package is available in MSYS2 we can include it in the AIO quite easily, although we try to keep the size reasonably small.
If the package is not available in MSYS2 then we would have to compile it ourselves in an MSYS2 environment.
Gramps is pure python and doesn’t actually need to be compiled. Likewise any other pure python modules should also work. Standalone executables should also not cause any problems.
Similar solution such as Chocolatey would also be possible, but in this case it appears not to contain a Gtk3 package.
I think the AiO for the main package is great, but it should be possible to either install/copy needed packages for gramplets/addons and to use them, either natively in Windows or in the gramplet sub folders that needs them…
And seriously, for today’s Windows the size of the Gramps doesn’t matter, it doesn’t matter if it is 60MB, 160MB or 500MB, most of today’s larger software packages is somewhere between 200MB and 2,7GB (the latest download of the On1 PhotoRaw software).
If a miniatyr installer is needed because someone is running a really old version of Windows, then maybe a “minimal installer” would be a better solution than making the main installer full of limitations?
Just asking, is it not possible to make the updates or installation of gramplets dependencies in a way so that they do not need to depend on what’s in the AIO package?
It is no problem with the AIO for the core software at all, but it is when users that wants to use possible features in gramplets, e.g., your great mongodb experiment (just an example), the problems start, because from time to time the packages that some gramplet use and that is bundled with the AIO is seriously outdated, and actually can be a security risk, or they are outdated so that some feature might not be working at all…
I like the concept. But wonder if the install of dependencies from within Gramps would look like malware?
The only reason plugins seem safe is that they install into a User writable space. On the other hand, the dependencies probably go in a “writing is restricted to Admin” space.
Don’t think so, python packages do not trigger a mailware alert as long as it uses pip or some other “installer” and are “signed” or just copies the compiled files to a folder outside the system folders.
I have never had problems with python or Anaconda the same way that I have had problems with e.g., Java… or js…
The standard way to install Python on Windows is to install it to the “user space”, if you want it to be installed for all users (in program files), you will need elevated permission.
if the gramplet folder can hold python packages for a gramplet, then it will not be any problems unless the package actually calls commands that alter the system…
Eventually, there could be a function in the Gramps installer that created a virtual env. based on the natively installed python it is defined in the env. settings of Windows, if it is not there, just trigger a native installation of Python or tell the user that Python needs to be installed…
Maybe a virtual env. that was common for all gramplets?
At least least with the All in one installer we no longer need to do it our selves!
These are the original instructions for installation on Windows of Gramps 3.4.x and earlier They were written when a Windows user also had to (manually) install various other programs (“dependencies”) in order to get Gramps to work. A much easier way to install Gramps on Windows is to use the “all-in-one” installer, since it has all the needed programs (“bundled”). Download#MS_Windows
A quick check shows that the GTK project no longer provides MS-Windows binaries so that stopped my experiment to see if I can update that page for Gramps 5+ !
I contribute a bit to the MacPorts project so maybe I can add a little context even if it is not Windows specific.
Allowing a program to download its own pre-requisites opens a number of issues. As noted above, operating systems are trying to protect users from malware by controlling the process of installing new software. Every new avenue is a new potential attack vector. That doesn’t mean it is impossible to do properly–just that security concerns need careful consideration and handling.
Then there is version management. Pre-requisites may get updated at any time but updates sometimes create as many problems as they fix. Operating systems get updated all the time but sometimes create unanticipated problems. With an all-in-one installer, the packager can test to try to ensure that all of the pre-requisites will play nicely together AND that the installed app will work on the chosen OS version(s). Installing pre-req’s ‘on-the-fly’ will break things for some people, sometimes.
Related to the above, it potentially creates a support nightmare. Suppose there is a subtle interaction between some (version of a) pre-req and a central Gramps element. Only people with that pre-req (or version of a pre-req) would/might experience a bug. The fact is that a lot of users will be unable to recall which additional bits of software they’ve installed. Tracking down such conflicts is very much not fun. That leads to a bad user experience which can end up (unfairly) reflecting on the project overall.
BTW, I will note that installing Gramps via MacPorts DOES allow one to also easily install any desired pre-requisites. That is perhaps the most compelling difference between the .dmg installer (from the Gramps project) and the MacPort’s packaging. AFAICT Chocolately on Windows does not work like MacPorts. It is just another way to download and install the Windows all-in-one installer (as produced by the Gramps project).
I dont have enough knowledge to comment on any of the other things in this thread, but:
I agree with your comment about size. If gramps itself is 100MB or 1GB doesnt really matter it todays age. (If it became 10 gb it might be an issue tho, lol). Any PC the last quite a bit years wouldnt have a problem with it.
I can’t see this as a problem when using virtual environment for python, and no one have said that the core AIO needs to be altered at all, it is just the “addons” that needs anything not in the core AIO. Noone wants to alter the AIO after it has been installed.
But then again, I am NOT a developer, I just try to use software based on i.e. Python that uses virtual environments… and very often hit 10 meters thick walls because of a python driver dont support the latest version of a database engine, that needs to be updated because of security reasons etc.
Version conflicts is one of the reasons for using virt. env. in Python, isn’t it? just as it is in JAVA.
And we already know that 3rd party gramplets are not supported by the core development team.
Maybe I wasn’t clear enough, but ALL I write here is only about 3rd party gramplets, not the core installation of Gramps, not the gramplets included in the core installation, only the gramplets that can/will be installed with any of the plugin managers…
Maybe there need to be another way to include packages in the plugin scripts, I don’t know (e.g., include “PytonVirtEnv/xxPlugin/package.py”)?
and yes, the chocolatey and Powershell installers only download and install software already made as an installer i.e. a msi package. So, I did not suggest installing plugins or gramps with chocolatey, that was just 3 ways that python could be installed on Windows.
PS. I do understand the problems you mention, that’s why I only ask about plugins/gramplets not in the core installation e.g., the AIO installer, if it is possible to use packages installed either to a virtual environment natively or to use a native installation of Python…
There is no problem running multiple versions of Python on Windows, so that shouldn’t be a game stopper?.
And that is the “problem”, if a developer dont want to add a packages or update it, because he/she/hen don’t use that plugin, those who want to use it but are not python developers stands without the possibility to do so…
Ergo, the answer is… normal users cannot keep packages updated unless they learn pyton, install msys2 and figure out how to update packages and then create a new updated AIO for themself…
Yes, indeed, ad it seems possible, because when I install the LifeLineChartView plugin it tells me that it can’t find svgwrite version 1.4, and offers to download it for me. And when I accept that, it does work, so that module is installed somewhere, by the add-on.And it does that for another module too.
I also know that the FamilySearch Gramplet uses PIP to install a module named gedcomx, written by the same author when it’s not found, and also updates it, when the installed version is too old. And this is something that’s quite easy on Linux, where it’s quite easy to install PIP, at system level, so that the add-on can use that, but in Windows it can be more difficult. And @jmichault solved that by offering an experimental Gramps AIO that comes with PIP and some other modules that support these kind of live updates.
I know that some people frown on this, but it is possible, and PIP can install modules in user space, so that there is not much danger for your system.