Psycopg2 pre-requisite fails for PostgreSQL addon

While trying to reproduce a Login/Password error for other discussions (Database location preferences: Are there examples of using Host and Port and Anmeldung / Kennwort [login/password]), I realized that enabling the Preferences options for database host and login required the PostgreSQL database addon.

Installing the add-on seems impossible due to failing the psycopg2 prerequisite.

Using the Addon Manager setting tab’s “Allow Gramps install required Python modules” apparently (from another thread) uses PIP and is only appropriate for Windows OS. Addon Manager showed the Install button with the setting selected. But threw an error when used:

4279492: ERROR: grampsapp.py: line 188: Unhandled exception
Traceback (most recent call last):
  File "/app/lib/python3.11/site-packages/gramps/gui/plug/_windows.py", line 289, in __on_install_clicked
    subprocess.check_output(
  File "/usr/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.11/subprocess.py", line 1953, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'pip'

I am using Fedora.
(Gramps for Desktops 5.2.1 and Fedora Linux 37 with GNOME 43.9)
So asking Perplexity, it suggested several terminal options to install the package:

  • sudo dnf install python3-devel postgresql-devel
  • python3 -m pip install psycopg2
  • python3 -m pip install psycopg2-binary

Each of these claimed to have installed the package… but Addon Manager still claimed it was un-installed and no PostgreSQL database backend option appears in the Preferences.


from another message by @DavidMStraub

Both the Shared and Regular fail to install.

Yes. We should probably remove this feature for all operating systems other than Windows.

I suggest that you don’t use the PostgreSQL backend. It is provided for expert users.

2 Likes

I don’t want to use it. I just want to do bug reproduction testing and testing for updating the troubleshooting in the wiki.

Try installing the python3-psycopg2 package using the package manager.

1 Like

That’s nice, really, but it is quite difficult on another OS. I bet that it’s quite easy on mainstream Linuxes, like Debian, ubuntu, and derivatives, partly because it’s easier to find answers on the web. Finding answers for Fedora is probably more difficult, and your Fedora is also quite old. Their current offer is version 40.

In this case, things may be obscured by the fact that Volker has part of his data on a German cloud service, and I don’t even know whether he installed PostgreSQL at all. The error suggests that he did not, because there’s nothing listening to port 5432, which seems to be the standard port for PostgreSQL.

I really didn’t expect to find an answer for those postings. But the attempt is exposing unexpected complications that also are not documented. I am working on those rabbit holes.

It is rare that there isn’t some tweak (or fix of a complete ommission) needed in the wiki when searching for a what to post in a reply.

I prefer to link to the explicit workflow in the wiki while giving a workflow summary in the reply. So people who just need a nudge don’t have to wade through excess detail… and people who need detail can find the precise steps. (And it saves me from the drudgery of writing a novel.)

The wiki isn’t “readable”. It is a reference-style. So I’ve never done a read-through for ommisions. I doubt anyone has. (Although @GaryGriffin impressed me with his tenacity while generating the offline PDF process.)

I understand that, and IMO, the wiki isn’t even searchable, because when I try, it often sends me to old pages, and in many cases, it also gives too many of them.

I must admit that in the case of PostgreSQL, the search worked, and this page is also the 1st result on Google:

https://www.gramps-project.org/wiki/index.php/Addon:PostgreSQL

The contents of that page work quite well for me, because they were made for ubuntu, but you need to read between the lines to realize that. There is a note that says that they’re for Linux, but that’s a bad joke, because they don’t cover Fedora, nor any other flavor than Debian/ubuntu.

I can tell you that there are many omissions, like the missing texts for Fedora and Windows here, and I know many many more. And at the same time, I don’t really care, because I know that I can also use the readme and install documents that are included with the source.

Absolutely true! I think that is one of the best things about the downloaded PDFs. They are VERY searchable and only give matches for a current version. And users can limit the results to their own language. (Since the translations tend to updated in response to the English wiki pages being updated, foreign language pages are newer and rank higher in the wiki’s search results.)

1 Like

Don’t I know there are holes! Several of us have spent years (nearly a decade for me) patching them in End-User docs … mostly with information gathered be experiment from with Windows version. The developer docs are mostly impenetrable to people who aren’t conversant with a few flavors of Linux. (As you said, mostly the Ubuntu dialect.)

And then the question is: What can we do about that? Is it realistic to keep expanding the wiki? Do we have alternatives, like more readme files on GitHub, where we can use markdown for simple editing? Would users read those?

For me, as someone who likes building Gramps from source, it’s nice to have an INSTALL file which is up-to-date, meaning that it covers subjects like the extra --root=/ option that I need to install 5.2 on LMDE 6, or Debian bookworm, or using pip3, which works on many platforms, but not on LMDE 6. But that file is mostly likely only read by users who follow the same path as I do.

I tend to ignore most docs, because I know that they’re outdated at the moment that you save them, especially developer docs, for the simple reason that you can’t keep up with all new Linux flavors that appear, every day. And even when you concentrate on the big ones, it’s still difficult, because that means that you have to upgrade your Fedora from version 37 to 40, and others would need to install Ubuntu 24.04, which brings its own dependency problems, which I don’t see here, because my Mint 21.3 is based on Ubuntu 22.04. And then there’s Debian too, moving in its own pace.

Most GitHub repositories contain a README.md file in GitHub markdown. That conveys to the Plugins subfolder with the Addon is installed… even when the addon won’t run due to prerequisite problems.

And the README.md works fine when browsing the hosted version. The GitHub server parse the file and serve html. But the browsers freak out when trying to view .md files that are stored locally. The browsers open hundreds of tabs intil the application crashes.

It would be great if we had a markdown viewer for Addon Help so it doesn’t crash and the Addon specific help can be displayed when offline.

I don’t think the wiki needs infinite expansion. But it does need to document the interface elements.

Even if that documentation points out that while the elements that appear to be active, they are only needed for advanced network storage users and should be left blank. This might (that’s MIGHT, not always) include an example of what happens when they are misused. (So we don’t spend so many hours retracing causes of a problem that only happens once in a blue moon.)

Are you sure about that? Are enough users reading that? Does improving these texts really help to prevent problems?

I’m provocative here, because my experience is, that most users don’t read the docs, and either start clicking, thereby choosing weird database backends like PostgreSQL, or get scared and walk away. Or phrased in another way, they may think that they know what they’re doing, so they don’t need to read the fine manual, or the manual itself scares them away too.

I’m writing this, based on observations of my late mother, and of my former girlfriend, who is an engineeer, just like me, but had so much fear of breaking things, that she needed, personal guidance to update Android or Windows. I gave that, and now she can do that herself.

I saw this with media too, in recent messages, which is an area where we have texts on the wiki. And even though they exist, users still come here for examples, because that’s how they learn.

Not provocative but certainly showing the deep scepticism vital to the socratic method.

It is good to question the basic assumptions.

Particularly so since the Help button (which relies on links to the wiki or a fully qualified URL.) just had a major upgrade in plugin registration. Previously, only gramplets were able to register a spot in the wiki for their documentation. Now, ALL plugins can point to their documentation. (That includes built-in as well as add-on.)

One of the complexities of the plug-in system is that even experienced users may find it difficult to find a feature in the interface. The HOPE is that the Help system can alleviate finding the corresponding GUI control. Which means more anchors in the wiki for built-ins and figuring out how to make the anchors work with the offline PDFs and translations of the wiki and the translated PDFs.

I understand that, and I think that it helps to create more anchors, but that’s a huge task, because you need those for all languages, not just English. I just ran a test, and when I click Help in the Database Manager, or whatever that thing’s called in English, I’m sent to an empty page, both in 5.1 and 5.2. And when I open the Database tab in 5.2 Preferences, and click the Help button there, I find that there is a well written page for that, in Dutch, but it’s a page for all preferences, not just the Database tab, so I’m still sort of lost.

And in the mean time, we still don’t have a solution for PostgreSQL, which is the question that triggered this thread. We had a user who ran into the rabbit whole of choosing PostgreSQL, and then finding out that he was asked for a name and a password, and that there was no server running at the designated port, and I still have no idea how he got there. I can only guess that for some reason he skipped the initial database manager screen, and went to preferences where he chose PostgreSQL. And then the big question is: Why did he do that? And could we have prevented that with more anchors on the wiki? I really doubt that.

And that’s why I question this approach.

Quick question:

Is there anything I can do to help you solve this for Fedora? I can install version 40 in a VM, or on an empty partition.

I tried installing PostgreSQL on Windows (10), but Gramps didn’t see it, and for that, we have one or two users who got it running, so we can always ask them.

Thanks for your kind offer. I’ve tried multiple ways (sudo dnf and pip) to install psycopg2 which claim be successful. But Gramps doesn’t recognize the prerequisite exists.

I had to install a Python environment use Black, pylint and make.py work. Maybe psycopg2 installed there but Gramps must’ve installed with enough Python to compile caches. Possibly Gramps is testing that path Instead? I’m blundering around in an OS that I know too little about.

Nick suggested using the package manager. But when I asked Perplexity which package manager Fedora used, the reply was dnf… which is what was in the attempts described in the posting that prompted Nick’s reply.

DNF being an evolution of the YUM package manager.
I read the Perplexity response to the prompt “Please give an overview of DNF as it pertains to the YUM package managers for RPM-based Linux distributions. Include etymology of acronyms. Assume audience with some experience with Linux but no formal training.”

Nick is right, and that means that you can use the command line as specified on the wiki, replacing apt with dnf. And that’s not what Perplexity wrote. I never used that, but I know by experience that most AI is not really AI, but just a dumb (and large) language model, which gives you texts that look nice, but do not make much sense. The LLMs hallucinate quite a lot.

What I saw is that when I install pip3, and use that to install psycopg2-binary, which should be enough, the add-on manager still says that the prerequisite fails. And that’s probably because pip3, or python3 -m pip, has no right, not even with sudo, to install the libraries that psycopg2 needs to work.

When I used

sudo dnf install python3-psycopg2

I could see that dnf wanted to install an extra library, and when I acknowledged that, I got a working psycopg2, and the add-on manager showed that all was OK.

I had not tested this on my Linux Mint yet, but when I did, the results were exactly the same, meaning that the pip install looked OK, but wasn’t, and the package install was OK.

There’s still a problem though, because on Fedora, I was not able to switch to the ‘postgres’ account that’s listed in the next chapter on the wiki. And I have no idea how to cure that.

1 Like

It looks like on Mint, the postgres server is started right after install, and on Fedora it’s not, which is confirmed by what I just found on the Fedora site. And that means that you’ll have to do some research over there.

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.