Enhanced PostgreSQL Addon Submitted

This path doesn’t exist on Win10. What is it on Linux?

OK found the issue - there was a bug in the configuration loading logic that caused your connection_info.txt file to be ignored.

The Problem:
The code had dual configuration modes (environment variables for GrampsWeb, and file-based for standard Gramps). Due to incorrect OR logic, when Gramps passed your tree ID, it triggered the environment variable path which had my hardcoded development defaults (192.168.10.90:5432), completely bypassing your connection_info.txt.

The Fix:
I’ve corrected the logic so connection_info.txt is properly used for standard Gramps installations.


How to Get the Fixed Version

Option 1: Download Individual Files (Easiest)

  1. Download the fixed postgresqlenhanced.py:
    https://raw.githubusercontent.com/glamberson/gramps-postgresql-enhanced/update-gramps-606-compatibility/postgresqlenhanced.py

  2. Right-click → Save As → save to your desktop

  3. Replace the file in your plugin directory:

    • Location: C:\Users\Ulrich Demlehner\AppData\Roaming\gramps\gramps60\plugins\PostgreSQLEnhanced\
    • Replace: postgresqlenhanced.py
  4. Also update INSTALL.md (has Windows AIO instructions now):
    https://raw.githubusercontent.com/glamberson/gramps-postgresql-enhanced/update-gramps-606-compatibility/INSTALL.md

Option 2: Download Entire Branch

  1. Go to: GitHub - glamberson/gramps-postgresql-enhanced: Enhanced PostgreSQL database backend for Gramps genealogy software with psycopg3, JSONB support, and advanced queries
  2. Click the green “Code” button
  3. Select “Download ZIP”
  4. Extract the ZIP
  5. Copy all .py files from the extracted folder to:
    C:\Users\Ulrich Demlehner\AppData\Roaming\gramps\gramps60\plugins\PostgreSQLEnhanced\

After Updating

  1. Verify your connection_info.txt is in the correct location:
    C:\Users\Ulrich Demlehner\AppData\Roaming\gramps\gramps60\plugins\PostgreSQLEnhanced\connection_info.txt

  2. Ensure it contains your settings:

    host = 192.168.178.20
    port = 5434
    user = your_postgres_username
    password = your_postgres_password
    database_mode = monolithic
    shared_database_name = gramps_monolithic
    
  3. Restart Gramps

  4. Check it’s working:
    The connection should now use 192.168.178.20:5434 instead of defaulting to localhost:5432


Enable Debug Logging (Optional)

To see what’s happening, before starting Gramps:

Windows Command Prompt:

set GRAMPS_POSTGRESQL_DEBUG=1
"C:\Program Files\GrampsAIO64-6.0.6\gramps.exe"

Check the log file at:
C:\Users\Ulrich Demlehner\.gramps\postgresql_enhanced_debug.log

The log will show which config file is being loaded and what connection parameters are being used.


Let me know if this resolves the connection issue, or if you need help locating any of these files/directories.

Greg

Where should this file go? I was assuming that is should also go into the plugin folder.

I now don’t see the connection error message any more so the fix appears to have resolved this issue. But I now get the following error when trying to create a new tree with pg_enh as db backend:

Error Details:

13015: WARNING: _manager.py: line 328: Plugin error (from ‘postgresqlenhanced’): No module named ‘concurrency’
13017: ERROR: grampsapp.py: line 187: Unhandled exception
Traceback (most recent call last):
File “C:\Program Files\GrampsAIO64-6.0.6\gramps\gui\dbman.py”, line 1058, in __new_db
self._create_new_db(dbid=dbid)
File “C:\Program Files\GrampsAIO64-6.0.6\gramps\gui\dbman.py”, line 1069, in _create_new_db
new_path, title = self.create_new_db_cli(title, create_db, dbid)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Program Files\GrampsAIO64-6.0.6\gramps\cli\clidbman.py”, line 341, in create_new_db_cli
newdb = make_database(dbid)
^^^^^^^^^^^^^^^^^^^
File “C:\Program Files\GrampsAIO64-6.0.6\gramps\gen\db\utils.py”, line 87, in make_database
raise Exception(f"can’t load database backend: ‘{plugin_id}’")
Exception: can’t load database backend: ‘postgresqlenhanced’

System Information:

Gramps version: AIO64-6.0.6–1
Python version: 3.12.12
BSDDB version: not found
sqlite version: 3.51.0
orjson version: 3.11.4
LANG: de_DE.UTF-8
OS: Windows

GTK version    : 3.24.51
gobject version: 3.54.3
cairo version  : (1, 28, 0)

Are there other modules that are not included in the Gramps “site packages archive” but are needed by pg_enh?

OK sorry . I’m rushing. You’ve hit a dependncy issue - postgresqlenhanced.py imports other modules (concurrency, connection, schema, etc.) that also need to be updated.

The easiest way to get ALL the required files:

Download Complete Plugin as ZIP

  1. Go to: GitHub - glamberson/gramps-postgresql-enhanced: Enhanced PostgreSQL database backend for Gramps genealogy software with psycopg3, JSONB support, and advanced queries

  2. Click the branch dropdown (says “master”) and select: update-gramps-606-compatibility

  3. Click the green “Code” button → Download ZIP

  4. Extract the ZIP file

  5. From the extracted folder, copy these .py files to your plugin directory:

    postgresqlenhanced.py
    postgresqlenhanced.gpr.py
    connection.py
    schema.py
    schema_columns.py
    migration.py
    queries.py
    concurrency.py
    __init__.py
    
  6. Copy them all to:

    C:\Users\Ulrich Demlehner\AppData\Roaming\gramps\gramps60\plugins\PostgreSQLEnhanced\
    

    (Replace the existing files)

  7. Also copy INSTALL.md and README.md if you want the updated documentation

  8. Restart Gramps

That should resolve the “No module named ‘concurrency’” error.


To answer your question: No, there are no external Python packages needed beyond psycopg. All the modules (concurrency, connection, schema, etc.) are part of the PostgreSQL Enhanced addon itself - they just all need to be present in the plugin directory together.

The issue was my original instructions only mentioned downloading postgresqlenhanced.py, but it depends on the other .py files being there too.

Greg

Ok. I had to copy all the py files, not only those that you listed. Now I’m back at the connection error: port 5432, no authentication. I just checked the connection_info.txt and the Gramps preferences, both are ok and point to localhost:5434. So I guess that old problem of ignoring both settings still exists. But on the bright side, we resolved the dependency issues for the time being.

Can we circumvent this problem with hardcoding my local setting into a python file? That would be a quick and dirty way to resolve it until you have had enough time to figure out what the real problem is.

Connected to this is another issue. If I understand your config instructions correctly, I need a PostgreSQL database called gramps_monolithic or whatever the connection_info.txt says in

shared_database_name = gramps_monolithic

Is that understandig correct?

Update:

I set the environment variables mentioned in connection.py (PGHOST etc) to my values. Now the connection error (localhost at 5432) is gone and I’m at another error:

Sorry for the German, but it basically says that it tried the connection at localhost:5434 so the environment variables did the trick. Now the database (file?) could not be found. The last string shown (C:\Daten.Backup\gramps\grampsdb\6945aaf1) locks like the folder names Gramps will use for the trees defined in Gramps and the path is where I have my Gramps trees.

Ok, I guess, I get the idea from the other tree folders I have. There must be a database.txt in that folder 6945aaf1 with the backend being used by the tree (here: postgresqlenhanced) and a name.txt with the name of the tree. I provided both files but still got the same error message. What’s obviously missing is the name of the PostgreSQL database (in my case gramps_monolithic) but I don’t see where it is missing. Now I’m stuck …

Apologies for the complexity - I’ve been making significant changes to properly integrate with Gramps’ configuration system, and you’re hitting moving targets.

The issue is that the latest version uses per-tree configuration files instead of the central connection_info.txt. Here’s how to fix it:

Create a settings.ini file for your tree:

  1. Find your tree’s directory. It should be at:
    C:\Users\Ulrich Demlehner\AppData\Roaming\gramps\grampsdb\<tree_id>

    The tree_id is an 8-character code like 6894f362. You can find it by looking at what folders are in the grampsdb directory.

  2. In that tree directory, create a file called settings.ini with this content:

[database]
host=localhost
port=5434
user=your_postgres_username
shared-database=gramps_monolithic
pool-size=5

Replace your_postgres_username with your actual PostgreSQL username.

  1. Restart Gramps

  2. When you open or create the tree, Gramps should prompt for the password (it’s not stored in the file for security).

Regarding your database question: Yes, you need to create the PostgreSQL database. On your PostgreSQL server (localhost:5434), run:

CREATE DATABASE gramps_monolithic;

Then grant your user access to it.

Please don’t hardcode settings in Python files - that will make updates difficult and is unnecessary. The settings.ini approach is the standard Gramps pattern and should work once it’s in the right location.

Let me know if this resolves the connection issue.

Greg

No, the settings.ini does not resolve the issue. I still get the same strange error message with the database. Why does the error message show the folder name of the the tree instead of the PostgreSQL database that exists?

And yes, of course I see your point and I fully agree with it, but when trying to figure out what’s going wrong, a hack of the kind I proposed will save a lot of time and help to identify where the problem is. So please have mercy with me :wink:

Yeah, OK. For debugging, hardcoding can help isolate whether it’s a config loading issue or an actual connection problem.

To hardcode your settings for testing:

Open postgresqlenhanced.py and find the _build_config_from_env method (around line 1057). Replace it with:

def _build_config_from_env(self):
    """Build configuration - HARDCODED FOR TESTING."""
    return {
        'host': 'localhost',
        'port': '5434',
        'database': 'gramps_monolithic',
        'user': 'your_postgres_username',
        'password': 'your_postgres_password',
        'database_mode': 'monolithic'
    }

Replace the username and password with your actual PostgreSQL credentials.

Then set this environment variable before starting Gramps:

Windows Command Prompt:

set POSTGRESQL_ENHANCED_MODE=monolithic
"C:\Program Files\GrampsAIO64-6.0.6\gramps.exe"

This forces it to use the hardcoded settings via the environment variable path.

If this works, we know the config is correct and the issue is with settings.ini loading. If it still fails, we know there’s a deeper connection problem.

Let me know what happens.

Greg

Oh, I guess you overlooked that I resolved the problem by using environment variables. The connection issue is gone, we are now at the database issue. Why is Gramps not able to figure out the correct name of the database?

1 Like

Our post keep crossing each other … :wink:

Ok. My current status: I put everything (host, port, database, user, password) in environment variables and this did the trick. Gramps now connects to the database and I can see those connections in pgAdmin. Some 30 seconds or so nothing happens and then an error message appears:

So the connection times out. I see in pgAdmin that the connection is still active. The connection stays active even after closing Gramps which is not a good sign. I have no idea what this genealogy_user:… has to say since my IP address is 192.168.178.20 and my port 5434. Does this come from your Python code?

Now I’ll call it a day. Looking forward to your comments tomorrow!

Yes I know this screen without knowing German. First confirm you’re configuring for monolithinc mode. Right? Also, when you look in preferences under the Family Tree tab, does it say PostgreSQL Enhanced only in the Backend type or does it say PostgreSQL Enhanced (Monolithic) or (Separate) ?

I’ve been redesigning that, and I’ve lost track of what version I had up when you downloaded it.

There’s a hack that happens to make monolithic mode work. It actually puts a prefix on each database you connect to, making the actual databases entries in a table. So at this point besides the issues we’re discussing now, I also need ot understand if you’re creating a new database now or triyng to use osmehting that exists. I assume this is all new, but I am now starting to lose the ability to think so I’m losing detail.

Just give me a little mroe detail, and I’ll put it together. You’re almost there.

Greg

1 Like

I’ll go to bed now, too. But I’ve submitted a new version which may or may not resolve your issues. (See: Update PostgreSQLEnhanced to v1.7.3 by glamberson · Pull Request #781 · gramps-project/addons-source · GitHub .) The genealogy_user and GenealogyData2025 strings equate to the postgres username and password respectively of my test scripts.

Your issues are minor, but I’e lost the ability to think, so I’ll check back tomorrow.

Greg

1 Like

Hi Greg,

I hope you slept well :wink:

Let me summarize what my current status is:

(1) I have to confess I’m not familiar with GitHub so the UI is quite confusing for me. I’m confident (but not 100 % sure) that I managed to download your last file versions. So it would be a good idea if you gave me some ways to verify that (e g a few code lines you changed so that I can check if they are really in my downloaded files).

(2) Based on the assumption that I got the correct files, I replaced to old versions in the plugin folder with the new ones. No hacks from my side, just the files I got from GitHub. There is a connection_info.txt in that plugin folder. Its content:

# PostgreSQL Enhanced Connection Configuration

# This file controls how the addon connects to PostgreSQL

# Connection details

host = 192.168.178.20    # PostgreSQL server address
port = 5434              # PostgreSQL port
user = postgres          # Database username
password = password      # Database password

# Database mode: ‘separate’ or ‘monolithic’

database_mode = monolithic

# For monolithic mode only: name of the shared database

shared_database_name = gramps_monolithic

# Optional settings (uncomment to use)

# pool_size = 10         # Connection pool size
# sslmode = prefer       # SSL connection mode
# connect_timeout = 10   # Connection timeout in seconds

Additionally, I have everything in enviroment variables (PGHOST etc).

(3) Now I see in the preferences dialog of Gramps 6.0.6 that there are two PostgreSQL Enhanced DB backends:

I guess this is kind of a proof that I downloaded the correct file versions from GitHub.

(4) I created a new tree with the monolithic DB backend. No problems with that, all the old problems from yesterday are gone.

This is the Gramps DB folder for this new tree containing one single line in both files. So this appears to be ok, but no settings.ini or something else.

(5) When I open this new tree on pg_enh_mono, I see in pgAdmin that a session is started with a connection to my database gramps_monolithic.

So we are obviously standing at the door of gramps_monolithic and the door appears to be open meaning that my settings and authentication are fine. But of course, I do not know and have no way to check it, if the settings and authentication are based on my environment variables or something else.

(6) After some 30 sec I get the good old timeout error message and the tree is closed in Gramps (i e the status line at the bottom says: no active object, the tree management dialog shows no open tree). The session is still open in pgAdmin.

(7) After closing Gramps completely, the session is still open in pgAdmin. If I try to close it manually, I get an error message in pgAdmin without any details about the error. I guess I have to shutdown the PostgreSQL 18 service to kill the session.

Any open questions from your side?

Hi Ulrich,

I’m sorry I kind of took the day off. Yes, I didn’t want to have to get you into the guts of github, but it became clear your problem was going to involve a patch.

I’m looking at your problem now, and frankly I’m confused. Could you confirm exactly what version of Gramps you’re using?

No problem, I somehow figured out what to do on GitHub.

Sorry for the confusion: for the PostgreSQL tests, I’m using Gramps 6.0.6, but for my daily work I’m using Gramps 5.1.6 on BSSDB because of the regrettable performance of 6.0.6 on SQLite. So everything what I said in my “status description” above refers to Gramps 6.0.6. I’m using PostgreSQL 18 for the tests.I’m using the same PostgreSQL 18 server for my “Gramps work-arounds” I mentioned above, so I feel confident to say that the database server is basically ok.

OK thanks.

Right now I’m looking at a few different things. One is I introduced a regression in error checking and correction in my haste, and I also left my testing values hardcoded into what you have.

Looking at your setup, I think I see what’s happening. You have PGHOST/PGPORT environment variables set, but not POSTGRESQL_ENHANCED_MODE=monolithic. This creates a confusing hybrid:

  1. The plugin tries the ConfigManager path (not env-var path)
  2. settings.ini should be created but isn’t
  3. Connection still works because connection.py separately reads PGHOST/PGPORT

To get clean diagnostics, please temporarily:

  1. Unset all PG environment variables* (PGHOST, PGPORT, etc.)
  2. Set only this one: POSTGRESQL_ENHANCED_MODE=monolithic
  3. Create/verify settings.ini in your tree folder with your connection details

Or, show me the output of this in Command Prompt before starting Gramps:
set PG
set POSTGRESQL

This will show what environment variables are set that might be interfering.

Greg

Note I completely edited and changed the above as I went back to look at everyhting you;ve done previously.

Sorry Greg, but this will have to wait until tomorrow. I’m currently running my “nightly build” of https://landgericht-griesbach.info/bayernatlas.php and this means that no other instance of Gramps can be started.

no, this is set:

PGDATABASE=gramps_monolithic
PGHOST=localhost
PGPASSWORD=password
PGPORT=5434
PGUSER=postgres
POSTGIS_ENABLE_OUTDB_RASTERS=1
POSTGIS_GDAL_ENABLED_DRIVERS=GTiff PNG JPEG GIF XYZ DTED USGSDEM AAIGrid
POSTGRESQL_ENHANCED_MODE=monolithic

Greg, may I propose a slightly different approach? I guess that it will be quite trivial to figure out which way I get access to the PG database once we resolved the timeout issue. I’d assume that this timeout issue is the more important one since it appears to point to some fundamental connectivity and communication issues. Those could hypothetically be connected to my way of getting access to the database but I do not really believe in that hypothesis. Once I have access to the database (which I obviously have), it should be irrelevant how I got there and the database should start listening to my commands. But the database appears to hear nothing, so the timeout. I guess we should focus on this problem first.

Your opinion?