LLM + Gramps = a useful chatbot

Development
61

Thanks for investigating. I have been using your work with the Flatpak to decide which packages to include in the Snap.

The litellm package that I added to the Windows AIO is untested. I don’t use Windows myself. It was on my to-do list because someone requested it for testing.

Update: The request was linked to PR #762 and the addition of the ChatWithTree gramplet.

1 Like
62

As mentioned elsewhere here, I (and others) have been looking into an alternative method: using Model Control Protocol (MCP). This doesn’t require any dependencies on the Gramps side, but would require some way to run the MCP server.

I believe that the MCP server that talks to gramps-webapi is now packaged with gramps-web. But there are other possibilities that I’ll talk about later that are simpler.

In any event, I think working on incorporating litellm in the gramps stack is probably not the right approach. If it was easily pip-installable, then that would be different.

2 Likes
63

OK. I’ll remove litellm from the next Windows AIO release.

Let me know if you need any funding for a server.

64

@Nick-Hall @dsblank I got a Gramps 6.0.6 flatpak to compile with the LiteLLM dependencies. The archived flatpak can be downloaded as a PreRealease at Release Gramps 6.0.6 with LiteLLM for GrampsChat · OzarkShepherd/flatpak · GitHub Size of the Gramps 6.0.6 flatpak with LiteLLM is 70.9 MB, compared to the current Gramps 6.0.6–1 flatpak size at 46.2 MB. I have not counted or listed the dependencies yet for the LiteLLM flatpak, but the module can be seen at flatpak/DepsAddonLiteLLM.yml at v6.0.6-GrampsChat · OzarkShepherd/flatpak · GitHub My rough estimate of dependencies is in the 40-50 range.

I have not tested the flatpak with Gramps and LiteLLM yet, first I need to figure out how to access the configuration files for the flatpak since it is a little different from a system installation. To be production ready for flatpak users, I believe a configuration window to input API keys and set models like the WebSearch Addon would be necessary.

If nothing else, I got my son’s help in making a script to convert a flatpak-pip-generator json file to yaml so I can manually add specific whl files for amd64 and aarch64 arches when the generated json’s tar won’t compile. So this exercise was beneficial to me regardless of whether LiteLLM gets integrated into Gramps or not.

1 Like
65

I opened the flatpak environment while it was running and tried to run the export and python GrampsChat commands inside the environment. I got some errors, but perhaps it was from running GrampsChat.py since I couldn’t find chatbot.py? Perhaps I was looking in the wrong directory?

Here is the terminal output edited for brevity and to remove the api key. (BTW, this is how to enter the flatpak environment if testing Release Gramps 6.0.6 with LiteLLM for GrampsChat · OzarkShepherd/flatpak · GitHub The flatpak ps command gives the PID for all running flatpaks, which can be used with the flatpak enter command to navigate the environment in terminal.)

[scott@localhost gramps.flatpak6.0.6-GrampsChat]$ flatpak ps
Instance  PID   Application               Runtime
354792817 11993 org.gramps_project.Gramps org.gnome.Platform
[scott@localhost gramps.flatpak6.0.6-GrampsChat]$ flatpak enter 11993 /bin/bash
bash-5.3$ cd .var/app/org.gramps_project.Gramps/data/gramps/gramps60/plugins/GrampsChat
bash-5.3$ export GRAMPS_AI_MODEL_NAME="openai/gpt-4.1-mini"
bash-5.3$ export OPENAI_API_KEY=
bash-5.3$ ls
GrampsChat.gpr.py  GrampsChat.py  __pycache__
bash-5.3$ python3 GrampsChat.py

Fontconfig error: No writable cache directories
	/usr/lib/fontconfig/cache
	/app/cache/fontconfig
	/.cache/fontconfig
	/run/host/fonts-cache
	/run/host/user-fonts-cache
[this fontconfig error looped several times]	

/home/scott/.var/app/org.gramps_project.Gramps/data/gramps/gramps60/plugins/GrampsChat/GrampsChat.py:4: PyGIWarning: Gtk was imported without specifying a version first. Use gi.require_version('Gtk', '4.0') before import to ensure that the right version gets loaded.
  from gi.repository import Gtk, GObject, Pango, GLib
  
 Fontconfig error: No writable cache directories
	/usr/lib/fontconfig/cache
	/app/cache/fontconfig
	/.cache/fontconfig
	/run/host/fonts-cache
	/run/host/user-fonts-cache
[this fontconfig error looped many times]	

Traceback (most recent call last):
  File "/home/scott/.var/app/org.gramps_project.Gramps/data/gramps/gramps60/plugins/GrampsChat/GrampsChat.py", line 14, in <module>
    from gramps.gen.plug import Gramplet
  File "/app/lib/python3.13/site-packages/gramps/gen/plug/__init__.py", line 27, in <module>
    from ._pluginreg import (
    ...<35 lines>...
    )
  File "/app/lib/python3.13/site-packages/gramps/gen/plug/_pluginreg.py", line 44, in <module>
    from ..const import IMAGE_DIR
  File "/app/lib/python3.13/site-packages/gramps/gen/const.py", line 122, in <module>
    OLD_HOME = os.path.join(USER_HOME, ".gramps")
  File "<frozen posixpath>", line 77, in join
TypeError: expected str, bytes or os.PathLike object, not NoneType
bash-5.3$ ls
GrampsChat.gpr.py  GrampsChat.py  __pycache__
  1. the fontconfig error: the flatpak environment limits the ability of the app and its dependencies to write to certain environments. If this is a fatal error, I will have to find a solution and recompile the flatpak. Do I need to fix this?
  2. I found GrampsChat in the data directory. This is a post 5.2 installation on a VM that never had 5.1, so there is no .gramps directory. It is following xdg. Am I in the wrong directory?
  3. Is the type error at the end due to fontconfig in Gnome 49 platform not being able to write to cache?
  4. If it is a writable cache error, would the explicit addition of python-fontconfig to the gramps-flatpak possibly resolve this issue? (I have been using Gnome platform dependencies for fontconfig.)
66

So the problem for the Gramps flatpak with adding LiteLLM is not whether it compiles, but rather the flatpak container and its security constraints. These security constraints caused the looping error for fontconfig above. Even though the Gramps flatpak is SUPPOSED to have full access to the home directory so users don’t lose their databases, the flatpak still quietly restricts system cache, config, data, etc access from the container. So in the paste above, if I use echo $HOME inside the bash environment for the flatpak, it returns a null value. I do not know if the Snap containers have this problem also. This despite the following output for the flathub gramps in Fedora 43

scott@fedora:~$ flatpak run --command=env org.gramps_project.Gramps | grep -E '(HOME|XDG)'
XDG_RUNTIME_DIR=/run/user/1000
HOME=/home/scott
XDG_CACHE_HOME=/home/scott/.var/app/org.gramps_project.Gramps/cache
XDG_CONFIG_DIRS=/app/etc/xdg:/etc/xdg
XDG_CONFIG_HOME=/home/scott/.var/app/org.gramps_project.Gramps/config
XDG_CURRENT_DESKTOP=GNOME
XDG_DATA_DIRS=/app/share:/usr/share:/usr/share/runtime/share:/run/host/user-share:/run/host/share
XDG_DATA_HOME=/home/scott/.var/app/org.gramps_project.Gramps/data
XDG_MENU_PREFIX=gnome-
XDG_SESSION_CLASS=user
XDG_SESSION_DESKTOP=gnome
XDG_SESSION_TYPE=wayland
XDG_STATE_HOME=/home/scott/.var/app/org.gramps_project.Gramps/.local/state

So even though the command above did return a value for HOME, substituting $HOME returns null. Even if code was added to this addon to workaround this issue to accommodate containers, it will probably happen again for other addons. So I think the better solution for the flatpak is either writing directly to or fixing a writable substitute directory in the flatpak manifest. However, the Gramps flatpak already pushes the boundary of home directory access that has been approved by Flathub reviewers. So any flatpak that allows enough cache access to eliminate the fontconfig error might get rejected by Flathub. However, a modified Gramps flatpak with better cache access to work with GrampsChat could potentially be made available as a release on the gramps-project github.