LLM + Gramps = a useful chatbot

Development
41

The remote gemini flash-2.5, very strong model.

The local machine is very old, but that is fine, it can execute each gramps query at the expected speed of this old machine.

I the model usually does select the correct tools, and it also says what kind of tools might be missing. That is, A reply once had:

“Sorry I can not search for a family this person is a parent in.” Or “I don’t have access it know how to access that information”

Sometimes it is hallucinating responses and comes up with people that are not even in the database!

We should probably extend the descriptions of each individual tool to explain what all the nitty gritty details and fields in a person response mean to act upon that.

Anyway, fun to play with.

42

2 posts were split to a new topic: DeepWiki as a Gramps API documentation tool

43

Making progress on a grampschat server:

image
image827×829 88.9 KB

TODO:

  1. Find an ollama model powerful enough to handle tools (above is using openai/gpt-4o-mini)
  2. Find a machine in the cloud that is powerful enough to run the local model
  3. Figure out cost of server

Then lots of little things:

  • how to upload database?
  • currently allowing github and google OAUTH. need others?
2 Likes
44

Hi @dsblank finally managed to add a “find_people” tool, so that we can search on people by name in the database.

I needed to also restrict the AI Model from looping, and added a final prompt after a few tries, not providing any tools anymore, to force an answer from the local running modal.

You might want to try out or just look at some of the code updates:

45

Thanks for the update.

FYI, the method can be pointed to using the github URL: genealogychatbot/chatbot.py at master · MelleKoning/genealogychatbot · GitHub

I’m going to try to make such a query function much more efficient. But what you have should work ok for small trees.

46

True about small trees. When a tree contains tens or hundreds of thousands people this won’t perform.

Also notice the summarization of the results with “desired fields”

As I found that even a result on a lastname with about 25 matching people, resulted in a huge json addition to the context window of the chat history, so we have to be careful, especially with local smaller models.

47

That subroutine seems to ignore some of the common name options. Will it make the chatbot too formal to recognize all the possibilities?

For instance, when searching for people who colaborate online, I might only recall their online name. So if I search a Gramps tree for kku, I’d want to find Kari Kujansuu. Some people haven’t shared their personal names. For instance, I only know CallMeDave (on the Gramps Reddit and the wiki) by that name.

48

Have tried the routine to search on a lot of name options, but then limit the output to the common names.

I’m a bit surprised that you mention to want to find nicknames? Usually the genealogy tree that is being searched by the tool has “real names”, maybe I’m misunderstanding the concern, or I might indeed have missed an important name field like alternate names to be found.

Other then that: when playing with the tools, also tried several small AI models, and some of those have a hard time understanding the tooling provided. That is, not all models understand when to provide the appropriate arguments for tools, it get into indefinite loops, so the mayor changes are also in the final decision: not providing the tools and force the model to provide an answer.

Updating the prompts seems important.

49

All of your points are good, and why this is ongoing development.

  1. Smaller models seems to use tools differently (so may require special handling)
  2. Smaller models may not perform well with tools, and longer context windows (each tool adds a lot of tokens)
  3. The prompt is really critical in smaller models
1 Like