yeh, there are quite a few of these around now… gigantic prompts and “guardrails”, but they’re meant for paid APIs and extremely large online LLMs.
You really don’t want to use something like that with a relatively small, specialized local model.
I haven’t tested them myself either, because I don’t really trust having someone else define a whole set of rules for how I am supposed to use a language model.
But please be very careful trusting these “assistants”, because all they really do is add a huge overlay on top of your prompts — and they cannot remove hallucinations or false positives. At best they might reduce them a little.
And for a local model, a prompt like this simply won’t work. You’ll use up most of your context window just on the system‑prompt itself, which eats memory and slows the model down — and in the end the model still won’t be able to follow all those instructions anyway.
There’s always a cost. It might not be tokens but the processing cost may be too high to bear locally. Although someone might figure out how to do split processing: most local; some computing hungry or greater complexity than someone without a graphics processor or using a small laptop can allocate locally)
I actually think the best way to reduce local resource usage is what I’ve outlined above — and something I’ve mentioned a few times in other threads as well: moving more of the Gramps logic, which has been refined for decades, from the frontend application layer into the backend database layer, and even into the database itself through functions, views, and queries.
If we expose that logic as tools through db-api or similar, a local LLM — especially one that is specialized for genealogy and the Gramps data schema — can rely on those tools instead of trying to “think” everything out from scratch.
That approach can give you extremely high accuracy even with a relatively small local model, and the model doesn’t even need to run in VRAM (though of course it will be faster if you have enough VRAM available).
This approach will also benefit users who rely on online paid (or free) models.
If the heavy lifting is done in the database layer — with decades of Gramps logic exposed as functions, views, and queries — then both local and online models can use those tools instead of trying to reason everything out themselves.
Yes, it makes sense and it has a name; it is called the “system prompt”. You tell the AI system how and what to answer, and any details that you want to refine the responses. Gramps Assist comes with a way to edit the system prompt, and you can change the behavior.
I think your explanation makes sense, but I’d just add one practical note: when people talk about “system prompts”, it’s easy to forget that the system prompt counts toward the total context size. On small or medium local models, a large system prompt can consume most of the available memory, which reduces accuracy and makes the model behave worse, not better.
That’s why I think it’s generally more effective to expose Gramps logic as tools rather than trying to encode everything into the system prompt. Tools scale; large prompts don’t.
Oh, yeah, totally agree. In fact this is the sum total of the system prompt I have been using:
“You are a helpful genealogy assistant with access to the user’s Gramps database.
Answer questions about people, families, events, and relationships.
Use the available tools to look up information when needed.”
It took the meaning of “the selected person” to be “the currently selected person”, which is reasonable (and I don’t think would work any other way in Gramps).
There is some confusion between the “Filter Gramplet” and realizing that this is the same thing as the “Filter Sidebar”.
I’ve found that places where the LLM gets confused is also the places where humans get confused. Specifically stating in the docs that (2) are the same could help here for humans and AI.
This is just a good model that has been trained on the internet of text. No tool use here.
No, it can’t do that as that is beyond what a standard LLM can do. But we could add tools that to that (or call other models that can do that). And even if it could read a PDF, this isn’t a very good question for a LLM. This is better for a full on-line tool-calling system (like Claude Code, $20/month). Which we could consider integrating.
Hope that gives an idea of where we are, and where we could be.
On a somewhat related note, I see that ollama offers a cloud service. It is $20/month. It is like running a local model, but in the cloud. You’re basically renting a powerful computer.
Yes, thanks. I was trying to think of things that take me more time than I care to spend on them. On the other hand, there may be other things where I would like the Assistant to actually teach me how to do, rather than just saying “here’s the code you need to enter”. (Of course, I’d prefer to learn from another human, and there are many helpful people in this forum to whom I’m grateful.)
Can you please clarify the following:
By “tools” I think you mean the particular aspects of Gramps that you have somewhere/somehow “explained” to the Assistant? What would it take for addons to do that as well, so that it doesn’t say it can’t interact with them? (Or is that not the issue?)
Although I like to do things on the cheap, that seems incredibly inexpensive if I could use it effectively. Each of us has aspects of our genealogical work that we would never want to outsource for various reasons, and may not want to let another person, much less a machine, update our database. On the other hand, I have a big backlog of “grunt work” like data entry for sources and citations from which I derive no particular pleasure.
Looks like even their free version includes “light usage” in the cloud.
Sorry, I veered into some Agentic AI-speak A “tool” is a function that has its own description of what it does. So the system prompt doesn’t have to say “if you want to do X use tool Y”. That allows the system prompt to focus on other instructions.
No problem. In this discussion, we are talking about LLM/AI systems. Specifically a new addon-on that I ma developing that I am calling “Gramps Assistant.” It is being developed for Gramps 6.1 as it requires a new addon type.
There have been at least three other “talk-to-your-tree” LLM addons mentioned in the last few months that can answer questions, and one allows you to change data.
But “Gramps Assistant” can control the UI, and run functions for you. The ability for an AI to “act” is often called a “tool”. I am talking about something very specific to this thread.
I would also add that “tools” can include much more than UI‑actions. Gramps already has a very rich set of logical building blocks — for example the entire filter system. If an LLM has access to all filters (and the ability to compose them), it can translate a natural-language question into a sequence of filter operations and use the results to generate a report, a graph, a timeline, or a text-based answer. This lets the model rely on Gramps’ own logic instead of trying to infer everything from scratch.
Combined with what dsblank describes — the ability to control the UI and trigger functions — this becomes extremely powerful. The LLM can not only navigate the interface, but also use these logical tools to support workflows and produce accurate answers.
Other tools could also be exposed in the same way, such as export gramplets, file‑writing operations, the supertools collection etc., or any other well‑defined function that Gramps already provides.
At the same time, it’s important to set some boundaries for what a single gramplet should do. It’s usually better to create several specialized AI/LLM gramplets, each focused on a specific task, rather than one large gramplet filled with every possible rule and tool. This avoids overloading the model’s context and keeps each assistant efficient.
You can still use the same LLM behind all of these “specialists” — the key is simply to limit the scope of each gramplet so the model only receives the tools and instructions relevant to the task at hand.
Edited: I only added some additional context to clarify my earlier points. The meaning and content of the original text remain unchanged.
Yes, I know — it was just meant as an addition to your answer.
I wanted to highlight that a well‑designed set of tools and clear tool descriptions can already cover a lot of what people are asking for today, and that an AI can combine different types of tools to accomplish a task.
I think this type of AI‑tools (gramplets) is a good example of how to use LLMs/AI in a correct and practical way. It combines UI‑control with well‑defined tools and existing Gramps logic, which is exactly the kind of structure that makes AI both useful and predictable.
Maybe someone (or some AI ) could kindly generate an infographic that compares/contrasts them?
Peeking under the hood, I think I begin to understand a bit more. You had mentioned:
And now I notice things like the comments in the code below, which I guess are actually for consumption by the AI? Or maybe I still don’t get it (but I’m trying!).
def get_family(self, family_handle: str) -> Dict[str, Any]:
"""
Get the data of a family given the family handle in the argument.
* family handles are different from a person handle.
* a person has family handles in two different fields:
- "parent_family_list" has the list of family handles the person is a child in
- "family_list" has the list of family handles the person is a parent in
The result of "get_family" tool contains several handles as follows:
"father_handle": person_handle of the father in the family
"mother_handle": person_handle of the mother in the family
"child_ref_list": list of person_handles of children in the family,
each item in the "child_ref_list" has a "ref" which is the person_handle
of children of the family.
Details of the persons can be retrieved using the "get_person" tool
"""
With the proliferation of LLM addons for Gramps, is this particular wheel being reinvented every time? Meaning, could/should there be some standard library of these descriptions, in order to achieve more standard/predictable behavior and reduce the need for redundant testing and maintenance?
(By the way, it looks like it does not yet distinguish the different types of parent/child relationships, i.e. birth, adopted, etc.?)
A “tool” is a function that has its own description of what it does. So the system prompt doesn’t have to say “if you want to do X use tool Y”. That allows the system prompt to focus on other instructions.
) could kindly generate an infographic that compares/contrasts them?