Should we (can we?) enable automatic reviews of PRs by an AI bot, such as Copilot?
Manual review would also be interesting - inviting the bot to review when the developer is ready - but I believe that requires write permissions which most of us do not have (for good reason!)
In addition to voting with emojis, it would be enlightening to hear the reason behind your votes. Recently @dsblank has been posting Claude reviews of PRs and I find them helpful, for a few reasons:
We can go back several years and look at commits and find so many non-trivial changes have not received any reviews and have been committed to the repo. Given that, the automated review is a great kickstarter. It often looks at things that I personally may not have thought about, and helps improve code quality consistently. I see no downside as it’s easy enough to ignore suggestions that aren’t deemed required.
Finally, some may find find AI reviews neutral and perhaps more palatable than one from a fellow contributor.
For the avoidance of doubt, this is just an AI driven automatic review.
I’m strongly in favour of continued manual review.
This does not imply automatic merging either.
It is just another data point to help us produce a better quality gramps.
You could also have it review past PRs for any hitherto undiscovered bugs. Maybe having a larger number of such reviews might also help you assess or improve their reliability.
I have just been looking into this. The text below the “Automatically request Copilot code review” option reads “Request Copilot code review for new pull requests automatically if the author has access to Copilot code review and their premium requests quota has not reached the limit.”
Looking at the Copilot plans it appears that code reviews are not available to people on the free plan. For others it will use their premium requests quota. We could select the one review per PR option to limit usage.
Indeed. There are areas that are pretty complex, and not every developer has expertise in all areas. And writing tests for some parts has been difficult in the past.
There are so many niche areas of complexity in Gramps: Gtk, database, reports, plugins, dates, gramplets, translations, installers. But I will say that using Claude Code (given that I give well-refined prompts) is better than I am!
The automated reviews, like you suggest, might offer slightly misguided advice. But that will get better as we improve the consistency of gramps (better type hints helps the AI, and the humans).
We can have AI agents review existing code. That is very productive. However, I would hold off on that for now. I don’t see how @Nick-Hall is going to keep up the PRs (and in fact he is falling behind). It is easier everyday to do comprehensive PRs, but if they sit in the repo for months and months, it will create even larger piles of merge conflicts with no end in sight.
I tend to review PRs in batches and I’ll start again soon. The AI reviews are helpful, and I think that the current approach of running them manually is probably sufficient for now.