@Nick-Hall , there is a critical performance bug that I was hoping that it would be included in any of the 6.0.x versions, or 6.1. It is simple looking code that hides a O(n**2) slowdown that can reduce the time every filter is applied. The fix is O(n). Here is the troubled code:
res = sorted(
[handle_tuple[handle] for handle in res],
key=lambda x: id_list.index(x),
)
It looks banal, but the issue is in the .index() which is O(n). Since it is in the key function, it gets called n times, thus n * n.
I have made the fix twice (I forgot about the first one):
master ← dsb/optimize-optimizer
opened 11:08AM - 30 Dec 25 UTC
Before:
We needed to loop over all optimized handles, in case there was a non… -optimized rule (a rule that didn't have `selected_handles`)
After:
We now detected if there are non-optimized rules, and only loop in that case. Otherwise we can return the optimized handles directly.
and then:
master ← dsblank:live-filter-ui
opened 02:16PM - 03 Mar 26 UTC
## Summary
- **Live sidebar timing display**: The filter sidebar now shows `Pre… pare time:`, `Apply time:`, and `Elapsed time:` as they happen, in a consistent order, rather than only after the filter completes. `Gdk.flush()` ensures each update is committed to the display immediately.
- **UI responsiveness on flat views**: `step_progress()` in `gui/user.py` now pumps the GTK main loop at most 10×/second when no `ProgressMeter` is active. This prevents the OS "not responding" freeze on flat views (People, Citations, etc.) which previously had no event pumping during filter execution.
- **Pulsing status-bar progress indicator**: `flatbasemodel._rebuild_filter()` now shows and pulses `uistate.progress` during `search.apply()`, giving the same visual feedback that tree views already had via `ProgressMonitor`.
- **O(n²) → O(n) result ordering fix**: The post-filter result reordering in `_genericfilter.apply()` used `list.index()` as a sort key — O(n) per item, O(n²) overall. On a 100k-person database this caused a ~50 second freeze after every filter run. Fixed with a single O(n) pass:
- `tupleind` path: filter `id_list` by a result set instead of sorting by index
- `tree` path: precomputed position dict replaces repeated `index()` calls
## Files changed
| File | Change |
|---|---|
| `gramps/gen/user.py` | Add `User.print()` and `_print_func` hook |
| `gramps/cli/user.py` | Wire `_print_func` for CLI user |
| `gramps/gen/filters/_genericfilter.py` | Timing messages + O(n²) fix |
| `gramps/gui/user.py` | GTK event pumping + pulse progress in `step_progress()` |
| `gramps/gui/filters/sidebar/_sidebarfilter.py` | Live label updates with `render`/`live_print`/`live_step` |
| `gramps/gui/views/treemodels/flatbasemodel.py` | Pulsing progress bar in `_rebuild_filter()` |
## Test plan
- [ ] Open People view, apply a sidebar filter rule on a large database
- [ ] Verify `msg_label` shows `Searching...\nPrepare time: X.XXXs` while apply phase runs
- [ ] Verify `Apply time:` appears in label immediately after apply completes (not only at the end)
- [ ] Verify final label shows `Prepare time` / `Apply time` / `Elapsed time` in order
- [ ] Verify status-bar progress bar pulses during filter execution and disappears on completion
- [ ] Verify no "not responding" OS dialog appears during filtering
- [ ] Verify filter on a tree view (e.g. Places) shows no regression
- [ ] Verify the ~50 second post-filter freeze is gone on large databases
🤖 Generated with [Claude Code](https://claude.com/claude-code)
The first includes another optimization, and the second adds infrastructure to time the parts of filter processing. I like the fix for the bug in the second PR better.
What needs to be done to get this critical fix into the next release?