Looks like this should be here also
That’s a different animal. This is for printing purposes. It just avoids printing GUI elements in hardcopy/PDFs.
I have been looking for performances with the web narrative for several weeks.
I tried threads and asyncio to copy files.
It seems that asyncio is faster than threads.
I had problems with the number of open files and I use setrlimit to fix this issue.
I do not know if all operating systems allow to increase the number of open files.
I tried gc.collect() to reduce memory resources: it’s worse when we use it.
So I use del to delete some variables.
I have now reduced my media copy time (~3GB) from ~2200 seconds to ~150s.
to be continued.
Wow! That seems like a fruitful experiment! Looking forward to reading about how things progress.
(I haven’t gotten into Media with my Gramps trees and have just lightly touched reports. But it is enjoyable to read about the way you evolve this report and the Geography view.)
Have you also considered creating symbolic links for media instead of copying them? When NarrativeWeb is generated for exclusive use on the computer (no transmission to “external” recipients via USB keys or others), this could be a viable alternative. Of course thumbnails still have to be created.
What would be the duration difference?
And a small sub-question… Does Thumbnail Generator have affect to the reports speed? Are thumbnails generated via addon the same as used in the report? And vise versa… If I generted report, do I need run the addon later?
This should be the solution, but we cannot do it because we do not know where you will put your data later.
Suppose you put the narrativeweb on a USB stick. When you move your data to another machine, you couldn’t see the images.
I don’t know. I have never tested. I’ll look at this later.
That’s why I restricted the use case to local use of the NarrativeWeb. Consequently, this possibility, if it results in a significant improvement, should be controlled by an option. Otherwise, if the performance boost is negligible, drop the suggestion.
I think we don’t need asyncio or threads to solve the copy problem.
In the media.py file, if you remove all gc.collect() lines, the result is very good.
Here is the results from my database with 721 images.
For the gramps 5.2.1 without modification:
If we remove all the gc.collect() in media.py, the result is:
As you can see, I create the narrative web for 6 languages and this is why we have 6 lines. In the standard narrative web, the time used to copy the images is 712 seconds for the first language and the time grow up for each language.
In the case with the removed gc.collect(), the time is now 53 seconds for the first language and between 2 and 4 seconds for the others languages. This is what I always expected. I started the report twice to see if the times changed.
I saw the gc.collect() was implemented with the narrative web in gramps 2.2
So If you want to try this, you can remove the lines 153, 179 and 321 in media.py
Removing the gc.collect() in media.py does make a huge difference for me.
With nearly 18000 media objects that get included in my report (after excluding over 300 that are marked private), the last time I created a report that included media objects using version 5.2.0, it took over 24 hours and less than an hour without media objects. Tonight I tried version 5.2.1 with gc.collect() removed, and it took about two hours to generate the report.
Since there don’t seem to be any changes to media.py from 5.2.0 to 5.2.1, I think it is safe to assume that gc.collect() has been the culprit in the long report generation times I’ve seen over the last several versions of gramps. I would guess this is what I’ve been looking for.
Allen Crider
The NarrativeWeb report does not support the REPORT_MODE_BKI in plg.report_modes plug-in registration attribute.
Even if the report was made ready-to-save with the Books feature, the books.xml lacks some data to make that saved data rot resistant for a Narrated Web Report.
The books.xml does have the database identified with the report. So a Book applied to the wrong Tree is less likely. And the loader for a Book will reset ‘pid’ to the Active Person if a different database is loaded when the Book is called up.
<booklist>
<book name="ActivePersonRule" database="/home/districtsupport/.gramps/disable/grampsdb/61f41273">
However, like all uses of report_options.xml, the filter references are by index… which is in constant flux. The chances that a filter surviving at the same index position between regenerations of a Narrated Web site are very poor.
<option name="filter" type="int" value="6"/>
It would be less fragile if the books.xml and report_options.xml used an embedded <filter> block. (Would each book need to maintain a separate <filter> block to avoid stepping on each others custom filter names and definitions?)
<filters>
<object type="Person">
<filter name="Active Person" function="and">
<rule class="IsActivePerson" use_regex="False" use_case="False"> </rule>
</filter>
</object>
</filters>
But this would raise a potential Custom Filter issue. Loading a Book could add the Custom Filter to your collection if it did not exist. And if the same name/ID custom filter was identical in the Book and Gramps session, it could be skipped. But what should be done if Book Custom Filters of the same ID had conflicting parameters? (It is the same old Merge problem.) Asking the User to resolve (by renaming or overwriting) seems reasonable with up to about 5 conflicts. More than that is onerous.
I cannot see anyone wanting to squeeze a Narrated Web into a Book. A book was what I created for my grandmother and mother back in the 1980’s. By design, the Narrated Web is designed for the World Wide Web.
Saving/loading a particular report configuration may be a worthwhile endeavor by first removing its settings from report_options.ini and into its own options ini file.
The NarrWeb and WebCal already save different setting options based upon the database/tree.
Updating my Narrated Web site is a periodic thing. Online is monthly or quarterly as needed. Family Reunion sites are generated annually and shared on USB thumbdrives.
Since there are 2 groups of sites that I maintain for different families (my 4th cousins through different progenitor lines so they consider themselves unrelated), the sites use totally different styles and report settings.
I’d like to replicate the previous report options each time for a specific website with less worry of overlooking something. And a LOT fewer notes about the settings.
I agree and support the endeavor. But I see trying to add save/load configurations to NarrWeb and WebCal as a better selling point than trying to fit the web reports into the Books process.
After creating a Narrweb for a cousin, I save its report_options along with my saved copy of the output. I just saw a possible hack, giving the reports their own web_report_options ini file, as an easier way to manually switch out option files.
Having @SNoiraud buy into that endeavor seems a much more realistic possibility.
As a practical approach, such one-offs sound easier.
But the idea of a new functionality that is similar to another existing functionality adds confusion (I know I saw that feature somewhere!)
And choosing to not evolve Books could lead to its extinction.
I know I have never used Books to actually print out a report and I have been using Gramps since 3.4.9 ??. I have helped another user with it recently (this year) I suppose for that great-aunt that does not do computers. Using Books to save report config files is in my bag of tricks but truthfully, I do not even use Books for that purpose.
I tried adding REPORT_MODE_BKI to webplugins.gpr.py. The reports were added to the list of reports for Books but as unsupported. It would take a lot more work to get Books to be able to be used to save the config settings.
Maybe Books can be used as a template to create a new Saved Report Configurations report tool that save and load configs for single reports without regards to the output. Again, it would be a lot of work.
I just explored the settings for the Narrated Web stored in report_options.ini. As previously noted, different options are stored based upon the name of the tree/database. In my report_options, I have…
<module name="navwebpage EXAMPLE"> and
<module name="navwebpage Master Family"> and a few others.
I will most often create narrwebs for a cousin after importing my main “Master Family” database into a new tree, “Cousin1”. Creating a narrweb from Cousin1 leaves behind <module name="navwebpage Cousin1"> and maybe <module name="navwebpage Cousin2">.
To reuse past narrweb settings, it would be a matter of creating trees/databases that reuse the tree names from past endeavors.