In this thread, a MacBook user complained about problems upgrading from an iMac, with an Intel processor, to his current machine, with a chip from the Apple silicon series. He reported that he can see the the tree statistics, but he can’t do anything else, and since his old database was made on Intel, I asked him to share that with me, so that I could see what it does on my PCs, in Linux or Windows, which have AMD and Intel processors. I asked that, because it once worked to restore data for a friend with an Intel based MacBook.
I hoped to get his old BSDDB files, but he sent his current grampsdb contents, which are a single SQLite database, that has the symptoms described above. I can see the statistics, but any attempt to switch to another view leads to Gramps hanging, showing all sorts of errors that suggest that the tree data can’t be read. And closer inspection of the database with the SQLitebrowser program suggests that the tables are there, with lots of rows,and that most of the data columns have value NULL.
He also sent some ZIP files made during the conversion process, with similar contents, which suggest that Gramps seems to start a conversion from BSDDB to SQLite, which seems to work at first. And I can imagine that if it’s just a format conversion that does not look at the actual data. The subsequent schema conversion does seem to fail though.
If this scenario is true, it feels quite scary, because it suggests that we don’t have anough protection against architecture changes. I would hope that BSDDB would simpy quit when it reads a file created with another processor architecture, but it seems to have no idea.
Or I have no idea.
Any ideas?
P.S. I’m trying to get hold of his BSDDB files, to see what Gramps does here. It can read files from Intel MacBooks, that I know.
I dont think this has to do with architecture. I tried to open a BSDDB tree on my Intel Mac on 5.1.2 and it failed.
gary$ ./Gramps
2024-10-11 16:04:25.177: ERROR: dbloader.py: line 111: unsupported pickle protocol: 5
Traceback (most recent call last):
File “/Applications/Gramps5.1.2.app/Contents/Resources/lib/python3.6/site-packages/gramps/gui/dbloader.py”, line 206, in read_file
password=password)
File “/Applications/Gramps5.1.2.app/Contents/Resources/lib/python3.6/site-packages/gramps/plugins/db/bsddb/write.py”, line 254, in try_
return func(self, *args, **kwargs)
File “/Applications/Gramps5.1.2.app/Contents/Resources/lib/python3.6/site-packages/gramps/plugins/db/bsddb/write.py”, line 652, in load
if not self.version_supported():
File “/Applications/Gramps5.1.2.app/Contents/Resources/lib/python3.6/site-packages/gramps/plugins/db/bsddb/write.py”, line 254, in try_
return func(self, *args, **kwargs)
File “/Applications/Gramps5.1.2.app/Contents/Resources/lib/python3.6/site-packages/gramps/plugins/db/bsddb/write.py”, line 507, in version_supported
dbversion = self.metadata.get(b’version’, default=0)
File “/Applications/Gramps5.1.2.app/Contents/Resources/lib/python3.6/site-packages/bsddb3/dbshelve.py”, line 226, in get
return pickle.loads(data)
ValueError: unsupported pickle protocol: 5
H’m, weird. Google suggests that pickle support can vary with Python versions, and it’s something that I have not really been aware of, until now. It’s not what the user got, and I don’t remember seeing it, when I tried to help this old friend. And that was probably at a time when I was still using 3.4, or maybe 4.x.
I don’t know the version that this user had, on his iMac, but forgot to mention the thread:
TS plans to extract the files from the iMac again, so I hope that I can report later this weekend, or nexy week.
And just for the sake of testing or preparation: Would you mind sharing some BSDDB files, Intel, or both platforms, with me? It can be through email, or iCloud, and I will pass my address in a minuite or so.
I’m writing a public reply, to report on these issues, which I could test with the 5.1.6 test tree that you made and put in my mail. And with 5.1.6 running on Windows 10, I see the same pickle protocol problems as you got.
Opening the test tree with 5.2.3 AIO worked very well. It offered an upgrade for the BSDDB version, which is quite a bit newer on Windows, and then did a schema upgrade, which was OK too. Check & Repair showed no errors, amd that proves that a Gramps AIO running on Windows can be used to convert a BSSDB tree from Macs running on Intel.
I will try another test on Linux tomorrow. That may even work in 5.1 if that Linux has a more modern Python than the AIO version has.
I tested all files that you sent me on Linux Mint, running my own Gramps 5.1.7, and they were all upgraded to my BSDDB version, which is newer, and showed no pickle problems, probably because the Mint Python is more modern than the one in the Windows 5.1.6 AIO, or your Gramps 5.1.2. I have Python 3.10 here.
One tree (4.2.4) required a schema upgrade, and all could be converted to SQLite too, which makes me conclude that Mac Intel databases can be restored on PCs running Linux or Windows (with 5.2.3).