Lately we encountered a serious problem with a site. The site uses Dexterity and our content objects have relationfields. In the beginning of the Project, we had a dependency on Products.LinguaPlone but removed it later in favor of plone.multilingual.
Sometimes, removing a product is hard, especially if persistent utilites are involved. Luckily, one can remove persistent utilities with wildcard.fixpersistentutiliries.
So we thought we are pretty safe now, but after a while the following traceback crept up:
1348150731.020.336585243806 http://somesite/edit Traceback (innermost last): Module ZPublisher.Publish, line 126, in publish Module ZPublisher.mapply, line 77, in mapply Module ZPublisher.Publish, line 46, in call_object Module plone.z3cform.layout, line 70, in __call__ Module plone.z3cform.layout, line 54, in update Module plone.dexterity.browser.edit, line 52, in update Module plone.z3cform.fieldsets.extensible, line 59, in update Module plone.z3cform.patch, line 30, in GroupForm_update Module z3c.form.group, line 138, in update Module z3c.form.action, line 99, in execute Module z3c.form.button, line 315, in __call__ Module z3c.form.button, line 170, in __call__ Module plone.dexterity.browser.edit, line 27, in handleApply Module z3c.form.group, line 119, in applyChanges Module zope.event, line 31, in notify Module zope.component.event, line 24, in dispatch Module zope.component._api, line 136, in subscribers Module zope.component.registry, line 321, in subscribers Module zope.interface.adapter, line 585, in subscribers Module zope.component.event, line 32, in objectEventNotify Module zope.component._api, line 136, in subscribers Module zope.component.registry, line 321, in subscribers Module zope.interface.adapter, line 585, in subscribers Module z3c.relationfield.event, line 82, in updateRelations Module zc.relation.catalog, line 546, in unindex Module zc.relation.catalog, line 556, in unindex_doc Module zc.relation.catalog, line 621, in _remove KeyError: <InterfaceClass wildcard.fixpersistentutilities.classfactory.IFakeInterface>
In that line of code, zc.relation tries to extract the key
<InterfaceClass wildcard.fixpersistentutilities.classfactory.IFakeInterface> from an OOBtree.
That OOBTree claimed that the OOBTree does not contain that key, but when iterating over the keys of the OOBTree, the key was right there.
After a lot of reading the wrong docs (I assumed that the key really does not exist in the BTree and assumed I found a bug in zc.relation, thus trying to understand zc.relation internals first) I reread the documentation of BTrees itself, especially the part about Total Ordering and Persistence.
It turned out that the IFakeInterface from above was not in the correct order. I used the check() method from BTree.check on the btree to confirm that.
Luckily, there is a fast way to repair a BTree so I went on to fix it:
> ./bin/instance -OPlone debug ... from zope.component import getUtility ... from zc.relation.interfaces import ICatalog ... catalog = getUtility(ICatalog) ... btree = catalog._name_TO_mapping['to_interfaces_flattened'] ... catalog._name_TO_mapping['to_interfaces_flattened'] = btree.__class__(btree)
Creating a copy of the BTree will readd everything in the right order.
I am going to improve the error reporting in zc.relation to give a better hint of what might be wrong, but I am not sure of a good way to repair such a problem automatically.
Anyway, I hope this helps!