Translator

Human → Computer

Debugging zc.relations / Broken BTrees

written by Patrick Gerken on 2012-09-20

Lately we encountered a serious problem with a site. The site uses Dexterity and our content objects have relationfields. In the beginning of the Project, we had a dependency on Products.LinguaPlone but removed it later in favor of plone.multilingual.

Sometimes, removing a product is hard, especially if persistent utilites are involved. Luckily, one can remove persistent utilities with wildcard.fixpersistentutiliries.

So we thought we are pretty safe now, but after a while the following traceback crept up:

1348150731.020.336585243806 http://somesite/edit
Traceback (innermost last):
  Module ZPublisher.Publish, line 126, in publish
  Module ZPublisher.mapply, line 77, in mapply
  Module ZPublisher.Publish, line 46, in call_object
  Module plone.z3cform.layout, line 70, in __call__
  Module plone.z3cform.layout, line 54, in update
  Module plone.dexterity.browser.edit, line 52, in update
  Module plone.z3cform.fieldsets.extensible, line 59, in update
  Module plone.z3cform.patch, line 30, in GroupForm_update
  Module z3c.form.group, line 138, in update
  Module z3c.form.action, line 99, in execute
  Module z3c.form.button, line 315, in __call__
  Module z3c.form.button, line 170, in __call__
  Module plone.dexterity.browser.edit, line 27, in handleApply
  Module z3c.form.group, line 119, in applyChanges
  Module zope.event, line 31, in notify
  Module zope.component.event, line 24, in dispatch
  Module zope.component._api, line 136, in subscribers
  Module zope.component.registry, line 321, in subscribers
  Module zope.interface.adapter, line 585, in subscribers
  Module zope.component.event, line 32, in objectEventNotify
  Module zope.component._api, line 136, in subscribers
  Module zope.component.registry, line 321, in subscribers
  Module zope.interface.adapter, line 585, in subscribers
  Module z3c.relationfield.event, line 82, in updateRelations
  Module zc.relation.catalog, line 546, in unindex
  Module zc.relation.catalog, line 556, in unindex_doc
  Module zc.relation.catalog, line 621, in _remove
KeyError: <InterfaceClass wildcard.fixpersistentutilities.classfactory.IFakeInterface>

In that line of code, zc.relation tries to extract the key <InterfaceClass wildcard.fixpersistentutilities.classfactory.IFakeInterface> from an OOBtree. That OOBTree claimed that the OOBTree does not contain that key, but when iterating over the keys of the OOBTree, the key was right there.

After a lot of reading the wrong docs (I assumed that the key really does not exist in the BTree and assumed I found a bug in zc.relation, thus trying to understand zc.relation internals first) I reread the documentation of BTrees itself, especially the part about Total Ordering and Persistence.

It turned out that the IFakeInterface from above was not in the correct order. I used the check() method from BTree.check on the btree to confirm that.

Luckily, there is a fast way to repair a BTree so I went on to fix it:

> ./bin/instance -OPlone debug

... from zope.component import getUtility
... from zc.relation.interfaces import ICatalog
... catalog = getUtility(ICatalog)
... btree = catalog._name_TO_mapping['to_interfaces_flattened']
... catalog._name_TO_mapping['to_interfaces_flattened'] = btree.__class__(btree)

Creating a copy of the BTree will readd everything in the right order.

I am going to improve the error reporting in zc.relation to give a better hint of what might be wrong, but I am not sure of a good way to repair such a problem automatically.

Anyway, I hope this helps!

xxx