Monthly Archives: January 2011

Deploying Cyn.in 3.1.3 on CentOS 5.4

Last week we were upgrading Cyn.in to the latest version. After a few days of testing on a local server it was time to deploy it to the server.

Since Cyn.in needs quite some RAM to operate normally, we chose Virpus VPS instance with 2 gigs of RAM, running CentOS 5.4.

Running buildout on the server produced this error:

...
Modules/constants.c: In function ‘LDAPinit_constants’:
Modules/constants.c:184: error: ‘LDAP_OPT_X_TLS_NEWCTX’ undeclared (first use in this function)
Modules/constants.c:184: error: (Each undeclared identifier is reported only once
Modules/constants.c:184: error: for each function it appears in.)

After a bit of googling around it seemed that the problem was caused by a bug in python-ldap. A quick look at CHANGES revealed that 2.3.10 version of python-ldap mentions work being done around LDAP_OPT_X_TLS_NEWCTX. Maybe it was fixed in the next version, 2.3.11?

I tried pinning the version of python-ldap to 2.3.11 and buildout finished without errors. Great!

Adding the following lines to buildout.cfg makes buildout more repeatable and future-proof:

versions = versions

[versions]
python-ldap = 2.3.11

Testing memoized methods

For a recent Plone project we needed to write unit tests for methods that were using plone.memoize for caching return values.

With a standard Plone’s PloneTestCase I got the following error:

Error in test test_main_image (niteoweb.elcond.tests.test_stol.TestContent)
Traceback (most recent call last):
 File "/Users/zupo/work/python/parts/opt/lib/python2.6/unittest.py", line 279, in run
 testMethod()
 File "/Users/zupo/work/niteoweb.elcond/src/niteoweb/elcond/tests/test_stol.py", line 75, in test_main_image
 image = self.portal.stol.main_image()
 File "/Users/zupo/work/niteoweb.elcond/src/niteoweb/elcond/content/stol.py", line 91, in main_image
 images = self.images(sort_limit=1)
 File "/Users/zupo/.buildout/eggs/plone.memoize-1.1-py2.6.egg/plone/memoize/view.py", line 21, in memogetter
 annotations = IAnnotations(request)
TypeError: ('Could not adapt', None, <InterfaceClass zope.annotation.interfaces.IAnnotations>)

The cause of the problem is that TestRequest (used by PloneTestCase) does not allow IAnnotations adapter to store data in an attribute named __annotations__. The solution is to add the lines below to afterSetUp() method of your TestCase.

from zope.annotation.interfaces import IAttributeAnnotatable
from zope.interface import directlyProvides
from zope.publisher.browser import TestRequest

request = TestRequest()
directlyProvides(request, IAttributeAnnotatable)

Order of ‘parts’ when compiling lxml

CentOS’s repos don’t have a working version of libxslt (you need 1.1.20, repos have 1.1.17) so we need to statically compile it for collective.xdv to work.

But, there is a catch! You need to be careful about how you order your parts in your buildout.cfg.

For examle, the following buildout.cfg works perfectly fine, it downloads libxml and libxslt and compiles them in your buildout environment for your use:

[buildout]
extends = http://dist.plone.org/release/4.0/versions.cfg
parts =
  lxml
  zopepy

[zopepy]
recipe = zc.recipe.egg
interpreter = zopepy
eggs =
  lxml

[lxml]
recipe = z3c.recipe.staticlxml
egg = lxml
static-build = true

Problems start when you change the order of ‘parts’ section to be like this:

[buildout]
parts =
  zopepy
  lxml

...

In this case, zopepy wants to fetch the lxml egg and compile lxml immediately. Since the ‘lxml’ part has not been run yet, there is no libxml2 and libxslt available in your buildout and the whole thing crashes. The point being: always put the lxml part at the top of your parts list.

The fix is a one-liner and I just spent 2 days figuring it out. I wish I knew this earlier.

UPDATE: Florian Schulze pointed out that “an even better fix is to reference the lxml part from zopepy.

[buildout]
extends = http://dist.plone.org/release/4.0/versions.cfg
parts =
  lxml
  zopepy

[zopepy]
recipe = zc.recipe.egg
interpreter = zopepy
eggs =
  ${lxml:egg}

[lxml]
recipe = z3c.recipe.staticlxml
egg = lxml
static-build = true 

That way buildout always installs the lxml part first, because it installs a part before resolving any references to it. This way you actually only need zopepy in parts and can leave out lxml completely.”

Upgrade Cyn.in 2.1 to 3.1.3

We started using Cyn.in for internal documentation repository about 2 years ago. We are a small team of developers and we don’t use all of it’s features, but we still do use it daily and have grown to love it. It’s just great to have a place to store all little bits and pieces of information and have it searchable quickly.

Now it was time to upgrade it so we keep up with the community around it in case anything goes wrong. Here’s how we did it.

Prepare a new version of Cyn.in

Obviously you first need to get the newest version of Cyn.in up and running locally. Following Cynapse’s guide on buildout helps you do it without much pain. When you have the environment ready, start up Zope and try if your Cyn.in installation works on a plain Plone instance.

Upgrade Data.fs

When you have confirmed that Cyn.in works it’s time to upgrade the actual data.

  • Shutdown Zope and remove everything in var/filestorage/ folder.
  • Copy Data.fs from your old Cyn.in installation into var/filestorage/ folder.
  • Start Zope.
  • Go to http://localhost:8080/<plone_instance_id>/portal_migration and migrate Plone content to the latest Plone version
  • Go to http://localhost:8080/<plone_instance_id>/portal_quickinstaller and reinstall ubify.site_policy.

The reinstall will fail at this point. We had to perform the following (ugly) patches for reinstall to work:

  • In file src/ubify.policy/ubify/policy/migration/onetimeinstall.py comment out line 114:
- logger.info("Unable to assign collection portlet for smartview : %s" % (item["id"],))
+ # logger.info("Unable to assign collection portlet for smartview : %s" % (item["id"],))
+ pass
  • In file eggs/Products.CMFQuickInstallerTool-2.1.7-py2.4.egg/Products/CMFQuickInstallerTool/QuickInstallerTool.py comment out lines 550 and 592:
- msg=str(res)
+ # msg=str(res)

- res += str(r)+'\n'
+ # res += str(r)+'\n'
+ pass

That’s it. After a successful reinstall of ubify.policy we were able to use our old data in a new version of Cyn.in. Only a matter of deploying the new version along with migrated Data.fs to the server and we were able to start using it. Hooray!