Skip to content

Sprint report ArtSprint 2012

February 17, 2012

I’ve spent a lovely and frankly hilarious week here in Vienna, and amazingly suceeded in finishing something!

I was starting with just setting up a site using, in the aim of trying to fix some failing tests or otherwise being useful. However, when testing that I could create recurring events, I made an event with an infinite recurrence, and discovered that creating that event would take a Very Long Time. This turned out to be the DateRecurringIndex that is currently being used by

As a result I changed tracks, and started writing performance tests for different date indexes. I’m using funkload (because that was easiest, but although funkload is awesome, it’s made for load testing web applications, so it’s not really a good fit for this, but it was the best I could come up with).

I’m running the tests on two different sets of data. One that simply has one event per day, that repeats every 8 days for ten times. This set of data is good for verifying that the tests test the right thing, and that you get the right data back, etc. The other set of data is a set of 10.000+ events that is the collection of real data people have been nice to send me. That set of data is useful to get real world performance data once you know the tests work properly.

I test three sets of indexes: DateIndex (without recurrence), DateRecurringIndex and


DateIndex is of course the fastest, but it doesn’t support recurrence, so it’s for comparison only.

DateRecurringIndex is almost exactly the same as DateIndex, and almost as fast. However, I did discover during this is that in fact DateRecurring index does not currently work as expected. There is no way to query it to get a correct result. I looked a bit at the code to try to fix that, but wasn’t able to come up with a quick solution.

Instead I looked at and it’s performance. And it was *slooooow*. Badly slow. But by looking at the DateIndex and DateRecurringIndex code, I’d already noticed two things I probably could do to make it faster. And I already implemented one, namely using the “multiunion” function instead of doing multiple unions. It was a method whose existence I had missed. That speeded up the index significantly. I’m also currently using timetuples as data in the indexes. I can probably change that to timestamps with minute resolution, and make it even faster.



I did a test where I reindexed all the events for the respective indices. It tests how many times you can reindex all 10.000+ events during 300 seconds.

DateIndex: 17
DateRecurringIndex: 13 13

So no performance hit for compared with DateRecurringIndex


This test makes 365 queries (one per day of a year) and checks how many times it can run that test in 120 seconds:

DateIndex: 18
DateRecurringIndex: 18 (but remember that it didn’t actually work) 15

 There is some performance hit here, but not significant.

From → calendaring, plone

  1. Unfortunately the current buildout for is broken.

  2. Yup, you are right. I broke it. Will fix tomorrow.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: