06 Feb 2012
Planet Plone
Tarek Ziade: Scaling Crypto work in Python
We're building a new service at Services called the Token Server - The idea is simple : give us a Browser ID assertion and a service name, and the Token Server will send you back a token that's good for 30 minutes to use for the specific service.
That indirection makes our live easier to manage user authentication and resource allocation for our services . A few examples:
- when a new user wants to use Firefox Sync, we can check which server has the smallest number of allocated users, and tell the user to go there
- we can manage a user from a central place
- we can manage a user we've never heard about before without asking her to register specifically to each service - that's the whole point of Browser ID
I won't get into more details because that's not the intent of this blog post. But if you are curious the full draft spec is here - https://wiki.mozilla.org/Services/Sagrada/TokenServer
What's this post is really about is how to build this token server.
The server is a single web service that gets a Browser ID assertion and does the following:
- verify the assertion
- create a token, which is a simple JSON mapping
- encrypt and sign the token
The GIL, Gevent, greenlet and the likes
Implementing this using Cornice and a crypto lib is quite simple, but has one major issue : the crypto work is CPU intensive, and even if the libraries we can use have C code under the hood, it seems that the GIL is not released enough to let your threads really use several cores. For example, we benched M2Crypto and it was obvious that a multi-threaded app was locked by the GIL.
But we don't use threads in our Python servers - we use Gevent workers, which are based on greenlets. But while greenlets help on I/O bound calls, it won't help on CPU bound work : you're tied into a single thread in this case and each greenlet that does some CPU work blocks the other ones.
It's easy to demonstrate - see http://tarek.pastebin.mozilla.org/1476644 If I run it on my Mac Book Air, the pure Python synchronous version is always faster (huh, the gevent version is *much* slower, not sure why..)
So the sanest option is to use separate processes and set up a messaging queue between the web service that needs some crypto work to be done and specialized crypto workers.
We're back in that case to our beloved 100% I/O bound model we know how to scale using NGinx + GUnicorn + GEvent
For the crypto workers, we want it to be as fast as possible, so we started to look at Crypto++ which seems promising because it uses CPU-specific calls in ASM. There's the pycryptopp binding that's available to work with Crypto++ but we happen to need to do some tasks that are not available in that lib yet - like HKDF.
Yeah, at that point it became obvious we'd use pure C++ for that part, and drive it from Python.
Message passing
Back to our Token server - we need to send crypto work to our workers and get back the result. The first option that comes in mind is to use multiprocessing to spawn our C++ workers and to feed them with work.
The model is quite simple, but now that we have one piece in C++, it's getting harder to use the built-in tools in multiprocessing to communicate with our workers - we need to be lower level and start to work with signals or sockets. And well, I am not sure what would be left of multiprocessing then.
This is doable but a bit of a pain to do correctly (and in a portable way.) Moreover, if we want to have a robust system, we need to have things like a hearbeat, which requires more inter-process message passing. And now I need to code it in Python and C++
Hold on - Let me summarize my requirements:
- inter-process communication
- something less painful than signals or sockets
- very very very fast
I got tempted by Memory Mapped Files, but the drawbacks I've read here and there scared me.
ZeroMQ
It turns out zeromq is perfect for this job - there are clients in Python and C++, and defining a protocol to exchange data from the Python web server to the crypto workers is quite simple.
In fact, this can be done as a reusable library that takes care of passing messages to workers and getting back results. It has been done hundreds of times, there are many examples in the zmq website, but I have failed to find any Python packaged library that would let me push some work to workers transparently, via a simple execute() call - if you know one tell me!.
So I am building one since it's quite short and simple - The project is called PowerHose and is located here : https://github.com/mozilla-services/powerhose.
Here is its descriptions/limitations:
- Powerhose is based on a single master and multiple workers protocol
- The Master opens a socket and waits for workers to register themselves into it
- The worker registers itself to the master, provides the path to its own socket, and wait for some work on it.
- Workers are performing the work synchronously and send back the result immediatly.
- The master load-balances on available workers, and if all are busy waits a bit before it times out.
- The worker pings the master on a regular basis and exits if it's unable to reach it. It attempts several time to reconnect to give a chance to the master to come back.
- Workers are language agnostic and a master could run heterogeneous workers (one in C, one in Python etc..)
- Powerhose is not serializing/deserializing the data - it sends plain strings. This is the responsibility of the program that uses it.
- Powerhose is not responsible to respawn a master or a worker that dies. I plan to use daemontools for this, and maybe provide a script that runs all workers at once.
- Powerhose do not queue works and just rely on zeromq sockets.
The library implements this protocol and gives two tools to use it:
- A JobRunner class in Python, you can use to send some work to be done
- A Worker class in Python and C++, you can use as a base class to implement workers
Here's an example of using Powerhose:
- The Server - https://github.com/mozilla-services/powerhose/blob/master/examples/square_master.py
- The Python worker - https://github.com/mozilla-services/powerhose/blob/master/examples/square_worker.py
- The C++ worker (don't look at the code
- https://github.com/mozilla-services/powerhose/blob/master/examples/square_worker.cpp
For the Token server, we'll have:
- A JobRunner in our Cornice application
- A C++ worker that uses Crypto++
The first benches look fantastic - probably faster that anything I'd have implemented myself using plain sockets ![]()
I'll try to package Powerhose so other projects at Mozilla can use it. I am wondering if this could be useful to more people, since I failed to find that kind of tool. How do you scale your CPU-bound web apps ?
06 Feb 2012 8:17am GMT
05 Feb 2012
Planet Plone
Reinout van Rees: Apple lion reinstall experience and surprise
My year-old macbook installation was showing its age. Or rather, there were some things wrong with it:
- The original OS was 10.6, snow leopard. I upgraded it to lion (10.7) half a year ago. This was an in-place upgrade, not a fresh install. I wanted a fresh install to clean some stuff up and because it started to feel slow. I heard that a clean install would help a lot regarding speed.
- I work a lot with geographic libraries, Django and geodjango. So originally I installed everything via the kyngchaos packages. Mapnik, gdal, spatialite and so on. But after the lion upgrade, I couldn't compile any python packages with C extensions anymore as gcc 4.0 (which everything had been build with) had been replaced by 4.2. And spatialite never would work right anyway. So I wanted to replace this.
- I used homebrew as a package manager for the gnu/unix side of things instead of macports I'd been using before. It works, but I missed some things, like Quantum GIS (QGIS), which is included in macports. I hoped to get everything python+gis related done with one package manager, in my case macports.
So I made sure my backups were OK, that my code was all committed, that my repositories were cleaned up, that all my dotfiles in my homedir were in version control and so on. Most of it was already OK, but of course there were some small things left. I'll do a write-up later on of my backup strategy and how I handle my dotfiles and so.
Time for the actual lion reinstall. How does that work? I bought Lion from the app store, so it was downloaded and installed by my mac: I didn't have an install DVD. Turns out to be easy: just restart and press command-r during bootup and you'll get a "lion recovery" menu. Choose the reinstall option and it will download the latest full version and install it for you. Simple and works.
The big surprise came when the computer rebooted. I expected a dialog to set up a main user. Instead, I got the regular login screen. Ok... Logging in... Hey! All my stuff is still there! All the settings, all my documents, all my music... No need to restore backups.
So: an OSX lion restore wipes only the OS and reinstalls it. Including xcode, btw. The rest (your own data, applications, settings) is retained. Actually pretty handy.
This did mean I had to clean up the kyngchaos packages and homebrew by hand. Just a matter of deleting some directories, telling homebrew to erase itself and adjusting my paths.
05 Feb 2012 8:12pm GMT
04 Feb 2012
Planet Plone
David Glick: Using tiles to provide more flexible Plone layouts
Plone does great at in-place editing: navigate to the thing you want to edit, then click the button and edit it. However, this paradigm breaks apart as soon as there is a need for a page to have multiple editable areas-such as for a homepage or section landing page.
At Groundwire, we used to deal with this problem by creating an ongoing series of very similar hacky one-off templates: the sort of template that would have have several areas which each pulled in the content from some item in a hidden folder of page components. Unfortunately this approach did not scale very well: it was tedious for us to set up new templates, and it was cumbersome for editors to remember how everything was set up in order to successfully make changes.
Last year we worked on the Net Impact website which has a different multi-part layout for each section landing page, and we realized that we needed to come up with a better solution. The requirements:
- Someone writing a template should be able to define an editable area of that template very simply, by just adding a line to the template that specifies the name of the area.
- There should be support for different types of editable areas; each type may have different settings when editing the area.
- Editing an area should be triggered by a pencil icon that shows up while hovering over the area for users who have permission to edit the area,
- All this should be done in a way that is simple to reuse for new sites.
Tiles to the rescue
We realized right away that our requirements were very similar to the functionality provided by the Deco project's implementation of "tiles." Deco is an ambitious project to provide drag-and-drop layout capabilities within Plone. Deco as a whole was not mature enough for us to feel comfortable using it, but I knew that the tile rendering was one of the older and more mature parts of Deco, and we realized that it would not take a lot of effort to use tile rendering without the rest of Deco.
A tile is a snippet that can be inserted into a template as a div with a data-tile attribute, like this:
<div data-tile="/Plone/@@mytile" />
Then some machinery in the publisher provided by plone.app.blocks performs the following steps:
- It finds all the divs with a data-tile attribute (let's call them tile placeholders).
- For each one, it performs a subrequest to fetch the contents of the tile. Using a URI makes tiles very flexible: a tile could be a browser view, or it could come from some external system.
- The tile placeholder is replaced with the contents of the tile's body tag. If the tile has a head tag, its contents will be appended to the head of the page that includes the tile.
That's a great start! As it turns out, other parts of the tiles implementation also help support our use case:
- plone.tiles has a tile implementation which supports having multiple tile types. A tile turns out to basically be a browser view that also happens to have some associated data. (This is a lot like a portlet renderer, but one that can be added anywhere with a line in a template rather than needing to mess around with portlet managers.) Each type of tile can specify a different schema for its data, and that data can be persisted in different ways.
- plone.app.tiles provides an edit form that takes care of editing the data for a particular instance of a tile.
In practice: adding a rich text tile
So let's see how this plays out in practice. We are going to:
- Set up the basic tile rendering machinery.
- Implement a rich text tile that can be added anywhere, and that stores its contents in an annotation of the context where it is added.
- Make sure that editors see a pencil icon that brings up an modal overlay to edit the tile.
The basics
Okay, let's get the basics set up.
- Create a package that declares dependencies on: lxml, plone.app.blocks, plone.app.textfield, plone.app.tiles, and plone.tiles.
- At the time of this writing, you need trunk checkouts of plone.app.blocks, plone.app.tiles, and plone.tiles.
- Make sure that your configure.zcml includes
<includeDependencies package="."/>. - Make sure the metadata.xml in your package's GenericSetup profile runs the default profiles from plone.app.blocks and plone.app.tiles as dependencies.
- Install your package.
The tile
Add a tile.py with the following:
from zope.interface import Interface from plone import tiles from zope.schema import Text from plone.app.textfield import RichText from plone.app.textfield.interfaces import ITransformer class IRichTextTileData(Interface): text = RichText(title=u'Text') class RichTextTile(tiles.PersistentTile): def __call__(self): text = '' if self.data['text']: transformer = ITransformer(self.context, None) if transformer is not None: text = transformer(self.data['text'], 'text/x-html-safe') return '%s' % text
In configure.zcml, add:
<plone:tile name="groundwire.tiles.richtext" title="Groundwire rich text tile" description="A tile containing rich text" add_permission="cmf.ModifyPortalContent" schema=".tile.IRichTextTileData" class=".tile.RichTextTile" permission="zope2.View" for="*" />
This defines a new tile type, called groundwire.tiles.richtext. This tile type has a schema with a single rich text field, and when it is rendered the tile will run the configured text through the safe HTML transform to make sure it is safe.
Wiring in the edit form
Now we just need to make sure that editors will have a way to access the edit interface for tiles.
Add the following javascript. Make sure you put a condition on it like "python:object.portal_membership.checkPermission('Modify portal content', object)" so that it will only run and add the edit links for users who have permission to edit.
jQuery(function($) { $('div[data-tile]').each(function() { $(this).addClass('tile-editable'); var href = $(this).attr('data-tile'); var edithref = href.replace(/@@/, '@@edit-tile/'); $('<a class="tile-edit-link" href="' + edithref + '"><img height="16" src="pencil_icon.png" width="16" />') .appendTo($(this)) .prepOverlay({ subtype: 'iframe', config: { onClose: function() { location.reload(); } } }); }); // Check if tiledata is available and valid if (typeof(tiledata) !== 'undefined') { // Check action if (tiledata.action === 'cancel' || tiledata.action === 'save') { // Close dialog window.parent.jQuery('.link-overlay').each(function() { try { window.parent.jQuery(this).overlay({api: true}).close(); } catch(e) { } }); } } });
This adds an edit link to all the divs that have data-tile attributes. It also handles the "tiledata" which is how the plone.app.tiles edit form controls when the overlay it appears in should close.
And finally we need a bit of CSS to style the tiles and edit links:
.tile-editable { position: relative; outline: 2px dashed #e8e8e8; min-height: 1.5em; } .tile-editable:hover { outline: 2px dashed #b8b8b8; } .tile-edit-link { display: none !important; position: absolute; right: 1px; bottom: 1px; z-index: 500; } .tile-editable:hover .tile-edit-link { display: block !important; }
Adding a tile
Okay, now let's add one of these to a template. Pick your favorite template and add:
<div tal:attributes="data-tile string:${context/absolute_url}/@@groundwire.tiles.richtext/hello-world" />
Here's what it looks like in my instance (I added it to the document_view template):

And here's the editing interface that shows up when I click on the pencil:

(If you want to see how this code all comes together, look at the code in https://groundwire.devguard.com/svn/public/groundwire.tiles/branches/davisagli-blocks)
In conclusion
We are very happy with the way the tile approach turned out for the Net Impact site. Once we had mastered the basic technique for landing pages, we soon realized that tiles provided a useful way to add user-editable content areas anywhere in the site. Site needs a doormat in the footer? Use a tile with the Plone site as its context so it appears the same throughout the site and the client can edit the links. Client wants a block of text they can edit on the login form to promote registering for the site? No problem, just add a tile. Client is repeatedly asking for minor edits to the text introducing a custom form? No problem, we turned it into a tile and told them how to edit it. Since the presentation of tiles in the UI is simple and consistent, the barrier to entry for the client to learn how to edit a new tile was very low.
The approach as described here isn't perfect. One thing that needs some care is cache invalidation. In our case, we wrote an ObjectModified event handler for tiles that updates the modified time of the page on which the tile appears. Another limitation is that text in tiles won't be included in the fulltext index unless you go to extra lengths. Whether that's a feature or a bug depends on your use case.
Overall though, we love the technique and have also started using tiles in other sites. I know that Six Feet Up has also successfully used tiles with at least one client. If you want to expand your Plone layout repertoire without using experimental technology like Deco or removing control over content from your clients, I encourage you to give it a try!
04 Feb 2012 9:01pm GMT
03 Feb 2012
Planet Plone
Connexions Developers Blog: Connexions Development Update 2-3-2012
Well, the dream of posting a weekly update is not working out. Things are very busy here and it is a challenge to find the time to write this post. I am committed to writing an update periodically so I'm going to modify my plan to make this bi-weekly.
CCAP
- The Physics rendering in Prince XML is complete except for some math issues. The code is on the development server and should move to production soon with Sociology.
- Our current focus is on getting the Sociology rendering tested and released. The current release date is Feb. 8th or 9th.
- There are several Math issues in the Physics book. Some of the problems were caused by the Word importer, some by the original structure of the math in the Word documents and others are font issues. When the Physics book is migrated to production, we will run a script to clean up some of the import problems to minimize the need for human intervention. Some of the font issues will not be able to be resolved because the Stix fonts do not have the needed font. An example is an italicized delta.
- We have over 50 people signed up for the Sprints! We are very excited about this. As best we can tell, about 30 of these will be developers/coders/designers. With this many people involved, we are going to make a huge effort to have clear easy install instructions for the various options. Next week, we will be testing everything and finalizing the instructions.
- Once the instructions are complete, we will post the link on the Rhaptos list so you can install Rhaptos or anything else you might need prior to arriving at the conference.
- Many thanks to all that are planning to Sprint with us!
- We released the latest version of the OERPub API last week. It will be used during the Conference Sprints by the sprinters working on content.
- We have had a little time to continue our discussion in house regarding a new editor for Connexions.
- Most of the discussion has been Microdata vs Microformats. Neither has much traction in the wild.
- CSS3 seems to have support for Microdata which would allow us to use it to decorate semantic elements. Microformats also have CSS support.
- There has also been discussion on which editor to select. TinyMCE and Aloha have been our focus.
- TinyMCE has a nice UI, but does not support all of HTML5.
- Aloha supports all of HTML5, but has an odd UI. The UI is not bad, just takes a little getting use to. The demos on Aloha's site are very specific so we need to install it and play with the configurations of the UI.
- I'm sure the editor will be discussed at the conference so let us know your thoughts if you are attending.
03 Feb 2012 3:18pm GMT
Four Digits: Behold: the Ploneconf 2012 team
As from now, Four Digits has started the initial preparations on the upcomming Ploneconf in october. We have the venue and we have formed a team to make it happen. If you have suggestions, ideas or questions, don't hesitate to contact us.
While writing the bid for the conference, we also formed a task force that will handle the organisation. It is a lot of work so we need all the help we can get. Down the road we anticipate on more people who can assist, but Four Digits will handle the main things to do. Here is how it is done.

From left to right
Maarten Kling: Attendee manager
Maarten was responsible for writing the bid and during the conference, he is in charge of registration, the budget, information desk and welcome. Questions about where to stay, your fee or anything related to money and sponsoring, contact him.
Ralph Jacobs: Social Activity manager
Ralph is the man you need to contact for information about Arnhem, what to see and what to do. He makes sure you will visit the best bars, the best restaurants and the thing you must see when visiting the city. It is his responsibility to come up with a great beter half program.
Sjoerd van Elferen: Venue & Sprint manager
Sjoerd will handle everything that happens at the venue. He is in contact with the people there and he is in charge of things and stuff we need there: beamers, tables, chairs and things like coffee and lunch. Sjoerd also coordinates the lightning talks, sprints and stand ups.
Rob Gietema: Program manager
Rob is assigned to scheduling and programming. He will make a day to day program that contains all the talks, sprints and open spaces. At the venue, we have multiple rooms and a great auditorium; Rob has the overview on what is happening where and when. Apart from that, Rob takes care of the key note speaker and other VIP's who will speak at the conference.
Yadi Dragtsma: Marketing manager
Our marketing and PR guy. He does the contact with the press, handles the website and social platforms like Twitter, this blog and other meda. If all goes well, everybody in Arnhem is going to know that Plone is in town. During the conference, Yadi will do reports, stories and updates for the people back home.
Martijn Jacobs: Party manager
Martijn is not in the photo, but nevertheless he plays a vital part in the organisation. Martijn is developer, DJ and part of a successful dance music act. He is the party manager and takes care of the conference fest. The music, the equipment, the acts...Martijn is on top of things.
For information or questions regarding the conference:
info@ploneconf.org
03 Feb 2012 1:15pm GMT
02 Feb 2012
Planet Plone
Netsight Developers: '10 Lessons' Intranet Event back by popular demand

On March 21st Netsight is hosting it's second half-day event '10 Lessons learned from building successful corporate intranets and portals'.
Due to the popularity of the first event we have decided to host the event again at the SS Great Britain.
Our consultants and developers will be giving talks and demonstrations based on our experiences of delivering over 200 projects from clients ranging from start-ups to multinational corporations, public sector bodies and NGOs.
This is a free event ideal for anyone involved in the implementation or management of a corporate intranet or portal, or anyone considering building an intranet.
For more information:
Download our event overview PDF
THE DATE21st March 2012 THE PRICEIt's Free THE VENUEThe SS Great Britain
|
HOW TO REGISTERSecure your place either online10intranetlessons.eventbrite.com or by email |
(Attendees will have the opportunity to take a tour of the ship either before or after the session programme.)
02 Feb 2012 6:22pm GMT
01 Feb 2012
Planet Plone
Connexions Blog: OpenStax College Goes Live!
OpenStax College is our new textbook initiative. Free peer reviewed textbooks that are readable and accurate. The site went live today so check it out.
We will be talking more about OpenStax College at the Connexions Conference in a couple of weeks.
You can follow OpenStax College on Google+, Facebook and Twitter.
01 Feb 2012 10:31pm GMT
Lennart Regebro: Release announcement: jquery.recurrenceinput and plone.formwidget.recurrence
The first beta release of jquery.recurrenceinput and plone.formwidget.recurrence has been released.
jquery.recurrenceinput is a Javascript widget, supporting both input and read only modes, for recurring events, supporting iCalendar type recurrence. plone.formwidget.recurrence is the Plone integration, with an Archetypes and a z3c.form implementation.
It is to my knowledge the only open source, generic (ie not written for a specific platform) recurrence widget, and it is amongst the most advanced and full-featured widgets I've seen, full stop.
jquery.recurrenceinput uses jQuery and jQuery Tools Dateinput widget to transform a textinput fields into an input that will not only allow you to show a dialog box where you can in a user friendly matter set up recurrence, but it will also (with the appropriate backend implementation) show a list of upcoming instances of this recurrence, and allow you to add more as well as exclude some.
If you are in any way involved in open source and web and need to have some sort of events or calendars, you should check this out, test it and come with feedback.
Filed under: calendaring, django, javascript, plone, python, zope Tagged: jquery, recurrence
![]()
01 Feb 2012 5:33pm GMT
Lennart Regebro: Finding the ideal Plone Conference location
This saturday, Karl Horak tried to find a good place for the Plone Conference, but by taking an weighted average of the location, he only found where the average Plone Conference is located. Which seems very cold and wet, and not ideal at all. I think his methodology can be improved on. The average place is not necessarily the ideal place.
If we look at where the conference has been located as an average of the attendance, we get this sort of graph for the the longitude:
We can see that the longitude makes no noticeable difference. The latitude is a different story, though:
The two outliers are Washington and Bristol, and we can of course ignore them as statistical flukes. And then we get a linear formula for attendance as a function of latitude:
attendance = 400 + latitude in degrees * 1.1
So, the more easterly, the better. Being just on the edge of the international date line would then yield the best results with 598 attendants, but it will be hard to find a good conference location there. Instead I recommend New Zealand. But as east becomes west as an arbitrary location, there is a small risk that the cutoff point is not the international dateline, and that New Zealand is on the "negative" side, and yield only 195 attendants, but I find that unlikely. If we want to be on the safe side, we do need to go gradually east and see. As we like nice weather in the Plone community, good locations would be Cairo (434 attendees), Mogadishu (450 attendees) or if you want to take a bigger risk, The Maldives with 480 attendees, or possibly 84, depending on where the cutoff is.
There is also a possibility that the function isn't linear. It could for example be cubic, which would make the projected attendances wrong, but the conclusion should still be to go east, although the east cost of the US would work well too, with Florida in preference for weather. Or for that matter, Cuba.
Or it could be a cyclic function:
In this case the best latitudes would be 148W, 108W, 68W, 28W, 12E, 52E, 92E, 132E and 172E. That gives us the following desirable locations:
- Anchorage, Alaska, US (149W)
- Billings, Montana, US (108W)
- La Paz, Bolivia (68W)
- The Azores (28E-25E)
- Tripoli, Libya (12E)
- Tehran, Iran (51E)
- Lhasa, Tibet (91E)
- Vladivostok, Russia (131E)
- Christchurch, New Zealand (173E)
What we do see here is that no matter what the function is, New Zealand would be a good bet, and therefore the inevitable conclusion is that there should be no call for proposals, we just you tell Tim Knapp to do the Plone Conference. Remember: Correlation *is* causation.
Filed under: plone, plone conference, python
![]()
01 Feb 2012 5:20pm GMT
Plone.org: Registration Now Open for Plone Symposium East 2012
Registration is open for the 5th annual Plone Symposium East 2012. This event will be held in State College, Pennsylvania, May 22 and 23, 2012. Your hosts will be the indefatigable WebLion group of Penn State University.
We keep our costs low by cunning and craft. Register by March 19, and pay a discounted fee of $285, and only $360 thereafter.
Proposals for presentations at the Symposium are also open, and if yours is selected, your registration fee is reduced to $210. Proposal accepted through March 12, 2012.
Just prior to the Symposium - May 19-21 - training workshops will be held (propose a workshop here). Training proposals are due by February 13th, 2012. So time is running out.
After the Symposium, May 24-26, we'll gather for development and PloneEdu sprints.
See you in State College!
01 Feb 2012 2:15pm GMT
31 Jan 2012
Planet Plone
Jorgen Modin: Plone training in London March 27-29, 2012
We will do Plone training in London March 27-29. Same contents as usual, with the addition of Diazo skinning and Dexterity.
We will most likely be at Portman Square (depending a bit on the size of the group).
31 Jan 2012 1:42pm GMT
Six Feet Up: A Solution to Fixed-Bid vs. Hourly Contracts (Part 1)
Right now, I'm waiting for Audi to give my car an oil change and fix a problem with the power steering. While the technician told me that it shouldn't take more than two hours, we're nearly at three (and counting.)

Of course, the moment I wrote that, the technician came over to explain the delay. Turns out they found a couple of unexpected (but legitimate) issues which they needed to fix. Things weren't quite as simple as he originally assumed.
Pretty typical.
Which brings me to this. If an auto-mechanic can't accurately estimate a quick and simple car fix, what hope do software developers have when building something new and complex, over the course of many months (or years,) that's never been exactly done before?
Technology projects are notorious for cost overruns.
The Hubble Space Telescope, which has allowed us to look back in time nearly all the way to the big bang, was originally supposed to cost $400 million to build. Final construction costs actually came in at $2.5 billion.
Boston's "Big Dig" and Denver's International Airport were each supposed to cost $2.8 billion. They cost $14.6 billion and $4.8 billion respectively.
A study performed a few years back by the Standish Group discovered that 71% of IT projects were over budget with a total average overage of 43%.
Why does this happen? And what does a space telescope or a major construction project have to do with your website or intranet?
The answer is they are all creative exercises. They are unique. There isn't another tunnel exactly like Boston's in the entire world, and there isn't another website exactly like yours. You can't build your website on an assembly line, and if you could, you really wouldn't want to.
Sure, it would be great if we could send a bunch of commodity-priced raw materials into an airport construction factory, click the "on" button, wait a predictable amount of time, and out comes your airport, ready for takeoffs, landings, and weather delays.
Predictable inputs. Predictable outputs. Ah….
But websites and intranets are more like art, music, and literature. I want to commission you to compose a 45-minute orchestral masterpiece for our symphony. How much time will that take you? Um…
"Websites and intranets are more like art, music, and literature..."
OK, not all web projects are complex. Some are simple and straightforward and can be easily developed by your nephew who learned some HTML as part of a school project. Those just aren't the kinds of projects we work on.
Six Feet Up builds sophisticated websites, intranets, and extranets.
For example, we develop a lot of websites on Plone, an open source, Python-based content management system. Plone is perfectly suited for complex content management, customized content types, intricate workflows and permission structures, and integration with other discrete technology platforms (i.e. CRM systems like Salesforce, authentication systems like LDAP, etc.) With Plone, you can create and manage numerous sub-sites that inherit content and templates from a parent site. You can build a global website, managing content in dozens of different languages.
Plone is unbelievably powerful and flexible. It's also quite user-friendly for content managers.
But, Plone development is not simple. For those with generic, simpler needs, a solution like WordPress may be more appropriate (and easier to estimate.)
So when it comes to budget estimates, where does this leave us? When every project has unique requirements, how can a precise estimate be developed? (It can't.) And how can any project move forward without a reliable budget estimate? (Again, it can't.)
What kind of payment structure can be created that's fair for all parties?
"What kind of payment structure can be created that's fair for all parties?"
We have an answer that's worked extremely well for our clients. Our solution and process aligns interests and minimizes risk. It provides transparency and allows flexibility. It's relationship-building and hands-on.
What is it?
Stay tuned for part 2.
Want to be notified when part 2 is ready? Sign up here….
31 Jan 2012 11:30am GMT
Plone.org: News from the Plone-iverse - January 2012
Plone Conference 2012 Coming to Arnhem, the Netherlands
Arnhem, the Netherlands will host the 2012 Plone Conference, the Plone Foundation announced today following a vote of the foundation membership. The Arnhem proposal, championed by Dutch Plone Developers Four Digits, beat out a competing proposal for the conference to be held in Paris, France which was made by Abstract and Fanelli Consulting.
The Plone Conference 2012 will be held in Arnhem on October 10-12, with pre-conference training on October 8-9 and a post-conference sprint October 13-14.
You can find out more here - http://plone.org/news/plone-conference-2012-host-announced
Munich Plone Conference is Just Around the Corner
The first Plone Conference in Munich is being held from February 22-24, featuring talks, training and Sprints. Keynote Speakers include Plone Foundation President Matt Hamilton, Elizabeth Leddy of the Plone Framework Team, and Plone co-founder Alan Runyan.
You can find out more about the conference by visiting their website - http://konferenz.plone.de/
Introducing Ploud
by Saba Resnick
Ploud is a free software as a service platform for the Plone content management system (Plone SaaS).
Evaluating and deploying a basic Plone site got a lot easier recently thanks to the folks at Enfold Systems. They have developed Ploud, a platform that allows users to create a Plone site in under a minute (or as it is written on the company's web site, in 10 seconds). Ploud can be used by individuals, consultants and organizations to evaluate or deploy the Plone CMS without having to install any software. In addition to Plone's standard features, Ploud includes popular, trusted, add-ons for enhanced functionality like web forms, image rotation, calendaring, Facebook integration, and a choice of themes.
Ploud is free for basic usage. If later your site requirements grow in complexity or you need expanded support, you can easily migrate to a fee-based hosting platform.
Ploud is deployed using a high-availability VMware vCloud solution on multiple redundant SANs with redundant switch fabric. The facilities feature low-latency IP connections through multiple providers on enterprise Cisco routers and switches. The facility has 24×7 onsite security staffing with surveillance and access control and is SAS 70 Type II certified.
Ploud provides a valuable service to the Plone community by allowing anyone to quickly and easily create a free Plone site. It is currently in Beta status and Enfold Systems is soliciting feedback to help them improve the service. Give it a try!
Plone Welcomes New Contributors
Since the recent Plone Conference 2011 in San Francisco, we've seen an influx of new contributors sign up to add their code to Plone. We'd like to welcome the newest members of the contributors group and say how much we appreciate your deciding to be involved.
The latest contributors are:
Adam Cheasley
Adam Terrey
Aleksi Korvenranta
Alin Voinea
Andre Gustavo E Nogueira
Asko Soukka
Chris Thomas
Cleber J Santos
Daniel Marks
David Mote
Derrick Stone
Douwe uld Mey
Emanuel Sartor
Esa Matti Suuronen
Federico Marcelo Castro
Franco Pellegrini
Ghica Alexandru
Giacomo Spettoli
Gustavo A. Lepri
Heather Harter
João Sebastião de Oliveira Bueno
Jukka Ojaniemi
Kai Lautaportti
Kim Paulissen
Lukas Graf
Marco Martinez
Mark Corum
Michael Davis
Noe Nieto
Nuyens Andre
Philippe Gross
Richard Barrell
Richard Mitchell
Roberto Allende
Roman Susi
Ross Reedstrom
Thiago Tamo Sauskas
Tom Blockley
Tyler Randles
William Deegar
Chrissy Wainwright and Maurizio Delmonte Chosen Plone Foundation Members
The Plone Foundation Board of Directors has announced the selection of Chrissy Wainwright and Maurizio Delmonte as the newest members of the Plone Foundation.
Chrissy Wainwright is the Senior Template Developer at Six Feet Up, based in Indianapolis, Indiana (USA). She has implemented Plone themes for dozens of sites, including indianahistory.org, uvahealth.com, and sfari.org. She has taught Plone theming classes for several years, including the Plone Symposium East at PSU and the Plone Conference 2011 in San Francisco.
Maurizio Delmonte is a Plone consultant with Abstract Open Solutions in Potenza Picena, Italy. Part of the Plone community from the outset, he contributed the first italian translation of Plone, and still maintains the current version with the Plone Italian Translation Group.
You can find out more here - http://plone.org/news/new-foundation-members-wainwright-delmonte
Plone Symposium East Puts Out Calls for Speakers and Training
Plone Symposium East 2012 (PSE12) at Penn State University is just around the corner - May 22-23, 2012.
Speakers
Share your knowledge with the Plone and Zope communities at the Plone Symposium East 2012. Tell others about what you are doing. Teach others about your latest development. Show off your latest tools. Weblion has begun accepting proposal for Speaker sessions. Speaker slots fill up quickly and speakers get a reduced registration fee, so get your proposal in TODAY! Proposals will be accepted until March 12, 2012, so don't delay! Here is the link for proposing talks - http://weblion.psu.edu/symposium/session-proposal
Training
The folks at Weblion have put out the call for trainers who would like to hold training sessions in the two days prior to PSE - on May 20-21st. It is an opportunity to teach others how to be successful with open source. Interested trainers have time to submit their proposals between now and February 13. You can find out more about the proposals process by visiting http://weblion.psu.edu/symposium/training-proposal
Meet the Plone Editorial Team
We're happy to announce that a brand new group of writers and editors located around the world has stepped up to help make that news more up-to-date and global. The new Plone Editorial Team will oversee the writing and approval of news on Plone.org as part of our overall efforts to better market and communicate about our favorite CMS.
Jules Allen
Mark Corum
Jorge Gonçalves
Gustav Franzsen
Carol Ganz
Jean Jordaan
Sally Kleinfeldt
Rose Pruyne
Saba Resnick
Jon Stahl
Rijk Stofberg
You can find out more here - http://plone.org/news/plone-editorial-team-announced
Got a story you'd like to see in next month's News from the Plone-iverse? Send a note to news@plone.org and we'll do what we can to include it.
31 Jan 2012 6:28am GMT
30 Jan 2012
Planet Plone
Plone.org: Plone Symposium East -- Speaker Proposals Due March 12, 2012
Tune up your voice! Spruce your notes!. Dust off your tux! The Plone Symposium East 2012 (PSE12) at Penn State University is looking for speakers to share their experiences.
Share your knowledge with the Plone and Zope communities at the Plone Symposium East 2012. Tell others about what you are doing. Teach others about your latest development. Show off your latest tools. Weblion has begun accepting proposal for Speaker sessions.
This is the time to submit your proposal to speak at PSE12 events on May 19-21, 2012.
Speaker slots fill up quickly and speakers get a reduced registration fee, so get your proposal in TODAY! Proposals will be accepted until March 12, 2012, so don't delay!
Here is the link for proposals: http://weblion.psu.edu/symposium/session-proposal
30 Jan 2012 8:30pm GMT
Netsight Developers: '10 Lessons' Intranet Event

On October 5th Netsight is hosting a half day event in Bristol focussing on our 10 Lessons learned from building successful corporate intranets and portals.
Our consultants and developers will be giving talks and demonstrations based on our experience of delivering over 200 projects for clients ranging from start-ups to multinational corporations, public sector bodies and NGOs.
This is a free event ideal for anyone involved in the implementation or management of a corporate intranet or portal, or anyone considering building an intranet.
For more information download
our event overview PDF
THE DATE5 October 2011 THE PRICEIt's Free THE VENUEThe SS Great Britain
|
HOW TO REGISTERSecure your place either online10intranetlessons.eventbrite.com or by email |
(Attendees will have the opportunity to take a tour of the ship either before or after the session programme.)
30 Jan 2012 4:01pm GMT
