fidzu : Turbogears

One of the key features of TurboGears2 is the great CRUD extension. Mastering the CRUD extension can really make the difference between spending hours or just a few minutes on writing a web app prototype or even a full application.

The CRUD extension provides two main features, the CrudRestController which is meant to help creating totally custom CRUDs and the EasyCrudRestController which provides a quick and easy way to create CRUD interfaces.

I'll focus on the EasyCrudRestController as it is the easiest and more productive one, moving forward to the CrudRestController is quite straightforward after you feel confident with the Easy one.

The target will be to create, in no more than 40 lines of controller code, a full featured photo gallery application with:

• Multiple Albums
• Uploads with Thumbnails Generation
• Authenticated Access, only users in group "photos" will be able to manage photos
• Contextual Management, manage photos of one album at time instead of having all photos mixed together in a generic management section

If you don't already know how to create a new TurboGears project, start by giving a look at TurboGears Installation for The Impatient guide. Just remember to add tgext.datahelpers to dependencies inside your project setup.py before running the setup.py develop command.

I'll start by providing a Gallery and Photo model. To store the images I'll use tgext.datahelpers to avoid having to manage the attachments. Using datahelpers also provides the advantage of having thumbnails support for free.

from tgext.datahelpers.fields import Attachment, AttachedImage
 
class Gallery(DeclarativeBase):
    __tablename__ = 'galleries'
 
   uid = Column(Integer, autoincrement=True, primary_key=True)
   name = Column(Unicode(100), nullable=False)
 
class Photo(DeclarativeBase):
    __tablename__ = 'photos'
 
    uid = Column(Integer, autoincrement=True, primary_key=True)
    name = Column(Unicode(100), nullable=False)
    description = Column(Unicode(2048), nullable=False)
    image = Column(Attachment(AttachedImage))
 
    author_id = Column(Integer, ForeignKey(model.User.user_id)))
    author = relation(app_model.User, backref=backref('photos'))
 
    gallery_id = Column(Integer, ForeignKey(Gallery.uid))
    gallery = relation(Gallery, backref=backref('photos', cascade='all, delete-orphan'))

Now to be able to start using our galleries we will have to provide a place where to view them and a gallery management controller to create and manage them. Viewing them should be quite straightforward, I'll just retrieve the galleries from the database inside my index method and render them. To access a single gallery I'll rely on the datahelpers SQLAEntityConverter which will retrieve the gallery for us ensuring it exists and is valid. For the management part I'll create an EasyCrudRestController mounted as /manage_galleries

from tgext.crud import EasyCrudRestController
 
class GalleriesController(EasyCrudRestController):
    allow_only = predicates.in_group('photos')
    title = "Manage Galleries"
    model = model.Gallery
 
    __form_options__ = {
        '__hide_fields__' : ['uid'],
        '__omit_fields__' : ['photos']
    }
 
class RootController(BaseController):
    manage_galleries = GalleriesController(DBSession)
 
    @expose('photos.templates.index')
    def index(self, *args, **kw):
        galleries = DBSession.query(Gallery).order_by(Gallery.uid.desc()).all()
        return dict(galleries=galleries)
 
    @expose('photos.templates.gallery')
    @validate(dict(gallery=SQLAEntityConverter(Gallery)), error_handler=index)
    def gallery(self, gallery):
        return dict(gallery=gallery)

Logging in with an user inside the photos group and accessing the /manage_galleries url we will be able to create a new gallery and manage the existing ones.

To configure how the crud controller forms should appear and behave the __form_options__ property of the EasyCrudRestController can be used. This property relies on the same options as Sprox FormBase and customizes both the Edit and Add forms.
The next part is probably to be able to upload some photos inside our newly created galleries. To perform this we will create a new EasyCrudRestController for gallery photos management.

from tgext.crud import EasyCrudRestController
from tw.forms import FileField
from tw.forms.validators import FieldStorageUploadConverter
from webhelpers import html
 
class PhotosController(EasyCrudRestController):
    allow_only = predicates.in_group('photos')
    title = "Manage Photos"
    model = model.Photo
    keep_params = ['gallery']
 
    __form_options__ = {
        '__hide_fields__' : ['uid', 'author', 'gallery'],
        '__field_widget_types__' : {'image':FileField},
        '__field_validator_types__' : {'image':FieldStorageUploadConverter},
        '__field_widget_args__' : {'author':{'default':lambda:request.identity['user'].user_id}}
    }
 
    __table_options__ = {
        '__omit_fields__' : ['uid', 'author_id', 'gallery_id', 'gallery'],
        '__xml_fields__' : ['image'],
        'image': lambda filler,row: html.literal('‹img src="%s"/›' % row.image.thumb_url)
    }

Mounting this inside the RootController as manage_photos = PhotosController(DBSession) it will be possible to upload new photos inside any gallery. To manage the photos inside the first gallery for example we will have to access /manage_photos?gallery=1url.

Each parameter passed to the EasyCrudRestController is used to filter the entries to show inside the management table and the keep_params option provides a way to keep the filter around. This makes possible to edit the photos of only one gallery at the time instead of having all the photos mixed together. Also when a new photo is created it will be created in the current gallery.

The PhotosController got more customization than the GalleriesController, through the __field_widget_types__ and __field_validator_types__ options we force the image field to be a file field and using the __field_widget_args__ we ensure that the newly uploaded photos have the current user as the author.

__table_options__ provide a way to customize the management table. The available options are the same as the Sprox TableBase and Sprox TableFiller objects. in this case we hide the indexes of the rows on the database and the gallery itself, as we are managing the photos of a specific gallery we probably don't need to know which galleries the photos belong to. Using the __xml_fields__ we also specify that the image field provides HTML and so doesn't have to be escaped. The image entry forces the table to show the image thumbnail for the image column of the table instead of printing the AttachedImage.__repr__ as it would by default.

At first sight it might sound a bit complex, but once you start feeling confident, the CRUD extension makes possible to create entire applications in just a bunch of code lines. With just a few lines of code we created a photo gallery with multiple albums support and we can now focus on the index and gallery templates to make the gallery as pleasant as possible for our visitors.

The complete implementation of the photo gallery is available as a pluggable application on bitbucket, feel free to use it in your TurboGears projects.

31 Jan 2012 8:22pm GMT

I can call 2011 quite a good year on my behalf however when I look at things I've done, I feel I've done much less than I could. Here is a list of what I have done in 2011.

Things I've Done in 2011

• Executed an idea we had with my brother, http://compector.com.

• Built the start-up above with Ruby on Rails so I became more familiar with Ruby world.

• I have made the switch to PostgreSQL from MySQL.

• Got a job in one of Turkey's top web sites where we deal with thousands of concurrent users...

31 Dec 2011 7:07pm GMT

I have been developing the new version of www.osesturkiye.com for the Turkish version of the show called "The Voice". It's already built with TurboGears, mako and SQLAlchemy. In the new version we have a gallery and many photos in it. My SQLAlchemy model is like this:

class GalleryPhoto(DeclarativeBase):
    __tablename__ = 'gallery_photo'

    id = Column(Integer, primary_key=True)
    photo_image = Column(UnicodeText)
    photo_description = Column(UnicodeText)
    dateline = Column(DateTime, default=datetime.now)

The problem I have is, in the photo detail page I will display "Previous" and "Next" links. I can implement this with two ways..

21 Dec 2011 8:28pm GMT

Tonight, VooDooNOFX was asking in IRC in #turbogears how to disable the injection of jquery.js by tw2.jquery into her/his TG2 app. Using the inject_resources=False middleware config value wouldn't cut it, since she/he wanted tw2 to inject all other resources, they were loading jQuery via google CDN beforehand and tw2′s injection was clobbering their code.

I came up with the following hack to myapp/lib/base.py which will remove tw2.jquery.jquery_js from the list of resources tw2 would inject into each page served by a TG2.1 app.

At the top of myapp/lib/base.py import:

import tw2.core.core
import tw2.jquery

and then replace:

        return TGController.__call__(self, environ, start_response)

with the following:

        stream = TGController.__call__(self, environ, start_response)

        # Disable the injection of tw2.jquery
        offending_link = tw2.jquery.jquery_js.req().link
        local = tw2.core.core.request_local()
        local['resources'] = [
            r for r in local.get('resources', None) if r.link != offending_link
        ]

        return stream

The two tricks to this are

• Simply knowing that tw2 resources register themselves with the 'request_local' object and that during the return-phase of the WSGI pipeline, the tw2 middleware refers to that list when injecting resources
• Figuring out where in a TG2 app's request flow to place the call to alter that object after all widgets that might register jquery have declared their resources but before the resources list is injected into the output stream.

We came out of it with a bug filed in the tw2 issue tracker so we can take care of it properly in the future.

13 Dec 2011 5:44am GMT

12 Dec 2011 8:18pm GMT

Every so often I find myself working on that odd job that requires syncing files with Amazon's S3. In the beginning, I tried some of the various S3 FUSE interfaces-hoping for something that would play nice with rsync-but FUSE's stability always left something to be desired and more often than not I'd be left with that one transfer that never would quite finish correctly.

Eventually I discovered boto and settled in to using a crude (yet stable) Python/boto solution for these type of tasks-all the while wondering why nobody took the time to write a "real" rsync-like client for S3.

Well, this last time around I finally decided to stop whining and take matters into my own hands. After a couple of late nights fleshing out my basic boto solutions, I'm happy to announce what I'm calling "boto rsync"-an rsync like wrapper for boto's cloud storage interfaces (S3 and Google Storage).

Please take a look at the project on github and let me know what you think: http://github.com/seedifferently/boto_rsync

12 Dec 2011 12:55am GMT

If you are using Python and looking for a web framework or if you are just looking to do web programming with Python, you will look out your options and among Django, web2py and pyramid you will notice TurboGears. As a Python developer and TurboGears user, I will let you guys know what is going on with TurboGears lately and will let you have a chance with TurboGears.

A Short Introduction First

TurboGears is a full stack MVC web framework that is built on top of Pylons and which includes an ORM, a templating system, database migration tool, web helpers, authentication and authorization system by default just like Django. If you are wondering, there are two main differences between Django and TurboGears.

• TurboGears is using class based controllers and object dispatch.

• TurboGears is built using external Python libraries such as Pylons, WebOB, paster whereas everything in Django is built for Django.

19 Nov 2011 11:55pm GMT

Recently some work has been done to extend the hooks support in TurboGears, to play a little with the new hooks I decided to try creating the famous and envied Django Debug Toolbar. I'm quite happy of the result and most of the features are there. In a few days I'll be able to place it on a public repository and I'll release it concurrently with the 2.1.4 release of TurboGears.

Debug Toolbar

Timings

Request and Headers

SQLAlchemy Queries

Mounted Controllers

The code has been heavily inspired by the Pyramid Debug Toolbar and have to thank the Pyramid team for the good job they did at making the Toolbar code clean and simple.

12 Nov 2011 7:30pm GMT

yum install nspr-devel
env LIBS="-lnss3 -lnspr4 -lplds4" ./configure
make && make install

export LD_PRELOAD="/usr/lib64/libnss3.so /usr/lib64/libnspr4.so /usr/lib64/libplds4.so"

23 Oct 2011 3:27pm GMT

So, a favor to ask of people: I'm working on a web application to help people manage their resumes. As I've gotten further in, I've realized I don't have an actual todo list for it. So, since I'm making this to be used by others, I'll ask everybody here:

What would you want to see? Currently, I've added enough code to allow the program to output something very close to my online resume ( http://www.icelus.org/ ). Next up, I have the following features on my todo list already:

• add docutils to output strings where appropriate (everywhere?)
• add login
• add ability to edit own resume
• add ability to compile new custom resumes
• add ability to output PDF
• add ability to output ODT
• add ability to output DOC
• add generic logo
• add generic photo / head (shadow) shot
• add ability to login using Google, Yahoo, Facebook, etc
• add ability to send resumes directly from Hiring Pond
• add ability to track where resumes have gone
• add ability to track job hunt progress
• add ability to display current employment status (employed, unemployed, contracting)
• add ability to display current employment seeking status (not at all, active, passive)
• add logo to qrcode
• change colors of qrcode
• change color scheme for all of hiring pond to sunset and water
• add fisherman silhouette as hiring pond logo

What else would you all want to see in order to make you want to use this?

18 Oct 2011 8:44pm GMT

As recently some effort has been involved in improving the performances of TurboGears2, I was curious to see how much things improved. As usually, the test isn't really reliable in any way and was just for fun.

All the graphs report the request/sec the application has been able to perform on my computer with only 1 concurrent client. So higher is better.

Here is the comparison between TG2.0 and TG2dev (will be 2.1.4)

I also compared various setups with different template engines on TG2dev

The comparison happened on an application similar to the quickstarted one.
Actually as there is no database involved in this application the template engine impacts a lot and so was a good benchmark for the template engines themselves.

04 Oct 2011 3:35pm GMT

It's done. An over-engineered WSGI middleware component that adds a velociraptor to every page served. Fact: Every WSGI app is better with a raptor.

It's called raptorizemw (pronounced "awesome") and the only way to use it is in production.

03 Oct 2011 8:35pm GMT

We are pleased to announce the latest release of TurboGears, 2.1.3!

This release adds support for the Mongo object database and the Ming ORM for it, in addition to bringing in some small bugfixes. The full changelog is below

The number of changes is, admittedly, small, but don't underestimate the importance: NoSQL databases are an important tool for the web, and adding the support for Mongo brings TurboGears into the group of frameworks that not only supports it, but supports it well.

We're already looking forward to expanding support and closing even more issues for 2.1.4, so we'll see you next month for that release!

* Added support for MongoDB into the quickstart template
* Added lurl (lazy_url) support for late evaluation
* Adding tests module back into source distribution. Solves ticket 115
* Template caching is now manageable by TG applications
* Clean up output for flash() in default quickstart
* Performance enhancements
* Fixed missing dependencies in TG quickstart on Python 2.4

28 Sep 2011 9:22pm GMT

I wrote an app that scrapes foreclosure data from my county of residence and plots it six ways from Sunday in a TurboGears2.1 app. You can find it at http://monroe-threebean.rhcloud.com/, hosted on redhat's openshift cloud.

It's used by activists with Take Back the Land, Rochester and my local branch of the ISO to find upcoming evictions before they happen and organize the neighborhoods to stop the shuttering of homes. Get a hundred people at the door of the house before the cops come, and no-one is getting evicted (we've had some successes).

We're living in some absurd times where banks got bailed out by the trillions yet still get to collect on our student debt and mortgages. Most of us are being ruined. If you're not, then your neighbor is.

If you're in Boston, check out Vida Urbana

or if you're in Chicago, check out the Chicago Anti-Eviction Campaign. Anywhere you go, check out the ISO.

Fork my code, port it to your home town, and start organizing!

24 Sep 2011 4:01pm GMT

Warum man zur Zeit beim Spazierengehen immer große Taschen mitnehmen sollte...

22 Sep 2011 8:38pm GMT

I know that my followers are few, and my posts are too infrequent. I'm looking for ways to change my posting frequency.

But, even still, I have an important request: TurboGears 2.1.3 is getting ready to be released. We're adding Ming/Mongo support in this release. Anybody who can, please review the changes (core, devtools, and docs), run tests, etc. We need to make sure we've got this as correct as possible for everybody.

Thank you for your time!

18 Sep 2011 9:29pm GMT