Bjørn Stabell 白熊

Although easy_install might be sufficient to install a Python package (aka egg) onto your system, it doesn’t help you find available packages or see what you have on your system. Most package managers does, so this was a bit surprising. I was frustrated enough that I started looking at what it would take to make one. I came so far as to create a little utility built on top of setuptools that downloaded the Python Package Index (aka cheeseshop, aka PyPI) using httplib2 with caching.

Searching for pypi using this utility I saw a couple of packages, including one called yolk that promised to be a “Command-line tool querying PyPI and Python packages installed on your system.” Great! Saved me a lot of work.

Yolk not only allows you to search PyPI using yolk -S <name>, it also gives you a way to query the packages that you have installed on your system using yolk -l. I’ve NEVER seen this before for Python eggs, so it was quite fun. Yolk doesn’t cache stuff from the PyPI, however, so that’s a bit of a downer for us living in bandwidth-challenged areas like China.

If you wonder, yolk is a word play on “egg yolk“, which is the yellow part of the egg. Python packages as done by setuptools are called eggs, thus the name.

By the way, httplib2, although not part of the standard library, is quite a bit better than urllib2 or httplib, and apparently the only way to easily make HTTP requests that use HTTP methods besides GET and POST; i.e., PUT, DELETE, HEAD, etc, as required by REST. It also exposes a very feature-complete client-side cache, so you can do:

I hope the standard library cleanup project that is planned for Python 3000 includes this module, and updates it so that it can support file-like interface to reading, not just returning a string like it does now. A compatibility module with the old http/url modules would be cool too.

Update! Flávio blogs about this too, includes a script for updating Python packages, and in a comment mentions enstaller, an alternative package manager for Python eggs which can do similar things to yolk, but also upgrading of packages. Mikeal also blogs about this and includes a simpler script for updating Python packages.

After hearing about it on an ITC podcast (also mentioned in previous post) I ordered the “RESTful Web Services” book from Amazon (and got into an argument with Sam Ruby regarding Python pain points; he seems to be a Ruby-convert these days). It arrived last week, and I spent a lot of this weekend browsing through the book, and this is a review of sorts.

This book has a star rating of 5 out of 5 on Amazon, and I agree it’s worth it. It covers a lot of ground and includes material rarely found elsewhere.

The first part covers REST basics, like the meaning and usage of the HTTP methods GET, PUT, DELETE, POST, as well as HEAD. I’m pretty familiar with that part of REST and HTTP, but people that don’t know the basic premise of REST, and haven’t heard of conditional GETs, partial GETs, caching, content negotiation, compression, and the various HTTP return codes should read the first part carefully; the whole premise of REST is to use HTTP the way it was designed, so you’d better know HTTP. The REST Wikipedia entry gives a good basic intro to REST. I find many web site optimization sites give a good introduction to HTTP features.

The authors also show how to use existing best-practice web services such as Amazon’s S3 and Google.

The meat of the book is in the middle where the authors show how to design a read-only and read-write web application in a RESTful way. There were a few new and interesting things for me:

1. A step-by-step guide to designing a RESTful web services app
2. A discussion of URI design conventions
3. An overview of hypermedia formats (e.g., XML and microformats) that can be used to represent state
4. Discussions about the tricky situations where REST just doesn’t seem to fit, e.g., when you need transactions

I’ll summarize the first three.

(more…)

This post shows how name components, which one is sorted on, and which one is normally used, vary around the world. The Chinese section is a bit weak as it doesn’t mention common titles such as 老毛 and 小毛, and the common use of nicknames, often by doubling given name characters like 东东, but the author gets his point across. Maybe it’s time to just use a full name field, and perhaps a field for an alias or preferred shorter name?

Martijn Faassen (whom apparently started EuroPython years back) did the funniest lightning talk at this year’s EuroPython, promoting his new web framework Grok, built on Zope:

Talking to him, he seemed very passionate to reinvigorate Zope through Grok. I must say I think both Zope 3 and Grok are quantum leaps forward compared to Zope 2.x, and I’m sure they’ll pick up more developers over time. It’s going to take some time for me, though, who got pretty burnt by the Zope/ZODB combination, to give it another try again. And I don’t think I’ll ever trust a database that doesn’t explicitly enforce a schema; it’s just a recipe for maintenance hell.

This is of course pretty controversial stuff, and in all honesty, Arlo only has his own data and the feedback of 6 teams that tried it as supporting evidence that it works. Of the 6 teams, all had productivity boosts. 4 teams continued, 2 stopped. The reasons why they stopped was interesting, though: they liked specializations, and were happy with slightly less productivity.

(more…)

The final practice Arlo described was team-owned tasks. Avoiding assigning tasks to people is very important to encourage team responsibility, which leads to team accountability, which is a necessary requisite to building strong self-organizing teams.

(more…)

Another question they had was:

• Who should do which tasks?

The normal approach is to put the most qualified implementor on a task. Naturally, that should ensure the task got done faster and better, right? Well, just to see how it worked, they once tried the exact opposite: putting the least qualified implementor on a task. Again, what they found was astonishing.

(more…)

Arlo’s team wanted to be extreme, and they started with one of the XP practices, pair programming. Going extreme meant that they would pair program all the time. No code produced by a single person was allowed to be checked into the repository.

But, pair programming is also a pretty new and relatively undefined practice. There were still a lot of questions open, among them:

• How long should each pairing session last?

  (more…)

The most interesting and eye-opening talks at EuroPython 2007 were probably Arlo Belshee’s talks on agile methodologies and his team’s experiments and results in that regard. The following blog posts summarizes what I learned from listening to his talks, doing his XP workshop, discussions with him, and digging up related information on the web (such as a paper and a podcast). I strongly believe this could be the next phase in how we develop software, and I hope I’ll be able to whet your aptetite for change as well

(more…)

Via Simon Willison’s blog, SlideShare has an impressive list of slideshows about web application scalability.