Morethanseven

I was just thinking about the Design it Build it conference later in the year (full disclosure: I’m speaking). Specifically the people speaking on the developer track. Between myself, Michael Brunton-Spall from The Guardian, David Singleton from Last.fm and Emma Persky from Gumtree four of the six speakers work on in-house teams. Not early stage start-ups, not large software/advertising companies, not as freelancers but in a reasonable sized company on a development team.

My original background was working in agencies, and then a stint working for myself and I’m constantly interested by the different facets of ...

I’ve been doing more operations related work of late and am starting to use Cucumber-nagios for various monitoring tasks. Nagios might not be the most attractive of web interfaces but it’s so simple to get clients up and running and extend to do what you need. Cucumber however has a lovely, text based, user interface. And although I’m mainly working with Python at the moment cucumber-nagios (written in Ruby) really is the easiest way I’ve found of writing simple functional tests.

Cucumber-nagios is the creation of Lindsay Holmwood and after several brief conversations over Twitter I ...

I found myself using a couple of powerful but underused command line applications this week and felt like sharing.

My problem involved a large text file with over three million lines and a script that processed the file line by line, in this case running a SQL query against a remote database.

My script didn’t try and process everything in one go, rather taking off large chunks and processing them in turn, then stopping and printing out the number of lines processed. This was mainly so I could keep an eye on it and make sure it wasn’t ...

I’ll be heading back up to Newcastle in April to give a talk at what’s shaping up to be a good looking conference to kick off the year with. DIBI is trying to please everyone, with both front and backend focused streams.

Created for both sides of the web coin, DIBI brings together designers and developers for an unusual two-track web conference. World renowned speakers leading in their fields of work will talk about all things web. Taking place in Newcastle upon Tyne, (it’s oop north) at The Sage Gateshead on the 28th April 2010, we’re ...

I’ve been trying to learn Erlang for a while. What I actually mean is it’s been on my list of things to learn for months, along with all sorts of other incredibly interesting bits and pieces. I spend a little bit of time at home but the majority of my learning time is now spent commuting to London and back most days. Sometimes I’m even going all the way to Swindon which gives me even longer to not learn Erlang.

The main problem with learning something new on the train is space. Reading a book (or my ...

The Hadoop wiki has a great introduction to installing this piece of software, which I wanted to do to have a play with Dumbo. The Dumbo docs also have a good getting started section which includes a few patches than need to be applied.

Dumbo can be considered to be a convenient Python API for writing MapReduce programs

Unfortunately it’s not quite that simple, at least on Ubuntu Jaunty. Hadoop now uses Java6, but if you just follow the instructions on the wikis you’ll hit a problem when you run ’‘ant package’’, namely that a third party application ...

I’m keep meaning to get around to writing about why I think the future of web developers is operations but in lieu of a proper post here’s a list of things I’ve been spending my work life getting to know this month:

• Puppet – It’s brilliant. Define (with a Ruby DSL of course) what software and services you want running on all your machines, install a daemon on each of them, and hey presto central configuration management.
• VMWare vsphere – puppet makes more sense the more boxes you have. With vsphere I can have as many boxes as ...

On the first day at Barcamp Brighton this year I did a brief talk about getting started with automating deployment. I kept it nice and simple and didn’t focus on any specific technology or tool – just the general principles and pitfalls of doing anything manually. You can see the “slides on Slideshare”:

As part of the presentation I even did a live demo and promised I’d upload the code I used. The following is an incredibly simple fabric file that has most the basic set of tasks. Fabric is a python tool similar to capistrano in Ruby. I ...

DJUGL is back, the monthly Django meetup in London. I think the last few times have been as much about useful Python stuff as just using Django, and this time it’s officially a bit more broad ranging. If you’re in or around London on the 24th September then come along.

You can get more information on Twitter or by following Rob. But expect a few short talks, some interesting conversations and maybe some beer with other like minded developers.

I’m going to be talking about automating deployment of Python web applications. If you follow me on Twitter ...

Now I’m generally an Ubuntu guy, but I’ve just had the need to setup some boxes running Solaris for Django and a handful of WSGI applications. I know my way around Ubuntu pretty well. I know all the packages I need to install and in what order. Hell, I even have all that scripted so I can just run a command and it works by magic. I’ll script the following steps if I can do when I get round to it but here, in one list, are the installation instructions for Apache, mod_wsgi, Mysql, MySQLdb, setuptools and ...

So one of the problems with using pip or easy_install as part of an automated deployment process is they rely on an internet connection. More than that, they rely on PyPi being up as it’s a centralised system, unlike all the apt package mirrors.

The best solution seems to be to host your own PyPi compliant server. Not only can you load all the third party modules you use onto it, but you could also upload any internal applications or libraries that you like. By running this on your local network you ensure your not dependent on pypi or ...