You’ve got a great idea! You want to build a web site that saves people time and money and helps them make friends! You’ve written a rough business plan, now you want to find someone to build your site.

Software development is complicated, expensive, error-prone, regularly boring and complicated. As a programmer, here’s what I think you should know before we meet for coffee.

1. Avoid developing software

As a general rule, you should avoid doing anything you don’t need to do in a startup or new project. You have plenty of things to worry about. If you can possibly avoid writing software, do so. Need a website? Use WordPress. Intruders.tv and I Can Has Cheezburger are 2 great businesses built on WordPress. Need something like a social network? Use Ning. Prove that you can build the community, then transition to a bespoke platform to build features. Since 2004, an amazing number of high quality, free or low-cost tools have come on to the market. Tools don’t have to be perfect. Maybe half your product is already implemented by someone else’s API. Use that, build the other half. Worry about strategic risks later. You’re prototyping remember.

2. Software is complex

Software is complex and fragile. Even a simple piece of software is many times more complex than a car engine for example. Even web apps built on top of an MVC framework don’t tend to share that many common parts. Understanding an application later and modifying it is much more complex than an engine.

It’s many times easier to change an idea or a document or a diagram than to change software. Try to make changes early and avoid making them late if possible. The agile methodology, building software bit by bit in short sprints, makes it more likely that you’ll change something before it’s been written, tested and deployed.

3. Don’t make a big long list of features stretching until the end of time and space

Decide what the very core of your offering is, find out if your customers are interested and build the minimum that can solve their problem.

This approach costs less and is less risky than building your perfect dream. When you have the minimum viable product built, solicit feedback from your customers. By making the big list, you’re second guessing what they want. You’re probably wrong.

The big list is toxic. It allows you to stay busy and ignore the real world for too long. Do a month of development and put something out there.

4. Let your developers do their job

Cheaper development teams are great when you have a known problem and a known solution. When you’re launching a new product, you have neither. Find an experienced developer or team who have the experience and talent to work in an agile way, dealing with changes as they arise.

Experienced software engineers have seen a lot of applications. They’ve built a few, they use tens every day. If they say Rails isn’t right, maybe Rails isn’t right. Be flexible enough to create an environment where your team can get on with delivering a great product, and you can get on with everything else.

5. Don’t micromanage

This is kind of the same as the previous point, but it’s vitally important for software. The blue you choose or the exact positioning of a graphic might be important one day, it might not be, it certainly isn’t right now. If you’re launching a product and you start involving yourself in the minutiae of development, you will slow the process down hugely. Development takes focus, handling a micromanaging client takes time and destroys that focus, leading to mistakes and delays.

Let small decisions go. Revisit them if they really are wrong. You don’t know if your product will be a hit yet. If you need to change the core product, time spent fine tuning the details before launch is time wasted.

6. Software is only part of the puzzle

Finding money, figuring out a product and having it built is not easy, but it is well understood and, given a sensible spec, can be acheived in a finite amount of time.

Most new products don’t fail because the implementation was bad, they fail because the customers never came. Make sure the customer is there before you start. Make sure you know where they will come from, why they will use your product. Make sure you know this inside out. It’s very possible that your idea, though neat, isn’t a workable business. You really want to learn that before investing time and effort in building a product.

7. Rewriting is common

Once a piece of software has been around the block once or twice, the task it’s being used for ends up pretty far from the one is was intended for. Maintaining it becomes increasingly difficult. At this point, it’s time to consider a rewrite. This is a good thing. New tools and practices will have emerged that you can take advantage of. You know more about the business you’re in and you can cut the features you don’t need and concentrate on making the ones you do much better. Rewriting is good. Plan to rewrite. Get stuff done simply and quickly at first, revisit it later when you know more.

And finally…

There are many great books about software engineering, but I can personally recommend none more than Robert L Glass’s Facts and Fallacies of Software Engineering. It is a mine of empirical data about what makes software projects succeed and fail. If your company’s business is making software (which it is if you’re a web business), you need to read this book.

There are also a number of great websites about startups, few are finer than Eric Ries’ Startup Lessons Learned. Read it all. Twice. Carefully.

Django is an awesome framework but different projects have different needs. The last 2 projects I’ve been involved with have been using Drupal. Other projects I’m planning call for very RESTful designs. Webmachine, an Erlang framework, is a great fit for these.

I do still very much love Python and Django, perhaps even more that I’m using PHP day to day. I miss the REPL. I miss first class functions. I miss Django’s very tidy organisation of code.

Drupal is something I’ve studiously avoided for a long time, thinking it to be a Zope-like mire. That’s true to an extent: there are many versions and a lot of code. Drupal apps do have good separation of concerns. The internal organisation of modules and themes is useful, although there’s a little bit too much function name magic going on (“Why isn’t my validator firing? Who knows!”). I’m interested in hooks, although I haven’t needed them yet. The same concept has served Django hackers well.

Leaky abstractions

Django and Drupal are both leaky abstractions. It’s easy to create great big joins with Django’s ORM. Drupal generates a mammoth set of CSS and JS imports per page. Both of these can be addressed with programmer discipline, but sometimes it’s nice to have a thinner level of abstraction to make you think carefully about each requirement and how best to implement it. Webmachine is such an abstraction. What it does provide, however, is system management built on top of Erlang and OTP. People doing scale (a group I’m not a member of) find that writing the initial app is easy, scaling it is hard.

Easy hacking, easy scaling

Newer frameworks seek to make initial implementation easy and scaling easy too. Webmachine is undoubtedly somewhat harder to code for than Django, there are less batteries included, but, in theory at least, scaling is easier. Only a little of that is due to implementation, most is due to architectural style. Webmachine does get in your way a bit less when hacking RESTfully. Django does things like setting cookies on every hit, making caching harder, increasing load on your app.

I think the current leader for easy hacking, easy scaling is Google App Engine. The core of Django runs happily, the system scales to Google’s infrastructure, deployment is very simple. GAE has one flaw though, porting to another platform involves work modifying code and extracting data. While the code scales effortlessly, scaling a business around that code seems harder. If your goals don’t chime with Google’s, you’re stuck. Frameworks running on open source software stacks are more trustable and it’s for this reason that EC2 is so much more popular than GAE to date.

I’ve enabled syntax highlighting on this site using the very tidy Chili.

The standard distribution of Chili doesn’t include a Python language definition (or Erlang, or Haskell…), so I wrote one.

Download Python recipe for Python

Fabric is a very useful Python tool for scripting administration of remote servers. Like Capistrano it allows you to define tasks as a mixture of local and remote operations and then run them for lots of hosts, different groups of hosts, etc.

Increasingly I’m using configuring sshd to allow public key authentication only. Using this method makes your server more secure against increasingly common SSH brute force attacks. You can also configure an ssh-agent app to allow password-less logins.

If you want your Fabric tasks to access machines using public key authentication, add something like to your Fabfile:

from paramiko import RSAKey

config.fab_user = "jhacker"
config.fab_pkey = RSAKey.from_private_key_file("/path/to/keyfile")

Simple, and very useful.

I’m flying long haul from Bali to London. I’m returning from 3 months spent in Bali, during which I worked on and off, sometimes even for money. Long-distance telecommuting is an interesting experience. Being away from London has made me think about living and working there in a new light.

Telecommuting from a relatively well-developed place like South Bali is very easy. Free wifi is readily available and backed by “fast enough” internet connections (up to 1Mbps at time of writing). Enough for a Skype call, sometimes even video. Coffee and food are cheap enough that you can pick a café and make it your full-time office. (My thanks go to the proprietors and staff of Grocer and Grind, Sticky Fingers, Bali Buddha and Café Lazumba and Café Local.)

Contract software development lends itself well to telework. It requires some kind of rigorous technical specification to be agreed by contractor and client. The work itself involves periods of focus. This means communication is generally quite minimal. It can be concentrated into daily bursts for example, and is often better than way.

I found it difficult to work around the time difference. My UK clients would start work as my day was finishing, so the end of the day should have been set aside for communication. In reality I was often busy with other things by then, my girlfriend finished work at 5 and I got into Pilates and other pleasant evening activities. I ended up having email conversations with a 24-hour reply frequency.

I also found it difficult to sell clients my services from Bali. Work I agreed before I left went well, but people were reluctant to commit to new work without face to face meetings, though they weren’t explicit about this. After the window of remaining time shrank to less than a month, conversations degenerated to “email me when you get back” frustratingly quickly. I have a list of 14 people to email this week.

I’m looking forward to coming back to London. I’ve missed my friends, my family, the tech start-up community (and my bikes). I’m planning a busy period to make up for all the slacking I did in Bali. Amusingly enough, much of the “slacking” involved hacking away on personal stuff or projects for charity. It was great to get the mental space for things like Followize, the redesign of this site, learning more Haskell and doing some serious reading.

My busy period kicks off with a return to the tech start-up scene — attending a bunch of events to catch up with people and activity — and with lots of meetings with people I haven’t seen for a while. When I left in December, startups in London was perceived to be coping with the economicrisis better than those in some other cities. I’m intrigued to see how views have changed in the first quarter of this year, whether people are starting to feel the pinch more accutely. Many of the companies I know well are pre-revenue. Others (e.g. Huddle) offer concrete cost-savings to their customers and thus may even benefit from the downturn.

Since becoming a freelancer again I’ve been considering what I want my future role to be. I have a fair amount of web tech experience and quite a bit of startup experience (all learning from my mistakes). I like to think that my role could be as a consultant technologist attractive to some of the great companies building their products. I wonder how long I want to stick at a non-scalable profession. I took on the challenge of MyMart to chase that big pay-off that comes to the companies that win. We failed sadly, but I’m interested in buying another lottery ticket.

Rich Internet Applications (RIAs) are still increasing in popularity. We’ve reached the stage where they aren’t called RIAs any more, they’re just cool websites. (Ever heard someone call Facebook an RIA?) There is a increasing demand for tools to build great UIs easily and to build JavaScript-only applications for platforms such as OpenSocial. There are a range of toolkits for building RIAs but most of them are tied to a specific server platform.

A framework takes care of repetitive tasks. In an RIA, these include building screens, including providing rich controls, validation etc. The framework should also provide an abstraction for maintaining state and requesting data from web services via XHR. While building an OpenSocial application in December, I could have saved a lot of time developing on top of such a framework.

MVC is the dominant model for UI development in the desktop world. A modified form is very popular in web development. It makes a lot of sense to stick to MVC for RIA development. RIAs running in the browser (or Flash or AIR) bear close resemblance to desktop applications.

Of the server-and-client toolkits, Google Web Toolkit is perhaps the best known. You write Java code and it compiles it to a web application complete with HTML, JavaScript and CSS. This is an interesting idea, but the server-side component is not interchangeable and requires Java. It also leaves virtually no visibility in the compiled code.

Wouldn’t it be great if there were a client-side only JavaScript framework which allowed you to build nice UIs quickly and easily and load data from any server back-end? Given the variety of technologies for server-side web app code, you think this would be an active area. Doesn’t it slightly suck that you have an interesting web toolkit in GWT and a cool cloud hosting platform in App Engine, both from the same company, but you can’t use them together?

I was pleased to see that the cunningly-named JavaScriptMVC framework got it’s version 1.5 release. This version adds some new development tools and documentation, but it’s still back-end independent. On the other hand it does look a little heavyweight.

Topper Bowers just open-sourced his Mamoo framework. Although Mamoo doesn’t seem to be a run-of-the-mill RIA platform, it does provide some of the MVC goodness that those applications need.

I look forward to checking out both of these projects in more depth. Certainly JavaScript-only MVC frameworks are needed if smaller development operations are to produce applications with the level of quality and UI finesse seen in, for example, Facebook or GMail.

Version 2.0 of aftnn.org is here!

The new site is powered by the Byteflow with a few some custom bits. Byteflow runs on top of Django.

Some of the features I’m happiest about:

• My FriendFeed collects all the stuff I’ve been doing on the web and I re-use that feed here.
• The vast majority of old urls still work. My nginx config contains a rewrite line for each blog post. Not pretty, but fast.
• The Content and Photos sections are dynamically generated, making them easier to maintain.
• I can finally write blog posts in Markdown, not raw HTML. Byteflow actually allows me to select a text renderer per-post, so I can write one post in Markdown, the next in MediaWiki markup.
• Comments are hosted locally again, not by Intense Debate, and are protected by CAPTCHA. Alternatively, you can log in with your OpenID to comment.
• nginx serves up static content very quickly and gzips everything too.

There are always some down-sides though.

• Blanket is gone. It was a resource hog and I didn’t want to port it to Python. Sorry.
• Byteflow and Django don’t play as nicely with caches as I would like. Byteflow doesn’t provide Last-Modified, Expires or ETag headers for it’s pages. Django sets cookies immediately, instead of when a session is actually used, so it’s page are effectively uncacheable. Also, Django does not provide Content-Length headers for responses. All this degrades the performance of my site.
• The Intense Debate comment import process lost the threading. I’ll try to fix the relevant posts by hand some time.

I hope you like the new site. Please post a comment if you find a bug or have any other feedback.

WordPress is a great platform for building a simple website. It’s a featureful and flexible blogging engine and CMS. Even better WordPress.com allows you host your site for free. Reasonably priced upgrades allow you to host your site on your own domain and apply custom CSS. This all adds up to a great tool for hosting even professional sites.

WordPress.com has some restrictions compared to the version you download. Super-cheap, zero-hassle hosting is very compelling though, so it’s worth bending your requirements to fit work WordPress.com’s constraints. I’ve built a few sites this way. The latest, Yakeba, for a Balinese charity, took me just a day preparing images and writing custom CSS. The final custom CSS is not drastically more horrible than a stylesheet prepared for a site over who’s HTML you have full control.

Here are 5 techniques I found particularly useful while building Yakeba:

1. Using a static home page
2. Picking the right theme
3. CSS hackery
4. Using the classes and IDs in the theme HTML
5. Widgets

1. Static home page

The ability to create static pages and to use one as a home page is what lifts WordPress from being a blogging engine to being a simple CMS. You can use this feature even on a blog hosted on WordPress.com.

1. Create a page in the WordPress admin, title it “Home.”
2. Create a second page, call it “Blog.”
3. Go to Settings > Reading.
4. Select “A static page” for the Front page displays option.
5. Select your “Home” page in the drop down menu for the Front page option.
6. Select your “Blog” page for the Posts page option. This tells WordPress to use the url from the “Blog” page as an index of your blog posts. It will ignore the actual page content and put a list of the latest posts there.

Any pages you create will automatically appear in your theme’s navigation. If you want more control over the navigation, create a text widget and position it absolutely with CSS.

2. Pick the right theme for custom CSS

WordPress.com’s theme selection is not perfect, but there are a range of different layouts and the themes have good separation of content and presentation. The theme defines the HTML, so spend a bit of time working out which theme has the best HTML for you to start with. For Yakeba that meant the theme had to have 3 columns and navigation links.

Once you pick a theme, you can add custom CSS to either modify that theme a little, to change colours for example. Or you can get serious with the CSS magic and create a new look altogether.

3. Know your CSS

CSS has got steadily better and better since it’s introduction in 1996. Even IE 6 will let you drastically change the layout of a page, create effects like buttons and menus and much more. If you have a good command of CSS, you can create a really excellent theme without changing a site’s HTML. You’ll need to buy WordPress.com’s Custom CSS upgrade, but it’s very cheap.

Here are some examples from Yakeba:

Create navigation menus and buttons with rounded corners

Yakeba has a big donate button on the right side. If you’re sensible and are using Firefox or Safari, you’ll see nice rounded corners. It’s all CSS and degrades nicely to square borders in IE and older browsers.

Here is the CSS I used:

.bigbtn {
    text-align: center;
    padding: 8px 0;
}
.bigbtn a {
    background-color: #060;
    border: 2px outset #090;
    color: white;
    padding: 6px 14px;
    font-size: large;
    font-weight: bold;
    -moz-border-radius: 4px;
    -webkit-border-radius: 4px;
}
.bigbtn a:link, .bigbtn a:visited {
    color: white;
    text-decoration: none;
}
.bigbtn a:hover, .bigbtn a:active {
    text-decoration: underline;
}

Clickable logo with a large header background

The Yakeba header is an image applied with the CSS background-image property. The image stretches the width of the page and contains the logo on the left. The actual header in the theme HTML is the blog name. I hid the text by setting font-size:0; and setting the same colour as the image background. I then hard-coded the sized of the <a> tag linking to the homepage (#header p a) so that it covered the area of the header background containing the Yakeba logo. Bingo, clickable logo.

4. Learn to love WordPress.com’s HTML

The HTML generated by WordPress.com’s limited selection of themes can be frustrating at times. You’re forced to find a “good enough” fit and then work around it. However, there are a broad range of IDs, classes and structures which can use to your benefit if you are canny with your CSS selectors. Some examples that I employed when building Yakeba:

Main content and sidebars

The Fadtastic theme defines a main content area in #content (wrapped by #content_wrapper) and 2 side bars, #side_one and #side_two. These are the main areas of content in the page and the elements you should lay out. I built a simple float layout for the columns and then created vertical separators using a vertically repeating background-image. This is a nice trick if you’re happy with a fixed-width layout. The required CSS is simple enough for mere mortals and IE 6.

Hiding elements to remove content

I hid the footer (mainly because it attributes the theme to someone else!) and several areas of the navigation. Hide elements with display:none; On the homepage I even hid the page title (#post-24 h1 { display:none; }) because I wanted the navigation link text to be “Home” but for the page to have a single paragrah overview in large type. All pages, blog posts and widgets have IDs like this which you can use to apply CSS properties to specific parts of the page.

Lists within lists within widgets

The Fadtastic theme creates widgets as lists and then creates content within those widgets as more lists. Selectors like .side ul and h3.widgettitle, li.linkcat h3 allowed me to grab those lists by the horns and pull them into shape.

5. Widgets are wonderful

Another feature that sets WordPress apart is widgets. Widgets are just little blobs of content that you can add to your theme’s sidebars. Widgets include recent posts, RSS feeds, content from a select few other sites (e.g. Flickr), and, most usefully, boxes of custom HTML.

Text boxes

You can add as many boxes of text (actually HTML) as you need and optionally give each one a title. This allows you to create bits of static content that appear on all pages. Because the boxes can contain HTML, you can create any kind of content and then style it with custom CSS. You can also embed widgets from other sites, which gives you quite a few options for customisations.

RSS feed magic

I wanted to display latest posts in the central column of Yakeba’s pages. WordPress.com doesn’t actually have a widget to do that, but it does have an RSS widget which will display a snippet of each item. Adding an RSS widget and setting the feed url to the feed of the site’s blog gave me enough of what I wanted.

Conclusion

This wasn’t meant to be an exhaustive list of WordPress.com tricks. Their forums have a lot more info. I hope, however, I’ve given you a sense of how much can be done with WordPress.com’s features if you set your mind to it.

REST provides a set of architectural constraints that, when applied as a whole, emphasizes scalability of component interactions, generality of interfaces, independent deployment of components, and intermediary components to reduce interaction latency, enforce security, and encapsulate legacy systems.

Roy Fielding’s doctoral dissertation surveys architectural styles for building networked hypermedia systems and introduces the REST architectural style. Fielding was one of the original architects of the web and his dissertation attempts to describe “the model for how the modern Web should work”.

Designing web applications is a complex task. Each application has a number of requirements and there is a huge range of possible approaches and toolkits available to help software engineers meet those requirements.

Fielding surveys architectural styles, a very high level of abstraction describing the bits of a running networked system and how they fit together. He then describes REST, a style designed to maximise the required properties of a networked hypermedia system (i.e. a world-wide web). It’s almost like a memoir of the construction of the web, an explanation of rationale.

The REST style is incredibly elegant. It simplifies so much that it almost seems invisible.

• Resources are named by urls.
• The big 4 HTTP methods GET, POST, PUT and DELETE are the only operations used on resources.
• Systems are built as clients, servers and connectors (proxies, caches, etc) speaking a common protocol, HTTP.

The properties introduced by these constraints are:

• Performance, both absolute and user-perceived.
• Scalability.
• Simplicity.
• Modifiability.
• Visibility.
• Portability.
• Reliability.

These are obviously very nice properties for a system to have. Visibility is especially interesting and subtle. Use of the HTTP protocol’s simple verbs and request and response structures allows the effect of individual requests to be well-understood and verifiable and intermediate tools (such as caches, proxies or transforming middleware) to be constructed easily.

Writing to your reading list

If you are developing web applications, I recommend Fielding’s dissertation as a summary of styles and as a description of the REST style.

Architectural Styles and the Design of Network-based Software Architectures

I also found the Wikipedia entry on REST very useful, particularly the grid of HTTP methods and their effects on various types of resource.

Stefan Tilkov has a nice article which lists some REST anti-patterns. It defines REST by subtracting what is not REST. An informative approach.

Finally, you may also be interested in Fielding’s presentation at Railsconf Europe last year, entitled “The rest of REST.”

Paul R Brown’s web application perpubplat (personal publishing platform) differs from a lot of the other Haskell web code I’ve seen — it doesn’t try to reinvent every wheel.

HAppS and Turbinado are 2 example Haskell frameworks. Both implement a web server. The HAppS project is implementing transactional ACID-compliant in-memory state, SMTP, IRC, DNS and much more. Turbinado is building a Rails-like stack.

These projects present large untested codebases. Neither project has a list of sites running their software. As a programmer considering using these tools, there’s just too much that could go wrong, too much I will have to understand and fix myself.

perpubplat is not a framework, but it’s a concise easy to understand codebase. It also contains a number of familiar looking patterns that every framework needs — logging, url mapping/routing. Most importantly, it doesn’t try to reinvent the wheel, it uses standard Haskell code where possible and serves pages to a web server via FastCGI.

Only serving responses through FastCGI makes complete sense to me. Apache, Lighty, Nginx and others are all great tools for getting static content out to the world. They are very configurable, offering flexibility for security rules, url rewriting, etc. Why try to build a product to mimic this existing code when you can piggyback on top of it through FastCGI? Instead all that programmer time can be used to build everything else required by a Haskell web application.

Wheel-reinvention (also known as Not Invented Here syndrome) makes it harder to use your code and harder to improve upon it, as parts become interdependent. Small pieces, loosely coupled means the Haskell framework conversation can evolve more quickly.