Ben Godfrey

Haskell web frameworks reinvent too much

Paul R Brown’s web application perpubplat (personal publishing platform) differs from a lot of the other Haskell web code I’ve seen — it doesn’t try to reinvent every wheel.

HAppS and Turbinado are 2 example Haskell frameworks. Both implement a web server. The HAppS project is implementing transactional ACID-compliant in-memory state, SMTP, IRC, DNS and much more. Turbinado is building a Rails-like stack.

These projects present large untested codebases. Neither project has a list of sites running their software. As a programmer considering using these tools, there’s just too much that could go wrong, too much I will have to understand and fix myself.

perpubplat is not a framework, but it’s a concise easy to understand codebase. It also contains a number of familiar looking patterns that every framework needs — logging, url mapping/routing. Most importantly, it doesn’t try to reinvent the wheel, it uses standard Haskell code where possible and serves pages to a web server via FastCGI.

Only serving responses through FastCGI makes complete sense to me. Apache, Lighty, Nginx and others are all great tools for getting static content out to the world. They are very configurable, offering flexibility for security rules, url rewriting, etc. Why try to build a product to mimic this existing code when you can piggyback on top of it through FastCGI? Instead all that programmer time can be used to build everything else required by a Haskell web application.

Wheel-reinvention (also known as Not Invented Here syndrome) makes it harder to use your code and harder to improve upon it, as parts become interdependent. Small pieces, loosely coupled means the Haskell framework conversation can evolve more quickly.

Comments

Creighton Hogg's avatar

Creighton Hogg

I can't really speak for Turbinado, but I can tell you that the current development on HAppS, reborn as Happstack, has already cut out things like the SMTP & DNS packages. We're trying to streamline it into a set of very orthogonal libraries for applications.
Cancel

Comments are closed for this post.

Daniel Patterson's avatar

Daniel Patterson

Something to check out, for which development has petered out a bit, was a novel little framework named 'hvac' - written by sterling clover. It is a lightweight not-inventing much full framework, that is used by at least me :). It also shares one of the good things I've seen in HAppS - a parse tree style url dispatcher. If I get some time, I'm going to try to revive it, but for now, check out most of what has been written about it (including original announcement) at: http://fmapfixreturn.wordpress.com/tag/hvac/ also, I wrote a tiny bit about it (when I had just started using it, and using haskell as well) at http://dbpatterson.com/articles/9
Cancel

Comments are closed for this post.

Daniel Patterson's avatar

Daniel Patterson

A little more about why I chose it (instead of HAppS) - I wanted something lightweight (my simple blog application, using ByteStrings for efficiency, still leaked memory gradually... on a small VPS, that's a problem), and I wanted to be able to pick up my data and drop it into anything - so having a common data backend would be a very nice thing (should I get sick of hvac or anything else) - sqlite was perfect for this. So I got haskell (which I wanted), a parse tree style dispatcher (which I wanted, and liked after I saw it in HAppS), but still had a very lightweight FastCGI app (I dont go over 5mb memory, and it has been running constantly, no restarts, for about 6 months now). and it is very fast.
Cancel

Comments are closed for this post.

Matthew Elder's avatar

Matthew Elder

Happstack (the successor to HAppS) is dropping the SMTP and DNS packages. Also, I think I addressed alot of the counterpoints in this article here: http://softwaresimply.blogspot.com/2009/01/happst...
Cancel

Comments are closed for this post.

Mighty Byte's avatar

Mighty Byte

Accusing HAppS of reinventing the wheel is like accusing Google of copying Altavista...true on a superficial level, but totally misses the point.
Cancel

Comments are closed for this post.

Adam's avatar

Adam

Straw-man simile alert...
Cancel

Comments are closed for this post.

Ben Godfrey http://aftnn.org

If the point that I miss is that Haskell frameworks have to invent the wheel because there aren’t alternatives, there are.

One example, for a database there are a tonne of alternatives. Plain old MySQL may not be very Haskellish, but it is very solid and proven. HAppS/Happstack’s store is a really interesting project (not unlike Erlang’s Mnesia), but it’s unproven.

The Django authors reinvented a lot of wheels when they started out, not least templating and the ORM. They (just about) got away with it because their code was proven in operation at the Lawrence Journal and the resulting framework is so pleasurable to use. The same effect could probably work for Happstack. If someone posted here saying “Actually VeryLargeCompany used Happstack to build a site that serves 200M req/day and it took them <1,000 SLOC,” I would seriously re-evaluate my assumptions.

By keeping the framework small, bits can be well understood, deployed, proven with much lower risk to the developer. I’m no big fan of Turbogears (I haven’t actually built anything with it), but perhaps their approach makes sense for Haskell frameworks.

Cancel

Comments are closed for this post.

Ben Godfrey http://aftnn.org

Firstly, Happstack is a great name. I read the interview and the intro to HAppS. Your new direction for Happstack, to simplify the framework, is a good one.

I think it would also be beneficial to concentrate on creating a simple experience for newbie developers. Many Haskell programmers (myself including) have had their appetites wetted by Haskell’s simpler features (purity, the type system, currying), but are scared of the darker magic.

Rails and Django separate concerns within an application. Url mapping/routing goes here, models go here, views/controllers go here. This does lead to editing quite a few files, but each part of the application is simpler.

Cancel

Comments are closed for this post.

Mighty Byte's avatar

Mighty Byte

No, the original argument is a straw man. It focuses on the downsides of reinventing the wheel, but ignores the fact that to do what HAppS wanted to do, the wheel had to be reinvented. And to be honest, it's not really all that big of a wheel. HAppS.Server.SimpleHTTP is 800 lines of code. Everything in HAppS.Server.* is a little over 5000 lines, and I'm sure a bit of that will be cleaned out when Happstack gets through a few more revisions.
Cancel

Comments are closed for this post.

Ben Godfrey http://aftnn.org

5,000 lines of Haskell is a big wheel. I ignored the idea that HAppS *had* to reinvent the wheel because I simply don't think it's true. I support this argument by pointing to the way Django, Rails and a whole bunch of other frameworks slot very nicely into the LAMP stack. Each of layers in that stack can be switched for your preference MySQL to Apache SimpleDB, Linux to Solaris. This gives the developer freedom and security. HVAC put it's data into SQLite, if HVAC sucks, I can serve data from that exact same DB with PHP, Python, Ruby, Perl, Java, C/C++, Scala, Groovy, Clojure, monkeys, pigeons... I can even run several at the same time. The unix philosophy: do one thing well.
Cancel

Comments are closed for this post.

Matthew Elder's avatar

Matthew Elder

This gives the developer freedom and security: Or dooms them to creating lots of meaningless boilerplate :)
Cancel

Comments are closed for this post.

Gour's avatar

Gour

Hi! Will Happstack become (more) shared-hosting-friendly or still out of reach for those not running VPS-es?
Cancel

Comments are closed for this post.

Mighty Byte's avatar

Mighty Byte

Why would HAppS use MySQL when the whole point was to avoid a relational database? Blindly sticking to the don't-reinvent-the-wheel mantra means that you'll miss out on all technology advances that require a wheel that hasn't been invented yet. If "solid" and "proven" are characteristics that are absolutely crucial to your application, then don't use HAppS. But don't assume (as is implied by the title) that anyone not fitting into your requirements is doing something wrong.
Cancel

Comments are closed for this post.

Matthew Elder's avatar

Matthew Elder

Happstack by design is not shared-host friendly. All state is kept in memory. It's meant for scalability and high availability. Even if we supported something like fcgi the memory usage would prevent you from finding cheap hosts that support it anyways. You can get a vps from linode for 20 bucks a month.
Cancel

Comments are closed for this post.

tphyahoo's avatar

tphyahoo

the OP states "Neither project has a list of sites running their software."... Well, true, but we're hoping to change things. I just launched http://patch-tag.com using this platform, including the macid-instaead-of-traditional-rdbms approach. Running on happstack was a calculated risk... sure it's less tested, but there's also the potential for it bringing major savings in the long run, both in computer resources and human time. Check it out, and wish us luck!
Cancel

Comments are closed for this post.

sclv's avatar

sclv

I tend to agree that reinventing too much at once is a problem. On the other hand, reinventing piece-by-piece is much less so. So if they happstack team can really provide simple and separate packages for the http server, with the dispatch stuff abstracted out, for the state stuff, etc., then that would be great. By the way, Artyom Shalkhakov has pulled the dispatcher stuff out of hvac into a lightweight library that can be used with any CGI or fastCGI app, which I'm very happy about. My mistake with hvac was also in a bit of "reinvention" in that I was trying to pull off a sort of transactional boosting using STM, and it turned out the ghc runtime wasn't quite up to the task yet. The other piece of hvac that I'd really like to see split out (and maybe reinvented with hlists or some simpler subset) was the parameter marshaling /validation stuff. Formlets come slightly close, but feel like they're packing too much in -- they're great if you want to define input and output at once, but feel too rigid for just parsing data out of requests in a clean, typesafe way.
Cancel

Comments are closed for this post.

Ben Godfrey http://aftnn.org

I completely agree. Network.UrlDisp looks quite interesting and I'd be interested to have a play with it in a FastCGI context. Haskell feels intuitively like it should be able to express input validators simply and concisely. Formlets looks interesting, but again syntactically somewhat scary! I guess I'm just too used to reading Python.
Cancel

Comments are closed for this post.

Alson Kemp's avatar

Alson Kemp

Turbinado isn't reinventing too much. It's more like a straight copy of RoR since RoR was my favorite framework. I did include a webserver simply because my experience with Mongrel+mod_proxy was so good and my experience with *-fast_cgi was so bad that it seemed the obvious thing to do. Other than that Turbinado tries to adopt as many technologies and best practices as it can...
Cancel

Comments are closed for this post.

Ben Godfrey http://aftnn.org

You're re-inventing every component of the framework. Taking the architecture from Rails means that is more tried and trusted, but the implementation is still new. Your blog states that building Turbinado is hard work, that doesn't reassure me that using it will be a productive process. Experimental stuff is incredibly important, but with this post I was complaining about the total lack of prospects for building production apps in Haskell. To build a production app, I need to use trusted components. That's a bootstrap process, but the size of the leap of faith is proportional the number of lines of code I need to import. That said, I really think Turbinado is an important project and I wish you every success with it. A bit of Darwinian selection will be a positive for everybody.
Cancel

Comments are closed for this post.

Alson Kemp's avatar

Alson Kemp

I guess that my original point was that I was distinguishing "re-inventing" from "copying". I'm copying many of Rails' features so that potential users are comfortable with the system, I'm focused on "inventing" more interesting parts of the system (e.g. Components (similar to Rails' Cells)) to distinguish Turbinado from Rails. As with many open-source projects, Turbinado derives from a desire to play (to learn Haskell in the case of Turbinado) and to give something back to the Haskell community. I've never claimed that it's production ready and (natch) I'd suggest using Rails for production sites. More than the hassle of copying Rails, the major problem in the development of Turbinado has been the inconsistent state of libraries and documentation in Haskell. That's getting much better with the maturing of Cabal and hopefully in a year or so, Turbinado will be ready for production-ish usage.
Cancel

Comments are closed for this post.

Add a new comment

Comments are closed for this post.