Mike West

Fallow fields, revisited

2009-10-04T16:31:23+00:00

I'm currently in the process of gutting my website, and rebuilding it piece by piece. I suspect I'm doing this to distract myself from the fact that I don’t seem to have anything interesting floating around in my head to write about. “Surely it’s the site’s fault; raze it to the ground!”, the large, simple, and shouty part of my brain tells me. So I build anew (this is possibly ironic, but I'm ignoring that).

Happily, the small, quiet, and generally reasonable portion of my brain agrees with the plan, at least insofar as it’s clear that the current system (fallow) was a solid idea but poorly implemented. The system works, and I'm happy I wrote it. It was a good introduction to Ruby and Git, and a good reason to migrate off the almost-as-inefficient-as-wordpress Textpattern. But it’s failing me in a number of ways, the most important being that I literally forgot how to get content onto the site, and it took me 45 minutes of reading through painfully structured Ruby code to figure it out again. That’s the sort of thing that happens when you don’t touch a website for 6 months.

Rather that catalog the failings of the system I'm replacing (for they are legion), I'd like to touch on the carefully considered bits I'm keeping:

URL structure: Posts live at /[year]/[month]/[URLified Title] which seems more or less perfect to me. It’s meaningful, while containing just enough temporal context to make completely outdated information easy to spot. Moreover, it provides a natural /[year]/ and /[year]/[month]/ for yearly and monthly archive pages. Tag pages live under /tags/[tag], which makes sense, and ad hoc pages have ad hoc URLs (/is/, for instance). This strikes me as a clean setup, one which I can’t see any way to improve upon.
Content storage: The site’s content consists entirely of UTF-8 encoded text files on disk. Text files are simple to work with, and have a more or less infinite shelf life. A site like this one simply doesn’t need a database, a single flat text file per piece of content is good enough. Metadata (title, tags, etc.) is contained in a YAML block at the top of each file. It’s a format that is clear, human readable, and easily parsed, and I'm especially pleased to see that the format I'd decided upon for Fallow matches up quite well against more widely used systems like Jekyll.
Static HTML: Dynamic content doesn’t really exist on this site. I write an article, then post it online. That’s the extent of the processing that particular page needs. The server shouldn’t be working to rebuild an article from last week (or last year!) every time it’s requested, that’s simply wasteful. This site, therefore, generates a page once when it’s created, or when a template changes, and then simply serves that cached copy over and over again. Similarly, overview pages (like tag pages, or current archives) are regenerated when a new article is published, then served straight from disk. On a small VPS, I can serve upwards of 300 static requests per second through Nginx with extremely low load. Textpattern would fall over and die at those absurd traffic levels.
Historical redirects: The (miserable) /blog/id/[ID] URL structure I decided upon in 2005 still works for the content I've kept from that period. The (also bad) /archives/[Title]/ structure from 2007-8 works too. The (not so lovely) Tumblr-generated links for content that used to be at blog.mikewest.org will redirect nicely. All these old URLs will continue to generate nice, clean permanent redirects for the foreseeable future: why make the reader jump through hoops created by my lack of foresight?

So, those are the good bits I'd like to keep going as I rebuild. With the understanding that I'm about to make one of those dangerous “forward looking statements” that I never seem to follow through on as cleanly as I'd like, I expect mikewest.org to be running a new Jekyll-based backend sometime in October. With luck, no one will notice a thing but me. With even more luck, I’ll squeeze out a post or two about the bits of Jekyll I'm adjusting, and the places where it’s falling down completely.

Productivity, or my lack thereof

2009-09-23T17:41:25+00:00

I just spent the last half-hour screwing around with my Vim configuration to set up a “more perfect” writing environment. Fullscreen! Eighty columns! Proper line-wrapping! Large, readable font! Markdown syntax highlighting! Spellcheck! Etcetera, etcetera! Hilariously wrongheaded, of course, since I created this environment instead of writing anything. Likewise, I've played around with todo.sh quite a bit in the last week or two (it’s nice), trying to put together a good system for recording my ever-growing list of things I need to take care of. Perhaps there’s a reason that the list is ever-growing.

Building new bash scripts that make recording tasks easier might feel like Doing Important Work, but it isn’t. Not really. This is unfortunately true even when “write new bash scripts to make recording tasks easier” is on my newly created todo list.

This is a long way of saying that Marco Arment is dead on:

The best way to increase your productivity, hack your life, and be minimalist is to stop reading those sites.

Also, hello again Internet. It’s been a while.

Also also, dear God is it a nightmare to publish anything here. Who wrote this crap?

Instapaper is Amazing

2009-03-22T20:37:07+00:00

I'm subscribed to a slowly-expanding list of something like 230 RSS feeds (I take the firehose approach to news-gathering, apparently), and for all practical purposes, my internet experience centers around NetNewsWire. Most mornings, I sit down, skim through the headlines of some subset of my feeds, and open tabs for all the the articles think I might find interesting. Tabs are NetNewsWire’s killer feature, so far as I'm concerned. I can pop open a few tabs and NNW happily keeps track of them across restarts so I don’t miss out on whatever it was that I found interesting if I don’t read the article immediately.

Unfortunately, this is sometimes as far as I get. On Friday, I had 97 tabs open; some had been sitting there since January, which is a bit absurd. These were articles that I'd almost certainly find interesting, but that I hadn’t made time to actually read. Reading ought to be the entire point of the tab-opening process, but here I was, opening tab after tab after tab, regardless of the queue of content I'd already identified. Obviously something has gone seriously wrong with my workflow.

Carlo has been singing the praises of Instapaper for a while now, this mountain of built up content seemed like a perfect test case. After digging around a bit, I came up with a script to “integrate” my Instapaper queue with NNW (NNWInstaPost), and I started pushing articles to Instapaper, and closing tabs. This worked halfway decently (I'd like a ‘to Instapaper’ toolbar button in NNW. Or a keyboard shortcut. Or anything other than clicking through the Scripts menu for each article individually). After a day or two, I've come to the conclusion that Carlo was right: Instapaper (in particular, Instapaper.app on my iPhone) is gobsmackingly brilliant.

I think I've read more articles with Instapaper in the last two days than I have in the last two weeks with NNW alone. Walking around with a queued-up list of content in my pocket is really quite wonderful. I can read on the bus. I can read while waiting for the bus. I can read while walking from the bus to the subway. I can read on the subway. Etc, etc.

I love it. I think you will too, go try it out.

Opera Web Standards Curriculum: The JavaScript Bits

2009-02-08T12:19:27+00:00

Last year, I jumped on the opportunity to sit down and write some articles for Opera’s Web Standards Curriculum. I bit off a bit more than I could chew, and Chris Mills exhibited the patience of a saint as I finished the first quickly, the second slowly, the third very slowly, and then completely failed to deal with the rest. Regardless, those were released along with the rest of the JavaScript bits to complete the curriculum.

I'm not particularly proud of my end of the process of getting these articles released, but I'm happy to see them released, and happier still to see the company they keep. Chris did a brilliant job organizing a brilliant group of authors; Opera (and Chris in particular) have organized a great body of work, which I'm proud to have had a hand in.

My articles are:

I hope you enjoy them.

Centralized Bug Tracking

2009-01-10T19:32:15+00:00

I liked many things about working at Yahoo. I'm coming to realize that what I (in hindsight) like most is probably the piece of software I thought about the least positively, namely Yahoo’s mostly centralized and completely open bug tracking system: Bugzilla. We abused it more than a bit, attempting to layer task and project management on top of a system that wasn’t really designed to support it, but all told, Bugzilla made my work life better.

As a generic employee, the centralization of bug tracking meant that I was able to quickly and easily file bugs against any Yahoo property. I didn’t have to know who was responsible for a project in order to raise bugs against it. I didn’t need the group responsible for a project to know me. When I saw an issue on a Yahoo site, I filed a bug against the project, and knew someone with the capability to fix the issue would be notified about it. Bugzilla minimized the friction caused by unclear answers to the question “I found a bug, now what?”. Instead of sending out a few emails, looking for someone to stick with a problem, it gave everyone in the company a clear “next step”, and (in the best cases) fostered a corporate culture of reporting bugs rather than avoiding them.

As a developer, Bugzilla meant that I didn’t have to keep the list of bugs on my projects. The bug database was maintained for me, triaged and prioritized by my managers, and brutally honest. Every bug that was reported against News sat in my queue, staring at me pleadingly until I fixed it. I made appropriate comments on each bug when necessary, which simple integration with CVS made trivial, with the cumulative effect that I didn’t worry about forgetting to fix something, or losing track of a bug’s status. Everything was maintained for me, removing a burden from my shoulders.

This isn’t to say Bugzilla was perfect. It was a bit of a mess, honestly, often difficult to use, full of confusing forms and confused categorizations, and plagued by an understaffed team of developers who played with the UI far too often. For these reasons and more, it probably annoyed me more than any piece of software at Yahoo, but it’s existence was hugely advantageous. In hindsight, I'm coming to consider this a critical component of any development team; a central bug tracking system provides visibility and accountability in a way difficult (impossible) to replicate with personal to-do lists and email.

Don’t read this as an endorsement of Bugzilla in particular, but as an endorsement of the concept of bug tracking. Working without a centralized bug database makes your work life more difficult for no good reason. It’s something I highly suggest that you avoid.

If you'd like to get started quickly with an externally hosted bug tracking system, I've heard good things about Lighthouse and Sifter. I'm still looking for a locally hosted system that I like, but I've been recommended Mantis, FogBugz, and, of course, Bugzilla. Honestly, even a hand-maintained text file in Tasks format that you print out and pin to the wall for people to write on is better than nothing. For the sake of your own sanity, use something.

Some Thoughts Regarding Caja

2008-12-16T20:39:27+00:00

Yesterday, Yahoo! made some announcements regarding The Future™ of many of their high profile properties. Specifically, they’re (slowly) opening up, enabling third-party developers to build applications that can be seen on and interact with your My Yahoo! page, or your mailbox. I think this is a great step, and one I wish they'd made before they laid me off.

Ah well.

One of the core technologies that’s behind this set of features is called Caja. Caja is a code sanitizer: it takes an HTML fragment, and JavaScript that operates on it, and “cajoles” it into a chunk that can be embedded into a page without the risk of maliciousness. I'd like to ramble about that, briefly, at a very high level. I'm still trying to wrap my head around it’s details

JavaScript is, simply, dangerous.

If you've paid attention to any of Doug Crockford’s presentations, you’ll know that the browser security model is simply broken-as-designed. The internet, therefore, is a place where one can barely trust first-party code, much less code written by your neighbor. You have to keep a constant eye out for new cross-site scripting vectors, and be very careful about how you filter third-party input before making it available as “user generated content.”

Seen in this light, Yahoo! has a massive problem to confront with it’s new “open” initiatives. On the one hand, they must protect the security of their sites. On the other, they want to pull in content from their users, and not just text, but code. Working applications, written outside of Yahoo!, running directly on a Yahoo! site. The project specification itself is basically a nightmare scenario for the security team. They need to find a way to include third-party JavaScript safely and sanely onto Yahoo! pages. This mechanism needs to be pretty automatic, as they can’t dedicate an engineering team to manually review (potentially) thousands of applications.

There are two broad paths to take to this end:

Code sanitization, which reads unknown code, processes it, and outputs a sandboxed version (if possible). Caja is the best known example of this tact.
Static analysis, which reads unknown code, parses it, and gives a thumbs-up if it only does known-safe things. AdSafe is a work-in-progress along these lines.

Yahoo!’s running with the former, so let’s dive in.

Code Sanitization

Untrusted JavaScript must not be allowed access to the document or window objects. This means that it must not gain direct access to any DOM node, as every DOM node enables you to crawl back up to document. event objects are right out as well, as they contain dangerous references as well. Really, when you get right down to it, you have to throw out practically everything of practical use.

I mentioned that JavaScript was broken, right?

Caja is an attempt to distill a safe subset of JavaScript out of this mess through server-side sanitization. A third-party uploads an application, consisting of JavaScript, CSS, and HTML fragments, and Caja transforms it into something that can be guaranteed not to damage the page into which it’s embedded. This is a good thing.

Caja is a capability-based system, meaning in practice that it begins by defining an extremely restrictive sandbox in which code must run, and enabling well thought-out bits of functionality by selectively injecting access as needed.

Think of it this way: when you hand your car to a valet, you would be better off if you gave them the valet key, which only enables them to drive the car. You shouldn’t give them your key, which would also let them rummage through your glove box, etc. In the same way, you shouldn’t give a program access to everything in the DOM if it’s only supposed to change a background colour in a particular location. You'd be better off if you could restrict it’s scope of access. Caja attempts to do this.

The token you give the valet, the key, can be looked at as a set of capabilities. The valet key enables “drive the car”, your key enables much more. Similarly, handing an object to a program is handing it capabilities, enabling it to act in whatever ways are exposed by that object. Since JavaScript’s default objects are so overpowered, Caja exposes a new set of objects that completely wrap things like the DOM or the event model, and rewrites input programs to use these objects instead, severely limiting the damage they can do. These wrapper objects contain a series of runtime checks to ensure that a malicious program hasn’t somehow broken out of their constraints, and the entire rewriting process fails if the program is written in such a way as to make it impossible to sandbox.

Big Drawback

Though I understand Caja’s practical necessity, I really don’t like the way it works. In short: it breaks progressive enhancement completely, and introduces a hard dependency on JavaScript for functionality.

As a quick demo, let’s look at the following code:

<script src="searchbox.js"></script>
<link rel=stylesheet href="searchbox.css" /> 
<form> 
  <input type="text" size="60" name="q"> 
  <input type="button" value="Search" onclick="doSearch(this)"> 
</form>

After “cajoling”, it will look something more like:

...
IMPORTS___.htmlEmitter___.p('form') 
    .a('onsubmit', 'return false') 
    .ih('  <input type="text" size="60" name="q">\n' 
      + '  <input type="button" value="Search"' 
      + ' onclick="return plugin_dispatchEvent___(…)">\n') 
    .e('form'); 
...

The HTML is transformed into a series of JavaScript method calls that generate HTML. This makes sense, as it enables Caja to retain complete control over what’s written to the page, but it has the side effect of making the form completely inaccessible to anyone who isn’t running JavaScript.

I'd much prefer more thought to be put into AdSafe, which sets up the same sort of wrapped-object sandbox, as well as a series of rules which third-party developers must follow. The system them simply verifies that they have done so, rather than rewriting their code to ensure that they have. If the rules are solid, the effect will be the same as can be achieved with Caja, but much more elegant, and with more respect for the fundamentals of the web.

Crockford has (finally) put up some example code on the AdSafe site. I'd suggest that you go take a look at it. It looks like a very interesting way to program indeed.

My job's value

2008-11-30T15:50:24+00:00

Recently, I wrote a short article on the effect a team’s sense of ownership in it’s projects can have on the finished product. The surprising twist in my professional life last week has led me back onto the same train of thought, but I'm coming to it from a slightly different angle. I discussed “ownership” in a narrow sense, relating to a team’s involvement with and responsibility for decisions made about a project’s vision and direction. This isn’t the only way in which the word has impact; one also has “ownership” of a project in the broader sense of pure possession.

A relatively famous study offered a representative sample of students a straight choice between a bar of chocolate and a coffee mug. Opinions were split: neither bars nor mugs were substantially preferred. If, however, a group was given one, and then asked if they'd like to trade it for the other, surprisingly few would. Even though the items were roughly equivalent in the abstract, possession (however brief) seemed to imbue them with irrationally high value. My coffee mug is much more important to me than some random bar of chocolate with which I have no relationship, even if I've only had my mug for a few minutes. Even if it’s a bit chipped and cracked, it’s mine, and that counts for something.

The economist Richard Thaler coined the term “endowment effect” to describe this phenomenon, and I'm reminded very strongly of it as I start looking around for a new job. Now that my projects at Yahoo! have been taken from me, I'm starting to wonder whether I perhaps I've been valuing them more than I rationally ought. There are a lot of good jobs out there, and I'm relatively sure of finding one quite quickly. Perhaps my job wasn’t the best for me after all. Perhaps that bar of chocolate across the room is significantly tastier than I expect. Perhaps.

This, of course, is nothing more than a thinly veiled attempt to rationalize to myself the thought that it’s really not all that bad to have my position yanked out from under me. It’s not quite working.

Yet.

Has Mike been laid off? Yes. Yes he has.

2008-11-21T17:13:13+00:00

Yahoo! has decided to close down the engineering team in it’s German office, of which I am (er… was) a part. I'm suddenly incredibly motivated to look for new work.

If you happen to know of an exciting web company somewhere in or around Munich that’s in need of a solid webdev, please drop me an email (mike@mikewest.org), ping me on twitter, take a look at my resume (or linkedin profile, if that’s your style), or give me a call ( +49 176 4854 6453 ). I'd really love to hear any and all leads you might have. :)

I ♥ GitHub

2008-11-16T16:53:09+00:00

Over the last two or three weeks, a substantial subset of my friends and colleagues have started using GitHub to host some of their personal projects. I'm really enjoying this influx, and it’s inspiring in a way I didn’t really expect. GitHub has done nothing less than to make my friend’s coding activity visible to me, and mine visible to them. This doesn’t sound like much, but it’s simply transformative; If this is how “normal” people feel about Facebook, then I can start to understand how it’s captured so much mindshare.

Coding in the Open

Visibility is inspiration and accountability. Watching talented developers Get Things Done™ around me gives me impetus to start putting something together of my own. Neil was creating a new repository every day at one point, and Norm’s potential set of projects is great to see. The whole group of Londoners are setting an example I'd like to live up to, and at the same time generating gentle social pressure for me to build something exciting of my own. Watching practically everyone I know fork Norm’s homedir immediately after he put it online is simply brilliant. I want that to happen to my projects. You couldn’t ask for better, more constructive peer pressure.

To that end, I'm putting as much of my current code as possible out into the open. I'm not generating a whole lot of code that I'd expect to be of use to anyone but myself at the moment, but even so, I work harder when I know that people I respect will be seeing what I'm producing. I'm coding for myself, but an audience changes the way I think about what I'm doing; The simple fact that my friends are following my progress on GitHub is reason enough to try to exceed their expectations.

Moreover, I've been able to convince myself that even robot’s opinions matter. Just knowing that CalendarAboutNothing is watching my every commit gives me reason to make sure that I make a little bit of time every day to sit down and write something resembling quality code. It’s a simple thing, and has been much more effective than I thought it would be over the last ~2 weeks. I highly recommend trying it out; soon you’ll be just as hooked on big red X’s as I've become.

An Admonition Regarding Details

2008-11-11T22:33:41+00:00

If Apple’s taught me anything about design, it’s that details are everything. The overall product might be brilliant, but it’s the tiny bits of perfection that really bring things together and imbue an experience with a sense of wonder and care. When I noticed that Rob Goodlatte (who has gone dark, apparently?) replaces the ampersands on his Lucida Grande dominated page with lovely, lovely Baskerville, I was thrilled. The first time I saw the little bit of bounce-back at the end of an iPhone’s scrolled list, I was hooked. These almost insignificant changes have an effect on the overall experience far out of proportion to their apparent importance.

It’s important, however, not to miss the forest for the trees. Attention to details will often make or break a project, but first laying down a solid foundation of functionality in broad strokes is critical. If you haven’t yet built a bit of your application, worrying about making it pixel-perfect cross-browser and subtly animated to amaze your users is nonsensical and counterproductive.

Put (virtual) pen to (virtual) paper, and start working. Details will fall into place naturally, either in the nooks and crannies of unconnected code you cleverly hack together to solve a problem, or in the long periods of iteration and polishing that you’ll start to go through near the middle of a project when things mostly work.

Test-Driven Development generally advocates that you should begin by ignoring (irrelevant) details and “Do the simplest thing that could possibly work.” Neil Crosby similarly says “Make it work, Make it pretty, Make it right.” My Dad (enthralled with the message while completely missing the commercial point of the Nike campaign) always told us “Just do it.” These are starting to resonate with me, and I like the idea of the development process as a continual process of iteration, building something delightful

This, of course, is a long-winded way of justifying the gaudy hack I've just put into Fallow to handle simple conditionals in templates. It’s ugly, but functional, and I know I can make it cleaner tomorrow. But right now, it works; That’s better than yesterday, and I can live with that.