Keeping “require” modules DRY in Node

Update: See the follow-up on StackOverflow. I agree with those who rallied against use of global, but no need to throw the baby out with the bathwater; as shown in that thread, we can instead use “eval” to keep it DRY.

You see a lot of NodeJS code like this:

javascript
< view plain text >
  1. connect = require 'connect'
  2. express = require 'express'
  3. redis = require 'redis'
  4. sys = require 'sys'
  5. coffee = require 'coffee-script'
  6. fs = require 'fs'

BTW this is CoffeeScript, not pure JavaScript, but the same logic applies.

The point is, Node’s module system gives you a layer of abstraction you probably don’t need. 9 times out of 10, people just assign a module to a variable by the same name. You would, wouldn’t you?!! You don’t have to think about alternatives, and you can easily cut and paste other people’s code. And it’s now just an industry convention.

So why not practice convention over configuration, and keep things DRY with the following idiom:

javascript
< view plain text >
  1. "underscore,connect,express,redis,sys,coffee-script,fs"
  2.   .split(',').forEach (lib) -> global[lib] = require lib

Thanks to the “global” object, which is like the “window” object in the browser, we can easily require new libraries without unnecessary clutter.

And – as with any decent convention-over-configuration setup – we can still override the convention if we desire:

javascript
< view plain text >
  1. _ = underscore

Announcing Faviconist

faviconist screenshot

I have made a new Favicon generator, Faviconist. Since most favicons are just a letter with some colour theme, I found it crazy to have to open up a photo editor every time I wanted a new favicon. Or alternatively, it’s easy enough to convert an image with ImageMagick, but most images don’t scale down nicely to a tiny icon! So the aim here is not an editor as such, but a true “favicon generator”.

Using Faviconist

To create a favicon, it’s as simple as typing the first letter of your website. But you probably don’t want the default colour scheme, so it’s easy enough to tweak. Under the covers, Favicon has a concept of a “FillStyle” and both foreground and background are instances of FillStyle, so they have the same flexibility. You can have a solid fill style (just a single colour), a linear gradient, or a radial gradient.

For simplicity, Linear gradients don’t have stops (it’s just a linearly increasing spectrum from start to finish), though you can choose the direction. And in the same sprit of simplicity, radial gradients may only begin at some points, and their span is fixed. The guiding principle here is “YAGNI” – You Ain’t Gonna Need It. I’m catering for the 95% of people who don’t want to micro-design their icons; the rest will still need to resort to PhotoShop et al.

The preview images are PNGs, which can easily be downloaded directly. However, another gripe of mine is the whole thing of downloading to hard drive and uploading back to a server somewhere else. Hot-linking, aka “bandwidth theft”, is rightly a problem for the original image host…but it’s a shame the only way we can deal with it is to silo everything. (A parallel vision of the web, which might have almost succeeded and still might, saw it more as embedded content than a collection of copies and links.) In this case, it’s only favicons, so I’ll just say hot-link and be merry. When you click “Save Favicon”, it saves the icons on my server and you can just cut-and-paste into your own project.

I hope this is useful for quick hacks and hackathons, where favicons are often forgotten! These days Favicons are more useful than here. Of many reasons to use a favicon, things one might forget are: they’re used on third party sites like Google Plus, tabbed browsing is super-popular these days; if you don’t provide favicons, you’ll get 404s in your log files.

And here’s a little tip for you – don’t just use letters. Try special characters, and don’t just limit yourself to ASCII. You have the entirety of Unicode at your disposal!

How Favicon Works

Buzzword Bingo time. There are a number of technologies at work here:

  • HTML5 Canvas In the past, a site like this would have to be powered by the server – every change would require a long wait for the server to generate a new image. But now, we can do it in real-time, in the browser, using Canvas. The only server interaction is conversion to “.ico”; unfortunately, Canvas will output to PNG, but not ICO. And while a JavaScript ICO encoder is certainly a possibility, no-one has yet written one.

  • Canvas Text To draw text, Faviconist uses Canvas’s fillText method. e.g. context = canvas.getContext('2d'); context.fontStyle = '25px arial'; context.fillText('Yo!'). When Ace Editor effort got started (called Bespin), in the early days of canvas, text support was patchy. But now it’s very well supported.

  • MeasureText Library I thought Faviconist would be easy because I thought there would be simple metrics for fonts. Turns out canvas’s measureText() returns a “TextMetrics” object, and guess what TextMetrics contains:

interface TextMetrics {
  readonly attribute double width;
};

Huh? We can only find the text’s width, not its height. And as a I went down the path, I learned you can’t just say “draw this text with its top-left at (0,0)”. To center the font, as we do in Faviconist, we actually need to determine text offset as well – left and top.

So I built the MeasureText library. It works as a drop-in replacement (~shim) for the built-in HTML5 Canvas MeasureText mechanism, and also provides a new convenience function not requiring an existing canvas. It works by drawing the text and scanning the resulting bitmap to get a true representation of the metrics. Using this, and with a little geometry, it’s possible to center text.

  • Favicon Library Figured it would be cool to show a preview of the favicon by changing the site’s actual favicon! For which I used ye olde Favicon library, which I originally created via this blog and I’ve now officially moved to Faviconist.com. The library also incorporates a new “badge” feature, so you can show number of pending emails, etc. Go try the demo!

  • Font Selection You can choose fonts, with the fonts coming from the Google Font Directory. It wasn’t feasible load all fonts, so they’re loaded dynamically, but I wanted to show previews … so there’s a script to build a 38,000 pixel sprite map! It’s loaded after the page loads, with a canvas-based check to see if it loaded successfully (since it doesn’t work on Firefox). To show it in a dropdown, I used a fork of the Chosen library, a fork which retains the class of each option in the select list.

  • Chrome Frame Didn’t want to support IE6-IE8, so introduced Google Chrome Frame, based on the code in HTML Boilerplate, but increasing the version check from 6 to 8.

  • JavaScript Libraries Faviconist uses several useful libraries: jQuery, Underscore, Backbone, and jQuery Tools. And just to say, jQuery Tools is the very picture of Developer Experience heaven. Great explanations, many easy-to-follow demos, lots of options…and a CDN for the libraries.

  • NodeJS, Connect, Express The server is NodeJS running the Express-Connect stack. Considering that the app is almost all client-side, I spent a ton of time getting up to speed with this stack, partly due to performance optimisatins, and partly due to the next point …

  • Jade, Stylus, Kaffeine Being a fan of DRY, I’m very happy to be using higher-level abstractions of HTML, CSS, and JavaScript. I certainly am happy with Jade, with the exception of dropping in raw HTML script snippets. And Stylus is absolutely awesome. Kaffeine makes JavaScript much easier, though I’ve found it difficult to incorporate everywhere because the ecosystem is very much geared around CoffeeScript. e.g. Connect comes with a default CS compiler, Nodemon (a node watch-and-run-tool by Remy Sharp) supports CS, there’s a ton of doco about getting CS working with things. So that was really the biggest challenge. I mainly wanted Kaffeine because I’m not such a fan of significant whitespace, but the ecosystem’s preference for CoffeeScript did present challenges.

I hope you enjoy Faviconist. It’s a work in progress, so please leave feedback if you have suggestions.

Whereupon HTML5 Doctype Messes with My Layout (and my head)

TL;DR Use “5px” in your CSS, not just “5″. It will be ignored with HTML5 doctype.

About to release my favicon tool and did the right thing with a <!doctype html>, which in Jade is simply !!! 5. I previously had no doctype.

Okay, but then I get some weird happenstance going down with the CSS. And Chrome devtools smugly crossing out style rules without telling me what’s actually the matter. (For reasons having to do with the absurd time it takes to open a UK business bank account and my general lack of enthusiasm for paperwork, I’m currently using a computer so slow it’s not economical to try this in Firebug.)

It took me a while to track down as I thought I’d made some other change in the meantime and simply didn’t put it down to impulsively changing the doctype at some random moment of inspiration.

Specifically, I found with Chrome 14 at least that CSS needs a unit of measurement. You can’t just say “5″, you have to say “5px”. I was actually saying “5″ and assuming Stylus would translate it to 5 pixels. Serves me right anyway for using absolute measurements and now I’ve updated everything to use “em” units.

I also discovered negative padding is not possible (regardless of doctype). And I agree with Aza, that sucks. There’s definitely cases where it would save work and avoid adding wrapper elements just for the sake of it.

Website Migrations: CodePlane, Nginx, Passenger

I’m just about done (famous last words) with a long, and not just a bit tedious, migration from Slicehost to Linode. Both are “cloud-style” VPSs, where you can do immediate one-click backups, cloning, upgrading, etc. As VPSs, you get to choose a raw initial Linux stack, and you’re then on your own to add on web servers, command-line tools, etc.

While SliceHost has worked out fine, notwithstanding [the 2008 exit of its founders]((http://37signals.com/founderstories/slicehost), Linode has much better rates these days and also seems to be gaining traction with the NodeJS community. Maybe it’s the fortunate name affinity. There’s also the distinct possibility Slicehost’s parent Rackspace will close down Slicehost itself and move things over to Rackspace. They also have great support docs. In any event, I’ve really been wanting to migrate from lighty to nginx and start using CodePlane for code repo, so I was going to be stuck doing some work anyway.

A few random notes …

Setup

I don’t intend this to be a review of Linode. Suffice to say, it was a super-easy process, and the Getting Started guide made everything nice and simple. Anyone coming from a similar environment like Slicehost would have no trouble. A nice bonus is the built-in monitoring charts, basically the kind of thing you’d get from a monit install, but right there on the admin panel, with no need to install anything. They’re powered by RRD.

Another nice bonus is ipv6 support. It’s been a long time coming, but I’m finally ready for the internet of IP-labelled things and beyond! 173.255.208.243? That’s not cool. You know what’s cool? 2600:3c01::f03c:91ff:fe93:5a3e/64.

Passenger on Nginx

I’d previously used Thin on Nginx, but Passenger promised to be a lot easier. Turned out maybe not, Passenger needs a custom Nginx install. I’d already set up Nginx with apt-get, and in retrospect, I should have tried to roll it back. So there was some challenges configuring things (the default Passenger-powered nginx goes into /opt/nginx, whereas the Ubuntu one is customised for the usual suspects on ubuntu, binary in /usr/sbin, conf in /etc/, and so on).

With the custom Passenger-powered Nginx, the core Nginx config needs no passenger or ruby reference. (ie You don’t do the apache/lighttpd equivalent of declaring a dynamic modules – the lack of such things is why the custom nginx install is necessary). You only need to declare passenger in the site-specific config. For a Sinatra app, with config.ru at /path/to, you do this:

server {
  passenger_enabled on;
  location /static {
  root /path/to/public;
  index index.html index.html;
}

(Yes, you never need to point directly to the config.ru or its directory.)

PHTML Suffix

I still have some weird bug preventing *.phtml files running. If no-one knows the answer, I’ll just rename it.

Mediawiki

After much messing around, I ended up with something that works for the (unendorsed, I think) “cool URI” scheme, i.e. site.com/EntryName. (versus site.com/index.php?title=EntryName).

  location / {
    rewrite ^/([a-zA-Z0-9_:]+)$ /wiki/index.php?title=$1 last;
    rewrite ^/$ /wiki/index.php?title=Main_Page last;
    root  /path/to/public;
  }

Self-explanatory I think. Along the way, I learned about nginx’s “try_files” mechanism, which fits nicely with many PHP CMSs like WordPress and Mediawiki, where a single front controller is the gateway to the entire app. You can do stuff like try_files $uri $uri/ /index.php?page=$request_uri…though I didn’t need it here.

WordPress

wordPress was similarly simple:

    if (!-e $request_filename) {
      rewrite ^/(.+)$ /index.php?p=$1 last;
    }

One quirk I discovered was I couldn’t do online changes using the SFTP facility. This happened at Slicehost too. I eventually discovered the cause. The directory needs to be owned by the webserver. The confusing thing is it looks like SFTP isn’t set up right or you have the password wrong or something. Glad to know this.

CodePlane

On a slightly separate note, I started using CodePlane as a code repo. It’s similar to GitHub’s private repo, but allows for unlimited projects. While I’d want to support GitHub, given all the value it provides, it’s not feasible to store dozens of small throwaway projects there, so CodePlane is really neat so far. And the developer is highly responsive. I noted on GetSatisfaction that the homepage doesn’t personalise if you’re logged in, he fixed it in a matter of hours. He’s also been open to engaging about some other suggestions I had. So it’s working out nicely so far.

WordPress Comments Out, Disqus Comments In

A New Comment Experience

Continuing the navel-gazing theme, I’ve finally migrated 6 glorious years of your WordPress comments to the cloud. Oh wait, they’re already in the cloud. But I’ve migrated them to someone else’s cloud. This blog being about software and the like, wanted to reflect on the motivation and process to move comments to Disqus …

I primarily decided to do this because it’s incredibly hard to “follow the conversation”, despite social media pundits telling us that’s exactly what we should be doing. Technorati is effectively no longer. Other systems which try to alert you to new links lag and miss many. Spam is not the only cause; the proliferation of URL shorteners is another factor. I’m hopeful Disqus will do a better job. So far, so good.

Furthermore, Disqus lets commenters post via their online identity – Twitter, etc – and also show posts or comments though those media. As a user commenting elsewhere, I’ve found it vastly preferable to the old school “enter your name, email, url, AGAIN”.

To be fair, I’m still not crazy on the user experience of dynamic (“Ajax”) comments, something I experimented with myself in Scrumptious. There are other comment widgets around (UserKit, Facebook, Google Friend Connect which one could assume will be replaced at some point by a G+ equivalent), but I like Disqus for its flexibility – not tied to any one user/social platform – and user experience. They’ve done a great job in both desktop and mobile browsers of making the dynamic comment experience pretty smooth.

I heard a podcast where someone was complaining about people “giving up their comments” to third parties. When challenged to say if he’s ever “used” those comments, he was silent. I don’t expect to do anything with comments other than display them. The only risk would be Disqus shutting down, and they provide an XML backup dump. I couldn’t see if anyone has actually scripted the migration back to WordPress, but we can be certain it will happen in a jiffy in the unlikely event Disqus was to shut down.

Installation

Installation was pretty simple, took around 15 minutes. I did have to sign up to Akismet and tell Disqus my key, which fine, but a surprise as I thought spam prevent was one of Disqus’s core competencies.

I also appreciate that I get a mail for each comment, which I can reply to, or reply with “Delete” if it’s unwanted. (It’s only sent me one false negative spam in 48 hours.)

The WordPress plugin is nice. It works (not a foregone conclusion in the WordPress plugin universe), it has a nice UI, and it integrates with Comments as a complete replacement.

The migration itself actually took about 12 hours and led me to be slightly concerned. We’re so used to instant gratification, but for a free service, I think this is fine. I just wish they’d made it more clear about average times to save me trawling through the forums to discover it’s normal.

For whatever reason, the resulting text was way small. So I tweaked the WordPress Cutline theme’s custom.css for a larger font:

  1. #disqus_thread { font-size: 1.4em; }

Integrated Google Plus on the Homepage

I’m getting more convinced Plus is the new Twitter, and also the new Posterous. I’ve been posting things on there I previously would have stuck on the Twitter or the Posterous, and so it was time to integrate Plus on my homepage alongside the existing Twitter and Posterous links.

Latest Post

It was pretty easy to integrate my latest Google Plus post (we don’t really have a name for a Plus post yet; a plust?), as I already have a framework in place for showing the last post from an Atom or RSS feed.

First, I found my Plus feed URL thanks to Russell Beattie’s unofficial Plus Atom Feed service:

http://plusfeed.appspot.com/106413090159067280619

Using MagpieRSS, you can easily get the last post.

  1. define('MAGPIE_CACHE_ON', false);
  2.   require_once('magpierss/rss_fetch.inc');
  3.   $feed = "http://plusfeed.appspot.com/106413090159067280619";
  4.   try {
  5.     $rss = fetch_rss($feed);
  6.     $recent_post = $rss->items[0];
  7.     $title = $recent_post[title] . " ...";
  8.     $link = "http://mahemoff.com/+", $recent_post[link];
  9.     $timeAgo = timeAgo(strtotime($recent_post[updated]));
  10.     // show the post
  11.   } catch(Exception $ex) {
  12.     // log exception
  13.   }

Me

Inside the CSS3-rendered vcard, there’s a link to my plus alongside twitter etc.:

  1. <a rel="me" class="url" href="https://plus.google.com/106413090159067280619">plus</a>

/+ …. redirect to Plus

Following Tim Bray’s suggestion, I redirected http://mahemoff.com/+ to the plus page. It’s nice to have a memorable URL.