Migrating user accounts from Google OpenID to Google OAuth to Google Plus

Background

Over here, Ade has asked me to permalink a comment I made about moving Player FM accounts from Google Open ID to OAuth. The reason I did it was because Android sign-in is really based on OAuth, but what I didn’t know was Google at the time was preparing to launch “Sign in with Google Plus”, also based on OAuth. Bottom line: Google Open ID, afaict, is going the way of the dodo, so any services using it probably want to migrate their users to G+. (Googlers may please correct me if I’m wrong about Open ID’s demise.)

There were many permutations to be considered for this kind of migration, each with trade-offs to developer complexity and user experience. What follows was the optimum balance for me after a repetition of thought experiments, in between wishing I had a pony (ie that I’d opted for OAuth in the first place, the only reason I didn’t was availability of an Open ID Rails plugin). This is all web-based as we (thankfully) hadn’t launched on Android yet.

The problem

The first concern here is largely about user perceptions. To us developers, Google OAuth and Google Open ID are distinct login mechanisms, as similar to each other as they are to Facebook or Twitter. But to the user, they’re the same thing – Googles all the way down. So you can’t present the user with separate login buttons for Google OAuth and Google Open ID…you only get to present one Big G button.

The other concern is that sites who present “Existing users – log in here” versus “New users – sign up here” buttons … are doing it wrong. A major benefit of third-party sign-in is you can just present a “Connect with X” button and the user doesn’t have to care or remember if they previously connected with X or not. Don’t make me think!

Put concern A together with concern B and Houston, we have a problem. You present that one big G button with Google OAuth and what happens if the user is unrecognised? Is this a new user or someone who had previously logged in using Open ID. (It’s fine if the user is recognised, that means they’re one of the post-OAuth-era people.)

The solution

The solution depends if you’re willing to ask for email permissions on the new oAuth flow.

If you are willing to ask for email, that will make it easy to link the two accounts, because Open ID already relies on email, so you have their email. You can just switch right now to oAuth and once the user authenticates, link the account with the account having the same o8 ID. (Note: this scenario is purely speculative and not based on my experience.)

Since I chose not to ask for email, I had to do the uncool thing and temporarily divided Login from Signup.

In the Login area, the app prompted users for their email or login, and immediately made an XHR call to detect whether that account is using o8 or oAuth, then showed the corresponding button (the button is just a Google button, looks the same either way but the link will be different for o8 vs oAuth). (In addition, the Twitter and classic login form were shown.)

For people who logged in with Open ID, I built an !IMPORTANT! big red notification when the user logged in via Open ID, telling them we’ve updated Google login procedure, and when they click to set it up, taking them through the oAuth flow with a special callback for the migration. At this point, the server recognises the two accounts are linked (they’d already logged in with Open ID, now they’ve just logged in with OAuth), so we can save the user’s OAuth credentials. This user now has two third-party accounts – Google Open ID and Google OAuth. Just as they might also have a Facebook account and a Twitter account.

The Signup area of course only contained an OAuth button (as well as Twitter, which was exactly the same as Twitter in the login area, and classic signup form).

I published advance notice on the blog I would be shutting down the old Google IDs, kept the migration alive for two months, and then deleted all the Open ID accounts at that time. Anyone who didn’t log in at the time lost their accounts, but of course a few people mailed me about it (mainly when we launched the Android app and they tried to log in again), so I helped them migrate manually.

I did this for a couple months before deprecating o8 and returning to the nicer single Google button setup. And just manually merged the accounts when a few users asked me why the Google button is not getting them back to their old account.?

Epilogue

It was well worth the pain, as the vast majority of Android users now choose to log in with Google, even though we also support classic login and guest mode. The G+ API was a big bonus to come out of it. I’ve done some experiments on the social side and expect to do much more with G+ accounts in the future, to help people discover new shows to listen to.

Why Google killed off Google Reader: It was self-defense (GigaOM guest post)

Guest-posted this on GigaOM today.

Backstory is I started writing it on Thursday night after seeing all the Reader tweetstorm and figured it’s probably of more general interest, so I submitted it there. The original draft was ~1400 words and I wasn’t sure how seriously they take their guideline fo ~800, so just left it at 1400, but turns out they are, in fact, serious. So we edited it down.

For the record (since some people asked), I used Bloglines for as long as I could cope with its downtime, as I always found Google Reader too magic (unpredictable) with its use of Ajax. Eventually Bloglines was outaging for hours and IIRC whole days, so I made the switch to Reader, but could never get into the web app – too much Ajax magic – and instead used Reeder, sync’d to Reader when it came along. When I switched to Android for my primary device, I couldn’t find a satisfactory app, so just used Reeder on the iPad occasionally.

Meanwhile, with podcasts, I preferred the cloud approach of Odeo and Podnova, but both sadly died. I tried podcasts with Reader, but it just wasn’t the right experience so I mostly used iTunes, and then on Android, mixed it up between several apps (DoggCatcher, BeyondPod, PocketCasts, etc…the usual suspects) until eventually creating my own (still in beta). I really had problems with Listen though, so again, no didn’t do the Reader sync.

So bottom line is I did use Reader “somewhat”, but mostly as an API; and it’s no great loss to me like I appreciate it is to others. The responses to this article certainly demonstrate how passionate people are about a product they get to know and love, and use on a daily basis. It’s never easy giving up on muscle memory. The bright side of the equation is exactly what people like about it: RSS and OPML are open, so at least people can move on to Feedly, Newsblur, and so on. And I truly believe this decision ultimately liberates the standard and allows it to thrive among smaller players.

About WebWait and Caching

I received today a question often asked about WebWait, so I’ll answer it here for reference.

WebWait User asks:

I have been using webwait for a while and have a quick question for you. When running multiple calls on the same website, is each call downloading the entire page again, or is the information being loaded from the browser cache?

My answer:

It will do whatever the browser would do if the page was loaded normally, so that would usually mean the 2nd-Nth time it will download from the cache. To counter-act that, you can simply disable your browser cache while performing your tests. Or if you do want to test cache performance, just open your site once (either in the browser or WebWait) and then start the WebWait tests, obviously keeping the cache enabled throughout.

Discovering Users’ Social Path with the New Google+ API

Google announced a slew of identity and social updates today, most excitingly, the ability to browse users’ social paths. This happens after similar services recently blocking some folks from doing so, which tells you Google gave it due consideration and is committed to supporting this feature indefinitely.

Here’s how the authentication looks:

Now there’s a whole set of widgets and JavaScript APIs, but I was interested in the regular scenario for apps already using the “traditional” OAuth 2 dance. After asking on the G+ APIs community, I was able to get this running and I’ll explain how below.

Step 1. Visit the API doc: https://developers.google.com/+/api/latest/people/list

Step 2. Scroll to the interactive part below and turn on OAuth 2.0 on the top-right switch.

Step 3. To the default scope, add a new one: https://www.googleapis.com/auth/plus.login. That’s the magic scope that lets your app pull in social graphs.

Step 4. For userID, enter “me”. For collection, enter “visible”. (This collection property, representing circles/people the user can identify to the app, only has that one value at present.)

Step 5. Now hit execute and (as a test user) you’ll see the dialog shown at the top of this article. Then hit accept.

Step 6. I got a confirmation dialog saying “Clicking Confirm will let Google APIs Explorer know who is in your circles (but not the circle names). This includes some circles that are not public on your profile.” which is surprising as I believe circles are always private (for now), so I guess users will always see that. Accept it.

Step 7. The JSON response will now be shown below the form. It includes a top-level field called “items”, which is the list of your (the authenticated user’s) G+ people. If the list is too long, there will also be a “nextPageToken” field so the app can page through the list.

So that’s an overview of the new G+ social API. It’s a straightforward OAuth implementation and should be easy for anyone with a Google login to adopt. I’ve been looking forward to adding this functionality on Player FM so people can see what their friends are listening to … I think it’s a nice model where users can choose how much of their social graph they share with any app.

Phablet-Only

Mid-Range Tablets: Their Time Has Come

I was initially skeptical about mid-range tablets. I figured I have my phone in my pocket and my big-ass tablet in my living room; why would I need something in the middle. Niche at best, right?

I was wrong.

I got interested in this form factor when I saw my G+ stream starting to light up with rave reviews about the Nexus 7. Biased crowd, admittedly, but they weren’t the usual fanboys/girls. After Google dropped the price for the holidays, it was too much too ignore. I yoinked a 16GB Nexus for £159 and I haven’t looked back. It’s one of the best devices I’ve owned, up there with iPod, iPhone, and iPad for sheer delight of getting to know it. I find myself reaching for the Nexus even when the iPad 2 is nearby. It’s lighter and less bulky to hold, and works fine for any kind of surfing and reading. (And it doesn’t hurt that it’s running what has become a fantastic OS.) Only video suffers from the size, though the trade-off still makes it worth it for video sometimes, especially in environments like public transport.

The little secret about this form factor, now revealed to the masses, is that it fits fairly comfortably in most adult jeans. Not to mention handbags and glove boxes. I did have a crappy knock-off 7″ model from early 2011, and it was simply too fat to fit comfortably in the pocket. But – thanks mostly to battery improvements, apparently – the Nexus 7 and iPad Mini are way thinner, and that really makes the difference. Furthermore, the grippy backplate of the Nexus 7 is genius, one of those “little things” that makes a huge difference and elevates the form factor overall.

The New Must-Carry Device

Given that (a) mid-range tablets are the sweet spot for many interactions, and (b) they are feasible to carry around, the natural deduction is I want to carry them around with me. Even more so as I’m often using it to listen to podcasts or watch videos prior to stepping out and want to continue that experience without switching over to the phone.

The mid-range tablet has begun to occupy the special place traditionally occupied by the phone: A personal device, always carried. Not a shared device like the iPads of yore, but a device as personal and omnipresent as the smartphone. So I’m often carrying two devices now; the convergence trend has been reversed and after shedding the dedicated MP3 player and camera, suddenly my gadget count has doubled. A smartphone and a tablet, both in my pockets? Don’t want.

Why even bother carrying a smartphone anymore? What does it offer that the mid-range tablet doesn’t? Well, two things for now: Bandwidth and actual phone services.

Bandwidth is mostly what I still need the smartphone for. The S3 has increasingly a dumb appendage which sits in my pocket and is only there to provide tethering support for the Nexus. Well, that happens as long as I have a wifi-only model, but if I could choose again, I’d splurge on a 4G model.

That leaves only one thing the smartphone is good for: the “phone” bit. And that’s why I’m talking about Phablets here. Modern tablets do in fact allow for phone services. We can use VOIP solutions like Facetime, Google Talk, and Skype, all with plenty of options for buying traditional phone numbers and interfacing with the regular network (including SMS and voicemail). Furthermore, it should be possible to access the radio and make actual phone calls using the standard dialling app (requires a rooted Android device for now, but it’s possible). One could easily make regular phone calls with a headset, bluetooth or wired, or speakerphone, or – yes – the comedy scenario of just holding the damn thing to your face for a few minutes.

Having explained why those two barriers are surmountable, I believe Phablet-Only is possible and something I want to do. I think we’ll see a little Phablet-Only trend gain momentum in the next year.

One Size Fits No-one

Phablet-Only is not for everyone. I know. Not everyone wants to interact with their phone using a headset. It’s super-convenient to just hit Call and hold the phone against your face. Others might object to the one-hand experience; if you’re standing on a crowded train every day, you probably want to hold a device where your thumb can reach every point on the screen. And the size itself, of course. If you don’t have big enough pockets and don’t want to carry a bag around, you can’t do this.

The real point is that everyone will have a range of options available. It’s likely that we’ll converge to one personal device, because most people will be too inconvenienced by keeping multiple devices with them at all time. Even with cloud syncing, you still have to install apps twice, set up your homescreen again, etc. Only a revolution in wearable devices, like Google Glass, will bring about more than one device. So assuming for now, we have only one device, what will that device be? In 2009, we could confidently say it will be an iPhone or similar form factor. But for 2013, I believe we won’t be able to say much at all as it could be anything from 3″ to 8″. For Phablet-Only users, they might still keep their phone, but it would switch to a secondary device for occasions where a tablet isn’t practical (e.g., the running/clubbing/gyming scenario). The implications for developers are obvious: Get Responsive! And I mean this in the broader sense: Native apps must be responsive too and designers must consider how different form factors affect different usage patterns.

New Look, New Host

This blog is now hosted on WPEngine. I was having trouble managing it on the Linode VPS for some time now. It seemed to cause DB issues for some reason, which would in turn lock up my other sites (WebWait, AjaxPatterns etc). So I had to isolate it on a separate Linode, and decided if I’m doing that, I may as well just go for a dedicated WordPress host. So here we are at WPEngine. And took the opportunity to cut the clutter and go for a minimalist theme. So thanks Sayanee for this here IceCap theme. Update: Or not. Having some scrolling issues, oddly enough, so reverting back to the old theme for now. Update again: Fixed. It was a conflict with the MailChimp Social plugin. Luckily, Social has an option to disable its own comment view, so I can keep both plugins active.

Mentoring and Judging

I’ve been blogging about some of my own projects lately, but wanted to capture/announce/disclose/link a few other things I’m please to be involved in:

  • I’m now a technical advisor to MinuteBox. I’ve been singing these guys praise since I discovered the product at SeedCamp last year. The concept is so damn useful and the founders are capable of delivering on it, so I’m proud to be associated with it.
  • Judged for Node Knockout. Did so in its inaugural year in 2010, and again this year, now from the perspective of someone who’s Noding daily. My votes.
  • Judged for HTML5 Open Call, for GDD Russia. HTML5 Canvas and SVG galore! Actually, it was great to see an upswing in SVG use…really, it has a lot of benefits over canvas in some cases, and it seems people are beginning to see it.
  • Mentoring at SeedCamp next week, as I did last year. Looking forward to seeing what’s new, and I’m pleased to have made lasting connections at the last event (MinuteBox being one of them).
  • Mentoring at GammaRebels next week (remotely). Looking forward to this event too. Polish startup community is thriving right now.

Website Migrations: CodePlane, Nginx, Passenger

I’m just about done (famous last words) with a long, and not just a bit tedious, migration from Slicehost to Linode. Both are “cloud-style” VPSs, where you can do immediate one-click backups, cloning, upgrading, etc. As VPSs, you get to choose a raw initial Linux stack, and you’re then on your own to add on web servers, command-line tools, etc.

While SliceHost has worked out fine, notwithstanding [the 2008 exit of its founders]((http://37signals.com/founderstories/slicehost), Linode has much better rates these days and also seems to be gaining traction with the NodeJS community. Maybe it’s the fortunate name affinity. There’s also the distinct possibility Slicehost’s parent Rackspace will close down Slicehost itself and move things over to Rackspace. They also have great support docs. In any event, I’ve really been wanting to migrate from lighty to nginx and start using CodePlane for code repo, so I was going to be stuck doing some work anyway.

A few random notes …

Setup

I don’t intend this to be a review of Linode. Suffice to say, it was a super-easy process, and the Getting Started guide made everything nice and simple. Anyone coming from a similar environment like Slicehost would have no trouble. A nice bonus is the built-in monitoring charts, basically the kind of thing you’d get from a monit install, but right there on the admin panel, with no need to install anything. They’re powered by RRD.

Another nice bonus is ipv6 support. It’s been a long time coming, but I’m finally ready for the internet of IP-labelled things and beyond! 173.255.208.243? That’s not cool. You know what’s cool? 2600:3c01::f03c:91ff:fe93:5a3e/64.

Passenger on Nginx

I’d previously used Thin on Nginx, but Passenger promised to be a lot easier. Turned out maybe not, Passenger needs a custom Nginx install. I’d already set up Nginx with apt-get, and in retrospect, I should have tried to roll it back. So there was some challenges configuring things (the default Passenger-powered nginx goes into /opt/nginx, whereas the Ubuntu one is customised for the usual suspects on ubuntu, binary in /usr/sbin, conf in /etc/, and so on).

With the custom Passenger-powered Nginx, the core Nginx config needs no passenger or ruby reference. (ie You don’t do the apache/lighttpd equivalent of declaring a dynamic modules – the lack of such things is why the custom nginx install is necessary). You only need to declare passenger in the site-specific config. For a Sinatra app, with config.ru at /path/to, you do this:

server {
  passenger_enabled on;
  location /static {
  root /path/to/public;
  index index.html index.html;
}

(Yes, you never need to point directly to the config.ru or its directory.)

PHTML Suffix

I still have some weird bug preventing *.phtml files running. If no-one knows the answer, I’ll just rename it.

Mediawiki

After much messing around, I ended up with something that works for the (unendorsed, I think) “cool URI” scheme, i.e. site.com/EntryName. (versus site.com/index.php?title=EntryName).

  location / {
    rewrite ^/([a-zA-Z0-9_:]+)$ /wiki/index.php?title=$1 last;
    rewrite ^/$ /wiki/index.php?title=Main_Page last;
    root  /path/to/public;
  }

Self-explanatory I think. Along the way, I learned about nginx’s “try_files” mechanism, which fits nicely with many PHP CMSs like WordPress and Mediawiki, where a single front controller is the gateway to the entire app. You can do stuff like try_files $uri $uri/ /index.php?page=$request_uri…though I didn’t need it here.

WordPress

wordPress was similarly simple:

    if (!-e $request_filename) {
      rewrite ^/(.+)$ /index.php?p=$1 last;
    }

One quirk I discovered was I couldn’t do online changes using the SFTP facility. This happened at Slicehost too. I eventually discovered the cause. The directory needs to be owned by the webserver. The confusing thing is it looks like SFTP isn’t set up right or you have the password wrong or something. Glad to know this.

CodePlane

On a slightly separate note, I started using CodePlane as a code repo. It’s similar to GitHub’s private repo, but allows for unlimited projects. While I’d want to support GitHub, given all the value it provides, it’s not feasible to store dozens of small throwaway projects there, so CodePlane is really neat so far. And the developer is highly responsive. I noted on GetSatisfaction that the homepage doesn’t personalise if you’re logged in, he fixed it in a matter of hours. He’s also been open to engaging about some other suggestions I had. So it’s working out nicely so far.

WordPress Comments Out, Disqus Comments In

A New Comment Experience

Continuing the navel-gazing theme, I’ve finally migrated 6 glorious years of your WordPress comments to the cloud. Oh wait, they’re already in the cloud. But I’ve migrated them to someone else’s cloud. This blog being about software and the like, wanted to reflect on the motivation and process to move comments to Disqus …

I primarily decided to do this because it’s incredibly hard to “follow the conversation”, despite social media pundits telling us that’s exactly what we should be doing. Technorati is effectively no longer. Other systems which try to alert you to new links lag and miss many. Spam is not the only cause; the proliferation of URL shorteners is another factor. I’m hopeful Disqus will do a better job. So far, so good.

Furthermore, Disqus lets commenters post via their online identity – Twitter, etc – and also show posts or comments though those media. As a user commenting elsewhere, I’ve found it vastly preferable to the old school “enter your name, email, url, AGAIN”.

To be fair, I’m still not crazy on the user experience of dynamic (“Ajax”) comments, something I experimented with myself in Scrumptious. There are other comment widgets around (UserKit, Facebook, Google Friend Connect which one could assume will be replaced at some point by a G+ equivalent), but I like Disqus for its flexibility – not tied to any one user/social platform – and user experience. They’ve done a great job in both desktop and mobile browsers of making the dynamic comment experience pretty smooth.

I heard a podcast where someone was complaining about people “giving up their comments” to third parties. When challenged to say if he’s ever “used” those comments, he was silent. I don’t expect to do anything with comments other than display them. The only risk would be Disqus shutting down, and they provide an XML backup dump. I couldn’t see if anyone has actually scripted the migration back to WordPress, but we can be certain it will happen in a jiffy in the unlikely event Disqus was to shut down.

Installation

Installation was pretty simple, took around 15 minutes. I did have to sign up to Akismet and tell Disqus my key, which fine, but a surprise as I thought spam prevent was one of Disqus’s core competencies.

I also appreciate that I get a mail for each comment, which I can reply to, or reply with “Delete” if it’s unwanted. (It’s only sent me one false negative spam in 48 hours.)

The WordPress plugin is nice. It works (not a foregone conclusion in the WordPress plugin universe), it has a nice UI, and it integrates with Comments as a complete replacement.

The migration itself actually took about 12 hours and led me to be slightly concerned. We’re so used to instant gratification, but for a free service, I think this is fine. I just wish they’d made it more clear about average times to save me trawling through the forums to discover it’s normal.

For whatever reason, the resulting text was way small. So I tweaked the WordPress Cutline theme’s custom.css for a larger font:

  1. #disqus_thread { font-size: 1.4em; }

Best Practices for Mailing List Unsubscription

Having spent the past too many hours gearing up for inbox zero or some such, I’m pleased to say almost every mailing list now includes an “unsubscribe” links (even those which I never signed up to! Which I avoided clicking on as they are probably a dodgy way to get you somewhere else). I assume this is some requirement of Spamhaus etc which the lists must follow to avoid falling in the spam folder, which affects their spam status in GMail etc. Also, if lists make it hard to unsubscribe, users will mark them as spam, which is also detrimental to their status on GMail etc.

After hitting “unsubscribe” many times, here’s what works well (of course, this is just from the perspective of an interested user):

  • While most let me unsubscribe immediately, some require login. This is a bad move as I assume many users simply won’t bother, especially if it’s some list you signed up to years ago. Interestingly, those which unsubscribe immediately fall into different categories – some are off-site, so you can only unsubscribe. Others are onsite and once you unsubscribe, you’re also logged in, which is possibly a security risk (maybe a reasonable one, but I don’t think they’ve always considered this happens). In a few cases (e.g. LinkedIn I think), you can unsubscribe, but then you have to log in after that. A fine example of the awesome Amazon-style “semi-logged in” state.
  • A good practice is to provide settings for what you want to receive. But not too many options! LinkedIn is particularly verbose with its new group feature – turns out each group has its own mail settings – a handful of checkboxes for every group. Just say no!
  • Some have a delay of up to three days, which is not just an annoyance, but breaks the UX principle of immediate feedback. You unsubscribe, then you see a new mail a few hours later and think maybe you didn’t unsubscribe after all.
  • Automatic Unsubscribe is best, but as with all “shoot first, ask later” style interfaces (e.g. Auto-Save), you also want to provide an “Undo” facility. One list did that, with a “Did you really mean to Unsubscribe? Resubscribe“. Smart.
  • Some said “We’re sorry to see you go” etc, but a smarter thing I saw was “You can still keep in touch with us on Twitter and Facebook“.