Artificial Horizon

Articles and links about the design process
by Thibaut Sailly


January 7, 2010

✍ updated at the end, January 21st

As anyone who's interested in Apple products and interface design, I came across a lot of views, wishes and predictions about the new forthcoming Apple product known for now as the tablet.
But I didn't see any mention of handwriting recognition as a feature, even to argue why it wouldn't be an option. It's just as if the "who wants a stylus ?" from the iPhone intro had wiped the idea all along. For a device that has been designed to be operated with a single hand, it makes perfect sense. For a bigger one you'd need both hands: one to hold it, one to interact with the touch screen. The idea of a stylus as an input device (along with the multitouch input) does seem less out of context, as far as ergonomics go.

I got the Newton out for a run the other day, and played a bit with it. How slow this thing is for today's standards... When I recalled this technology is 15 years old, I began to think that we could have today the technology allowing much faster and more reliable handwriting recognition. So let's pretend.

Let's pretend there's this magical power inside the tablet, that this machine understands everything you write. It doesn't visually convert your writing into type as the Newton almost did ; you just write as on paper, and it knows what's written. It can store and communicate your scribbles as they are, and it can produce an indexable version of it instantly, so you could paste it into some other document. What would be the benefits of entering text by handwriting ?

The first one which comes as a definite added value to me is formatting personal emails. Handwritten letters have so much more emotional impact than our helvetica emails. Have a look at Jeff Bridge's website and realize how much you get from his personality because it's handwritten. The variations of text you can use to express emotions are way faster to format by hand than by a text editor. Who needs emoticons with handwriting ?
More expression and more personality in your family and friends emails, that's one.

Second, from a usability perspective : words as the interface. Handwriting controls context, data input and data flow. Let's say you're reading a nice article and you want to tweet about it. Get the stylus to write "twitter : what a good read + link" and draw a right arrow to send. In a single sentence you call an application, assign data to it, and act on it. This single sentence would launch a twitter client, fill in a tweet, add a link and post it, but it would be invisible to you (see the illustrated sequence below). Interface minimalism at its best. Your handwriting would overlay the context, and would be instantly understood as content, destination and action. Compare this to what happens when you want to do the same thing with an iPhone. It would be way faster than having to switch apps, add content to the right place with the virtual keyboard and come back to your article. Other situations I can think of are building lists, sending short emails, messaging and search. It covers a lot.
Small recursive context independent tasks made easier, that's second.

Another place to have such a handy input and magical power is at work - or any situation where you take notes and draw on paper really. It could also be school. My notepad pages have a lot of different informations on a single page : todo items, phone numbers, website layout sketches, notes taken during discussions. It would be practical to have this magical power know where all these informations belong and selectively propose actions to assign to them. You could call an actions menu around a name and a phone number with a gesture or a stroke, and it would suggest to add it to your contacts. You could select a note and write "email Georges" and it would send it by email to Georges, in its graphic version and/or translated as type.

It's often seen as more convenient to enter text with a keyboard ; composing texts like this one for example. But this input method also forces us into interaction models that feel more "administrative" than the natural flow of a sheet of paper. The keyboard has his history, dating back to the Renaissance. In Robert Bringhurst's The Element of Typographic Style, p.19 : "Typography is just that : idealized writing." Because we wanted to spread words and knowledge faster and cheaper, monks were not enough and we had to use a standardized form of writing : type was therefore invented with printing. Keyboards came when we figured we could build a little printing machine, the typewriter. And when we needed to find a way to talk to a computer system with characters, these four rows of keys looked pretty good.

We may, today, be able to pack enough computing power in a tablet to spread words and knowledge simply with our own handwriting, giving us the choice of leaving it as is, or to format it Gutenberg style. Is this still science fiction or a close reality, I have no idea. Is this economically viable ? From what I've outlined, possibly, but I'm thinking out loud and dreaming here. Maybe it's not "entertaining" enough to use a stylus on a tablet, maybe it's too much of a pen nerds niche. You also have the form factor of the stylus in itself, and the relationship it has with the tablet - pull out styluses really look dated and odd in the way it was done on Palms and Newtons. Stylus only input : no way. Multitouch only notepad-sized tablet ? That would be frustrating for sure.

Update : here are some illustrations about the process of tweeting described above. Le scribe accroupi du Louvre

This would be a webpage you're viewing, and you'd want to tweet about it (browser navigation interface is missing)

Starting the tweet

At the moment you start writing, an overlay appears so you can focus on what you write. As it understands it's a tweet, you get an instant character count.

Tweet completed

The right arrow sends the text and url to twitter

Tweet confirmed

Confirmation message of the completed task. The overlay would then dim.

Tweet published

And of course, your tweet is published as scripted text.
☛  Table of contents

Copyright Thibaut Sailly · Subscribe to the feed · Powered by MovableType