Touch UI and the Art of Intent

Observation: One of the great things about the mouse as an input device is the idea of "intent" - that your cursor indicates your locus of attention when used for interaction (Roll-over states and tool-tips being trivial expressions of this).

You have a simpler model of this semantic with the Blackberry trackball (and the Blackberry jog wheel before that), and a crappier version with things like using your remote control with a TV EPG (guide) or the arrow keys on your cell phone to navigate menus.

Touch interaction systems, like the iPhone, lack that model completely - just like most older (read: HW only) Consumer Electronics UIs (think VCR or DVD player).

In some cases, that really doesn't matter much... and in other cases, the directness of interaction provides a far better paradigm.... but, it suggests the question: is "intent" a semantic that will disappear for Touch UI? Or is it a temporarily "lost" item, like tactile feedback - just a gap to be crossed?

Review: Beowulf (in 3D!)

First, the short version: it was a fast paced, man-movie of an adventure (especially in 3D!). Not quite as testosterone-ly epicly big screen worthy as, say, 300, but still a movie well worth seeing in a theater (in 3D!). Really fun.

The longer version: It was good, but won't hold up - even for re-watching in the near term, and certainly not in the longer term as a "film".

Obviously, there's the whole 3D(!) -motion capture thing. Definitely a huge step forward here, especially as compared to Zemeckis' previous outing with this technology in The Polar Express. In particular, I thought they really nailed the close shots (especially the eyes), but in attempting to service the "real" left a lot of the motion looking very stiff. Ironically, it was the big motion action sequences and distance shots that looked the most fake - great effort, but still short of the "reality bar" and likely to look Dr. Who bad in a decade or so. Still - there are moments when you really, completely get drawn in (and then, *whack* get snapped out :P...)

Story-wise, the conceit of the movie (i.e. why the filmakers are "re-visiting" the well known story) is this: Imagine if everything in the original were literally true, but embellished by the narrator (Beowulf himself for much of the tale) and edited for "mature content" by the transcribers (likely cleric/priest scribes) of the epic Olde Englishe poeme. It definitely adds a layer of pathos to the story - filling in the missing "back story", but also afflicts the story with that Soap opera interconnectedness that's drowned action/adventure storytelling (especially sci-fi and fantasy) in the wake of the Empire Strikes Back.

Sometimes a dragon is just a big evil dragon , you know?
(apologies in advance for the politics of the previous link - but, whattya gonna do?)

Still - its was fun to watch, and fun to note where (and why) it diverged from its source. And it was co-written by Neil Gaiman, who, even when just off, is miles better than most...

The Great SetWorkingSetSize() Scam...

I saw this post at ZDNet about Firefox 3 memory usage,. Setting aside for a second whether Firefox 3 is better than IE 7 or Firefox 2, this reminds me of one of the great cheats of "small applications" developers everywhere:

SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1)

This Windows API makes your application look *very* efficient without actually doing anything, and has been employed by MANY a popularly considered"lightweight" application (and some, um, less light) - because the "Memory Size" column in Task Manager on Windows doesn't reflect memory usage.

"Hunh?!?!", you say?

That column actually reflects the working set of memory for your app - which is the amount of memory currently "realized" (in active use) by your process. Let's look at some use cases to illustrate what that actually means:

1) Allocate a bunch of memory and free it. Your app isn't reserving the memory space, but the working set may still be high - Windows will lazily reclaim if its needed by another application.
2) Minimize all your application windows. This does the equivalent of the Windows API call I listed above, and the memory working set for that application gets *totally* paged out. Then Windows will load back the memory pages as they're accessed - its the equivalent of clearing a cache.

This last is confusing (and illustrates the issue): after SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1), "Memory Size" in Task Manager doesn't reflect what's been "reserved" (allocated) by an application, just what blocks of memory are being/have been actively "touched" since the working set was "cleared".

If all that's confusing, fortunately for you, its easy to boil down to a simple action: Use the "Virtual Memory Size" column in Task Manager instead to look at application memory usage. You can find under the "View... Select Columns..." menu. It reflects what the application has requested from the OS, but not yet released, i.e. the real memory consumed by the application!

More info here.

Android: Sound and Fury signifying... ?

So... I've been looking at Android a little bit (what's actually available thus far is really the SDK, not the platform itself).

A few comments:
- Custome Java(-ish - more on this shortly)VM/bytecode engine
- Integrated Webkit for HTML/JavaScript authoring
- Fast, robust 2d graphics (native and software only) with a custom presentation engine
- OpenGL ES, with possibility of HW acceleration
- Rich storage semantics through SQLite
- Still a little fuzzy on the licensing terms: Google says (emphasis mine)"Over time, more of the code that makes up Android will be released..." - not sure what that means...

The SDK support Java application only at this stage (and the forseeable future?) - though, in theory, platform source code being available under "non-restrictive" terms creates the opportunity for other types of enhancements.

So - still early, but on the good side: LOTS of (welcome/needed) attention to the graphics and presentation layers.

On the bad: fracturing Java (which only recently has started to make progress on getting past the "build once/debug everywhere" problems). If Sun doesn't address this with Google, it'll be hard for them to maintain any credibility or cohesion with the Java Community Process for managing Java's roadmap.

Interestingly this hilights a truism of mine: for software, single implementation trumps single specification - more on this in a future post.

How this will work with open source and Android is unclear, because as Johnathon Schwartz (Sun's CEO) points out: "Companies compete, communities simply fracture".

In particular, it'll be interesting how Google rationalizes their rumoured "non-fragmentation" clause and the idea of "non-restrictive" licensing...

Watch this Space, pt 1 (AKA the Android cometh)

Updated: Up and available around noon EST (someone sent me the link already! :)). Will look at it tonight... amusingly, win and mac tarballs downloading now, but consistently crapping out on linux set...

So, despite all the "news" last week (*cough* tease) - and all the subsequent pontificating - today (in theory) is when the Android SDK - the guts of the long rumoured Google "gPhone" - is actually released, and we get some real meat.

I have to hand it to the Symbian guys, though who take "quote of the week" for John Forsyth's likening of a Linux mobile initiative to the common cold: "It keeps coming round and then we go back to business."

Heh.

So... watch this space (I'll share some thoughts once I get an "open" look...)

Toddlers bond with Robot

Pretty cool story in the New Scientist on Monday: Robot becomes one of the kids. Basically, researches found that, with the correct behaviour emulations, an advanced robot was able to integrate into a toddler group as a peer (as opposed to "as a toy", or "as a pet", even, based on touching clues and other interations).

Video below (link):


This has implications for group behaviour theory and social development (it hints at a lot about how we develop thinking about "us"). But mostly it speaks to the evolution of robotics assistance in the classroom, especially for young children.

There is clear evidence that the "uncanny valley" gets substantially farther and wider as we age - the question that this study ponders, but only partially answers, is: Why would we ever use/need *humanoid* robots?

Still: cool :)

ES4: The Javascript 2.0 Blaze

Updated: Now on Slashdot.

A bit of a flame war going on in the ECMAScript working group (which spices up an otherwise reasonably boring mailing list).

This blog post (from Mozilla Foundation CTO and Javascript creator Brendan Eich) is really only tip of the iceberg - you should follow some of the links from his post, or check out the mailing list archive.

Javascript 2.0, or more formally ECMAScript Edition 4 (or simply ES4) has been in the works for a good, oh.... 8 years now. With the ES4-in-motion work from Adobe (nee Macromedia) in the form of ActionScript 3 (AS3), and the rise of Firefox, Safari, et al. its been getting a serious push to completion over the last year and a half, especially in the face of Microsoft's C#, Silverlight, and (though no one said it directly), I think even Adobe's Flex and AIR.

The battlelines are pretty clearly drawn, with Microsoft and Yahoo on one side, and the Mozilla team, Opera, Adobe (interesting, eh? "Enemy of my Enemy" anyone?), and oh, pretty much everybody else on the other side. Or, as you might first opine from that cast, Evil v. Good.

MS and Yahoo think the language is changing too much, whilst the others think that it needs to in order to be competitive for the larger scale programming projects the web is increasingly requiring.

My opinion? As is usually the case, they're both right - you only have to look at the Flash community's response to ActionScript 3 .

In short: They like it - a lot, but its very different than AS2 development.

And, oh, from an implementor's perspective, AVM1 and AVM2 (which, roughly, correspond to ES3/AS2 and ES4/AS3) are two completely different VMs. Is it possible to make one that does both? Sure... but there's no denying ES4 requires *substantially* more effort and complexity to implement (note I'm *not* making an argument about code size here....)

An so, in a deliciously Shakespearean turn, Doug Crockford of Microsoft of Yahoo asserts (correctly I think) that the issue is fundamentally one of nomenclature.

Of marketing.

If its not called Javascript, would anyone use it? And if it is, how similar should it be?

Quite frankly, Brendan's probably right in that, whatever justifications the opposition might even believe, there is a bias to keep Javascript "ghetto-ized" to a degree - because of existing investments and strategy considerations.

That doesn't, however, make it wrong to push in that direction.

Personally, I do wish there were less emphasis on the "compiler/VM" split that Java brought into vogue - it seems to be at the the heart of a lot of the design decisions that make AS3 and ES4 feel less "Javascript-y" to me.... but that's both good and bad - ECMAScript 3 is forgiving of errors well past the point of stupidity.

And so, lightweight stuff really is harder, but its also a lot easier to write stuff well...