1 year at Comcast

I just completed my first year at Comcast. Its been a gratifying experience thus far, both personally and professionally. I've been having a blast living in downtown Philly - the lifestyle has been wonderful, and the 10 minute walk to work everyday has completely spoiled me for commuting in the future. And Comcast has been a great place to work: an interesting and challenging set of problems, and a good group of people to work with (and for). I'd be lying if I said it was moving as quickly as I'd like - although I think I've made some meaningful contributions throughout my time thus far, the bigger consumer impacting things I've been working on (tru2way and Project Infinity and the like) will only begin showing up in the back half of '08 and only really at significant scale and volume (from a consumption perspective) well into 2009.

That's a long time.

Still, its a complicated technology environment - its a heady mix of legacy technology infrastructures, permutations of configurations and network designs, and a significant scale of problem set and quality-of-service requirements - and just to help, we enjoy a... challenging regulatory environment. (Boy, Those sound like excuses, even as I type them)

In celebration of my year (sort of) I spent last week at NCTA's (National Cable and Telecommunications Association) "The Cable Show". I spoke on a number of panels at the show, which is always fun. The Q&A's are always a bit frustrating, as people come to the shows to get answers, which I have .... but usually can't give. We have enough problems internally distinguishing between *goals* ("we're committed to getting there") and *plans* ("we actually have a path to get there") - no good way to make that distinction clear externally - especially for a public company.

The only real drag has been my travel calendar - would like to slow that down a bit. All in all, though, feels like I'm just getting warmed up here, which is both exciting and depressing, on many levels :)

You can't make this stuff up...

My blog is blocked by our corporate firewall... (and I quote):

Access to this page has been denied by web filtering.

If the site you are trying to access is critical to your job function, please open a support center ticket and provide the full address of the site that you were trying to access and the following message in its entirety:

Access to http://sree.kotay.com/ for user adapps.cable.comcast.com OU=Users - CHQ,OU=1500 Market,OU=Corporate,DC=cable,DC=comcast,DC=com\Rouleau-Hellhake\, Shari has been denied for the following reason:
The Websense category "Social Networking and Personal Sites" is filtered.

I guess my blog isn't work related... not really sure WHAT it's related to, come to think of it...

Secrets of the Cable Universe #2: Bandwidth, pt 2

Continuing from Part 1.

To cut to the chase, your coaxial cable (and hence your cable company) is capable of delivering (roughly somewhat less than) 5Gbps into your house.

The math for that number is pretty simple. There's about 750mhz (or so) or RF spectrum avaible, divided into 6mhz "channels". Why 6mhz? Because that's about how much spectrum you need to deliver an uncompressed analog NTSC AV signal - in the digital world, that translates into (about) 40Mbps. So 40*125 = 5 (you know, adjusting for decimal places).

That sounds like a big number, and it is, but there are a few mitigating factors, as discussed last time.

First, it is a "shared" connection. There a certain number of households grouped into a "service group", usually between 200 to 2000 (very broadly), which connect to some physical networking gear at the cable plant (I use the term "plant" very loosely here). Within that service group, DOCSIS (the cable networking data interface protocol) basically works like a form of encrypted ethernet. Everybody in that service group sees all the packets, but cannot decode those packets.

So right off the bat, your effective sustained speed (more on this concept in a bit), is 5Gbps divided by [number of homes in your service group].

Additionally, some portion of that spectrum, those channels, is consumed by television - In fact, almost all of it. Today, only a single 6mhz channel is allocated to your High Speed Data (HSD) connection. That's a big part of what DOCSIS 3.0 promises - the ability to "bond" channels to effectively multiply the available bandwidth by N number of 6mhz channels.

Assuming there's channel spectrum to allocate...

In digital form, some 10 to 16 or so Standard-Def (SD) channels can fit in a single 6mhz channel (multiplexed into a single MPEG container over that channel, for those curious - this is also important because it has implications for compressions; specifically regarding CBR v. VBR). Two to maybe 3 or four High-Def (HD) channels can fit in the same 40Mbps. The range incidentally, is largely a function of compression quality per digital channel; this is probably worth a future post.

Short version: you get either 1 analog channel, 10 to 16 (or so) SD digital channels, or 2 to 4 (or so) HD digital channels.

Thr rub is this: most of the U.S. is still analog. Or at least, enough of it that most cable operators carry around 80 or so analog channels (out of a possible 110 or so). Then they consume another 6 to 8 or so "double carrying" the same channels in digital form, and the rest is allocated to HD channels (triple carrying many channels) and VOD (Video-On-Demand - more on how this works in a future post also).

Which doesn't leave a lot of room for your HSD connection.

Interesting, the core technology and bandwidth available compares reasonably favorably even with newer technologies like optical fiber (remember that we're talking about the connection to the home from the "edge" of the network - "inside" the cable network is often optical fiber, already; the challenge here is the "last mile"). Mostly the advantage of networks like FiOS is smaller service group sizes (owing to larger capital investments and other "late mover" advantages), and less legacy encumberances.

Secrets of the Cable Universe #2: Bandwidth, pt 1

As with previous installments (ok, the one :P), this post also isn't so much a "secret" as it is a clearer explanation, with some implications, of widely available information. There's been an increasing fervor building around bandwidth, bandwidth management, and its implications as consumers are now finally beginning to consume rich web content at scale (if "Monkeys kissing women" , "Man in underwear" or crappily digitized stolen content can be considered "rich" :P).

So how much bandwidth does the Cable infrastructure provide? In past lives, I'd always heard that Cable was a "shared" pipe, while technologies like DSL were not - so let's explore what that means.

To go back to first principles, the coaxial cable coming in from the street to your house delivers about 750mhz (or so that's usable) of information spectrum (I'm not going to get into RF modulation and how it works here). In the old days, that 750mhz was split into 6mhz channels, which turned out to be about what you needed to deliver a single ucompressed "standard defintion" (NTSC) audio/video signal - a channel, basically.

And that's why you basically had about 100 or so channels on Cable, and no more, really - there's a limited capacity to what the "cable" from the Cable head-end into your home could deliver. Fundamentally, it was constructed as a multicast technology - broadcast from the Cable company's head-ends, down your street, and split into homes in your neighborhood. Each node from the head end could pump a signal of sufficient strength to service from 200 to 2000 or so homes (really, really, roughly).

And everybody got basically the same content (though so-called "Conditional Access" would encrypt at the head-end and decrypt either at entry to your house, or on your settop box - again, beyond the scope of this discusion).

Then came digital signals, and things got interesting. Turns out, that over these 6mhz bands, you could send, oh, about 40Mbps of information (still multicast, of course - meaning everyone gets the same information). And if you MPEG-2 compressed your video, that worked out to - for Standard-def (SD) video - about 10 or so video streams per channel (figuring 3.5 to 4Mb per video) - or to put it another way, you could pack 10 or more digital "channels" into the bandwidth occupied by a single analog channel.

In part 2, I'll cover how this maps to Internet Connectivity and your Cable modem and how much bandwidth Cable really delivers to the home(its not the number you think - do the math implied above) .

Presence is not an Identity feature

I realized in my "Unique Visitors" post that I skipped some background - didn't "show my work" as every math teacher I had always said.

When Google released GTalk, I remember feeling like they had missed an opportunity to correct the Presence, Identity, Messaging and Accounts model - that they should have made "location" more than just metadata, but I don't think that's right, actually.

On the PC side of the Instant Messaging (IM) universe, Presence is exerted by an Identity, and (nowadays) often qualified by Location (for Location Based Services) and/or Network of Origin (for interoperation messaging systems), but I think Presence, especially for Comcast, is really a device feature, and Identities are transiently bound to those devices, generally with an (essentially) eternal TTL (time-to-live), though in the case of PCs, that TTL amounts to IM session length.

Identity is a kind of user virtualization construct (usually reasonably synonymous with "Credentials": username and password - at least for our purposes), while Presence advertises availability for messaging, and Messaging is generally a point-to-point session negotiation and communication pipe, though it may be one-to-many (broadcast), or many-to-many (chat). Note that Messaging and Presence need not necessarily be interdependent. Finally, Accounts are a billing relationship construct.

A typical use case in a househould is likely to involve multiple STBs (set top box or other consumer electronics device on the Comcast networks), multiple .net IDs (e-mail addresses), and multiple PCs in the house, where Presence, Identity, Message and Accounts overlap each. And of course, wireless, school/office, and multiple residences complicate the picture.

Usually, scalability for Presence is scale limited by subscription/notfication events, and by that I mean the "m X n" implied by the "Buddy List" construct, where any user may be subscribed to presence notification of any n users (effectively O(n^2) ). Messaging is bound by active simultaneous communication sessions (and type - multimedia for example) and is usually O(n).

In any case, I'm just rambling now - all of that's a roundabout way of saying that messaging is a service feature, not a product :)

Secrets of the Cable Universe #1: VOD ffwd/rwd

I've now been working at Comcast for about 6 weeks-ish, and so far, I'm having a blast. Both culturally and technically its a fairly dramatically different environment (along a different axis than the small-to-big transition that AOL was... more on that another time) - which is always fun.

Although I'm finding my skills, talents, and experience are useful (thankfully), the whole Cable/Telecom universe is completely new to me, so the learning curve is both vast and interesting.

In that vein, I though I might share some of the random but interesting technical tidbits that manifest themselves in odd ways, whether operationally or in terms of the consumer experience. Nothing I'll share is (obviously) actually a secret - its either public information and/or subject to trivial induction from public information.

For example, one of the significant features all the Cable operators (MSOs) have introduced over the last few years is Video-on-Demand (VOD). Unlike the "your-movie-starts-every-15-minutes -on-4-different-channels" model that the satellite providers started with, the new VOD systems actually dynamically allocate a unique "physical" channel from your local cable head end when you select a movie. The video asset is then played over that channel which your set top box (STB) is then tuned to automatically, as if it were any other channel in your channel line up.

So the interesting "secret" is that in order to enable fast forwarding (and rewinding) of the assets, each media file actually has additionaly "trick files": copies of the asset at +/- 2X (or whatever the speed multiplier is). When you press the ffwd or rwd button on your remote control while watching a VOD asset, it's actually dynamically switching to another asset at the right time code, and playing from there.

And that is why you (currently) only have one speed for fast forwarding or rewinding: more would have required many more multiple media assets (one at each speed) in the VOD storage systems.

Clever, but strange...

Unique Vistors are not users

Being at a Cable/Media giant now, as you might imagine, we discuss advertising a fair amount, and in particular, exploring the strengths, weaknesses, and, really, differences in how the advertisers, content creators, and distributors think about the eyeball value chain on the web vs broadcast media.

One of the obvious but interesting observations for web metrics is that the commonly used measure for audience is not actually people. That is to say, the "visitors" referred to by "unique visitors" isn't people at all, but devices. And even that's a bit of a misnomer, because its really, for PC users, a per computer per OS user account metric. Whether its a browser cookie, Flash local shared object, or Google Gears data store (the latter two don't get cleared when you delete browser history in your browser, btw) - nevertheless, they are all at the same level of "user" granularity. I'm going to suggest that OS user account is really a poor man's device and data virtualization technology, much in the same way that Multifinder was a poor man's multitasking technology back in the days of the original MacOS, and thus, we're talking about a device metric.

Unique visitors (UVs) really is a direct measure of how many devices connect to a given site. And it is correlated, of course, but not identical to the number of actual users visiting that property. Some sites you may use only at home or at work (one UV per user), while some may be used at work and home (two UVs per user), or, in cases where many users share the device (home computers, or set top boxes, for example) it may be one UV for many actual users.

Magazines will often refer to the "pass along" index of a magazine: that is, how many people might actually read it, but may not have purchased it (House&Garden magazine has a pass along readership of 14 or 15 people per sold copy, whilst National Geographic is around 5 or 6).

In this area, the Internet is surprisingly immature, given the promise (and increasing reality) of behavioural, demographic, and metric oriented targeting of the world's many-to-many publishing medium. This kind of thing becomes important not just for CPM advertising (that is, impression and brand based advertising), but even more so when considering the efficacy of CPA advertising (so called "Cost-Per-Action" advertising).

I mentioned magazines rather specifically, because it appears that Internet advertising growth is coming most directly from print and publishing, and not at all at the expense of broadcast (its shrunk nationally, but more than compensated in other channels). Perhaps we need to extend UV's to be UV/U's (Unique Visitors/Users), much as Nielsen's does for TV ratings/share to extend more actionable transparency to advertisers and targetting technologies?

The Bandwidth Shell Game

Whilst getting my slashdot groove on yesterday, I encountered this: Will ISPs Spoil Online Video?

The main thrust of the article is that no ISP can actually deliver the "promised" sustained bandwidth for all users on its network (or even a large percentage of its users) at any one time.

The article is basically true, in the facts, and I've touched on the topic of video bandwidth and the 'net in the past, but its (somewhat) unfair to narrow this to an ISP issue. (I say "somewhat" considering my previous gig was at an ISP, and my current employer offers Broadband ISP services, so perhaps I'm not the most objective here...)

For example, every website plans against peak load, not total possible usage - same problem: you can't access promised services (paid or free) as advertised/committed. And, more on point, Google's ever increasing g-mail mailbox size is also bogus in the same sense -they can offer that much storage because not everyone uses 2+GB for mail (very few do, in fact).

Really, all businesses do capacity planning (online AND offline) to determine pricing (and therefore marketing claims), and bandwidth is no different in this regard.

I can't even make a call for the first 30 minutes after American Idol ends - wireless capacity planning never forsaw the Seacrest effect. And although I went to the Buffalo Wing Factory in Va ("Home of the Flatliner") for some spicy buffalo wings one night after a goodbye party for a departing colleague, they were, in fact, out of Flatliners. Grrrr....

What makes it thorny for most connected users is that the usage profile of the service, of the Internet, continues to evolve very rapidly, making terms of service seem quickly antiquated. What people should bear in mind though, is that the terms of service are simply a reflection of the economic and topological constraints of the network itself - usually in place to guarantee some core QoS (Quality of Service) for as many customers as possible.

Nobody's trying to trick anybody, or game the system - but you can't plan for what you don't know, and the increasing interconnectedness of things make prediction a dicey thing. That is to say, the dumber the network, the less visibility available.

Consider, for example, P2P applications are good (i.e. cost) for the endpoints (origin and destination), but usually MORE traffic (i.e. cost) for the network itself.

New Gig: Comcast Chief Software Architect

Starting Monday, I'll be working again, joining Comcast as SVP and Chief Software Architect.

The first thing I said when talking to people at Comcast (early on) was "Comcast? Software? Isn't that an oxymoron?" Suffice it to say, today they're a Cable Company... but tomorrow?

Well... more information in future posts... but mostly I was motivated, at a personal level, by three (professional) considerations:

(a) At this point in my career, I wanted to either go do my own thing again (my usual default stance), or take a significant role at a significant organization; Comcast did $26B in revenue last year, and, though still facing far more demand than supply in the market place is still aggressively pursuing a deep business transformation for the new millennium. They're in a great place, and they're running very fast.

(b) I'm gratified that my particular skills and experience will be applicable and valuable, but also wanted to play in areas of complexity where I had little experience (Cable/Telecom) - nothing beats tackling new problems. And Comcast has no shortage of that, though they have a strong market position.

(c) I was quite impressed by the caliber, intellect, and alignment of the executive team. That last, in particular, I've found to be uncommon in large organizations (not just from my AOL experiences, but the half dozen or so "usual suspects" I've spoken to over the last few months).

Updated my LinkedIn profile last week - and am currently drowning in paperwork (employee stuff, moving, buying, schooling, re-locationg, eeeerkkk...) - but its all good :) In particular, my wife grew up in Philly (where Comcast Corp is headquartered), and it was while working near there that we met, so geography is actually a big part of the appeal, too - we have lots of friends and family in the area.