Metrics and Analytics

posted 02/16/08 by Rick Webb

We get a lot of questions from our clients about Metrics. How do we know if our campaign is successful? How do we know if it worked? What kind of stats can we give someone to prove this was a success? How can we track this.

Here is our unfiltered, gods-honest truth. You know it when you see it, and all stats are bunk. They can, will, and are always tailored to make a campaign look like a success. In our opinion, you’re wasting your money buying any tracking packages – there are perfectly good ones that tell you everything you need to know, for more or less for free. In our opinion, companies that track the success of viral efforts are bunk. Companies that track “buzz” are bunk.

There is, of course, much to be learned by looking at the traffic on your site, and there is much to be learned from things like the number of clicks a banner gets, and the number of times a video was downloaded from YouTube. These, however, are only useful comparing them against other campaigns, and the same metrics. 1 Million YouTube downloads isn’t a lot. 10 million page views sounds great but may or may not mean anything, depending on the circumstances.

There are a lot of marketing people out there who have grown very comfortable and happy with the statistics they can get on the broadcast side. Statistics are ridiculously well kept on that side. You can see the needle move in sales and calculate, to the penny, exactly how much money a campaign made you. God, if I were a marketing exec I would be eating that stuff up – I’d have paid some 18 year old programmer a bunch of money to make me a set of widgets for my computer that let me see, to the second, how effective my advertising was. In fact, I’m willing to wager that this already exists.

The challenge, then, is that these types of marketing people are hearing a lot of great stuff about the web. They’re hearing how it’s trackable, and it’s the great white hope for high bang-for-your-buck advertising.

This is all true. There is a ridiculous level of tracking ability with certain kinds of web advertising. And this, my friends, is where the catch lies. You can track a banner through to a website through to sales perfectly. You cannot track an online video game’s emotional impact on a customer and how it helped them to buy one type of cereal over another. It’s the age old branding vs. sales approach to advertising – it exists on the web just like everywhere else. Compound this with the fact that there is no means to compare, say, $1million spent on the Internet directly against $1million spent in print. And on top of that, there’s no way to tell yet, at all, if having 10 million people download your ad on YouTube was worth anything.

For a metrics-based marketer, it’s all kind of a nightmare. The temptation is, then, when moving your marketing dollars to the Internet, to place your money in media that is tracked – banners, websites – over media that is not – phantom campaigns, advergames, MySpace groups. This is not unreasonable, however, it is easy to forget: Just because an advertising vehicle is more trackable does not necessarily mean its more effective. You may measure greater success with ad vehicle A over ad vehicle B, but it is very possible this is purely a dynamic of the characteristics of the metrics themselves, not the efficacy of the vehicle.

It’s our belief that inconsistent metrics are false metrics, and all metrics on the web are inconsistent. Sure, we’re happy to track everything. We’re happy to tie into whatever tracking packages you already have. We’re happy to trot up to your clients with you and say “this campaign received 1.2 million unique page hits and blah blah blah.” We’ll do what it takes to get the job done. But in the end, we’re only doing it because we know, in our hearts, that the campaign is a good idea, and we want to help you. If you are metrics obsessed to the point where this paragraph upset you, we are probably not the right people for the job.

In the meantime, we do what we can to push the limits of the world of online metrics. We work on software that measures things that haven’t been measured yet. We offer insights to other metrics companies. We sit on panels and committees. We are dabbling even in startups. We believe there’s a long way to go with metrics, and we try and push it when we can. After all, analytics are definitely part of Marketing R&D.

Here are some recent posts from our employees about Metrics and Analytics:

Diagrams - 12 New Yrs Resolutions for '09

So it’s that time of year again…New Years Resolutions!
I’ve always been a big fan of setting and tracking goals. It’s gotten me to run marathons, finish grad school, and complete other daunting tasks. I suggest others to set realistic goals by using the SMART Goal Setting method)
This year I have 12 resolutions! Yep, it’s a lot. Surely full of unpredictable blows, as well as the following tasks I plan to accomplish. Because some goals are complex, I decided to dissect them into clear paths. This will help to achieve them, as well as aid in tracking my progress. Also, I’ve been missing work and aching to do some diagram sketching! Enjoy, hope it inspires you with your own goals!
Here’s my 12 resolutions for 2009, in no particular order….
You can also find the diagrams in a Flickr set
1) Buy a House in SF &
2) Get a Dog

- workflows
3) Compete in 1st triathlon
Page icon denotes a separate document coming soon,16-week training schedule.
- workflow
4) Run 3rd Marathon
- workflow
5) Drink 8 glasses of water a day
-12 hr clock
6) Maintain weight +/- 5 pds
- visual scale
7) Help 3 people with their health/fitness related goals
- Hyperbolic Tree
8) Turn 30!
hoping this one is relatively easy.
-Calendar date
9) Resurrect 3 lost/broken friendships
-Life Cycle Diagram
...and a couple extra without diagrams!
10) Regularly track finances/investments through Mint
- accounts set up, but need to add investments & set budgets
11) Take a writing and/or public speaking class
12) Blog at least once ever 2 weeks
And that’s it!
Happy 2009, hope all your dreams, goals, and wishes come true this year!

Playoff Baseball and Luck.

The owner of the Oakland A’s, Lew Wolff, recently suggested that the first round of playoffs in professional baseball, the League Division Series, should be shortened to a single game. He said it would be, in his words, exciting.
I find this interesting for a few reasons. First, the source of the suggestion: As was noted in the book Moneyball, the problem with the Playoffs is that it undoes the thing that makes a low-budget, strategically winning team such as Oakland successful: it introduces luck. The timeline of the playoffs is excruciatingly short when compared to the marathon regular season, so luck (a random vector that only time can average out) becomes a far more potent influencer. This could be the reason that, despite having winning seasons, the A’s particular management style never finds them in the World Series.
So this makes the suggestion by Wolff to shorten it to one game all the more curious. I could understand the impetus, but this only exacerbates the problem: shortening the gameplay from 5 games to 1 increases the possibility of luck as influencer dramatically (I would say it increases it 5x, but I’ve never been good at percentages). I suppose they hope for the best on the crapshoot: put it all on black.
If Major League Baseball really wanted to shorten the postseason while limiting luck as a deciding factor in the outcome, the structure should be something like this: LDS: 7 games. LCS: 5 games, WS: 3 games. That would ensure that luck as a factor was minimized during the early rounds, when there are a greater number of teams involved, ergo a greater number of games played (the math works out to a possible 28 games in the LDS, 10 games in the LCS, and 3 games in the WS) to spread the luck over. That way, by the time we get to the World Series, luck is only strongly influencing the outcome between 2 teams, so it’s a greater chance that a “deserving” team will be crowned champion.
Now, will Selig agree to a best-of-three WS? Severely doubtful. The World Series has always been a best of 7 situation, and tradition aside, limiting it to at worst 2, at best 3 games wouldn’t exactly make for a windfall in advertising dollars. So I humbly suggest the 3-5-7 game structure. Sure, it still increases the odds of an “undeserving” team making it to the World Series, but each successive round would prove out the stronger team’s particular strengths.
Just a thought.

Dot Surveys: Lo-Fi Analytics

While designing a custom web analytic project at work, I’ve found myself increasingly intrigued in data sets. At work I’m thinking about data sets particular to the web, and at home I’m thinking about data sets specific to….just about anything, such as… Q: Number of cookies eaten today? A: 3 Q: Ratio of plants to [...]

Bugs, Skins and Tickers

Lots of new ad units cropping up to support the burgeoning online video market. It’ll be interesting to see where it all shakes out. I’m sure it’ll have something to do with measurement and effectiveness.
Anyway, today’s news: skins. I guess their existence is not really news because a skin – a clickable, branded border around a video player – has been around for a year or so. The NEWS is that CBS is employing them on their on-line properties like CBSSports.com. Great WSJ article about the move. This article also describes very clearly other new options like “bugs” (logos on or next to video) and “tickers” (horizontal bars underneath video). Check it.
One question: what is up with the names? I feel like searching in my hairline for some ticks.