What's the trouble with media measurement: Part I

When a certain report was released several months ago in Hong Kong and Singapore which suggested as much as 65 per cent of marketing spend in the region was being wasted, and achieved no return at all, the marketing fraternity including clients, their agency brethren and media owners screamed blue murder.

How could this be when so much is invested in TV ratings reports, readership surveys, online analytics, in every kind of consumer sentiment and brand equity research and in emerging new measurement technologies?

The answer, at least to why we know so little about the effectiveness of marketing spend, lies in close analysis of the media measurement we either accept as the best possible method for aiming marketing messages at desirable targets, or begrudgingly use for want of a better system.

Across all the media marketers use there are inadequacies in the measurement systems. Some lie with stubborn media owners who refuse to consider standardised methodologies for fear they will be forced to financially wear the burden of improved research or worse, that standardised measurement might lower their audience, or the quality of their audience.

Other deficiencies include the inability to separate quality of audience from mere numbers, disparate methodologies for gathering data which penalise some media and unfairly favour others, and the simple fact that media measurement is failing to keep pace with changing media consumption habits.

Even online measurement, touted as the great hope for accuracy in measurement, is problematic, not only for the dozen or so measures of online effectiveness but right down to the question of what a marketer wants to measure in the online space.

And not only does measurement of each individual media have issues that inhibit strategic planning, it is very difficult for a marketer to measure one media against another media and assess ROI of spend placed across multiple channels.

Asia isn't unique in its search for accurate measurement metrics and systems and the demise of the recently shelved ambitious Project Apollo in the US (see page 45) which sought to tie spend more closely with results and consumer actions shows despite the best intentions fully transparent and accurate measurement is an elusive, if desirable, goal globally.

But the industry can't give up if media and agencies want to stand shoulder to shoulder with marketers in boardrooms across Asia to help them explain and justify marketing budgets and particularly that significant slice of it spent on media. This becomes even more important as the march of technology alerts CEOs to better measurement possibilities - even if they aren't realised - and with the looming global slowdown where marketing budgets may be the first to get slashed by short sighted bosses if every cent can't be accounted for.

Print Measurement: Print and be damned, audit and be saved

One of the bitterest measurement debates in Asia is over circulation auditing, with those who do passionately berating those who see no value in it and who dismiss audit figures which, in many more mature media markets, are considered gospel. Those who support auditing see it as an honesty in publishing issue and view those who simply provide unsubstantiated circulation figures in their media kits with no audit certificate as misleading their advertisers.

Against the fierce criticism of publishers who believe auditing is the only true measure of a print publication's reach and influence, those who refuse to audit often argue that they would be at a disadvantage if they audited because many of their competition don't. It's a weak retort, but there are other concerns in Asia over auditing procedures.

Cyril Pereira, principal of Telesis Consulting and executive board member & treasurer of Society of Publishers in Asia (SOPA), argues that local advertisers and agencies are also to blame, for not demanding transparent data and as a result valuable advertising dollars are being spent on "unsubstantiated metrics".

"It's misrepresenting what quality media should really be about," Pereira says.

"We have for a century or more settled on a thriving formula for advertisements based directly on the guaranteed circulation base, [but] large parts of the industry in Hong Kong and elsewhere are not audited, although auditing services are available."

Hong Kong's Audit Bureau of Circulations (HKABC) says its data and collection methods are rigorous and highly transparent - with PricewaterhouseCoopers holding the contract to audit all member titles and all data is made available online.

Pauline Yu, secretary of HKABC, says there are a number of moves to combat criticism and ensure the organisation is keeping pace with industry changes.

While the organisation can not force publishers to audit, she says the body can work a bit harder to get clients to demand audits.

Depending on who you talk to, the state of print measurement in the key markets of Asia is not in good shape. Criticism of the methods used to sample the quality and size of the audience including face-to-face interviews is strong, as is the criticism about the very rules governing the paid circulation system.

In the past few years, auditing bodies around the world have been forced (in many cases reluctantly) to review their circulation and readership rules. These reviews have resulted in sweeping changes to the way publishers collect and report circulation data.

The rapid emergence of content distribution tools like podcasts, RSS feeds, news blogs and social networking sites, together with a growing list of online media brands, have only added to the woes of publishers and auditing bodies.

Readership rewritten

It's no secret that traditional reading habits are changing. Media is no longer consumed in a silo and is often just one of many activities being undertaken at a particular time.

Business readers are getting busier and harder to reach, the younger generation are migrating online and added to this is the harsh reality that older readers are simply dropping off the other end. While publishers are largely supportive of the research and measurement tools available in Asian markets, the way they use the research is changing.

William Adamopoulos, president & publisher, Forbes Asia, says today the magazine's biggest clients are not buying on reader surveys.

"These days in a post-internet era, it's not so much about paid circulation, it's about proving you have an engaged readership. Quite frankly the agencies tend to buy on readership," he says.

While the Forbes Asia edition, launched two years ago, will be audited later this year, Adamopoulos stresses that readership is more important.

"Both surveys in the market are good solid surveys, but they need to be applied correctly. If people using PAX are more of your broad base more mass market it makes a lot sense but if you're talking about reaching C-suite executives then you better use BE:Asia or use your judgment to look at the title," he advises.

Readership surveys around the region are today growing as a dominant tool for measuring not just the paid circulation numbers, but the overall quality of targeted demographics.

BE:Asia, formerly ABRS, along with the PAX and Media Atlas studies are the dominant players in this field. But the face-to-face and recall-based surveys these groups undertake have come under fire for being out of sync with the lifestyle changes of readers and bucking the natural trend of declining numbers of readers.

Moving targets

If your most valuable target audience is working well beyond traditional hours in the office, or is caught in a relentless travel cycle, they are unlikely to make themselves available, even if you catch them at home, for an interview on their media habits and brand recall knowledge.

However, Craig Harvey, director of media research Asia Pacific at Synovate, says there will always be a place for recall-based surveys in media measurement and argues a complete overhaul of the measurement system is unnecessary.

"The industry needs to understand how media works together, when multi-tasking of media is happening, which media they are more engaged with. It's all about keeping up with the consumer. Today's measurement systems need to understand how, where, when and why media fits into their lifestyles."

Feli Tam, buying director, OMD Hong Kong, says today media planning and buying is a mix of client needs, research and in some cases personal opinion. She echoes publishers sentiments that circulation and readership are no longer the most important factor when developing a media schedule.

"We look at circulation, readership profile, readership image and the quality of the magazine. Every success factor will be taken into consideration, it's not based on a single dimension," she says.

"In Hong Kong not every publisher is being audited, so from our point of view it's difficult to only use the circulation as the single dimension for measuring print titles. Some clients have their own experience in using the titles from response rankings and have their own preferences because of that image."