Bigger Picture Research

Last week I came across the guardian.co.uk’s Data Store for the first time (along with its companion Data Blog).  If you haven’t visited I can recommend it as a splendid use of the Scott Trust’s money. It’s free to you and me.

The idea is simple enough. Using web 2.0 technology, guardian.co.uk aims to make ‘important data more accessible to people’, by publishing facts and figures from publicly available data sources, and encouraging users to analyse, re-interpret and share the results. The site even invites you to mashup the data, which as far as I can tell is a bit like a fry up but with more potato and a lot less saturated fat.

Now if truth be told, I don’t have a statistical bone in my body (unless, of course, you count my digits). Nonetheless I find this a really exciting development. There’s even a Guardian Data Talk Group, ‘to discuss data, tools, formats, standards, how to use the data The Guardian publishes, where to get it, etc.’. Judging by the archive, the Group hasn’t seen that much activity since it came online in late February, but it’s good to know it’s there should the need ever arise. I’ll certainly be chipping in.

If you’re wondering what any of this has to do with the usual film-related themes of BPR posts, I only came across the Data Store by chance while chasing a link to Charles Arthur’s technology blog post about music piracy, in which he uses Data Store numbers to advance the idea that file sharing is not cannibalising music sales to the extent claimed by the music industry.

Arthur contends the decline in music revenues since 1999 has been driven by people switching their spending to other forms of entertainment, notably video games and DVDs. It’s a plausible idea, although as presented it remains only a correlation that may in reality be no more substantial than the view he seeks to challenge. (What if people are spending more on video games because they can get their music for free?). But I think we should welcome the fact these issues are being raised and debated within the context of a growing community of data sharers and mashup artists (now there’s an image to conjure with). So three cheers for guardian.co.uk.

All this has set me thinking that it would be great if all public agencies made their data available like this, actively encouraging users to analyse and share results in interesting and novel ways, rather than issuing statistics in static publications and news releases. Even if they don’t have the web 2.0 infrastructure to do it themselves, perhaps public data owners should donate their holdings to the Data Store, or equivalents. This would deliver genuine public value, and the quid pro quo is that data owners may even learn something new about the information they hold, if enough people run inspired and innovative analyses.

Anyone who has ever visited the lamentably labyrinthine and obstructive web site operated by the Office for National Statistics (ONS) surely must agree that a bit of community datasourcing (crowdsourcing for anoraks like me) has got to be an improvement. The Data Store has left me hungry for more.

Pie chart and mashup, anyone?

'Impact' crops up every once in a while in movie titles. Within recent memory we’ve had Deep Impact (perhaps Shallow Glancing Blow might have been more apt, for all the effect it had on audiences). Then there’s Double Impact, which serves up twice as much of Jean Claude Van Damme as any self respecting movie goer probably deserves. And who could forget Dirty Harry’s fourth outing in Sudden Impact?

The reason for this noun’s popularity is obvious enough: it conveys a certain kinetic energy, lending whichever film it adorns a flavour of dynamism and the anticipation of consequential action. Which is also why the term has found favour in policy and advocacy circles, deployed by trade associations, film commissions and other screen agencies around the globe in their efforts to marshal evidence in support of their cause.

The most recent addition to the literature was published a couple of weeks ago by the Motion Picture Association of America (MPAA). ‘The Economic Impact of the Motion Picture & Television Industry on the United States’ is the second of its biennial studies seeking to establish evidence of the instrumental value of the movie industry in the US. This follows similar studies in the UK (and elsewhere), most notably a report by Oxford Economics on behalf of the UK Film Council and Pinewood Sheperton, entitled ‘Economic Impact of the UK Film Industry’ (July 2007).

As one would expect, the MPAA pulls no punches in its press release. Headlined ‘MOTION PICTURE AND TELEVISION INDUSTRY ADDS JOBS, WAGES IN ALL 50 STATES’, it leads with the idea that US film and TV production is a nationwide enterprise, thereby deserving the support (including tax breaks) of every state in the union, and not just California and New York.

“People don’t think of the business of making movies the same way they think of other American businesses such as manufacturers or large retailers,” Dan Glickman (MPAA CEO and Chairman) is quoted as saying. “However, our industry in many ways is just like every other – we employ millions of people, we create jobs all across America, and yes, we too feel the sting of declining economic times.”

So the message is clear enough for those on Capitol Hill: in these economically straitened times, please don’t forget about the film and TV industries. In your haste to prop up the automobile industry et al, spare some thought for this other great, all-American enterprise.

It is plainly in the MPAA’s interest to present the best possible picture of the film and TV industries’ contribution to US output and trade, but that doesn’t necessarily invalidate the research.

To my mind it’s actually a rather effective report- clear and direct, with enough methodological detail in an appendix to satisfy the curious and establish its credentials. The study draws upon various data sets, filleted and collated by the MPAA, including information from primary sources, like the major studios and TV networks, as well as government agencies (e.g. the Bureau of Labor Statistics) and other proprietary data providers, and it uses input-output multipliers to calculate Hollywood’s impact on employment and economic activity.  

In terms of the key findings, the report concludes that ‘the production and distribution of motion pictures and television programs is one of the nation’s most valuable cultural and economic resources’. Citing a range of indicators as evidence, the report reveals that in 2007 the motion picture and television industry was responsible for:

• 2.5 million jobs in the US (that’s 285,000 in feature film and TV production, marketing, and distribution; 478,000 in related businesses, including employees in cinema exhibition, broadcasters and cable companies; and 1.7 million employed in occupations indirectly linked, such as clothing retailers, catering, florists etc.);
• $41.1 billion in wages to workers in the US;
• $38.2 billion in payments to US vendors and suppliers, small businesses and entrepreneurs;
• $13 billion in income and sales taxes; and
• $13.6 billion in trade surplus.

These are indeed impressive headline statistics. And the report offers other fascinating insights. For example, production employees took home an average annual salary of $74,700 in 2007. That’s just over £50,000 at today’s exchange rate, which compares favourably with average feature production income on this side of the pond. Last year Skillset found those involved in making UK feature films took home on average around £28,000.

In keeping with its core purpose, the MPAA report provides a snapshot of production activity right across the US. Thus, we learn that Illinois, Texas and Florida are placed third, fourth and fifth in the production rankings, after California and New York (although it is not clear by what measures they are assessed as such).

Such is the strength of belief that production activity can have a beneficial effect on the local economy, forty states plus the District of Columbia have now enacted fiscal incentives to entice projects within their dominion. Such a strategy can pay dividends: for example, the state of Illinois, which hosted production of The Dark Knight, was home to 72 feature film and TV productions in 2007, with wages for those involved totalling over $1 billion.

As well as the bald numbers, the report is illustrated with a range of case studies. Returning to the previous example, The Dark Knight spent over $35 million in the Chicago economy, generating $17 million in payments to 793 local businesses including hotels, catering, office rental and building supplies (NB this is a film recognised as British by the authorities over here, which enabled it to benefit from the film tax credit available on UK expenditure).

Reference is also made to the positive effect of film and TV production on tourism, a theme explored in a study conducted by Olsberg SPI and published in 2007 by the UK Film Council (‘Stately Attraction: How Film and Television Programmes Promote Tourism in the UK’). According to the MPAA report, twenty years after Field of Dreams was released the eponymous cornfield in Dyersville, Iowa still gets around 65,000 visitors a year, bringing income into the area. If you film it, they will come.

The report is not, however, entirely without fault. To take one example, the authors claim that in essence ‘the motion picture and television production industry is largely entrepreneurial and dominated by small business and individuals.’ It is undeniably entrepreneurial, but I wonder in what sense is it ‘dominated’ by small businesses? The report is referring to the number of SMEs operating in the industry: 115,000 in total, according to estimates, 81% of whom employ fewer than ten people. Yet these businesses are highly dependent on a much smaller number of multinationals and corporate behemoths, those with the power to commission or green light projects and who effectively control the international means of distribution, including the studios, TV networks and larger theatre chains. Only in number do SMEs ‘dominate’ the film business; in all other respects it is the MPAA’s core membership, 'the six major producers and distributors of motion picture and television programs in the United States', that hold sway.

So what is the likely impact of this impact research? It's too early to say, and only the MPAA will be able to answer for certain. Over the coming months, lobbying for government favour and public money, in the US and in every other developed nation, will doubtless preoccupy film trade associations, screen agencies and the like. Yet one thing every party should bear in mind is that no matter how well conducted a piece of economic impact research, the methodology has inherent limitations. These are neatly summarised in a briefing paper issued by Screen Australia (when it was the Australian Film Commission) entitled ‘The Economic Contribution Of A Film Project: A Guide to Issues and Practice in the Use of Multipliers’.

With admirable concision, the paper concludes that ‘when using [economic multipliers] it must be recognised that they are estimates only and should be treated as such. To treat them as authoritative measures is a mistake but as a general indicator to suggest possible broader impacts, especially in the absence of the time and resources required to formulate more thorough studies or indeed in the absence of better measures, they are a comparatively simple and relatively effective measure’.

Simple + effective = impact.

This week law firm Olswang published the results of its annual ‘convergence’ survey, examining UK attitudes to different media platforms. Taken together the findings indicate that viewer choice and control are important drivers of consumer satisfaction, which suggests anything impeding easy access to content, like Digital Rights Management or release windows, is a demand-side no-no.

The research is based on a survey of 1,162 respondents (235 children aged 13-15 and 927 adults) conducted by YouGov between 25 September and 3 October 2008. YouGov also hosted a number of focus groups, designed ‘to explore in more detail the effects of convergence on the activities of distinct types of consumer’. These groups include ‘kids’, ‘mainstream’ adults, ‘tech vanguard’ and ‘laggards’. This last category comprises late adopters, rather than Luddites; the survey and focus groups were conducted online, so all respondents had access to the Internet.

Bear with me on this, I will get round to discussing statistics. But first up, a further word on the continued good fortune of Mamma Mia! (2008) at the UK box office.  I posted on this subject a couple of weeks ago and the gist of the story remains the same: namely the astonishing business this film has seen since its release 9 weeks ago.

According to the latest weekend box office update from the UK Film Council, the film grossed £1,431,775 last weekend (5-7 September), with a very respectable site average of £2,898 (it played at 494 cinemas, down marginally from 505 the week before). This pushed its cumulative box office takings to £57,687,650, comfortably past Casino Royale’s haul of £55.5 million in 2006, and it has just topped the total achieved by Lord of the Rings: The Two Towers in 2002 (£57.6 million).

Anecdotal evidence (and common sense) suggests this remarkable run has been achieved as a result of a good number of happy punters returning for second (or third or more) helpings. But much as we might like to, we can never know the full extent of such repeat business. Box office gross simply tells us the combined value of ticket sales. In fact, we can’t even say for certain how many tickets have been sold, as no-one tracks title-by-title admissions in the UK (unlike in France, where the CNC has such information at its fingertips).

In follow up to the previous post, with its silly season theme, I’d like to draw attention to a timely piece that caught my eye on the BBC news magazine site. Michael Blastland (author, with Andrew Dilnot, of The Tiger That Isn't: Seeing Through a World of Numbers) gives some handy tips for seeing through the misreporting of research findings.

He cautions readers against taking such reports at face value, encouraging us to compare the claims made about research findings with the research methodology employed, to see if the two stack up.

In lesson one Blastland discusses some of the common flaws in reporting survey data. ‘The news’ he notes, ‘is chock full of surveys. Many are blatant self-promotion and/or statistical garbage’. He's perfectly right- we should be wary of such things, especially at this time of year when there is a surfeit of quack research in place of proper news stories.

A news story that was doing the rounds at the tail end of 2007 has recently resurfaced (see Guardian.co.uk and Telegraph.co.uk). Ever mindful of the common man (or non-gender specific person), the Local Government Association has declared war on jargon, issuing a list of words to be banned from the Local Authority lexicon.

Fair enough, you may say. Language should help ease the passage to understanding not block the way like an impenetrable thicket. And scanning down the list of proscribed terms (or ‘non-words’ to use the LGA’s own jargon) one can readily sympathise: so out with ‘predictors of beaconicity’ and shame on ‘coterminosity’! Yah boo sucks to ‘Performance network’ and a big fat raspberry to ‘improvement levers’.

But before the literary lynch mob gets too carried away we should remember there is a difference between buzz words (empty neologisms) and technical terms.