TV Ratings

This is something that I've been thinking about lately.

Is there an over reliance on the current TV ratings system?
Is it a fair representative take a percentage of a small selection and multiply them into the millions?

I sometimes wonder how many programmes have been watched by high audiences, but axed because hardly any of the barb panel chose to watch it or found if interesting.

Are there any known alternative technologies being developed perhaps to add to set to boxes to record a higher number of audience?

How does our system compare to those on the continent and in the States?

No system will be 100% effective, but the sample is significantly large to give a result which in theory is 99% accurate. I've forgotten most of what I learned regarding statistics, but basically as you'd expect the larger a sample, the more accurately it reflects the true picture - but once you get to a certain point increasing the sample doesn't necessarily increase the accuracy.

I believe Sky have their own internal system which I assume collects data straight from subscribers digiboxes - though whether that's a sample of customers or all customers I don't know. I'm not sure what the legalities would be about collecting data about what people are watching if they don't know the data is being collected.

I'm struggling to remember my statistics too, but an online thingy I found suggests that to be 99% sure of being within 1% of the true figures, you need to sample 16,000 people in the UK. I seem to recall a figure of 5000 homes being bandied around a few years ago, so that sounds about right.

David

Its a fair system to use because of the diversity of the panel, but I don't believe the 5% error margin for a second. I think the ratings are at least double what we get told. I don't understand how 11,000 people in just over 5000 homes can represent 60m people, and I don't understand why BARB can make such claims of accuracy.

In terms of channels I think Channel 4 and digital channels are under represented, whilst in terms of demographics I think younger viewers are misproportionate.

Statistics is a complex and often counter-intuitive subject. This is BARB's bread and butter, so I think it's safe to assume, lacking evidence to the contrary, that their claims are broadly accurate.

David

For 11,000 to be representative of 60,000,000 people is just illogical and unrealistic. Studies in other areas using a similar sized sample are often dismissed for being too limited. You can never truly get a good enough cross section and the only way to truly know what is going on is to survey every single individual. Otherwise you're prone to big gaping errors. Its just a basic of statistics, its only informed projections and never fact. But I'm not opposed to the BARB system, its probably the closest we'll get. I just don't think its 100% reliable that's all.

I'm not being funny, but if you say that to a statistician you'll get a blank look, because you'll need to back up with - perhaps ironically - statistics.

It's like that old puzzle - how many people do you need to put in a room to have a 50% chance of two of them having the same birthday? The answer surprises a lot of people.



In all seriousness, can you give a specific example?

David

As with all forms of data gathering it's who/what you measure that matters far more than the quantity of measurements you take. For example, if you ask a bunch of Daily Mail readers a bunch of questions about immigration, you're not going to get an accurate representation of the country's opinion. Or if you wanted to do a survey on traffic levels across a town, setting up outside the busiest car park in town, or at the bottom of a cul-de-sac in a housing estate is not going to provide usable data. A significant amount of work has to go into deciding how to chose your sample set, and for all but the simplest of surveys random choice is insufficient.

In this case, if the 5,000 homes accuratly represent British society at large, then chances are the results will be accurate. Remember the main reason for accurately measuring viewing figures is not to keep producers happy, but to justify advertising costs, and I'm pretty sure if the advertising companies thought they were getting short changed they would sort out an alternative method of measuring.

As for ratings being double what is reported, where do you think the people come from?!

The birthday paradox is completely different, that's due to probability rather than proportionate statistics. Most research I've come across where a large sample size is criticised tends to be psychological, I'm sure you'll find some of it if you look hard enough/are genuinely that interested. A fair amount of it is surrounding IQ testing.



But you couldn't take a few readers of every paper and then use that to form the 'general' opinion either. I do appreciate the diversity of the sample, I just think that the raw numbers are lower than in reality but the percentages are about right. Whether or not they actually are is probably irrelevant because its an accepted system and with such a system all the ratings are relative to each other. As for advertisers being short changed, if more people are watching then surely they'd be getting a better deal?

Just to put this into perspective, putting aside your personal preferences do you genuinely believe that last year 40m people in the UK chose not to watch X Factor?

It's not for me to find anything - you made the claim.

Here's another example: suppose you wanted to know how many people did or did not (i.e., a yes/no question) watch the Doctor Who Christmas Special. You only need a sample size of 16583* to be 99% sure of being within 1% of the true answer (mathematical fact!).

(*assuming a population of 60m)

What's realy mental, to me, is that if the population of the UK was 100 (or even 10,000) times higher, you'd still only need a sample size of 16588 (i.e. five more people ) to reach the same level of accuracy. But it's true



If "a few" means >0.02% (see above) then yes, you can.



Yes, quite readily - the question is, why don't you?

(answers beginning with "I [don't] think" won't be accepted )

David

Yes.