This swell article on SMH yesterday arvo enlightens us on the fact that Australians no longer only use mobiles for phone calls & texting, lots of us are using it for email, web browsing and entertainment. These kinds of reports are critical to my job as it helps make or break business cases on where clients should invest their money.
Some interesting results in the research report got my attention including:
- 9% of participants have iPhones
- 32% of respondents accessed social networking sites from their handsets
- and half of those accessing social networks daily
However, skepticism started to creep in as the second stat seemed a little out of kilter with general social networking site usage in Australia…. So I dug a little deeper into the research report. One of the first places I usually look (and I’m no researcher) is who they surveyed – sometimes this skews the results – and in this case there was an issue.
The decision to implement the survey online and to promote the survey via mobile banner ads has some implications in terms of disseminating the results to the wider population.
SOME IMPLICATIONS?
I’d say a little more than SOME IMPLICATIONS – considering the survey is about mobile usage.
If you do a survey of people who are clicking on mobile banner ads it’s pretty stupid to ask them if they use mobile internet and applications – THEY JUST CLICKED ON A BANNER AD.
If AIMIA or the research agency can clarify the number of people that responded via mobile devices the report might be of some value – maybe not.
Footnote: I’m a digital advocate, believe in all things online/mobile/social, but flawed research reports make it difficult for me to judge the potential value of the research & therefore difficult to recommend it to my clients.
UPDATE: 13th October
Marisa from mNet has clarified the issues that I raised in this post in the comments below. Until I’ve reviewed the report in further detail assume that the research is sound. My question is why they didn’t clarify this important fact in the report.
implications in terms of disseminating the results to the wider populationThe decision to implement the survey online and to promote the survey via mobile banner ads has some
implications in terms of disseminating the results to the wider population
September 29th, 2009 on 9:21 pm
Hi Simon,
Good spotting!
Research is a funny thing. Unfortunately, more often than not, the truth gets in the way of a good story.
I always find it useful to start exactly where you did – who did the research and what would they ‘want’ it to say. Then, how will that benefit their business. The more often readers start at this point the more honest research will become.
Cheers,
Carmen
.-= Carmen Campbell´s last blog: Living Large – The World of Harold Mitchell =-.
September 29th, 2009 on 9:33 pm
The research itself is sound, it appears that the SMH have taken some creative license in interpreting the results and forming their headline.
Knowing that only 14m aussies are online (as per Nielsen – Aug 09), it’s a big call to state that “Aussies call an end to just phoning on mobiles”.
I guess the SMH are all about getting the click, which I’m sure we can all be guilty of at some point…
.-= Nick Smith´s last blog: _nicksmith: Spellcheck: GraffitiGeo =-.
October 9th, 2009 on 6:37 am
Hi Simon, my name is Marisa and I am the key author of the report. I would like to assure you that the research design, implementation and analysis are sound.
My response to your specific concern follows.
The perceived problem -
you have questioned the validity of the results of the AIMIA mobile phone survey, based on the fact that some respondents were made aware of the survey via the mobile phone. You ask how ‘we’ can say that 50% of people use the internet on their mobile phones, when the sample was made up of people who accessed the survey on the mobile web?
So is there a validity problem – the short answer is no!
There is no significant difference between the reported mobile web usage of those that accessed the survey via an email link (which includes those that became aware of the survey via mobile) and those that accessed it via a web-link online. We can therefore conclude that it is unlikely that the results have been biased as a result of the survey being advertised via the mobile.
Where is the evidence?
1) The samples from the last five studies were sufficiently similar to allow cross year comparisons. Despite the fact this year the survey was advertised on mobile for the fist time, as well as online the demographic profile of the respondents remains very similar.
2) We separated out the overall sample into two groups: 1) those that accessed the survey via email, and 2) those that accessed the survey via a web-link.
People who accessed the survey on the mobile phone were sent a link to the survey via email; as were those who were sent a link to the survey from opt in databases. Unfortunately we can’t separate them. While this is not ideal, the results below suggest that there is no difference in mobile web usage based on how recipients became aware of the survey. Had the inclusion of mobile as means to promote the survey impacted on the results, we would have expected to see some kind of significant difference between the two groups.
Of those that accessed the survey via a link sent to them, 52% said they had accessed the mobile website in the past year.
Of those that accessed the survey via a web link 54% said they had accessed a mobile website in the past year.
October 12th, 2009 on 4:36 pm
Hi Marisa,
Thanks for your follow up comment, that does seem to resolve the key issue I raised, I’ll continue to review the research now and report back my views.
Thanks