Assessing persona prevalence empirically February 21, 2009Posted by cnchapman in Blogroll, Market research, Technology, User research.
1 comment so far
I just obtained permission to post our latest paper on Personas. We argued previously that the personas method should not be considered to be scientific, and that a complete persona almost certainly describes few people or no one at all. In the new paper, we present a complete formal model, and evaluate the prevalence of “persona-like descriptions” with both analytical methods and empirical data. Full paper on persona prevalence.
There are two key implications here: (1) if you want to claim that a persona describes real people, you need strong multivariate evidence. (2) Without such evidence, we provide a formula you can use that will give a better estimate than simply assuming something. We show how this formula has a better than chance agreement with 60000 randomly generated persona-like descriptions in real data with up to 10000 respondents.
None of this says that personas are not inspiring or useful. It just says that they cannot be assumed to have verifiable information content, unless that is demonstrated empirically. As for alternatives to answer key design and business questions using empirical data, check out our paper on quantitative methods for product definition.
Personas February 8, 2009Posted by cnchapman in Blogroll, Market research, Technology, User research.
add a comment
One of my papers from 2 years ago is still causing discussion: “The Personas’ New Clothes: Methodological and Practical Arguments against a Popular Method” by me and Russ Milham. Email from researchers I didn’t know led me to look up citations, and the article appears to be commonly cited when people present criticism of the personas method. Google search. The paper itself is here.
There are a few misunderstandings of our position out there. Our basic argument is simple. Persona authors often make two claims: (1) personas present real information about users; and (2) using personas leads to better products. In a nutshell, we argue that neither claim has been supported by empirical evidence; rather, the claims for personas’ utility are based on anecdotes, generally from their own authors or other interested parties (such as consultants selling them).
This does not mean that personas are bad, but they cannot be taken at face value. As researchers, we suggest that persona authors should either provide better evidence (and we suggest how) or make weaker claims.
Some persona users don’t make claims about their personas’ usefulness or correspondence to reality; they simply say that personas might be helpful for inspiration for some people or teams. We take no issue with that, as long as they don’t forget those caveats and reify the persona. Unfortunately it is probably very difficult for people to read a persona and not think that it describes a user group.
We’ve recently published empirical work on (quasi-)persona prevalence using several large datasets, demonstrating that once a description has more than a few attributes it describes few if any actual people. I’ll put that paper up as soon as I get reprint permission. (If you have access to HFES archives, it is “Quantitative Evaluation of Personas as Information”, Christopher N. Chapman, Edwin Love, Russell P. Milham, Paul ElRif, James L. Alford, from HFES conference 2008, New York.)
What should one do instead of personas? I advocate stronger empirical methods that have more demonstrable validity.
New papers on user research February 7, 2009Posted by cnchapman in Market research, Technology, User research.
add a comment
Just uploaded 2 new papers on user research. First is work on a multi-factorial product interest scale, designed to be easily administered in survey format and applicable to consumer products. See the abstract on my “papers” page, or get the file directly: wip337-chapman.pdf
Second is an overview of quantitative methods that are helpful in early evaluation of product needs and strategy. The abstract is on my “papers” page, or the complete file is chapman-love-alford-quantitative-early-phase-ur-reprint.pdf
I’ll be uploading more papers soon.