My six years spent in Human Computer Interaction meant that I am familiar with statistical/mathematical approaches to solving scientific problems. I spent the latter part of those years diving into ethnography, and my PhD doing all that.

State of Play 2009 and some discussion with colleagues from engineering background revealed a lot of gaps in understanding of ethnographic and statistical/mathematical to problem solving. These, I wish to discuss.

I like the way Angela Tan from Defense Science Organization, Singapore, described ethnographic method as adapting to "how the situation unfolds”. Believing that a method can be linearly apply to something is a luxury only some scientific problems can believe in. Usability, for instance, is one. Because the question of usability is narrowly constructed, it can be described in as little as two dimensions – speed and accuracy. Experiments will then be pretty linear, users will be timed, users have to do a series of tasks. Even in such situations, Lucy Suchman had said, users in real life do not perform tasks in a linear sequence.

Worse. In real life, every social situation is multi-dimensional. How would we know what to measure if we do not know what users care about? Most businesses care about speed and accuracy, users don’t. Which is why HCI has been successful in industries such as process control, firefighting, aerospace, and military. In other businesses, such as gaming, which don’t always care about speed and accuracy, a pure statistical/mathematical approach would fail. When a problem is multi-dimensional, the ethnographer has to be adaptive. But this does not imply “doing whatever we like.” Real anthropology is based on a lot of drilling in theory of knowledge, culture, and psychology (of which there are many schools and therefore many perspectives of anthropology, which is another issue altogether). Based on the theoretical perspectives, our users (e.g. their concerns) and the place they reside also shape our method.

Engineers often say that ethnography cannot be replicated. It can! So long as the subsequent studies are done in the same context, including users. Because apart of theoretical approaches, ethnographers follow the lives and constraints of the users and more of less experience it for themselves*. If another ethnographer performs the ethnography in the same context, the same constraints should follow (more of less). If the ethnography is done in another context then a new approach should be mapped out.

This points out that any theory that does not consider environment as a factor that shape user behaviors as unsuitable for ethnography. This includes a lot of cognitive theories in Engineering Psychology. But the newer ones such as distributed cognition and ecological approaches are more suitable.

However, ethnography is not everything. I often like to think that ethnography is "accurate" but not "precise." It tells us how users feel and often convey too much information. Also, we found the data set too thin to precisely tell us if we need to take action. Is the 15 users interviewed representative of the entire population? I like the more recent suggestions that ethnography need to be extended and verified using either multiple ethnographers or mathematical techniques such as Social Network Analysis.

So said, ethnography and statistical/mathematical should go hand in hand. I think some of the best studies have both elements. I hope that practitioners would one day come together to ensure the next generation of students are well equipped with both apparatus.






*This is in fact an argumentative issue. I do think that ethnographers should have, to some extend, a level of openness and objectivity as part of his or her cognitive capabilities. However, this issue has never been discussed. We often hear, "anyone can be an ethnographer, so long as you have a new perspective." I think this view is inherently flawed and led to too much misunderstanding of the field. I would argue that ethnographers who are able to let go of their own prejudices would more or less get the same outcome from studies in the same context.