“It’s about the depth of your data”

Fantastic article (it’s about the depth of your data) on “qualitative data analysis”, clarifying the difference between “quali” and “quanti”, but also providing rich comments on the labour of fieldnotes when doing fieldwork. I want to emphasize this parragraph and then you can read the whole article:

Descriptions in fieldnotes need to be precise. Rather than using the word “said” for example (as in “She said”) we encourage sociologists to think more deeply about the way in which the communication unfolded. Did she seem angry, bored, thoughtful, unsure, frustrated, annoyed, or sad as she spoke? Was his tone of voice loud, boisterous, gentle, irritated, exasperated, discouraged, delighted, gleeful, cheery, or jovial?

Another great sentence:

Data analysis is ongoing and deeply entwined with data collection.

Here is the whole article:

A key strength of in-depth interviews and ethnography is obtaining textured insights into social phenomenon. Yet, many qualitative researchers try to invoke the reliability of quantitative methods by shrouding themselves in numbers as a way to legitimize their work. They offer up the number of interviews, the number of hours, weeks, and years spent in the field and they propose bigger and bigger samples. Even as qualitative researchers assert that they have carried out in-depth qualitative research, they often revert to the language of quantitative research to justify the legitimacy of the work. The nod to numbers is a way of claiming trustworthiness and, importantly, scientific expertise, which is usually equated with quantitative methods. This dependence on large sample sizes for qualitative research as a form of legitimacy, however, is misplaced. Indeed, we see this seeking of legitimacy through quantification as a distortion of where the value of qualitative research truly lies.

Instead, it is the depth of qualitative data that determines the quality of the work. Qualitative methods have the capacity to illuminate meaning—particularly the micro-level nuances of attitudes and daily behaviors. Qualitative research can highlight the impact of large-scale social structural forces on the rituals of daily life as well as many other spheres of life. This depth may in fact be linked to a larger number of interviews, or to more time spent in the field, but it should not be seen as reducible to this. We want to point to three factors that we see as being indispensable to achieving depth in qualitative research: collecting high-quality data, trenchant data analysis, and vibrant writing.

Qualitative data has the advantage of making readers feel they are hearing the interview or seeing the scene unfold in their presence. Our trust of qualitative data should thus rely (more than it currently does) on how vividly the researcher captures the micro-level nuances. The point is to study social interaction—how people act and how others react to their actions, as well as how people react to the reactions. Descriptions in fieldnotes need to be precise. Rather than using the word “said” for example (as in “She said”) we encourage sociologists to think more deeply about the way in which the communication unfolded. Did she seem angry, bored, thoughtful, unsure, frustrated, annoyed, or sad as she spoke? Was his tone of voice loud, boisterous, gentle, irritated, exasperated, discouraged, delighted, gleeful, cheery, or jovial? (A thesaurus is a crucial aid here.) The notes should be greatly detailed. Lareau’s rule of thumb for the data collection for Unequal Childhoods was to spend five to twelve hours writing fieldnotes after every two or three hour visit. At the very least, researchers want to take twice as long to write up their notes as they spent in the field.

These kinds of notes also can set the stage for description of interviewees. Although notes usually cannot be collected during the interview—since it breaks the flow and the connection between the respondent and the interviewer—they can be written immediately afterwards and must be done within 24 hours. As part of the creation of a high-quality data set, it is crucial to collect information on facial expressions, gestures, and tone of voice so as to better understand the social interactions being studied. It is also helpful to highlight the sounds, smell, and light in the setting researchers are trying to describe. Qualitative researchers want the reader to feel as if he or she is peering over the researcher’s shoulder to watch the events which are unfolding. But this kind of depth traditionally comes at the cost of scope in the number of sites and also in the number of interviewees. (Many researchers have concluded that they can keep about 50 people in their heads during data analysis.) Extremely large studies are difficult for one researcher to carry out, are expensive to transcribe, and are hard to represent through words. Smaller studies may create difficult decisions on balancing groups to study, but all studies involve hard choices. The goal is to achieve deep knowledge in a particular research setting.

Data analysis is integral to data collection in qualitative research. As the first bits of data emerge, researchers should read over fieldnotes and interview transcripts to search for emerging themes. Throughout the data collection process, researchers should consider the research question and try to figure out what interesting themes are surfacing. This analysis is almost always a pattern of discerning a focus (and letting go of other, interesting questions). But it is important to be skeptical as well. Researchers should search vigorously for disconfirming evidence to the emerging ideas. In the data analysis for Unequal Childhoods, for example, Lareau searched assiduously for middle-class, working-class, and poor families who had different behaviors than the general pattern for their social class. She found one white mother, who was raised in an affluent home, but—as a former drug addict living below the poverty level—her parenting style followed the “accomplishment of natural growth,” which was the cultural logic of child rearing in working-class and poor families. These and other examples increased Lareau’s confidence in her findings. In other cases, if researchers find disconfirming evidence, they need to investigate it thoroughly. Is it the exception that proves the rule? Or does it mean researchers should rethink their conclusions? Sociologists want to capture patterns that are decisively in the setting they are studying, and they want to be alert to variations on a theme. Writing memos, talking with others, giving “works-in-progress” talks all are helpful strategies to try to figure out what researchers are really doing in the field. Data analysis is ongoing and deeply entwined with data collection.

Our last point is that high-quality research is well written. Because doing qualitative research well is labor and time-intensive, it can be frustrating for scholars that they cannot share all of the collected evidence with the reader. Instead, researchers only share an extremely small fraction of the data. But qualitative researchers know that they have more data to support their claims than what they are able to present. This conviction is important. Yet, the writing and the quotes need to be judicious. Readers enjoy being told a story, and readers like to “connect” with a person in the text. Researchers who collect and analyze these rich details of social interaction are able to create clearer, more sophisticated arguments. Hence, it is valuable for these details to appear in the analysis. Too many studies using interview data prioritize including numerous quotes on the same analytical point as evidence of the robustness of their data. We find it better to present fewer people in more depth by helping the reader get to know a person in the study. For example, rather than presenting disembodied quotes in a research report, we believe it is ideal to help the reader understand who is speaking. Even in a brief fashion, the author can bring the respondent to life. This can be done by delving into the relevant back-story of the participant, detailing facial expressions, gestures, and tone of voice. Then, the common themes can be illustrated more briefly with evidence from others. Tables also can be a succinct way of capturing patterns in the sample with very brief quotes—often less than ten words—which illuminate key themes.

In the end, qualitative research is about words. It is not about numbers. The “arms race” to have bigger and bigger samples is unfortunate, since many researchers spend valuable time and energy collecting data only to leave it on the “cutting room floor.” Qualitative researchers need to evoke in readers the feeling of being there. But that knowledge of daily life comes from learning the details of a relatively small, non-random sample. It means systematically analyzing the experiences, looking for disconfirming evidence, and being sure that the patterns are solid. It also means bringing them to life through the written word. The value of qualitative research is not about brandishing the large number of cases in a study. Instead, qualitative researchers need to focus on the quality and the meaning of the data they have collected. This is the source of their legitimacy.

Annette Lareau is in the sociology department at the University of Pennsylvania. The author of Unequal Childhoods, she is writing a practical guide for doing ethnographic research.

Aliya Hamid Rao is a doctoral candidate in the sociology program at the University of Pennsylvania. She is completing a dissertation on how gender reverberates through the unemployment experiences of families of white-collar men and women.

Advertisements

Advantages of focus group

There are a number of broad advantages to the use of the focus group. These may be summarized as follows:

1. Group basis. They are an economical way of tapping the views of a number of people, simply because respondents are interviewed in groups rather than one by one (Krueger, 1994)

2. They can provide a “safe” forum for the expression of views, e.g. respondents do not feel obliged to respond to every question (Vaughn et al., 1996)

3. Simulation of everyday interaction. Focus group provides information on the “dynamics” of attitudes and opinions in the context of the interaction that occurs between participants, in contrast to the rather static way in which these phenomena are portrayed in questionnaire (Morgan, 1988) or in-depth interview studies. Top of that, such interacion may well be comparable in any way with everyday ineraction (Flick, 2009) and this is actually the point of the focus group, i.e. a simulation of everyday reality. It is attempted to collect the data in context and to create a situation of interaction that comes closer to everyday life than in an interview.

4. They may encourage a greater degree of spontaneity in the expression of views than alternative methods of data collection (Butler, 1996)

5. Check, balance and consistence. Interaction is not only between interviewee and interviewer (moderator) but among all participants. It increases the dynamic of the discussion which favors checks and balance among participants and “weeds out false or extreme views” (Flick, 2009). Somehow they correct each other. Additionally, it may provide with greater consistence to the focus group conclusions as long as all participants agree on a specific topics.

6. Participants may feel supported and empowered by a sense of group membership and cohesiveness (Goldman, 1962)

7. Less risk of bias by the interviewer. The interaction interviewer-interviewee always drives a risk of bias. Even when interviewer has a lot of experience, the responses are sometimes affected by the mere presence of an unknown person. This risk decreases in a focus group. Despite the obvious presence of the interviewer or so called moderator, the tendency is him/her to occupy a non-predominant role. Actually the moderator´s role is rather facilitate the group own dynamic and avoid that one person dominate the discussion. It depends on the research objectives but in some cases, the less involved the moderator, the better (Ibañez, 1986).

Despite the organization of a focus group may be seen as a simple meeting to discuss on a specific topic and so that doable even for non-experience researcher, most of the advantages above highlighted strongly depends on a number of issues to be cover in futures posts, such as the focus group design, group composition or the role played by the moderator. Bear in mind that a “well-conducted focus group research can determine the course of your company’s future” (Edmunds and Edmonds, 1999)

References

Carey, M. A. (1994). The group effect in focus groups: planning, implementing, and interpreting focus group research. Critical issues in qualitative research methods, 225-241.
Edmunds, H., & Edmonds, H. (1999). The focus group research handbook. Chicago: NTC Business Books.
Goldman, A. E. (1962). The group depth interview. The Journal of Marketing, 61-68.
Gutiérrez, J. (2008). Dinámica del grupo de discusiónCuadernos metodológicos del Centro de Investigaciones Científicas, (41).
Merton, R., Fisk, M., & Kendall, P. (1956). The focused interview: a report of the bureau of applied social research. New York: Columbia University.
Morgan, D. L. (1998). The focus group guidebook. Sage.
Ragin, C. C. (1994). Introduction to qualitative comparative analysis. The comparative political economy of the welfare state, 299, 300-9.
Vaughn, S., Schumm, J. S., & Sinagub, J. M. (1996). Focus group interviews in education and psychology. Sage.
Article updated on 24th of April 2014

Difficult interview participants and possible solutions

1. Participants are willing just to give monosyllabic answer, “yes”, “no”, “maybe”. Formulate questions as open as possible or use long pauses in silence to let them know you want hear more.

2. Participant become upset or start to cry. Explain that the question does not have to be answered. Avoid finalize the interview in this precise moment and just move on a different topic. It could make him/her feel guilty and cause greater damage.

3. They provide long answer and digress from the guide. Impose your direction discretly, without cause offence. For example, refering back to an earlier relevant point.

4. Participants starts interviewing you. You need to stress that you are interested in their opinion and that yours maybe provided at the end.

Reference

Lewis, Philip, Mark NK Saunders, and Adrian Thornhill. Research methods for business students. Pearson, 2009.

What is a market research interview?

What does first spontaneously come to your mind when listen the word “interview”? May be a conversation between a journalist and a relevant personage on television? May be a job interview? Although an interview for market research may have many things in common with a job interview, there are a number of particularities to bear in mind. A market reserch interview is a purposeful discussion between two or more people (Kahn and Cannel, 1957) Bellow you can see what are the essentials of an interview.

1. Open-ended questions. Unlike in a questionnaire, the interviewee is not forced to choose among a set of options like bellow:

  1. Strongly disagree
  2. Disagree
  3. Neither agree nor disagree
  4. Agree
  5. Strongly agree

On the contrary, the interviewee has absolute freedom to answer and does not need to stick to any preset categories. Actually, this is the essence and advantage of such technique, i.e. the possibility to obtain a spontaneous and non-conditioned answer. Some of examples are:

  • “Tell me about your relationship with your supervisor”
  • “How do you see your future?”
  • “What is the purpose of government?”
  • “Why did you choose that answer?”

3. Representation. When doing a research on trainers shoes consumptions, you must interview young people, adults and elder, or whatever other relevant categorization. It is going to assure a correct representation of the phenomena. Take into account that representation does not imply representativeness. When applying interviews in your research, the sample is usually low, 15 or 20 cases is enough. But these figures are insufficient to do inferences. On the contrary, other techniques like survey are precisely designed for reliable inferences.

4. Interaction. As well as other qualitative techniques as focus group, interaction is both the main advantage and handicap. On the one hand, the direct and close interaction interviewer-interviewee allows a deeper understanding. However, the way interviewer behaves may affect the answers. How do should behave? This question will be answer in future posts, but here is enough to say that a good interviewer must be neutral and refrain from giving his or her personal opinion.

Reference

Kahn, R. L., & Cannell, C. F. (1957). The dynamics of interviewing: theorie, technique, and cases. Cited in Philip, Mark NK Saunders, and Adrian Thornhill. Research methods for business students. Pearson, 2009
Martínez, P., & Rodríguez, P. M. (2008). Cualitativa-mente. ESIC Editorial.
VALLES, M. (2002). Entrevistas Cualitativas, CIS, Cuadernos Metodológicos nº 32S, Madrid. Trabajo personal de lectura y comprensión Trabajo personal para obtener informaciones de diversas fuentes Trabajo personal de análisis y síntesis.

Why qualitative research? (Case study and solution)

The qualitative approach within the market research industry is of increasing importance. Why this approach is required in many occasions?

Tempranillo varietal wine bottle and glass, sh...

Tempranillo varietal wine bottle and glass, showing colour Shot with Nikon D70s (Photo credit: Wikipedia)

CASE STUDY¹. Imaging your company want to export Spanish wine to Poland. In this country, it is well known that either beer or vodka are more popular drink than wine, but nothing is clear about the Polish delight in wine, and even less in Spanish wine. For this reason, the company has contacted a market research agency and they plan to develop a telephone survey of a Polish population representative sample. Results show that just 10% of population drinks more than one glass of wine a month, instead of the 20% in other eastern European countries. Also, results show that the consumption of wine is less common in the group between 25 and 35 years old. The researcher seems to have a clear marketing strategy.

But something important is still up in the air. How to address this population? A qualitative approach has complemented the quantitative data by mean a number of focus groups in Warsaw. The idea of the research is gathering at least eight people between 25 and 35 years old that do not consume wine usually and another group of eight people at the same age who consume wine at least once a month. Questions as “habits of alcohol consumption in general”, “willingness to increase wine consumption”, “reasons to not consume” were asked. Below you can see some of the sentences that were listened to in the discussions:

“…I do not drink wine because I never know what kind of wine goes with each food…”

“…I have a feeling that wine is more for old people, and I feel young…”

“I do prefer to support Polish industry of vodka and beer”

“…yeah! and also for high standard of living people, I feel that beer is more…how to say…more akin to my people”

“…bottles information are not translated into polish so I do not even know where this wine is from”

“…I get drunk too fast! beer is more kind of easy-going…”

“…I really like wine but all my friends prefer beer so… I don´t want to be a weirdo when go to a party…”

QUESTION: WHAT MAKES THE QUALITATIVE APPROACH DIFFERENT FROM QUANTITATIVE ONE?

________________________________________________________________________________________

Solution

The quantitative approach is very important to quantify the reality. The representative percentage of wine-consumers in Poland or the frequency of consuming is unobtainable but by mean questionnaire or statistics. Ad hoc questionnaire may help us also to understand opinions and attitudes of people toward your product by mean opinion scales as well as including some open-ended questions.

However, three factors make this approach inappropriate when investigator sought a more detailed opinion:

1. Time: especially in phone questionnaires, the interviewees feel generally in hurry either by the surveyor or by him/herself. On the contrary, the more calm and tranquil atmosphere provided by qualitative methods as interview or focus group, encourages a major commitment and engagement.

2. Interaction: while the interaction in quantitative methods in occasions does not even exist or is limited to a phone conversation, in the qualitative one, the number of interaction increases exponentially, allowing at the same time a major capacity to adjust questions and improvising new ones as the interviews progress. In the case of the focus group, the interaction is produced also among the members of the group which represent a great advantage of this technique, since many of the most profound opinions we have are just seen clearly by ourselves after discussing with others.

3. Depth of the analysis: quantitative approach usually deals with countable behaviors or resources: number of wine bottles sold in Poland last year, number of wine glass per week, etc. it hardly allow researcher gets know about emotions, values and beliefs. Making use of the iceberg metaphor, focus group, interview, but also the analysis of comments in forums or just graffiti in a wall expresses better than nothing our deepest view of the world.

Finally: critical reflection for a organization manager: What values do your brand transmit? Is your organization media strategy connected efficiently with your buyers? What is more, is your organization media strategy connected with your potential market?  Just when a company reach to understand its target´s deepest and detailed opinions, values and beliefs, the marketing and communication strategies implemented may cause a truly impact on the sales.

References

Flick, U. (2009). An introduction to qualitative research. Sage Publications Limited.
Martínez, P., & Rodríguez, P. M. (2008). Cualitativa-mente. ESIC Editorial.
Silverman, D. (2011). Interpreting qualitative data. Sage Publications Limited.
1 This case is not real, although some of the sentences were taken from real cases