|
Defined: A case study is a detailed examination of a single person, group, institution, social movement, or event. For convenience of discussion, case studies can be divided into two main types, with each type divided into a pair of subtypes. The first of the main types focuses on an individual's behavior and experiences. The second concerns the operations of a group, institution, or social movement. Under each of these types, a researcher can assume the role of either an outside observer or a participant-observer. A case study in which the researcher serves as a participant-observer within a group or institution qualifies as a typical ethnographic study, when ethnography is defined as the "descriptive study of a particular human society or the process of making such a study. Contemporary ethnography is based almost entirely on fieldwork and requires the complete immersion of the [researcher] in the culture and everyday life of the people who are the subject of the study" ( Ethnography, 1994, p. 582). General Purposes: Case studies are differentiated from historical research, biographies, and autobiographies chiefly by their time frame and emphasis. Whereas historical studies, biographies, and autobiographies typically encompass a span of several decades, case studies center attention on a limited time range-from as short as an hour to as long as several months, but seldom more than a year or so. In terms emphasis, the intent of histories, biographies, and autobiographies is usually to trace cause-and-effect relationships as they develop across the years, whereas the aim of the typical case study is to show how-during a restricted period of time--people interact and relate to their physical/social environments. Studying a single person from outside Purpose: One well-known example of this subtype is Roger Barker book One Boy's Day ( 1951) which depicts the way the different environments that a boy inhabited during a typical affected behavior. Another example is a master's thesis by Christine Williams entitled A Modern "Ship of Foods": An Account of One Boy's Journey through the Mental Health System ( 1981). Each of these reports the experiences of a person as recorded and interpreted by a researcher who has followed the individual around to observe his or her transactions with the people and environments he or she encountered. Procedure: In conducting an outsider's view of a subject, a researcher can gather information by observing the subject directly, by interviewing other people who know the subject, by analyzing items produced by the subject (diaries, letters, reports, art objects), and by inspecting documents in which other people offer appraisals of the subject's behavior and traits (school report cards and cumulative records, social workers' reports, medical records, police files).In approaching the data collection task, researchers can either (a) start with a mind-set (such as a theory) that guides what they should notice and record or (b) start with no preconception of what to look for.An illustration of the theory-guided strategy is Barker's ecological scheme that conceives of people occupying successive behavior settings (the person's physical/social environment at a given time) that are composed of two components: (a) the typical ways people act (standing patterns of behavior) and (b) the milieu that itself involves two elements--physical things (a restaurant's dining room or a high school classroom) and time boundaries (a two-hour dinner party or a 50minute class period) ( Barker & Gump, 1964; Barker et al., 1970). These components direct the observer's attention to what should be noted. Other mind-sets that focus researchers' observations can concern the studied person's interests, avoidances, likes, dislikes, problem-solving tech-niques, work habits, friendship patterns, and more.The opposite approach of intentionally beginning with nothing in particular to look for is intended to reveal life from the viewpoint of the person being examined. The investigator's hope is to produce a case study unblemished by preconceptions. Thus, the researcher tries to record everything that occurs during an event, and then later hunts through the collected records to locate patterns to be used as the themes around which the case-study report will be organized. In practice, however, it is unlikely that researchers can approach their task with a completely open mind. Everybody apparently brings some variety of preconceptions to a case. But at least investigators can try to be open minded, thereby recognizing circumstance they had not expected; thus they are equipped to include those features in their analyses. Important in the conduct of case studies is the researcher's description of the context in which people make decisions and seek to resolve issues. Stake ( 1995, p. 33) suggests that "the best research questions evolve during the study."Sample projects: Representative titles include | | One Week in the Life of a Chicago Social Worker (sociology) | | | A Stock Broker's Activities on Bullish and Bearish Days (business administration) | | | Teaching in an Sioux Reservation High School (education) | | | The Role of a Fulani Chieftain (anthropology) |
Advantages: The study of an individual from an observer's perspective can reveal the unique persona that the individual displays to the world and can show how that person's behavior affects other people and vice versa. Limitations: What the portrait by an outsider does not reveal is the subject's secret ambitions, motives, pleasures, fears, and ways of interpreting events. Furthermore, by focusing solely on one individual, the researcher can, only at great risk of error, infer generalizations from the case and apply those generalizations to the lives of other people.Resources: Guidance in carrying out case studies, and examples of such studies, can be found in: | | Kidder T. ( 1989). Among School Children. Boston: Houghton Mifflin. | | | Stake R. ( 1995). The Art of Case Study Research. Thousand Oaks, CA: Sage. | | | Thomas R. M. ( 1998). Conducting Educational Research (chapter 5). Westport, CT: Bergin & Garvey. |
Studying a single person from inside Purpose: The aim of the introspective case study is to disclose how events are interpreted by the person experiencing those events.To qualify as a thesis or dissertation, an insider's case study needs to represent either a person's experiences that are unique enough to represent a contribution to knowledge or else a style of depicting those experiences (such as a novel theoretical perspective) that is of scholarly import.Procedure: As with autobiographies, case studies from the insider's perspective can be composed solely by the insider (traditional) or can be produced in collaboration with someone else (mediated). If the subject of the study works alone in composing the account, the choice of what to include in the study will be directed either by that person's principal interests and concerns or by a formula for doing case studies derived from someone else, thus modeled on such existing reports as Barker One Boy's Day. If the study is a cooperative effort involving the subject and a collaborator, the sorts of information to be complied may be guided by a model that the collaborator provides.Sample projects: Illustrative titles of insider's studies can be | | A Precinct Worker's View of Local Party Politics (political science) | | | Life on an Automobile Assembly Line (anthropology) | | | Six Months in Jail (social psychology) | | | The Homes of a Foster Child (social work) | | | An Infantryman's War (social psychology) |
Advantages: Insider case studies can inform readers of the motives, values, beliefs, and interpretation of events of an individual who has participated in those events. Limitations: Again, by focusing solely on one individual, a researcher can err in applying generalizations from that case to other people's situations. Hence, authors of an insider's view, in order to extend their interpretations to others' lives, need to offer evidence in addition to their own experiences to support those extensions. Resources: Four examples of this sort of study are | | Brinkley D. ( 1988). Washington Goes to War. New York: Ballantine. (An insider's view of Washington, DC, as the nation prepared for World War II.) | | | Crouse T. ( 1973). The Boys on the Bus. New York: Ballantine. (An insider's version of the 1972 presidential campaign. Each chapter is a brief case study.) | | | Goffman E. ( 1959). The Presentation of Self in Everyday Life. New York: Doubleday Anchor. (Much of the information for this classic sociological study of people in the Shetland Islands was collected in restaurants.) | | | Noonan P. ( 1990). What I Saw at the Revolution. New York: Random House. (A series of brief incidents or snapshots of the Republican "revolution" by a speech writer for President George Bush.) | | | Cautionary guidelines for writing mediated autobiographies are offered in Causer G. T. ( 1998). "Making, taking, and faking lives: The ethics of collaborative life writing". Style, 32 ( 2), 334-338. |
Studying a group or institution--outsider versus insider In producing a case study or ethnographic account of a group or organization, the researcher's relationship to the operation of that social entity can vary from distant to intimate. Although in our introduction to this section we dichotomized the researcher's role as that of either an outsider or insider, in practice the relationship of observer to the observed is usually somewhere between those extremes. For instance, a psychologist studying the dynamics of a drug therapy group could view the operation of the group from any of several vantage points that extend along a scale from (a) the most distant, least involved to (b) the least distant, most involved. The researcher could assume any of the following roles, which range from distant to intimate: | • | Sometime after the end of the group session, view the group's activities as recorded on videotape. | | • | View the group's activities via a one-way-vision screen at the time the group is meeting. | | • | Sit in the meeting room at a distance from the group, attempting to appear as inconspicuous as possible. | | • | Sit among the group members, but take no part in the proceedings. | | • | Participate with the group as an observer who is studying the group process and is welcome to make comments. | | • | Participate in the group as one of those seeking help for a drug-abuse problem, but not revealing one's role as a researcher. |
Wolcott, as an anthropologist seeking to produce "a generalized description of the life-way of a socially interacting group," has noted advantages and disadvantages of being a participant-observer at different points along the distant/close scale. Ordinarily an outsider to the group being studied, the ethnographer tries hard to know more about the cultural system he or she is studying than any individual who is a natural participant in it, at once advantaged by the outsider's broad and analytical perspective but, by reason of that same detachment, unlikely ever totally to comprehend the insider's point of view. The ethnographer walks a fine line. With too much distance and perspective, one is labeled aloof, remote, insensitive, superficial; with too much familiarity, empathy, and identification, one is suspected of having "gone native." ( Wolcott, 1988, pp. 188-189) Purpose: Case studies of groups can have various aims. One is to identify the cultural characteristics that typify the group. The study becomes comparative whenever two or more groups are observed and conclusions are drawn about ways in which the groups are similar and different. | | A Minangkabau Wedding Ceremony (anthropology) | | | A Silicon Valley Office Party (anthropology) | | | Choosing a Chief--Tahitian and Fijian Practices (political science) | | | Sunday Church--Roman Catholic, Unitarian, Salvation Army (sociology) |
Another aim is to trace the dynamics of a group's inner workings--exposing the roles different people play, the power and prestige relationships among group members, interpersonal problems that arise, strategies adopted to resolve problems, and more. | | Community Building at the Atlanta Union Mission (social psychology) | | | Competition and Cooperation in a College Basketball Team (social psychology) | | | The Dynamics of an Alternative High School Classroom (education) | | | Power Relationships in a Business Office (business management) | | | Settling a Management/Labor Dispute in the Grape Industry (economics and political science) | | | The Internal Workings of a Feminist Organization (social psychology) |
Procedure: The intention that a researcher brings to collecting data for a case study can be at any point along a scale that extends from (a) a specific theory or precise question to answer to (b) no expectations at all about what to observe or how to interpret the results. That intention determines to a great extent the steps in the procedure that will be adopted. At the precise-question end of the scale, viewing a group or institution through the lens of a preconceived structure defines exactly what to look for and what to ignore. In contrast, at the other end of the scale, the task of observing a group is approached with an attitude of "I'll give attention to everything that happens. I'll not decide ahead of time which events and actions are most important. In other words, I'll try not to be biased by expectations I bring from my own culture. Instead, I'll attempt to see life from the viewpoint of the group's participants in order to identify their beliefs and values that explain why they behave as they do." These two extreme positions can be illustrated with a pair of anthropological investigations, one in the South Sea Island nation of Tonga and the other in West Africa. Specific preconceived questions. In a journal article entitled "Dealing with the Dark Side in the Ethnography of Childhood: Child Punishment in Tonga," Helen Kapavalu described the role of physical discipline as an instructional device in contemporary Tongan society ( Kapavalu, 1993). The content of Kapavalu's description implies that the following kinds of questions guided her research. | | The target variable: What role does physical punishment play in traditional Tongan child rearing, on what rationale is such punishment founded, and what consequences result from that practice? | | | Guide questions defining the domain of the case: What is the authority structure (the system of controllers and the controlled) in Tongan society? How is this structure reflected in the broader community, the family, and the school? What characteristics determine a person's place in the authority hierarchies? What personal-social qualities are valued in Tongan culture? What methods are used by authority figures to foster those qualities in children and youth? Upon what rationale and perception of child nature are these methods founded? How do children and youths respond to such methods, particularly to the use of physical punishment? What significant consequences appear to result from physical punishment--consequences for individuals and for the conduct of Tongan society? What changes, if any, are occurring in the techniques used for socializing the young; what are the likely causes of those changes; and what are the probable consequences of such changes for the welfare of individuals and for the traditional Tongan social system? How does the role of physical punishment as a child-rearing device in Tonga compare with its role in a typical Australian community? What problems can occur for people from a Tongan background when they enter Australian society, and vice versa? | | | Sources of answers to the guide questions: Incidents of punishment in the daily routine of life in Tongan homes, schools, church sessions, markets, playgrounds, recreational events, and work sites. Ethnographies and accounts of life in Tonga written over past decades by missionaries, anthropologists, and visitors to the islands. | | | Methods of collecting answers: Observations of child-rearing practices in Tongan settings. Interviews with Tongans in Tonga to learn their perceptions of physical punishment as an instrument of child socialization. Interviews with non-Tongan observers of life in Tonga to gather incidents of physical punishment and the observers' assessments of the desirability of such punishment. ( Thomas, 1998, pp. 97-98) |
No preconceived guide questions. An Indiana University anthropologist, Michael Jackson, identifies his approach to data gathering as phenomenological anthropology, which he defines as the scientific study of experience. It is an attempt to describe human consciousness in its lived immediacy, before it is subject to theoretical elaboration or conceptual systematizing. . . . [It consists of] prioritizing lived experience over theoretical knowledge. . . . Phenomenology seeks a corrective to forms of knowledge and description that, in attempting to isolate unifying and universal laws, lose all sense of the abundance and plenitude of life. ( Jackson, 1996, pp. 2, 6, 7) In effect, the researcher avoids forcing the events of people's daily lives into a preconceived theory of cause, such as offering an estimate of how hereditary and environmental factors have contributed to or "caused' an event. Instead, the investigator seeks to judge the meaning of events in terms of the question: What were the experiencing person's thoughts and actions that appeared during--or as a result of--those events? The phenomenologist suspends inquiry into the hidden determinants of belief and action in order to describe the implications, intentions, and effects of what people say, do, and hold to be true. ( Jackson, 1996, p. 11) To illustrate this attempt to understand life from the experiencing person's perspective, Jackson offers an example from his experience with the Kuranko people of Sierra Leone in West Africa. He describes a class of women in Kuranko society who were self-proclaimed witches facing imminent death. One must consider how the beliefs [in witches] are actually used by the women who confess to witchcraft during terminal illness--as desperate stratagems for reclaiming autonomy in a hopeless situation. The self-confessed witch . . . actively uses the imagery of witchcraft to give voice to long-suppressed grievances, coping with suffering by declaring herself the author of it. . . . Thus, she determines how she will play out the role which circumstance has thrust upon her. She dies deciding her own destiny, sealing her own fate. ( Jackson, 1996, pp. 11-12) As a further example, Jackson asserts that storytelling among the Kuranko serves the function of "reconfiguring" people's perceptions of their lives, thereby enabling them to bear the dilemmas and tensions of their daily existence. However, this is not what the Kuranko directly told him. Rather, it's Jackson's interpretation--his inference. Therefore, he has not adhered strictly to presenting his subjects' view of life in their own words. He seems to recognize this by admitting "it is difficult to capture the kinds of transformed awareness that storytelling or ritual effects" ( Jackson, 1996, p. 21). Advantages: Ethnographic research can perform several useful functions. It can reveal those characteristics shared among members of a group that make the group's culture distinctive, thereby contributing to the understanding of why one group differs from another. Case studies can also reveal the internal operations of a group or institution by identifying the relative influence of different members, tracing routes of communication, suggesting the sources of proposals that determine the group's activities, showing how people achieve and maintain their membership, exposing the sanctions used to ensure that members abide by group standards, and more. The case study is useful as a flexible approach that can be applied to diverse academic disciplines and in varied contexts. In addition, cases that trace happenings in detail allow readers to vicariously experience events as they actually occurred. As instructional devices, case studies provoke discussion that can lead to new learnings and lines of inquiry.Limitations: To disabuse people of the notion that a case study can be a portrayal of the "objective truth" about a group or organization, Denzin ( 1997, p. 3 ) has proposed: "Ethnography is that form of inquiry and writing that produces descriptions and accounts about the ways of life of the writer and those written about." Thus, although authors may assert that they have simply recorded "what really happened," their account of the case is inevitably a rendition filtered through their particular mental lens, so versions of events produced by different investigators will necessarily paint a somewhat different picture. This observation about multiple visions of the same group or same series of events is a disadvantage for readers who were hoping to learn a single "real truth" about a case. However, if a number of different researchers conduct independent studies of a group or institution, the resulting diverse products can be considered advantageous by readers who accept Denzin's proposal that different "truths" result from different accounts of an episode.Conclusions drawn in one ethnographic study can be applied to other ethnographic studies only at considerable peril because of the unique conditions that may determine the fabric of life in each setting. Furthermore, participantobservers can become so intimately immersed in a society that they diminish the objectivity of perception they sought to bring to the study. If, on the other hand, participant-observers fail to engage themselves intimately in the life of the society--and thus obtain a faulty grasp of the language, cannot view religious rites from a native perspective, or the like--they are apt to convey an inaccurate picture of what life in that setting means to the people who inhabit it.Resources: Guidance in producing ethnographic research is offered in the following volumes: | | Denzin N. K. ( 1997). Interpretive Ethnography. Thousand Oaks, CA: Sage. | | | Denzin N. K., & Lincoln Y. (Eds.). ( 1994). Handbook of Qualitative Research. Thousand Oaks, CA: Sage. | | | Jessor R., Colby A., & Shweder R. A. (Eds.). ( 1996). Ethnography and Human Development. Chicago: University of Chicago Press. | | | Hammersley M. ( 1992). What's Wrong with Ethnography? London: Routledge. |
Examples of the case study approach include | | Bussert J. ( 1986). Battered Women: From a Theology of Suffering to an Ethic of Empowerment. New York: Division for Mission in North America, Lutheran Church in America. | | | Doress P. B., & Siegal D. L. ( 1987). Ourselves Growing Older. Women Aging with Knowledge and Power. New York: Simon & Schuster. | | Hiss T. ( 1990). The Experience of Place. New York: Knopf. | | | Hochschild A. ( 1989). Second Shift: Working Parents and the Revolution at Home. New York: Viking. | | | Kozol J. ( 1991). Savage Inequalities: Children in America's Schools. New York: Harper. | | | Mead M. ( 1975). Male and Female: A Study of the Sexes in a Changing World. New York: Morrow. | | | Shulman L. ( 1992). "Toward a pedagogy of cases". In J. Shulman (Ed.), Case Methods in Teacher Education (pp. 1 - 30 ), New York: Teachers College Press. | | | | |
Experience Narratives Such narratives consist of relatively brief stories of influential events in people's lives. The nature of personal stories Defined: In recent decades, individuals' descriptions and interpretations of their personal experiences have been increasingly accepted as suitable versions of research by academicians of a postmodern persuasion. In the following section, while we recognize that biographies and case studies often include a large measure of personal views of life, we are limiting the meaning of experience narratives to accounts focusing on a particular time (in contrast to the extended time sequence of biographies) and on a more restricted body of subject matter than that found in most case studies.Purpose: The aim of experience narratives (or personal stories) is to reveal individualistic perceptions of selected life events. The emphasis is on differences among people in their experiences and their ways of viewing their lives as conveyed in their own modes of communication--words, gestures, songs, dances, symbols, art works--rather than in a researcher's modes. In other words, the people who are the objects of the research are mainly the ones who do the telling. The researcher acts chiefly as an organizer and compiler of the narratives. Studies of this sort are thus cooperative efforts in which the compiler (the thesis or dissertation author) and the informant (the person whose narrative is being reported) are credited with being co-researchers.Procedure: A typical approach to experience narrative research involves such steps as the following: | 1. | The compiler explains to the informant the realm of life experiences that is the focus of attention, such as the informant's (a) present conception of God, (b) becoming an abused wife, (c) most dramatic sexual episode, (d) suffering discrimination, (e) encounters with a particular ethnic group, or the like. | | 2. | The compiler describes (a) the informant's expected role and why the informant's narrated experiences are valued and (b) the compiler's own role. | | 3. | The informant speaks freely about the topic as the compiler records the narration verbatim, preferably through the use of an audio or video recorder so the account will be accurate. When such equipment is unavailable or the informant objects to its use, the compiler must depend on notes written at that time or as soon as possible after the session. | | 4. | During the narration, the compiler may feel it necessary to offer prompts that keep the informant on the topic and encourage an elaboration of aspects that have been unclear or inadequately developed. For example, when investigating a respondent's conception of God, a compiler may ask, "What do you feel is your relationship to God?" or "Does God ever help you? And if so, how?" | | 5. | In presenting the recorded narrative in the thesis or dissertation, the compiler prefaces the narrative with a description of: | | 5.1 The research topic, that is, the aspect of life which has been the focus of the informant's story. | | | 5.2 Who the informant was and why such an informant is a suitable source of information. | | | 5.3 The division of labor between the informant and the compiler in the conduct of the research. | | | 5.4 The context of the narrative session. | | | 5.5 Conditions that may have influenced the outcome of the session. |
|
An experience narrative project can assume a comparative form if more than one person's account is included in the study. Under those circumstances, the author may present the informants' stories without adding any analysis so that the narratives stand on their own. Or else the author may discuss themes, similarities, and contrasts observed among the several accounts.Sample projects: Illustrative titles of experience narratives include | | Participants' Perceptions of the Woodstock Concert | | | Incidents of Grieving among the Cree | | | What Basques Talk about at Home | | | A Zulu Shaman's Interpretation of His Work | | | Four-Year-Olds Tell Jokes |
Advantages: Narratives have been touted as offering two opposite advantages. First, they can show differences among people, enabling readers of research reports to discover and to "celebrate" the uniqueness of individuals' experiences and the curious ways they interpret events. Second, narratives can show similarities among people through demonstrating how ones who live under very different circumstances may have much in common when they display similar desires, emotions, and responses under dissimilar life situations.Limitations: Narratives do not provide what many consumers of research are seeking, including, | • | information about how characteristics of people are distributed throughout a population, | | • | generalizations that can be applied to understanding people other than those whose stories have been collected, and | | • | ways to correct undesirable personal or social conditions. |
Resources: Guidance in producing experience narratives is offered in | | Denzin N. K., & Lincoln Y. (Eds.). ( 1994). Handbook of Qualitative Research. Thousand Oaks, CA: Sage. | | | Durr M. ( 1998). "Oral narrative research with black women". Gender & Society, 12 ( 6 ), 766+. | | | Lieblich A. ( 1998). Narrative research: Reading, Analysis, and Interpretation. Thousand Oaks, CA: Sage. | | | Shaw C. L. M. ( 1997). "Personal narratives: Revealing self and revealing other". Human Communication Research, 24 ( 4 ), 203 -319. |
Surveys General Definitions Survey methods involve gathering information about a topic from a variety of sources, then reporting a summary of the findings. One useful way to classify surveys is to place them in two broad categories--direct-data types and research--literature types. A direct-data survey involves collecting information from individuals, groups, or institutions by means of interviews, questionnaires, opinionnaires, or observations. A research-literature survey involves compiling studies that others have conducted, then interpreting or evaluating those studies from a novel perspective. Direct-data surveys Purpose: The direct-data type is the kind people usually intend when they use the term survey. The aim of such investigations is to reveal the status of some phenomenon within an identified class of people, organizations, or regions at a particular time. Over past decades, thousands of surveys have provided information collected from a wide range of sources. To suggest something of the variety of topics on which surveys can focus, Table 7-1 offers a minuscule potpourri of topics (Information about), of sources (Information sources), and of devices for directly collecting information (Data-gathering techniques and instruments, such as those inspected in Chapter 8). Table 7-1 illustrates only five foci of surveys--achievement, customs, opinions about, policies about, and status of. Other kinds would have served equally well as illustrations, such as surveys centering on laws, regulations, habits, plans, practices, theories, traditions, worldviews, life styles, leisure pursuits, possessions, artistic pursuits, social systems, and more. Procedure: Direct-data surveys can be conducted in many ways. The following example illustrates one pattern of steps that can be adopted. | 1. | The researcher's guide question (or questions) defines the survey's focus. | | 2. | Potential people, organizations, or regions to survey are identified. | | 3. | Criteria are established to guide the choice of which options from among the potential ones at step 2 will actually be used. | | 4. | The criteria are applied to the options to arrive at the actual people, organizations, or regions to be studied. | | 5. | Potential instruments and methods of collecting survey data (as described in Chapter 8) are identified. | | 6. | Criteria for selecting the most suitable instruments and methods are established. | | 7. | Specific instruments and methods of data collection are created or adopted. | | 8. | A small sample of people, institutions, or regions that will not be used in the final survey is chosen for testing the instruments and methods in a pilot study in order to discover possible weaknesses in the methodology. | | 9. | The instruments and methods are tried out on the small sample. | | 10. | The results of the pilot study are examined; and the instruments and methods are revised to correct weaknesses found during the pilot study. If many shortcomings were identified, or if the researcher is not confident that the corrections have been sufficient, a second cycle of steps 7-10 may be carried out with a different sample of people, institutions, or regions (which will not participate in the final survey). | | 11. | In most surveys, the entire population that is being studied does not take part. Instead, for practical reasons, only a portion of that population (a sample) is chosen to be studied. Thus, at step 11, a system for drawing the sample is adopted; and a decision is made about which people, organizations, or regions will be asked to participate. (The next segment of this discussion reviews alternative sampling procedures.) | | 12. | The survey instruments and methods are administered. | | 13. | The survey responses are tabulated and classified. | | 14. | The classified results are interpreted to suggest what they mean in relation to the researcher's original guide questions. | | 15. | A description of the study is written in the form of a thesis or dissertation. |
Problems of sampling: Sometimes researchers intend to apply the conclusions drawn from their survey solely to the sources of information (people, documents, events) that directly participated in the survey. For instance, a sociology student interviews all members of a local truckers' union to learn their opinions of a proposed change in truck safety regulations. The results are then reported as representing only the viewpoints of those members, with no speculation about how truck drivers in general, or those in other union locals, might have answered the interviewer's questions. The results of such a study can be referred to as descriptive conclusions. Even though some researchers draw generalizations only about the actual participants in their survey, far more wish to extend their conclusions to embrace people or events that were not included in the study. In effect, the survey's subjects are considered to be only a portion--no more than a sample--of the population to which the research findings will be applied. Thus, on the basis of testing the reading skills of 1,500 high school seniors, a graduate student in the field | Table 7-1 | | Examples of Survey Topics | | Information about | Information sources & instruments | Data-gathering techniques | | Achievement: | | | | academic | school records | test scores, grades | | athletic | teams' record sheets | content analysis | | financial | income tax returns | content analysis | | political | election returns | content analysis | | | magazines, newspapers | content analysis | | scientific | science journals | content analysis | | theatrical | magazines, newspapers | content analysis | | Customs: | | | | clothing | people's appearance | direct observation | | | photos, drawings | content analysis | | dietary | restaurants | nutrition rating scales | | | people's self-reports | nutrition rating scales | | governance | constitutions | content analysis | | | people's experiences | interviews, questionnaires | | | history books | content analysis | | housing | housing sites | photographs | | | magazines, books | content analysis | | marriage | marriage ceremonies | direct observation | | | academic journals, books | content analysis | | religious | religious services | direct observation | | | religious publications | content analysis | | | anthropological studies | content analysis | | social interaction | social situations | direct observation | | | social-psychology books
& journals | content analysis | | Opinions about: | | | | human rights | people | interviews, opinionnaires | | politicians | people | interviews, opinionnaires | | | newspapers, magazines | content analysis | | Table 7-1 continued | | Examples of Survey Topics | | Information about | Information sources | Data-gathering techniques
& instruments | | Opinions about: | | | | social behavior | people | interviews, opinionnaires | | taxation | people | interviews, opinionnaires | | Policies about: | | | | college admission | college catalogs | content analysis | | hiring/firing | court cases | content analysis | | | personnel officers | interviews, questionnaires | | lending/ borrowing | bank & credit card
brochures | content analysis | | voting | election laws & regu-
lations | content analysis | | Status: | | | | social class | people's life styles | rating scales | | citizenship | people's self-reports | content analysis | | | census records | content analysis | | income | people's self-reports | content analysis | | | income tax returns | content analysis | | educational | people's self-reports | content analysis | | | school records | content analysis | | health | medical records | content analysis | | | people's self-reports | content analysis | | | | | |
of educational psychology may draw conclusions about the reading abilities of all 136,000 of an entire region's seniors. This application of a study's results to subjects that did not directly take part in the study is usually referred to as an inferential conclusion, because the pattern of scores of the 1,500 is inferred to be indicative of the pattern that would have resulted if the entire 136,000 had been tested. However, extending the conclusions about a directly studied group to a larger population always entails the danger of error, since the sample group may not truly represent the larger population. In other words, the sample may be biased. Consequently, as you plan a survey, it is important to decide how broadly you wish to apply your findings. Will you be content to regard the results as limited to the subjects you directly studied (people, events, documents), or will you consider those subjects to be a sample of a broader population to which your conclusions can appropriately apply? If the latter, then you need to specify (a) the characteristics of the population to which you will apply your conclusions and (b) how you can select a sample of subjects that will faithfully represent those characteristics. There are various methods of drawing samples, with each accompanied by particular advantages and disadvantages. In way of illustration, consider these four approaches to sampling: simple random, multistage, systematic, and convenience. Simple random sampling. The basic rule for drawing a random sample is that each member of the population should have an equal chance of being chosen. There are several ways that can be accomplished. One familiar way consists of assigning each subject in the population a number, then writing each number on a slip of paper (with all slips identical in size and texture), placing the slips in a hat or fish bowl, stirring up the slips, and drawing one slip at a time until the sample size has been reached. So, if there are 4,756 members in a college's junior class, and you plan to interview a sample of 100 juniors about their opinions of binge drinking among college students, then the first 100 numbers drawn out of the 4,756 would comprise your random sample. Or, instead of putting 4,756 slips of paper in a hat, you could obtain a list of random numbers from a table in a statistics book or from a computer program that generates random numbers. The first 100 numbers in that list identify which students (in terms of their assigned identification numbers) would be interviewed. The advantage of drawing a random sample is that you can now make a good estimate of the extent to which your sample of juniors probably reflects the opinions of the entire junior class (using statistical techniques described in Chapter 11). But you also may face several problems of ensuring that all 100 can be interviewed. For instance, you may have trouble getting in touch with all 100, or some of them may not agree to participate, while others may not be available at the times you need them. And if the population to which you wish to apply your conclusions is spread out geographically, the task of interviewing all respondents may be overwhelming. Such would be the case if you wanted to apply your generalizations to "college juniors in the United States." There are simply too many juniors in too many different locations to sample randomly. Multistage sampling. One way to simplify the problem of drawing a random sample is to divide the selection process into stages. That is, a population can often be described in terms of a hierarchy of sampling units of different sizes and types. As an example, for our binge-drinking study we could define a hierarchy of three stages or levels: (1) regions of the nation, (2) colleges and universities, and (3) juniors in those institutions. First, we divide the United States into regions, and by random sampling we pick one region. Second, we list the names of all colleges and universities in that region, then pick four by random means. Third, we obtain the names of the juniors in those institutions and from that list we randomly select the 100 to be surveyed by telephone interviews. This procedure meets the basic requirement for random sampling (each student has had an equal chance to be chosen) and has much simplified our task of conducting the survey. Variations of multistage sampling are available to accommodate the conditions of different studies and different types of populations ( Ross, 1985). Systematic sampling. Within relatively small populations, a systematic sample will usually represent a population's characteristics as accurately as will a random sample. Imagine that a political-science student plans to write a thesis on the attitudes of members of local luncheon clubs ( Kiwanis, Lions, Optimists, Rotary, and others) toward gun control legislation. The membership rosters show a total of 1,379 members in the local region. Rather than soliciting the opinions of the entire population of 1,379, the student plans to conduct telephone interviews with only 50 or 3.6% of the total. To select the 50 who will be asked to participate, the student assigns each club member a number ranging from 1 to 1,379. She then writes numbers 1 to 28 on a sheet of paper (since there are about 28 fifties in 1,379) and, with her eyes closed, touches a pencil point to the sheet. The point touches number 8. That number defines the first club member to be included in the sample. The next choice will be 28 numbers beyond 8 (member 36), and the third will be 28 numbers beyond 36 (member 64), and so on until all 50 have been selected. Because only chance errors, rather than other sources of bias, are apt to affect how closely the interview results approximate the population's opinions about gun control, the statistical techniques described in Chapter 11 for estimating sampling error (t-test and ANOVA) can be appropriately used with systematic sampling. Convenience sampling. Many surveys in the social sciences and humanities utilize what have been called available, convenience, or accidental samples. Such is the case when teachers in a junior high school test eighth-graders' mastery of computer keyboarding, when an anthropologist describes the division of household tasks among the members of nine families, or a social psychologist studies children's methods of settling disputes in an inner-city neighborhood. In these instances, the particular junior high school, families, and neighborhood were chosen because they were convenient to study, not because they were randomly selected representatives of a defined population. Therefore, the value of such research resides in what it tells about the people who participated directly in those investigations rather than in generalizations that might be proposed about junior high students, families, and neighborhoods in general. In short, there is no available statistical procedure for estimating how well convenience samples reflect the pertinent characteristics of whatever broad population a researcher wishes to speak about. The best an investigator can do in such situations is to (a) identify the features of the sample that seem to be causal factors (factors that affect keyboarding, family members' roles, and means of settling disputes), then speculate about whether those same factors might be influential--and to the same degree--in other groups that could be studied and that represent the same population. For instance, the anthropologist who studied the nine families might suggest that results similar to the ones obtained in her project would likely be found in other families that displayed similar--and apparently influential--conditions, such as, (a) approximately the same number of family members of about the same ages, (b) the same social-class status, (c) the same general cultural background, (d) the same general housing style, and (e) the same sort of climate. In effect, the investigator believes these five variables significantly affect family members' roles. However, generalizing from an available or convenience sample to an assumed population is a precarious venture, since there is no clear way of identifying and measuring the factors that may have biased the sample's results.Summary. The four types of sampling described above represent only a few of the available options for designing samples. Other varieties, detailed methods of using them, and their advantages and disadvantages can be found in the resources listed at the end of this discussion of surveys.Sample projects: Here are titles of studies utilizing survey data: | | Voters' Predictions of Election Results (political science) | | | Welfare Recipients' Complaints (sociology, social work) | | | Teachers' Influence on Students' Perceptions (psychology, education) | | | Families' Budgeting Practices (economics) | | | Errors in Spoken Spanish (linguistics) |
Advantages: Direct-data surveys offer useful information about groups by showing the dominant characteristics of a group as well as differences among members of a group. Inferences drawn from studying a sample of subjects (people, events, institutions, documents) may be applied to a larger population, thereby contributing to readers' understanding, not only of the individuals who were studied directly, but of a wider range of similar subjects. Surveys conducted with questionnaires distributed to respondents (by hand, by mail, by Internet) enable researchers to gather information from a large number of respondents in a short space of time with relatively little effort. Limitations: One worrisome problem with surveys that require detailed answers or touch on sensitive topics is that of ensuring that participants are diligent and truthful in answering the survey items. A further challenge is that of mounting a convincing rationale to support the extension of the findings beyond the studied sample to encompass a broader population. Surveys conducted by means of interviews are labor intensive in that they require a large expenditure of the researcher's time. However, interviews are able to reveal nuances of meaning that cannot be obtained by means of written questionnaires. Resources: Help with survey methods and sampling is offered in | Babbie E. R. ( 1990). Survey Research Methods ( 2nd ed.). Belmont, CA: Wadsworth. | | | Barnett V. ( 1991). Sample Survey Principles and Methods. New York: Oxford University Press. | | | Bickman L., & Rog D. J. (Eds.). ( 1998). Handbook of Applied Research. Thousand Oaks, CA: Sage. | | | Braverman M. T., & Slater J. K. (Eds.). ( 1996). Advances in Survey Research. San Francisco: Jossey-Bass. | | | Chaudhuri A., & Vos J. W. E. ( 1988). Unified Theory and Strategies of Survey Sampling. New York: Elsevier. | | | Dyer J. A. ( 1996). The art and science of survey research. In T. J. Flanagan & D. R. Longmire (Eds.). Americans View Crime and Justice: A National Opinion Survey. Thousand Oaks, CA: Sage. | | | Henry G. T. ( 1990). Practical Sampling. Newbury Park, CA: Sage. | | | Fink A., & Kosecoff J. ( 1998). How to Conduct Surveys: A Step-by-Step Guide ( 2nd ed.). Thousand Oaks, CA: Sage. | | | Kish L. ( 1965). Survey Sampling. New York: Wiley. | | | Krosnick J. A. ( 1999). Survey research. Annual Review of Psychology, 50, pp. 537-567. | | | Lemeshow S., et al. ( 1990). Adequacy of Sample Size in Health Studies. New York: Wiley. | | | MacNeill I. B., & Humphrey G. J. ( 1987). Applied Probability, Stochastic Processes, and Sampling Theory. Boston: Klumer. | | | Rea L. M., & Parker R. A. ( 1992). Designing and Conducting Survey Research. San Francisco: Jossey-Bass. | | | Schwarz N. ( 1996). Survey research: Collecting data by asking questions. In G. R. Semin et al. (Eds.). Applied Social Psychology. London: Sage. | | | Weisberg H. F., Krasnick J. A., & Bowen B. D. ( 1996). An Introduction to Survey Research, Polling, and Data Analysis. Thousand Oaks, CA: Sage. | |
Research-literature surveys Defined. Frequently the information on which a graduate student's project depends is not acquired directly from people, institutions, or observed events. Instead, the information comes from previously conducted studies that bear on the project's questions. Purposes: Literature surveys can serve numerous functions, including those of (a) synthesizing studies, (b) revealing diversity, (c) exposing inconsistencies and exceptions, (d) illustrating applications, and (e) generating propositions and principles. Synthesizing studies. Most research projects are very limited in scope. They focus on a specific place, time period, and restricted number of people. But, as noted above, researchers usually wish to draw conclusions that apply to a broader range of places, times, and people than they have studied. However, doing so can entail considerable risk of error. Some investigators seek to reduce this risk by collecting the results of a variety of separate studies so as to increase the places, times, and subjects on which to base generalizations. In effect, their purpose is to synthesize studies so as to show which conclusions validly apply to the entire collection and which apply to no more than one or a few of the studies.One form of synthesizing that has become increasingly popular in recent decades bears the title meta-analysis and is founded on the following line of logic. The traditional process of integrating [the conclusions from] a body of research literature is essentially intuitive and the style of reporting narrative. Because the reviewer's methods are often unspecified, it is usually difficult to discern how the original research findings have contributed to the integration. A careful analysis can sometimes reveal that different reviewers use the same research reports in support of contrary conclusions. . . . The most serious problem for reviewers to cope with is the volume of relevant research literature to be integrated. Most reviewers appear to deal with this by choosing only a subset of the studies. Some take the studies they know most intimately. Others take those they value most highly, usually on the basis of methodological quality. Few, however, give any indication of the means by which they selected studies for review. ( McGaw, 1985, p. 3322) One solution proposed for the apparent shortcomings of the typical intuitive synthesizing process has been to adopt a form of meta-analysis. The term metaanalysis can be applied to any quantitative integration of empirical research results. Two of the most popular meta-analytic approaches to defining the domain of pertinent studies and calculating the commonalities and differences among studies are those described by Glass, McGaw, and Smith ( 1981) and by Hunter, Schmidt, and Jackson ( 1982). Three examples of studies employing metaanalysis are ones entitled | | Common Themes across Counseling Theories (counseling psychology) | | | Similarities among Polling Systems (political science) | | | Defining "Family "--A Cross-Cultural Study (anthropology) |
Revealing diversity. The main intent of some research-literature surveys is not to identify characteristics shared in common by many studies but, rather, to show how various investigations of the same topic differ from each other. Imagine, for example, that a sociology student wishes to learn how social exchanges are manifested in different cultures. As her guide, she adopts a socialexchange theory that focuses on the principles governing transactions between individuals or groups, with those transactions contributing either to the benefit or to the detriment of the participants. The assumption behind her socialexchange theory is that an individual who benefits from another person's acts is obligated to reciprocate by furnishing benefits to that person in turn. For many common types of social interaction, proper exchange is dictated by cultural tradition in the form of expectations about fairness, expectations that assume the form of exchange norms. Such norms are adopted in a culture as devices for coercing the parties in a social transaction to abide by what is considered fair. Members of society impose social pressure to encourage people to comply with those norms. The extent to which people abide by exchange norms influences their status in the society's hierarchy of respect, prestige, and power ( Eve, 1986). The central purpose of the student's dissertation is thus to reveal and explain diverse forms in which exchange norms appear in different cultures. Here are titles of three studies focusing on diversity: | | Variations among Conceptions of Leadership (social psychology) | | | Disparate Policies Governing Test Cheating (education) | | | Conceptions of Justice in Six Societies (criminology, social philosophy) |
Exposing inconsistencies and exceptions. Sometimes a reader believes that a theory, proposition, or conclusion offered by an author fails to account for all instances of the phenomenon under study. In other words, the author's proposal is valid under some circumstances (such as those situations cited in the author's account) but not in others. A student may thus seek to correct the resulting oversimplification by means of a literature survey that exposes instances which cannot be convincingly explained unless the author's proposal is revised. Hence, the student's thesis contributes to the body of knowledge by both (a) revealing inconsistencies in, and exceptions to, the original author's research and (b) suggesting the manner in which the investigated author's proposal should be revised so as to resolve the inconsistencies and accommodate the exceptions. | | When a Market Economy Explanation Does Not Suffice (economics) | | | Off Target: Melford's Theory Misapplied to Preindustrial Cultures (anthropology) | | | Problems with Voucher Systems under Differing Versions of Democracy (education, political science) |
Illustrating applications. Some researchers are strong in formulating theories or methodology but weak in showing how their theories or methods can be applied to solve problems met in everyday life. A student's project may therefore be designed to illustrate practical implications of theoretical or methodological proposals. | | Skinner's Operant Behaviorism in Classrooms (education) | | | Applying Sander's Grief Theory to Distressed Children (social work) | | | Managing One's Money from Friedman's Vantage Point (economics) |
Generating propositions and principles. The intent of a literature survey can also be to discover features that are shared by a variety of studies, then to cast those features in the form of propositions, hypotheses, or principles that help explain the nature of the phenomenon on which the studies focus. | | Toward a Theory of Ethnic Shame (anthropology) | | | The Development of Sibling Rivalry (developmental psychology) | | Principles of Reconciliation for Divorced Couples (counseling psychology) |
Procedure: Here are the steps in one method of conducting a research-literature survey with the aid of a computer connected to the Internet. | 1. | State the issue to be investigated in the form of a guide question or a series of questions derived from your research topic. | | 2. | To direct the hunt, select key words and phrases from the research questions, along with their synonyms and related terms. | | 3. | Using a computer that is linked to the World Wide Web, | a. | Locate a university library's home page via a browser. | | b. | From the array of data bases that the library offers, select one or more that you think will enumerate the kinds of studies you wish to survey. For instance, here are the names of several data bases in the University of California's Digital Library: Anthropological literature, Chicano data base, English short-title catalog, ERIC, GeoRef, Magazine & Journal Articles, Melvyl (books), Newspaper Articles, PsychInfo. |
| | 4. | Open the data base and begin entering your key words and phrases to locate citations of studies relevant to your project. | | 5. | Record the results of your search by means of the methods described in Chapter 3 for compiling material from the professional literature. |
Resources: Methods of conducting literature surveys, and examples of the results of such surveys, can be found in | | Conroy H. ( 1988). "Pacific Island studies: A survey of the literature." Pacific Affairs 61( 3), 573-574. | | | Shields D. C. ( 1988). "So you'd like to do a literature search: An ERIC report." Communication Education, 37 ( 2), 165-172. | | | Stevenson R. ( 1988). "Black politics in the U.S.: A survey of recent literature." Black Scholar, 19 ( 3), 58-61. | | | Stirling K. ( 1989). "Classical political economy: A survey of the recent literature." Journal of Economic History, 49 ( 1), 252-254. | | | Swing E. S. ( 1994). "Textbooks in kaleidoscope: A critical survey of literature and research on educational texts." CHOICE, 31 ( 8), 1342. | | | Weaver D. B. et al. ( 1982). How to Do a Literature Search in Psychology. Dallas, TX: Resource Press. |
Correlation Analyses Calculating positive and negative relationships Defined: Correlation studies are guided by the generic question: What happens to one variable when another variable changes? Whenever that question is applied to particular variables, it can assume any of the following forms. | • | Are females emotionally more stable than males? | | • | Are Germans more stubborn than the French? | | • | Are children from one-parent families more likely to get into trouble with the law than children from two-parent families? | | • | Does the incidence of illness increase with an increase in a community's population density? | | • | What is the relationship between the frequency of rain dances and the amount of precipitation falling in a region? | | • | Is the rate of divorce related to the level of married couples' formal education? |
Correlations can be either positive or negative. In positive correlations, an increase in one variable parallels an increase in the other. For example, research on 14-year-olds' science achievement in 23 countries showed that pupils whose parents had more formal education scored higher on science tests than did students whose parents had less formal education ( Postlethwaite & Wiley, 1992, p. 162). In negative correlations, an increase in one variable is accompanied by a decrease in the other. A study by Barker and Gump ( 1964) showed that the larger the student enrollment in a high school, the fewer the number of extracurricular activities in which the average student engaged. If, when one variable changes, nothing happens to the other, then the two are not correlated.Procedure: Researchers can describe correlations in various ways, ranging from a general, imprecise statement in a historical account to a statistically precise statement in a direct-data survey. | • | As more settlers moved into the Southwest, Apache resistance became increasingly violent. | | • | Lincoln's oratorical skill grew with each additional opportunity to speak before political gatherings. | | • | Eighty-three percent of Republicans and 17 percent of Democrats voted in favor of the tax bill. | | • | The relationship between general intelligence and extroversion was +.19 among girls and +.21 among boys. |
It's apparent that not all correlated variables are related to each other in the same degree. At the highest level of correlation, the extent of change in one variable is accompanied by the same extent of change in the other variable. In contrast, at the opposite extreme of correlation, change in one variable is associated with no change at all in the other. There are many methods for expressing degrees of relationship. One popular method, applied when the variables are expressed in graduated quantities, is Pearson's product-moment correlation coefficient. In the Pearson system, the highest level of correlation is labeled 1.00, indicating that any change in one variable is always attended by the same extent of change in the other. The lowest level is labeled 0.0, meaning that change in one factor is never accompanied by any change in the other. Between these two extremes are graduated degrees of relationship. The closer the coefficient is to 1.00, the greater the relation between the variables. The closer the coefficient is to 0.0, the less the two variables are related. Thus, in the following examples, the highest level of relationship (+.82) is between fourth-graders' scores on reading and on mathematics tests (a strong tendency for skilled readers to get higher math scores than do poor readers). A considerably lower level of relationship (+.35) obtains between students' school achievement and their self-concepts (students with high grades tend to have higher self-concepts than those with lower grades, but there are still a good many exceptions to that tendency). Only to a very slight extent (-.16), do Mexican high school applicants from smaller families have greater command of the Spanish language than do ones from larger families (the correlation is negative, because greater skill in Spanish is accompanied by smaller, rather than larger, family size). However, a coefficient as low as -.16 is so close to no correlation at all (0.0) that knowing the size of a girl's family is of almost no aid in our predicting her mastery of Spanish or predicting her family size from knowing her Spanish-language test scores. | • | Fourth-graders' scores on a standardized reading test and their scores on a mathematics test = +.82 ( Slavin, 1984, p. 15). | | • | School achievement and self concept = +.35 ( Follman, 1984). | | • | Family size and Spanish-language achievement of secondary-school applicants in Mexico = -.16 ( Palafox, Prawda, & Velez, 1994, p. 173). |
Additional ways to calculate degrees of correlation are described in Chapter 11.Purposes: Correlations are extremely important in research designed to identify the causes of events, because all statements about causation are statements about correlations among variables.Correlations not only help people identify causes of events, but even when the relationship between two variables does not represent a causal connection, knowing the extent of relationship between two variables may help us predict the condition of Variable A by knowing the condition of Variable B. In way of illustration, by learning that a fourth-grade boy in Slavin's study has a high score on the reading test, we can predict with some confidence that the boy will have an above-average score in mathematics as well, since the relationship between the two variables is +.82. And as noted above, because the correlation between family size and Spanish-language skill among Mexican secondary-school applicants is so slight (-.16), we cannot with any confidence estimate a girl's command of Spanish by knowing the size of her family.Sample projects: Correlation is the essence of such studies as: | | Causal Factors in Autism (clinical psychology) | | | Methods of Crowd Control in Public Crises (sociology) | | | Predicting Vocational Success and Failure (counseling psychology) | | | Connections between Worldviews and Customs (anthropology) | | | Early Signs of Business Failure (business administration) |
Advantages, Limitations: The fact that two variables are correlated does not necessarily mean that one contributed at all to the outcome of the other. Thus, it is critical that researchers recognize the difference between casual or incidental correlations and causal or determining correlations. For example, in a small town in Kansas they tell of an astute observer who, while spending both winter and summer hours lounging in front of the general store, noted a positive correlation between the softness of the pavement on Main Street and the speed at which milk soured. The softer the pavement, the faster milk curdled. Hence, he proposed that the town council install firmer paving (replace asphalt with concrete) to retard the souring of milk.So, merely demonstrating a positive or negative correlation is not sufficient to support a claim of cause. What is also needed is a convincing line of logic demonstrating that one variable was at least partially the result of one or more other variables. Frequently, the supporting rationale is designed to show that one factor (the cause) preceded the other (the effect) and that the two could not have occurred in reverse order. Furthermore, constructing a persuasive argument that links two or more variables in a causal relationship often entails proposing how such a relationship is mediated by a chain of variables--a relationship involving a sequence of linkages extending from underlying causes to immediate ones.Resources: Statistical correlation procedures are provided in | | Glass G. V., & Hopkins K. D. ( 1996). Statistical Methods in Education and Psychology ( 3rd ed.). Boston: Allyn & Bacon. | | | Hays W. L. ( 1994). Statistics ( 5th ed.). Fort Worth, TX: Harcourt Brace. | | | Jaccard J., & Becker M. A. ( 1990). Statistics for the Behavioral Sciences ( 2nd ed.). Belmont, CA: Wadsworth. | | | Kendall M. G., & Gibbons J. D. ( 1990). Rank Correlation Methods ( 5th ed.). New York: Oxford University Press. | | | Siegel S., & Castellan N. J., Jr. ( 1988). Nonparametric Statistics for the Behavioral Sciences ( 2nd ed.). New York: McGraw-Hill. | | | Sirkin R. M. ( 1995). Statistics for the Social Sciences. Thousand Oaks, CA: Sage. | | | Sprinthall R. C. ( 1997). Basic Statistical Analysis ( 5th ed.). Boston, MA: Allyn & Bacon. |
Experiments General definition An experiment typically consists of applying a treatment to an individual, group, or institution, then describing the apparent effect of the treatment and estimating why that effect occurred. An assumption on which experiments are founded is that events are the result of one or more causal variables. General purpose The aim of an experiment is to manipulate the variables in a way that reveals (a) which ones are responsible for the outcome and (b) how much each variable has contributed to the observed result. Designs of experiments can vary from the extremely simple to the highly complex, with each design bearing particular advantages and limitations. The simpler the design, the easier it is to carry out the research. The more complex the design, the better the experiment accounts for variables that affect the outcome and, as a result, the greater the confidence a researcher can place in the conclusions drawn from the result. Consequently, ease and feasibility are paid for by insecurity in accepting conclusions drawn about cause. On the other hand, confidence in conclusions is paid for by more difficulty in executing the experiment and by greater complexity, time, expense, and bother. As with surveys, conclusions drawn from experiments can be applied solely to the people or events that have been directly studied, or the conclusions can be inferences applied to a population which the studied sample is thought to represent. In recognition of this distinction, patterns of experiments are often referred to as either authentic experimental designs or quasi-experimental designs. The term authentic refers to the practice of randomly assigning people or events to different treatments, then assuming that the results of the study can also be validly applied to the population from which the subjects were randomly drawn. A random sample is usually easier to draw in the physical sciences than in the social sciences. Researchers in chemistry, physics, and biology are typically freer to manipulate the objects they study than are researchers in the social sciences, who are obliged to accommodate their methodologies to people's everyday lives while disturbing those lives as little as possible. Thus, the great majority of experiments in the social sciences--and certainly those conducted with the limited resources available to most graduate students--render random sampling impracticable. Consequently, experiments reported in theses and dissertations typically involve convenience samples, since students usually study groups and events that are readily available. And because those people and events probably do not accurately represent of the intended population, the resulting experimental schemes are referred to as quasi-experimental designs. There are far more experimental designs available than can be described in this one chapter, so in the following pages we limit our attention to key characteristics of only four of the more common types. For more detailed descriptions of many more types, you can consult the resources at the end of this section. Ex post facto Procedure: The simplest design consists of (a) applying a treatment to an entity (person, group, series of events), (b) evaluating the entity's performance following the treatment, and (c) estimating how much that treatment contributed to the outcome of the event. For example, a political candidate's advertisement is aired on a television station (the treatment); then a researcher questions a number of viewers to discover their opinions of the candidate (the evaluation). If the interviewees' opinions of the candidate are favorable, then the researcher may conclude that the TV ad has been responsible for that outcome. This is an ex-post-facto or after-the-fact design, because the researcher has drawn conclusions about the apparent effect of the treatment--the TV ad--solely on the basis of evaluating viewers' opinions after they experienced the treatment.Advantages: The main advantage of the ex-post-facto model is that it's easy to use. It requires minimal time and bother for the researcher and for the participants in the experiment.Limitations: The most obvious shortcoming is that the design fails to answer questions about the viewers' beliefs before they saw the TV program. Possibly the advertisement had no affect on their attitudes, or it may have damaged the politician's image in some viewers' eyes while enhancing it in others'. Hence, an ex-post-facto plan leaves the researcher in an indeterminate position when trying to explain the effect of treatments.Sample projects: Using an ex-post-facto design is most defensible when the ability of the subjects (such as individuals or groups) to display the evaluated knowledge, skills, or attitudes could not have been acquired prior to the treatment. For instance, it seems highly unlikely that students in a typical North American college would have any ability to read, write, or speak the Javanese language before they enrolled in a course in Javanese. Thus, after six weeks in the course, their performance on a test of Javanese probably reflects the effect of the course (the treatment) and not the effect of prior knowledge.Sample projects: An ex-post-facto design could appropriately be applied in projects bearing such titles as | | Mastering Micronesian Ocean Navigation Skills | | | Learning to Identify Islamic Art Influences | | | Applying Jungian Dream Symbolism | | | Identifying Key Elements of Voodoo Belief | | | Critiquing Piaget's Theory of Child Development |
Pretest-treatment-posttest Procedure: To furnish a more convincing answer to the question about the effect of the politician's TV commercial, the researcher could adopt a pretesttreatment-posttest design. In this case, at the outset of the study the investigator would have the viewers take a pretest in the form of an interview or a printed questionnaire focusing on the respondents' present knowledge and opinions about the politician before they saw his TV ad (treatment). After viewing the advertisement, the respondents would once again express their opinions of the politician by means of interviews or questionnaires (posttest). To determine the apparent influence of the treatment, the researcher subtracts each participant's pretest results from his or her posttest results, then computes the average of those differences to obtain an overall assessment of the effect of the ad for the group as a whole. Consequently, the investigator's judgment of the advertisement would be based, not on the final posttest scores, but on the extent of change from pretest to posttest. Advantage: Compared to the ex-post-facto design, a pretest-treatment-posttest (p-t-p) experiment enables a researcher to estimate with greater confidence and precision the apparent influence of the treatment.Limitations: The p-t-p design can still leave the investigator in doubt about (a) possible reactive effects and (b) the comparative worth of alternative treatments.As Ball ( 1985, p. 4200) has explained, Reactive effects in measurement occur when the behavior elicited by a measurement procedure is not characteristic of the behavior that would have occurred in the absence of the measurement procedure. One type of reactive effect is the influence that a pretest may exert on a posttest result. In the case of the political-ad experiment, the pretest interview may, in itself, have altered participants' attitudes toward the politician and thereby served as a kind of treatment that influenced their posttest results. For example, the questions posed in the pretest could have alerted participants to think about the politician in a manner that would have shown up on their posttest, even if they never saw the TV advertisement. Thus, the posttest scores might derive from a combination of both the pretest and intended treatment, leaving the researcher in doubt as to how much the TV ad itself influenced viewers' opinions.Our illustrative version of the p-t-p design also fails to reveal how the lone treatment option--the single TV ad--influences people's judgments in comparison with other potential treatments, such as a series of TV ads, a televised debate between the candidate and other candidates, a newspaper article, or a radio talk show.Sample projects: Titles of studies using a pretest-treatment-posttest model can include The Influence of Movies on Empathic Responses The Effect of Humorous Anecdotes in Public Lectures Memorizing the Succession of British Monarchs Increasing the Speed and Accuracy of Mathematics Calculations tive treatments) is one that requires multiple sets of participants. In our hypothetical experiment, we will attempt to determine the comparative effectiveness of two contrasting treatments--a TV commercial prepared by the political candidate's staff and a radio talk show on which the candidate appears. As shown in the following diagram, the first step in conducting the experiment consists of randomly assigning participants (TV viewers or radio listeners) to four groups of equal size--groups A, B, C, and D. The practice of randomly assigning subjects to different treatments is based on the assumption that the four groups will be equal in regard to the attitudes and knowledge of the candidate that they bring to our experiment. In other words, the number of people who already have a favorable opinion of the candidate will be about the same in each group, the number who bring an unfavorable opinion of him will be about the same in each, and the number who will know nothing of the candidate will be about the same. Therefore, any differences we find in the groups' opinions in the posttest will be the result of what goes on within the experiment. We plan to have two of the groups (A and B) view the TV advertisement and the other two (C and D) listen to the radio talk show. The results on the posttest are expected to tell us whether the TV commercial or the talk show had the greater effect on participants' opinions of the candidate and whether that effect was positive or negative. To determine what influence, if any, pretesting exerted on the ultimate outcome, we will pretest members of Groups A and C, but not pretest members of B and D. The information derived from the pretesting and final testing should equip us to judge (a) the comparative effectiveness of the four types of experience and (b) whether pretesting affected posttest results. In way of illustration, let's assume that 600 university students took part in the experiment, with 150 students randomly assigned to each group. On the final 100item posttest questionnaire, the groups' average scores were: Group A = 82 Group B = 72 Group C = 90 Group D = 81 These results suggest that the pretest did indeed function as a kind of treatment, because the students who took the pretest in either a TV-ad or a radio-program group scored about 10 points higher than students who were not pretested. It would also appear that the TV advertisement had more influence than the radio broadcast, since the scores under groups C and D conditions were higher than those under groups A and B conditions. Multiple treatments Procedure: A design intended to compensate for the above-mentioned limitations of the p-t-p form (the reactive effect of pretesting and the lack of comparaSample projects: Projects employing such a multiple-treatments design can bear such titles as | | Textbook versus Lecturing in Teaching American History (education) | | | Reducing Race Prejudice--A Comparison of Role Playing and Pen Pals (social psychology) | | | Cost-Accounting Methods: Markinson's and Carswell's Approaches (economics) | | | Effects of Three Incentives on Welfare Recipients' Attitudes (social work) |
Advantages: The multiple-treatments design, when it includes the pretest/notest feature, enables a researcher to compare two or more treatments and to identify the possible influence of pretesting on the final outcome. Limitations: The problems of finding enough subjects to participate and of administering the experiment mount with each increase in the number of conditions and groups in the design. Such tasks as scheduling testing and treatment sessions, ensuring that participants follow directions, and compiling test results become increasingly complicated when many participants are involved. But if the total number of participants is much reduced, such as to a total of 24 or 36, each group will contain only 6 or 9 individuals. Then any attempt to apply the results of the study to a larger population becomes very hazardous since the conclusions have been based on evidence from so few subjects. Something else that we would like to learn in our study of ways to publicize political candidates is how lasting a given treatment will be as an influence on people's opinions. Our multiple-treatments design, in the above form, does not give us that information; but, as illustrated below, changing the design into a time series can help answer our query. Time series Procedure and Advantage: Information about how the result of a treatment may diminish or may increase with the passing of time can be obtained by adding posttesting at various junctures following the end of the treatment. Limitations: If the experiment has been conducted with subjects who are not readily available for a considerable time after the treatment ended, then a timeseries design can be difficult to implement, because participants can be lost before the later phases of posttesting are administered. Furthermore, such a design does not provide information about what factors following the end of the treatment have caused any decrease or increase in participants' test scores, opinions, or skills that appear in the later stages of posttesting. Additional models The four experimental designs described above represent only a small number of the forms that experiments can assume. Numerous other types are available to suit the demands of different research agenda. The nature of diverse designs, the situations for which they are suited, and the steps to be followed in applying those models are described in such publications as the following: | | Campbell D. T., & Stanley J. C. ( 1966). Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally. | | | Glass G. V., McGaw B., & Smith M. L. ( 1981). Meta-analysis in Social Research. Thousand Oaks, CA: Sage. | | | Miles M. B., & Huberman A. M. ( 1994). Qualitative Data Analysis ( 2nd ed.). Thousand Oaks, CA: Sage. |
Selecting an appropriate design The task of choosing a suitable design can entail three principal steps: (1) considering the characteristics of different experimental designs, (2) deciding whether one of those designs will be the most appropriate method of study for answering your research question (or will some other approach--historical, case-study, survey--be more relevant?), and (3) choosing a design that will furnish the most convincing evidence about the problem you are investigating within your research context. The phrase "convincing evidence about the problem you are investigating within your research context" refers to such variables as (a) the magnitude of the consequences that could potentially result from your study and (b) the availability of the time, funds, facilities, and participants that the experiment would require. Magnitude of consequences: When estimating the magnitude of potential consequences, you can properly consider how greatly decisions derived from an experiment could influence people's welfare and how many people might be affected. The influence of this estimate on your choice of an experimental design can be illustrated with the following examples. Imagine that a high school teacher wishes to assess the comparative effectiveness of two videotapes of the Civil War so as to decide which one the school should purchase for use in American history classes. She intends to derive her assessment from an experiment in which one group of students views Tape A and the other group views Tape B. In terms of the welfare of students, it likely makes little difference which tape is judged more effective. One tape is not apt to effect students' overall academic progress much more than the other. Furthermore, a relatively small number of people will be influenced by whichever tape is judged the better, since the number will be limited to students who enroll in that one school's American history classes over the next few years. Thus, the teacher is not risking students' lives if she adopts a simple, rather than a complex, experimental design. Probably a two-group pretest-treatment-posttest form will be sufficient as a guide to choosing the tape to purchase. It's true that such a design will fail to account for a number of variables that might affect students' test scores--such variables as sample size (the two groups will be rather small), distractions during the videotape viewing, the reactive effect of pretesting, and memory loss over the coming months. However, adopting a design that would accommodate all such variables would entail more bother and expense than the choice between tapes is worth. The simple pretest-treatment-posttest seems to represent an acceptable compromise between feasibility and precision. Although the results of this quasi-experiment will not be foolproof, they should be sufficient for the teacher's rather modest aim. Now imagine that a doctoral candidate in the field of clinical psychology is cooperating with a large city's schools in studying the effects of three drugs for the treatment of hyperactive elementary school pupils. The aim of the study is to determine which of the drugs reduces distractibility in pupils, enabling them to concentrate on their school work without suffering such undesirable side effects as drug addiction, lethargy, depression, and long-term memory loss. The doctoral student, aided by school counselors, intends to conduct an experiment that will reveal which of the drugs is best. In this case, the choice of a research design is far more crucial than in the case of the Civil War videotapes. The final decision about which drug, if any, should be administered to hyperactive pupils could critically affect the academic success and general health of large numbers of children--not only children in this one city's school, but hyperactive pupils throughout the nation if the experiment's results are widely disseminated. Therefore, it will not suffice to employ a simple pretest-treatment-posttest design that includes four groups of hyperactive pupils (one group taking no medication and each of the other groups taking one of the three drugs). It will be more appropriate to adopt a time-series design that includes large numbers of participants who are randomly assigned to the treatments. Adequately assessing the outcome of the experiment will also require a far more complex evaluation scheme than the one called for in the videotape study. Whereas the American history teacher's pretesting and posttesting could reasonably consist of paper-pencil tests of the content of the videotapes, the psychology student's drug study requires a variety of measures, including appraisals of hyperactivity (type, magnitude, frequency), pupils' concentration skills, environmental distracters, mood (depression, optimism, self-confidence), speed and accuracy of learning, study habits, and length of memory. Available time, funds, facilities, participants: Apparently, most master's and doctoral students wish to earn their degree within a few years. Hence, the time that an experiment will require becomes a serious concern. An experimental design that provides for the collection of data within a few weeks or few months is therefore better suited to the researcher's time constraints than one that requires several years, as can be the case with a time-series study. The choice of an experimental design is often influenced as well by the expense it will involve. If the cost of a preferred design will be excessive, the researcher must settle for a design that is less sophisticated but affordable. The same is true of the instrumentation, personnel, and facilities required by different experimental designs. A design that will yield results short of the ideal may be necessary if the needed facilities are beyond a graduate student's means. In studies that involve people as the objects of interest, large numbers of participants are usually desirable so that conclusions drawn from the results will represent a wide range of conditions and types of people. However, the necessary kinds of participants may be difficult to find, or the manner of studying them may be so complex and time-consuming that a graduate student finds it feasible to study no more than a few individuals. In summary, there is no experimental design that is superior to all others for all research situations. The choice of a design requires achieving a reasonable balance among costs (time, energy, funds, facilities, number of participants, bother, undesirable side effects), the gravity of actions that could result from the study's findings, and the benefits that could derive from the research results. With our overview of popular research methods now complete, we return to the four hypothetical projects described at the beginning of this chapter. MATCHING METHODS TO RESEARCH QUESTIONS We suggested that you might find it useful, as you read the chapter, to estimate which of the chapter's types of research methods would be most suitable for answering the questions. To close the chapter, we now propose what we believe would be appropriate matches. Our choices are displayed within brackets following each question. Project title 1: One Size Fits All: State and Federal Legislators' Solutions for Students' Unsatisfactory School AchievementGuide questions: | | How do educational experts (teachers, school psychologists, researchers) diagnose and treat students' problems of unsatisfactory achievement? [direct-data survey of experts, published-studies survey] | | | How do legislators, as reflected in laws they pass, propose that students' unsatisfactory achievement be diagnosed and treated? [direct-data survey of legislation] | | | How well do the solutions proposed by educational experts match the solutions proposed by legislators? [researcher's own analysis] |
Project title 2: Destined to Preach the Gospel: A Social-psychological Study Guide questions: | | What factors in the life of the Reverend Delevon Johnson determined that he would become a lifelong missionary in Africa? [interpretive biography based on a literature survey] | | | How did the causal factors in the Reverend Johnson's life compare with those in other African missionaries' lives? [interpretive biography based on a literature survey] |
Project title 3: The Comparative Effectiveness of Same-Sex Therapists Versus Opposite-Sex Therapists with Teenage Drug UsersGuide question: | | In counseling teenage drug users to stop using illicit drugs, is greater success achieved when the counselor is of the same sex as the client than when the counselor is of the opposite sex? [multiple-treatments experiment] |
Project title 4: The Dynamics of Choosing Candidates to Run for Political Office in Adams CountyGuide questions: | | Who are the candidates that have run for political office in Adams County over the past 12 years? [direct-data survey of public documents] | | | What other individuals were potential candidates but were not selected to run? [direct-data survey via interviews with political party leaders] | | | By what processes were candidates selected to run for different offices? [direct-data survey of public documents and direct-data survey via interviews with political party leaders] | | | What inferences can be drawn about those processes' effect on the quality of political office holders in Adams County? [researcher's own analysis] |
PLANNING CHECKLIST As an aid in applying the contents of this chapter to your own thesis or dissertation, you may find it helpful to carry out the following activities. | 1. | What is the target question or series of questions that my research is intended to answer?__________ | | 2. | Which approach or combination of approaches will I employ in my research? | | _____2.1 Historical | | _____2.1.1 Descriptive chronicle | | | _____2.1.2 Interpretive history | | | _____2.1.3 Biography--descriptive | | | _____2.1.4 Biography--interpretive | | | _____2.1.5 Autobiography--traditional | | | _____2.1.6 Autobiography--mediated | | | _____1. 1.7 Other (describe__________ |
| | | _____2.2 Case Study--Ethnography | | _____2.2.1 Single person, from outside | | | _____2.2.2 Single person, from inside | | | _____2.2.3 Group or institution--from outside | | | _____2.2.4 Group or institution--from inside | | | _____2.2.5 Other (describe)__________ |
| | | _____2.3 Survey | | _____2.3.1 Direct data | | | _____2.3.2 Research literature | | | _____2.3.3 Other (describe)__________ |
| | | _____2.4 Correlation Analysis | | | _____2.5 Experiment | | _____2.5.1 Ex post facto | | | _____2.5.2 Pretest--treatment--posttest | | | _____2.5.3 Multiple treatments | | | _____2.5.4 Time series | | | _____2.5.5 Other (describe)__________ |
| | | _____2.6 Other (describe)__________ |
|
|
