Grand Theft 3
0 General Document comments
0 Sentence and Paragraph comments
0 Image and Video comments
CHAPTER 3
Science, Nonsense and Common Sense
This essentially echoes my thoughts from reading the two studies. I think anyone who researches and uses common sense in looking into these studies can realize how much BS they are. The claims are, as the authors say, overreaching and assuming.
I seriously doubt video games are the root of any violence at all, outside of violence in video games, and that just like television and film, video games will probably come to be accepted as an art, violence or no. I think this reading, and the studies we read, have established that.
More than 3500 research studies have examined the association between media violence and violent behavior; all but 18 have shown a positive relationship.
- "Media Violence," American Academy of Pediatrics Committee on Public Education, November 20011
It is not true, for example, that "more than 3500 research studies have examined the association between media violence and violent behavior [and] all but 18 have shown a positive relationship." ... In fact, there are probably fewer than 300 empirical studies that try to measure the effects of violent media-with even and ambiguous results.
-National Coalition Against Censorship, December 20012
LET'S GO FROM HISTORY TO SCIENCE FOR A BIT. TO UNDERstand the claims made by researchers, it's useful to know how scientists approach a complex problem like the causes of violence. Scientific research is like solving a jigsaw puzzle in which you don't know if you have all the pieces; the pieces that you do have can fit together in many different ways and you're not sure what the finished picture will look like. If you're not a scientist or you don't have an interest in research, it might be tempting to skip this chapter. Some of it may be hard sledding.
58
Grand Theft Childhood
We urge you to bear with us. Don't let the technical terms or the details intimidate you.
Why should you, as a parent, care about the process of video game: research? As you'll see in chapter 8, academic studies turn into valuabl ammunition in political battles on the local, state and national levels. with. the fate of many tax dollars and careers at stake-not to mention policies designed to shape the behavior of game makers, game retailers and your children.
Our goal for this chapter is not to bore you with an exhaustive analysis of research on media violence but to address one seemingly simpl question that has some surprising answers:
Why Don't the "Experts" Agree?
You'd think that things would be clear-cut by now. After all, there are so many studies. Many of the conclusions bandied about sound pretty obvious. But that's not the case. Video game violence research is highly complex for a wide range of reasons. We'll start to explore some of those in a few pages, but let's begin by examining some of the conflicting claims.
These results clearly support the hypothesis that exposure to violent video games poses a public-health threat to children and youths [and] is positively associated with heightened levels of aggression in young adults and children, in experimental and nonexperimental designs, and in males and females .... In brief, every theoretical prediction derived from prior research and from GAM [General Aggression Mod£[J was supported by the metaanalysis of currently available research on violent video games.
-Craig Anderson, PhD, and Brad Bushman, PhD, Iowa State Uniuersity?
I have read nearly all the published English-language research on electronic games, which includes video and computer games, CDROM and online games. Neither the quantity nor the quality of research on video games does much to inspire confidence in solid conclusions about their effects.
58
Grand Theft Childhood
We urge you to bear with us. Don't let the technical terms or the details intimidate you.
Why should you, as a parent, care about the process of video game: research? As you'll see in chapter 8, academic studies turn into valuable ammunition in political battles on the local, state and national levels, with. the fate of many tax dollars and careers at stake-not to mention policies designed to shape the behavior of game makers, game retailers and your children.
Our goal for this chapter is not to bore you with an exhaustive analysis of research on media violence but to address one seemingly simple question that has some surprising answers:
Why Don't the "Experts" Agree?
You'd think that things would be clear-cut by now. After all, there are so many studies. Many of the conclusions bandied about sound pretty obvious. But that's not the case. Video game violence research is highly complex for a wide range of reasons. We'll start to explore some of those in a few pages, but let's begin by examining some of the conflicting claims.
These results clearly support the hypothesis that exposure to violent video games poses a public-health threat to children and youths [and] is positively associated with heightened levels of aggression in young adults and children, in experimental and nonexperimental designs, and in males and females .... In brief, every theoretical prediction derived from prior research and from GAM [General Aggression ModeIJ was supported by the metaanalysis of currently available research on violent video games.
-Craig Anderson, PhD, and Brad Bushman, PhD, Iowa State University 3
I have read nearly all the published English-language research on electronic games, which includes video and computer games, CDROM and online games. Neither the quantity nor the quality of research on video games does much to inspire confidence in solid conclusions about their effects.
Science, Nonsense and Common Sense
59
Nearly every study suffers from unclear definitions (of violence or aggression), ambiguous measurements (confusing aggressive play with aggressive behavior, or using questionable measures of aggression, such as blasts of noise or self-reports of prior aggression), and overgeneralizations from the data. Experiments that claim to study the effects of playing electronic games rarely study play at all.
-Jeffrey Goldstein, PhD, University of Utrecht (the Netherlands)'
If you scan the chronology of research, congressional debates and other documentation that has gone into attempting to deal with media violence since 1952, you should be outraged. How is it that millions of dollars and an uncountable number of hours and energy have been put to work against TV, movie, and video game violence for decades-and things have only gotten worse?
-Lieutenant Colonel Dave Grossman and Gloria DeGaetano, authors of Stop Teaching Our Kids to Kill"
On balance, given that video game play is highly prevalent among children and adolescents in industrialized countries, there is little evidence that moderate frequency of play has serious acute adverse effects. Adverse effects, when they occur, tend to be relatively minor and temporary, resolving spontaneously with decreased frequency of play. More evidence is needed on excessive play and on defining what constitutes excess in the first place.
-Mark Griffiths, PhD, Nottingham Trent University (England)»
Those are pretty dramatic contrasts. How can these experts arrive at such different conclusions? Sometimes, it's a matter of timing. In 1999, when Grossman and DeGaetano published their book Stop Teaching Our Kids to Kill: A Call to Action Against TV, Movie and Video Game Violence, the available data on youth violence in America painted a grim picture. Keep in mind that government crime data are always a few years old. As we write this in mid-2007, the most recent data available are from 2004.
60
Grand Theft Childhood
When they were writing their book in the late 1990s, the most recen a crime data available to them were from the early 1990s.
Using data supplied to the Federal Bureau of Investigation by loca = law enforcement agencies, the U.S. Office of Juvenile Justice and Delinquency Prevention reports that the rate of juvenile arrests increased in the late 1980s and peaked in 1994. This looked like an alarming trend. Media reports at the time were filled with speculation about a new breed of "super-predator" juvenile delinquent.
Meanwhile, in 1992, Mortal Kombat hit the video arcades with its over-the-top, blood-spewing "fatalities." According to Jeff Greeson, who manages a Web site devoted to the game series, "Mortal Kombat not only stood out, it grabbed you by the shirt collar and demanded your attention. Mortal Kombat had the biggest and most realistic characters ever featured in a video game at that time. You wrwere literally watching digitally animated photographs of people flying through the air and beating the living hell out of each other.'"
By 1995, Mortal Kombat 3 had invaded the home arena on multiple fronts, including the original PlayStation, Sega Genesis, and Super Nintendo, along with other scary-sounding titles such as Doom. It seemed natural to connect the video game violence and the real-life violence.
But juvenile arrests declined in each of the next seven years.
Between 1994 and 2001, arrests for murder, forcible rape, robbery and aggravated assaults fell 44 percent, resulting in the lowest juvenile arrest rate for violent crimes since 1983. Juvenile murder arrests reached a high of 3,790 in 1993. By 2004, arrests were down 71 percent, to 1,1l0.s
In retrospect, we can see what drove the increase in murders. We don't need to cast about for an explanation. In their 2006 report for the Office of Juvenile Justice and Delinquency Prevention, Howard N. Snyder and Melissa Sickmund of the National Center for Juvenile Justice wrote, "Ninety percent of the overall increase was murders of nonfamily members committed by males with a firearm-generally a handgun .... Nearly three-quarters of the increase was the result of crimes committed by black and other minority males-and in two-thirds of these murders, the victims were minority males. The decline in murders by
illvpnilp" frrnTI 1 QQA t" ?()()? r""a,·C'arl tho a~~l;Ar ;~~~~~~~ "9
Science, Nonsense and Common Sense
61
There's no evidence that black teens' use of violent media differed significantly from that of other young people, though there is ample evidence that as a group, they had and still have greater exposure to other risk factors for violence. For example, as of 2002, almost one third of black Americans aged seventeen and under were living in poverty.'" African-American youth are also more likely to witness real-life violence."
Changes in social priorities and ideas about crime also affected juvenile arrest rates in the early 1990s. Fears of youth violence led some communities to increase arrests for curfew violations; arrests for marijuana possession rose-although surveys did not show increased use. Acts of domestic violence were being redefined as criminal rather than family matters. Behavior once classified as a juvenile "status" offense might now lead to arrest for assault."
For whatever reason, the various experts who cited the 1990s increase in crime as evidence of harm from media violence are not rushing to take back those statements in the face of reduced crime or the more direct explanations for the temporary rise. Nor are they addressing the dramatic growth in the popularity of video games, including violent video games, during the years when violent crime rates were plummeting.
Smoke and Mirrors
Some researchers have described what they call "instructive parallels" between the relationship of media violence to youth aggression and the relationship of smoking to lung cancer," By using meta-analyses that pool the results of many different studies (more on that laterl. they conclude that the effect size (the strength of the relationship) of the violent media-aggression link is almost as big as the smoking-lung cancer link.
We agree that there are instructive parallels here. But they don't strengthen the case for media violence increasing aggression-they undermine it. Here's why:
-
Lung cancer was a rare disease before cigarette smoking became common; now it's the leading cause of cancer deaths. Aggressive behavior was not rare before electronic media became popular.
62
Grand Theft Childhood
-
Lung cancer rates increase as smoking rates go up and decrease as they go down. We can see this not only in the population as a whole, but among subgroups (men, women, rich, poor, East Coast, Midwest and so forth). No matter how you look at the group, the more smoking, the more cancer. We see nothing like this relationship when we look at violent video games and such measures of real-world violence as violent crimes comrnitted by minors .
-
There is a well-understood physiological mechanism for how cigarette smoke triggers cancer. The process by which violence in media leads to real-life violence still inhabits the realm of speculation and ad-hoc theory .
-
There's a dose-response relationship between smoking and lung cancer.
Although it's mediated by genes and environmental factors (such as exposure to asbestos), in general the more a person smokes, the higher his or her risk of lung cancer. It's not clear whether greater exposure to violent video games (More hours of play? More years of play? Choosing games with more blood?) is related to greater effects on children. Our surveys of middle school youth did find some interesting correlations, which we'll describe in the next chapter.
-
Lung cancer is a clearly defined set of diseases; we know it when we see it, and experts pretty much agree on diagnostic methods and results. A pathologist looking at a cancer cell in Boston will almost always reach the same diagnosis as a pathologist looking at the same cell in Bangkok or in Barcelona. But experts definitely differ on how to define and "diagnose" aggressive or violent behavior, both in artificial (laboratory) settings and by real kids on playgrounds or in classrooms .
-
The relationship of smoking to lung cancer is obviously a one-way street, with a clear direction of causality. People who have cancer or a genetic predisposition to cancer are not more likely than other people to seek out cigarettes! But it's likely that children who are already more aggressive than their peers prefer and seek out violent media programs and games. It's possible that time spent with a violent game or movie could help shape or trigger aggressive impulses in aggressive children. Perhaps a high level of violent media consumption could be a marker for existing aggression .
-
Exposure to cigarette smoke is easy to observe and can be measured in aoreed.unnn w:!u<: /"Ithnotnh ;t'~ +" •. "L.."_ l __ - --
Science, Nonsense and Common Sense
63
"Exposure" to video game violence is subject to argument and difficult to measure. In some ways, measuring exposure to violent video games is easier than measuring exposure to TV violence; at least you know the child is paying attention when playing a video game. But in a television program or film, everyone who watches is exposed to the same violence. In a video game, different players can be exposed to dramatically different amounts and types of violence even though they play the same number of hours .
-
The reliabilitv (consistencvl of measurement over time is also different.
Cancerous cells will still be present three hours later, while aggressive thoughts or behavior may have gone away .
-
We know that starting smoking at a young age increases the risk of addiction, long-term use and cancer. We don't know if young people of a particular age are at higher risk from voluntary play of violent video games. For example, we might want to pay extra attention to the media use of young teens. Why? Because parents monitor them less closely than they watch younger children, just when their stage of emotional, social and brain development might make them more vulnerable to the influence of violent media.1415
My Way or the Highway
The wildly divergent opinions about video game violence may partially reflect professional training and background. Different researchers and theorists sometimes see things quite differently because of their experiences and perspectives. One of the challenges faced by our own highly diverse research team was keeping track of all its members' different underlying assumptions. We soon realized that when it comes to research design, psychologists think differently than public health people, who think differently than biologists, who think differently than psychiatrists. We had to get all of our assumptions and biases out on the table, argue a little, then draw on the best of our combined training to maximize the links between our research and the real world.
In 1995, Dave Grossman wrote the book On Killing: The Psychological Cost of Learning to Kill in War and Society. It was based on his experience as an army ranger and as a teacher at the U.S. Military Academy at West
64
Grand Theft Childhood
Point. He claimed direct connections between the training (in values anc in methods of killing) that military recruits receive and what childrer experience through "shooter" video game play. He saw "point and shoo i video games" as simulators for practicing murder. (We'll address the clallr that children can literally learn to shoot from a video game in chapter 8.)
Henry Jenkins, PhD, director of the Comparative Media Studies Program at the Massachusetts Institute of Technology, has published widely on the use of video games for education. He's skeptical of the assumptions that underlie the proposed link between playing violent video games and learning to become a killer:
Grossman's model only works if:
-
we remove training and education from a meaningful cultural context.
-
we assume learners have no conscious goals and that they show no resistance to what they are being taught.
-
we assume that they unwittingly apply what they learn in a fantasy environment to real world spaces.
The military uses games as part of a specific curriculum, with clearly defined goals, in a context where students actively want to learn and have a need for the information being transmitted. There are consequences for not mastering those skills."
Craig Anderson and Brad Bushman are experimental psychologists.
In that role, they develop laboratory experiments that try to shed light on what's happening in the real world. When done well, such laboratory data can be extremely powerful, much as laboratory data using test tubes and petri dishes can give us tremendous insights into biological processes that take place in living organisms. But for a laboratory experiment to provide useful information about the real world, the assumptions (model) the researchers use have to be explicit, and have to make sense. Anderson and Bushman state:
It is important to note that real-world aggression measures (e.g., violent crime) share few surface features with laboratory aggression measures (e.g., delivery of electric shock). However, these
64
Grand Theft Childhood
Point. He claimed direct connections between the training (in values anr; in methods of killing) that military recruits receive and what children experience through "shooter" video game play. He saw "point and shoot video games" as simulators for practicing murder. (We'll address the claim that children can literally learn to shoot from a video game in chapter 8.)
Henry Jenkins, PhD, director of the Comparative Media Studies Program at the Massachusetts Institute of Technology, has published widely on the use of video games for education. He's skeptical of the assumptions that underlie the proposed link between playing violent video games and learning to become a killer:
Grossman's model only works if:
-
we remove training and education from a meaningful cultural context.
-
we assume learners have no conscious goals and that they show no resistance to what they are being taught.
-
we assume that they unwittingly apply what they learn in a fantasy environment to real world spaces.
The military uses games as part of a specific curriculum, with clearly defined goals, in a context where students actively want to learn and have a need for the information being transmitted. There are consequences for not mastering those skills."
Craig Anderson and Brad Bushman are experimental psychologists.
In that role, they develop laboratory experiments that try to shed light on what's happening in the real world. When done well, such laboratory data can be extremely powerful, much as laboratory data using test tubes and petri dishes can give us tremendous insights into biological processes that take place in living organisms. But for a laboratory experiment to provide useful information about the real world, the assumptions (model) the researchers use have to be explicit, and have to make sense. Anderson and Bushman state:
It is important to note that real-world aggression measures (e.g., violent crime) share few surface features with laboratory aggression measures (e.g., delivery of electric shock), However. these
Science, Nonsense and Common Sense
65
aggression measures do share the conceptual features of delivering a noxious stimulus to a victim with the intent and expectation of harming the victim."
Here, the researchers fail to differentiate between aggression and violence. Their logic assumes that the subjects in these experimentsusually college students who participate to earn some spending money or to get extra credit for a class-cannot tell fantasy from reality and don't know that "punishing" a person with a mild electric shock or a 9 mm pistol will lead to different outcomes. Can someone who delivers a brief blast of noise really be said to have the same malicious intent as someone who shoots a convenience store clerk or stabs someone in a bar fight?
Consider the famous experiment" conducted in the early 1960s at Yale by Stanley Milgram, PhD, in which participants were duped into believing that they were delivering painful or even life-threatening electric shocks to a person they could hear but could not see. The participants typically showed symptoms of acute stress during these experimentsincluding profuse sweating, trembling and stuttering-things not seen in video game violence experimental research.
To understand other factors behind the standoff on video game violence's effects, we need to take a closer look at the research process.
Designing a Video Game Violence Study: A Primer
In fairness, studying the influence of violent video games is much harder than it appears. What do researchers actually do to find out whether media violence makes kids aggressive? Let's walk step-by-step through some of the basic decisions and assumptions that scientists have to make when they design and carry out a study.
WHOM WILL WE STUDY? As we've seen, much of the fuss over video games has involved putative links to school shootings or other real-life violence. So, why don't researchers study teens or young adults who've committed or attempted murder? Here, we run into our first problem. Teenage murderers and rapists may have played Mature-rated video
66
Grand Theft Childhood
games, but, as you'll see in chapter 4, so have most of the law-abiding boys on your local middle school soccer team.
Violent video game play is extremely common, and violent crime i:;:; relatively rare. This makes it tough to document whether and how violen1 video and computer games contribute to serious violence, such as criminal assault or murder. Criminals are also much more likely to have past exposure to other factors, such as poverty, alcoholism, family violence or: parental neglect, that are known contributors to violent behavior.
There are also practical and ethical barriers. Most research conducted today that involves people must be approved by institutional review boards (IREs), also called human subjects committees. To IREs, prisoners are a "vulnerable population" because they are subject to coercion and control. Because of past research abuses, research subjects must be fully informed about the nature of the research and must feel free to decline to participate without fear of reprisal or loss of benefits. (In the best-known case of abuse, the Tuskegee Syphilis Study, participants were so shockingly exploited that the survivors received a presidential apology in 1997.19) While studying felons is not impossible, getting a large and representative group to sign on would be very difficult.
What about just studying teenagers? There are lots of them around.
The question then becomes, what kinds of kids do we include, and whom do we exclude? Do we want children who are at highest risk for aggression? If so, how will we identify them: A questionnaire? Interviews with teachers? Or, is there an age when children might be most vulnerable to the effects of video game violence, so that we should focus on studying that group's game play?
If we want the results of this study to help kids and want that help to be effective and cost-efficient, it's important to focus on those children who are at the greatest risk. Violent games might be a bigger influence on children who lack the protection of a nurturing relationship with a caring adult, don't feel connected to a community or are falling behind in school.
If we want to look at the effects of violent video games on young teenagers, does it makes sense to study a representative group of children under seventeen? Or could we accurately extrapolate from results of studies on high-risk kids, or even studies of college students enrolled
I
I
66
Grand Theft Childhood
games, but, as you'll see in chapter 4, so have most of the Iaw-abidin , boys on your local middle school soccer team.
Violent video game play is extremely common, and violent crime i relatively rare. This makes it tough to document whether and how violera video and computer games contribute to serious violence, such as crimi nal assault or murder. Criminals are also much more likely to have pas exposure to other factors, such as poverty, alcoholism, family violence OJ parental neglect, that are known contributors to violent behavior.
There are also practical and ethical barriers. Most research conductec today that involves people must be approved by institutional review boards (IRBs), also called human subjects committees. To IREs, prisoners are a "vulnerable population' because they are subject to coercion and control. Because of past research abuses, research subjects must be fully informed about the nature of the research and must feel free to decline to participate without fear of reprisal or loss of benefits. (In the best-known case of abuse, the Tuskegee Syphilis Study, participants were so shockingly exploited that the survivors received a presidential apology in 1997.19) While studying felons is not impossible, getting a large and representative group to sign on would be very difficult.
What about just studying teenagers? There are lots of them around.
The question then becomes, what kinds of kids do we include, and whom do we exclude? Do we want children who are at highest risk for aggression? If so, how will we identify them: A questionnaire? Interviews with teachers? Or, is there an age when children might be most vulnerable to the effects of video game violence, so that we should focus on studying that group's game play?
If we want the results of this study to help kids and want that help to be effective and cost-efficient, it's important to focus on those children who are at the greatest risk. Violent games might be a bigger influence on children who lack the protection of a nurturing relationship with a caring adult, don't feel connected to a community or are falling behind in school.
If we want to look at the effects of violent video games on young teenagers, does it makes sense to study a representative group of children under seventeen? Or could we accurately extrapolate from results of studies on hizh-risk kids. or even studies of rol
Science, Nonsense and Common Sense
67
in Psychology 101? The latter group is a popular choice, because children under eighteen are also considered a "vulnerable population" for research purposes.
Often, researchers are required to obtain a signed parent consent form for each participating child, plus a signed "assent" from the child. If you've ever helped round up signed permission forms for a class trip to the museum, you can imagine how much harder it is to get detailed research consent forms returned! *
Since school is a primary gathering place for children, researchers often try to use schools as sources of research SUbjects. Principals and teachers can be less than enthusiastic. With increased government mandates and testing, there is very little discretionary time in which to hand out questionnaires. They also dislike the disruption of routines, especially when there is no clear benefit to the school. In areas with many colleges and universities (such as the Boston area, where we work), many schools are overwhelmed with requests and are fed up.
Let's say that we successfully woo a principal and gain access to a school. We send out the consent forms, and even with follow-up mailings or phone calls, only 40 percent come back with parent signatures. (That's not unusual.) How do we know whether the group of signed-up children is similar to the group whose parents declined or didn't bother to fill in the form? Is their video game use the same? What about their household income, their cultural or ethnic background, their academic achievement, parental supervision of their game play, or any number of other factors that might affect the link between violent video games and aggressive behavior? This creates a problem with "generalizing" the results of our study to the overall population of children.
Another problem with studying children is: they're kids. They don't read and write as well as adults do. They get bored and make things up.
* Because we conducted our research under the auspices of the IRB at Massachusetts General Hospital, we were required to use the hospital's standard "informed consent" form, which covers all of its highly diverse research-most of which involves drug trials or new medical and surgical procedures. Consequently, even though we were only talking to people in an office, we were obligated to tell the participants in our focus groups that if they suffered any side effects from our conversations, they should immediately go to the nearest hospital emergency room!
68
Grand Theft Childhood
They have trouble remembering or estimating potentially importan things, such as how many hours they play video games during a typica, week. At what age can kids be expected to fill out questionnaires or give: accurate responses? Can older kids accurately recall what they did noonly last week, but what they did a few years earlier?
As you can see, there are many possible answers just to the questior of whom to study. Researchers make different choices based on what they want to find out {their "research question")-and frankly, on whorr: they can get to sign up. In the real world, time and resources are limited: and we sometimes have to make the best of things. However: researchers who study different populations may end up with divergent or even contradictory results. Studies that try to combine the results oi many small studies on very diverse groups (e.g., ten-year-olds in Belgium, college undergraduates in Minnesota, and urban sixteen-year-olds who played arcade games in 1987) should be taken with a grain-make that a lump-of salt.
What Type of Study? It Depends on the Question
The design of a research study has a profound effect on what its findings can (and cannot) demonstrate. Here are three of the most common designs for research on the effects of violent media:
-
A cross-sectional study. This involves looking at behaviors at a particular point in time or during a relatively short period of time. We might find, for example, that children who report playing a particular type of violent video game, or who play for significantly more hours per day than their peers, are more likely to have lower grades in school. One problem with these studies is that the data only show correlations; we can't say that violent video game play causes poor grades, that frustration at school leads to more violent game play or that a third factor causes them both. We simply don't know.
-
A longitudinal study. This type of study measures children's video game play multiple times, over a long period of time. Some longitudinal studies in other fields have gone on for a half century or longer. As you can imagine,
68
Grand Theft Childhood
They have trouble remembering or estimating potentially importan, things, such as how many hours they play video games during a typicsa week. At what age can kids be expected to fill out questionnaires or give: accurate responses? Can older kids accurately recall what they did noonly last week, but what they did a few years earlier?
As you can see, there are many possible answers just to the questior of whom to study. Researchers make different choices based on wha, they want to find out (their "research question")-and frankly; on whorr they can get to sign up. In the real world, time and resources are limited and we sometimes have to make the best of things. However researchers who study different populations may end up with divergen i or even contradictory results. Studies that try to combine the results 0: many small studies on very diverse groups (e.g., ten-year-olds in Belgium, college undergraduates in Minnesota, and urban sixteen-year-old s who played arcade games in 1987) should be taken with a grain=-make that a lump-of salt.
What Type of Study? It Depends on the Question
The design of a research study has a profound effect on what its findings car (and cannot) demonstrate. Here are three of the most common designs fOI research on the effects of violent media:
-
A cross-sectional study. This involves looking at behaviors at a particular point in time or during a relatively short period of time. We might find, for example, that children who report playing a particular type of violenl video game, or who play for significantly more hours per day than then peers, are more likely to have lower grades in school. One problem with these studies is that the data only show correlations; we can't say thai violent video game play causes poor grades, that frustration at school leads to more violent game play or that a third factor causes them both. We simply don't know.
-
A longitudinal study. This type of study measures children's video game play multiple times, over a long period oftime. Some longitudinal studies in other fields have gone on for a half century or longer. As you can imagine,
Science, Nonsense and Common Sense
69
these are extremely expensive to run. They also tend to involve smaller samples of participants, some of whom will probably drop out or move away before the study ends. Also, by the time you finish your study, social or technological changes may make your findings irrelevant. The graphics and plotlines of 1 980s arcade-based games such as Pac-Man and the original Mortal Kombat bear little resemblance to the home-based violent video games of today. In a few years, games with holographic images and fullbody interactivity could make todav's studies less relevant.
-
Laboratory experiments, such as pre/post experimental studies. This is the most common type of study done in this field. Participants playa game in a controlled situation, which can range from a spare room in an afterschool program to inside an fMRI machine at a neuroimaging center. After subjects are exposed to the video games, researchers measure some sort of behavior or physiological response, either immediately or after a delay. They look for relationships between the exposure (say, to a violent game versus a nonviolent game) and the behavior or response.
Because laboratory experiments are, by definition, carried out in controlled environments, one has to be careful about making the leap not only from correlation to causation (e.g., exposure to video games causes participants to feel more aggressive) but from the laboratory to the real world (e.g., that a D.l-second difference in the length of an air horn blast immediately after playing means that this person is more likely to behave violently after playing a violent video game at home).
VIDEO GAME VIOLENCE: HOW MUCH, AND WHAT KIND? Now that we've chosen a group to study, how will we measure whether, and how much, our group is exposed to video game violence? Suppose we've settled on a cross-sectional study of real children or adolescents. We might ask them (or their parents) about games they playa lot or their favorite games. We could give them diaries and ask them to record what they play, and how long, every day for a week. We could watch them play (live or on videotape) and count how many murders of human-like characters they see per hour of game play. We could go to kids' homes and look at the number or proportion of game cartridges and discs that are rated Teen or Mature.
70
Grand Theft Childhood
Each method has pros and cons and requires a set of additional de sions. For example, if we count game cartridges: Do we only inclu those played within the last month? Six months? At least once a week, for at least an hour per week? What about rented games or games play at friends' houses?
Another complication is that unlike movies, video game play is nc linear; each child who plays follows a somewhat different sequence. A the more flexible the playa game allows, the more variation in exposu to violence there will be. For example, in Grand Theft Auto: Vice City, yl can make Tommy Vercetti rampage through the city in ugly plaid g( pants, literally chainsawing through everyone in his path, or he co peacefully deliver pizzas. In this case, a child's game play would need be observed for hours to get a sense of how much vicarious violence 1 or she is experiencing.
One might use ESRB ratings or content descriptors to assess leve of game violence. (Our surveys used a variation of this method.) Bi there is some debate about the accuracy of ESRB ratings, includir how violence is defined and labeled. In a study of violent content in I rated games.:" researchers Kimberly Thompson, ScD, and Kevi Haninger, PhD, chose not to count routine aggressive behavior i sports games-such as tackling in football or checking in hockey-c violent, because there is no real intent to harm. As you'll see in char ter 7, parents in our focus groups shared this view. They also woul probably agree with the researchers' definition of violence as "acts i which the aggressor causes or attempts to cause physical injury c death to another character."
Interestingly, the definition of "character" is where these researcher and our parents of preteens part company. The parents were bothered b violence against realistic human or at least humanoid characters. Thomp son and Haninger "defined characters broadly, including personifie objects that attacked either the player or other characters." Using thi definition, one of the study's most violent games, in terms of percentagi of screen time (72 percent), was Kirby 64: The Crystal Shards. Kirby is; smiling pink puffball who inhabits a number of Nintendo games. We have a hard time picturing Kirby as a violent role model.
Of course, parents of young children may be more concerned abou
Science, Nonsense and Common Sense
71
animated violence, and researchers have every right to define terms as they see fit. Sources of honest disagreement would be easier to spot if all researchers explained their assumptions and methods as clearly as in this study.
Also, how much violence is too much? If we assume that any and all exposure to violent games is harmful, we end up treating the majority of preteens and teens as abnormal. This doesn't make sense. As Griffiths noted above, "More evidence is needed on excessive play-and on defining what constitutes excess in the first place.' ?' We need to know what "normal" is so that we can meaningfully measure behaviors against it.
Finally, as you'll see in chapter 5, not all violence appears to have equal effects on game players' thoughts, feelings and behaviors.
A Rose by Any Other Name?
A key part of any research design is establishing what's known as "validity." In other words, are you really measuring what you think you're measuring? This can be tricky.
Let's begin by looking at some famous longitudinal studies. Because video games are a relatively new medium, researchers looking for evidence about their long-term effects often point to well-known studies of TV violence. One such studv" followed children from northern New York State for seventeen years and found that young teens (and young adults) who watched three or more hours of television a day were later involved in more fights, robbery and aggressive acts in general.
However, the researchers didn't bother to measure what these children watched on TV. They assumed all content was basically the same. After all, as they pointed out, "violent acts are depicted frequently on television."
They also noted that "childhood neglect, growing up in an unsafe neighborhood, low family income, low parental education, and psychiatric disorders were significantly associated with time spent watching television at mean age 14 and aggressive acts at mean age 16 or 22." Even though researchers tried to account for these factors in their analyses, it's not at all obvious that more time with television causes more aggressive behavior. It's just as plausible that heavy TV-watching represents a poverty of options-no
72
Grand Theft Childhood
safe places to play outside, poorer social skills, or no access to social, inte lectual or artistic activities that support healthy development.
A study of Chicago qrade-schoolers> found that watching violent show and identifying with aggressive characters was linked to aggressive behavia fifteen years later. Researchers asked children to choose their favorites fron lists of popular programs and whether they watched these shows "every tim, it's on," "a lot but not always," or "once in a while."
Two raters coded the amount of physical violence in each show on a five point scale from "no violence" to "very violent," based on "visually depicted interpersonal acts that were intended to harm." Nature-show violence wa: excluded. The examples of "very violent" programs listed by the researcher: were Starsky and Hutch, The Six Million Dollar Man, and Road Runner car toons. (This is a great example of how changing social standards make longi tudinal studies so chancy. When we describe this study in presentations, i always gets a lauqh.]
Children were also asked about how much they acted like particular rnah or female "aggressive characters" such as the Bionic Woman or the Six-Mil lion-Dollar Man. It's unclear why kids were not supposed to identify witl these heroic figures: a tennis pro and an astronaut who overcome cata. strophic injuries and pursue assorted bad guys and spies.
Measuring exposure to game violence in laboratory studies has 2 completely different set of challenges. If the study will compare reactions to a violent game versus a nonviolent game, how do we choose appropriate games for our tests? In some studies, an action-packed violent game goes head-to-head against a tranquil nonviolent game. In one case, researchers compared Wolfenstein 3D (which involves killing Nazis in the corridors of a fortress prison) to Myst (a surreal puzzle-solving and exploration game) on the basis that both involved a "3-D walk-through format." 24 For these comparisons to make sense, games need cornparable levels of emotional excitement and physiological arousal.
In this study of college students, as in many similar studies, play times were extremely short. Participants were seated at a computer, read an instruction sheet, played for fifteen minutes-then completed a questionnaire on how they were feeling. As those of us who've struggled
72
Grand Theft Childhood
safe places to play outside, poorer social skills, or no access to social, intellectual or artistic activities that support healthy development.
A study of Chicago qrade-schoolers» found that watching violent shows; and identifying with aggressive characters was linked to aggressive behavior fifteen years later. Researchers asked children to choose their favorites from lists of popular programs and whether they watched these shows "every time it's on," "a lot but not always," or "once in a while."
Two raters coded the amount of physical violence in each show on a fivepoint scale from "no violence" to "very violent," based on "visually depicted. interpersonal acts that were intended to harm." Nature-show violence was excluded. The examples of "very violent" programs listed by the researchers were Starsky and Hutch, The Six Million Dollar Man, and Road Runner cartoons. (This is a great example of how changing social standards make longitudinal studies so chancy. When we describe this study in presentations, i1 always gets a laugh.)
Children were also asked about how much they acted like particular mals or female "aggressive characters" such as the Bionic Woman or the Six.Million-Dollar Man. It's unclear why kids were not supposed to identify witr these heroic figures: a tennis pro and an astronaut who overcome catastrophic injuries and pursue assorted bad guys and spies.
Measuring exposure to game violence in laboratory studies has c completely different set of challenges. If the study will compare reactions to a violent game versus a nonviolent game, how do we ChOOSE appropriate games for our tests? In some studies, an action-packed violent game goes head-to-head against a tranquil nonviolent game. In one case, researchers compared Woifenstein 3D (which involves killing Nazis in the corridors of a fortress prison) to Myst (a surreal puzzle-solving ant exploration game) on the basis that both involved a "3-D walk-throng} format." 24 For these comparisons to make sense, games need compara. ble levels of emotional excitement and physiological arousal.
In this study of college students, as in many similar studies, play times were extremely short. Participants were seated at a computer read an instruction sheet, played for fifteen minutes-then completed c questionnaire on how thev were feelinc. As thoss- of us whrv'v
Science, Nonsense and Common Sense
73
to survive even a few minutes in an unfamiliar game can attest, any increase in hostility or anger could easily be due to frustration over trying to learn a new game so quickly! Also, this experience is far removed from that of a typical child who plays self-selected games in a familiar environment, perhaps with friends, over a long period of time. There's also the issue of whether we can accurately extrapolate from college students to young teenagers.
WHAT IS THIS THING CALLED "AGGRESSION"-AND HOW DO WE MEASURE IT? If we're going to study aggression, we need to know it when we see it. "Violence" is somewhat easier: a person or thing gets physically attacked with the intent to injure or break it. But the definition of "aggression" can be surprisingly slippery and is another factor behind the warring interpretations of video game research.
Some researchers use "aggression" and "violence" almost interchangeably, implying that one inevitably leads to the other. That's simply not true. A common definition of aggression as used by psychologists is "behavior leading to self-assertion; it may arise from innate drives and/or a response to frustration, and may be manifested by destructive and attacking behavior, by hostility and obstructionism, or by self-expressive drive to mastery." 25 As you can see, some behaviors can be aggressive but not violent.
For most media violence studies, some aspect of aggression is the "outcome" that's measured. A classic example is the 1960s experimental studies by Albert Bandura, PhD, of Stanford University on young children's imitation of aggression." He used what are known as Bobo dolls-inflatable plastic clowns several feet tall that have a weighted round bottom. If you hit or push the top, the doll topples over and then springs back up.
Children watched an adult (on film) either attacking a Bobo doll (punching it or hitting it with a mallet) or playing quietly with other toys while ignoring the Bobo doll. Children were more likely to attack the doll when they saw that behavior modeled by an adult. (One problem with this study is that there's not much you can do with a big inflatable Bobo doll other than punch it and watch it bounce back!)
Some of the most-publicized experimental studies done by other
74
Grand Theft Childhood
researchers use a "noise blast" as a stand-in for aggressive behavior. AE one study" describes it, "The participant's goal is to push a button faste.a than his or her [nonexistent] opponent. If participants lose this race they receive a noise blast at a level supposedly set by the opponen t (actually set by the computer)." The study participants, in turn, are tole to set a loudness level for the noise that their opponent will hear if he O::J: she loses. "Aggression is operationally defined as the intensity and duration of noise blasts the participant chooses to the deliver to the opponent." This is basically white noise, the "whishing" sound you hea r between channels when tuning a radio. The test is an adaptation of ar: earlier measure that used electric shocks instead of sound.
Christopher J. Ferguson, PhD, at the Department of Behavioral Applied Sciences and Crimina] Justice at Texas A&M International University, takes issue with the noise-blast test:
Most people outside of research associate aggression with physically trying to hurt someone. Whether the noise blast corresponds with anything like that is very debatable. My guess is that it's actually a better measure of competitiveness, which is right in the name: the Taylor Competitive Reaction Time test. In my opinion, it's a far cry from beating someone to death, or even spanking or slapping someone, because it's not that aversive.
In science, new measures are supposed to be "validated," or proven to represent something in the real world. For example, if we developed a new questionnaire or physical test to screen for depression, we would need to show that the test scores of people who'd been formally diagnosed with depression were different from the scores of people known not to be depressed. At minimum, we'd need to compare our test results to some existing validated questionnaire. But the "noise blast" test has never been validated as a measure of aggression. Without such validation, we are simply asked to accept someone else's belief that the test means what is being claimed.
As Ferguson points out:
The test ranges from zero to ten. But it's not, "If you score above
......... ~,_ ............. : ..... 1 ........... :_..J:_ .. :-I. __ 1 L _1 .' . __ :._ ~1 __ ,.,,_
74
Grand Theft Childhood
researchers use a "noise blast" as a stand-in for aggressive behavior. A one study" describes it, "The participant's goal is to push a button fasts than his or her [nonexistent] opponent. If participants lose this racs they receive a noise blast at a level supposedly set by the opponen (actually set by the computer)." The study participants, in turn, are tot to set a loudness level for the noise that their opponent will hear if he 0 she loses. "Aggression is operationally defined as the intensity and dura tion of noise blasts the participant chooses to the deliver to the oppc nent." This is basically white noise, the "whishing" sound you hea between channels when tuning a radio. The test is an adaptation of a earlier measure that used electric shocks instead of sound.
Christopher J. Ferguson, PhD, at the Department of Behaviora Applied Sciences and Criminal Justice at Texas A&M International Uni versity, takes issue with the noise-blast test:
Most people outside of research associate aggression with physically trying to hurt someone. Whether the noise blast corresponds with anything like that is very debatable. My guess is that it's actually a better measure of competitiveness, which is right in the name: the Taylor Competitive Reaction Time test. In my opinion, it's a far cry from beating someone to death, or even spanking or slapping someone, because it's not that aversive.
In science, new measures are supposed to be "validated," or prove) to represent something in the real world. For example, if we developed; new questionnaire or physical test to screen for depression, we wouk need to show that the test scores of people who'd been formally diag nosed with depression were different from the scores of people knownot to be depressed. At minimum, we'd need to compare our test result. to some existing validated questionnaire. But the "noise blast" test ha: never been validated as a measure of aggression. Without such valida tion, we are simply asked to accept someone else's belief that the tes means what is being claimed.
As Ferguson points out:
The test ranges from zero to ten. But it's not, "If you score above seven you're a violent individual, below seven you're in the nor-
Science, Nonsense and Common Sense
75
mal range, and below three you're a pacifist." Say that study Group A gets a 5, and Group B gets a 5.4. Does that mean that Group B is ready to go out and riot? No. Nobody knows what any of these scores mean. I think the noise blast test could be salvageable if put into a standardized, reliable format and subjected to rigorous testing.
As we write this, Ferguson is undertaking studies to try to validate the noise-blast test.
Some real-world studies, especially of younger children, look at aggressive behavior on the playground after exposure to violent games or cartoons. But is this really unhealthy, potentially dangerous aggression, or is it normal rough-and-tumble play?
Michael [ellinek, MD, is a professor of psychiatry and pediatrics at Harvard Medical School, the chief of child and adolescent psychiatry at Massachusetts General Hospital and the president of Newton-Wellesley Hospital. He points out:
A normal kid on the playground who's playing army or spaceman, and pushing his friend or trying to vaporize him but never hurting him-that's normal, healthy fantasy play. It's totally consistent with the biological nature of 10-year-old boys.
Another problem is that today's aggressive first-grader may be tomorrow's thoughtful teen. According to the Surgeon General's 2001 report on youth violence." most children who are aggressive or engage in antisocial behavior do not grow up to be violent adolescents or adults-and most violent adolescents were not notably aggressive as children.
Finally, how do we know whether kids with different personalities or behavioral styles respond differently to the same video games? What about how they're feeling just before they start to play? Several Australian researchers looked at those issues with respect to anger, which they saw as related to aggression and violent behavior:
The results demonstrated that some people increase, some decrease and the majority saw no change in anger ratings. Unlike
76
Grand Theft Childhood
past research, we also demonstrate that these changes are mediated by the player's feelings immediately prior to game play and a labile temperament-one predisposed to aggression."
DO THE RESULTS MATTER? SO, we've measured changes in aggre sive behavior, mood or attitudes. How do we decide whether exposure media violence has made a difference? Also, what's the effect size? Th is, how much of a difference does what you're measuring really make?
For Ferguson, this is a sore point when he sees experimental resul described on television and in newspaper stories:
When these things get reported to the general public, there's no discussion of what an effect size is and what it means. It's "Study A shows a relationship between video games and aggression." And people think that's a very conclusive, important finding. But when you look at the effect size, sometimes it's near zero.
For example, twin studies of antisocial or violent behavior find that about 50% of the variation [the R-squared effect size] is due to genetics. That's a pretty powerful effect. By comparison, the Rsquared effect size for media violence-including televisionranges anywhere from one percent to nine percent. So you're getting anywhere from one fifth to one fiftieth the bang for the buck you'd get from a genetic effect.
lf you were four percent more aggressive today than yesterday, cou1 anybody tell the difference?
Most important: Does the new study help parents recognize whethe their child is at risk of, or already has, problems related to violent vide games? Academic research on video game effects is not geared to th needs of parents. You can't scan your child's brain at home, even if an) thing meaningful could be found that way. Parents need markers the they can observe and specific things they can do to prevent problems c catch them early.
One could make the argument, as Anderson does, that small effec sizes can be significant because of the overall long-term exposure:
Science, Nonsense and Common Sense
77
If youths spent only a little time playing violent video games (e.g., less than 30 minutes per week), or if only a few youths spent a lot of time playing such games (e.g., 1 in 10,000), then the overall cost to society would likely be fairly small. But ... a lot of youths are playing violent video games for many hours per week. When large numbers of youths (including young adults) are exposed to many hours of media violence (including violent video games), even a small effect can have extremely large societal consequences.v
It's absolutely true that across a population, small things can add up to a big difference. Consider salt intake and cardiovascular disease, for example. But we don't assume that all people are equally affected by a medical risk and that the same remedy is good for all. Some people are not salt sensitive and would give up better-tasting food for no benefit; others might be healthier with a little extra salt. One glaring problem with most studies of violent game effects is their failure to consider which groups are most at risk and should therefore lower or monitor their time spent playing games, which groups are probably at little or no risk (at least with moderate use) and whether any groups might even benefit from a little downtime with a joystick.
Violent Video Games and the History of the Bathtub
Now there's an odd juxtaposition! But it's relevant as a cautionary tale when you hear published research results being bandied about. One of the most important skills you can use as both a scientist and as a parent is skepticism.
During World War I, the eminent journalist H. L. Mencken wrote an article in the New York Evening Mail that purported to give a history of the bathtub. It was a joke, designed to provide his readers with some respite from the awful news of the day. Mencken later described his article as "a tissue of absurdities, all of them deliberate and most of them obvious.' ?' These
'''I
78
Grand Theft Childhood
included descriptions of physicians calling the bathtub dangerous and laV\. passed against it in Massachusetts and Pennsylvania.
His essay struck a chord. It was reprinted in newspapers across the cou I try. Then Mencken started noticing something strange. His intentional I absurd claims were being quoted in reputable publications as undispute facts. "They began to be cited by medical men as proof of the progress ( public hygiene. They got into learned journals. They were alluded to on th floor of Congress .... Finally, I began to find them in standard works of refel ence. Today, I believe, they are accepted as gospel everywhere on earth. 1 question them becomes as hazardous as to question the Norman invasion.":
Take another look at the quote at the beginning of this chapter, the on about the "3500 research studies." It doesn't make any sense. But it's cite all the time. Where did that number come from? We decided to trace it bac as far as we could.
The list of citations at the end of the American Academy of Pediatrics pol icy statement gives the book Stop Teaching Our Kids to Kill as the sourer That book, in turn, lists a 1 998 article published in Sweden by the Unitel Nations Educational. Scientific and Cultural Organization (UNESCO). whicl states, "Over the past forty plus years more than 3,500 research studies 01 the effects of television violence on viewers have been conducted in th! United States." 33 There is no information given about how the authors arrive( at that number.
In all fairness, we should point out that the media researchers in the ne:x quote who claim that "there are probably fewer than 300 empirical studie: that try to measure the effects of violent media" don't demonstrate how the\ came up with that number, either. But it sounds much more logical.
NOT ALL STUDIES ARE CREATED EQUAL There are even fewa studies about the effects of violent video games on aggression. In a 200] review, John L. Sherry, PhD, of Purdue University found "32 independ ent studies in which violent video game play was the independent vari able and some measure of aggression was the dependent variable." > HE concluded, "There is a small effect of video game play on aggression, ant the effect is smaller than [that of] violent television."
Science, Nonsense and Common Sense
In a separate review published the same year, Anderson and BL:l..shman found thirty-five reports (some with multiple studies) in a psychology research database." Their analysis included many of the same studies used by Sherry. Yet their conclusion was different: "ThesE results clearly support the hypothesis that exposure to violent videa games poses a public health threat to children and youths."
We decided to take a closer look at the studies reviewed by Anderson and Bushman. Unusually, there is no listing of these studies in their paper; instead, the reader is directed to a Web site. We reviewed the list and tracked down full-text copies of as many of those studies as we could.
Of the twenty-one experimental and thirteen correlational studies included, about one third are dissertations or other unpublished research, making descriptions difficult to obtain. Across the peerreviewed correlational studies, subjects and measures varied widely. Many focused on time spent playing games and did not ask about content; those few that did ask about content used vague measures such as a stated preference for aggressive games. Outcomes had little to do with real-world aggressive behavior (perhaps the closest being teacher observations of free play). Some studies were from the 1980s, when children still primarily played video games in arcades. Study samples were small. (The largest is a survey of 278 Dutch children.) There is little or no information on response rates or other means to assess the characteristics or the representativeness of the samples. (One sample included students at a high school for "disturbed young people.") The study that came closest to looking at effects of violent games on children's aggressive behavior (a British study of two hundred and four students aged twelve to fourteen) found that "aggression scores were not related to the number of games with aggressive content [children] named among three favourite games." 36
As we said at the start of this chapter, this is not an exhaustive review of research on video game violence but an attempt to examine why experts disagree about its effects. However, it seems clear that despite the worrisome conclusions that appear in the popular press, there are very few studies involving current violent video games and real children.
80
Grand Theft Childhood
Statistical Significance vs. Real-World Significance
The term "statistical significance" refers to the odds against something oc ring simply by chance. Let's say that you claim that you have supernat powers and can use them to blindly select an ace from a full deck of ca You know this because you tried it once and picked an ace.
How could we decide whether your supernatural powers are real? L leave aside issues of research design such as making sure that you agreE what an ace is, that the cards are adequately and randomly shuffled and 1 no one is cheating, and focus on interpreting the results.
In a standard deck, four of the fifty-two cards (one in thirteen) are ac That means that you'd choose an ace, on average, one out of every thirn tries simply as a matter of chance. But what if you were able to select t aces in a row? Would that prove that you have supernatural powers? Wh. you did it twice, then failed the next twenty-eight times, and then were s cessful again?
Statisticians can look at those different success rates and calculate odds that the results are occurring simply by chance. That's what a tesi statistical significance is. It's usually expressed as a probability or "p valu If the difference between your actual performance (you selected three ar out of thirty-one tries) and your expected performance that's attributable chance or dumb luck (you select three aces out of thirty-nine tries=-ons thirteen) is described as significant at the p < .05 level, that means tl there's less than a 5 percent chance that you've simply been lucky. If it's s nificant at the p < .01 level, then the odds that it's a fluke are less than one a hundred.
There are two technical things to keep in mind here. The first is that t more people you're dealing with-that is, the larger the sample size-t more you should be looking for higher levels of significance. For example, our survey of over 1,200 teenagers, we didn't focus on findings at the p <. level; we looked for findings at the p < .01 or p < .001 levels before we p. serious attention to them.
Also, some statistically significant differences may be mathematica
Science, Nonsense and Common Sense
81
sound but irrelevant to the real world. For example, an experimental study may find small but consistent differences in the heart rates of game players exposed to violent vs. nonviolent games. The findings may be significant at the p<.05 or even the p<.Ollevels. But that doesn't mean that those differences actually make a difference in predicting a person's future or even immediate behavior.
META-ANALYSES AND THE "FILE DRAWER PROBLEM" The two research reviews mentioned above used a statistical method called "meta-analysis." It is a very powerful and respected technique when used as intended, gleaning important information through synthesizing studies that are too small or otherwise problematic to rely on individually (such as studies of the best drug treatment for a particular heart problem, or among people of a certain age group). But as one how-to article on meta-analysis puts it, "Even though the statistics used in metaanalysis are quite sophisticated, the end product will never be better than the individual studies that make up the meta-analysis.?" In other words: garbage in, garbage out.
Goldstein raises another important issue that affects meta-analyses: the "file drawer problem":
Published research in scholarly journals does not represent all the research on electronic games. Studies that fail to find statistically significant results are less likely to be accepted for publication. So the published record is an unknown fraction of all research, and it tends to consist of those studies with statistically significant results. This is known as the "file drawer problem" because studies that do not find any effects of video games remain unpublished, locked away in the researcher's files."
The file drawer problem is an open secret among academic researchers. The editors of academic journals generally think that positive results are more interesting and more likely to be cited by future researchers, which is one measure of a journal's influence and prestige.
82
Grand Theft Childhood
They're more likely to publish research that finds a relationship or a E nificant difference between two things than a study that shows no rE tionship.
If findings don't fit the hypothesized results, researchers may assu their study was somehow flawed in methods or analyses and keep try for a more acceptable outcome. Or they may simply decide to bury urn sired or embarrassing results.
The existence or extent of publication bias in a given body of reseal can be estimated using advanced statistical methods. In a recent me analysis," Ferguson found a pattern of missing studies, particularly experimental studies of aggressive behavior and nonexperimental sn ies of aggressive behavior and thoughts.
The problem is this focus on statistical significance testing-this sort of black-and-white, either-you-have-it-or-you-don't kind of thing. You get these tiny, tiny little effects that become statistically significant the bigger your sample size is. But how meaningful are those results?
Our point is simply this: be skeptical of claims about violent vic games. You don't need a degree in statistics to judge research for yo self. When you hear about a new study that "proves" the link betwe video games and aggressive or violent kids, ask yourself:
.: . Whom did they actually study?
.: . How did they define "violence" in video games or television progran .: . How did they measure exposure to video game violence?
.: . How did they define "aggression"? What did they use as a measure aggression? How do they justify the relationship of this measure to re world aggression or violence?
.: . Does that measure pass the common-sense test?
In the next chapter, we'll describe some findings from our 0 research and what they mean to you as a parent. We urge you to ap these questions to our research as well.
Science, Nonsense and Common Sense
83
Justified Violence
Imagine the following gruesome plot: A mob rapes and beats a concubine all night. They leave her bloodied corpse on a doorstep to be discovered in the morning. Her master takes her home, carves her body into twelve pieces and incites another mob to seek revenge. They do, killing tens of thousands of men, women, children and animals, and destroying a city.
It sounds a bit like a violent video game. It's not; it's a story from an obscure passage in the Bible (Judges 19-21) that was used in two interesting studies of the influence of religious beliefs and attributions on responses to violent media. Half the participants who read the passage were told of its Biblical origins. The other participants were told that the text was from an ancient scroll. In each group, half were given a version that included two sentences from the original text in which God commands the mob to undertake the violent retaliation; the version that the other half read did not include the commands from God.
In the first study, the participants were students at Brigham Young University, of whom 99 percent said that they believed in God and in the Bible (the religious group). In the second study, they were students at Vrije Universiteit in Amsterdam, of whom 50 percent said that they believed in God and 27 percent said that they believed in the Bible (the nonreligious group).
The researchers measured differences in the intensities of noise blasts on a competitive test conducted immediately after reading the material. (Note that this design suffers from the same weaknesses as other research using these noise blasts as stand-ins for violent behavior, as we described above. It also only measured immediate effects, not long-term effects.)
They summarize their research results: "We found compelling evidence that exposure to a scriptural depiction of violence or to violence authorized by deity can cause readers to behave more aggressively. . . . Aggressive responses were greater when a violent depiction was attributed to a scriptural source than when it was attributed to an ancient scroll and were also greater when the violence was said to be sanctioned by God than when God was not mentioned .... Even among our participants who were not religiously devout, exposure to God-sanctioned violence increased subsequent aggres-
Science, Nonsense and Common Sense
83
Justified Violence
Imagine the following gruesome plot: A mob rapes and beats a concubine all night. They leave her bloodied corpse on a doorstep to be discovered in the morning. Her master takes her home, carves her body into twelve pieces and incites another mob to seek revenge. They do, killing tens of thousands of men, women, children and animals, and destroying a city.
It sounds a bit like a violent video game. It's not; it's a story from an obscure passage in the Bible (Judges 19-21) that was used in two interesting studies of the influence of religious beliefs and attributions on responses to violent media. Half the participants who read the passage were told of its Biblical origins. The other participants were told that the text was from an ancient scroll. In each group, half were given a version that included two sentences from the original text in which God commands the mob to undertake the violent retaliation; the version that the other half read did not include the commands from God.
In the first study. the participants were students at Brigham Young University. of whom 99 percent said that they believed in God and in the Bible (the religious group). In the second study. they were students at Vrije Universiteit in Amsterdam, of whom 50 percent said that they believed in God and 27 percent said that they believed in the Bible (the nonreligious group).
The researchers measured differences in the intensities of noise blasts on a competitive test conducted immediately after reading the material. (Note that this design suffers from the same weaknesses as other research using these noise blasts as stand-ins for violent behavior, as we described above. It also only measured immediate effects. not long-term effects.)
They summarize their research results: "We found compelling evidence that exposure to a scriptural depiction of violence or to violence authorized by deity can cause readers to behave more aggressively .... Aggressive responses were greater when a violent depiction was attributed to a scriptural source than when it was attributed to an ancient scroll and were also greater when the violence was said to be sanctioned by God than when God was not mentioned .... Even among our participants who were not religiously devout, exposure to God-sanctioned violence increased subsequent aggres-
84
Grand Theft Childhood
sion. That the effect was found in such a sample may attest to the insidioi, power of exposure to literary scriptural violence." ?
We should note that the lead author of this research article, Brad Bus t man, PhD, is well known for his research on the potentially harmful effects ( video games, and we commend him for exploring this "politically incorrecl topic. But we also note that in their conclusion, the researchers offer advic that is quite different from that offered by many video game researcher: including Bushman, who found less-compelling results when measuring th effects of that medium:
"Does this ultimately mean that one should avoid reading religious cano for fear that the violent episodes contained therein will cause one to becom more aggressive, or that individuals who read the scriptures will becom aggressive? Not necessarily. Violent stories that teach moral lessons or ths are balanced with descriptions of victims' suffering or the aggressor' remorse can teach important lessons and have legitimate artistic merit." 41
Unfortunately, the story they used in their experiments does not teach an moral lessons. ITo the contrary, the full text describes murder, capturin women as slaves and ways to get around promises that you've made.) Ther is very little description of the victims' suffering aside from the insult felt b the master of the concubine. Nor is there much remorse for their acts of gene cide. So it's difficult to see what important prosociallessons are being taugh
The authors tie this finding to the behavior of contemporary terrorists wh are religious fundamentalists, stating that reading selected violent scripture might partially account for their behaviors. In essence, they say that this typ of violent religious story will have a good effect on most people, but a ba effect on a population that's at greater risk of violent behavior due to oths causes. But video game opponents argue the opposite when it comes to the technology: the games are dangerous because their influence over real-worl aggression and violence extends to a broad range of people.
What's sauce for the goose should be sauce for the gander.
General Document Comments 0
I don’t believe it is farfetched to say that the Hollywood code was established as a means to controlling the hysteria created by the possible effect that people believe the movies had on people. It was a new media readily available to the masses and since no one really understood it at that point many things could be attributed to it, good and bad. Since most medias cannot be fully understood until something new comes around I feel that this same hysteria that video games has taken away from films will be around till something new comes around. If you think about it we refer back to films and television a lot when we are talking about video games. In part two there’s a quote that explains what I mean “It was the change in technology that triggered grave concerns about the influence of violent media on youth.” But he was talking about the kids in the 1800s. He said that since graphic books (in description, not graphic novels) did not worry people because few people could read back then and few people could afford it. So when dime novels came around everyone went hysterical. In that same way video games are now readily available to the masses. Just like the novel, just like movies. But since they are the new thing they take all the heat. They even say that “researchers looking for evidence about their long-term effects often point to well-known studies of TV violence”. Why? Because just like us, they don’t know how to really define video games as a media and without a working definition they cannot be examined and judged, so instead they compare it to older medium and use though standards but is that okay? I know I do it too, all the time. I compare videogames to movies because I see the connections and I can’t really compare video games to them. But since video games are a media all on its own shouldn’t the masses have a working definition for it? (yes this is kind of like our paper prompt but I’m talking about the masses not individual scholars and not ourselves)
This entire chapter gives an excellent illustration of the problem of study the effect of video games (or any media) on violence (or any behavior). Researchers make myriads of assumptions about various media, often based in their own biases. Then researchers often use dubious methods to attempt to measure ill-defined variables (like the horn test for aggressiveness). There is also the problem of generally small sample sizes (how can you control for thirty variables when you only have thirty participants?). Finally there is the persistent incentive to overstate the results of a study so the paper might actually get published. Any research into this subject ultimately has to broached with a great deal of skepticism, yet news outlets do not often impose even a small amount of criticism on these studies.
icons on the left to see existing comments.
0 archived comments