Skip to content
Search AI Powered

Latest Stories

A Study Asked Whose Lives an Autonomous Vehicle Should Be Programmed to Save, and Yes, It's As Disturbing As It Sounds

A Study Asked Whose Lives an Autonomous Vehicle Should Be Programmed to Save, and Yes, It's As Disturbing As It Sounds
Tetsuya Lijima from Nissan giving a demonstration around the roads of east London of a prototype Nissan Leaf driverless car. (Photo by Philip Toscano/PA Images via Getty Images)

Well, that's unsettling.

While designers of autonomous vehicles (AVs) continue their quest to make them safer, the realities of complex roadways call for complex ethical decisions about who lives or dies. To address the technological version of what’s known as the age-old “trolley problem,” a worldwide study asked questions such as: if one or more pedestrian is suddenly crossing the road, should the AV be programmed to swerve and risk going off the road with its passengers or hit the people head-on?

Most respondents objectively lean toward protecting the greatest number of people, but also show a reluctance to ride in an AV that doesn’t guarantee protection for its passengers as priority number one.


The Study’s Design

Researchers at MIT launched the initial online study in the United States in 2015—an interactive video game that allows people to make ethical choices about how AVs should respond in dangerous situations.

From 2017 to 2018, MIT expanded that reach worldwide to include 4 million people. At the Global Education and Skills Forum in Dubai on March 18, 2018, MIT Professor Iyad Rahwan told the audience that his Moral Machine study is now the largest global ethics study ever conducted. As fate would have it, that was the day before the first United States pedestrian fatality by an AV.

Moral Machine (MM) allows users to choose alternatives when risky road conditions arise, judge which is the most ethically acceptable, and even design alternative solutions from those presented.

MM’s website calls itself: “[a] platform for gathering a human perspective on moral decisions made by machine intelligence, such as self-driving cars.” The instructions explain, “We show you moral dilemmas where a driverless car must choose the lesser of two evils, such as killing two passengers or five pedestrians. As an outside observer, you judge which outcome you think is more acceptable.”

The software creates even more nuanced scenarios, especially if participants click on the button asking to “Show Description,” rather than simply choosing based on the depicted image. The people in each image are described in greater detail: by gender, age, and with characteristics including “homeless” and “executives.” Live animals in the road, such as dogs and cats, are also presented in the situations. People can compare their answers with those of others and discuss them in a comments section.

Results of the Studies

When Rahwan previewed the global studies’ results, which will be published later this year, they largely mirrored the results of the US study: in theory, people wanted autonomous vehicles designed for the greater good, sacrificing the lives of passengers to save the lives of more pedestrians. But they didn’t want to ride in those cars.

When explaining the original US study results, co-author of the study Rahwan said, “Most people want to live in in a world where cars will minimize casualties.” However, he added: “But everybody wants their own car to protect them at all costs.”

Indeed, 76 percent of respondents found it more ethical for an AV to sacrifice one passenger over 10 pedestrians. But that rating fell by one-third when respondents considered the possibility of riding in such a car. In fact, the global study found nearly 40 percent of participants chose to have their own cars run over pedestrians rather than injuring passengers.

The aptly named paper, “The social dilemma of autonomous vehicles,” was published by the journal Science:

“We found that participants…approved of utilitarian AVs (that is, AVs that sacrifice their passengers for the greater good) and would like others to buy them, but they would themselves prefer to ride in AVs that protect their passengers at all costs. The study participants disapprove of utilitarian regulations for AVs and would be less willing to buy such an AV.”

“I think the authors are definitely correct to describe this as a social dilemma,” according to Joshua Greene, professor of psychology at Harvard University. Greene wrote a commentary on the research for Science and noted, “The critical feature of a social dilemma is a tension between self-interest and collective interest.” He said the researchers “clearly show that people have a deep ambivalence about this question.”

The study’s more detailed questions showed clear preferences to protect certain groups and more division in the results as situations grew increasingly complex. Respondents generally chose to spare children’s lives over adults. Furthermore, the more elderly the subject, the more respondents were willing to risk those people’s safety. With multi-faceted situations, such as a pedestrian crossing legally while multiple people crossed illegally, results were evenly split.

Researchers also noted some intriguing cultural differences between eastern and western countries in the global study, with Germany standing somewhat as an outlier. Generally, the preliminary results from the global study indicate that people in Western countries, including the US and Europe, favor the utilitarian ideals of minimizing overall harm, according to Rahwan. However, respondents from Germany ran against the grain from their surrounding neighbors.

“When we compared Germany to the rest of the world, or the east to the west, we found very interesting cultural differences,” said Rahwan. Presumably, these specific differences will be detailed in the study to be published later this year.

In line with Rahwan’s findings, German company Mercedes-Benz executive Christoph von Hugo commenced a scandal with his comments in early October 2016, by stating that the company will prioritize the lives of its AV passengers—even at the expense of pedestrians.

“If you know you can save at least one person, at least save that one,” von Hugo said at the Paris Motor Show. “Save the one in the car. If all you know for sure is that one death can be prevented, then that’s your first priority.”

A few weeks later, von Hugo claimed he was misquoted, attempting to clarify the company’s official position that “neither programmers nor automated systems are entitled to weigh the value of human lives.” He added that the company has no legal authority to favor one life over another in Germany or any other country.

Moving Forward with Ethics in AVs

While society come to terms with the idea that AVs will unavoidably kill people—but hopefully less often than those driven by humans—some important decisions need to be made about the ethical judgments those AVs will make.

Rahwanexplained: "The problem with the new system [sic] it has a very distinctive feature: algorithms are making decisions that have very important consequences on human life."

But the goal of the study is not simply to decide what ethical guidelines should drive the AVs’ algorithms, but also what society will require before they are willing to accept the risks and use them.

The researchers acknowledge in the 2016 US study that public-opinion polling on this issue is at a very early stage—indicating current findings “are not guaranteed to persist” if the landscape of driverless cars evolves. However, the early findings in the world-wide study two years later present similar results.

But NuTonomy CEO Karl Iagnemma, whose Cambridge-based company has piloted self-driving taxis in Singapore, questions the value of the detailed study, emphasizing that AV technology is nowhere near ready to distinguish between many of the details tested in the global study.

"When a driverless car looks out on the world, it's not able to distinguish the age of a pedestrian or the number of occupants in a car." Iagnemma, added, "Even if we wanted to imbue an autonomous vehicle with an ethical engine, we don't have the technical capability today to do so."

Likewise, Noah Goodall, a scientist at the Virginia Transportation Research Council, said the intense focus on the rare and dramatic “trolley problem” consumes too much energy. He believes the time would be better spent on typical day-to-day algorithmic issues that still need addressing, including how much space to leave between cars, pedestrians and cyclists.

"All these cars do risk management. It just doesn't look like a trolley problem," Goodall said.

But Rahwan explained, “I think it was important to not just have a theoretical discussion of this, but to actually have an empirically informed discussion.”

With this research, he’s trying to prevent a situation where AVs are designed in such a way that people refuse to buy them, and the life-saving industry collapses as a result.

"There is a real risk that if we don't understand those psychological barriers and address them through regulation and public outreach, we may undermine the entire enterprise," Rahwan explained. "People will say they're not comfortable with this. lt would stifle what I think will be a very good thing for humanity."

Even if people grew more comfortable with the technology, and it improved over time, the study clearly demonstrates that “defining the algorithms that will help AVs make these moral decisions is a formidable challenge.”

More from News

Jason Ritter; John Ritter
@jasonritter1980, ABC

Fan Asks Jason Ritter To Do His Dad's 'Creep Face' From 'Three's Company'—And He Nails It

Actor Jason Ritter humbly obliged a fan who requested he mimic his late father John Ritter's goofy expression often seen in the 1970s sitcom that made him famous.

John Ritter was known for playing the likable culinary school student Jack Tripper on Three's Company, which ran for eight seasons from 1977 to 1984.

Keep ReadingShow less
Ariana Grande during interview with lie detector test
Vanity Fair/YouTube

Ariana Grande Hilariously Frets After Lie Detector Suggests She Thinks Moon Landing Was Fake

In the excitement over the film adaptation ofWicked, Ariana Grande and Cynthia Erivo have toured for various appearances and interviews, and one of their most anticipated appearances was with Vanity Fair to have an interview while wearing a lie detector.

When asked who would like to go first, Grande was quick to volunteer but also asked if the machine would be able to pick up on her anxiety.

Keep ReadingShow less
Screenshot of Corey DeAngelis
Fox News

Anti-LGBTQ+ Activist Whose Gay Adult Film Past Was Exposed Now Claims He Was A 'Victim'

Following allegations that he starred in several gay adult films, anti-LGBTQ+ activist Corey DeAngelis admitted in a post on X, formerly Twitter, that he'd made "mistakes" while claiming he was the "victim of poor decisions and poor influences."

DeAngelis, a senior fellow at the American Federation for Children—a conservative organization that promotes "school choice," the use of public dollars to help families pay for private school tuition and homeschooling—was placed on leave after the organization learned of allegations that he appeared in videos on the adult-film site GayHoopla under the alias Seth Rose, including a film titled Jerk Off Race.

Keep ReadingShow less
JD Vance and Tim Walz
CBS News

Harris Campaign Turns JD Vance's 'Damning Non-Answer' On 2020 Election Into Brutal New Ad

Following last night's vice presidential debate, Vice President Kamala Harris's campaign hit back at former President Donald Trump's running mate J.D. Vance after Vance refused to say Trump lost the 2020 election, turning his "damning non-answer" into a brutal new ad.

During the debate, Kamala Harris's running mate Governor Tim Walz asked Vance directly who won the 2020 election and instead of answering, Vance pivoted away from the question altogether. Vance also minimized the significance of the insurrection of January 6, 2021, when a pro-Trump mob stormed the U.S. Capitol in an attempt to halt the certification of the election results.

Keep ReadingShow less
Mark Zuckerberg
Chris Unger/Zuffa LLC via Getty Images

Mark Zuckerberg Sparks Debate After Using AI To Illustrate His Young Daughter's Book

Meta CEO Mark Zuckerberg divided the internet by using AI to bring his six-year-old daughter August's illustrations to life.

Zuckerberg posted a video of his presentation at the annual Connect conference explaining how he and his second daughter used Meta AI to illustrate and create a trailer for her book, The Mermaid Crystal.

Keep ReadingShow less