In one of the odder moments of his presidency, President Donald Trump thanked musician Kanye West for boosting his support among African Americans.
"Kanye West must have some power because I doubled my African-American poll numbers. We went from 11 [percent] to 22 in one week," Trump said during his NRA convention speech in Dallas. "Thank you, Kanye, thank you. Even the pollsters thought there must be some mistake."
He continued:
Now we've come a long way. You remember I'd come into big rooms, big audiences, and I'd say, 'What do you have to lose?' Because the Democrats have always had their votes. Horrible on crime, horrible on education, horrible on everything. I'd say 'What do you have to lose?' And they voted for me. And we won, but now the numbers are much higher than they ever were before with African Americans, and we're happy.
Trump's comments came after West pledged his support of the president, attributing their respective successes to the "dragon energy" they share.
West has only continued to assert his support on Twitter.
But as much as it might please the president that West––who was named one of Time magazine's 100 most influential people in 2010––has pledged his support, there is no reliable data to support his claim that his support among African Americans has increased.
On May 2, two days before Trump made his comments at the NRA convention, The Daily Caller published an article titled "Black Male Approval For Trump Doubles In One Week." The website cited Reuters/Ipsos polling, which tracked Trump's approval rating among black men, not the African American community as a whole. On April 22, the weekly tracking poll had "Trump's approval rating among black men at 11 percent, while the same poll on April 29, 2018, pegged the approval rating at 22 percent."
"Reuters only sampled slightly under 200 black males each week," The Daily Caller noted, without explaining why Reuters sampled such a low number and how that would affect the interpretation of the results. The actual results soon became lost in the social media shuffle as people shared the article on Facebook and Twitter. Charlie Kirk, the founder and director of Turning Point USA, even tweeted a snippet from The Daily Caller's story, saying that "If this trend continues and holds, it will be the death of the Democrat party."
The problem with the poll, however, is that it uses a non-probability online sample. A 2014 Pew Research study (which assessed nine non-probability samples) found this methodology doesn't necessarily accurately represent African Americans, let alone other minorities:
Online nonprobability sample estimates based on Hispanics and blacks show particularly large biases. Across the nine nonprobability samples, the average deviation from the benchmarks was 15.1 percentage points for Hispanic estimates and 11.3 percentage points for estimates for blacks. Sample I and the ATP are the only samples examined that have average benchmark deviations in the single digits for both of these subgroups.Estimated biases were also particularly large for young adults. The pattern of larger average biases for younger adult estimates than older adult estimates (11.8 points for ages 18-29 versus 9.6 points for ages 65 and older) is somewhat surprising given that young adults have much higher levels of internet usage, suggesting that they might be better represented in online panels.
Estimated bias also varied by gender. All of the samples in this study had larger biases when making inferences about men than about women. Across the nine nonprobability samples, the average deviation was 9.9 percentage points for men versus 7.6 points for women.
It's unclear how Trump came across the Reuters data, which wasn't common knowledge. Nevertheless, his comment that pollsters "thought there must be some mistake" about his support among the African American contingent bears some weight, because Chris Kahn, the U.S. political polling editor for Reuters, noted in an email with CNNMoney that the 11% and 22% data points cited by The Daily Caller were, in fact, misunderstood.
"The sample sizes for those two measurements were too small to reliably suggest any shift in public opinion," Kahn told CNNMoney through a spokeswoman.
As CNN itself notes:
This is common sense: The more narrowly you slice the data, the less reliable it becomes. Reuters and Ipsos survey about 350 people a day, 11,000 a month, on the internet. The outlets call the relative precision of the results as a "credibility interval."In this case, the president's approval rating among black men, "the credibility interval was more than +/- 9 percentage points for each measurement, which leaves open the possibility that his approval also could have dropped in this time frame," Kahn said.
It could have dropped!
This is why the president's words require constant and close examination. It's risky, even irresponsible, to just quote someone without fact-checking the content -- particularly when the person has a track record like Trump's.
Non-probability online samples do not meet CNN's standard for reporting. The polls that do meet CNN's standard do not show approval ratings that are as high as Trump might suggest.
CNN's director of polling and election analytics Jennifer Agiesta notes that "Gallup's latest tracking data among black respondents finds 13% of African Americans approving of Trump."
Meanwhile, a Pew Research Center survey from late April shows Trump's approval rating at 13 percent. A Quinnipiac University survey from a few days earlier showed only 14 percent approval.
