I Object, Your Honor: Pew Is Leading the Witness (and Confirming Its Own Bias) in Its Survey on Automation and Jobs

There’s an old adage about polling—if you tell a pollster what you want to find, then the pollster can design questions to get you those answers. I was reminded of this when I read a recent study by Pew Internet and Society, which polled Americans on their attitudes toward artificial intelligence, robotics, and the future of jobs. The study finds—wait for it—that Americans are worried about automation.

But before we respond with a tax on robots and universal basic income, it’s worth looking a little more closely at the study and the poll. First, Pew itself doesn’t really seem to understand the literature or studies on the issue. The authors cite the infamous Oxford study that claims—with very little evidence and a shoddy methodology, at best—that 47 percent of U.S. jobs will be automated. But they fail to mention that estimate is over a 20-year period, and they fail to mention studies, including ITIF’s, that have called into question Oxford’s results. Surprisingly, they also misinterpret a McKinsey Global Institute study on automation, claiming it said that up to half of the activities people perform could be automated, when in fact McKinsey clearly estimated that only around 5 percent of jobs would be automated. Having failed to adequately look at the logic or the evidence, Pew then accepts the erroneous premise that “robots will be competitive with humans on a widespread scale.”

After such an inauspicious start, the authors proceed to ask biased survey questions that virtually guarantee answers that will confirm their notions of an impending robo-pocalpyse. First, they ask people if they have heard robots could take everyone’s jobs. In fact, they present respondents with the following scenario: “in the future, robots and computers with advanced capabilities may be able to do most of the jobs that are currently done by humans today.” Talk about leading the witness.

Once the pollsters have planted to seeds in the minds of respondents that robots can take all our jobs, they proceed to ask respondents what they think of robots taking all our jobs.

They then ask about views on impacts. Rather than ask how robots will impact inequality, they ask people if they think robots will make inequality much worse than today. Under positive impacts, the pollsters fail to ask the really key question, which is: “Do you think that by reducing the price of goods and services that you and your family need, robots and automation will increase your standard of living?”

The pollsters go on to ask people if they would support or oppose policies that could be put in place if robots are doing many human jobs. But here they fail to ask about likely possibilities—for example, “if the government used the massive tax revenues that a larger economy would produce to cut your personal income taxes,” or “if the government used the massive tax revenues from a larger economy to increase spending on infrastructure and K-12 education.”

When asked whether government has an obligation to take care of displaced workers, even if it means raising taxes substantially, why not ask the more accurate question: “Should government help workers gain the skills needed to find new employment if they are displaced by robots and computers?” I would bet upwards of 80 percent of respondents would say yes.

Finally, the Pew report fails to get at a fundamental contradiction. Most respondents think the odds of their job being automated is low (just 70 percent think it’s likely), but 77 percent think it’s realistic that robots and computers could eliminate most jobs. In other words, it’s easy to imagine someone else’s job being eliminated—what does an insurance salesman know about what a carpet installer does? But when everyone is asked about the job they know best—their own—most people think robots are for vacuuming their floors, not replacing their jobs.

So, Pew, if you are serious about doing good and accurate work in this important area, please go back to the drawing board and start over. This issue is too important for America to be guided by shoddy analyses.