The government reported 165,000 new jobs created according April's nonfarm payroll numbers (176,000 in total in the private sector) – a pleasant surprise to most economists, who were anticipating fewer. Part of the reason that expectations were off was because the Automatic Data Processing (ADP) jobs report predicted that only 119,000 would be created, an apparent error of 57,000. Why is this discrepancy a big deal?
Jobs reports used to be exciting only for economists and stockbrokers, but since the election season, every political junkie and their dog seems to have taken an interest. People recognize that the economy plays a vital role in deciding votes; these reports, therefore, offer a vital clue to predicting the politicians' election chances. So, now we have both the economic and political class yearning for 8.30am on the first Friday of the month, all to learn about the jobs numbers.
But as in so many arenas in America, people can't wait to see what happens. They race to get the answer as quickly as they can, picking up on whatever clues they deem fit. Enter the ADP jobs report, a jobs survey released two days before the official Bureau of Labor Statistics (BLS) government report. Many use the ADP to predict the BLS, but past ADP surveys have sometimes been far off actual BLS results. As Steven Russolillo noted in September, "some months, it's spot on; others it's wildly off base."
The ADP, hoping to make its data more accurate, made some major changes for its October 2012 report. That month, the ADP started using ADP payroll data, BLS employment data, and the Philadelphia Federal Reserve's Aruoba-Diebold-Scott Business Conditions Index. As a result, ADP surveyed 62,000 more clients than previously, 2 million more employees, and two more company-size classes and industries. They brought on Moody's Analytics to replace Macroeconomic Advisers for processing data. To put it mildly, these are not small changes.
Have the adjustments brought ADP any closer to solving the monthly jobs mystery?
To answer this question, I've compared ADP forecasts of the past seven months with the same seven-month period last year, and looked at the ADP's accuracy in predicting final BLS numbers. The BLS produces an initial, second, and final report as it calculates more data, and the data between reports can differ greatly. The ADP wants to land as close as possible to the BLS's final report, though most attention is usually paid to the initial report.
The past seven months have seen an average difference between the ADP and initial BLS report of 42,286 jobs. (You can see all the data here.) Some months, such as October and November 2012, had errors of under 30,000 jobs, while March and April 2013 saw errors of 57,000 or greater. No month had an initial error of less than 26,000; the error in margin ends up within +/-19,000 of 45,000.
Compared to the same time last year, the average error has, in fact, diminished. Last year, ADP was off by an average error of 55,429 jobs, which is 13,143 jobs greater than their more recent average. This difference, however, is not statistically significant, due to a small sample size (seven observations) and the fact that the old ADP results could sometimes be very accurate.
Last year, three months under the old methods had errors of 17,000 or less, compared to the initial BLS report – far more accurate than any month per the new ADP. The problem for the old ADP was that four months last year had errors of 66,000 or greater, which less accurate than all seven months of the new ADP.
In that light, the new ADP does look better than the old. When it comes to their forecasts and the initial jobs report, we still haven't seen an error so wrong it makes your eyes pop out. Of course, we haven't seen stunning accuracy either.
The BLS's final jobs report, however – what ADP should supposedly be best at predicting – apparently confounds ADP. We see zero consistency in their results. Four out of six final reports (or second report for March 2013, since we don't have the final one yet) have had errors of 28,000 or less. Two final reports, December 2012 and March 2013, have been within 4,000. November 2012 and February 2013, though, have seen errors of over 120,000 jobs! The old ADP, by comparison, had its biggest miss last year, in January 2012, at 107,000 jobs.
The average error of the new ADP on final BLS reports has been 51,167 jobs, which is actually worse than the ADP's error on initial BLS reports. It's better than the average error of 58,333 from last year, but it's not better by a statistically significant amount.
When you put it all together, I can't really say that ADP has done better with its new methodology than it did with its old. There are some signs that the changes have made it more accurate – perhaps those huge misses of November 2012 and February 2013 will turn out to be anomalies – but we'll need a larger sample size to know for sure. But at this point, it looks as likely as ever that the ADP numbers will be way off-the-mark measured against the BLS's final reports.
The smart bet right now? Have a little patience and wait for the actual government statistics.