The search for better labour market data
14 Dec 2023
Daniel Muir, Research Economist (Fellow)
This blog was originally published on the website of Adzuna, one of the largest online job search engines in the UK.
One area currently of concern to policy makers is the extent to which labour market tightness that has created upwards pressure on wages has been feeding through into prices, and whether interest rate rises aimed at combatting inflation might now be leading to an economic slowdown and rising unemployment. In order to answer these questions, researchers require high quality labour market data.
Historic sources: The LFS
The Labour Force Survey (LFS) has historically been one such source. This statistical survey conducted by the Office for National Statistics has been collecting data on the employment circumstances of the UK population since 1992. It is the largest household study in the country and provides the official measures of employment and unemployment, crucial headline indicators of the health of the nation’s labour market.
However, response rates to the survey have been falling for a while, with the pandemic accelerating this decline when face-to-face interviews ceased, forcing the ONS to rely on phone interviews. A smaller sample of survey responses reduces the confidence we can have in the headline estimates (i.e., the margin of error around the central estimate). Furthermore, differences in the likelihood of an individual responding to the surveys by group introduces the potential for the estimates to also be biased (i.e., not centred on the true figure).
This issue came to a head last month, when the ONS was unable to publish official estimates based on the LFS, relying instead on experimental estimates which took previous headline estimates from the LFS and updated them using HMRC’s PAYE RTI data and the Claimant Count, which themselves have their own issues making the fact that these were deemed more reliable that the official LFS estimates even more alarming.
The ONS is more than aware of these issues, and has put in place plans to improve the LFS in order to reintroduce it as the regular source of official estimates on labour market activity, as well as continuing with the transition towards a ‘Transformed’ LFS next year.
Alternative data sources: ValueMyCV
Whilst these are all positive steps, survey based measures will always face the potential of low response rates and a biased sample. There is arguably thus the need to find alternative data sources, based on actual labour market activity as opposed to self-reported labour market activity, devoid of these issues that can provide reliable indicators of the headline data points that policy makers use to shape public policy.
One such potential dataset is the online CV data collected by Adzuna, one of the UK’s largest job search engines. Jobseekers can upload their CVs to Adzuna in order to use its ValueMyCV tool, which helps individuals improve their CV as well as providing suggestions on the types of roles they might be suited to. Adzuna has collected this data since January 2017, with hundreds of thousands of CVs uploaded every year since.
This simple datapoint (the counts of CVs uploaded each month) might have the potential to offer leading insights on movements in unemployment. (The data is collected real-time and could be analysed accordingly, whereas the LFS releases occur with a two-month lag). Job search activity is naturally directly related to whether someone is unemployed or not, and online job search is the main method of job search nowadays – the launch report of the Commission on the Future of Employment Support showed that searching newspapers, journals or online (likely heavily skewed towards the latter) is by far the main method used by the unemployed to search for work.
Therefore, using online job search data represents a potential source of data devoid of some of the problems currently being faced in measuring unemployment using traditional data sources.
Before looking at how the count of CVs uploaded tracks to measures of unemployment over time, we need to consider which measures of unemployment it might be related to.
In a given period, a certain proportion of the stock of unemployed individuals will upload their CV to Adzuna, according to some sort of survival model. And it is also likely that as an individual enters unemployment, there is a certain likelihood that they will also engage in this behaviour. As such, we present the CV upload count alongside both the level of unemployment and flows into unemployment (from employment and economic inactivity).
Of course, you can also be a jobseeker without being unemployed – individuals in employment regularly search for alternative employment to what they currently have. As such, we’ll also look at how data on job-to-job flows tracks against the CV upload counts. These two different sources of job search activity will likely affect the CV upload count in different ways – an individual will largely enter unemployment before then uploading a CV to Adzuna (potentially affecting the ability of this measure to act as a leading indicator), whilst a job-to-job move would follow the CV upload. This difference in the order of events means that more complex analysis than simply looking at trends in this activity over time (as we do here for simplicity’s sake) is warranted.
Alternatively, the link between CV uploads and the two different sources could be isolated by splitting CVs uploaded into those whose most recent employment history was open (i.e., a potential job-to-job move) or closed (i.e., relating to unemployment).
Another point to consider with the extent to which CV upload counts might link to the level of unemployment in the UK is that individuals from outside the UK can also upload their CVs to Adzuna. Looking at a sample of 100 CVs uploaded in January 2021, of the 76 that contained a location, 8 of these came from individuals overseas (so approximately 11%). At this stage we aren’t able to remove these individuals from the measure, which is one source of noise that will interfere with any relationship with unemployment (there are several others that we discuss later on).
So, let’s take a look at the data
We plot three month moving averages of the CV upload count, the level of unemployment, the level of flows into unemployment, and the level of job-to-job moves against each other over time since February 2017, using the previous and next month in the average for the month in question – this reduces the leading nature of the CV upload count by a month, but aids comparability to the other datapoints.
Flows data (inflows to unemployment and job-to-job moves) is only released by the ONS quarterly, so we interpolate the quarterly data using a cubic spline to approximate monthly counts (obviously not a perfect option).
We use the measures of unemployment and job-to-job flows that are not seasonally adjusted, given that there is no seasonal adjustment to the CV data. Also, given the differences in the levels of each of these data points, we index them using their February 2017 level as the base.
Observations
At the start of the period, there appears to be very little connection between the CV upload count and any of the potential measures that it might be linked to.
Across Autumn 2017 to Summer 2018, trends in unemployment inflows do look like they somewhat lead trends in the CV upload count, although this largely disappears through the rest of 2018 and into 2019 where the CV upload count remains fairly flat. However, from the end of 2019 through to the end of 2021, movements in the CV upload count track remarkably well with movements in unemployment inflows (aside from between November 2020 and February 2021), with the peaks in these measures at the end of this period occurring in the order that (according the hypothesised order of events around a CV upload) we would expect i.e., unemployment then CV upload count then job-to-job flows (although on the later this may just be coincidence given the high degree of seasonality in this measure in particular).
Moving into November 2021 through to February 2022, all three measures of labour market activity that might be linked to CV uploads are falling whilst the upload count is rising steeply. Across the remainder of 2022 and the start of 2023, there looks like there is some commonality in trends, although diverging trends again emerge in Spring 2023. The latest CV upload count data is ticking upwards, which might link to the rises in unemployment we have seen in recent months’ data.
Based on this then, the evidence on any link between labour market activity and the CV upload count is mixed. There are periods where the link looks quite strong, and other periods where trends are divergent. An indicator that works well sometimes and doesn’t work at all at others is not particularly helpful if we aren’t able to know which of these periods we’re in.
Caveats
While I suggested earlier that job search activity is directly related to unemployment, naturally unemployment is not solely related to job search activity. Individuals have to select into uploading a CV to Adzuna, and this decision will likely be affected by a range of factors that vary over time other than just whether the individual is unemployed or not, thus affecting any link between the CV upload count and the component of this process directly related to employment status which is what we want to try to isolate.
The prevalence of online job search, awareness/ exposure of Adzuna and the ValueMyCV tool, and the composition of the unemployed population are a few examples of such factors. Through more complex modelling, these factors could be controlled for, although finding the right measures to use for this might be quite challenging.
On this issue, let’s also remind ourselves of how unemployment is defined according to the International Labour Organisation:
“a person who is without a job, has been actively seeking work in the past four weeks and is available to start work in the next two weeks;
or who is out of work, has found a job and is waiting to start it in the next two weeks.”
For the first type of individual, being without a job can be identified by whether the individual’s most recent employment history is open or closed, and the actively seeking work part can be either assumed based on the fact they have uploaded a CV to a job search engine or (if one wanted to be more rigorous) based on whether they are actively searching on the site and/ or clicking on job ads (data which Adzuna also holds). The being available to start work part can also to a certain extent be assumed based on them actively engaging in job search activity – one could also look at the start dates of the ads they were looking at – although this is arguably not known.
The second type of unemployed person (out of work because they’re waiting to start a job) however would not be able to be picked up by this data. In the four quarterly LFS datasets that combined cover July 2022 through to June 2023, approximately 62,000 of the on average 1.2 million unemployed people were waiting to take up a job already obtained i.e., about 5%.
Other datapoints
There are other potential job search activity datapoints that could be used to understand what is happening to unemployment beyond the CV upload count.
For instance, Adzuna also tracks the number of searches that occur on its site. The level of search activity will likely closely track to the level of unemployment. The intensity of search activity among jobseekers who upload a CV to Adzuna that have their search activity tracked could be used to account for the potential impact that changes in search intensity could have on the link between total search activity and unemployment. This ‘indicator’ will also suffer from some of the other issues previously discussed (including non-unemployed individuals also performing searches), but nonetheless there might be something of value in this too.
Another point to consider in all of this is the fact that we are effectively exploring whether the CV data might provide an alternative, potentially better, measure of unemployment than the LFS which based on the survey response issues detailed might not be measuring the true level of unemployment accurately, but in order to test the accuracy of the CV data as a measure of the true level of unemployment we are comparing it to the very data which it is suggested might no longer be accurate. Indeed, part of the breakdown in the link between the CV upload count and unemployment data in more recent periods might be in part due to the fact that this is when the reliability of the LFS has got even worse.
Experimental statistics, such as a measure of unemployment based on Adzuna’s CV data, will always be viewed as in second place to traditional data sources. The best use of an unemployment measure based on the CV data (that would be based on much more thorough work than this) would likely be as part of a dashboard of indicators which in combination with traditional sources like the LFS, as well as other experimental statistics such as the claimant count, can triangulate the true trends in unemployment based upon which policy decisions want making.
Conclusions
As policy makers continue to have to tackle seismic socio-economic challenges, and our traditional sources of key economic and labour market data are struggling to provide accurate insights, there is a real need to seek out alternative data sources that can fill this information gap that is necessary for effective policy responses. While a lot more work needs to be done to truly understand the link to unemployment and how reliable the data is, online job search activity datasets including online CV data from Adzuna could be one such solution.
A simple comparison of trends between the CV upload count and measures of labour market activity suggests that there might be a strong link between the two during certain periods, while in others this appears to breakdown entirely. I believe that the instances where this link looks to be quite strong gives support for further work to be done looking into this data in order to understand whether there is value to found in it as a labour market indicator. Should it be found to be an accurate, and potentially leading, measure of unemployment, this would be of immense use to policy makers.
Any views expressed are those of the author and not necessarily those of the Institute as a whole.