In this work we propose a methodology for developing an occupational classification by applying Natural Language Processing methods, such as document clustering and distributed word representations, to UK online job adverts. The new occupational classification will be directly aligned with employer needs and group jobs into occupations based on similar skill requirements. Unlike the existing UK Standard Occupational Classification taxonomy, the skills based occupational classification methodology will prioritise skill specialisation over skill level. The term skill level refers to the amount of education and training required as well as the range of tasks performed; skill specialisation refers to domain-specific expertise, technology and materials used, and the products and services produced in a given occupation. The resulting classification will have the potential to enable measurement of an individual’s career progression within the same skill domain, to recommend jobs to individuals based on their skills and to mitigate occupational misclassification issues.
To develop the classification methodology, we apply semi-supervised machine learning techniques to a dataset of 37 million UK online job adverts collected by Burning Glass Technologies. The resulting occupational classification comprises four hierarchical layers: the first three layers relate to skill specialisation and group jobs that require similar types of skills. The fourth layer of the hierarchy is based on the offered salary and indicates skill level. The proposed classification will have the potential to enable measurement of an individual’s career progression within the same skill domain, to recommend jobs to individuals based on their skills and to mitigate occupational misclassification issues. While we provide initial results and descriptions of occupational groups in the Burning Glass data, we believe that the main contribution of this work is the methodology for grouping jobs into occupations based on skills.
The resulting occupational classification comprises 16 broad groups, 33 skill categories, 50 skill sub-categories and 150 skill levels (Figure 9).
In this paper we propose a methodology to group occupations on the basis of skill requirements contained in 37 million UK job adverts. The resulting occupational classification captures both the skill specialisations and skill levels of occupations. In its current form, the methodology comprises four hierarchical layers. At the first three layers, we use skills from the adverts to place jobs into groups that require similar domain- specific skills. By identifying these distinct skillsets, we lay the groundwork for quantifying skill demands and analysing the composition of the UK workforce by skill type. The fourth layer of the hierarchy reflects a job’s skill level, on the basis of the salary offered. Integrating a skill level dimension into the classification provides a pathway for the analysis of individuals’ career progression within a given domain-specific skillset.
We believe that this work contributes to the occupational classification field in a number of ways. First, we offer a data-driven approach for dynamically capturing skills, competencies and knowledge required by employers. A vast collection of job adverts is used to develop the methodology, which means that we can gauge the needs of employers across the UK with high resolution and accuracy. The approach is cost effective, because it requires little manual input. The methodology can also be easily extended to work with any skills taxonomy and thus offers policymakers, educators and researchers the flexibility to choose a taxonomy that is most closely aligned with their objectives. Finally, the proposed approach can be applied to analyse skill requirements across all occupations on an on-going basis or to focus on a skillset/occupation of interest. Apart from the choice of the skills taxonomy, the methodology is algorithmic in nature, which means that the methodology can be used to automatically code large volumes of job adverts to occupations.
Further research will help to validate the methodology and increase its relevance to occupational classification practitioners. There is also scope to refine the analytical methods used to develop the methodology by training an occupation-specific word embeddings model and to improve the accuracy of job assignment to reference categories. The results of our work will be released publicly and shared with labour market researchers, with the aim of showing how online job advert data can be used to improve our understanding of labour markets.
Chosen excerpts by Job Market Monitor. Read the whole story at Classifying occupations according to their skill requirements in job advertisements