Open Data Open Jobs
In 2016, the Commonwealth Center for Advanced Research and Statistics ( CCARS ) initiated a pilot project to create an open “real-time” data set of advertised job postings in Virginia. This data set is the initial outcome of the pilot. Work will continue to build upon what we were able to accomplish in just a few short months with more data streams offered in “real-time” and more robust data enrichments.
Currently, the Open Data Open Jobs data set combines data from three sources:
- A daily feed of jobs from the National Labor Exchange made available by the DirectEmployers Association, Inc.
- A snapshot of jobs in the Virginia Workforce Connection from mid-February 2016 made available by the Virginia Employment Commission
- A feed of schema tagged jobs available through an open API built by Devis for the Veterans Job Bank.
We at the Discovery Analytics Center, Virginia Tech ingested, cleansed, enriched, and de-duplicated the data from each source to come up with a single data set. Below is an overview of the major steps taken to achieve this result:
- Mapped job postings from all three sources to the job-posting schema standard
- Enriched job postings with average wage data from the Georgetown University’s Center for Education and the Workforce and job title normalization assistance from Glassdoor
- De-duplicated the data using an algorithm to identify identical job postings
Currently, this data is limited in its reach and should be used with some caveats in mind. This data set does not cover all job openings in Virginia advertised online. Not all sources of data used to create this set are “real-time.” Currently, data supplied from the Virginia Workforce Connection is a snapshot of data from a point in time. Efforts are underway to explore access to these jobs in “real-time.” Additionally, the schema tagged jobs pulled into this data set are limited to those jobs tagged with a “Veteran Hiring Commitment”. Work is in progress to find an alternative source of these jobs that extends beyond the Veteran population. Skills and education data enrichments partnerships are in progress as well.
Why online job postings data?
Online job postings data is a relatively new resource for understanding labor markets in “real-time.” But, it offers enormous potential in helping employers to signal their need for talent, and educators and trainers to respond to those needs. We still have a lot of work to do to understand the quality of information employers signal in job postings for talent with specific qualifications, such as skills, workforce credentials, and degrees. But, that’s why we have made this data open to challenge the academic and public sectors to help us understand the potential this data has in helping to connect Virginians to jobs!
What can I do with the data?
You can use the data to create applications or visualizations that can help connect Virginians to job opportunities, offer insights into the needs of employers by occupation, skills, or education requirements, or create predictive models to help Virginia determine its future needs for talent!
Here are some burning questions you can help us to answer:
• What are the skills required of jobs? How do those skills align to available training and curricula at Virginia’s community colleges?
• What jobs require an academic certificate or associate degree in addition to a license or industry certification? What jobs require a high school diploma and a license or industry certification?
• What is the demand for degrees and credentials by level and type? How are they connected to education programs and majors?
• Are the real needs of employers adequately reflected in job postings?