Real-world time series forecasting is challenging for a whole host of reasons not limited to problem features such as having multiple input variables, the requirement to predict multiple time steps, and the need to perform the same type of prediction for multiple physical sites.
In this post, you will discover a standardized yet complex time series forecasting problem that has these properties, but is small and sufficiently well understood that it can be used to explore and better understand methods for developing forecasting models on challenging datasets.
The competition and motivation for addressing the air-quality dataset.
An overview of the defined prediction problem and the data challenges it covers.
A description of the free data files that you can download and start working with immediately.
Photo by someone, some rights reserved.
The dataset was used as the center of a Kaggle competition.
Specifically, a 24-hour hackathon hosted by Data Science London and Data Science Global as part of a Big Data Week event, two organizations that don’t seem to exist now, 6 years later.
The competition involved a multi-thousand-dollar cash prize, and the dataset was provided by the Cook County, Illinois local government, suggesting all locations mentioned in the dataset are in that locality.
The EPA’s Air Quality Index is used daily by people suffering from asthma and other respiratory diseases to avoid dangerous levels of outdoor air pollutants, which can trigger attacks. According to the World Health Organisation there are now estimated to be 235 million people suffering from asthma. Globally, it is now the most common chronic disease among children, with incidence in the US doubling since 1980.
The competition description suggests that winning models could be used as the basis for a new air-quality prediction system, although it is not clear if any models were ever transitioned for this purpose.
The competition was won by a Kaggle employee, Ben Hamner, who presumably did not collect the prize given the conflict of interest. Ben described his winning approach in the blog post titled “Chucking everything into a Random Forest: Ben Hamner on Winning The Air Quality Prediction Hackathon” and provided his code on GitHub.
There is also a good discussion of solutions and related code in this forum post titled “General approaches to partitioning the models?“.
The data describes a multi-step forecasting problem given a multivariate time series across multiple sites or physical locations.
Given multiple weather measurements over time, predict a sequence of air quality measurements at specific future time intervals across multiple physical locations.
Incomplete data. Not all weather and air quality measures are available for all locations.
Missing data. Not all available measures have a complete history.
Multivariate inputs: The model inputs for each forecast are comprised of multiple weather observations.
Multi-step outputs: The model outputs are a discontiguous sequence of forecasted air quality measures.
Multi-site outputs: The mode must output a multi-step forecast for multiple physical sites.
The dataset is available for free from the Kaggle website.
You must create an account and sign-in with Kaggle before you can get access to download the dataset.
This file contains a list of site locations marked by unique identifiers and their precise location on Earth measured by longitude and latitude.
All coordinates appear to be relatively close in the North-Western Hemisphere, e.g. America.
Below is a sample of the file.
This file has the same format as SiteLocations.csv and appears to list all of the same locations as that file with some additional locations.
As the filename suggests, it is just an updated version of the list of sites.
This file contains the training data for modeling.
The data is presented in an unnormalized manner. Each row of data contains one set of meteorological measurements for one hour across multiple locations as well as the targets or outcomes for each location for that hour.
Time information, including the block of time, the index within the contiguous block of time, the average month, day of the week, and hour of the day.
Wind measurements such as direction and speed.
Temperature measurements such as minimum and maximum ambient temperature.
Pressure measurements such as minimum and maximum barometric pressure.
The target variables are a collection of different air quality or pollution measures at different physical locations.
Not all locations have all weather measurements and not all locations are concerned with all target measures. Further, for those recorded variables, there are missing values marked as NA.
This file contains a sample of the submission for the prediction problem.
Each row specifies the prediction for each target measure across all target locations for a given hour in a chunk of contiguous time.
A large part of the challenge of this prediction problem is the vast number of ways that the problem can be framed for modeling.
This is challenging because it is not clear which framing may be the best for this specific modeling problem.
For example, below are some questions to provoke thought about how the problem could be framed.
Is it better to impute or ignore missing observations?
Is it better to feed in a time series of weather observations or only the observations for the current hour?
Is it better to use weather observations from one or multiple source locations for a forecast?
Is it better to have one model for each location or one mode for all locations?
Is it better to have one model for each forecast time or one for all forecast times?
General approaches to partitioning the models?
In this post, you discovered the Kaggle air-quality dataset that provides a standard dataset for complex time series forecasting.
A description of the free data files that can download and start working with immediately.
Have you worked on this dataset, or do you intend to?
In this book, “Multivariate” is not dealt with.
Multiple inputs or multiple input series is multivariate.
I believe he is referring to your book ad at the end of the post.
Yes. I hope to write a book dedicated to the topic next.
I don’t know of any sorry.
Hi Jason, it will be interesting to see how you will approach this problem. Do you think a reliable predictive model can be built from a dataset with this much missing variables? I look forward to your feedback.
Perhaps, it really depends on the specifics of the data.
Great post! I’m trying to immitate this analysis in python and am having trouble dealing with the missing values in the random forest regression. From what I’m reading, the random forest can deal with the NaNs for the tree splitting but not for the regression.
Looking at Ben’s matlab code it seems he’s “imputing” the missing values to 0. Am I correct in this? I’m not sure imputing the missing values to 0 for a regression is best. Any thoughts?
It really depends on the specific implementation.
Perhaps try the xgboost library? or the sklearn implementation?
Recently I have read Your article on this blog. Since that time I have read a lot of Your articles published here. Your knowledge and expierience is impressive.
I’m trying to solve a multivariete (multiple input data) multi-step (multiple output) forecasting problem and I don’t know, which method should I use.
I have time series data that represent weather(temperature, pressure, humidity, windspeed, wind direction etc.); pollution(pm10,pm2.5, NO2 etc.); traffic(average speed, street traffic congestion factor). I have a data from about ~100 localization inside middle-size city (327 km^2). Measurements are peformed each hour.
Maybe there are other methods I have not heard about. If You had to solve this type of problem, what methods would You use?
I would recommend a deep MLP on this type of problem.
I have one more question. Is there any article on Your blog about making multivariete/multi step timeseries forecasting using MLP?
I was trying to find some materials on the Internet about this method, but i canno’t find anything interesting. I have seen some scientifc papers, but none of them discribed exact the same problem.
I hope to write a book on this topic next.
It would be a very good idea. I would like to read such a book.
Regardless, I will try to solve this problem myself. I just thought you had materials that might be useful to me, but now I understand why You have not published them yet.
Hi Jason, what is your ETA for this book covering deep deep MLP for multivariate time series forecasting?
I bet fixed windowed data i key, what your take on that?
I know its VERT implementation dependent, but in these cases just how deep is deep in your view?
No fixed ETA at this stage. In progress.
Thank you very much for your share in the LSTM timeseries prediction,it help me a lot in learning LSTM.But I still have some confusion about several time steps prediction by LSTM.
The problem is that I have 12 years one dim timeseries data and want to use 84 hours data to predict the next 12 hours data by BP network and LSTM.I find the BP have the better performance,but I think the LSTM will do better,perhaps I have something wrong,could you tell me how to build a multi-steps prediction LSTM network?
I am looking forward to hearing from your!
1) I want to know the difference between sequence prediction and time series prediction . Whether can we use the model that deal with sequence prediction to make time series prediction or not?
I hope to receive your reply.
Time series is a type of sequence prediction. Sequence prediction models can be used for time series.
LSTMs may be good for your problem, but I would encourage you to test a suite of other methods as well to confirm that the LSTMs are skillful.
When working with LSTMs with a variable number of input time steps, you must pad the inputs and use a masking layer to ignore the padding. I have examples on the blog.
Could you elabaorate more on using random forest to do a multi step time series forecast in Python? One of your further reading topics covers this on matlab. However, I don’t have a lot of experience in matlab and I am finding it difficult to understand.
You must split the data into windows with input vectors of past obs mapped to the next observation.
I’ve been looking at your blog all day and found plenty of very interesting articles.
Forecasting the power production (in different fields – solar, wind, etc), with historical production data. I was thinking to improve the predictions by using the dates as an input too (month can be a good information, for solar exposition, and for the wind). I was also thinking of using weather dataset as an additionnal input.
Do you thing a multivariate, multistep time serie could be a good approach (maybe with a LSTM network) ?
I have a more general question. In one of your tutorials you said that one should make the time series stationary. If I have a multivariate and test each variable for stationarity and get the result that some variables should be differenced and some are already stationary, what would you do? Probably it is not reasonable to difference only some of the variables..?
Start by modeling the data directly, then try making each/all stationary and see how it impacts the models ability to learn the problem.