Multi-task learning of time series and its application to
the travel demand

arXiv:1712.08164v1 [cs.LG] 21 Dec 2017

Boris Chidlovskii
Naver Labs Europe
6 chemin Maupertuis, 38240 Meylan, France
boris.chidlovskii@naverlabs.com

Abstract. We address the problem of modeling and prediction of a set of temporal events in the context of intelligent transportation systems. To leverage the
information shared by different events, we propose a multi-task learning framework. We develop a support vector regression model for joint learning of mutually dependent time series. It is the regularization-based multi-task learning
previously developed for the classification case and extended to time series. We
discuss the relatedness of observed time series and first deploy the dynamic time
warping distance measure to identify groups of similar series. Then we take into
account both time and scale warping and propose to align multiple time series by
inferring their common latent representation. We test the proposed models on the
problem of travel demand prediction in Nancy (France) public transport system
and analyze the benefits of multi-task learning.

1

Introduction

Time-dependent entities and their analysis represent a great challenge in multiple domains, including finance, science, medicine, entertainment, etc. In the domain of transport, intelligent transportation systems have been designed to track and analyze a large
number of time-dependent events, such as vehicle position, road load, travel time and
demand, service reliability, traffic density, etc. Efficient planning and management of
transportation network, in particular, an adequate response to changing traffic conditions, require an accurate modeling and real-time prediction of all these time-dependent
entities.
It is common to represent time-dependent entities as time series. In this paper we
address the problem of simultaneous modeling and prediction for a set of time series.
Our work is motivated by the problem of travel demand prediction in a public transport
system. Figure 1 shows the public transport network of Nancy city (France), Figure 2
reports three time series representing the daily passenger travel demand in different
network locations. In these series, one item refers to the number of passengers boarding
public transport vehicles (buses and trams) during a certain period of time. Similar and
dissimilar patterns are easily recognizable in the series, they are often relevant to peak
and off-peak times.
In statistics and signal processing, autoregressive models have been intensively used
to study time-varying processes. An autoregressive (AR) model specifies that the output variable depends linearly on its own previous values. Combined with the integrated

Fig. 1. Nancy public transport network.

moving average (IMA) it forms a more general ARIMA model of time series; the later
can be further generalized to a SARIMA model that takes into account seasonal effects [11]. Multiple techniques developed by the machine learning community, such as
neural networks, perceptrons, support-vector machines and regressions, have been also
adapted to model time series [17].
In the transportation domain, the time series has been a subject of multiple studies
in the context of traffic flow forecast [20]. The most recent study presents an extensive
experimental protocol allowing to compare both autoregressive models and machine
learning techniques, and to infer a general picture of the advantages and limitations of
each approach [9].
In the most conventional setting, all time series are modeled individually. However,
if they are related and share important information, a multi-task learning approach can
be proposed to model all of them in a joint way.
Complex dependencies between multiple time series is of high interest for anomaly
detection [14,4]. Causality analysis discovers the temporal dependencies between a
large number of time series data coming from different source, like sensors, logs or
physical measurements. Inferring the Granger causality between series became particularly popular, for its scalability and interpretable insights on the possible reasons of
anomalies [14].
The multi-task learning is aimed at leveraging the information of multiple, mutually
related learning tasks to make more accurate predictions for the individual tasks [3]. Related information contained in tasks can be exploited to mutually increase the quality
of predictions. Examples of successful application domains for multi-task learning include computational biology, natural language processing, computer vision [8], where

multiple biological, textual and visual object classes may share some of the relevant
features.
In the multi-task learning, the prediction accuracy in each task is leveraged by making use of data from the other tasks. Previous work has focused on relating the tasks via
regularization [5], mutualization [6], covariance sharing [2] and mainly addresses the
classification case. In this paper we extend the regularization-based technique [5] to the
time series.

Fig. 2. Three daily travel demand time series.

Our motivation example for developing a multi-task learning approach is the travel
demand prediction in a public transport network. We integrate a number of relevant
features in prediction models.
We first collect historical data of the travel demand in a public transport system.
It is extracted from a massive collection of individual passenger boarding events in
the network observed over some period of time. Second, we provide the calendar data,
including weekends, school vacation, public holidays; these data are often critical to
estimate the level of travel demand. Third, we integrate weather information, such as
temperature, precipitation and wind speed, as factors that may also impact the demand.
Lastly, we can benefit from the traffic information, describing the traffic status in a
network segment relevant to a given task.
Unlike multi-task learning for the classification, the mutual relatedness is harder
to assess in time series, as they are often a subject of time shifting, distortion and rescaling effects. To address this problem, we first use the dynamic time warping distance
measure, it helps identify similar series and group them together. Then we develop
an alternative approach which attempts to infer a latent representation common to all
observed time series. Such a common profile is used to align all series in Viterbi-like
manner used in Markovian models.
The contribution of this work can be summarized as follows:
1. We develop a multi-task learning approach to the joint learning for a set of time
series.

2. For a set or related time series, we design a support vector regression model derived
from regularization-based techniques developed for the classification case.
3. For a given set of time series, we first use the dynamic time warping to measure
their similarity; this helps identify clusters of related time series.
4. We then propose another approach to address both time and scale warping of a set
of time series, it infers their common latent profile and use this profile to align them
among themselves.
5. We test both methods on the travel demand prediction in Nancy public transport
system and analyze advantages of each method.
The remainder of the paper is organized as follows. The problem of time series
prediction in single and multi-task setting is introduced in Section II. In Section III, we
describe the regularization-based support vector regression for multi-task learning. Section III.A addresses the series relatedness and describes how the dynamic time warping
distance can be used to identify similar time series. Section IV proposes an alternative
approach to the series relatedness problem; it introduces the generative model for inferring a common latent representation of a set of time series and aligns them by both time
and scale warping. Section V reports evaluation results of experiments with multi-task
learning on a travel demand data collected in real public transport network and analyses
the results. Finally Section VI concludes the paper.

2

Time Series Prediction

Time series analysis and prediction theory have been intensively studied over last 40
years [17,11]. Well known applications are financial market prediction, environment
and weather forecasting, biology, control systems, and many other applications involving non-linear processes [13].
The goal of time series prediction is to estimate value y at time i based on past
values yi−1 , yi−2 , . . .. It is common to limit the depth to the last k values: yi =
f (yi−k , yi−k+1 , ..., yi−1 ), i = k, . . . , n, where k ≥ 1 is the modeling depth (lag).
Moreover, it can take into account a number of external characteristics vi at time i
which may have impact on the series values:
yi = f (vi , yi−k , yi−k+1 , ..., yi−1 ), i = k, . . . , n.

(1)

The objective of time series prediction is to find a function f (xi ), where xi =
(vi , yi−k , yi−k+1 , ..., yi−1 ) such that ŷi , the predicted value of the time series at a future
point in time, is consistent and minimizes a regularized fit function.
The underlying system models and time series data generating processes are generally complex and the models for these systems are usually not known in advance.
Accurate and unbiased estimation of the time series data produced by these systems
cannot always be achieved using linear techniques, and thus the estimation process requires more advanced time series prediction algorithms.
2.1

SVM for Time Series Prediction

The Support Vector Machine (SVM) is machine learning technique which has been
successfully used for different tasks such as pattern recognition, object classification,

and in the case of time series prediction and regression analysis. In Support Vector
Regression (SVR), a function for a time series is estimated using observed data. Being
a data-driven methodology, SVMs differ from more traditional time series prediction
methodologies, like ARIMA or Kalman Filter [17].
The underlying motivation for using SVMs for time series is the ability to make
accurate prediction when the underlying system processes are typically nonlinear, nonstationary and not defined a-priori. In a number of applications, SVMs have been proven
to outperform other non-linear techniques including neural-network based non-linear
prediction techniques such as multi-layer perceptrons [9,20,12].
Given a set of time series data (xi , yi ), i = 1, . . . , n, a time series prediction method
defines a function f (x) that will have an output equal to the predicted value for some
prediction horizon. By using regression analysis, the prediction function for the linear
regression is defined as f (x) = wT x + b. If the data is not linear in its input space,
the data x are mapped into a higher dimension space, via a kernel function φ(x), then
perform a linear regression in the higher dimensional feature space [12,16]:
f (x) = wT φ(x) + b.
The goal is therefore to find optimal weights w and threshold b, as well as to define the
criteria for finding an optimal set of weights.
Given training data (xi , yi ), i = 1, . . . , n, xi ∈ Rd , yi ∈ R, SVM first maps input
vectors x onto the feature space Φ, φ(x) ∈ Φ and then approximates the regression
by a linear function f (x) = wT φ(x) + b. This is obtained by solving the following
optimization problem in the ε-insensitive tube [19]:
Pn
1
2
∗
min 
i=1 (ξi + ξi )
2 ||w|| + C
T
 yi − (w φ(xi ) − b) ≤ ε + ξi
(wT φ(xi ) + b) − yi ≤ ε + ξi∗ , i = 1, . . . , n
s.t.
 ∗
ξi ≥ 0, ξi ≥ 0, i = 1, . . . , n

(2)

where ξi , ξi∗ , i = 1, . . . , n are slack variables, measuring the deviation from ε-insensitive
tube, C is the regularization parameter.

3

Multi-task SVM Regression

Suppose we dispose a set of m related time series, m > 1. Training samples are represented as a set {(xri , yri ), r = 1, . . . , m, i = 1, . . . , n}. We denote the indices from
series r by Tr = {ir1 , . . . , irn }, r = 1, . . . , m. 1
Making an assumption of relatedness between the tasks serves us to design the
multitask learning method. We assume that the tasks are related in a way that the true
models are all close to some common model w [5] and every task model wr can be
written as w + wr , r = 1, . . . , m, where the individual vectors wr are small when the
tasks are similar to each other.
1

Without loss of generality, we make assumption of the same length n for all series.

We adapt the regularization approach [5], according to which vectors of each task
xi , i ∈ Tr are mapped into two different spaces. One is a space Φ common to all tasks,
φ(xi ) ∈ Φ; another is a correction space Φr , specific to every task r, φr (xi ) ∈ Φr .
Individual task information is accounted in the slack variables, defined as follows:
ξi = wrT φ(xi ) + br , i ∈ Tr , r = 1, . . . , m
ξi∗ = wr T φ(xi ) + b∗r , i ∈ Tr , r = 1, . . . , m.

(3)

All slack variables are non-negative, ξr (xi ), ξr∗ (xi ) ≥ 0, r = 1, . . . , m. Thus samples maped in the correcting space have to lie on one side of the corresponding correcting function. The correcting function also has to pass through some points with slack
variables being zero.
The goal of the multi-task version of SVR is to estimate m regression models, one
model per task. Multi-task learning SVR incorporates individual task information into
estimated regression functions through the slack variables. We specify the following
parameterized functions for m regression models:
f (x) + fr (x) = wT φ(x) + b + wrT φr (x) + br , r = 1, . . . , m,

(4)

where f (x) = wT φ(x)+b is the common estimation function and fr (x) = wrT φr (x)+
br is a correction function for series r, r = 1, . . . , m. Therefore the MTL SVR formulation solves the following optimization problem:
Pt
Pn
1
2
2
r
r∗
min 
i=1 ||wr || + C
i=1 (ξi + ξi )
2 ||w|| + µ
r
T
T
 yi − (w φ(xi ) + b + wr φr (xi ) + br ) ≤ ε + ξir
(wT φ(xi ) + b + wr T φr (xi ) + br ) − yir ≤ ε + ξir∗
s.t.
 r∗
ξi ≥ 0, ξir ≥ 0, i = 1, . . . , n, r = 1, . . . , m.

(5)

where µ is a regularization parameter for the individual correction functions. Using
the dual optimization technique (similar to standard SVM), the dual form of the above
optimization problem is as follows:
Pn
Pn
max −ε Pi=1 (αi∗ + αi ) + i=1 (αi∗ − αi )yi
n
− 21 i,j=1 (αi∗ − αi )(αj∗ − αj )φ(xi )T φ(xj )
Pt P
T
1
− 2µ
(α∗ − αi )(αj∗ − αj )φr (xi ) φr (xj )
 P r=1 ∗ i,j∈Tr i
i∈Tr (αi − αi ) = 0, r = 1, . . . , m,
s.t.
0 ≤ αi , αi∗ ≤ C, i = 1, . . . , n.

(6)

Then vectors w, wr can be expressed in terms of training samples:
w=
wr =

Pn
∗
i=1 (αi − αi )φ(xi ),
P
1
∗
i∈Tr (αi − αi )φr (xi ), r = 1, . . . , m.
µ

(7)

Note that in addition to usual parameters of individual SVR, C and ε, the MTL
extension requires also to tune parameter µ.

3.1

Multiple Series Relatedness

In the previous section, multi-task learning was assumed to be applied to a set of mutually related time series. Ensuring the task relatedness is crucial; applying the multi-task
learning to unrelated series may have a negative impact on the prediction quality.
A pair of time series is said to be mutually related if they behave similarly and
show up the same tendencies. To address the relatedness between time series, we first
consider conventional measures of similarity which is of fundamental importance for
time series analysis [7]. The straightforward approach to measure the similarity between
two time series is to evaluate the Euclidean distance on their normalized representation.
However, Euclidean distance ignores any shifting and distortion in time series and has
been found unsuitable in different domains [18].
Besides the Euclidean-based distance measure, the time-warping distance has become one of the most popular and field-tested similarity measures. The dynamic time
warping (DTW) distance is efficient as the time-series similarity measure for minimizing the effects of shifting and distortion in time; it allows elastic transformation of time
series in order to detect similar shapes with different phases. Given two time series,
DTW algorithm yields the optimal solution in the O(n2 ) time.
Since many time series mining algorithms use similarity search as a core subroutine,
To avoid that the time taken for DTW similarity becames a bottleneck, multiple efforts
have been undertaken to improve the complexity of the DTW distance algorithms. In
recent work [15], a novel DTW similarity algorithm for time series is proposed in order
to search and mine massive time series. It permits to search with DTW similarity faster
than with the Euclidean distance algorithm. For such problems as motif discovery and
time series clustering, it scales up to very large datasets accounting for billions of time
series.
Below we use the DTW similarity to identify groups of similar time series and
then apply the multi-task learning to each of the identified clusters. Algorithm 1 below
summarizes the multi-task learning on a given set of time series.

4

Time series common profile

In the previous section, we grouped time series by using their DTW distance as similarity metric. This helps identify similar tasks but does not guarantee their relatedness.
In this section, we make a step further in the analysis of time series relatedness, by
taking into account both time warping and scale warping when aligning two or more
time series. Our approach is based on an assumption that all series in the set are noisy
and rescaled replicates of one common and latent representation [10]. To leverage the
information contained in such noisy replicates, such a common representation can be
inferred and used to align the series among themselves in an appropriate way.
The latent trace is modeled as an underlying, noiseless representation of the set
of observed noisy time series. Output time series are generated from this model by
moving through a sequence of hidden states and emitting an observable value at each
step, as in Hidden Markov Machines (HMMs). To account for changes in the value
amplitude within and across replicates, the latent time states are augmented by a set of

Algorithm 1 Multi-task learning for related and unrelated time series.
Require: Set of time series y1 , . . . , ym , yr = (yr1 , . . . , yrn ), r = 1, . . . , m
Require: Number of clusters L
Require: Observations (vr , . . . , vrn ), including weather, calendar and traffic data, for every
series yr .
1: for every pair (yr , yt ), 1 ≤ r < t ≤ m do
2:
Measure the similarity between series yr and yt by the dynamic time warping (DTW)
distance.
3:
If series are too long (more 10,000 elements), uniformly sample series before applying the
DTW algorithm.
4: end for
5: Apply agglomerative clustering algorithm to form L clusters G1 , . . . , GL of series using
DTW distance as metric. Any series yr belongs to exactly one group.
6: for each group G of relates tasks do
7:
Apply the multi-task learning described in Section 3
8: end for
9: return Multi-task models for all groups and tasks

scale states, which control how the emission signal will be scaled relative to the value
of the latent trace. The latent trace is learned in unsupervised manner, as well as the
transition probabilities controlling the Markovian evolution of the scale and time states
and the overall noise level observed data.
The Continuous Profile Model (CPM) [10] is a generative model for a set of m time
series, y1 , y2 , ..., ym . Time series in the set are noisy replicas of a common and latent
representation, defined as a trace z = (z1 , . . . , zM ). Any given time series yr is a subsampled version of z after applying local scale transformations. Size M of z is larger
than time series size n so that any experimental data could be mapped precisely to the
correct underlying trace point. In practice, M = λn with 1 < λ < 3.
The resolution of the latent trace z is higher with respect to the observed time series,
so the experimental time can be made effectively to speed up or slow down by advancing
along the latent trace in larger or smaller jumps. The sub-sampling and local scaling
used during the generation of each observed time series are determined by a sequence
of hidden state variables. The state sequence for series yr is denoted π r . Each state in
the sequence maps to a pair (time state, scale state), πir → {τir , ψir }, where time state
τir ∈ (τ1 , . . . , τM ), and scale states ψir ∈ (ψ1 , . . . , ψQ ), Q is the number of scale states.
The probability of emitting observed value yir in state πir is defined by the emission
probability distribution Aπir (yir |z) ≡ p(yir |πir , z, σ, ur ) which is defined by probability distribution N (yir ; zτir ψir ur , σ), where σ is the noise level of the observed data,
N (a; b, c) denotes a Gaussian probability density for a with mean b and standard deviation c. The ur are real-valued scale parameters, one per observed time series, that
correct for any overall scale difference between time series r, r = 1, . . . , m, and the
latent trace z.
The transitions between time states are independent from transitions between scale
r
states, so p(πir |πi−1
) = p(ψi |ψi−1 )pr (τi |τi−1 ). To enforce that time must move forward, any time state can jump ahead no more than Jτ states. Similarly, scale state tran-

sitions are allowed between neighboring scale states only, this disallows arbitrary jumps
on the local scale. These constraints keep the number of legal transitions to a tractable
computational size. Each observed time series yr has its own time transition probability
distribution to account for experiment-specific patterns. The time transition probability
distribution for yr is given by multinomial distribution as follows
 r
dl , if a − b = l, l = 1, . . . , Jτ ,
r
p (τi = a|τi−1 = b) =
(8)
0, otherwise.
where

PJτ

i=1

dri = 1. The time scale probability transition for all series is given by

 s0 , if D(a, b) = 0,
p(ψi = a|ψi−1 = b) = s1 , if D(a, b) = ±1,
(9)

0, otherwise,

where D(a, b) = ±1 means that a is one scale state larger (+1) or smaller (-1) than b,
and D(a, b) = 0 means that a = b and the scale state does not change. The distribution
Jτ are constrained by 2s1 + s0 = 1.
Jτ determines the maximum allowable speedup of one portion of a time series relative to another portion, within the same series or across different series. However, the
length of time for which any series can move so rapidly is constrained by the length
of the latent trace; thus the maximum overall ratio in speeds achievable by the model
between any two entire time series is given by min(Jτ , λ).
After training, one may examine either the latent trace or the alignment of each observed time series to the latent trace. Such alignments can be achieved by several methods, including use of the Viterbi algorithm to find the highest likelihood path through
the hidden states [1], or sampling from the posterior over hidden state sequences. Viterbi
alignment is found to work well in our experiments.
Training the CPM with the EM algorithm is similar to HHMs with Baum-Welch
algorithm. The E-Step is computed exactly using the Forward-Backward algorithm,
which provides the posterior probability over states for each time point of every observed time series and the pairwise state posteriors. The M -Step consists of a series of
analytical updates to the various parameters, see [10] for more detail.
Figure 3 shows an example of alignment for five travel demand times series.

5

Evaluation

We run experiments to test learning methods described in previous sections on the task
of travel demand prediction in Nancy (France) public transport network. Two alternative
approaches to model a set time series are tested in different settings and scenarios. One
is a single-task learning approach where a separate support vector regression model
is estimated for each task independently. Another is the multi-task learning approach
where one common and a number of individual correction models are jointly learned to
estimate related regression models. A set of time series representing the travel demand
are analyzed and pre-processed. In one instantiation, they are clustered by using the
DTW distance as described in Algorithm 1. In another, a latent profile common to all

Fig. 3. Time and scale alignment of 5 travel demand time series.

the series is inferred and then all series are aligned to this common profile, as described
in Section IV.
Public is GDP time series include annual per capita GDP time series for several
countries. One interest lies in studying the ”periodic” behaviour of such series in connection with understanding business cycles. Another lies in forecasting turning points.
Another lies in comparisons of characteristics of the series across national economies.
We dispose the following dataset to evaluate different prediction models:
– Historical data of the travel demand, which accounts for the total of 5,29 M passenger boarding events registered at 1,067 stops and 28 tram and bus lines in the
city network; the dataset covers the period of 90 days from March to May 2012, including 13 weekends, 2 weeks of school vacations and three public holidays. Each
boarding transaction includes the route, stop, direction and the timestamp of the
event. Time series describing the travel demand are then generated by the aggregation of boarding timestamps at a regular time period. In the tests, two different
periods are used to produce the time series, of 10 and 60 minutes.
– Weather data include the air temperature (minimum and maximum), air humidity,
precipitation, wind speed, etc. All data are registered hourly 2 .
– Calendar data, including the binary tags for weekends, school vacations and public
holidays.
– Traffic information on the network segment relevant to a given task. The average
bus adherence delay accumulated between timestamps i − 1 and i for each series is
used.
2

Available from weather station Nancy-Ochey at http://www.meteociel.fr/temps-reel/
obs villes.php?code2=7181.

Protocol and experimental setup We compare the performance of different methods
for time series by using the standard measure of Mean Square Error (MSE). It is defined
as follows
n
1X
2
(yi − ŷi ) ,
(10)
EM SE =
n i=1
where yi is the true value at time point i in the test set while ŷi is the prediction.
Experiments for the travel demand prediction are conducted on two levels of the
public transport network; one on city zones, another for individual stops. For the zone
demand experiment, a preliminary analysis of the travel dataset has been done, with the
goal to extract all travel patterns required for the inference of origin-destination (OD)
matrices. The city network is then segmented into zones, where one zone is defined as
a compact city region with stops showing up a high similarity of their travel patterns.
Figure 5 demonstrates the segmentation of Nancy city network into 15 zones, where
blue points refer to the bus and tram stops and each zone is shown as a convex hull
built around geographical locations of all stops included in the zone. For this case, we
generate 15 times series describing the travel demand in all zones.
For the stop demand experiment, one tram and four bus lines have been selected.
For each line, a representation set of 10 stops has been selected to produce the set of
travel demand time series.
The evaluation prototype has been implemented in MATLAB as extension of MALSAR toolkit for multi-task learning via structured regularization [21].

Fig. 4. GDP per capita: a) 9 series, b) 1T and MT learing.

In all experiments, all time series are split into training and testing sets in the same
way. The training set includes starting P % items (days); the testing set includes remaining 100 − P %, where P varies from 40 to 90.
For each of two experiments, we evaluate three different learning approaches in
different scenarios:
1. We compare the prediction performance of the three following methods:
– Single-task (ST) learning, where individual SVR models are learned and tested
independently; the average of their MSE values is reported;

Fig. 5. Nancy city zoning by travel demand.

– Multi-task (MT) learning, where one common SVR model and one correction
model per time series are learned; the series are clustered by their DTW distance, as described in Section 3.
– MT learning with CPM alignment, where the time series get aligned to their
common profile, as described in Section 4.
2. We run experiments with two different demand aggregation periods, of 10 minutes
and 1 hour.
All time series are normalized and scaled to the logarithmic space. In SVR, the
linear kernel function φ(x) = x is preferred to the Gaussian one. The depth k = 2
for the past values (lag) was determined by cross-validation and remains the same in
all experiments. Parameters C and ε are automatically tuned as in the standard SVM
regression [19], the optimal value of parameter µ is determined by re-sampling.
Algorithm 1 for clustering time series by their DTW distance takes as input the
number of clusters L. In the zone experiment, where we produce 15 time series, the
number of clusters to test is between 2 and 15. In the stop experiment, 10 time series
represent any selected line, and the number of clusters is between 2 and 10. The optimal
number of clusters is detected by the cross-validation, the best results are reported for
L between 3 and 6 in the stop experiment and 4 or 5 in zone experiment.
For the CPM inference and alignment, seven latent traces z1 , . . . , z7 are inferred
from the daily times series in the training set, one for each day of week. At the test
time, a daily time series yi as well as relevant observations vi get aligned to the same
day trace z using the Viterbi algorithm. The size M of the latent for series of size n
is λn where λ = 2.5. The scaling space size Q is limited to 5 states, they are evenly

spaced on logarithmic scale. The MSE values are then averaged over 7 day-specific
experiments. Finally, the number of iterations for EM algorithm was limited to 20.
In all figures presented below, the whisker plots are used to show the mean, variance
and outliers of MSE values.
Stop experiments Figure 6 reports evaluation results of single- and multi-task learning
for the stop experiment with 1 hour period. In addition to the three referred methods,
it also includes MSE values for the single-task learning without external observations
vi (weather, calendar and traffic features). As one can see, taking external observations into account reduces the error values by 40%-60%; this measures the impact of
external observations on the predictive behavior of support vector regression models
3
. The multi-task learning with DTW outperforms the single-task learning by 2%-7%,
multi-task learning with CPM further reduces the error by 3%-12%.
Figure 7 reports the error values for the stop experiment, this time with 10 minute
period. Shorter period of data aggregation leads to a higher variance in time series; this
results in higher error values. Both multi-task learning methods improve the predictive
performance with respect to single-task learning, with multi-task learning with CPM
being the best.

Fig. 6. Stop demand prediction for 1 hour period.

Zone experiments Figure 6 reports the evaluation results of the referred methods for
the zone experiment in 0-delay scenario, with 1 hour period. The multi-task learning
with DTW outperforms the single-task learning by 0%-9%, multi-task learning with
CPM reduces the error by 3%-12%.
Figure 6 reports results for the same experiment in 1-delay scenario. Unavailability
of the last value in series makes the prediction harder for all methods, both in terms of
the mean and the variance. The multi-task with DTW plays the equal game with the
single-task learning or outperforms it in some cases.
3

Analysis of individual contribution of external observations is beyond the scope of this study.

Fig. 7. Stop demand prediction for 10 minute period.

In this experiment, multi-task learning with DTW performs better than the singletask learning for few values of L; in other cases, it often shows a worse performance
(see Figure 9.a for cases P =40% and 90%). The multi-task learning with CPM resists
better to the data delay, it outperforms the single-task learning by 3%-16%.

Fig. 8. Zone demand prediction.

Discussion Analysis of the evaluation results in the previous subsections suggests that
multi-task learning can offer a comfortable performance improvement over single-task
setting. However, unlike the multi-task learning for classification, the task relatedness
raises a particular concern. A simple normalization is insufficient and advanced techniques are required to leverage the information shared by different time series.
Between two methods presented in Sections III and IV, the continuous profile model
(CPM) gives an advantage over the DTW distance based clustering. Considered as a
straightforward choice to identify related time series, Algorithm 1 is sensible to the way
the times series get clustered. Moreover, DTW-based alignment does not guarantee the
relatedness of selected tasks.

Fig. 9. a) MT with DTW for different number of clusters. b) MT with CPM for different numbers
of stops.

Time series alignment with CPM seems to solve the problem, the method appears
more stable, it yields a performance improvement in a majority of experiments and
scenarios. The number of time series we tested in both zone and stop experiments is
modest and do not raise particular issues when inferring daily latent traces, even for
10 minute period. In Figure 9.b we relax the number of stops in the stop experiment.
Instead of 10 representative stops, we pick up randomly 2 to 20 stops, and compare
multi-task CPM to single-task learning, for P =40% and 90%.
Computationally, multi-task DTW and CPM methods behave similarly, due to the
straightforward dynamic programming implementation of the DTW algorithm and a
reasonable convergence of the EM algorithm when inferring 7 daily latent traces and
Viterbi-based alignment of the observed time series.

6

Conclusion

In a public transport network, an accurate estimation of the number of travelers entering
the network at any period of time is crucial for efficient planning of transport services.
The estimation model is learned from historical data and relevant observations. For a
set of time series describing the demand, the single-task learning can be further turned
into by multi-task learning in order to leverage the information shared by different time
series. In this paper, we describe the regularization-based support vector regression for
multi-task learning and propose two ways of addressing the series relatedness. One
way is to cluster times series by their dynamic time warping distance. Another way
is to infer a latent representation common to all time series and then align the series
by this common representation. We apply the proposed methods on the travel demand
prediction in Nancy public transport network and report the evaluation results for singleand multi-task learning methods.

References
1. L. E. Baum, T. Petrie, G. Soules, and Weiss N. A maximization technique occurring in
the statistical analysis of probabilistic functions of markov chains. Ann. Math. Statist.,
41(1):164–171, 1970.
2. Edwin Bonilla, Kian Ming Chai, and Christopher Williams. Multi-task gaussian process
prediction. In Proc. Neural Information Processing Systems (NIPS) 2007, Vancouver, BC,
Canada, 2007.
3. R. Caruana. Multitask Learning. Machine learning, 28(1):41–75, 1997.
4. S. Chawla, Yu Zheng, and Jiafeng Hu. Inferring the root cause in road traffic anomalies. In ,
2012 IEEE 12th International Conference on Data Mining (ICDM), pages 141–150, 2012.
5. Theodoros Evgeniou and Massimiliano Pontil. Regularized Multi-Task Learning. In Proc.
10th ACM SIGKDD Intern. Conference on Knowledge Discovery and Data Mining, KDD
’04, pages 109–117, New York, NY, USA, 2004. ACM.
6. Jean Baptiste Faddoul, Boris Chidlovskii, Rémi Gilleron, and Fabien Torre. Learning Multiple Tasks with Boosted Decision Trees. In Proc. ECML/PKDD, pages 681–696, 2012.
7. Tak-Chung Fu. A review on time series data mining. Engineering Applications of Artificial
Intelligence, 24(1):164 – 181, 2011.
8. J.M. Leiva-Murillo, L. Gomez-Chova, and G. Camps-Valls. Multi-task Remote Sensing
Data Classification. IEEE Transactions on Geoscience and Remote Sensing, 51(1):151 –
161, January 2013.
9. Marco Lippi, Matteo Bertini, and Paolo Frasconi. Short-term traffic flow forecasting: An
experimental comparison of time-series analysis and supervised learning. IEEE Transactions
on Intelligent Transportation Systems, 14(2):871–882, 2013.
10. Jennifer Listgarten, Radford M. Neal, Sam T. Roweis, and Andrew Emili. Multiple alignment
of continuous time series. In Proc. NIPS, 2004.
11. Terence C. Mills. Time Series Techniques for Economists. Cambridge University Press,
1990.
12. Klaus-Robert Müller, Alex J. Smola, Gunnar Rätsch, Bernhard Schölkopf, Jens Kohlmorgen,
and Vladimir Vapnik. Predicting Time Series with Support Vector Machines. In Proc. 7th
Intern. Conference on Artificial Neural Networks, ICANN ’97, pages 999–1004, London,
UK, UK, 1997. Springer-Verlag.
13. S. J. Orfanidis. Optimum Signal Processing: An Introduction, 2nd ed. New York: MacMillan, New York, 1988.
14. Huida Qiu, Yan Liu, N.A. Subrahmanya, and Weichang Li. Granger causality for time-series
anomaly detection. In Data Mining (ICDM), 2012 IEEE 12th International Conference on,
pages 1074–1079, 2012.
15. Thanawin Rakthanmanon, Bilson Campana, Abdullah Mueen, Gustavo Batista, Brandon
Westover, Qiang Zhu, Jesin Zakaria, and Eamonn Keogh. Searching and mining trillions
of time series subsequences under dynamic time warping. In Proc. 18th ACM SIGKDD Intern. Conference on Knowledge Discovery and Data Mining, KDD ’12, pages 262–270, New
York, NY, USA, 2012.
16. Stefan Rüping. SVM Kernels for Time Series Analysis. In Proc. of Tagungsband der GI
Workshop-Woche, 2001.
17. N. Sapankevych and R. Sankar. Time Series Prediction Using Support Vector Machines: A
Survey. Computational Intelligence Magazine, IEEE, 4(2):24 –38, may 2009.
18. Pavel Senin. Dynamic time warping algorithm review. In Technical Report CSDL-08-04.
Information and Computer Science Department. University of Hawaii at Manoa. Honolulu,
USA, 2008.

19. Alex J. Smola and Bernhard Schlkopf. A Tutorial on Support Vector Regression. Statistics
and Computing, 14(3):199–222, 2004.
20. Man-Chun Tan, S. C. Wong, Jian-Min Xu, Zhan-Rong Guan, and Peng Zhang. An aggregation approach to short-term traffic flow prediction. IEEE Transactions on Intelligent
Transportation Systems, 10(1):60–69, 2009.
21. J. Zhou, J. Chen, and J. Ye. MALSAR: Multi-tAsk Learning via StructurAl Regularization.
Arizona State University, 2011.

