Your Comprehensive Guide to Time Series Analysis and Forecasting

When it comes to our data and the vast amounts of time and effort invested in collecting the same, we like to assume, it is quite robust. Perhaps we’d be stretching things if we said the data is sufficient by itself (which is never quite true), but it serves the purpose of analysis and fitting a good model onto the processed and engineered data.

Potentially our dataset is comprised of information collected over a significant period of time, going into years or even decades. Think of volatile weather and storm prediction for a region where the history of occurrence may date back to the early 20th century. And yet the spate of recent storms puts more emphasis on the last 15 years alone.

More often than not we consider all parts of the whole as equal. Yes, we may seclude the part of the data that has more bearing and then treat the analysis like any other. But the time factor as a varying dimension is ignored. It is only with the introduction of a time component into our analysis that we begin to express the scenario more accurately. The data then speaks of occurrences that formed a sequence and were recorded in such a sequence. Data thus considered sequentially is known as a time series.

A look at Time Series Analysis

Think of the patterns generated by stock prices or the fluctuations in demand for a certain beauty product, or the demand for seasonal fruit. By nature, the time-dependent data for these occurrences allows for developing patterns. We then observe attributes of relevance and come up with insights gained from analyzing such a series. This is also called descriptive modeling.

Here are the key attributes of a time series analysis:

Level — This is the base component or platform upon which the variations of time series can be explored. A straight line is used to represent it.

Trend — The series may have a trend if a pattern of increase or decrease in values can be observed over time. This regularity in the correlation with time may sustain for a period.

Figure 1: A linearly increasing trend is apparent in this time series

Seasonality — If a behavior becomes cyclical such that the pattern is seen to be repeating over a period of time, we have seasonality in the time series.

Figure 2: Seasonality can be spotted with a pattern repeating itself over a period of time

Noise — The inexplicable portion of the variability observed with time is known as noise.

Let’s have a quick overview of the attributes or features because a combination of these attributes in many ways results in the time series for the observations at hand.

The level is a platform that must always be present. We may observe trends and seasonality in the series though not as a rule. It is worth noting that most observations made close together in a sequence are often related when we have a trend or seasonality present in the time series. There is almost always some noise in the series that does not hint at any pattern but depicts an element of randomness. The noise may not contribute to the time series as much as the other features but cannot be neglected.

Time Series Forecasting

The time series analysis, as mentioned before, is a form of descriptive modeling. In order to take this beyond analysis and attempt prediction, we must look at forecasting. If we had to define this in a nutshell, forecasting can be seen as extrapolating the future observations or future of the series based on the sequential data or the existing time series available.

The word available is not to be taken lightly. Our forecasting hinges on how much data we actually have access to. The other key aspect of forecasting is setting realistic expectations for how far we can go into the future. The shorter the time period for which we are extrapolating, the more reliable our forecasting is.

Stationarity

Prior to describing models in regular usage, we shall touch upon stationarity to further our understanding. A time series is said to be stationary when its observations are independent of the time at which they were recorded. In other words, statistical measures like the mean and variance of the series are free from the influence of time and therefore appear constant over a period.

As a rule of thumb, there will be no sign of a trend or seasonality (attributes that reflect a time-dependent pattern) in a stationary series. The phenomenon known as stationarity is a common assumption when we work with forecasting and data is often manipulated to make it stationary for the convenience of modeling.

A measure of stationarity can be obtained by the autocorrelation function ACF also known as a correlogram.

Any fluctuation in values of the correlogram would occur rather slowly as in the first plot below.

Figure 3a: Correlogram for a non-stationary time series

If the series has turned stationary these fluctuations will be more sudden in nature moving from positive to negative values instantaneously.

Figure 3b: Correlogram for a stationary time series

Models in Popular Usage

AutoRegressive: The predicted values for a future time are a linear function of the past values for the same variable. Observations that were documented at these posterior points in time are better known as lags.

yt= βo + β1yt-1 + β2yt-2 + … + βnyt-p + ε

In the above equation, we have n number of past observations. This can be called an autoregressive model of the pth order. The coefficients of the linear function are developed using the training data. The intercept ε refers to noise in the series. This is a simplistic model that works reasonably well with data that is stationary.

Moving Average: Moving averages are handy tools for neutralizing fluctuations in available data. If the first term of the equation is the mean of a certain number of past values, sliding one step backward gives the second term with another coefficient. It is represented by MA(q) where q is the order of the series.

yt = µ + εt + θ1 εt-1 + θ2 εt-2 + θ3 εt-3 +… θq εt-q

A term of a few days, weeks, or another fixed period is decided upon as the moving period. Here, the linear function would incorporate the residual errors from the moving periods and hence deliver a predicted value for a future period.

AutoRegressive Moving Average: Commonly referred to as ARMA, the linear function in this case would combine the coefficient terms from the autoregressive series and the errors from the moving period to obtain predicted values.

yt= βo + β1yt-1 + β2yt-2 + … + βpyt-p + εt + θ1 εt-1 + θ2 εt-2 + θ3 εt-3 +… θq εt-q

AutoRegressive Integrated Moving Average: We may choose to use autoregression and moving average techniques for non-stationary data with this model better known as ARIMA. Integration refers to a modification performed on the data in order to make the series stationary. The orders are taken for each of the aspects — autoregression (p), moving average (q), and integration (d) and passed as parameters to the model.

The order of integration (d) refers to the number of differencing operations performed on the series.

Seasonal Autoregressive Integrated Moving Average: A seasonal component added to the ARIMA model enables both trend and seasonality to be accounted for. It works well for non-stationary data.

Seasonal Autoregressive Integrated Moving Average with Exogenous Regressors: When we wish to add the ability to deal with variables in a parallel time series that matched the observation sequence of the original time series, we may use this model better known as SARIMAX. In such cases, the presence of an exogenous series has a significant impact on the original time series. The presence of a trend or seasonality in the series would be welcome as far as the model is concerned.

Holt Winter’s Exponential Smoothing (HWES): The future periods in this model will be predicted with exponential weights alongside the observations in a linear function. With trend and seasonality included, this model is suited for non-stationary data.

Vector Autoregression Moving Average: The generalization of AutoRegression Moving Average to run multiple time series in parallel; this may also be used to develop vector models without one of these parameters, viz. excluding moving average or autoregression. This would be typically used for bivariate or even multivariate analysis.

Vector Autoregression Moving Average with Exogenous Regressors: The general assumption with a stationary time series allows the Vector Autoregression Moving Average to be utilized without the presence of any trend or seasonality. Parallelly, if we were to introduce a series with observations that coincided with the sequence of the original series, we would term the primary series as endogenous and the add-on as exogenous. This would be well suited to multivariate analysis.

Facebook Prophet: This open-source algorithm developed by Facebook scientists is generally seen as a well-designed model to account for seasonality. There are multiple components that make up a prediction or forecast viz. growth, seasonality, holidays, and error. Growth here refers to not only the trend in the time series but also the direction shift in data, which the model understands as a changepoint. Varying the changepoints can conveniently change the forecast projected by the series. Holiday as an element of the forecast function refers to events during the calendar year that are expected to alter immediate forecasts. This element clearly requires some domain knowledge and takes into account the past trend.

Evaluating a Forecasting Model

When it comes to time series forecasting, the process of evaluation is known as backtesting. The model built upon past data is tested for its robustness, keeping in mind the temporal nature of the series. Having observations that are time-dependent defies the conventional evaluation metrics used which require that the sequential data be independent.

A few evaluation techniques are summarized as follows:

Time Series Split — The data is split into train and test datasets multiple times with different proportions even as the test set is kept constant. The different splits can be compared to determine which works better for the unseen test data.

Walk Forward Validation — This method uses the training data to validate the subsequent prediction value each time and compare the model’s forecasts with actual values. The process ends up creating several models, making it expensive. It however addresses the problem with time series split where training data is not changing for newer models.

When it comes to tuning our model, the parameters we pass can be iterated over a range of values. For each permutation, we can compute the Akaike Information Criterion or AIC. The formula for the metric is a function of the sum of squares of the values and the number of degrees of freedom. Our objective would be to minimize this AIC score and try running the model with a combination of parameters that give the lowest scores.

Conclusion

As with any machine learning challenge, the time series depends on the availability of data and certain assumptions. The idea of choosing the right model is working with parameters that can make the most out of the available data.

DataMites is a global institute offering specialized training in some of the happening fields in the current market, including data science, machine learning, python, MLOps, artificial intelligence, python, and many more. Together with international certification from the IABAC, DataMites courses do offer internship opportunities and job assistance.