Basic Concept of Time Series (2)

In my last post, I talked about some basic concept of time series, for example, components of time series data, feature engineering, and visualization. This post is the second part of introducing some fundamental concepts about time series and ARIMA model.

The data we will use in the ARIMA model could be downloaded from here.

First thing first, let me continue some basic concepts of time series. Let’s recall the feature engineering in time series first. In univariate time series, the data file only includes the date, and the data series. In order to utilize a supervised learning algorithm to model, we need to transform the data into input feature and output target. The approach to do it is using shift function with the number of lag observations, and the rolling window statistics.

After create input features by applying lag feature and rolling window statistics, sometimes we need to conduct resampling and interpolation.

Resampling

Resampling refers to change the frequency of time series observations. Resampling includes upsampling and downsampling. Upsampling means we increase the frequency of the sample, and downsampling means we decrease the frequency of the sample.

The reason for resampling is that, sometimes the data may not available at the same frequency that we want to make predictions. For example, I want to predict a product demand in the next day, but I only have the data at each month. Or I want to predict the demand in the next month, but my data is for each day. The other reason is for feature engineering. Resampling can also be used to provide additional structure or insight into the learning problem for supervised learning methods.

Pandas provides a function named resample() to easily resample the series. For example, if I have data series that has the frequency of the series at Month and I want to upsample it to increase the frequency of series to Day.

upsampled = series.resample(‘D’).mean()
upsampled.head(30)

After the resample, we will have the data for the rest days of one month as NaN.

The next step is using interpolate to fill the data points. There are two ways to interpolate the data points. One is using the method as linear, which draws a straight line. The other way is using polynomial or spline to connect the values.

Another resampling is downsampling as mentioned above. Downsampling is to decrease the frequency of the series, so that it does not need interpolation. The code and plot below shows how to downsample from month to quarter.

Data Transform

Another important data preprocessing in time series is data transform. The purpose of data transform is to remove noise and improve the signal in time series forecasting. There are many different transforming techniques, and it can be very difficult to select a good, or best transform for a given prediction model. Power transform is a popular and useful approach to try. Power transform normally include square root transform, log transform, and box-cox transform. Box-cox transform actually is very flexible and can conduct square root transform and log transform by setting different values of Lambda.

Box-cox:

1. lambda = -1.0, reciprocal transform
2. lambda = -0.5 reciprocal square root transform
3. lambda = 0.0 is a log transform
4. lambda = 0.5 is a square root transform
5. lambda = -1.0 is no transform

The example below shows how to use box-cox to make a log transform:

Moving Average Smoothing

Moving average smoothing is a naive but effective technique in time series forecasting. It can be used for data preparation, feature engineering, and for directly making prediction. Smoothing refers to remove the fine-grained variation between time steps. The goal of smoothing is to remove noise and better expose the signal of the underlying causal process.

Calculate moving average is pretty straightforward: specify a window size, or window width by defining the number of raw observations used to calculate the average value. Then calculate the average of the number of raw lag observations in the original series.

There are two main types of moving average:

1. centered moving average:

center-ma(t) = mean(obs(t-1), obs(t), obs(t+1)

As we can see that the centered moving average requires knowing the value of the future values (t+1), so it is used on time series analysis to understand the dataset. It can be used to remove trend and seasonal components from a time series.

2. Trailing moving average:

trial_ma(t) = mean(abs(obs(t-2), obs(t-1), obs(t))

Obviously, trialing moving average uses historical data, so that it is used on time series forecasting. The example below shows hoe moving average smoothing using centered moving average to remove noise.

The next example shows how to use trialing moving average to predict future.

White Noise

Another important concept in time series forecasting is white noise. A time series is white noise when the variables are independent and identically distributed with a mean of zero. IID, we learned this concept in the Ordinary linear regression already. When a time series is IID, it means all variables have the same variance and each value has a zero correlation with each other.

Why understanding if a time series is white noise or not is important in time series forecasting? Basically because if a time series is white noise, it means it is random, and we cannot reasonably model it and make predictions.

Check if a time series is white noise:

1. if it has a non-zero mean?
2. variance change over time?
3. values correlate with lag values?

Conclusion

Today we covered some more basic concepts about time series. In the next post, I will continue to talk about more concepts about time series, the ARIMA model, and the LSTM. Thanks for reading. Please keep tuned!