LSTM for Time Series Forecasting

Long Short Term Memory model (LSTM) is a recurrent neural networks (RNN). RNN is a type of neural network that is powerful for modeling sequence data such as time series, natural language, or speech recognition.

In RNN, connections between nodes form a directed graph along a temporal sequence, which allows it to exhibit temporal dynamic behavior. RNN perform the prediction for every element of sequence with the output that is dependent on the previous computations. RNN makes the use of information from all previous computation as a ‘series’ input, which means RNN captures the relationship across all previous inputs meaningfully. RNN takes one or more input vectors and produce one or more output vectors by not only weights on inputs, but also a hidden vector that representing the context based on previous inputs and outputs. The same input could produce different output according to previous inputs in the series.

Why LSTM model in Time Series

LSTM model is able to predict more than one step into the future. LSTM is very powerful time series models that can predict arbitrary number of steps into the future.

The essential components in LSTM allows it to model both long-term and short-term data.

1. Cell state (Ct) — This represents the internal memory of the cell which stores both short term memory and long-term memories.
2. Hidden state(ht) — This is output state information calculated with current input which is eventually used to predict the future stock market prices. The hidden state can decide to only retrieve the short or long term or both types of memory stored in the cell state to make the next prediction.
3. Input gate(it) — Decides how much information from current flows to the cell state.
4. Forget gate(ft) — Decides how much information from the current input and the previous cell state flows into the current cell state.
5. Output gate(ot) — Decides how much information from the current cell state flows into the hidden state, so that if needed LSTM can only pick the long-term memories or short-term memories and long long-term memories.

The next part is an example using LSTM in stock price prediction.

The dataset could be found here.

First step, load the data, and take a look:

Decomposition of the data:

Second step: Normalize the data for modeling:

Let recall a bit here about normalization.

Feature normalization:

1. Scaling: Standard Scaler z = (x-u) / s
Min Max Scale:xit = (x_i — x_i_min) / (x_i_max — x_i_min)
Normalizer: scales each data point such that the feature vector has a Euclidean length of 1, and often used when the direction of the data matters, not the feature vector.

2. Pipeline: from sklearn.pipeline import Pipeline

pip = Pipeleline[(‘s’, MinMaxScaler(copy=True), (‘lr’, LinearRegression())]

In this case, MinMaxScaler was used for normalization.

Step3 build the model:

Visualize the loss:

Step4: Prediction

Visualize the prediction:

As we can see, the model performance is not very high due to I only ran 10 epochs for the consideration of computational cost. You can try to tune the model.

Thanks for reading.