This content originally appeared on Level Up Coding - Medium and was authored by Tatev Karen
Who hasn’t heard about the most famous digital money in the world, the cryptocurrency of decade? That’s right, I am talking about Bitcoin. Everyone is talking about Bitcoin and what will happen to its record high price, including famous financial brands such as JPMorgan and Morgan Stanley. This year, Bitcoin reported its all-time high price of $61,556.59 in March of 2021, according to the CoinDesk 20.
The biggest question is, of course, what will happen to the price of this magic money. This is a question that no one has a certain answer to because predicting the exact price of Bitcoin, or in fact, any other financial instrument, is nearly impossible. However, what we can do is to use Data Science techniques to predict the Bitcoin price with high precision and that’s exactly what we are going to do in this article. We will built a Deep Learning model that will use Bitcoin’s historical data to predict its price.
By the end of this article you will learn about:
- Deep Learning (RNNs, LSTMs)
- Data Preprocessing
- Evaluation of the Bitcoin Price Prediction Model
About Bitcoin
Bitcoin offers an efficient money tranferes over the internet and is controlled by a decentralized network with a transparent set of rules and works as an alternative to central bank-controlled fiat money. The key technology behind this cryptocurrency is the phenomenon of blockchain which is a growing list of records (blocks) linked using cryptography where each block contains a cryptographic hash of the previous block, a timestamp, and transaction data.
By design, a blockchain cannot be modified because once it is recorded, the data in any given block cannot be altered afterwards without alteration of all subsequent blocks. Hence, any changes in a single block will make all following blocks in a blockchain to be invalid making it obvious that the original data has been manipulated.
Bitcoin Historical Price Data
To train our pricing model, we use Bitcoin’s public price data reported in the period of 2016–2020, 5 years of financial data downloaded from Yahoo Finance. To evaluate our model we use Bitcoin price data in the period of January 2021-March 2021. We use the model to predict the Bitcoin prices for this period and compare it to the real prices in order to understand how good the model was able to predict the prices.
Data Preprocessing
Step 1: Normalization Our model is based on RNN with LSTM layers which uses Sigmoid Activation Function to transform input vectors to vectors with entries that have value in a range of [0,1]. Therefore, we use normalization to scale model features given that the denominator will always be larger than the nominator, the output value will always be a number between 0 and 1.
from sklearn.preprocessing import MinMaxScaler
sc = MinMaxScaler(feature_range=(0,1))
training_set_scaled = sc.fit_transform(training_set)
Step 2: Data Transformation For the RNN we make an assumption regarding the number of time periods the model needs to go back in time to learn about the current time period. Given that there are in general 30 days in a month, 3 month, that is 90 days, time period seems like a reasonable assumption to make for our RNN model to learn from the Bitcoin’s prices in the past. Hence, per price, the X_train will contain the Bitcoin prices of the last 60 days and the Y_train will be the Bitcoin price of that day and this holds for all time periods.
import numpy as np
X_train = []
y_train = []
X_test = []
for i in range(90,training_set_scaled.size):
X_train.append(training_set_scaled[i-90:i, 0])
y_train.append(training_set_scaled[i, 0])
X_train, y_train = np.array(X_train), np.array(y_train)
for i in range(90,inputs.size):
X_test.append(inputs[i-90:i, 0])
X_test = np.array(X_test)
Step 3: Shaping Data Adding extra dimension to the model that will allow using extra indicators helping to predict the price (e.g. external events causing public anxiety). In this way we transform the data from 2D to 3D.
X_train = np.reshape(X_train, [X_train.shape[0], X_train.shape[1], 1])
X_test = np.reshape(X_test, [X_test.shape[0], X_test.shape[1], 1])
Download Bitcoin Training and Test Data here: https://github.com/TatevKaren/recurrent-neural-network-stock-price-predicition/tree/main/bitcoin_data
Deep Learining
The entire idea behind Deep Learning is to mimic the functional capabilities of human brain. If Artificial Neural Networks (ANNs) are responsible for the long term memory, like the Temporal lobe of our brain, and if Convolutional Neural Networks (CNNs) are responsible for the image recognition and visual interpretation, like the Occipital lobe of our brain, then the Recurrent Neural Networks (RNNs) are responsible for the short term memory, like the Frontal Lobe of our brain.
Recurrent Neural Networks (RNNs)
On one hand the information flows from the input layers to the output layer and on the other hand, the calculated eerror is back propagated through the network to update the weights. Unlike ANN and CNN, in case of RNN, hidden layers not only give an output but also feed themselves.
What RNN does is that it translates the provided input features to a machine readable vectors. Then the system processes each of this sequence of vectors one by one, moving from very first vector to the next one in a sequential order. While processing, the system passes the information through the hidden state (memory state) to the next step of this sequence.
Once the hidden state has collected all the existing information in the system from the previous steps, it is ready to move towards the next step and combine this information with this current step’s input (Xt) to form a new information vector.
Long Short-Term Memories (LSTMs)
When training a RNN model, during the optimization process, in the very initial epoch, the weights are randomly chosen values and in case these chosen values are very small, multiplying them with the same recurrent weight for many times, the gradient becomes less and less and at some point the gradient vanishes. Then, the lower the gradient is, the harder it is to update weight which means the slower the optimization process will be.
Additionally, there is a domino effect and one improperly updated weight effects the calculation of the remaining weights and makes them inaccurate as well, given that all weights are related. This issue in RNNs is referred as Vanishing Gradient Problem and Long Short-Term Memories (LSTMs) are solving this problem of RNNs. The difference between the usual RNN and LSTM is the set of operations that are performed on the passedinformation and the input value in that specific step. The information in LSTMs flows through its gates:
- forget gate: a gate that decides what information should be thrown away or kept
- input gate: a gate to update the cell state
- cell state: a gate to update the cell state to new values that the network finds relevant
- output gate: a gate to decide what the next hidden state should be
Building Bitcoin Pricing Model
The goal is to predict the stock prices which is a continuous output value, hence we have a regression rather than classification problem. We initialize this regressor as an object with sequential layers for which we use Sequential module from Keras that allows to create a Neural Network object with sequential layers. Then, we add LSTM layers and given that predicting a price of a financial product is a quite complex task, we like to have a model with high dimensionality that can capture upward and downward trends in the stock price, hence we use large number of LSTM units per LSTM layer and we use multiple LSTM layers. Moreover, we add a Dropout layer, for regularization to ignore part of the neurons in the LSTM layers. Finally, we use the Dense module to add an output layer
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Dropout
regressor = Sequential()
# LSTM layer 1
regressor.add(LSTM(units = 50, return_sequences=True, input_shape = (X_train.shape[1], 1)))
regressor.add(Dropout(0.2))
# LSTM layer 2,3,4
regressor.add(LSTM(units = 50, return_sequences=True))
regressor.add(Dropout(0.2))
regressor.add(LSTM(units = 50, return_sequences=True))
regressor.add(Dropout(0.2))
regressor.add(LSTM(units = 50, return_sequences=True))
regressor.add(Dropout(0.2))
regressor.add(LSTM(units = 50, return_sequences=True))
regressor.add(Dropout(0.2))
# LSTM layer 5
regressor.add(LSTM(units = 50))
regressor.add(Dropout(0.2))
# Fully connected layer
regressor.add(Dense(units = 1))
# Compiling the RNN
regressor.compile(optimizer = 'adam', loss = 'mean_squared_error')
#Fitting the RNN model
regressor.fit(X_train, y_train, epochs = 120, batch_size = 32)
Bitcoin Pricing Model Predictions
Once the pricing model is trained, we use it in combination with test data to generate the Bitcoin price predictions for the testing period, January 2021-March 2021 which we then compare with the real Bitcoin prices reported in the same time period.
As we can see our RNN model based on multiple LSTM layers, was able to properly predict upward and downward trends because we see that the blue line corresponding to the predicted Bitcoin prices follows the same pattern as the yellow line which corresponds the real Bitcoin prices.
import matplotlib.pyplot as plt
predicted_stock_price= regressor.predict(X_test)
predicted_stock_price= sc.inverse_transform(predicted_stock_price)
plt.plot(real_stock_price, color = '#ffd700', label = "Real Price January - March 2021")
plt.plot(predicted_stock_price, color = '#4782B4', label = "Predicted Price January - March 2021")
plt.title("Bitcoin Price Prediction")
plt.xlabel("Time")
plt.ylabel("Bitcoin Price")
plt.legend()
plt.show()
Resources
More
Google Stock Pricing Model with RNN: https://github.com/TatevKaren/recurrent-neural-network-price-predicition
Customer Churn Prediction Model with ANN: https://github.com/TatevKaren/artificial-neural-network-business_case_study
Image Recognition Model with CNN: https://github.com/TatevKaren/convolutional-neural-network-image_recognition_case_study
Predicting Bitcoin’s Price With Recurrent Neural Networks was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Tatev Karen
Tatev Karen | Sciencx (2021-03-30T00:34:36+00:00) Predicting Bitcoin’s Price With Recurrent Neural Networks. Retrieved from https://www.scien.cx/2021/03/30/predicting-bitcoins-price-with-recurrent-neural-networks/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.