Informatica

Neural networks have been successfully applied in fields such as finance (Sezer et al., 2020), macroeconomics (Šestanović and Arnerić, 2020), engineering (Hegde and Rokseth, 2020), weather forecasting (Purwandari et al., 2021), medicine (Han et al., 2024), and many other (Čorić et al., 2023). Their forecasting ability has recently been tested on time series data, which exhibit features that have to be taken into account and addressed appropriately (Kalinić Milićević and Marasović, 2023; Šestanović, 2024). Especially interesting are financial time series data, which are not stationary, they can have seasonal pattern or cyclical behaviour, they are nonlinear, meaning they can exhibit occasional presence of aberrant observations and the possible existence of regimes within which returns display different dynamic behaviour (Franses and van Dijk, 2003).

The dynamic behaviour of cryptocurrencies, as financial time series, displays extreme observations, asymmetries, and several nonlinear characteristics that are difficult to model and forecast (Šestanović, 2024). Additionally, the importance of cryptocurrency forecasting lies in their constantly increasing financial market, characterized by high volatility and extreme price fluctuations.

Bitcoin prices are highly volatile since they are influenced by a vast number of factors including but not limited to the supply of bitcoins, the cost of the mining process, market demand, as well as political and economic data (Cavalli and Amoretti, 2021). Some papers use internal factors for prediction (Polasik et al., 2015; Jang and Lee, 2017; Sovbetov, 2018; Liu and Tsyvinski, 2020; Spilak, 2018; Fahmi et al., 2018; Ji et al., 2019; Chen et al., 2020; Cavalli and Amoretti, 2021), the others use only Open, High, Low, and Close (OHLC) prices (Indera et al., 2017; Fahmi et al., 2018; Uras et al., 2020; Li and Dai, 2020), while Azari (2019) and Abu Bakar and Rosbi (2017) use only past closing prices. Technical indicators are also used as predictors (Indera et al., 2017; Spilak, 2018; Pabuccu et al., 2020; Li and Dai, 2020; Cavalli and Amoretti, 2021). Few papers use macro-finance factors (Polasik et al., 2015; Sovbetov, 2018; Liu and Tsyvinski, 2020; Spilak, 2018; Li and Dai, 2020; Chen et al., 2020) and report the lack of statistical significance if used in parametric models. Contrary, Walther et al. (2019) found that economic activity is the most important exogenous volatility driver, while the results in Jang and Lee (2017) suggest that macro-financial markets can have a small impact on cryptocurrencies. Moreover, Aljinović et al. (2022) report significant dynamic conditional correlations between cryptocurrencies and real estate, S&P500 and gold. However, the majority of papers confirm attractiveness as an important factor that influences cryptocurrency prices (Polasik et al., 2015; Sovbetov, 2018; Li and Dai, 2020; Šestanović, 2021; Cavalli and Amoretti, 2021). This factor is even included among the other indicators in appropriate, comprehensive manner in models for portfolio optimization that include cryptocurrencies (Trimborn et al., 2019; Aljinović et al., 2021). On the other hand, (Kalinić Milićević and Marasović, 2023) report that different models cannot agree upon the importance of Tweets and macro-financial variables in Bitcoin direction forecasting, but show that technical indicators are the most influential, followed by blockchain and market variables. Since different types of attractiveness measures can be found in the literature, the most commonly used measures and their predictive power are compared in this paper, i.e. Google Trends and Tweets.

Previous research has also ambiguous conclusions regarding the appropriate NN model, which calls for further investigation (Cavalli and Amoretti, 2021; Zhang et al., 2021; Li and Dai, 2020; Livieris et al., 2021; Lahmiri and Bekiros, 2019; Ji et al., 2019). Additionally, Lahmiri and Bekiros (2019) revealed that cryptocurrencies exhibit fractal dynamics, long memory and self-similarity. Therefore, an accurate and reliable forecasting model is an essential tool for portfolio managers, which has to be developed and tested.

The feedforward neural networks (FFNNs) are the most popular NN model. Despite their power and proven properties as universal approximators (Hornik et al., 1989), FFNNs have limitations that each input (independent variable) and output (dependent variable) are handled independently, i.e. temporal or space information is not incorporated into the model, which is a significant drawback for time series analysis. Recurrent neural networks (RNNs), however, are adapted to time series data as they incorporate recurrent connections from output or hidden layers and the so-called self-connected neuron, which allows learning the temporal dynamics of time series data (Madaeni et al., 2022). However, their main problem is capturing long-term dependencies as they suffer from vanishing or exploding gradient. RNN with long short-term memory (LSTM) units has a cell state which enables stable gradients, while the presence of filters can control the information flow (Paranhos, 2021). LSTM emerge as a model most commonly used in financial time series prediction (Sezer et al., 2020). However, convolutional neural networks (CNNs) have recently challenged the LSTMs in their predictive power when working with sequences. Namely, CNNs use filters, which help them learn spatial features from raw time series data (Fawaz et al., 2019).

In this paper, the most commonly used NNs for time series prediction, i.e. FFNNs, LSTM and CNNs, which have proven their forecasting abilities on time series data, are used to forecast Bitcoin returns. The proposed models are compared across different periods, including bullish, bearish and stable market periods, using different performance measures such as means squared error (MSE), accuracy and Diebold-Mariano test, through different inputs and, finally, through different NN architectures. That is, neither input nor NN architecture selection is straightforward and should be chosen with caution. Additionally, Uras et al. (2020) confirmed that partitioning datasets into shorter sequences, representing different price “regimes”, enables obtaining precise forecasts. Therefore, Bai-Perron multiple structural break test is used.

To sum up, the main contributions of this paper to the current literature are:

The remainder of the paper is organized as follows. Section 2 provides a literature review of related work, Section 3 describes the proposed methodology, including dataset definition, data preprocessing, a description of neural network architectures, as well as model evaluation criteria. Section 4 presents experimental results with discussion. Finally, conclusions and directions for future research are provided in Section 5.

There are papers that compare FFNNs to other linear and nonlinear models and find NNs to have the highest predictive performances (Greaves and Au, 2015; Pabuccu et al., 2020). Namely, Greaves and Au (2015) compare Support Vector Machine (SVM), Logistic regression (LR), Baseline model and NNs in classification. They obtain the highest classification accuracy of 55.1% with NNs. Pabuccu et al. (2020) aim to forecast Bitcoin prices applying four different machine learning (ML) methods, i.e. SVM, NNs, Naive Bayes (NB), Random Forest (RF) and LR as a benchmark model. They conclude that in a continuous dataset, RF has the best forecasting performance, while NB, the worse. In a discrete dataset, NN has the best forecasting performance, while NB again lags behind other models. Šestanović (2021) confirmed the ability of simple FNNs with lower number of hidden neurons to accurately predict the Bitcoin price direction, compared both to previous research considered and to LR, while Šestanović (2024) predicted Bitcoin price, returns, direction, and volatility. Return and volatility predictions are stable regardless of model or period. Return and direction prediction is best with NNs. ARIMAX and NNARX models predicted prices effectively. All models predict volatility in a similar way. The price prediction was the most accurate, whereas JNNX showed poor performance. However, these papers did not use any sophisticated machine learning models for prediction, which have been proven in the literature to have superior performance.

Since CNN and LSTM methods have proven their forecasting abilities, more and more research has recently been testing their abilities in new circumstances. Several studies confirm that CNN has superior prediction abilities in comparison to LSTM and other NN architectures (Cavalli and Amoretti, 2021; Zhang et al., 2021; Šestanović and Kalinić Milićević, 2023), while some improve the accuracy by combining the CNN and LSTM (Li and Dai, 2020; Livieris et al., 2021). Other research confirms that LSTM exhibits superior predictive abilities when compared to various NN architectures (Lahmiri and Bekiros, 2019; Ji et al., 2019; Spilak, 2018). Contrary, Uras et al. (2020) indicate linear regression models outperform NNs, while in Chen et al. (2020) LR and Discriminant Analysis outperform more complicated machine learning algorithms. Since the literature does not give a unique answer, this calls for further investigation. Table 1 provides a brief overview of the key features of related research and a comparison of related work concerning variables, data, models and key findings.

Cavalli and Amoretti (2021) predict Bitcoin direction with One-Dimensional (1D) CNN and demonstrate, using large datasets collected in a cloud-based system, that the 1D CNN allows for the prediction of the Bitcoin trend with higher accuracy compared to LSTM models.

Among other papers that combine different deep neural network architectures is Livieris et al. (2021) who proposed a multi-input deep learning (MICDL) model based on CNN-LSTM approach for predicting prices of Bitcoin, Ether and XRP. The proposed model is compared to two CNN-LSTM models: model trained with only one cryptocurrency and model trained with all three cryptocurrencies. The utilization of all cryptocurrencies in the training data of the MICDL yielded a forecasting model with the best return and direction predictions.

Motivated by the high correlations among different cryptocurrencies as well as the powerful modelling efficiency exhibited by DL models, Zhang et al. (2021) propose a CNN-based Weighted and Attentive Memory Channels model to predict the daily closing price of cryptocurrencies. The results indicate that the proposed model outperforms the baseline models (ARIMA, RF, XGBoost, MLP, LSTM, CNN, GRU, SVM) in predictive performances. The hyperparameter setting of baseline models is chosen by default. Additionally, this paper does not use any other inputs.

Li and Dai (2020) propose a hybrid NN model based on CNN and LSTM. CNN is used for feature extraction, which become inputs to LSTM for training and prediction of the Bitcoin price. They conclude that CNN-LSTM can effectively improve the accuracy of both value and direction prediction compared with simple NNs.

Uras et al. (2020) forecast daily closing prices of Bitcoin, Litecoin and Ether, using Simple and Multiple Linear Regression model (RM), as well as FFNN and LSTM models. The best results were found with RM and LSTM models. However, the linear RMs outperform NNs.

Chen et al. (2020) predict Bitcoin price at daily and high-frequency intervals. LR and Discriminant Analysis (DA) achieve an accuracy of 66%, outperforming more complex machine learning (ML) models. ML models include RF, XGBoost, Quadratic DA, SVM and LSTM.

Lahmiri and Bekiros (2019) implement LSTM and generalized regression NN (GRNN) to forecast the prices of Bitcoin, Bitcoin Cash and XRP. The predictability of LSTM is significantly higher compared to the GRNN. LSTM are proved highly efficient in forecasting cryptocurrency prices.

Ji et al. (2019) compare deep NN (DNN), LSTM, CNN, deep residual network, and their combinations for Bitcoin price prediction, as well as SVM, GRU and linear/logistic RMs which performed worse or equally to SVM. They conclude that LSTM slightly outperforms the other models. Moreover, DNN performed the best in the classification problem.

Spilak (2018) uses FFNN, RNN and LSTM in classification tasks to predict price directions of 8 major cryptocurrencies with a rolling window RM. The study reveals that LSTM has the highest accuracy for direction prediction of the most important cryptocurrencies, FFNN has the best generalization power for three cryptocurrencies, while RNN shows poor prediction performances, seemingly failing to extract the necessary information.

Jaquart et al. (2022) train models to predict binary relative daily market movements of the 100 largest cryptocurrencies. They use only daily closing prices and market capitalization data. GRU and LSTM models perform the best, as portfolios based on these models’ predictions yield the highest performances.

Šestanović and Kalinić Milićević (2023) estimated FFNN, CNN and LSTM models in the downturn period for Bitcoin return prediction by applying the multi-criteria decision-making approach for model selection. They concluded that optimal model is CNN.

In line with previous research findings that the majority of optimal models are constructed using CNN and LSTM, this article presents a comprehensive approach to forecasting Bitcoin returns using FFNN, CNN and LSTM. The employed NN architectures are thoroughly explained and compared across various architectures and periods in order to reach a conclusion regarding the important NN configurations. Namely, this paper emerges as a stress test for NN architectures, since it tests the abilities of more sophisticated NN architectures in different sub-periods, which include bullish, bearish and stable market conditions, obtained in a logical way, i.e. using Bai-Perron structural break test. Previous researches do not include downturn periods in their analysis, which are usually more difficult to predict. The results are compared through different performance measures and tested using Diebold-Mariano test. Additionally, although some previous researches use sentiment analysis variable, they do not compare their performances. Finally, although other papers sometimes use even more inputs for prediction, they narrow them down to the use of only technical indicators or internal factors, while in this paper the representatives of important factors are used in a comprehensive manner.

The models with the best forecasting performances on both observed performance measures are those in Subset 3, which is characterized by an increase in price but lower volatility. This confirms the findings of previous research, in which models tested during periods of stable Bitcoin price growth performed well. Subset 4 was characterized by a significant slump in Bitcoin prices due to the uncertainty of the Covid-19 crisis, followed by a more volatile price increase. Thus, on average, all NNs predicted the movement properly. However, predicting direction was more challenging for more complex NNs. Namely, FFNN outperformed other NNs in direction forecasting. This result comes as a surprise since it contradicts previous research findings that CNN outperforms other NN architectures (Cavalli and Amoretti, 2021; Zhang et al., 2021; Šestanović and Kalinić Milićević, 2023). In Subsets 2 and 5, which cover the significant downturn period in Bitcoin prices, including extremely volatile periods, CNN managed to capture price fluctuations, while FFNN dominated on average in predicting Bitcoin returns. CNNs have the advantages inherent in their architecture, incorporating FFNN model after the convolutional layer. This enables a lower number of hidden neurons and higher predictive performances, while avoiding the overfitting problem in the process. CNNs are robust and need less training time compared to RNNs or FFNNs, and can reduce the complexity of the model (Madaeni et al., 2022). Based on accuracy, the model with LSTM architecture is superior to all others, confirming the findings in Ji et al. (2019) and Spilak (2018). CNN outperforms other models in downturn period, while FFNN outperforms the other models in bullish market conditions. This means that simpler NNs can be used for predictions in bullish market, while more complex NNs should be used for predictions in bearish market. The results of MSE produce diametrically opposed conclusions. Namely, MSE and accuracy do not always agree when it comes to identifying the optimal prediction model. Since MSE is composed of both bias and variance, if one estimator has lower MSE than another, it is not known whether this is due to lower bias or lower variance (i.e. higher precision). Therefore, further investigation is needed to determine whether it is more appropriate to observe regression models through these two perspectives or to simply use classification models to find the model which has better prediction of price direction.

In this paper, the predictive performances of three commonly used NN models were compared using different performance measures on different subsets of datasets, differentiated by two attractiveness measures. These subsets entail different market conditions, i.e. bearish or bullish periods. Thus, this paper examined the ability of these machine learning models in all types of environments, including bullish, bearish, and stable periods, as well as periods characterized by high volatility. All NNs performed best in an environment of bullish market, where CNN stood out as the optimal NN model using MSE, while FFNN and LSTM emerged as optimal models in direction forecasting. However, based on accuracy, CNN outperformed other models in downturn periods, while FFNN outperformed other models in bullish market conditions. The results of MSE produced diametrically opposed conclusions. Moreover, based on accuracy, the dataset with Tweets as the attractiveness measure outperformed Google Trends, whereas based on MSE the results did not differ significantly. Finally, using a lower number of hidden neurons, as well as lower learning rate values and lower batch size values yielded optimal results. Future research direction includes forecasting the cryptocurrency returns using the sentiment-enriched data. Additionally, since cryptocurrency prices are increasingly reactive to macroeconomic shocks, it is proposed to examine which macroeconomic variables have the greatest influence on their movement. Finally, sophisticated neural network models can be used for the prediction of cryptocurrency prices, returns, direction and volatility in a comprehensive manner.