Stock Price Predictions with LSTM Neural Networks and Twitter Sentiment

Marah-Lisanne Thormann; Jan Farchmin; Christoph Weisser; Rene-Marcel Kruse; Benjamin Säfken; Alexander Silbersdorff

doi:10.19139/soic-2310-5070-1202

Marah-Lisanne Thormann Faculty of Economics, University of Göttingen， Germany
Jan Farchmin Faculty of Economics, University of Göttingen， Germany
Christoph Weisser Faculty of Economics, University of Göttingen， Germany
Rene-Marcel Kruse Centre for Statistics，University of Göttingen， Germany
Benjamin Säfken Centre for Statistics，University of Göttingen， Germany
Alexander Silbersdorff Faculty of Economics，University of Göttingen， Germany

DOI: https://doi.org/10.19139/soic-2310-5070-1202

Keywords: Twitter, TextBlob, Stock Price, Prediction, Neural Networks, LSTM, RNN

Abstract

Predicting the trend of stock prices is a central topic in financial engineering. Given the complexity and nonlinearity of the underlying processes we consider the use of neural networks in general and sentiment analysis in particular for the analysis of financial time series. As one of the biggest social media platforms with a user base across the world, Twitter offers a huge potential for such sentiment analysis. In fact, stocks themselves are a popular topic in Twitter discussions. Due to the real-time nature of the collective information quasi contemporaneous information can be harvested for the prediction of financial trends. In this study, we give an introduction in financial feature engineering as well as in the architecture of a Long Short-Term Memory (LSTM) to tackle the highly nonlinear problem of forecasting stock prices. This paper presents a guide for collecting past tweets, processing for sentiment analysis and combining them with technical financial indicatorsto forecast the stock prices of Apple 30m and 60m ahead. A LSTM with lagged close price is used as a baseline model. We are able to show that a combination of financial and Twitter features can outperform the baseline in all settings. The code to fully replicate our forecasting approach is available in the Appendix.

References

A. A. Adebiyi, and C. K. Ayo, and M. O. Adebiyi, and S. O. Otokiti, Stock price prediction using neural network with hybridized market indicators, Journal of Emerging Trends in Computing and Information Sciences, vol. 3, no. 1, pp. 1–9, 2012.

A. A. Adebiyi, and A. O. Adewumi, and C. K. Ayo, Comparison of ARIMA and artificial neural networks models for stock price prediction, Journal of Applied Mathematics, vol. 2014, pp. 1–7, 2014.

G. V. Attigeri, and M. P. MM, and M. Radhika, and A. Nayak, Stock market prediction: A big data approach, TENCON 2015-2015 IEEE Region 10 Conference, pp. 1–5, 2015.

M. Baughman, and C. Haas, and R. Wolski, and I. Foster, and K. Chard, Predicting Amazon Spot Prices with LSTM Networks, in Proceedings of the 9th Workshop on Scientific Cloud Computing, Tempe, AZ, USA, pp. 1–7, 2018.

Y. Bengio, Practical Recommendations for Gradient-Based Training of Deep Architectures, in Neural Networks: Tricks of the Trade: Second Edition, edited by G. Montavon, and G. Orr, and K.-R. M¨uller, Springer-Verlag, Berlin, Heidelberg, pp. 437–478, 2012.

J. Bergstra, and Y. Bengio, Random search for hyper-parameter optimization, The Journal of Machine Learning Research, vol. 13, no. 1, pp. 281–305, 2012.

C. M. Bishop, Pattern recognition and machine learning, Springer-Verlag, New York, 2016.

J. Bollen, and H. Mao, and X.-J. Zeng, Twitter mood predicts the stock market, Journal of Computational Science, vol. 2, no. 1, pp. 1–8, 2011.

K. Chen, and Y. Zhou, and F. Dai, A LSTM-based method for stock returns prediction: A case study of China stock market, 2015 IEEE international conference on big data (big data), pp. 2823–2824, 2015.

R. Chen, and M. Lazer, Sentiment analysis of twitter feeds for the prediction of stock market movement, stanford edu Retrieved January, vol. 25, pp. 1–5, 2013.

S. Das, and R. K. Behera, and S. K. Rath,and others, Real-time sentiment analysis of twitter streaming data for stock prediction, Procedia computer science, vol. 132, pp. 956–964, 2018.

M. De Choudhury, and H. Sundaram,and A. John, and D. D. Seligmann, Can Blog Communication Dynamics Be Correlated with Stock Market Activity?, in Proceedings of the Nineteenth ACM Conference on Hypertext and Hypermedia, pp. 55—60, 2008.

L. Di Persio, and O. Honchar, Artificial neural networks architectures for stock price prediction: Comparisons and applications, International journal of circuits, systems and signal processing, vol. 10, no. 2016, pp. 403–413, 2016.

T. Fischer,and C. Krauss, Deep learning with long short-term memory networks for financial market predictions, European Journal of Operational Research, vol. 270, no. 2, pp. 654–669, 2018.

A.-C. Florea, and R.Andonie, Weighted Random Search for Hyperparameter Optimization, International Journal of Computers Communications & Control, vol. 14, no. 2, pp. 154—169, 2019.

Y. Gal, and Z. Ghahramani, A Theoretically Grounded Application of Dropout in Recurrent Neural Networks, arXiv preprint arXiv:1512.05287, 2016.

A. G´eron, Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems, O’Reilly Media, 2019.

I. Goodfellow, and Y. Bengio, and A. Courville, Deep Learning, MIT Press, 2016.

A. Graves, and J. Schmidhuber, Framewise phoneme classification with bidirectional LSTM and other neural network architectures, Neural Networks, vol. 18, no. 5, pp. 602–610, 2005.

K. Greff, and R. K. Srivastava, and J. Koutn´ık, and B. R. Steunebrink,and J. Schmidhuber, LSTM: A search space odyssey, IEEE transactions on neural networks and learning systems, vol. 28, no. 10, pp. 2222–2232, 2016.

A. Hasan, and S. Moin,and A. Karim, and S. Shamshirband, Machine learning-based sentiment analysis for twitter accounts, Mathematical and Computational Applications, vol. 23, no. 1, pp. 1–11, 2018.

M. Hiransha, and E. A. Gopalakrishnan, and V. K. Menon, and K. P. Soman, NSE stock market prediction using deep-learning models, Procedia computer science, vol. 132, pp. 1351–1362, 2018.

S. Hochreiter, and J. Schmidhuber, LSTM can solve hard long time lag problems, in Advances in neural information processing systems, pp. 473–479, 1997.

S. Hochreiter, and J. Schmidhuber, A new model for stock price movements prediction using deep neural network, in SoICT ’17: Eighth International Symposium on Information and Communication Technology, pp. 57–62, 2017.

S. Jain, and R. Gupta,and A. A. Moghe, Stock price prediction on daily stock data using deep neural networks, in 2018 International Conference on Advanced Computation and Telecommunication (ICACAT), pp. 1–13, 2018.

G. Kant, and C. Weisser, and B. S¨afken, TTLocVis: A Twitter Topic Location Visualization Package, Journal of Open Source Software, 5(54), 2507.

M. M. Khani, and S. Vahidnia, and A. Abbasi, A Deep Learning Based Methods for Forecasting Gold Price with Respect to Pandemics, 2020.

T. Kim, and H. Y. Kim, Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data, PloS one, vol. 14, no. 2, pp. 1–23, 2019.

D. P. Kingma, and J. Ba, Adam: A Method for Stochastic Optimization, arXiv preprint arXiv:1412.6980, 2017.

S. Loria, and P. Keen, and M. Honnibal, and R. Yankovsky, and D. Karesh, and E. Dempsey, and others, Textblob: simplified text processing, Secondary TextBlob: simplified text processing, vol. 3, 2014.

W. Ma, and Y. Wang, and N. Dong, Study on stock price prediction based on BP neural network, in 2010 IEEE International Conference on Emergency Management and Management Sciences, pp. 57–60, 2010.

P. Malhotra, and L. Vig, and G. Shroff, and P. Agarwal, Long short term memory networks for anomaly detection in time series, Proceedings, vol. 89, Presses universitaires de Louvain, pp. 89–94, 2015.

Y. Mao, and W. Wei,and B. Wang, and B. Liu, Correlating S&P 500 stocks with Twitter data, in Proceedings of the first ACM international workshop on hot topics on interdisciplinary social networks research, pp. 69–72, 2012.

D. Masters, and C. Luschi, Revisiting Small Batch Training for Deep Neural Networks, arXiv preprint arXiv:1804.07612, 2018.

S. Mehtab and J. Sen, A robust predictive model for stock price prediction using deep learning and natural language processing, Available at SSRN 3502624, 2019.

A. Mittal, and A. Goel, Stock prediction using twitter sentiment analysis, Standford University, CS229 (2011 http://cs229. stanford. edu/proj2011/Goel Mittal-Stock Market Prediction Using Twitter Sentiment Analysis.pdf), vol. 15, pp. 1–5, 2012.

M. Nikou, and G. Mansourfar, and J. Bagherzadeh, Stock price prediction using DEEP learning algorithm and its comparison with machine learning algorithms, Intelligent Systems in Accounting, Finance and Management, vol. 26, no. 6, pp. 164–174, 2019.

V. S. Pagolu, and K. N. Reddy and G. Panda; and B. Majhi, Sentiment analysis of Twitter data for predicting stock market movements, in 2016 international conference on signal processing, communication, power and embedded system (SCOPES), pp. 1345–1350, 2016.

P.-F. Pai, and C.-S. Lin, A hybrid ARIMA and support vector machines model in stock price forecasting, Omega, vol. 33, no. 6, pp. 497–505, 2005.

K. Pawar, and R. S. Jalem, and V. Tiwari, Stock market price prediction using LSTM RNN, in Emerging Trends in Expert Applications and Security, Springer, Singapore, pp. 493–503, 2019.

V. Pham, and T. Bluche, and C. Kermorvant, and J. Louradour, Dropout improves Recurrent Neural Networks for Handwriting Recognition, arXiv preprint arXiv:1312.4569, 2014.

R. Pimprikar, and S. Ramachandran, and K. Senthilkumar, Use of machine learning algorithms and twitter sentiment analysis for stock market prediction, International Journal of Pure and Applied Mathematics, vol. 115, no. 6, pp. 521-526, 2017.

A. Porshnev, and I. Redkin, and A. Shevchenko, Machine learning in prediction of stock market indicators based on historical data and data from twitter sentiment analysis, in 2013 IEEE 13th International Conference on Data Mining Workshops, pp. 440–444, 2013.

M. Qasem, and R. Thulasiram, and P. Thulasiram, Twitter sentiment classification using machine learning techniques for stock markets, in 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 834–840, 2015.

S. Raschka, and V. Mirjalili, Python Machine Learning, 3rd Ed., Packt Publishing, Birmingham, UK, 2019.

J. Roesslein, tweepy Documentation, Online] http://tweepy. readthedocs. io/en/v3, vol. 5, 2009.

M. Roondiwala, and H. Patel, and S. Varma, Predicting stock prices using LSTM, International Journal of Science and Research (IJSR), vol. 6, no. 4, pp. 1754–1756, 2017.

S. Ruder, An overview of gradient descent optimization algorithms, arXiv preprint arXiv:1609.04747, 2017.

B. S¨afken, and A. Silbersdorff, and C.Weisser, Learning deep: Perspectives on Deep Learning Algorithms and Artificial Intelligence, Universit¨atsverlag G¨ottingen, G¨ottingen, 2020.

H. Sak, and A. Senior, and F. Beaufays, Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition, arXiv preprint arXiv:1402.1128, 2014.

L. Sayavong, and Z. Wu, and S. Chalita, Research on Stock Price Prediction Method Based on Convolutional Neural Network, in 2019 International Conference on Virtual Reality and Intelligent Systems (ICVRIS), pp. 173–176, 2019.

S. Selvin, and R. Vinayakumar, and E. A. Gopalakrishnan, and V. K. Menon, and K. P. Soman, Stock price prediction using LSTM, RNN and CNN-sliding window model, in 2017 international conference on advances in computing, communications and informatics (icacci), pp. 1643–1647, 2017.

J. Sen, and T. Datta Chaudhuri, Stock price prediction using machine learning and deep learning frameworks, in Proceedings of the 6th International Conference on Business Analytics and Intelligence, Bangalore, India, 2018.

D. Shah, and W. Campbell, and F. H. Zulkernine, A comparative study of LSTM and DNN for stock market forecasting, in 2018 IEEE International Conference on Big Data (Big Data), pp. 4148–4155, 2018.

M. Skuza, and A. Romanowski, Sentiment analysis of Twitter data within big data distributed environment for stock prediction, in 2015 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 1349–1354, 2015.

Y.-G. Song, and Y.-L. Zhou, and R.-J. Han, Neural networks for stock price prediction, arXiv preprint arXiv:1805.11317, 2018.

K. S. Vaisla,and A. K. Bhatt, An analysis of the performance of artificial neural network technique for stock market forecasting, International Journal on Computer Science and Engineering, vol. 2, no. 6, 2104–2109, 2010.

I. Vasilev, and D. Slater, and G. Spacagna, and P. Roelants,and V. Zocca, Python Deep Learning: Exploring deep learning techniques and neural network architectures with Pytorch, Keras, and TensorFlow, Packt Publishing Ltd, 2019.

T. T. Vu, and S. Chang, and Q. T. Ha, and N. Collier An experiment in integrating sentiment features for tech stock prediction in twitter, in Proceedings of the workshop on information extraction and entity analytics on social media data, pp. 23–38, 2012.

P. D. Yoo, and M. H. Kim, and T. Jan, Machine learning techniques and use of event information for stock market prediction: A survey and evaluation, in International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06), pp. 835–841, 2005.

W. Zaremba, and I. Sutskever, and O. Vinyals, Recurrent neural network regularization, arXiv preprint arXiv:1409.2329, 2014.

T. Zhang, and S. Song, and S. Li, and L. Ma, and S. Pan, and L. Han, Research on Gas Concentration Prediction Models Based on LSTM Multidimensional Time Series, Energies, vol. 12, no. 1, pp. 1996-1073, 2019.

X. Zhang, and H. Fuehres, and P. A. Gloor, Predicting stock market indicators through twitter “I hope it is not as bad as I fear”, Procedia-Social and Behavioral Sciences, vol. 26, pp. 55–62, 2011.