Machine Learning for Stock Price Forecasting: LSTM vs Transformer Approaches

Abstract

Stock price prediction, rightly considered one of the most complex and risk-sensitive undertakings in therealm of finance, was never quite so much in fashion. Statistical models given to financial marketprediction have never enjoyed universal acceptance because of the nonlinearities in the data-generationprocess and temporal dependence characteristic in financial time series data. This paper explores twofrontline deep learning-based approaches to stock price prediction: LSTM and Transformer. LSTM RNNsare well known for learning sequential dependencies, whereas Transformers triggered a revolution in NLPby addressing the long-term dependency problem through attention without using recurrence. This paper, taking real financial datasets from public stock exchanges, implements both LSTM andTransformer architectures and tests factors such as MSE, MAE, and R^2. Our comprehensive performance analysis is done over many stocks to assess the robustness, generalizability, andinterpretability of the tested models. The study's results reveal that while LSTM fares better with smalldatasets displaying trends that are less volatile, Transformer models outshine in volatile and highfrequencyenvironments in terms of accuracy and ability. This infers that the Transformer, at the expenseof computational effort, could be the next paradigm in time series forecasting for finance. Going further, this manuscript presents visualizations comparing predicted and actual stock prices, whilealso including a feature analysis for determining which factors contributed most to the prediction. Theconclusion provides potential trajectories for hybrid models that combine the strengths of eacharchitecture.