Abstract

Predictive maintenance PdM has helped, in recent decades, manufacturing and industry to save costs and keep their operations safe. This study outlines how advanced machine learning systems, including LSTMs and Transformers, could enhance data-driven maintenance planning. Mainly using C-MAPSS datasets to test Deep Learning (DL) methods and estimate Remaining Useful Life (RUL) values, this research aims to compare LSTM networks and Transformer performance in prognostic via different evaluation criteria. The analysis manifests the peculiarities of both the proposed learning approaches with a marked difference in the performances in favor of the recurrent architecture (e.g., 40% in term R² and 43% in terms of MSE), thus not generally suggesting the usage of transformer-based architecture, especially in a data-scarcity situation common condition when working of critical and costly units, but opening to a new perspective in otherwise operational conditions where data is prosperous.

References

1. Pop GI, Titu AM and Pop AB. “Enhancing Aerospace Industry Efficiency and Sustainability: Process Integration and Quality Management in the Context of Industry 4.0”. Sustainability 15.23 (2023): 16206.
2. Ucar A, Karakose M and Kirimca N. “Artificial intelligence for predictive maintenance applications: key components, trustworthiness, and future trends”. Applied Sciences 14.2 (2024): 898.
3. Meddaoui A, Hain M and Hachmoud A. “The benefits of predictive maintenance in manufacturing excellence: a case study to establish reliable methods for predicting failures”. The International Journal of Advanced Manufacturing Technology 128.7-8 (2023): 3685-3690.
4. Lee WJ., et al. “Predictive maintenance of machine tool systems using artificial intelligence techniques applied to machine condition data”. Procedia Cirp 80 (2019): 506-511.
5. Zhang J., et al. “Long short-term memory for machine remaining life prediction”. Journal of manufacturing systems 48 (2018): 78-86.
6. ElDali M and Kumar KD. “Fault diagnosis and prognosis of aerospace systems using growing recurrent neural networks and LSTM”. In 2021 IEEE aerospace conference. IEEE 50100 (2021): 1-20.
7. Wu Y., et al. “Remaining useful life estimation of engineered systems using vanilla LSTM neural networks”. Neurocomputing 275 (2018): 167-179.
8. Dong D, Li XY and Sun FQ. “Life prediction of jet engines based on LSTM-recurrent neural networks”. In 2017 prognostics and system health management conference (PHM-Harbin). IEEE (2017): 1-6.
9. Vaswani A., et al. “Attention is all you need. Advances in neural information processing systems”. Advances in neural information processing systems 30 (2017).
10. Zeng A., et al. “Are transformers effective for time series forecasting?”. In Proceedings of the AAAI conference on artificial intelligence 37.9 (2023): 11121-11128.
11. Ramasso E and Saxena A. “Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets”. International Journal of Prognostics and Health Management 5.2 (2014): 1-15.
12. Ferreira C and Goncalves G. “Remaining Useful Life prediction and challenges: A literature review on the use of Machine Learning Methods”. Journal of Manufacturing Systems 63 (2022): 550-562.
13. James G., et al. An introduction to statistical learning 112 (2013): 18.
14. Hastie T., et al. “The elements of statistical learning: data mining, inference, and prediction”. New York: Springer 2 (2009): 1-758.
15. Draper NR. “Applied regression analysis”. John Wiley & Sons (1998).
16. F Moscato., et al. “Solution Workflows for Model-Based Analysis of Complex Systems”. IEEE Transactions on Automation Science and Engineering 9.1 (2012): 83-95.
17. F Amato and F Moscato. “Model transformations of MapReduce Design Patterns for automatic development and verification”. Journal of Parallel and Distributed Computing 110 (2017): 52-59.
18. F Amato, F Moscato and F Xhafa. “Generation of game contents by social media analysis and MAS planning”. Computers in Human Behavior 100 (2019): 286-294.
19. Bai S, Kolter JZ and Koltun V. “An empirical evaluation of generic convolutional and recurrent networks for sequence modelling”. arXiv preprint arXiv:1803.01271 (2018).
20. Narwariya J., et al. “Graph neural networks for leveraging industrial equipment structure: An application to remaining useful life estimation”. arXiv preprint arXiv:2006.16556 (2020).
21. Zhou H., et al. “Informer: Beyond efficient transformer for long sequence time-series forecasting”. In Proceedings of the AAAI conference on artificial intelligence 35.12 (2021): 11106-11115.
22. Wu H., et al. “Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting”. Advances in neural information processing systems 34 (2021): 22419-22430.
23. Oreshkin BN., et al. “N-BEATS: Neural basis expansion analysis for interpretable time series forecasting”. arXiv preprint arXiv:1905.10437 (2019).
24. Challu C., et al. “Nhits: Neural hierarchical interpolation for time series forecasting”. In Proceedings of the AAAI conference on artificial intelligence 37.6 (2023): 6989-6997.
25. Salinas D., et al. “DeepAR: Probabilistic forecasting with autoregressive recurrent networks”. International journal of forecasting 36.3 (2020): 1181-1191.
26. Lim B., et al. “Temporal fusion transformers for interpretable multi-horizon time series forecasting”. International Journal of Forecasting 37.4 (2021): 1748-1764.
27. Arena F., et al. “Predictive maintenance in the automotive sector: A literature review”. Mathematical and Computational Applications 27.1 (2021): 2.
28. Gasthaus J., et al. “Probabilistic forecasting with spline quantile function RNNs. In The 22nd international conference on artificial intelligence and statistics”. PMLR (2019): 1901-1910.
29. Sasinthiran A, Gnanasekaran S and Ragala R. “A review of artificial intelligence applications in wind turbine health monitoring”. International Journal of Sustainable Energy 43.1 (2024): 2326296.
30. Steckel J., et al. “Tool Wear Prediction in CNC Turning Operations using Ultrasonic Microphone Arrays and CNNs”. arXiv preprint arXiv:2406.08957 (2024).
31. Tang Y, Zhang Y and Li J. “A time series driven model for early sepsis prediction based on transformer module”. BMC Medical Research Methodology 24.1 (2024): 23.
32. Tian Y., et al. “Federated vehicular transformers and their federations: Privacy-preserving computing and cooperation for autonomous driving”. IEEE Transactions on Intelligent Vehicles 7.3 (2022): 456-465.
33. Gupta S and Torra V. “Differentially Private Traffic Flow Prediction Using Transformers: A Federated Approach. In European Symposium on Research in Computer Security”. Cham: Springer Nature Switzerland (2023): 260-271.
34. Prechelt L. “Early stopping-but when?”. In Neural Networks: Tricks of the trade. Berlin, Heidelberg: Springer Berlin Heidelberg (2002): 55-69.
35. Arlot S and Celisse A. A survey of cross-validation procedures for model selection (2010).
36. Al-Kababji A, Bensaali F and Dakua SP. “Scheduling techniques for liver segmentation: Reducelronplateau vs onecyclelr”. In International conference on intelligent systems and pattern recognition. Cham: Springer International Publishing (2022): 204-212.
37. Li L., et al. “Hyperband: A novel bandit-based approach to hyperparameter optimization”. Journal of Machine Learning Research 18.185 (2018): 1-52.
38. Snoek J, Larochelle H and Adams RP. “Practical bayesian optimization of machine learning algorithms”. Advances in neural information processing systems 25 (2012).
39. Jaderberg M., et al. “Population based training of neural networks”. arXiv preprint arXiv:1711.09846 (2017).
40. Izmailov P., et al. “Averaging weights leads to wider optima and better generalization”. arXiv preprint arXiv:1803.05407 (2018).
41. Smith LN. “Cyclical learning rates for training neural networks”. In 2017 IEEE winter conference on applications of computer vision (WACV). IEEE (2017): 464-472.
42. Mukherjee K, Khare A and Verma A. “A simple dynamic learning rate tuning algorithm for automated training of DNNs”. arXiv preprint arXiv:1910.11605 (2019).
43. Goodfellow I., et al. “Deep learning”. Cambridge: MIT press 1.2 (2016).

Citation

Alessandro Del Prete., et al. “Predictive Algorithms for Maintenance Planning and Optimization in Industrial Applications". Clareus Scientific Science and Engineering 2.5 (2025): 41-51.

Copyright

© 2025 Alessandro Del Prete., et al. Licensee Clareus Scientific Publications. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.