Evaluation and Validation of a Developed Hybrid Models for ERP System Usability and Its Influence on User Satisfaction in Enterprises
Abstract
Enterprise Resource Planning (ERP) systems have become indispensable for integrating organizational processes and improving operational efficiency. However, evaluating ERP system effectiveness remains challenging because conventional evaluation methods often rely on subjective assessment techniques and fail to capture the complex relationships among multiple organizational and operational performance indicators. This study proposes a hybrid framework that integrates Inverse Variance Weighting (IVW), Tabu Search Algorithm (TSA), and supervised Machine Learning (ML) models for objective ERP effectiveness evaluation. Operational and organizational Key Performance Indicators (KPIs), including throughput, lead time, defect rate, machine utilization, system uptime, response time, employee productivity, order fulfillment rate, supply chain responsiveness, customer satisfaction, cost efficiency, and return on investment, were employed to construct a composite ERP effectiveness index. The IVW technique objectively assigned KPI weights based on statistical stability, while Tabu Search optimized KPI selection and feature combinations. Linear Regression, Support Vector Regression, and Random Forest Regression models were subsequently trained to predict ERP effectiveness. Model performance was evaluated using Mean Absolute Error (MAE), Root Mean Square Error (RMSE), coefficient of determination (R²), and five-fold cross-validation. The proposed framework provides an objective, scalable, and intelligent decision-support mechanism capable of improving ERP performance evaluation in manufacturing environments. The study contributes a novel hybrid optimization and machine learning framework that enhances predictive accuracy while reducing the subjectivity associated with conventional ERP evaluation techniques.
Keywords
Download Options
Introduction
Enterprise Resource Planning (ERP) systems have transformed organizational operations by integrating business processes into a unified platform that facilitates information sharing, operational efficiency, and strategic decision-making. Manufacturing organizations increasingly depend on ERP systems to coordinate procurement, inventory management, production scheduling, finance, human resource management, and customer relationship management. The integration of these functions enables organizations to reduce operational costs, improve productivity, enhance customer satisfaction, and achieve sustainable competitive advantage.
Despite these advantages, evaluating the effectiveness of ERP systems remains a significant challenge. Traditional ERP evaluation approaches primarily rely on subjective assessment methods, expert judgment, balanced scorecards, or simple Key Performance Indicator (KPI) aggregation techniques. Although these approaches provide useful managerial insights, they often fail to capture the nonlinear relationships that exist among organizational performance variables. Furthermore, subjective weighting methods such as the Analytic Hierarchy Process (AHP) introduce human bias into the evaluation process, thereby reducing the reliability and objectivity of ERP performance assessment.
Conclusion
The findings demonstrate that the proposed hybrid framework provides an objective, scalable, and intelligent mechanism for ERP performance evaluation in manufacturing organizations. By combining optimization and machine learning, the framework reduces evaluator bias, improves predictive accuracy, and supports evidence-based managerial decision-making. Consequently, the study contributes to both ERP research and intelligent manufacturing by providing a practical approach for monitoring and improving enterprise system performance.
References
[1] Alsharari, N. M. (2023). Enterprise resource planning systems and organizational performance: A review of recent developments.
[2] Alzubaidi, L., Zhang, J., Humaidi, A. J., et al. (2023). Review of machine learning: Concepts, applications, and challenges. Journal of Big Data, *10*(1), 1-74.
[3] Ashok, M. (2024a). Machine learning approaches for demand forecasting in supply chain management.
[4] Ashok, M. (2024b). Reinforcement learning for inventory optimization in enterprise systems.
[5] Balić, J., et al. (2022). Digital manufacturing and ERP data acquisition for industrial analytics.
[6] Cabello-Solorzano, D., et al. (2023). Data preprocessing techniques for industrial machine learning applications.
[7] Chen, X., Zhang, Y., & Wang, H. (2021). Optimization of enterprise resource planning workflows using genetic algorithms and particle swarm optimization.
[8] Dachepalli, R. (2025). Reinforcement learning framework for dynamic resource allocation in enterprise resource planning systems.
[9] Dubey, R., Gunasekaran, A., Childe, S. J., et al. (2022). Big data analytics and enterprise resource planning performance: A structural equation modeling approach.
[10] Faveto, G., et al. (2024). Key performance indicators for warehouse systems: A systematic review.
[11] Ghodake, P., et al. (2024). Feature scaling techniques for machine learning-based industrial applications.
[12] Glover, F. (1986). Future paths for integer programming and links to artificial intelligence. Computers & Operations Research, *13*(5), 533-549.
[13] Glover, F. (1989). Tabu search—Part I. INFORMS Journal on Computing, *1*(3), 190-206.
[14] Hatefi, S. M. (2023). Objective weighting methods in multi-criteria decision-making: A comparative review.
[15] Ismail-Fawaz, A., et al. (2023). Sensitivity analysis methods for machine learning prediction models.
[16] Jawad, Z. N., & Balázs, V. (2024). Machine learning-driven optimization of enterprise resource planning (ERP) systems: A comprehensive review. Beni-Suef University Journal of Basic and Applied Sciences, *13*.
[17] Kumar, R., & Singh, P. (2022). Hybrid AHP–TOPSIS framework for enterprise resource planning system evaluation.
[18] Marín-Martínez, F., & Sánchez-Meca, J. (2009). Weighting by inverse variance or by sample size in meta-analysis.
[19] Martínez-Caro, E., et al. (2023). Predictive analytics and automation in enterprise resource planning systems: A systematic review.
[20] Nabi, M., et al. (2024). Machine learning-based disruption prediction for resilient supply chain management.
[21] Qureshi, M. A. (2022). Critical success factors for enterprise resource planning implementation in sustainable supply chains.
[22] Rahman, M., & Islam, M. (2023). Predicting enterprise resource planning project success using machine learning techniques.
[23] Tortorella, G. L., et al. (2022). Industry 4.0 technologies and enterprise resource planning integration: A systematic review.
[24] Zakaria, N., et al. (2024). Machine learning applications in enterprise human capital management.
[25] Zhao, Y., et al. (2023). Reinforcement learning for enterprise resource planning process optimization.