Periodicity.: September - October 2019
e-ISSN......: 2236-269X
Cover Image

Systems success measurement: instrument & framework a new perspective

Ibrahim Eskandar Ibrahim Fadhel, Syed Zulkarnain Bin Syed Idrus, Mohd Syukri Yeoh Abdullah, Amani Ali Elmetwaly Ali Ibrahim, Mazni Binti Omar

Abstract


There are major concerns regarding the previous research that can be said as having poor measurements, lack of theoretical grounding, and concern heavily on investment justifies. Some are also considered as having weak survey instruments, inappropriate data collection approaches, and lack of agreement on the dependent variables measurement that resulted in its incompatibility. Several related issues pertaining to high rates of systems’ failure that justify the heavy investment costs and affect the systems’ success of measurement. This matter had raised a high concern, especially for the researchers, practitioners, organization managers and systems administrators. Thus, the successful measurement of any systems is vital. The purpose of this study is to provide a framework and a high-quality validated instrument based on the literature review and experts’ validation. Subsequently, it can be used in the future studies to efficiently assist in the systems’ success measurement. The statistical test of 344 users with the smart PLS for the instrument shows an excellent result.

Keywords


ISO 25010; Systems Success; Systems Fail; DeLone and McLean; TAM; Instrument.

Full Text:

PDF HTML

References


ADEBOLA, B. D. (2014) The determinant of information security practices towards organizational performance in the banking sector evidence from Nigeria. Universiti Utara Malaysia.

AGHAZADEH, S.; PIRNEJAD, H.; ALIEV, A.; MORADKHANI, A. (2015) Evaluating the effect of software quality characteristics on health care quality indicators. Journal of Health Management and Informatics, v. 2, n. 3, p. 67–73.

ALADWANI, A. M.; PALVIA, P. C. (2002) Developing and validating an instrument for measuring user-perceived web quality. Information.; Management, v. 39, n. 6, p. 467–476.

ALARCÓN, D.; SÁNCHEZ, J. A.; DE OLAVIDE, U. P. (2015) Assessing convergent and discriminant validity in the ADHD-R IV rating scale: User-written commands for Average Variance Extracted (AVE, p. Composite Reliability (CR, p. and Heterotrait-Monotrait ratio of correlations (HTMT) Composite Reliability (CR, p. and Heterotrait-Monotrait Ratio of Correlations (HTMT, p. p. 1–39. In Spanish STATA Meeting

ALSHIBLY, H. H. (2015) Investigating decision support system (DSS) success: A partial least squares structural equation modeling approach. Journal of Business Studies Quarterly, v. 6, n. 4, p. 56.

ALVES, J. M.; WANGENHEIM, C.; LACERDA, T.; SAVARIS, A.; WANGENHEIM, A. (2015) Adequate software quality evaluation model v1. 0. Instituto Nacional Para Convergência Digital–INCoD, Tech. Rep.

ARDITO, C.; COSTABILE, M. F.; DE MARSICO, M.; LANZILOTTI, R.; LEVIALDI, S.; ROSELLI, T.; ROSSANO, V. (2006) An approach to usability evaluation of e-learning applications. Universal access in the information society, v. 4, n. 3, p. 270–283.

ASTANI, M.; ELHINDI, M. (2008) An empirical study of university websites. Issues in Information Systems, v. 9, n. 2, p. 460–465.

AL-AZAWEI, A.; LUNDQVIST, K. (2015) Learner differences in perceived satisfaction of an online learning: An extension to the technology acceptance model in an Arabic sample. Electronic Journal of E-Learning, 13(5, p. 408–426.

AL-MANASRA, E.; KHAIR, M.; ZAID, S. A.; TAHERQUTAISHAT, F. (2013) Investigating the impact of website quality on consumers’ satisfaction in Jordanian telecommunication sector. Arab Economic and Business Journal, v. 8, n. 1–2, p. 31–37.

BARCLAY, D.; HIGGINS, C.; THOMPSON, R. (1995) The Partial Least Squares (pls) Approach to Casual Modeling: Personal Computer Adoption Ans Use as an Illustration.

BHARATI, P.; CHAUDHURY, A. (2004) An empirical investigation of decision-making satisfaction in web-based decision support systems. Decision Support Systems, v. 37, n. 2, p. 187–197.

BROWN, R. M.; MAZZAROL, T. (2006) Factors driving student satisfaction and loyalty in Australian universities: the importance of institutional image. In 20th Annual Australia.; New Zealand Academy of Management (ANZAM) Conference Paper, p. 1–12.

CAVANA, R. Y.; DELAHAYE, B. L.; SEKARAN, U. (2001) Applied business research: Qualitative and quantitative methods. John Wiley.; Sons Australia.

CHIANG, L.; HUANG, C.-Y.; YANG, W.-C. (2011) The effect of user’s satisfaction of web security on trust in eGovernment. In the Proceedings of the 11th European Conference on EGovernment: Faculty of Administration, University of Ljubljana, Ljubljana, Slovenia, p. 140.

CHIU, P.-S.; CHAO, I.-C.; KAO, C.-C.; PU, Y.-H.; HUANG, Y.-M. (2016) Implementation and evaluation of mobile e-books in a cloud bookcase using the information system success model. Library Hi Tech, v. 34, n. 2, p. 207–223.

CHOW, W. S.; CHAN, L. S. (2008) Social network and shared goals in organizational knowledge sharing.

CONSTANTIN, A. M. (2013) The antecedents of e-satisfaction and e-loyalty. Timisoara Journal of Economics, v. 5, n. 2, 18, p. 236–252.

COUCHMAN, P. K.; FULOP, L. (2006) Building trust in cross-sector R&D collaborations: Exploring the role of credible commitments. In 22nd EGOS Colloquium, Bergen, Norway.

CRONIN, J. J.; BRADY, M. K.; HULT, G. T. M. (2000) Assessing the effects of quality, value, and customer satisfaction on consumer behavioral intentions in service environments. Journal of Retailing, v. 76, n. 2, p. 193–218.

DAVARPANAH, A.; MOHAMED, N. (2013) Human resource information systems (HRIS) success factors in a public higher education institution context. In Research and Innovation in Information Systems (ICRIIS, p. 2013 International Conference on, p. 79–84, IEEE.

DAVIS, F. D. (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, p. 319–340.

DELONE, W. H.; MCLEAN, E. R. (2016) Information systems success measurement. foundations and trends® in Information Systems, v. 2, n. 1, p. 1–116.

DELONE, W. H.; MCLEAN, E. R. (1992) Information systems success: the quest for the dependent variable. Information Systems Research, v. 3, n. 1, p. 60–95.

DERNBECHER, S. (2014) Having the mind in the cloud: organizational mindfulness and the successful use of desktop as a service. In System Sciences (HICSS, p. 2014 47th Hawaii International Conference on, p. 2137–2146, IEEE.

DEVARAJ, S.; FAN, M.; KOHLI, R. (2002) Antecedents of B2C channel satisfaction and preference: validating e-commerce metrics. Information Systems Research, v. 13, n. 3, p. 316–333.

DOHI, T.; NAKAGAWA, N. (2013) Stochastic reliability and maintenance modelling. Springer.

DREHEEB, A. E.; BASIR, N.; FABIL, N. (2016) Impact of system quality on users’ satisfaction in continuation of the use of e-learning system. International Journal of E-Education, e-Business, e-Management and e-Learning, v. 6, n. 1, p. 13.

DYBA, T. (2005) An empirical investigation of the key factors for success in software process improvement. IEEE Transactions on Software Engineering, v. 31, n. 5, p. 410–424.

EPPLER, M. J.; ALGESHEIMER, R.; DIMPFEL, M. (2003) Quality Criteria of content-driven websites and their influence on customer satisfaction and loyalty: An Empirical Test of an Information Quality Framework. In IQ, p. 108–120.

FADHEL, I. E. I. (2015) An evaluation of information system success based on students’ perspective. Universiti Utara Malaysia.

FADHEL, I. E. I.; IDRUS, S. Z. S.; IBRAHIM, A. A. E. A.; OMAR, M.; BAHASHWAN, A. A. A.;.; AL-ANSI, A. A. M. (2018a) An integration between information systems engineering and software engineering theories towards engineering a novel framework of web-based systems success for institutions based on students’ perceptions. Journal of Physics: Conference Series, v. 1019, n. 1, p. 12081. Retrieved from http://stacks.iop.org/1742-6596/1019/i=1/a=012081

FADHEL, I. E. I.; IDRUS, S. Z. S.; IBRAHIM, A. A. E. A.; OMAR, M.; BAHESHWAN, F.; ALBZEIRAT, M. K.;.; ALBZEIRAT, S. K. (2018b) Measuring System Success In New Context By Adapting DM 2003 Framework With The External Factor Management Support. Journal of Physics: Conference Series, v. 1019, n. 1, p. 12003. Retrieved from http://stacks.iop.org/1742-6596/1019/i=1/a=012003

FIELD, A. (2005) Discovering statistics with SPSS. London: Sage.

FORSGREN, N.; DURCIKOVA, A.; CLAY, P. F.; WANG, X. (2016) The Integrated user satisfaction model: Assessing information quality and system quality as second-order constructs. In system administration. Communications of the Association for Information Systems, v. 38, n. 1, p. 39.

FORNELL, C.; LARCKER, D. F. (1981) Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, p. 39–50.

GORLA, N.; SOMERS, T. M.; WONG, B. (2010) Organizational impact of system quality, information quality, and service quality. The Journal of Strategic Information Systems, v. 19, n. 3, p. 207–228.

GUADAGNOLI, E.; VELICER, W. F. (1988) Relation to sample size to the stability of component patterns. Psychological Bulletin, v. 103, n. 2, p. 265.

HAIR JR, J. F.; HULT, G. T. M.; RINGLE, C.; SARSTEDT, M. (2017) A primer on partial least squares structural equation modeling (PLS-SEM) Thousand Oaks: Sage Publications.

HENSELER, J.; RINGLE, C. M.; SARSTEDT, M. (2014) A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, v. 43, n. 1, p. 115–135.

HOLBROOK, A. L.; KROSNICK, J. A.; MOORE, D.; TOURANGEAU, R. (2007) Response order effects in dichotomous categorical questions presented orally: The impact of question and respondent attributes. Public Opinion Quarterly, v. 71, n. 3, p. 325–348.

HUANG, C.-C.; WANG, Y.-M.; WU, T.-W.; WANG, P.-A. (2013) An empirical analysis of the antecedents and performance consequences of using the moodle platform. International Journal of Information and Education Technology, v. 3, n. 2, p. 217.

JANSEN, H.; HAK, T. (2005) The productivity of the three-step test-interview (TSTI) compared to an expert review of a self-administered questionnaire on alcohol consumption. Journal of Official Statistics, v. 21, n. 1, p. 103.

JEON, M. M. (2009) Impact of perceived website service quality on customer e-loyalty on a lodging website. Iowa State University.

JONGCHUL, O.; YOON, S.-J. (2014) Validation of haptic enabling technology acceptance model (HE-TAM): Integration of IDT and TAM. Telematics and Informatics, v. 31, n. 4, p. 585–596.

KHAWAJA, K. F.; BOKHARI, R. H. (2010) Exploring the factors associated with quality of website. Global Journal of Computer Science and Technology, v. 10, n. 14, p. 37–45.

KHRED, D. A. (2017) Systems development in our country.

KIRAN, K.; DILJIT, S. (2011) Antecedents of customer loyalty: Does service quality suffice? Malaysian Journal of Library.; Information Science, v. 16, n. 2, p. 95–113.

KLINE, R. B. (2011) Principles and practice of structural equation modeling, 3rd edn Guilford Press. New York.

LIAW, S.-S.; HUANG, H.-M. (2013) Perceived satisfaction, perceived usefulness and interactive learning environments as predictors to self-regulation in e-learning environments. Computers.; Education, v. 60, n. 1, p. 14–24.

LIU, I.-F.; CHEN, M. C.; SUN, Y. S.; WIBLE, D.; KUO, C.-H. (2010) Extending the TAM model to explore the factors that affect Intention to Use an Online Learning Community. Computers.; Education, v. 54, n. 2, p. 600–610.

LUDIN, I. H. B. H.; CHENG, B. L. (2014) Factors influencing customer satisfaction and e-loyalty: online shopping environment among the young adults. Management Dynamics in the Knowledge Economy, v. 2, n. 3, p. 462.

MACCALLUM, R. C WIDAMAN, K. F.; PREACHER, K. J.; HONG, S. (2001) Sample size in factor analysis: The role of model error. Multivariate Behavioral Research, v. 36, n. 4, p. 611–637.

MADAN, A.; DUBEY, S. K. (2012) Usability evaluation methods: a literature review. International Journal of Engineering Science and Technology, v. 4, n. 2, p. 590–599.

MALIK, B. H.; SHUQIN, C.; MASTOI, A. G.; GUL, N.; GUL, H. (2016) Evaluating citizen e-satisfaction from e-government services: A Case of Pakistan. European Scientific Journal, ESJ, v. 12, n. 5.

MARDIANA, S.; TJAKRAATMADJA, J. H.; APRIANINGSIH, A. (2015) Validating the Conceptual Model for Predicting Intention to Use as Part of Information System Success Model: The Case of an Indonesian Government Agency. Procedia Computer Science, n. 72, p. 353–360.

MBIWA, P. N. (2014) Evaluating the quality of management information systems: A case of SACCOs in Nairobi, Kenya. University of Nairobi.

MCGILL, T. J.; HOBBS, V. J.; KLOBAS, J. E. (2003) User developed applications and information systems success: A test of DeLone and McLean’s model. Information Resources Management Journal, v. 16, n. 1, p. 24–45.

MEBRATE, T. (2010) A framework for evaluating academic website’s quality from students’ perspective. TU Delft, Delft University of Technology.

MOHAMMADI, H. (2015) Investigating users’ perspectives on e-learning: an integration of TAM and IS success model. Computers in Human Behavior, n. 45, p. 359–374.

OFORI, K. S.; LARBI-SIAW, O.; FIANU, E.; GLADJAH, R. E.; BOATENG, E. O. Y. (2016) Factors influencing the continuance use of mobile social media: The effect of privacy concerns. Journal of Cyber Security and Mobility, v. 4, n. 3, p. 105–124.

OLSON, K. (2010) An examination of questionnaire evaluation by expert reviewers. Field Methods, v. 22, n. 4, p. 295–318.

PADAYACHEE, I.; KOTZE, P.; VAN DER MERWE, A. (2010) ISO 9126 external systems quality characteristics, sub-characteristics and domain specific criteria for evaluating e-Learning systems. The Southern African Computer Lecturers’ Association, University of Pretoria, South Africa.

PALLANT, J. (2007) SPSS survival manual: A step guide to data analysis using SPSS for Windows (3rd Ed) Australia: Allan.; Unwin.

PALLANT, J. (2013) SPSS survival manual. McGraw-Hill Education (UK)

PAPANIKOLAOU, K.; MAVROMOUSTAKOS, S. (2008) Web2Train: A design model for corporate e-learning systems. In BIS (Workshops), p. 155–163.

PARK, N.; RHOADS, M.; HOU, J.; LEE, K. M. (2014) Understanding the acceptance of teleconferencing systems among employees: An extension of the technology acceptance model. Computers in Human Behavior, n. 39, p. 118–127.

PRESSER, S.; BLAIR, J. (1994) Survey pretesting: Do different methods produce different results? Sociological Methodology, p. 73–104.

ROCHA, Á. (2012) Framework for a global quality evaluation of a website. Online Information Review, v. 36, n. 3, p. 374–382.

SEKARAN, U. (2003) Research methods for business: A skill building approach. John Wiley.; Sons.

SEKARAN, U.; BOUGIE, R. (2010) Research methods for business: A skill building approach. Chichester: John Willey.; Sons Ltd. 2010.

SELVAKUMAR, J. J. (2016) Impact of service quality on customer satisfaction in public sector and private sector banks. Purushartha: A Journal of Management Ethics and Spirituality, v. 8, n. 1.

SENATE, A. (2001) Universities in crisis. Canberra: Commonwealth of Australia.

SHIRATUDDIN, N. (2015) Evaluation of e-Book applications using ISO 25010. In Technology Management and Emerging Technologies (ISTMET, p. 2015 International Symposium on, p. 114–118, IEEE.

STEVENS, J. (1992) Applied multivariate statistics for the social sciences (2nd edition) Hillsdale, NJ:Erlbaum.

TABACHNICK, B. G.; FIDELL, L. S. (2007) Using multivariate statistics. Allyn.; Bacon/Pearson Education.

TANDON, U.; KIRAN, R.; SAH, A. (2017a) Analyzing customer satisfaction: users perspective towards online shopping. Nankai Business Review International, v. 8, n. 3, p. 266–288.

THEIS, B.; FROOD, J.; NISHRI, D.; MARRETT, L. D. (2002) Evaluation of a risk factor survey with three assessment methods. Chronic Diseases and Injuries in Canada, v. 23, n. 1, p. 1.

TEO, T. S. H.; SRIVASTAVA, S. C.; JIANG, L. (2008) Trust and electronic government success: An empirical study. Journal of Management Information Systems, v. 25, n. 3, p. 99–132.

UNIT, M. C. AND B. S. (2013) What thresholds should I use for factor loading cut-offs? Retrieved from http://imaging.mrc-cbu.cam.ac.uk/statswiki/FAQ/thresholds

VAEZI, R.; MILLS, A.; CHIN, W.; ZAFAR, H. (2016) User satisfaction research in information systems: Historical roots and approaches. Communications of the Association for Information Systems, v. 38, n. 27, p. 501–532.

VALVI, A. C.; WEST, D. C. (2013) E-loyalty is not all about trust, price also matters: extending expectation-confirmation theory in bookselling websites. Journal of Electronic Commerce Research, v. 14, n. 1, p. 99.

VAN IWAARDEN, J.; VAN DER WIELE, T.; BALL, L.; MILLEN, R. (2004) Perceptions about the quality of web sites: a survey amongst students at Northeastern University and Erasmus University. Information.; Management, v. 41, n. 8, p. 947–959.

WANG, Y.-S.; LIAO, Y.-W. (2008) Assessing eGovernment systems success: A validation of the DeLone and McLean model of information systems success. Government Information Quarterly, v. 25, n. 4, p. 717–733.

WIXOM, B. H.; WATSON, H. J. (2001) An empirical investigation of the factors affecting data warehousing success. MIS Quarterly, p. 17–41.

WOLFINBARGER, M.; GILLY, M. C. (2003) eTailQ: dimensionalzing, measuring and predicting etail quality. Journal of Retailing, v. 79, n. 3, p. 183–198.

ZAIED, A. N. H. (2012) An integrated success model for evaluating information system in public sectors. Journal of Emerging Trends in Computing and Information Sciences, v. 3, n. 6, p. 814–825.

ZEHIR, C.; SEHITOGLU, Y.; NARCIKARA, E.; ZEHIR, S. (2014) ES-quality, perceived value and loyalty intentions relationships in internet retailers. Procedia-Social and Behavioral Sciences, n. 150, p. 1071–1079.

ZEITHAML, V. A.; BERRY, L. L.; PARASURAMAN, A. (1996) The behavioral consequences of service quality. The Journal of Marketing, p. 31–46.




DOI: http://dx.doi.org/10.14807/ijmp.v10i5.872

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Refbacks

  • There are currently no refbacks.


Copyright (c) 2019 Ibrahim Eskandar

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

LIBRARIES BY

Logo Gaudeamus

Logo INDIANA

Logo CHENG KUNG

Logo UTEP

Logo MOBIUS

Logo UNIVEM

Logo Kennedy

Logo Columbia

Logo UCS

Logo MSG/UFF

Logo OPT

Logo Biblioteca Professor Milton Cabral Moreira

Logo UFL

Logo ULRICHSWEB

Logo UNISA