Program Learning Outcomes of Students in Bachelor of Science and Engineering Degrees: A Systematic Review
Abstract
This study aims to explore the assessment of program-level learning outcomes for students in Bachelor of Science and Engineering programs through a systematic review of 65 documents from ERIC, IEEE Xplore, Scopus, Web of Science, and Google Scholar during the past 14 years. The methodology involves synthesizing and analyzing assessment methods, their effectiveness, and the challenges reported in the studies, with a focus on geographic distribution, program types, study designs, outcome types measured, and alignment with accreditation standards. Key findings reveal that direct methods, such as course-based assessments, and indirect methods, such as surveys, are widely used, while mixed approaches emerge as a comprehensive strategy for evaluating technical skills, soft skills, and professional competencies. The studies primarily originate from Asia and North America, aligning with prominent accreditation frameworks. The review proposes an optimal approach—a flexible, mixed strategy integrating direct and indirect tools, technology (e.g., AI), and stakeholder input—to ensure thorough and feasible program-level learning outcomes assessment, providing a practical solution for universities, educators, and researchers to enhance outcome assessment in Bachelor of Science and Engineering programs.
https://doi.org/10.26803/ijlter.24.3.19
Keywords
Full Text:
PDFReferences
Agrawal, E., Tungikar, V., & Joshi, Y. (2021). Method for assessment and attainment of course and program outcomes for Tier-I institutes in India. Journal of Engineering Education Transformations (JEET), 34(3), 35–41. https://doi.org/10.16920/jeet/2021/v34i3/157723
Alhakami, H. H., Al-Masabi, B. A., & Alsubait, T. M. (2020). Data analytics of student learning outcomes using ABET course files. Intelligent Computing: Proceedings of the 2020 Computing Conference, 1, 309–325. https://doi.org/10.1007/978-3-030-52249-0_22
Ali, Q. I. (2023). Surveying different student outcome assessment methods for ABET-accredited computer engineering programs. Research Reports on Computer Science, 56–76. https://doi.org/10.37256/rrcs.2120232577
Ali, Q. I. (2024). Towards more effective summative assessment in OBE: A new framework integrating direct measurements and technology. Discovery Education, 3, 107. https://doi.org/10.1007/s44217-024-00208-5
Altamirano, E., Gazca, L., López-Malo, A., & Palou, E. (2013). Direct and indirect assessment of Universidad de las Américas Puebla’s Food Engineering program outcomes. ASEE Annual Conference and Exposition, Conference Proceedings. https://peer.asee.org/direct-and-indirect-assessment-of-universidad-de-las-americas-puebla-s-food-engineering-program-outcomes.pdf
Alyahya, S., & El-Nasr, A. A. (2012). Outcomes-based assessment of the engineering programs at Qassim University for ABET accreditation. IEEE International Conference on Interactive Mobile Communication, Technologies and Learning (IMCL). https://doi.org/10.1109/IMCL.2012.6396445
Alzubaidi, L., & Jabur, I. (2024). Student outcomes assessments using deep learning. Journal of Electrical Systems, 20(7s), 3647–3654. https://www.researchgate.net/publication/384232231
Ammar, K., & Rais, R. N. B. (2023). Outcome-based performance evaluation approach using rubric-based capstone projects assessment. 20th International Multi-Conference on Systems, Signals & Devices (SSD), 777–783. https://doi.org/10.1109/SSD58187.2023.10411275
Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology: Theory & Practice, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616
Awad, W. S., & Almhosen, K. A. (2023). A framework of outcome-based assessment and evaluation for computing programs. International Journal of Advanced Computer Science and Applications (IJACSA), 14(1), 371–380. https://doi.org/10.14569/IJACSA.2023.0140140
Balaji, N., & Pai, K. B. H. (2020). Course outcome and programme outcome in OBE: An illustration in engineering and technology. Journal of Engineering Education Transformations (JEET), January 2020. https://doi.org/10.16920/jeet/2020/v33i0/150188
Balasubramani, R., & Chiplunkar, N. N. (2017). Attainment of programme outcomes through course outcomes in outcome-based education: A case study. Journal of Engineering Education Transformations (JEET), 31(2). https://journaleet.in/articles/attainment-of-programme-outcomes-through-course-outcomes-in-outcome-based-education--a-case-study
Basri, N. E. A., Taib, K. A., Jaafar, O., Zain, S. M., Suja’, F., Kasa, A., Osman, S. A., & Shanmugam, N. E. (2011). An evaluation of programme educational objectives and programme outcomes for civil engineering programmes. Procedia - Social and Behavioral Sciences, 18, 56–64. https://doi.org/10.1016/j.sbspro.2011.05.009
Battistini, A., & Kitch, W. A. (2021, July). Make assessment straightforward: A case study on the successful implementation of ABET student outcomes 1-7. Proceedings of the 2021 ASEE Virtual Annual Conference & Exposition, Virtual Conference. https://doi.org/10.18260/1-2--37472
Bhatia, J., & Singh, H. (2017). Indirect assessment of outcomes in education: A review. International Journal of Computer Sciences and Engineering (IJCSE), 5(12), 273–278. https://doi.org/10.26438/ijcse/v5i12.273278
Bhatti, S., Memon, M., & Meghji, A. F. (2023). Scrutinizing outcome assessment of outcome-based education using Q-OBE in engineering education. International Journal of Innovation in Teaching and Learning (IJITL), 9(1), 1–10. https://doi.org/10.35993/ijitl.v9i1.2609
Booth, A. (2016). Searching for qualitative research for inclusion in systematic reviews: A structured methodological review. Systematic Reviews, 5, Article 74. https://doi.org/10.1186/s13643-016-0249-x
Briner, R. B., & Denyer, D. (2012). Systematic review and evidence synthesis as a practice and scholarship tool. In D. M. Rousseau (Ed.), The Oxford handbook of evidence-based management (pp. 112–129). Oxford University Press. http://dx.doi.org/10.1093/oxfordhb/9780199763986.013.0007
Clarke, M. (2017). Rethinking graduate employability: The role of capital, individual attributes and context. Studies in Higher Education, 43(11), 1923–1937. https://doi.org/10.1080/03075079.2017.1294152
Critical Appraisal Skills Programme (CASP). (2018). CASP checklist: For qualitative research. https://casp-uk.net/casp-checklists/CASP-checklist-qualitative-2024.pdf
Cruz, M. L., Saunders-Smits, G. N., & Groen, P. (2019). Evaluation of competency methods in engineering education: A systematic review. European Journal of Engineering Education, 45(5), 729–757. https://doi.org/10.1080/03043797.2019.1671810
Damaj, I., & Yousafzai, J. (2019). Effective assessment of student outcomes in computer engineering programs using a minimalistic framework. International Journal of Engineering Education, 35(1), 59–75. https://doi.org/10.13140/RG.2.2.32415.71841
Davis, W. J., Ghanat, S. T., Brown, K. T., & Michalaka, D. (2017, June). Application of indirect and direct measures for student teamwork outcome assessment within an undergraduate civil engineering curriculum. 2017 ASEE Annual Conference & Exposition. https://doi.org/10.18260/1-2--27597
El-Kady, M., Alsadaawi, A., Al-Zahrani, S., & Ajbar, A. (2014). Assessing learning outcomes in electrical engineering education: A case study from Saudi Arabia. International Journal of Electrical Engineering Education, 51(4), 355–367. https://doi.org/10.7227/IJEEE.0007
Exter, M. E., Tagare, D., & Sabin, M. (2024). Develop a competency-based curriculum that purposefully integrates computing skills, cross-disciplinary skills, and dispositions. Proceedings of the ACM Conference on Computer Science Education. https://doi.org/10.1145/3626253.3633410
Forcael, E., Garcés, G., & Orozco, F. (2022). Relationship between professional competencies required by engineering students according to ABET and CDIO and teaching–learning techniques. IEEE Transactions on Education, 65(1), 46–55. https://doi.org/10.1109/TE.2021.3086766
Gamboa, R., Namasivayam, S., Al-Atabi, M., & Singh, R. (2013). Quantitative measurement of students’ PO attainments for Taylor’s University engineering programmes. Procedia - Social and Behavioral Sciences, 103, 1147–1156. https://doi.org/10.1016/j.sbspro.2013.10.396
Ghaly, S. M. A. (2020). Indirect evaluation of program educational objectives and student outcomes for engineering programs: A case study. Engineering, Technology & Applied Science Research, 10(5), 6209–6213. https://doi.org/10.48084/etasr.3751
Gough, D., Oliver, S., Thomas, J., & Bean, M. R. (2017). An introduction to systematic reviews (2nd ed.). Psychology Teaching Review, 23(2), 95–96. https://doi.org/10.53841/bpsptr.2017.23.2.95
Gough, D., Thomas, J., & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1, 28. https://doi.org/10.1186/2046-4053-1-28
Hasan, H., Ambiyar, Rizal, F., Maksum, H., & Putri, T. T. A. (2024). Evaluation on students’ achievement for outcome-based education on engineering students. Journal of Computer Science, Information Technology and Telecommunication Engineering (JCoSITTE), 5(2), 1–10. https://doi.org/10.30596/jcositte.v5i2.20014
Hashim, N., & Hashim, H. (2010). Outcome-based education performance evaluation on final year degree project. Latest Trends on Engineering Education, 1–7. https://www.researchgate.net/publication/228496780
Hatipkarasulu, Y., Gunhan, S., Pishdad-Bozorgi, P., & Patin, J. (2012). Development and implementation of multi-level outcomes assessment plan for construction degree program. ASEE Annual Conference & Exposition Proceedings. https://doi.org/10.18260/1-2--21206
Hatipkarasulu, Y., & Hatipkarasulu, G. S. (2021, July). Direct assessment of student learning outcomes. 2021 ASEE Virtual Annual Conference Content Access. https://peer.asee.org/direct-assessment-of-student-learning-outcomes
Hensel, E., & Robinson, R. (2014). Direct assessment of student learning outcomes by analysis of performance evaluation of student employees. Proceedings of the 12th LACCEI Conference. https://www.laccei.org/LACCEI2014-Guayaquil/RefereedPapers/RP193.pdf
Higgins, J. P. T., & Green, S. (2011). Cochrane handbook for systematic reviews of interventions (Version 5.1.0). The Cochrane Collaboration. http://handbook-5-1.cochrane.org
Hinojosa-Rivera, M., Reyes-Melo, M. E., & Cazares, J. A. (2015). Educational objectives, outcomes and competencies assessment for a Latin-American materials engineering program. Proceedings of the ASEE Annual Conference & Exposition. https://www.researchgate.net/publication/278704389
Hussain, W., Spady, W. G., Khan, S. Z., Khawaja, B. A., Naqash, T., & Conner, L. (2021). Impact evaluations of engineering programs using ABET student outcomes. IEEE Access, 9, 46166–46190. https://doi.org/10.1109/ACCESS.2021.3066921
Kalaani, Y., & Haddad, R. J. (2014). Continuous improvement in the assessment process of engineering programs. 2014 ASEE Southeast Section Conference. http://se.asee.org/proceedings/ASEE2014/Papers2014/4/44.pdf
Kamran, M., Fazal, M. R., Abid, M. I., & Abid, I. (2020). Implementation of the outcome-based education system in engineering programs for Pakistan Engineering Council accreditation under Washington Accord signatory. Science International (Lahore), 32(2), 197–206. https://www.researchgate.net/publication/340095539
Lavanya, C., & Murthy, J. N. (2022). Assessment and attainment of course outcomes and program outcomes. Journal of Engineering Education Transformations (JEET), 35(4), 104–111. https://doi.org/10.16920/jeet/2022/v35i4/22109
Luzan, P., Titova, O., Kurok, R., & Mosia, I. (2021). The methodology for assessment of engineering students outcomes. 2021 IEEE International Conference on Modern Electrical and Energy Systems (MEES), 1–6. https://doi.org/10.1109/MEES52427.2021.9598666
Mahrishi, M., Jain, P. K., & Hosseini, S. (2023). Towards assessment and attainment of engineering graduate attributes in outcome-based education (OBE). Proceedings of the IEEE Conference on Future of Educational Innovation, 1–7. https://doi.org/10.1109/IEEECONF56852.2023.10104817
Manteufel, R. D., & Karimi, A. (2016, June). Promoting consistent assessment of student learning outcomes over multiple courses and multiple instructors in continuous program improvement. Proceedings of the 2016 ASEE Annual Conference & Exposition. https://doi.org/10.18260/p.25986
Mishra, A., & Singh, S. (2024). Smart pedagogical learning and assessment methodologies—Outcome-based education: A novel quality assurance framework with attainment computation. Journal of Computers in Education, 12(4). https://doi.org/10.1007/s40692-024-00344-9
Mistamiruddin, E., & Nasri, N. M. (2024). Challenges in integrating outcome-based education (OBE) in higher education institutions: A systematic literature review. International Journal of Academic Research in Progressive Education and Development, 13(4), 1135–1150. http://dx.doi.org/10.6007/IJARPED/v13-i4/23378
Mohd Noor, N., Muhamad, N., Alias, R., Alias, A., & Rosli, S. (2024). Analysis of program outcomes attainment for diploma in civil engineering at Universiti Teknologi MARA Pahang. International Journal of Evaluation and Research in Education (IJERE), 13(3). https://doi.org/10.11591/ijere.v13i3.26996
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analysis: The PRISMA statement. PLOS Medicine, 6(7), e1000097. http://dx.doi.org/10.1371/journal.pmed.1000097
Mynderse, J. A. (2022). Assessing ABET student outcome 7 (New Knowledge) with measurement systems [Paper presentation]. American Society for Engineering Education (ASEE) Annual Conference & Exposition. https://doi.org/10.18260/1-2--40917
Nandy, A. (2022, August). A highly integrated and successful approach to program development and implementation of accreditation strategies for an engineering technology program [Paper presentation]. 2022 ASEE Annual Conference & Exposition. https://doi.org/10.18260/1-2--40640
Naqvi, S. R., Akram, T., Haider, S. A., Khan, W., Kamran, M., Muhammad, N., & Nawaz Qadri, N. (2019). Learning outcomes and assessment methodology: Case study of an undergraduate engineering project. The International Journal of Electrical Engineering & Education, 56(2), 140–162. https://doi.org/10.1177/0020720918790108
Nurjannah, I., Cholik, M., Muhaji, Wiyanto, T., Wijanarko, D. V., & Arsana, I. M. (2022). Development of OBE-based learning evaluation model in mechanical engineering education program. Advances in Engineering Research, 209, International Joint Conference on Science and Engineering 2021 (IJCSE 2021), 7(12), 2352–5401. Atlantis Press. https://www.atlantis-press.com/proceedings/ijcse-21/125966512
Passow, H. J., & Passow, C. H. (2017). What competencies should undergraduate engineering programs emphasize? A systematic review. Journal of Engineering Education, 106(3), 475–526. https://doi.org/10.1002/jee.20171
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Blackwell Publishing. https://doi.org/10.1002/9780470754887
Premalatha, K. (2019). Course and program outcomes assessment methods in outcome-based education: A review. Journal of Education, 199(3). https://doi.org/10.1177/0022057419854351
Raihan, M. Z., & Azad, M. A. K. (2023). A bibliometric review on outcome-based learning for graduate employability: Mapping the research front. Journal of Education, 203(1), 73–91. https://doi.org/10.1177/00220574211016444
Rajak, A., Shrivastava, A. K., & Tripathi, A. K. (2019). An approach to evaluate program outcomes and program educational objectives through direct and indirect assessment tools. International Journal of Emerging Technologies in Learning (iJET), 14(23), 110–123. https://doi.org/10.3991/ijet.v14i23.11018
Rashideh, W., Alshathry, O. A., Atawneh, S., Al Bazar, H., & AbualRub, M. S. (2020). A successful framework for the ABET accreditation of an information system program. Intelligent Automation & Soft Computing, 26(6), 1286–1306. https://doi.org/10.32604/iasc.2020.012995
Sala, A. L., & Riddell, J. (2012). Successful ABET accreditation of a two-year electronics technology program: Lessons learned [Paper presentation]. Proceedings of the ASEE Annual Conference & Exposition. https://doi.org/10.18260/1-2--21965
Shafi, A., Saeed, S., Bamarouf, Y. A., Iqbal, S. Z., Min-Allah, N., & Alqahtani, M. A. (2019). Student outcomes assessment methodology for ABET accreditation: A case study of computer science and computer information systems programs. IEEE Access, 7, 13653–13667. https://doi.org/10.1109/ACCESS.2019.2894066
Sloan, J. A., & Frank, T. E. (2023). Student self-assessment data: An instrument for program assessment. Intersection: A Journal at the Intersection of Assessment and Learning, 4(2). https://doi.org/10.61669/001c.84511
Sundararajan, S. (2014). A strategy for sustainable student outcomes assessment for a mechanical engineering program that maximizes faculty engagement. Proceedings of the ASEE Annual Conference & Exposition. https://doi.org/10.18260/1-2--20002
Syeed, M. M., Shihavuddin, A. S. M., Uddin, M. F., Hasan, M., & Khan, R. H. (2022). Outcome-based education (OBE): Defining the process and practice for engineering education. IEEE Access, 10, 119170–119192. https://doi.org/10.1109/ACCESS.2022.3219477
Thiruvengadam, J., Baskar, S., Jeyamala, C., & Abirami, A. M. (2022). Systematic approach in assessment of course outcomes/program outcomes for undergraduate engineering programs – A case study. Journal of Engineering Education Transformations, 35, 270–275. https://doi.org/10.16920/jeet/2022/v35is1/22039
Tshai, K. Y., Ho, J.-H., Yap, E. H., & Ng, H. K. (2014). Outcome-based education – The assessment of programme educational objectives for an engineering undergraduate degree. Engineering Education, 9(1), 1–10. https://doi.org/10.11120/ened.2014.00020
Waly, M. I. (2020). A modified evaluation approach for direct assessment of program outcomes: Medical equipment technology program as case study. International Journal of Emerging Technologies in Learning (iJET), 15(4), 125–137. https://doi.org/10.3991/ijet.v15i04.11792
Zain, S., Badaruzzaman, W. H. W., Rahmat, R. A. O., Jaafar, O., Basri, N. E. A., & Basri, H. (2012). Learning outcomes for environmental and sustainable development component in the field of civil engineering. Asian Social Science, 8(16), 153–162. https://doi.org/10.5539/ass.v8n16p153
Refbacks
- There are currently no refbacks.
e-ISSN: 1694-2116
p-ISSN: 1694-2493