Written By: Ms. Nodumo Dhlamini – Director ICT Services, Communications and Knowledge Management at AAU

Background

The ability of any institution to use data is critical because when data is well managed, it can be used to support decision making through the mapping and understanding of emerging data trends. Well managed data facilitate key strategic initiatives and actions that help improve business performance and relationships with stakeholders.

African Universities collect great amounts of data on students, staff, curricula, research, partnerships, and other areas – but more still needs to be done for African Universities to benefit from the data that they collect. Issues hindering them from maximizing the benefits from their data, include weak management information systems, limited data skills, gaps in the data collected, inadequate human resources to manage data functions, and limited institutional policies to promote data prioritization.

In addition, this century is characterized by the prevalence of new data sources that is sometimes referred to as Big Data. This new data emanates from the social media, various online systems, and many ‘silent’ data collection processes. In line with these developments, it is very important that African Universities sift through such data and turn useful and important new data into a resource that informs curricula, student services, partnerships, resource mobilization, among several other important functional areas of universities.

Benchmarking Initiatives for African Universities

There are two major benchmarking initiatives that were initiated for African Universities – the PASET (Partnership for Applied Skills in Engineering and Technology) Regional Benchmarking Initiative developed by the World Bank and the AQRM (African Quality Rating Mechanism) developed by the African Union Commission. Both benchmarking initiatives aim to strengthen the capacities of African Universities to self-study, self-analyze and self-assess using set standards and ultimately use these results to improve their processes and performances. Both the PASET Regional Benchmarking Initiative and the AQRM have a set of indicators that universities use to self-report on their status in relation to these indicators.

The PASET Regional Benchmarking methodology is based on five institutional performance and eight institutional health indicators. The five institutional performance indicators concentrate on Inclusion and Equity; Learning Achievement; Labor Market Outcomes; Research Results; and Technology Transfer Results. The eight institutional health indicators focus on Inclusion and Equity; Quality of Teaching and Learning; Relevance; Internationalization; Research; Community Service & Technology Transfer; Governance & Management; and Financing.

The AQRM benchmarking methodology has three parts, namely institutional general information, institutional level self-rating and programme level self-rating. The institutional general information indicators include Institutional Profile; Student Profile; Facilities; Faculty / Staff Profile; Governance & Management; Teaching & Learning; Linkage with the Industry Sector; Research & Community Outreach; Internationalization; and Rating of Best Three Departments /Subject Areas. The self-rating indicators at institutional level concentrate on Governance & Management; Infrastructure; Finance; Teaching & Learning; Research, Publication & Innovation; and Community/Societal Engagement. The self-rating indicators at programme level focus on Programme Planning & Management; Curriculum Development; Teaching & Learning; Assessment; and Programme Results.

PASET Regional Benchmarking Initiative

At the just ended 7^th ACE Impact Regional Workshop that was hosted in Benin, Dr Jingwen Mu from the University of Auckland, New Zealand, spoke on the progress that the ACE Impact Host Universities have made towards achieving the goals of the Disbursement Linked Result 7.4 (DLR7.4) which encourages the Host Universities to participate in the PASET Regional Benchmarking initiative. DLR7.4 is one of the sub- indicators that is used by the ACE Impact Project to measure the impact of the project on the overall universities’ functions, that is, how the project is impacting the processes at the universities’ institutional levels.

The first PASET benchmarking pilot in 2014 involved seven universities, the second round in 2016 involved 31 universities, and the third round in 2018 involved 26 universities. These earlier benchmarking activities revealed that the participating universities were challenged with reporting and finding the data that was required as part of self-benchmarking. For example, during the 2018 and 2021 PASET Benchmarking exercises, the participating universities said that they needed support in the areas of institutional research, quality assurance, MIS (management information systems) capacity development and conducting benchmarking and graduate tracer studies. These declared needs by the universities proved that they were facing difficulties in meeting the full benchmarking data requirements of the PASET self-benchmarking methodology. MIS support was identified by participating institutions as a top critical area where support was most needed.

‘As part of revising the PASET self-benchmarking methodology, the project team decided to initially assess the data systems maturity of ACE Impact Host Universities – instead of requiring that they respond to the set self-benchmarking questions’, said Dr Jingwen Mu. ‘It was within this context that DLR 7.4 proposed a two-phase approach, in which Phase 1 asked ACE Impact host institutions to complete a self-assessment of their institutional data systems’, added Dr Jingwen Mu.

Data System Maturity Survey involving 36 ACE Impact Host Universities

It was reported that a free open source data maturity framework called DataOrchard was chosen and adapted to assess the “data maturity journeys” of all the ACE Impact Host Universities. The data maturity assessment aimed to inspire the universities to work “towards improvement and increased capability in using data”.  This Framework, created by Data Orchard CIC, has been developed specifically for the non-for-profit sector and has seen its adoption by higher education institutions. It was mentioned that the adapted version of the Framework for ACE Impact institutions is a simplified version that encourages participants to focus on three key themes of a healthy data system – People, Technology, and Culture. The host institutions’ survey responses were aggregated and rated at both the ‘theme-level’ and the ‘overall level’ and classified into one of the five stages of data system maturity: Unaware, Emerging, Learning, Developing, and Mastering.

Dr. Jingwen Mu reported that 36 ACE Impact host institutions had participated in the Data System Maturity Assessment as part of preparing them for the PASET self-benchmarking. The findings from the data system maturity assessment were that the participating universities fell into three categories – 14 host universities were found to be in the Learning Group in terms of their data systems maturity; 19 host universities were placed in the Developing Group and 3 host universities were placed in the Mastering Group.

Three Host Universities graded in the mastering group were said to be most advanced in the way that they collect, manage, and use their data. Mastering Universities are very comfortable using data to ask difficult and complex questions, explained Dr. Mu.

Nineteen Host Universities in the developing group were reported to be starting to use and share data at institutional levels.  People from various teams and levels of seniority in these institutions regularly discuss data and how to use it, expounded Dr. Mu.

Fourteen Host Universities in the learning group were said to be characterized by the fact that data was beginning to be recognized as important at more senior levels, however access to data was still challenged.

Dr. Jingwen Mu said that even those universities that were graded to be masters, also have much more to learn towards improving their data management systems. Dr. Jingwen Mu also cautioned that the results of the ACE Impact Data System Maturity Assessment needed to be interpreted in the context of the adapted version of the data maturity framework and in the context of the PASET DLR 7.4 objectives.

As a follow up to the Data System Maturity Assessment, the ACE Impact host universities were given a personalized report that indicated areas of strengths and weaknesses. Currently these institutions are developing institutional intervention plans as their responses to the outcomes of the Data System Maturity Assessment, said Dr Jingwen Mu.

During the question-and-answer session, the participants asked for advice on how to handle resistance by team members to new ways of handling data. Dr. Jingwen Mu said that in the context of Africa, it was important to demonstrate what faculty currently do and what they can do with data or how they can improve their processes because of data. She advised that staff needed to be educated about the importance of data in helping universities better engage students and stakeholders. Such efforts in demonstrating the usefulness of data would reduce their resistance, advised Dr. Jingwen Mu.

Key Lessons for other African Universities

1. Data capability is very important for the sustainability and relevance of African Higher Education Institutions (HEIs); thus, it is important that HEIs strengthen their data skills, transform their data cultures and reinforce their data management technologies.
2. Assessing the data systems maturity of African Universities might be an important starting point as opposed to insisting that the universities move straight into self-benchmarking. The value of the data systems maturity assessment is that it prepares the universities to provide the required data during self-benchmarking.
3. There is an opportunity for the two benchmarking methodologies (PASET and AQRM) to integrate the good aspects of the other – and for the universities that have used these methodologies to share experiences.
4. The adapted data systems maturity assessment could be deployed across all interested universities in Africa – and this would assist the universities to advance their journeys towards data systems maturity.