FACTORS AFFECTING UTILISATION OF MONITORING AND EVALUATION FINDINGS IN PUBLIC HEALTH FACILITIES IN IBANDA DISTRICT, UGANDA

LIST OF ACRONYMS

AHSPR – Annual Health Sector Performance Report

B.C : Before Christ

IOCE : International Organization for Cooperation in Evaluation

AEA :American Evaluation Association

US :United States

M&E – Monitoring and Evaluation

MoH – Ministry of Health

NGO – Non Governmental Organisation

NPM – New Public Management

RBM – Results Based Management

CHAPTER ONE

INTRODUCTION

1.1 Introduction

The study will examine the determinants of utilization of monitoring and evaluation findings within public health facilities at level III and IV in Ibanda District. For this study technical, financial, and quality factors will be the independent variables, while utilization of monitoring and evaluation findings will be the dependent variable. The chapter will focus on the background of the study, statement of the problem, purpose of study, specific objectives of the study, research questions, research hypotheses, conceptual framework, and scope of the study, significance, justification and operational definitions of key terms in the study.

1.2 Background to the Study

1.2.1 Historical Background

Evaluation is as old as the world itself and has moved side by side the journey of human civilization (Basheka, 2016). The practices of the field of evaluation can be traced as far back as the biblical times during the known creation story extensively addressed in Genesis (1:31) where the biblical account informs us that on the fifth day of creation God saw everything he had created and everything appeared good. From the philosophical works of Socrates, Plato and Aristotle to the mathematical methodologies of Pythagoras and Euclid, the ideas of the ancient Greeks shaped many institutions and contributions to many fields including evaluation (Zanakis, Theofanides, Kontaratos and Tassios, 2003). Existing scholarly accounts inform us that the Delphic oracle of the ninth to the third centuries BC was the first central intelligence database of the ancient world, an interdisciplinary think tank of approximately 90 priests, deemed the best educated experts of antiquity (Theofanides et al., 2003). They collected and evaluated information and advised ordinary people and leaders, among them Alexander the Great. Major project management in the fourth century BC existed where evaluation and monitoring were key. Griffin (2005) notes that: “The practice of management can be traced back thousands of years.  These great pyramids, which were built in 2900B.C., are a classic example of management and co-ordination. Griffin (2005) notes that: The practice of management can be traced back thousands of years. The Egyptians used the management functions of planning, organizing and controlling when they constructed the pyramids. These great pyramids, which were built in 2900B.C, are a classic example of management and co-ordination.

1.2.2 Theoretical Background

The study will be based on the General Systems Theory which was developed by Bertalanffy (1934), as cited in Tama (1987). The theory provides an analytical framework which can be used to explain the factors affecting the utilization of evaluation data. According to Bertalanffy (1968), a system is an assemblage of things connected or interrelated so as to form a complex unity: a whole composed of parts and sub-parts in orderly arrangement according to some scheme or plan. The following are the features of a system. A system is basically a combination of parts, sub-parts, sub-systems. Each part may have various sub-parts. A system has mutually dependent parts, each of which may include many sub-systems. Parts and sub-parts of a system are mutually related to each other, some more, some less; some directly, some indirectly.

1.2.3 Conceptual Background

Technical capacity refers to the extent to which the human resources in an organization are able to manage evaluations (Byamugisha, 2016). For this study, technical capacity will be measured in terms of whether staff are qualified, experienced and knowledgeable about using M&E systems.

Financial Capacity refers to the extent to which funds are available to finance and facilitate the monitoring and evaluation function in an organization (USAID, 2015).

1.2.4 Contextual Background

National efforts have been directed towards providing a basis for performance improvement as provided for in the National Development Plan (National Development Plan, 2010/11-2014/15) that the utilization of Monitoring and Evaluation findings have been valued to improve the performance of the health sector Such efforts have also been characterized by the establishment of an M&E function to support this role enshrined in the M & E national policy (National M&E Policy, 2013). Unfortunately, much as national efforts have been directed towards enhancing M&E capacity as well as ensuring that sound evidence-based data and information are available to inform decision making (The Republic of Uganda, National Integrated Monitoring and Evaluation Strategy, 2006), the use of M&E results remain questionable, according to a 2009 published report, there is one doctor for every 7,272 Ugandans. The related statistic is 1:36,810 for nurse/midwifery professionals. The shortages are worse in rural areas where 80 percent of the population resides, as 70 percent of all doctors are practicing in urban areas. There are 61 institutions that train health workers, with five medical colleges, twenty-seven allied health training schools, and twenty-nine nursing schools, which indicates that Uganda’s Health sector is at risk of total collapse (Ogwal et al., 2020).  This study will therefore attempt to examine the factors affecting utilization of M&E findings in Uganda’s health sector using a case study of Ibanda District.

1.3 Statement of the Problem

Findings from the M&E function ought to be used to inform decision making, planning and organizational learning (Basheka, 2016). However, at the public health facilities at level III and above in Ibanda, health information is not adequately used (Ibanda District, 2017). In order to increase use of health information, the Ministry of Health has devoted significant human, technical and financial resources to the Monitoring and Evaluation function. Despite the significant investment in M&E, there is widespread concern that the utilization of evaluation findings in public health facilities in the district is low. According to the Annual Health Sector report (AHSPR) 2018/2019, Ibanda District ranked 18^th in the whole country out of 128 districts, and in the AHSPR 2019/2020, it ranked 76^th. The AHSPR tracks performance of public health facilities in terms of coverage of health services, quality of health services, Human Resource for health services, as well as reporting for health services.

The current study thus seeks to determine the factors affecting utilization of M&E findings within the public health facilities in Ibanda District. Without this information, utilization of evaluation findings may deteriorate further and affect the performance of the public health facilities.

1.4 Main objective

The study seeks to establish the factors affecting the utilization of M&E findings in the level III and IV public health facilities in Ibanda District.

1.5 Specific objectives

1. To establish the relationship between technical capacity on the utilization of M&E findings in the public level III and IV health facilities in Ibanda District;
2. To assess the relationship between financial capacity on the utilization of M&E findings in the public level III and IV health facilities in Ibanda District;

• To examine the relationship between the quality of M&E systems on the utilization of M&E findings in the public level III and IV health facilities in Ibanda District.

1.6 Study hypotheses

H₁. There is no relationship between technical capacity and the utilization of M&E findings in public health facilities

H_2: There is no relationship between financial capacity and utilization of M& E findings in public Health facilities.

H₃ : The is no significant effect of the Quality of M&E  findings on the utilization of M&E findings in public Health facilities.

1.7 Conceptual Framework

Independent Variables (Factors)           Dependent Variable (Utilization of M&E Findings)

Source: Adopted from Kasule (2016) and modified by the researcher

The conceptual framework above shows the factors affecting the utilization of M&E findings. The model assumes that utilization of M&E findings is enhanced by improved technical, financial and quality of M&E systems within organizations.

1.9 Significance of the Study

The study will provide information to the future scholars on the relationship between technical capacity on the utilization of M&E findings in the public level III and IV health facilities in Ibanda District and this will enable them in drawing conclusions with the present situation and other times.

1.10 Justification of the Study

It is critical that the factors that affect the utilization of M&E findings are thoroughly examined and understood by the public sector implementing Monitoring and Evaluation system. Without clear understanding of these factors, the public health sector may continue underutilizing evaluation findings. This may affect the performance of the sector and subsequent underperformance of the public health sector.

1.11 Scope of the Study

1.11.1 Content Scope

The study will examine the factors affecting the utilization of M&E findings. The study will specifically focus on the effects of technical, technological and quality factors on the utilization of M&E findings in Ibanda District.

1.11.2 Geographical scope

The study will be conducted in Ibanda District which lies in South Western Uganda. It will be conducted at public Health centers IIIs and Health Centre IVs within the district.

1.11.3 Time scope

The study will look at quarterly HIV data/reports for the financial year 2015/2016 and how it was used to inform the clinic processes or facilitated good quality data, monitoring and evaluation, or planning at the level III health centres and the general hospital.  In addition, the field study will be conducted between September and October 2017.

1.12. Definition of Key Terms and Concepts

Technological capacity refers to the capacity of the human resources/people to use M&E findings.

Technological capacity refers to the ability of the organizations to utilize technology to manage M&E activities

Quality of M&E Findings refers to the extent to which the evaluation findings meet the specified requirements and standards

Utilization of Monitoring and Evaluation Findings. This will refer to the application of Monitoring and Evaluation findings in decision-making, quality improvement and learning.

CHAPTER TWO

LITERATURE REVIEW

2.1 Introduction

This chapter presents a review of literature on the topic under investigation. The chapter presents a review of the relevant theories. It also presents empirical literature on the factors affecting the utilization of M&E findings.

2.3 Conceptual Review

This section presents the literature review as reviewed by various scholars in line with the study objectives and conceptual frame work.

2.3.1 Technical capacity

Adequate personnel

Building an adequate supply of human resource capacity is critical for the sustainability of the M&E system and generally is an ongoing issue. Furthermore, it needs to be recognized that “growing” evaluators requires far more technically oriented M&E training and development than can usually be obtained with one or two workshops. Both formal training and on-the-job experience are important in developing evaluators with various options for training and development opportunities which include: the public sector, the private sector, universities, professional associations, job assignment, and mentoring programs (Acevedo et al., 2010:12).

2.3.2 Financial capacity

Availability of funds

Financial Capacity refers to the extent to which funds are available to finance and facilitate the monitoring and evaluation function in an organization (USAID, 2015). In this study therefore financial capacity will be measured in terms of whether there are adequate financial resources, whether the funds are released in a timely manner and whether the funds are accounted for.

Timely funds

The study by Koffi-Tessio (2002), on Efficacy and Efficiency of Monitoring-Evaluation Systems (MES) for Projects Financed by the Bank Group that was done in Burkina Faso, Mauritana, Kenya, Rwanda and Mozambique, through desk review and interviews, for projects approved between 1987 and 2000. Monitoring-Evaluation systems are not meeting their obligatory requirements because of financial constraints.

Accountability

A study conducted by Gamba (2016) to determine the factors affecting the utilization of evaluation findings in malaria control projects in Uganda found that management support was low and financial resources allocated for M&E was insufficient. This greatly affected effective outcome and impact monitoring and evaluation of projects.

2.3.3 Quality of Evaluation Findings

Timeliness

Quality of M&E Findings refers to the extent to which the M&E systems and activities meet the specified requirements and standards (Mulandi, 2013:12). For this study, the qualities of M&E findings meet the standards and requirements.

Methodologically correct

According to Cornielje, Velema and Finkenflugel (2008:15), only when the monitoring system is owned by the users the system is it likely to generate valid and reliable information. However, all too often the very same users may be overwhelmed by the amount of daily work which in their view is seen as more important than collecting data and subsequently the system may become corrupted.

Relevant

In a study report of an Australian NGO conducted by Spooner and Dermott (2008:45), staff reported that, as WAYS evolved over time, they were unsure about what works in the current system of monitoring and evaluation. Additionally, resources had not been dedicated to data analysis; and the data was rarely analyzed.

Well written

Obure (2008:5) in a study of RBM in Northern Ghana indicates a problem associated with post collection data management. As confessed by many field officers, the storage, processing and interpretation of data was ineffectively handled. Results from the study strongly point to a weakness in the system arising from the inability of stakeholders to handle and process data in a meaningful way. He concludes that this challenge could seriously lead to mere collection of large volumes of data which eventually might not be used in a helpful way. Data must be collected and analyzed regularly on the objectives and intermediate results.

2.3.4 Utilization of M&E findings

Utilization of Monitoring and Evaluation findings refers to putting monitoring and evaluation results to use. The use of monitoring and evaluation findings for decision making and project control ensure that there is a baseline against which to undertake new measurements (Mulandi, 2013)

Number of Decisions made

According to Gebremedhin, Getachew and Amha, (2010) establishes that the source of performance data is important to the credibility of reported results and thus their utilization in future programmes implementation. The author thus notes that it is important to incorporate data from a variety of sources if the results are to be validated. The processes of planning, monitoring and evaluation make up the Result-Based Management (RBM) approach, which is intended to aid decision-making towards explicit goals.

Learning

M&E systems will only add value to project implementation through interpretation and analysis, by drawing on information from other sources and adapting it for use by project decision makers and a range of key partners. Knowledge generated by the M&E efforts should never stop at basic capturing of information or relying exclusively on quantitative indicators, but also to address the “why” questions. Here the importance of more qualitative and participatory approaches become particularly important, to analyze relationship between project activities and results.  Evaluation therefore serves the purpose to establish attribution and causality, and forms a basis for accountability and learning by staff, management and clients. The challenge early on during project implementation is to design effective learning systems that can underpin management behavior towards results, and come up with strategies to optimize impact.

Project/program improvement

Bourckaert,  Verhoest and De Corte (2009) observe that indicators for measuring programme performance are difficult to identify unless the M&E results are produced on time. It is therefore important to clearly define an appropriate system of indicators to measure and monitor programme performance with time. In support, a study by Cunnen,  (2006)  finds  that  a  system  of  over  two  thousand  societal  indicators  to measure Results for Canadians across all sectors need to be timely.

CHAPTER THREE

METHODOLOGY

3.1 Introduction

This chapter presents the methodology that will be adopted during the study. It describes and discusses; the research design, sample size and selection, the data collection methods used and their corresponding data collection instruments, data management and analysis procedure as well as steps that will be taken to ensure validity and reliability during the study and measurement of variables.

3.2 Research Design

The study will adopt a cross sectional design. Explanatory research helps a researcher to analyze patterns, formulating hypotheses that can guide future endeavors.

3.3 Study Population

Sekaran (2018) defines a population as the entire group of people, events or things that a researcher wishes to investigate. According to the Annual Health Sector Performance report (AHSPR) FY19/20, only 49% of the approved posts at public health facilities have been filled with qualified personnel. 49% accounts for approximately 90 health workers

3.4 Study Sample

Mugenda and Mugenda (2003), argue that it is impossible to study the whole targeted population and therefore the researcher shall take a sample of the population. A sample is a subset of the population that comprises members selected from the population. Using Krejcie and Morgan’s (1970) table for sample size determination approach, a sample size of 73 employees was selected from the total population of 90 employees of Ibanda District.

3.5 Sampling Techniques and Procedure

A number of sampling techniques will be used to select respondents to the study namely; simple random and purposive sampling techniques. The lower level Staff will be selected using simple random sampling technique. Simple random sampling will be used because it ensures generalizability of findings and minimizes bias (Sekaran, 2003).

3.6 Data collection Methods

The section presents data collection methods which include survey method, interview and documentary review. The following data collections methods have been chosen because of their numerous advantages.

3.6.1 Survey Method

The study will use the questionnaire method to collect data. The questionnaire will be used because it allows for the collection of data from a big group of respondents in a short period as suggested by Mugenda and Mugenda (1999: 107). The questionnaire will also be used because it allows busy respondents fill it at their convenient time. It also allows respondents express their views and opinions without fear of being victimized (Oso & Onen, 2008:18).

3.6.2 Interview Method

The study will employ interview method. Interviews in this study will help the researcher obtain more information on the topic under investigation. Interviews will be used because they fetch a variety of ideas needed for the study and gives a deeper understanding of the topic.

3.6.3 Document Review Method

The researcher will review documents in order to obtain recorded information that is related to the issue under investigation. This method will be used because it enables the researcher access data at his convenient time, obtain data that are thoughtful in that the informants have given attention in obtaining them and enables the researcher obtain data in the language of the respondent (Oso & Onen, 2008: 45).

3.7 Data Collection Instruments

The instruments used in this study will be questionnaire, interview guide and document review checklist.

3.7.1 Self-Administered Questionnaire

The study will employ a questionnaire as a tool of data collection. The questionnaire for staff will have 5 sections.

3.7.2 Interview Guide

An unstructured interview will be used as a tool for collecting in depth information from the key informants. The guide will have list of topical issues and questions which will be explored in the course of conducting the interviews. The guide will be drawn with the questions soliciting for the perception of the key informants regarding the factors affecting the utilization of M&E findings within the public health facilities.

3.8 Validity and Reliability of the Research Instruments

3.8.1 Validity

Validity is defined as the extent to which results can be accurately interpreted and generalized to other populations (Oso & Onen, 2008).

3.8.2 Reliability

According to Mugenda and Mugenda, (2003) reliability is the measure of the extent to which research instruments are able to provide the same results upon being tested repeatedly.

3.9 Data Collection Procedure

The researcher will obtain a letter of introduction from university which will be presented to the authorities at in the public health units in Ibanda District and after that she will obtain a list of all the staff in the organization.

REFERENCES

Acevedo, G. L., Krause, P., & Mackay, K. (Eds.). (2012). Building better policies: the nuts and bolts of monitoring and evaluation systems, Washington DC, World Bank.

Amin, M. E. (2005). Social science research: Conception methodology and analysis. Kampala: Makerere University Printery.

Amkeni Wakenya. (2009). Strengthening the capacity of Kenyan civil society to participate more effectively in democratic governance reforms and in deepening democracy in Kenya. Amkeni Wakenya Annual progress report

Annual Health Sector Performance report 2019/2020 page 160 available at https://www.health.go.ug

Anywar, G., Kakudidi, E., Byamukama, R., Mukonzo, J., Schubert, A., & Oryem-Origa, H. (2020). Indigenous traditional knowledge of medicinal plants used by herbalists in treating opportunistic infections among people living with HIV/AIDS in Uganda. Journal of ethnopharmacology, 246, 112205.

Gamba, P (2016). Factors Affecting Utilization of Monitoring and Evaluation Findings in Implementation Of Malaria Control Programmes In Mukono District, Uganda. Unpublished Master’s Thesis, Uganda Technology and Management University.

Gebremedhin, B., Getachew, A. & Amha, R. (2010). Results based monitoring and evaluation for Organizations working in agricultural development: A guide for practitioners.

Gorgens, M.,& Kusek, J.Z. (2010). Making Monitoring and Evaluation Systems Work: A Capacity Development Toolkit. Washington D.C, World Bank

IFAD, (2002). A guide for project M & E, “Managing for impact in Rural Development” Rome, Italy. International Livestock Research Institute, Nairobi, Kenya

Krejcie,R. and Morgan,D. (1970). Determining Sample Size for Research activities. Education and psychological Measurement. 30, 607-610.

Kusek, J. Z., & Rist C. R. (2004). Ten steps to a Results-based Monitoring and Evaluation System. Washington DC, World Bank.

Mibey, H. K. (2011). Factors affecting Implementation of Monitoring and Evaluation Programs in Kazi kwa Vijana Project by government ministries in Kakamega Central District, Kenya (Unpublished master‟s thesis). University of Nairobi, Kenya.

Mugenda, O.M & Mugenda, A.G (1999).Research methods; Qualitative and quantitative approaches.ACTS publishers, Nairobi Kenya.

Duchoslav, J., & Cecchi, F. (2019). Do incentives matter when working for god? The impact of performance-based financing on faith-based healthcare in Uganda. World Development, 113, 309-319.

Obure, J. Otieno. (2008). Participatory monitoring and evaluation: A meta-analysis of anti povertyinterventions in Northern Ghana. (Unpublished master‟s thesis). University of Amsterdam.

Spooner, C. & S. McDermott (2008), Monitoring and evaluation framework for Waverley Action for Youth Service, Social Policy Research Centre Report, University of New South Wales

Turabi, A.E, Hallworth, M., T. & Grant, J. (2011). A novel performance monitoring framework for health systems; experiences of the National Institute for Health Research. England.

Zogo, N. Y. E. (2015). The State of Monitoring and Evaluation of NGOs’ Projects in Africa. Hill & Knowlton Strategies Regional Office of Eastern Africa.