Shahid Mahmood  ( Department of Epidemiology, Institute of Public Health, Lahore. )
Muhammad Ayub  ( Department of Public Health Practice, Institute of Public Health, Lahore. )

Objective: To determine the accuracy of monthly reports submitted by lady health workers of National program for family planning and primary health care in district Lahore.
Methods: A cross sectional descriptive study was conducted in district Lahore from November 2008 to February 2009. Forty lady health workers were selected using stratified random technique. Four monthly reports (one from each quarter of year 2008) submitted in health facilities by each lady health worker were retrieved from respective health facilities. These reports were then audited for accuracy using registers in health houses and verification of primary health care information in their respective communities. A scoring system was devised to grade reports for accuracy as good (80% and above score), satisfactory (60-70% score) and unsatisfactory (less than 60% score).
Results: Out of 40 lady health workers (LHWs) interviewed, 32(80%) had good knowledge, 6(15%) had satisfactory knowledge, while 2(5%) had unsatisfactory knowledge regarding data recording and reporting tools. Based on our scoring system, only 47.5 % of reports were found accurate. In addition, 35% of reports contained missed entries, misreported data and factitious information.
Conclusion: There are significant gaps in recording and reporting of primary health care data in National programme for family planning and primary health care in district Lahore. Considering the utility of this data for health planning at district, provincial and national levels, adequate supervision and regular auditing should be carried out at health facility and district levels before onward transmission primary health care data to provincial and national levels (JPMA 60:649; 2010).

Information is the basic tool of management and a key input for progress of any society. A well designed information system is almost a necessity and priority for planning and management of health services. Public health informatics is \'the application of information science and technology to public health practice and research.\' The assessment of health information system is essentially a measurement of the performance of selected components or subsystems of given health care system. Health information systems in most countries are inadequate in providing the needed management support.¹ Most health systems in developing countries equate information systems with filling endless registers and sending out reports without receiving any feedback. A great majority of clinical facilities either do not submit any health report or no standardized format exists. This in turn creates problems for data compilation and analysis, resulting in issues of drawing inference for managerial decision making.² Furthermore, data received from many health facilities are incomplete, inaccurate and unrelated to priority tasks and function of local health personnel.³
Accuracy of data refers to the degree to which the data or statistics measures what was intended to be measured when the data collection system was designed.⁴ It is important to keep in mind that primary function of a health information system is to provide data that enhance decision making in provision of health services. The process of tracking down information has changed dramatically in recent years. Introduction of computer technology and construction of World Wide Web, though allows easy access to heaps of data that previously was found in dusty reports of limited circulation, however, due to paucity of resources in developing countries, these still rely on hand written forms and reports.⁵
The health system in Pakistan is consistently facing under-utilization of primary health care facilities and overloading of secondary and tertiary level facilities .The reason for this phenomenon is multidimensional. There has been irrelevant and inappropriate planning for the basic health facilities especially in rural areas. Despite the large network of Basic health units, health care system in Pakistan has failed to attract the communities to utilize whatever services are available. Against this background, National Programme (formerly Prime Minister\'s Programme) for family planning and primary health care was launched in April 1994 as part of World Health Organization\'s global strategy "Health for All by Year 2000". The objective of the programme is to address the health problems in community by providing promotive, preventive and curative services through Community based lady health workers (LHW) who establish a "health house" in her home.⁶ A separate health information system was developed to gather primary health care information from communities. This information system aims at improving the capacity of service providers and health managers, monitoring various health services rendered by lady health workers and assess trend of health events in population. Each lady health worker is supplied with a set of registers to record data concerning vital events, family planning services and essential primary health care services to 200 families in her allocated catchment area. Every month, the health workers compile data in the form of monthly reports which are submitted to attached health care facilities. District report is compiled using individual\'s health care facilities for onward transmission to provincial and national levels.⁷ The aim of this study was to assess accuracy of primary health care data collected by lady health workers in terms of its completeness and reporting quality. The results of this study will assist health managers to understand the strengths and weaknesses of existing health information system and to take remedial measures to make this system effective and efficient.

This cross sectional study was conducted in district Lahore. All health houses were stratified for this study into ten groups based on existing administrative division and availability of basic health units. Sampling frame of all those lady health workers who had completed formal training and been working in field for at least one year were compiled from district database. Four lady health workers were then selected from each stratum through simple random technique. Four monthly reports (January 2008, April 2008, July 2008 and October 2008) of each lady health worker were retrieved from respective health facilities for data auditing using recording registers in health houses and verification of information by visiting households in respective catchment community. Data collection tools comprises a semi-structured questionnaire for interviewing lady health workers regarding her insight of recording/ reporting system and a checklist for verification of reported data from lady health worker\'s recording registers. Information was also collected regarding knowledge of these community based health workers about various health information system tools provided by the programme and how to submit their monthly reports as per programme policy. Checklist was devised based on important primary health care indicators of monthly reports which included number of reported births, infant deaths, children started immunization, maternal deaths, and low births weight babies, number of family planning clients and health committee meetings. The reliability and validity of the questionnaire as well as the checklist were tested outside study area in four health houses in district Kasur. Necessary changes in questionnaire and checklist were made accordingly.
Upon approval of ethical committee of institute of public health Lahore and taking permission from concerned health authority, investigators visited sampled health houses in community accompanied by female supervisor. The purpose of survey was narrated and informed consent was taken before starting each interview. Furthermore three entries from health worker\'s registers on selected variables were randomly selected and verified in the community for assessing the accuracy of entries in the registers.
Data was compiled and analyzed by using Computer software EpiInfo version 6. A scoring system was devised to grade knowledge of health workers as good (80% and above score), satisfactory (60-70% score) and unsatisfactory (less than 60% score). Each correct response to question was given mark 1 and 0 for incorrect response. Similar scoring system was used for checklist. Finally, based on these scores, monthly reports were categorized for accuracy from unsatisfactory (less than 60% score), satisfactory (60-70% score) and good (80% and above score) reports.

All sampled health workers fulfilled the selection criteria of the National Programme for family planning and primary health care. Educational status of community health workers is an added advantage in accomplishing their tasks in community. About 85% of health workers were educated up to middle and above. These health workers were trained by the Panel of three trainers comprising medical officer, lady health visitor and dispenser for duration of fifteen (15) months. Out of forty health workers, 32(80%) had good understanding about their job description, especially concerning recording and reporting health information.
Out of 40 lady health workers (LHWs) interviewed, 32(80%) had good knowledge, 6 (15%) had satisfactory knowledge, while 2 (5%) had unsatisfactory knowledge regarding data recording and reporting tools. Majority of study participants had satisfactory knowledge about the case definitions of diarrhea, family planning and immunization, whereas knowledge about the cases of maternal deaths and low birth weight babies was unsatisfactory. Monthly reports were received in health facilities in first week of every month by trainers in 87.5% of the cases whereas in rest of health facilities (12.5%), lady health supervisor did this task against the programme policy. The entries in reports at the time of submission were verified from recording registers in only 75% of the instances. Feed back on the monthly reports from health facility staff was received regularly by 28 (70%) out of 40 health workers. It was found that out of 40 health workers 23 (57.5%) were correctly reporting about the total catchment population whereas 17 (42.5%) were either under reporting or over reporting the information. Only 12.5% of health workers were under reporting the number of births in the serving communities whereas 15% are also over-reporting the numbers of births in their catchment population. Reporting accuracy regarding infant deaths and maternal deaths is 87.5% and 92.5% respectively. Based on our scoring system, 47.5% of monthly reports were categorized as accurate and good quality reports whereas 35% of these monthly reports were found to be unsatisfactory and inaccurate. Upon verification of primary health care information from catchment population, it was found that 17.5% of workers were over reporting and 22.5% were under reporting the immunization status of children. On the other hand, only 35% of health workers were correctly recording and reporting the family planning services data including over-reporting (35% of workers) of oral contraceptive consumption in population. Similarly, maternal and infant deaths were under-reported (12.5% and 10% respectively) by these community based health workers.

The most important function of the health information system is to generate evidence for improvement of health of individuals and community. In most countries inappropriate data recording and inadequate data analysis are common problems in information systems. Makombe et al. in evaluating the quality of data aggregated by anti-retroviral clinics in Malawi reported that most sites did not document valid data for several critical data fields. For instance, 38% of these sites reported inaccurate data about tuberculosis outcome variable and there were missing entries in 75% of the reports.² In another study conducted in Nigeria, authors recommended that special attention should be given to strengthen the ability of health workers to document information accurately.⁸
A well designed health information system should contain relevant indicators which are simple to understand and are standardized in the form of data recording instruments. There should be proper flow of information at various levels to facilitate evidence based decision making combined with feedback to those who generate and transmit the data. The system should provide information about the demographic, environmental health statistics, morbidity and mortality characteristics of the population.⁹ Data collection tools in National programme have been developed on the same basis. These recording and reporting data instruments not only help the lady health workers to assess the health status of the community especially mother and child health but also provide the tools for health facility staff and health managers for evaluating the working of these community health workers. The LHW-health information system has been designed to collect the information about demographic features of the community, morbidity/mortality details, and important vital statistics in the served catchment population.
Inappropriate data collection instruments, poor recording and reporting and error in processing of data are three possible sources which may result in impediments to good quality data.¹⁰ Another factor which creates fears among health workers in compiling and reporting data is that information system could monitor employee\'s work and that repercussion will occur if the employees will not attain a particular level on a performance indicator. Barnum has reported that Nurses in Cameroon under reported the number of patients treated because of similar fears. The data on supervisory visits are also frequently misreported.¹¹ Similar findings were observed in our study since health workers are under reporting the infants and maternal deaths and they are over-reporting the family planning clients and oral contraceptive consumption.
Weaker supervisory mechanisms result in generation of inaccurate data. For instance, a study in Nepal found that data obtained from the facility registers were lower than the data reported at the district level, showing a tendency of over-reporting to the higher levels. Similarly, in a quality data audit (DQA) of forty-one low income countries, Xavier et al. demonstrated that most frequent weaknesses in most information systems are inconsistency of data recording, late reporting and lack of feedback in relation to data recording and reporting performance.¹¹ Furthermore, several other studies^2,11,12 concluded that errors in reporting were due to lack of supervision and feedback from the superior levels as well as inadequate incentives to health workers.¹³ Results of our study are consistent with these studies. For instance, our study have clearly demonstrated that though community health workers have appropriate level of knowledge about the reporting instruments, however due to lack of feedback, data recording exercise is considered as worthless task by these health workers.
Another factor which makes the data recording and reporting task of health workers much complicated is the presence of several different types of data recording registers, hand written forms as well as increased patients/clients load in busy outdoors of health units, especially in developing countries. In this scenario, pressure of service provision and at the same time recording a single event in at least two to three different places might result in illegible, inaccurate records and missing entries. In a recently published article, Kunimitsu demonstrated that in Solomon Island, mean data discrepancy (over-reporting and under-reporting) between records of clinics and government statistics was 21.2% and significant associations (coefficient 0.09; 95% CI 0.04-0.15) were observed between this data discrepancy and average number of patients as well as with illegible writings of health nurses.¹⁴ Similar observations have also been reported in a study conducted in Ghana on remote rural maternity units.¹⁵
Recently, World Health Organization has devised a standardized and validated methodology named Data Quality Audit (DQA) to assess the accuracy of health data collected at health facilities. This methodology is based on tracking data from federal level to community based health units to calculate a proportion called Verification Factor (VF). This factor is taken as a measure of data accuracy.¹¹ Methodology adopted in our study has many similarities with the WHO validated data quality audit. Many health related agencies like Global Alliance for Vaccines and Immunization (GAVI) are increasingly using DQA surveys of individual countries for their funding decisions.¹¹
Likelihood that reported health information would be used for decision making by health managers and policy makers tends to be significantly higher if this information be considered valid and reliable. The extent to which health information system of National program for family planning and primary health care will produce desired results depends upon accuracy of data collected by these community based health workers. Considering our finding that almost half of health reports submitted by community based health workers are inaccurate, one can imagine the quality of decisions taken to deliver primary health care in Pakistan.

There is significant disparity in the knowledge and practices of community health workers regarding health data recording and reporting. The reporting inaccuracy issue needs to be addressed by programme managers on priority basis in order to make sure that reliable and valid community health data is available for effective and efficient planning and implementation of primary health care delivery to the community. It is recommended that regular data auditing activities should be planned from district programme management along with strengthening proper supervision of community based health workers.

We are grateful to district coordinator National programme for family planning and primary health care Lahore and all health supervisors and lady health workers of district Lahore and Kasur for their co-operation and facilitation for this study.

1.Allotey P A, Reidpath D. Information quality in a remote rural maternity unit in Ghana. Health Policy and Planning. Oxford: Oxford University Press 2000; 15: 170-6.
2.Makombe SD, Hochgesang M, Jahn A, Tweya H, Hedt B, Chuka S, et al. Assessing the quality of data aggregated by antiretriviral treatment clinics in Malawi. Bull World Health Organ 2008; 86: 310-4.
3.Lippeveld T, Sauerborn R, Bodart C. Design and implementation of health information system .WHO Geneva 2001; 1: 143-4.
4.Bodart C, Sapirie S. Defining essential information needs and indicators. World Health Forum 1998; 19: ISSN: 0251-2432.
5.Webb P, Bain C, Pirozzo S. Essential epidemiology: an introduction for students and health professionals. Finding and using public health data. 2007; 3: 61-75.
6.Revised PC-1.Prime Minister\'s Program for Family planning and primary health care Project Digest 1995; 1: 3-5.
7.Ministry of health Islamabad .National program for family planning and primary health care training manual. 2001; 9: 305-59.
8.Ogunbekun I. A management information system for Nurses/midwives. Bull WHO. World Health Organ 1999; 77: 186-9.
9.Park K. Text book of preventive and social Medicine. Health information and basic medical statistics 26th ed. India: Bhanat Publisher, 2007; 18: 638-9.
10.Andrew F, Patrick W, O \' Caroll. Last-Maxcy-Rosenau Textbook of public health and preventive medicine. 5th ed. Springer, 1998; 5 : 59-6.
11.Burnum J F .The misinformation era: the fall of the medical record. Ann Inteen Med 1989; 110: 482-4.
12.Bosch-Capblanch X, Ranveaux O, Doyle V, Remedios V, Bchir A. Accuracy and quality of immunization information systems in forty-one low income countries. Trop Med Int Health 2009; 14: 2-10.
13.Mavimbe JC, Braa J, Bjune G. Assessing immunization data quality from routine reports in Mozambique. BMC Public Health 2005; 5: 108.
14.Kunimitsu A. Accuracy of clinical malaria case reporting at primary health care facilities in Honiara, Solomon Island. Malar J 2009; 8: 80.
15.Reidpath D. Information quality in a remote rural maternity unit in Ghana. Health Policy and Planning. Oxford: Oxford University Press 2000; 15: 180-2.