Data Led Governance and Policy Making

Objectives

Evidence based policy making should be made integral to the overall governance structure in New India, 2022-23. To achieve this, timely generation and dissemination of robust data at all levels of governance would be a pre-requisite. This would require:

• Collecting data for new measurable parameters using latest technologies and improving efficiencies in processes related to existing data collection by government departments and agencies.
• Expanding warehousing facilities for storing and integrating data from different sources.
• Making data available for industry practitioners, academicians, researchers, etc . wherever feasible.
• Integrating data analysis and interactive data visualization into all policy formulation.

Current Situation

• Countries, where large-scale developmental efforts are needed, require their policy delivery mechanisms to be robust and efficient. However, paradoxically, these countries have very little data needed for the selection, implementation and evaluation of effective policies.
• Rapid advancements in technology have led to an explosive growth in the volume of data produced. Hence, public services and governance systems in India can better harness the value of this data.
• One important step taken towards creating the availability of non-sensitive data for public consumption on a common platform was the launch of National Data Sharing and Accessibility Policy (NDSAP) in 2012. This led to the creation of the Open Government Data initiative where the domain data.gov.in was registered in 2012

Essential components of a data governance policy

Let me expand on some of the key components of a comprehensive data governance policy framework:

• Data principles

High-level guiding principles establish the spirit of the governance approach. For example, principles of transparency, privacy, ethics, accountability help shape standards for responsible data use.

• Data architecture

Policies need to standardize how data is structured, formatted, integrated and catalogued across government systems. This enables interoperability and discoverability. Standards for master data also facilitate a unified view.

• Data quality

Rigorous processes and metrics should be established to assess and ensure high quality data collection. This includes guidelines for accuracy, completeness, validity, timeliness and consistency of data. 

• Metadata standards

Metadata policies specify what descriptive information must accompany datasets like source, date, purpose, access levels etc. This provides context on proper data usage.

• Data lifecycle

The policy should cover appropriate protocols for the end-to-end data pipeline - from collection, storage, security, processing, analytics, sharing, archival and destruction. Lifecycle stages may need different protocols.

• Oi5+9bData security

Detailed policies on access controls, encryption both at rest and in transit, network security, contingency planning etc. are essential to safeguard confidentiality, integrity and availability of data.

• Open data policy

Principles for proactive public release of non-sensitive government data. Guidelines on preparing data, platforms, stakeholder engagement and updating frequency help maximize impact.

Issues with Data Governance

• Data quality

Low quality data with errors, inconsistencies and lack of context reduces trust and reliability of insights derived from analysis. Garbage in, garbage out.

• Data silos

Data trapped in organizational silos prevents a unified view and full utilization. Breaking down data silos is difficult. 

• Privacy and security

Governance must balance open data benefits with privacy risks and ensure cybersecurity. Data breaches undermine trust.

• Legacy technology

Outdated IT systems constrain ability to capture, store, process and analyze data effectively. Modernization is expensive.

• Lack of skills

Public servants often lack data literacy and specialized skills like data science and analytics to maximize data value.

• Resistance to change

Institutional inertia and lack of leadership commitment slows adoption of data-driven approaches.

• Cost vs value

Governments struggle to quantify return on investment from data initiatives. Making the business case is hard. 

• Ethics

Concerns around ethics, algorithmic bias, and responsible use of AI with government data persists.

• Information overload

Too much low value data creates noise without useful insights. Determining high value datasets is difficult.

• Data monopolies

Reliance on data and tech from a few dominant platforms raises risks like vendor lock-in, bias, privacy issues.

Overcoming these challenges requires coordinated organizational effort, leadership vision, building talent and capabilities, appropriate funding and a culture that values data-based decision making.

Constraints

The following constraints need to be overcome to enable India’s transition to a data-led governance structure:

• There is over-reliance on data collection through surveys e.g., population census comes out once in ten years, the latest National Family Health Survey-4 was released in 2015-16 after a period of 10 years but huge delays, diminishes their usefulness in policymaking.
• There is a dearth of availability of real time operational/
  administrative data.
  • One challenge in this regard is that considerable number of stakeholders are involved in enabling data collection systems that are premised on a “bottom-to-top” approach. It will be a huge challenge to get all these stakeholders on board for a streamlined data collection and reporting mechanism as envisaged for 2022-23.
• There is a problem with the usability of data that is currently generated.
  • Large volumes of data collected by different government agencies and departments are not shared, even among the departments.
  • The data shared is often not available in machine readable format or cannot be integrated with data from other sources to help develop multidimensional insights.
  • Enabling adoption of the latest technology at the grass roots level would involve substantial investment along with skill development of local functionaries.
  • Furthermore, planning will be required to integrate different technologies so that ground level data can be aggregated.
• Lastly, there is considerable lack of awareness regarding currently available data sources.

Steps Required

• Data Collection Methods:

Both administrative and survey data need to move from paper based to digitally driven operations. This would require the adoption of latest technologies that require recording in digital format, geo-tagging etc. This will address the issues related to time lags, data cleansing, etc., associated with surveys to a large extent.

Enable data sharing in real time through Application Programming Interfaces (API) between data stored across different databases and across ministries in a central location for easy access to public.

• Data Integration and Quality assurance:

Most of the administrative and survey data are generated at the state level. It is recommended that after going through the process of quality assurance, where discrepancies are removed, and formats are standardized, the data should be integrated in a state data repository.

Necessary reform of our statistics and data collection system must be undertaken for quality assurance as soon as possible to achieve quality evidence based policy making.

Some state governments like Andhra Pradesh, Gujarat, and Rajasthan have taken important steps to leverage technology for evidence-based policymaking. However, these steps need to be further streamlined and adopted by all states. This will empower the officer on the ground to take data led decisions and technology would help in informed policy making.

• Data Protection:

The issue of confidentiality has to be addressed while dealing with citizen level data. Justice Sri Krishna Committee Report submitted its recommendation in July 2018. Its recommendations are under active consideration to formulate a data protection law in India.

• Role of tertiary Big Data:

For better governance and evidence-based policymaking, it is recommended that tertiary big data collected by private third parties should be used. Overtime, the National Data Analytics Portal aims at collecting, analyzing and disseminating various types of tertiary data of different levels of granularity.

• Skill Development and Restructuring:

  • Government statistical organizations responsible for data collection and reporting needs to be updated on new technologies.
  • Data scientists with multiple skills in the areas of statistics, analytics, computer science and programming are rare in the Indian government.
  • Ministry of Statistics and Programme Implementation (MoS & PI) needs to have an adequate number of data scientists to take advantage of new technologies.
  • Re-skilling needs to be promoted across government agencies, both at the state level and at the centre.
  • A roadmap for strengthening various government agencies including MoS & PI needs to be formulated and implemented in a time bound manner.