Delivering Evidence

Based Impact

Delivering Evidence

Based Impact

Integra’s experts use state-of-the-art analytical tools and methods to deliver the evidence based impact and results our clients expect – specific to each practice area.

Inclusive Growth Diagnostics (IGDs)

Integra conducts IDGs to assess and identify the most critical constraints to inclusive, broad-based economic growth on the Journey to Self-Reliance. Integra applies two approaches to our IDG efforts to form a cohesive picture of the constraints across key development sectors:

Disaggregated Growth Diagnostics – applies the “standard” HRV methodology, identifies binding constraints a country is facing and provides a framework for determining which will have a large impact and provide the most economic growth benefit.
Expanded Growth Diagnostic – expands the HRV to also consider possible employment-related and informal production binding constraints.

Our work is guided by the following principles, which we believe reflect key lessons learned from implementing growth diagnostics in a variety of countries:

Country context is key: We start a growth diagnostic with a deep understanding of the country, its economy, and its prior development path.
Don’t re-invent the wheel: We find and examine the useful data and analysis that may already exist.
Focus on what is important: We concentrate the bulk of our analysis on the most plausable binding constraints, while distinguishing between more and less binding constraints.
Keep it simple: We employ focused and concise growth diagnostics to be timely, to be read, and to have an impact. Not every bit of data adds value.
Use judgment: When our analysis yields conflicting results, we apply good economic judgment to select appropriate comparator countries or to reach sound conclusions.

FEATURED PROJECTS

April 12, 2020

Armenia IGD

December 12, 2019

Tanzania IGD

July 12, 2019

Kyrgyzstan Inclusive Growth Diagnostic (IGD)

Institutional Architecture Assessment

Integra provides a framework for mapping the institutional landscape, analyzing possible systemic constraints, and highlighting areas to improve the capacity and performance of the institutions involved in the policy change process. We examine six distinct and interrelated components:

Guiding Policy Framework.
Policy Development and Coordination.
Inclusivity, Stakeholder Consultation and Women’s Empowerment.
Evidence-Based Analysis.
Policy Implementation.
Mutual Accountability.

FEATURED PROJECTS

Applied Political Economy Assessments (PEA)

Integra uses USAID’s PEA methodology to assist in the design and implementation of development projects, to understand how political, economic, social, and cultural drivers affect development outcomes. These assessments help explain both the reasons why things work as they do and the competing incentives and constraints that drive behaviors. The ultimate goal is to find locally appropriate and sustainable solutions. This methodology explores questions including:

What are the embedded foundational factors (national, subnational and international) that shape the government, political, and socio-economic structures?
What are the rules (formal constitutional and legal frameworks) and norms (informal, social, and cultural) that guide the predictable behaviors of the state, civil society, and the private sector?
How might current events and circumstances influence the objectives and behavior of key actors and stakeholders, and how they respond to change?
What are the dynamics and interactions between the answers to the three questions above, and how might that impact a desired change?

Specific Examples:

In the Uganda PEA for example, Integra looked at how oil and gas development impacted land use and land rights. This PEA found that women are disproportionally affected by these demographic shifts because customary landholders with insecure tenure are displaced as the value of land increases.
In 2016, Integra facilitated PEA review workshops on behalf of USAID’s Bureau for Democracy, Conflict and Humanitarian Assistance to produce a ‘Summary of Lessons Learned Using USAID’s Applied Political Economy Analysis Framework’.

FEATURED PROJECTS

Cost Benefit (CBA) & Cost-effectiveness (CEA) Analyses

Integra uses state-of-the-art techniques in cost-benefit and cost-effectiveness analysis to weigh the costs of investments, delivering quantifiable results that enable decision makers to optimize project resources. CBAs and CEAs are effective tools for assessing the feasibility of interventions.

Integra’s process enables decision-makers to:

Better allocate resources by optimizing the design of projects.
Compare performance across projects.
Assess sustainability from alternative perspectives.
Inform resource decisions under uncertainty.
Forecast and understand an economic rate of return associated with a particular investment or activity.
Assess the feasibility of development investments.
Understand and communicate the benefits of development investments.

Integra combines established practices in CBA and CEA (such as the integrated approach developed by Harberger and Jenkins) with other innovative approaches to address unique issues such as streamlining of gender and ecosystem service valuation in CBA and CEA and conducting CBAs on policy and institutional reform projects.

Sample Projects: Over the past two years, Integra has conducted CBAs/CEAs in the past year for USAID/Mali, USAID/Madagascar, and USAID’s Office of Food for Peace

FEATURED PROJECTS

November 19, 2024

CBA Update and Training for USAID’s Resilience in the Sahel (RISE) Initiative

November 19, 2024

Assessment of USAID Guidelines for Cost-Benefit Analysis

September 23, 2024

Bureau for Humanitarian Assistance Cost Benefit Analysis (CBA)

November 16, 2023

Cost-Benefit Analysis and Training for RISE Initiative

September 12, 2020

Food for Peace (FFP) QR Code Cost Benefit Analysis (CBA)

Private Sector Assessment

Private Sector Assessments identify opportunities to engage with the private sector and collaborate for development impact. Integra researches and maps companies across sectors, industries, and geographies to determine where interests are aligned between donors and interested companies. Once mutual interest is identified, collaboration plans and models are developed to ensure efficiency and success are maximized.

Performance Evaluations

Integra designs and implements performance evaluations to measure what a project has achieved. Descriptive, recommendation-focused, and cause-and-effect questions are addressed in the process. Our approach aims to answer questions, such as:

Did the project achieve its stated objectives?
How do stakeholders perceive the success and impact of the project?
Are the expected results occurring?
Are the expected results sustainable?

Integra conducts a variety of mixed methodology and development evaluation approaches to inform decisions related to project design, management and operations. Qualitative analyses are used to provide valuable performance evaluation information including:

Focus Group Discussions
Key Informant Interviews
Observations
Spatially Derived Data
Remote Communication Strategies and Electronic Surveys

Sample Projects:

Mid-term Performance Evaluations

USAID USAID’s REPLACE IDIQ – Supporting Forests and Biodiversity in Northeastern Cambodia program, Integra conducted interviews in 14 communities focused on communal land titling and strengthening protected areas management.
USAID Indonesia’s Midterm Performance Evaluation of LESTARI – a $46 million, five-year project, Integra’s evaluation team conducted an independent review of progress toward agreed targets for reduced terrestrial emissions of greenhouse gases and improved conservation of forests and biodiversity.
USAID-funded Community Forest Protection (CFP) project in Zambia – Integra combined 55 key informant interviews, 88 focus group discussions, and geospatial analysis to understand how program activities impacted deforestation patterns and Government of Zambia policies.

Large-scale Regional Performance Evaluation

Mid-term Performance Evaluation of the Sustainable and Thriving Environments for West Africa Regional Development Program (STEWARD III) Under the USAID REPLACE IDIQ
Mid-term Performance Evaluation of the third phase of USAID’s Central Africa Regional Program for the Environment III.

FEATURED PROJECTS

Whole-of-project (WOPE) Evaluations

This performance evaluation primarily examines the progress of all the constituent parts of the project toward achievement of the Project Purpose. It may also inform how a project was implemented; how it was perceived and valued; and other questions that are pertinent to design, management, and operational decision-making.

Impact Evaluations

Integra applies experimental and quasi-experimental methodologies to identify impacts related to performance and other outcomes of development interventions. Impact evaluations are a powerful tool for answering the following question: Can an intervention or policy change be causally linked to observed changes in behavior, performance, or wellbeing at the individual or aggregate level?

Some key elements of our impact evaluation approach include:

Leveraging a network of economists, statisticians, research analysts, and in-country experts.
Conducting large-scale evaluations in the context of the program being evaluated.
Using technology solutions and local firms when collecting high volumes of surveys to ensure high quality and affordable evaluations.
Applying quasi-experimental estimation strategies, like propensity score matching, regressions discontinuity, or instrumental variable designs, to create a valid counterfactual for retrospective or ex-post evaluations.
Designing robust evaluation methods and sampling frameworks to address common challenges in developing an evaluation framework such as influence factors, design scale, and control group definition.
Ensuring that our analysis adheres to Integra’s high evaluation standards.

Example: Tanzanian Ex-Post Evaluation Integra is currently implementing a multi-prong ex-post evaluation for Heifer International. Based on Tanzanian secondary data, and existing monitoring data, we are creating a partial pre-project baseline and developing an ex-post evaluation strategy.

FEATURED PROJECTS

Collaborating, Learning and Adapting (CLA)

Development initiatives and investments can be made more effective through Collaborating, Learning, and Adapting (CLA). Integra supports donors in making CLA a more systematic and intentional element of development programing to improve outcomes and impact.

Integrating CLA ensures programs are grounded in evidence and adapted as needed to remain effective and relevant. Integra has experts who understand how to systematically apply CLA practices to address questions including:

Collaborating: Are we working with others to promote synergy?
Learning: What did we learn from each and every development initiative to inform decisions about future programming?
Adapting: How are we using what we learned to improve outcomes in subsequent initiatives?
Enabling Conditions: Is the environment within the organization one that embraces and supports CLA?

Capacity Building

Integra offers capacity building services to increase motivation, knowledge, and skills that support program design, analysis, and monitoring and evaluation. This support can be both direct and indirect depending on the need.
1. DIRECT Capacity

Direct organizational capacity building is grounded in new performance expectations and carefully designed training and development to build required staff knowledge and skills. Most organizations face severe time constraints, so training and performance support must be highly focused and effective. Our approach to building organizational capacity includes:

Identifying and analyzing skills/competency gaps through an assessment of the current status quo targeted skills and competencies, and other organizational or external factors.
Designing, building, and adapting targeted training for classroom and online training resources.
Delivering training and providing follow-up support and tools.
Evaluating, learning, and assessing training impact.

Furthermore, Integra knows that adult learning is effective when it is based on three key factors:

Understanding of an organization’s environment;
Recognition of the learner’s view of the “problem” to be addressed; and
Provides opportunities for practical application of learning to the job.

2. INDIRECT Capacity

Indirect capacity building is technical assistance that ensures timely staff augmentation. We assemble innovative and nimble teams designed to provide a wide range of sectoral and technical expertise.

Quickly Fielding Experts

Some projects require access to a large pool of international and local experts. Integra, together with our partners, maintains a combined database of 10,000 experts. Integra also has home office project specialists on staff that are fully trained to identify, recruit, and fielding consultants. Our clients have access to these project specialists as surge capacity, enabling Integra to scale-up field support quickly, while remaining cost efficient.

Building Local Partnerships

Integra works with local partners in ways that are purposeful, strategic, and cost-effective to build local capacity and self-reliance. Integra employs a ‘Learn-by-doing’ capacity building approach and creates a long-term capacity building plan for each local firm through the following:

Providing a range of support including institutional capacity building and financial management.
Training in theories of change.
Identifying performance indicators.
Applying internal quality control management.

As capacity is built, Integra provides local firms with a large share of activities to a degree that their contribution to projects grow year after year.

Example Project:

USAID/Mali Cost-Benefit and Cost-Effectiveness Training

FEATURED PROJECTS

Training Design & Development

Many development teams face severe time constraints, so training and performance support must be highly focused and effective. Integra has customizable trainings and a track record of delivering courses—minimizing lead time for course prep.

The Integra team can:

Identify and analyze skills/competency gaps through a CLA and Evidence Listening Tour and a CLA Practices Inventory;
Design, build, and adapt targeted training for classroom and online training resources;
Deliver training and provide follow-up support and tools; and,
Develop train-the-trainer programs for MEL Champions to coach others through peer learning, mentoring, and related programs.

Workshop Facilitation

Integra implements a range of CLA activities, including evidence-driven training design and agenda development, provision of technical experts, workshop facilitation, and data synthesis and dissemination.

Commercial Legal And Institutional Reform (CLIR) Diagnostics

Integra is a lead implementer of the CLIR framework, an analytical diagnostic that examines the systematic limitations that inhibit investment and business activity in a country. Integra provides this in-depth analysis of the legal framework, implementing institutions, supporting institutions, and social dynamics to examine market systems across different sectors. We study the constraints sectors face across an enterprise life cycle, from launching a business and accessing inputs to reaching end consumers through cross-border trade.

FEATURED PROJECTS

Policy Analysis Matrix (PAM)

Integra conducts analysis of the competitiveness of different sectors and subsectors. The PAM methodology provides information to study the impacts of price and investment policies. It is most commonly applied to the agricultural sector and seeks to address three central issues:

Ranking of the competitiveness of agricultural systems at market prices,
Ranking of the efficiency of agricultural systems, and
Measuring the transfer effects of policies.

Using the PAM, one can identify how a given public policy or investment would be likely to affect the general performance of the economy and the private income of farmers. The PAM allows Integra to compare the impacts of alternative policies on rural poverty as well as on national economic performance.

Domestic Resource Cost (DRC) Analysis

A DRC analysis, a subset of PAM, estimates the economic, as well as the financial profitability of entire value chains, as well as individual segments within these chains. It calculates the degree to which the chains are subject to positive or negative protection in relation to world market conditions. The DRC is an indicator of the efficiency with which a country’s domestic resources, such as labor and capital, are converted into useful output (i.e. the degree to which there is a comparative advantage).

Data Collection

Data collection varies depending on the context. The hallmarks to our approach include:

Using electronic data collection – increases the quality of data collection in four ways:

Minimizes data entry errors through real time quality audits during the data collection process.
Allows for seamless respondent identification, data entry, and monitoring while requiring multiple sign-offs, both from supervisors within the survey collection teams and from Integra’s staff, before finalizing a survey.
Allows for the quick revision of surveys during enumeration training sessions and for the changing of survey assignments from offsite locations.
Saves both time and money through automating the data entry process eliminating the possibility of data entry error from illegible or damaged paper surveys and the expense of manual data entry.

Using local data collection firms to lower the time and cost of data collection:

Provides greater local contextual awareness and to lower the time and cost of data collection.
Provides support for hiring and training interviewers, providing coordinators and enumerators, organizing logistical arrangements for transport, and translation.
Enables us to lower data collection costs and field teams on short notice.

Combined use of evaluation specialists and subject matter experts: Integra uses blended evaluation teams for data collection, including both subject matter and evaluation experts. By employing strong evaluation experts, Integra’s teams:

Design high quality qualitative and quantitative survey instruments that accurately capture the necessary information to conduct the evaluation.
Tailor survey instruments to fit the context of the project.
Rapidly revise piloted survey instruments based on feedback from the field.

Ethical data collection: Integra is committed to ensuring the protection of all human subjects by working with conscientious evaluators, following data collection guidelines laws, and responsibly sharing data. Integra requires all evaluation team members to:

Complete a course on human subjects research.
Comply with all local and international rules around research data collection.
Receive IRB approval before collecting data.
Confirm the human subjects have been adequately de-identified and that additional potential identifiers, like GIS locations, have been properly blurred or removed prior to release of data.

Applying modern technology-based approaches

We employ mobile devices with offline operational capabilities for data collection, enabling us to regularly monitor the quality of incoming data and make course corrections as necessary.

We also offer geospatial analysis and mapping capabilities to supplement and enhance on-the-ground data collection.

Example Project: Integra recently used this approach to conduct a qualitative evaluation of USAID’s Community Forest Program in Zambia under LEAP III, involving 88 focus group discussions over a three-week period, and used a mixed methods approach including both CAPI and GIS to identify and conduct over 84 focus group discussions in 43 villages.

Data Analysis

Integra brings strong experience in data analysis and integrating intentional learning into organizational cultures, systems, and processes through rigorous meta-analysis and dissemination of lessons learned and good practices.

Our team is skilled in accessing, combining, and analyzing data, including survey data, disaggregated monitoring data, and/or existing survey or administrative data. While adaptable to client needs, Integra normally works in STATA for quantitative evaluation, NVivo for qualitative analysis, and ArcGIS for GIS mapping. These software packages provide Integra with the tools to accurately interpret the data and present MCC with rigorous results. We can also seamlessly convert data from one format to another as needed to combine information from different sources for analysis.

We employ a variety of statistical techniques as necessary, including descriptive analysis and regression analysis, accounting for clustering of standard errors, if appropriate. When using survey data, we apply appropriate survey weights in our analyses to account for the probability of selection of survey respondents to the sample. We also bring to bear our high standards for structured and rigorous data analysis, as well as our experience in creating data documentation, data visualizations, and written reports.

Data Quality Assessment (DQA)

Integra employs robust procedures to quality assure existing data and maintain strong data systems. These procedures include logical, statistical, visual, and replication tests to validate the integrity of the data. By ensuring a high level of data quality, data is useful from a reproduction/replication and a data re-use perspective.

Validity: Integra matches the survey instruments to the project’s logical framework to test the consistency of the data collection effort to the project design. We take the following steps: 1) draft data collection instruments that pose the questions needed to capture the measures of interest; 2) present questions in a way that will lead to the intended interpretation—and thus a valid response—by the respondent; 3) conduct trainings and practice runs to ensure that step 2 is carried out as intended; 4) monitor all incoming data.
Reliability: To achieve data reliability, we ensure that data is collected in a consistent manner across all respondents, locations, and time frames and that the collected data is as complete and accurate as possible. We work with field staff to develop manuals that outline the proper data collection steps, and lead trainings so that all enumerators collect data in the same manner.
Timeliness: To ensure timeliness, we identify any constraints to collecting the data in advance (such as weather patterns, constraints to access, elections or other politically motivated events). Once we identify constraints and understand available resources, we determine the appropriate frequency of data collection.
Precision: During our mandatory data review, we 1) design programs to automatically conduct spot checks, duplicate observations, missing data, and out-of-scope responses, and 2) if needed, compare against original data entry (if in paper).
Integrity: We ensure that proper handling of data transfer and export and store all data (including audio recordings) on a secure server with safeguards in place to prevent unauthorized access. Project folders are secured so that only people authorized to review the data have access.

Data Management

The Integra team is capable of providing a wide range of data management services, including but not limited to:

Data cleaning, analysis, and visualization on an ad-hoc or routine basis.
Data mapping to bridge the transfer of information between multiple distinct data models.
Integrate raw data into real-time visualizations and project status dashboards.

Data Synthesis, Communication and Visualization

Integra employs specialist to identify, understand, and produce materials that meet our client’s learning and messaging needs. We help clients move to the next level in data visualization, including developing new approaches for presenting performance data. Data is integral to many performance monitoring processes and stakeholder conversations but telling stories through data requires skill and attention as viewers can quickly lose interest in data-intensive reports that lack intuitive hooks. We create interactive visualizations that simplify data into easily understandable graphics that facilitate data interpretation, use, and learning for adaptive management.

Data Transparency

Integra also believes in the importance of data transparency and is well versed in the procedures and systems required to create a transparent evaluation and comply with requirements. From the de-identification process to the standardization of evaluation documentation, Integra works with clients to ensure they make their evaluations replicable and their data reusable.

When possible, Integra posts data, statistical code, de-identification procedures, publications, and a readme file to the Harvard Dataverse. Before posting any materials to the Dataverse, Integra reviews and revises the data and code for any identifiable information. By including metadata with each posting, Integra encourages others to replicate the original analysis and repurpose the data for other projects.

Logistical Support Services

Integra provides a range of logistical support services for clients including conducting trainings, events, workshops and conferences. Our team works closely with relevant staff to meet a variety of events needs.

Providing logistics support and materials preparation
Planning and managing public and/or internal events
Facilitating group events to promote learning and collaboration

Examples:

We hosted an event for the Egyptian American Enterprise Fund at the National Press Club in Washington DC, as well as organizing a series of 12 monthly internal USAID trainings on different Private Sector Engagement (PSE) topics.
We also organized all logistics for the ECON2020 conference, including organizing speakers and promotional material for the event, and creating custom branded event materials.
In February 2020 under the USAID/Asia Emerging Opportunities (AEO) project, Integra implemented a knowledge management and learning workshop based on the findings from the 2020 Asia Environment Officer Planning Survey in preparation for the 2020 Asia Environment Officers Learning Event in Bangkok, Thailand. Integra is offering logistics and facilitation support, as well as assistance in identifying suitable external environment experts from the region to participate in the workshops.
Integra logistical support services were also requested by the Senior Advisor for Natural Resource Management in USAID’s Asia Bureau to conduct a “How to Save a Tiger” Knowledge Sharing Event under USAID/AEO in collaboration with UNDP on March 4th, 2020 in New York. Support for this activity included logistics such as venue and catering, advertising and outreach, and speaker coordination.

Public Financial Management (PFM) Analysis

Integra recognizes that strong PFM systems are critical for the effective, transparent, and accountable use of public funds. Integra brings several innovations in PFM, including:

- Transparency with technology. We view transparency as a necessary pre-condition for accountability and high-quality decision-making. Most governments, however, use outdated digital tools to manage their PFM systems. Integra leverages its ICT experience to support governments with cutting-edge technology.
- Public-Private Partnerships and innovative financing. The private sector offers a potential partner to address funding deficits in infrastructure, from transport to healthcare, to drive economic growth. Many countries however remain inexperienced with planning, budgeting, and managing private partnerships, and as a result, the success of their application have been mixed.

Gender in PFM. Integra supports the application of gender mainstreaming in the budgetary process. Gender-responsive budgeting helps ensure public resources are used efficiently, based on the needs and priorities of both men and women.

PFM Analytical Tools

- Public Expenditures Financial Accountability (PEFA): Measures the extent to which PFM systems, processes and institutions contribute to the achievement of desirable budget outcomes.

Public Financial Management Risk Assessment Framework (PFMRAF): Assesses, analyzes, and mitigates risks associated with partner country’s public financial systems.

Gather, Analyze, and Plan (GAP) tool: A simple Excel-based tool designed to help PFM officials understand and plan for the costs associated with public planning
Impact Now: An Excel-based model that estimates the economic impact of public spending.

Integra has implemented PFM reforms in Colombia, Kenya, Indonesia, Tunisia and the Philippines, including a PFM analysis of Indonesia Universal Service Fund (USF) operations under the USAID Global Broadband and Innovations Program.

FEATURED PROJECTS

Bridging The Gap Between CBA, Performance Evaluations, And Impact Assessments

One of the hallmarks of Integra’s method is a systematic approach to integrating feasibility analysis, performance monitoring, and impact measurement. In fact, Integra is one of the first evaluators to conduct a pilot evaluation for MCC (under the MCC Green Prosperity Evaluation in Indonesia) that combines an evaluability assessment, performance evaluation, and a CBA. As part of this assignment, Integra designed and implemented three evaluations: 1) an ex-post performance evaluation of MCC/Indonesia’s On-Grid Renewable Energy Grant Portfolio, 2) an ex- post performance evaluation of their Peatland Grant Portfolio, and 3) an expanded evaluability assessment of their social forestry portfolio. Integra presented the findings and recommendations to Bappenas in 2019 suggesting areas for improvement in MCC facility management and program implementation moving forward.