2019 Federal Standard of Excellence

Innovation

Did the agency have staff, policies, and processes in place that encouraged innovation to improve the impact of its programs in FY19? (Examples: Prizes and challenges; behavioral science trials; innovation labs/accelerators; performance partnership pilots; demonstration projects or waivers with rigorous evaluation requirements)

Score
6
Administration for Children and Families (HHS)
7.1 Did the agency engage leadership and staff in its innovation efforts?
  • HHS has embarked on a process called ReImagine HHS, which has engaged leadership and staff from around the department to identify strategic shifts to transform how HHS operates. One part of this larger initiative, called Aim for Independence (AFI), is using a human centered design approach to rethink how ACF does work and how that work translates into long-lasting, positive outcomes for parents and children. Engagement activities have included a leadership retreat and opportunities for staff input.
  • ACF leadership has proposed new Opportunity and Economic Mobility Demonstrations to allow states to redesign safety net service delivery by streamlining funding from multiple public assistance and workforce development programs and providing services tailored to their populations’ specific needs. The demonstrations would be subject to rigorous evaluation.
7.2 Did the agency have policies that promote innovation?
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • ACF projects that support innovation include:
    • ACF’s Behavioral Interventions to Advance Self-Sufficiency (BIAS) project was the first major effort to apply a behavioral economics lens to programs that serve poor families in the U.S. The project conducted 15 rapid-cycle randomized tests of behavioral interventions. The Behavioral Interventions to Advance Self-Sufficiency-Next Generation (BIAS-NG) project continues ACF’s exploration of the application of behavioral science to the programs and target populations of ACF. Additionally, the Behavioral Interventions Scholars(BIS) grant program supports dissertation research that applies a behavioral science lens to research questions relevant to social services programs and policies and other issues facing low-income families.
    • ACF’s Human Centered Design for Human Services project is exploring the application of human centered design across ACF service delivery programs at the federal, state, and local levels.
    • Several ACF grant programs are innovation projects, demonstration projects, or allow waivers. See details below in the response to Sub-Criteria 4 below.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
Score
5
Administration for Community Living (HHS)
7.1 Did the agency engage leadership and staff in its innovation efforts?
7.2 Did the agency have policies that promote innovation?
  • There are several funding streams that support innovation. The Older Americans Act, which funds ACL’s Administration on Aging, allows ACL to use up to 1% of its appropriations for nutrition innovation demonstrations designed to develop and implement evidence-based practices that enhance senior nutrition. One result is that, consistent with the Administrator’s focus on identifying new ways to efficiently improve direct service programs, ACL is using $3.5 million to fund nutrition innovations and test ways to modernize how meals are provided to a changing senior population. One promising demonstration (entitled Double Blind Randomized Control Trial on the Effect of Evidence-Based Suicide Intervention Training on the Home-Delivered and Congregate Nutrition Program through the Atlanta Regional Commission), currently being carried out by the Georgia State University Research Foundation, is an effort to train volunteers who deliver home-delivered meals to recognize and report indicators of suicidal intent and other mental health issues so that they can be addressed.
  • State Councils on Developmental Disabilities (SCDD) are charged with identifying and addressing the most pressing needs of people with developmental disabilities in their state and territory. Councils work with different groups in many ways, including educating communities to welcome people with developmental disabilities; funding projects to show new ways that people with disabilities can work, play, and learn; and seeking information from the public as well as state and national sources.
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • ACL uses innovation dollars provided under Title IV of the Older Americans Act as a means of testing new approaches to service delivery and developing replicable models that could then be embedded into core programs. In FY16, ACL established the Elder Justice Innovation Grants program to increase knowledge about effective prevention and intervention of abuse, neglect, and exploitation of older adults, native elders, adults with disabilities, people who self-neglect, and guardianship abuse. In FY18, ACL monitored the second year of 2-year grants awarded to 5 non-profit organizations in FY17, totaling $2.2 million. In FY18, ACL awarded grants under the Alzheimer’s Disease Programs to States and Communities HHS-2018-ACL-AOA-ADPI-0307 to pilot innovative  dementia-capable home- and community-based services (HCBS) programs to states and communities.
  • NIDILRR participates in the Small Business Innovation Research program to improve the lives of people with disabilities through research and development of innovative products generated by small businesses, and to increase the commercial application of NIDILRR-supported research results and development products.
  • The Engagement and Older Adults Resource Center, funded by ACL, provides technical assistance and serves as a repository for innovations designed to increase the aging network’s ability to tailor social engagement activities to meet the needs of older adults.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
  • ACL has a number of model programs and demonstration grants that propose and test the use of innovative approaches. For example, ACL funded cooperative agreements for the development and testing of model approaches towards coordinated and comprehensive systems for enhancing and assuring the independence, integration, safety, health, and well-being of individuals with intellectual and developmental disabilities living in the community (i.e. Living Well Grants). While the evaluation of this program is not yet complete, initial findings about what works were integrated into the requirements of the funding announcement for the FY18 award cycle.
Score
8
U.S. Agency for International Development
7.1 Did the agency engage leadership and staff in its innovation efforts?
  • In FY2019, USAID appointed a new Chief Innovation Officer to advocate for innovation throughout development and national security strategies across USAID, the U.S. Government, and the international community. The Chief Innovation Officer promotes opportunities for entrepreneurs, implementing partners, universities, donors, and others to test and scale innovative solutions and approaches to development problems around the world. In FY2019, the U.S. Global Development Lab also engaged USAID leadership and Mission staff from around the world at the Mission Directors Conference, the Contracting Officer and Controller Conference, the Foreign Service National Conference, and the Private Sector Engagement Forum.
  • For innovations specific to a particular sector, Agency leadership has supported technical staff in surfacing groundbreaking ideas, such as how the Bureau for Global Health’s Center for Innovation and Impact (CII) used open innovation approaches to issue the Saving Lives at Birth Grand Challenge and identify promising, life-saving maternal and newborn health innovations.
7.2 Did the agency have policies that promote innovation?
  • In December 2018, USAID released its first Acquisition and Assistance Strategy to guide changes to policy and practice to increase flexibility, creativity, and adaptability in USAID partnering approaches. The strategy states: “USAID will increase our use of collaborative and co-creative approaches by 10 percentage points in terms of total dollars and awards in FY 2019.” Building on past years of experimentation and innovation, USAID will challenge design and procurement officers to engage a much wider range of practices that emphasize collaboration and co-creation. USAID will diversify approaches to design, solicitation, and awards—designing activities less prescriptively and more collaboratively; simplifying access for new and local partners; and increasing usage of awards that pay for results, as opposed to presumptively reimbursing for costs.
  • In the same month, USAID released the first Private Sector Engagement Policy to transform how USAID engages the private sector throughout core operations across all sectors. The policy represents an intentional shift to pursue market-based approaches, investment, and private sector expertise and innovation to accelerate country journeys to self-reliance.
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • Since 2014, the U.S. Global Development Lab has advanced science, technology, innovation, and partnerships to accelerate development impact. In FY 2019, the Lab’s budget was $75 million with 76 direct hire staff (Civil Service and Foreign Service). The Lab is home to numerous innovation teams and programs, including those housed in the Center for Development Innovation.
  • In addition, the Center for Innovation and Impact (CII)—the Bureau for Global Health’s dedicated innovation office—takes a business-minded approach to fast-tracking the development, introduction, and scale-up of health innovations that address the world’s most important health challenges, and assessing and adopting cutting-edge approaches (such as unmanned aerial vehicles and artificial intelligence).
  • USAID and its partners have launched eleven Grand Challenges for Development since 2011. Across the Grand Challenges portfolio, partners have jointly committed over $535 million ($155 million from USAID) in grants and technical assistance for over 528 innovators in 107 countries. To date, more than $614 million in follow-on funding has been catalyzed from external sources, a key measure of success.
  • Feed the Future Partnering for Innovation partners with agribusinesses to help them commercialize and scale new agricultural innovations to help improve the livelihoods of smallholder farmers, increasing their productivity and incomes. To date the program has worked with 59 partners in 20 different countries, investing more than $43 million in new technologies and services, and leveraging nearly $100 million in private sector investment. The program has helped commercialize over 118 innovations, which resulted in an estimated $99 million in sales.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
  • Within the U.S. Global Development Lab, the MERLIN program works to innovate on traditional approaches to monitoring, evaluation, research and learning. While innovative in themselves, these approaches can also be better suited to evaluating an innovation effort. Two examples include Developmental Evaluation, which aims to provide ongoing feedback to managers on implementation through an embedded evaluator, and Rapid Feedback, which allows implementers to test various methods to reach certain targeted results (more quickly than through traditional midterm or final evaluations).
  • Many of the agency’s programs such as Grand Challenges and Development Innovation Ventures (DIV) have been reviewed by formal audit and other performance and impact interventions. DIV is USAID’s tiered, evidence-driven open innovation program. It awards grants for innovative solutions to any development problem, on the basis of rigorous evidence of impact, cost-effectiveness, and a pathway to scale via the public and/or private sectors. The DIV model is designed to source breakthrough solutions, to minimize risk, and maximize impact by funding according to outcomes and milestones, to rigorously evaluate impact and cost-effectiveness, and to scale proven solutions. Since 2010, DIV has supported 197 innovations in 45 countries with approximately $118 million. It has generated experimental or quasi-experimental evaluation studies of more than a third of those innovations. And a forthcoming working paper rigorously assesses the social rate of return of DIV’s early portfolio.
Score
4
Corporation for National and Community Service
7.1 Did the agency engage leadership and staff in its innovation efforts?
  • Staff at all levels of the organization participate in work groups focused on implementing CNCS’s Transformation and Sustainability Plan. The CEO has also conducted Service Jams to elicit feedback from staff to support the plan. Service Jam topics have focused on what a best-in class learning organization looks like and how CNCS could break down silos.
7.2 Did the agency have policies that promote innovation?
  • No examples available 
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • CNCS continued to learn from its evidence-based planning grant program which “awards evidence-based intervention planning grants to organizations that develop new national service models seeking to integrate members into innovative evidence-based interventions.” CNCS continued to learn from its research grantees, who receive grant funds to engage community residents and leaders in the development of new and innovative national service projects.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
  • As part of the evaluation of the Social Innovation Program, which was designed to identify and rigorously test innovative approaches to social service problems, CNCS continues to receive evaluation reports from grantees. As of May 2019, CNCS has received 72 final SIF evaluation reports, of which 20 (28%) were experimental designs and 39 (54%) were quasi-experimental designs. Further, the evidence-based planning grant program and the research grant program both seek to generate innovative national service models. The planning grants require an evaluation plan. The research grants use evidence to inform action planning and solutions.
Score
4
U.S. Department of Education
7.1 Did the agency engage leadership and staff in its innovation efforts?
  • In FY19, the Office of Elementary and Secondary Education made strategic investments in innovative educational programs and practices and administered discretionary grant programs. In FY19, the Innovation and Improvement account received $1.035 billion. The Department reorganized in 2019, consolidating the Office of Innovation and Improvement into the Office of Elementary and Secondary Education.
7.2 Did the agency have policies that promote innovation?
  • ED uses the Experimental Sites Initiative under section 487A(b) of the Higher Education Act of 1965, as amended, to test the effectiveness of statutory and regulatory flexibility for participating institutions disbursing Title IV student aid. ED has waived specific statutory or regulatory requirements at the postsecondary institutions, or consortia of institutions, approved to participate in the experiments. The outcomes of experiments have the potential to benefit all postsecondary institutions and the students they serve.
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • The Education Innovation and Research (EIR) program is ED’s primary innovation program for K–12 public education. EIR grants are focused on validating and scaling evidence-based practices and encouraging innovative approaches to persistent challenges. The EIR program incorporates a tiered-evidence framework that supports larger awards for projects with the strongest evidence base as well as promising earlier-stage projects that are willing to undergo rigorous evaluation. Funds may be used for: (1) early-phase grants for the development, implementation, and feasibility testing of an intervention or innovation, which prior research suggests has promise, in order to determine whether the intervention can improve student academic outcomes; (2) mid-phase grants for implementation and rigorous evaluation of interventions that have been successfully implemented under early-phase grants or have met similar criteria for documenting program effectiveness; and (3) expansion and replication of interventions or innovations that have been found to produce a sizable impact under a mid-phase grant or have met similar criteria for documenting program effectiveness.
  • The IES Research Grants Program supports the development and iterative testing of new, innovative approaches to improving education outcomes. IES makes research grants with a goal structure including “Goal 2: Development and Innovation,” which supports the development of new education curricula; instructional approaches; professional development; technology; and practices, programs, and policies that are implemented at the student-, classroom-, school-, district-, state-, or federal-level to improve student education outcomes. The Small Business Innovation Research Program provides funds for rapid prototype development and evaluation as well as for product development and evaluation. The related ED Games Expo promotes emerging education technology by allowing for demonstration of game-based learning.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
  • ED is currently implementing the Experimental Sites Initiative to assess the effects of statutory and regulatory flexibility for participating institutions disbursing Title IV student aid. ED collects performance data from all participating institutions, and IES is currently conducting rigorous evaluations of selected Experimental Sites, including two related to short-term Pell grants.
  • The Education Innovation and Research (EIR) program, ED’s primary innovation program for K–12 public education, incorporates a tiered-evidence framework that supports larger awards for projects with the strongest evidence base as well as promising earlier-stage projects that are willing to undergo rigorous evaluation.
Score
6
U.S. Dept. of Housing & Urban Development
7.1 Did the agency engage leadership and staff in its innovation efforts?
  • HUD has an Office of Innovation led by a Deputy Assistant Secretary. The office organized the five-day Innovative Housing Showcase on the national mall with federal and private sector partners in June 2019 to demonstrate new housing technology and discuss innovation barriers and opportunities. The entire local HUD staff was encouraged to attend and view the innovative technologies.
7.2 Did the agency have policies that promote innovation?
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • HUD established the Office of Innovation in 2019 to advance innovation in several domains. The office managed the 2019 Innovative Housing Showcase and is developing prize competitions to stimulate innovation in housing and HUD policy and programs.
  • HUD’s regulation of manufactured housing production is guided by a federal advisory committee, the Manufactured Housing Consensus Committee, to provide increased ability for the industry to produce some of the nation’s most innovative, safe, and affordable housing.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
Score
6
U.S. Department of Labor
7.1 Did the agency engage leadership and staff in its innovation efforts?
  • DOL’s Chief Evaluation Office (CEO) Data Analytics team developed a secure data analysis platform accessible to all DOL staff, pre-loaded with common statistical packages and offering the capability to access and merge various administrative data for analysis. DOL supports staff in executing limitless web-based A/B testing and other behaviorally-informed trials, with the shared service of the advanced Granicus platform’s GovDelivery communications tool, including free technical support. This tool enhances the Department’s ability to communicate with the public, such as through targeted email campaigns, and to adjust these communications, informed by testing and data, to increase engagement on relevant topics. CEO also has developed toolkits and detailed resources for staff to effectively design behaviorally informed tests.
7.2 Did the agency have policies that promote innovation?
  • DOL is strongly committed to promoting innovation in our policies and practices. For example, the Employment & Training Administration’s (ETA) competitive funding routinely funds innovative programming, since grantees typically bundle various program services and components to best meet the needs of the people being served by them in their local contexts. A particularly good example of where this innovation is happening is in the Administration’s high priority area of apprenticeships. DOL is funding $150 million to support sector-basedinnovations in apprenticeship. DOL has invested more than $95 million through the ApprenticeshipUSA initiative – a national campaign bringing together a broad range of stakeholders, including employers, labor, and states as well as education and workforce partners, to expand and diversify registered apprenticeships in the United States. This includes more than $60 million for state-led strategies to grow and diversify apprenticeship, and state Accelerator Grants to help integrate apprenticeship into education and workforce systems; engage industry and other partners to expand apprenticeship to new sectors and new populations at scale; conduct outreach and work with employers to start new programs; promote greater inclusion and diversity in apprenticeship; and develop statewide and regional strategies aimed at building state capacity to support new apprenticeship programs. All of these grants include funding for data collection; additionally, ETA and CEO are conducting an evaluation of the American Apprenticeship Initiative.
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • DOL has built capacity for staff innovation through the Performance Management Center’s Continuous Process Improvement (CPI) Program, an agency-wide opportunity which trains and certifies agency staff on Lean Six Sigma (LSS) methodologies through real-time execution of DOL process improvement projects. The program includes classroom sessions that prepare participants for LSS Black Belt certification examinations, including the American Society for Quality (ASQ) as well as DOL’s own certification.
  • The Wage and Hour Division’s (WHD) Transformation Team is one such example where continuous improvement efforts are driving innovation. Their work has identified potential areas where behavioral interventions and trials may inform program improvement. CEO is also working across agencies – including WHD, ETA, Women’s Bureau, Veterans Employment & Training Service (VETS), Office of Federal Contract Compliance Programs (OFCCP), and International Labor Affairs Bureau (ILAB) – to identify and assess the feasibility of other areas where insights from behavioral science can be used to improve the performance and outcomes of DOL programs.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
  • DOL typically couples innovation with rigorous evaluation to learn from experiments. For example, DOL is participating in the Performance Partnership Pilots (P3) for innovative service delivery for disconnected youth which includes not only waivers and blending and braiding of federal funds, but gives bonus points in application reviews for proposing “high tier” evaluations. DOL is the lead agency for the evaluation of P3. An interim report will be released in FY19.
  • DOL has two pilot projects that tested use of a Pay for Success (PFS) financing model in pilot projects from 2013-2017, with a final report on the outcomes of this pilot expected by the end of calendar year 2019.
  • DOL routinely uses Job Corps’ demonstration authority to test and evaluate innovative and promising models to improve outcomes for youth. Currently, CEO is sponsoring a rigorous impact evaluation to examine the effectiveness of one of these pilots, the Job Corps Experimental Center Cascades.
Score
6
Millennium Challenge Corporation
7.1 Did the agency engage leadership and staff in its innovation efforts?
  • In 2014, MCC developed an internal “Solutions Lab” that was designed to encourage innovation in program development and implementation by engaging staff to come up with creative solutions to some of the biggest challenges MCC faces. MCC promotes agency-wide participation in its Solutions Lab through an internal portal. To further encourage staff who pursue innovative ideas throughout the compact lifecycle, MCC launched the annual MCC Innovation Award as a part of the Agency’s Annual Awards Ceremony held each summer. The Innovation Award recognizes individuals who demonstrate “exemplary” leadership integrating innovation in project design, project implementation, and/or systems functionality and efficiency. Selections for the Innovation Award are based on a demonstrated ability to lead and implement innovative strategies from project conception that foster sustained learning and collaboration and add value to MCC and/or country partnerships.
7.2 Did the agency have policies that promote innovation?
  • MCC’s approach to development assistance hinges on its innovative and extensive use of evidence to inform investment decisions, guide program implementation strategies, and assess and learn from its investment experiences. As such, MCC’s Office of Strategic Partnerships offers an Annual Program Statement (APS) opportunity that allows MCC divisions and country teams to tap the most innovative solutions to new development issues. In FY19, the Monitoring and Evaluation division, using MCC’s APS and traditional evaluation firms, has been piloting partnerships with academics and in-country think tanks to leverage innovative, lower cost data technologies across sectors and regions. These include:
    • using satellite imagery in Sri Lanka to measure visible changes in investment on land to get early indications if improved land rights are spurring investment;
    • leveraging big data and cell phone applications in Colombo, Sri Lanka to monitor changes in traffic congestion and the use of public transport;
    • independently measuring power outages and voltage fluctuations using cell phones in Ghana, where utility outage data is unreliable, and where outage reduction is a critical outcome targeted by the Compact;
    • using pressure loggers on piped water at the network and household levels to get independent readings on access to water in Dar es Salaam, Tanzania; and using remote sensing to measure water supply in water kiosks in Freetown, Sierra Leone.
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • MCC recently launched an internal Millennium Efficiency Challenge (MEC) designed to tap into the extensive knowledge of MCC’s staff to identify efficiencies and innovative solutions that can shorten the compact and threshold program development timeline while maintaining MCC’s rigorous quality standards and investment criteria.
  • In September 2014, MCC’s Monitoring and Evaluation division launched the agency’s first Open Data Challenge, which continued into FY19. The Open Data Challenge initiative is intended to facilitate broader use of MCC’s U.S.-taxpayer funded data, encourage innovative ideas, and maximize the use of data that MCC finances for its independent evaluations.
  • MCC regularly engages in implementing test projects as part of its overall compact programs. A few examples include: (1) in Morocco, an innovative pay-for-results mechanism to replicate or expand proven programs that provide integrated support; (2) a “call-for-ideas” in Benin for information regarding potential projects that would expand access to renewable off-grid electrical power; (3) a regulatory strengthening project in Sierra Leone that includes funding for a results-based financing system; and (4) an Innovation Grant Program in Zambia to encourage local innovation in pro-poor service delivery in the water sector.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
  • Although MCC rigorously evaluates all program efforts, MCC takes special care to ensure that innovative or untested programs are thoroughly evaluated. In addition to producing final program evaluations, MCC is continuously monitoring and evaluating all programs throughout the program lifecycle, including innovation efforts, to determine if mid-program course-correction actions are necessary. This interim data helps MCC continuously improve its innovation efforts so that they can be most effective and impactful. Although 37% of MCC’s evaluations use random-assignment methods, all of MCC’s evaluations – both impact and performance – use rigorous methods to achieve the three-part objectives of accountability, learning, and results in the most cost-effective way possible.
Score
3
Substance Abuse and Mental Health Services Administration
7.1 Did the agency engage leadership and staff in its innovation efforts?
  • SAMHSA participates in collaborations with other HHS agencies to promote innovative uses of data, technology and innovation across HHS to create a more effective government and improve the health of the nation, via the HHS IDEA Lab. SAMHSA has co-developed and submitted several innovative data utilization project proposals to the Ignite Accelerator of the HHS IDEA Lab, such as Rapid Opioid Alert and Response (ROAR), a project to monitor and prevent opioid overdoses by linking heroin users to resources and information.
7.2 Did the agency have policies that promote innovation?
  • Pursuant to the 21st Century Cures Act, SAMHSA established the National Mental Health and Substance Use Policy Laboratory (NMHSUPL) as an office, led by a Director, within the agency to promote evidence-based practices and service delivery models. NMHSUPL staff and programs engage in the following activities: 1) Facilitate the implementation of policy changes likely to improve mental health, mental illness, recovery supports, and the prevention and treatment of substance use disorder services; 2) Work with CBHSQ to evaluate and disseminate evidence-based practices; and 3) Carry out other activities to encourage innovation and disseminate evidence-based programs and practices. 
7.3 Did the agency have processes, structures, or programs to stimulate innovation?
  • The SAMHSA Program Portal, a collection of technical assistance and training resources provided by the agency, provides behavioral health professionals with education and collaboration opportunities, and ample tools and technical assistance resources that promote innovation in practice and program improvement. Located within the Knowledge Network are groups such as the Center for Financing Reform and Innovation, which works with states and territories, local policy makers, providers, consumers, and other stakeholders to promote innovative financing and delivery system reforms.
7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?
  • SAMHSA does not list any completed evaluation reports on its evaluation website. Of the 10 evaluation reports found on the publications page, none appear to use experimental methods.
Back to the Standard

Visit Results4America.org

Copyright 2024 Results for America. All rights reserved. Privacy Policy

Site by Wide Eye Creative