Did the agency have senior staff members with the authority, staff, and budget to build and use evidence to inform the agency’s major policy and program decisions in FY19?

1.1 Did the agency have a senior leader with the budget and staff to serve as the agency’s Evaluation Officer (or equivalent)? (Example: Evidence Act 313)

• The General Deputy Assistant Secretary of the Office of Policy Development & Research (PD&R) serves as the Department of Housing and Urban Development (HUD) evaluation officer. HUD’s Office of Policy Development & Research (PD&R) is led by an Assistant Secretary and the career General Deputy Assistant Secretary. PD&R comprises six offices, 139 staff including a team of field economists that work in HUD’s 10 regional offices across the country, and a budget of $96 million in FY19. The Assistant Secretary and Evaluation Officer ensure that evidence informs policy development through frequent personal engagement with other principal staff, the Secretary, and external policy officials including consultation with Congress, speeches to policy audiences, sponsorship of public research briefings, and policy implications memoranda.

1.2 Did the agency have a senior leader with the budget and staff to serve as the agency’s Chief Data Officer (or equivalent)? (Example: Evidence Act 202(e))

• HUD’s Chief Data Officer is a senior advisor in the Office of the Chief Financial Officer. Budget and staffing have not been determined. The PD&R GDAS and Statistical Official are responsible for numerous data infrastructure functions such as the collection and analysis of national housing market data (including survey collaborations with the Census Bureau); developing income limits and factors to support program operations; advising and assisting program offices with the development and analysis of administrative data collections; and  developing open data products drawn from administrative systems, including geospatial data products.

1.3 Did the agency have a governance structure to coordinate the activities of its evaluation officer, chief data officer, statistical officer, and other related officials in order to inform policy decisions and evaluate the agency’s major programs?

• PD&R informs HUD’s policy development and implementation by conducting, supporting, and sharing research, surveys, demonstrations, program evaluations, and best practices. PD&R achieves this mission through three interrelated core functions:
  • (1) collecting and analyzing national housing market data (including with the Census Bureau); (2) conducting research, program evaluations, and demonstrations; and (3) providing policy advice and analytic support to the HUD Secretary and program offices. PD&R is supporting the decision-making and roles of the evaluation officer, chief data officer, and statistical official for the Department.

Did the agency have an evaluation policy, evaluation plan, and learning agenda (evidence-building plan), and did it publicly release the findings of all completed program evaluations in FY19?

2.1 Did the agency have an agency-wide evaluation policy? (Example: Evidence Act 313(d))

• PD&R has published a Program Evaluation Policy that establishes core principles and practices of PD&R’s evaluation and research activities. The six core principles are rigor, relevance, transparency, independence, ethics, and technical innovation.

2.2 Did the agency have an agency-wide evaluation plan? (Example: Evidence Act 312(b))

• A key feature of HUD’s Research Roadmap is a lengthy list of potential research and evaluation projects that feed into proposed evaluation plans that PD&R submits to Congress as part of its budget request. Actual research activities are substantially determined by Congressional funding and guidance.

2.3 Did the agency have a learning agenda (evidence-building plan) and did the learning agenda describe the agency’s process for engaging stakeholders including, but not limited to the general public, state and local governments, and researchers/academics in the development of that agenda? (Example: Evidence Act 312)

• HUD’s Research Roadmap serves as the Department’s evidence-building plan and learning agenda. A new Roadmap is under development in FY19. HUD’s participatory process (see for example pp. 14–16 of Roadmap Update 2017) engages internal and external stakeholders to identify research questions and other evidence-building activities to support effective policy-making. Stakeholders include program partners in state and local governments and the private sector; researchers, academics; policy officials; and the general public frequently accessing HUDuser.gov web portal. Outreach mechanisms include email, web forums, conferences and webcasts, and targeted meetings.

2.4 Did the agency publicly release all completed program evaluations?

• PD&R’s Program Evaluation Policy requires the publishing and dissemination in a timely fashion all evaluations that meet standards of methodological rigor. Completed evaluations and research reports are posted on PD&R’s website, HUDUSER.gov. Additionally, the policy includes language in research and evaluation contracts that allows researchers to independently publish results, even without HUD approval, after not more than 6 months.

2.5 What is the coverage, quality, methods, effectiveness, and independence of the agency’s evaluation, research, and analysis efforts? (Example: Evidence Act 315, subchapter II (c)(3)(9))

• PD&R is HUD’s independent evaluation office, with scope spanning all the Department’s program operations. PD&R’s Program Evaluation Policy defines six core principles for evaluation and research activities: rigor, relevance, transparency, independence, ethics, and technical innovation.

2.6 Did the agency use rigorous evaluation methods, including random assignment studies, for research and evaluation purposes?

• For decades, PD&R has been a federal leader in the use of random assignment and other rigorous methods for research and evaluation purposes. Examples of random-assignment program demonstrations found on HUDUSER.gov include landmark research in the Housing Allowance experiment, the Moving to Opportunity Demonstration, and the Family Options Demonstration. Ongoing random-assignment experiments include the Moving to Work Demonstration, Family Self-Sufficiency Demonstration, the First-Time Homebuyer Education and Counseling demonstration, the Rent Reform Demonstration, and the Integrated Wellness in Supportive Housing Demonstration (See the PD&R Biennial Report FY 2017–2018).

Did the agency invest at least 1% of program funds in evaluations in FY19? (Examples: Impact studies; implementation studies; rapid cycle evaluations; evaluation technical assistance, rigorous evaluations, including random assignments)

3.1 ____ (Name of agency) invested $____ on evaluations, evaluation technical assistance, and evaluation capacity-building, representing __% of the agency’s $___ billion FY19 budget.

• HUD invested $96 million on evaluations, evaluation technical assistance, and evaluation capacity-building, representing 0.18% of the agency’s $53.762 billion FY19 appropriation.

3.2 Did the agency have a budget for evaluation and how much was it? (Were there any changes in this budget from the previous fiscal year?)

• For FY19, Congress appropriated $96 million for the Office of Policy Development and Research’s (PD&R’s) Research & Technology account. FY19 funding was up $7 million from FY18, reflecting congressional support for the value of PD&R’s research, evaluations, and demonstrations. This budget includes $50 million for core research activities; up to $21 million for research, evaluations, and demonstrations; and not less than $25 million for technical assistance. The total represents an FY19 investment in evaluations and evidence amounting to 0.18 percent of HUD’s $53.762 billion gross discretionary budget authority, net of salaries and expenses, for FY19. The funding for core research is used primarily for the American Housing Survey, other surveys, data acquisition, and research dissemination that support evaluation of HUD’s mission activities in domains such as affordable housing and housing finance.

• PD&R’s FY19 appropriation of $26 million for Salaries and Expenses, up $2 million from FY18, also supports evidence in the form of PD&R’s in-house research and evaluation program; economic analyses; data linkage initiatives; and management of housing surveys, contract research, and evaluation.

3.3 Did the agency provide financial and other resources to help city, county, and state governments or other grantees build their evaluation capacity (including technical assistance funds for data and evidence capacity building)?

• For FY 2019, HUD is making available $25 million through the Community Compass NOFA for technical assistance to equip HUD’s customers with the knowledge, skills, tools, capacity, and systems to implement HUD programs and policies successfully and to provide effective oversight of federal funding. State and local governments and authorities are among the eligible applicants.

• HUD operates a Section 4 Capacity Building grant program that funds national intermediaries and rural jurisdictions in building capacity for functions including assessing needs, planning programs, and evaluation.

• HUD’s Community Development Block Grant (CDBG) program, which provides formula grants to entitlement jurisdictions, increases local evaluation capacity. Specifically, federal regulations (Section 24 CFR 507.200) authorize CDBG recipients (including city and state governments) to use up to 20% of their CDBG allocations for administration and planning costs that may include evaluation-capacity building efforts and evaluations of their CDBG-funded interventions (as defined in 507.205 and 507.206).

Did the agency implement a performance management system with outcome-focused goals and aligned program objectives and measures, and did it frequently collect, analyze, and use data and evidence to improve outcomes, return on investment, and other dimensions of performance in FY19?
(Example: Performance stat systems, frequent outcomes-focused data-informed meetings)

4.1 Did the agency have a strategic plan with outcome goals, program objectives (if different), outcome measures, and program measures (if different)?

• HUD’s FY 2018–2022 Strategic Plan defines strategic objectives, priority outcome goals, and program metrics supporting each objective. Progress on program metrics is tracked through HUD’s Annual Performance Plan.

4.2 Does the agency use data/evidence to improve outcomes and return on investment?

• HUD uses data and evidence extensively to improve outcomes and return on investment. The primary means are through PD&R’s investments in data collection, program demonstrations, program evaluations, and research guided by a multi-year learning agenda; HUD’s extensive use of outcome-oriented performance metrics in the Annual Performance Plan; senior staff oversight and monitoring of key outcomes and initiatives through the Prescription for HUD, the Advancing Economic Opportunity Task Force, and the Agency-Wide Integrity Task Force, which bring together senior staff for quarterly performance management meetings. In addition, the Standards for Success pilot is a new standardized data collection and reporting framework for discretionary grant programs.

4.3 Did the agency have a continuous improvement or learning cycle processes to identify promising practices, problem areas, possible causal factors, and opportunities for improvement? (Examples: stat meetings, data analytics, data visualization tools, or other tools that improve performance)

• HUD’s senior staff support continuous improvement and oversight and monitoring of key outcomes and initiatives through the Prescription for HUD, the regular meetings of the Advancing Economic Opportunity Task Force, and the Agency-Wide Integrity Task Force. These processes are supported by ongoing, significant investments in evidence-building as documented in the Annual Performance Plan and the iterative process of developing the Research Roadmap learning agenda.

Did the agency collect, analyze, share, and use high-quality administrative and survey data – consistent with strong privacy protections – to improve (or help other entities improve) outcomes, cost-effectiveness, and/or the performance of federal, state, local, and other service providers programs in FY19?
(Examples: Model data-sharing agreements or data-licensing agreements; data tagging and documentation; data standardization; open data policies; data-use policies)

5.1 Did the agency have a strategic data plan, including an open data policy? (Example: Evidence Act 202(c), Strategic Information Resources Plan)

• As HUD implements the DATA Act and Evidence Act, new open data policies, accountability principles, and automated reporting are being developed and implemented during FY19 (see FY20 Annual Performance Plan). HUD maintains a vigorous open data program including administrative datasets on data.hud.gov, spatially enabled data on the eGIS portal, PD&R datasets for researchers and practitioners, a robust partnership with the Census Bureau, U.S. Postal Service vacancy data, and health data linkages with the National Center for Health Statistics.

5.2 Did the agency have an updated comprehensive data inventory? (Example: Evidence Act 3511)

• HUD has an Inventory Schedule to identify tasks and timelines for engaging program offices to identify data assets, assess whether data can be made available to the public, and create metadata and guidance for shareable data. At present, the Enterprise Data Inventory remains incomplete while the Enterprise Data Management Policy is updated during FY19.

5.3 Did the agency promote data access or data linkage for evaluation, evidence-building, or program improvement? (Examples: Model data-sharing agreements or data-licensing agreements; data tagging and documentation; data standardization; downloadable machine-readable, de-identified tagged data; Evidence Act 3520(c)) 

• HUD has an updated list of open data assets; numerous PD&R-produced datasets for researchers and practitioners, including tenant public use microdata samples; and an eGIS portal providing geo-identified open data to support public analysis of housing and community development issues using GIS tools.

• PD&R has data linkage agreements with the National Center for Health Statistics and the Census Bureau to enhance major national survey datasets by identifying HUD-assisted households; making available major program demonstration datasets in secure environments; and to produce special open-access tabulations of census data for HUD’s partners.

• PD&R engages in cooperative agreements with research organizations, including both funded Research Partnerships and unfunded Data License Agreements, to support innovative research that leverages HUD’s data assets and informs HUD’s policies and programs. Data licensing protocols ensure that confidential information is protected.

5.4 Did the agency have policies and procedures to secure data and protect personal, confidential information? (Example: differential privacy; secure, multiparty computation; homomorphic encryption; or developing audit trails)

• HUD’s Evaluation Policy specifies that HUD protects client privacy by adhering to the Rule of Eleven to prevent disclosure from tabulations with small cell sizes.

• Data licensing protocols ensure that researchers protect confidential information when using HUD’s administrative data or program demonstration datasets.

• HUD has an interagency agreement with the Census Bureau to link administrative data from HUD’s tenant databases and randomized control trials with the Bureau’s survey data collection and other administrative data collected under the privacy protections of its Title 13 authority. These RCT datasets are the first intervention data added to Federal Statistical RDCs by any federal agency, and strict protocols and review of all output ensure that confidential information is protected.

5.5 Did the agency provide assistance to city, county, and/or state governments, and/or other grantees on accessing the agency’s datasets while protecting privacy?

• HUD has an updated list of open data assets, an open data program, numerous PD&R datasets for researchers and practitioners, and an eGIS portal providing geo-identified data to support public analysis of housing and community development issues related to multiple programs and policy domains using GIS tools. These accessible data assets have privacy protections. Researchers needing detailed microdata can obtain access through data licensing agreements.

• HUDExchange offers numerous resources and training opportunities to help program partners use data assets more effectively. Additional technical assistance is offered through the Community Compass program, a $25 million technical assistance program to equip HUD’s customers with the knowledge, skills, tools, capacity, and systems to implement HUD programs and policies successfully and provide effective oversight of federal funding.

Did the agency use a common evidence framework, guidelines, or standards to inform its research and funding purposes; did that framework prioritize rigorous research and evaluation methods; and did the agency disseminate and promote the use of evidence-based interventions through a user-friendly tool in FY19?
(Example: What Works Clearinghouses)

6.1 Did the agency have a common evidence framework for research and evaluation purposes?

• PD&R’s Program Evaluation Policy defines standards that prioritize rigorous methods for research and evaluation covering impact evaluations; implementation or process evaluations; descriptive studies; outcome evaluations; and formative evaluations; and both qualitative and quantitative approaches. It also provides for dissemination of such evidence to stakeholders in a timely fashion.

6.2 Did the agency have a common evidence framework for funding decisions?

• HUD seeks to employ tiered evidence in funding decisions by embedding implementation and impact evaluations in funding requests for program initiatives, including major program demonstrations that employ random assignment methods. These include the Moving To Work Expansion demonstration, the Rental Assistance Demonstration, the Rent Reform Demonstration, the Family Self-Sufficiency Demonstration, and the Housing Counseling Demonstration. Such trials provide robust evidence to inform scale-up funding decisions.

• In FY17, HUD developed and piloted a new standardized data collection and reporting framework for its discretionary grant programs called Standards for Success. The framework consists of a repository of data elements that participating programs use in their grant reporting, creating common definitions and measures across programs for greater analysis and coordination of services.

6.3 Did the agency have a user friendly tool that disseminated information on rigorously evaluated, evidence-based solutions (programs, interventions, practices, etc.) including information on what works where, for whom, and under what conditions?

• HUD provides resources and assistance to support community partners in evidence-based practice through the HUD Exchange web portal. PD&R provides the public, policymakers, and practitioners with evidence of what works through the Regulatory Barriers Clearinghouse and HUD USER, which is a portal and web store for program evaluations, case studies, and policy analysis and research.

6.4 Did the agency promote the utilization of evidence-based practices in the field to encourage implementation, replication, and application of evaluation findings and other evidence?

• HUD provides resources and assistance to support community partners in evidence-based practice through the HUD Exchange web portal. PD&R provides the public, policymakers, and practitioners with evidence of what works primarily through HUD USER, a portal and web store for program evaluations, case studies, and policy analysis and research; the Regulatory Barriers Clearinghouse; and through initiatives such as Innovation of the Day, Sustainable Construction Methods in Indian Country, and the Consumer’s Guide to Energy-Efficient and Healthy Homes. This content is designed to provide current policy information, elevate effective practices, and synthesize data and other evidence in accessible formats such as Evidence Matters. Through these resources, researchers and practitioners can see the full breadth of work on a given topic (e.g., rigorous established evidence, case studies of what has worked in the field, and new innovations currently being explored) to inform their work.

Did the agency have staff, policies, and processes in place that encouraged innovation to improve the impact of its programs in FY19?
(Examples: Prizes and challenges; behavioral science trials; innovation labs/accelerators; performance partnership pilots; demonstration projects or waivers with rigorous evaluation requirements)

7.1 Did the agency engage leadership and staff in its innovation efforts?

• HUD has an Office of Innovation led by a Deputy Assistant Secretary. The office organized the five-day Innovative Housing Showcase on the national mall with federal and private sector partners in June 2019 to demonstrate new housing technology and discuss innovation barriers and opportunities. The entire local HUD staff was encouraged to attend and view the innovative technologies.

7.2 Did the agency have policies that promote innovation?

• HUD administers five types of juried Secretary’s Awards to encourage excellence in addressing housing and community development challenges: Public-Philanthropic Partnerships, Opportunity and Empowerment, Healthy Homes, Historic Preservation, and Housing and Community Design. An Innovation in Affordable Housing Competition engages multidisciplinary teams of graduate students in addressing a specific housing problem developed by an actual public housing agency.

• The Office of Innovation is developing prize competitions to stimulate innovation in housing and HUD policy and programs.

7.3 Did the agency have processes, structures, or programs to stimulate innovation?

• HUD established the Office of Innovation in 2019 to advance innovation in several domains. The office managed the 2019 Innovative Housing Showcase and is developing prize competitions to stimulate innovation in housing and HUD policy and programs.

• HUD’s regulation of manufactured housing production is guided by a federal advisory committee, the Manufactured Housing Consensus Committee, to provide increased ability for the industry to produce some of the nation’s most innovative, safe, and affordable housing.

7.4 Did the agency evaluate its innovation efforts, including using rigorous methods?

• PD&R is conducting numerous random-assignment program demonstrations to test new, innovative program models, as described in PD&R’s biennial report and online: the Family Self-Sufficiency Demonstration, First-Time Homebuyer Education and Counseling Demonstration, Pre-Purchase Homeownership Counseling Demonstration, Support and Services at Home (SASH) Demonstration for elderly households, Supportive Services Demonstration for health services in elderly housing, Rent Reform Demonstration, Rental Assistance Demonstration, and the Small Area Fair Market Rent Demonstration.

• HUD also is using random assignment and administrative data linkages to test the impact of education navigators on rates of application for federal student aid by young residents of public housing.

• The Secretary’s Awards competitions are juried by professionals.

• In 2019, PD&R published an independent review of building technology innovation policies, programs, and strategies to increase the impact of federal research and development investments.

Did the agency use evidence of effectiveness when allocating funds from its 5 largest competitive grant programs in FY19? (Examples: Tiered-evidence frameworks; evidence-based funding set-asides; priority preference points or other preference scoring for evidence; Pay for Success provisions)

8.1 What were the agency’s 5 largest competitive programs and their appropriations amount (and were city, county, and/or state governments eligible to receive funds from these programs)?

• In FY19 HUD’s largest competitive programs are: 1) Continuum of Care ($2.2 billion; eligible grantees: state and local governments and coalitions); 2) Lead Hazard Reduction Grants ($234 million; eligible grantees: local governments); 3) Choice Neighborhoods Implementation ($150 million; eligible grantees:  state and local governments); 4) Indian Housing ($200 million; eligible grantees: tribes and tribally designated housing entities); 5) Section 202 Service Coordinators ($90 million; eligible grantees: housing providers).

8.2 Did the agency use evidence of effectiveness to allocate funds in 5 largest competitive grant programs? (e.g., Were evidence-based interventions/practices required or suggested? Was evidence a significant requirement?)

• The Continuum of Care program (CoC) provides homelessness assistance awards on the basis of system performance measures focused on outcomes and evidence of effectiveness. This includes up to 56 points (out of 200) for past “performance related to reducing homelessness” and 4 points for “reallocat[ing] lower performing projects to create new higher performing projects that is based on performance review of existing projects.”

• Lead Hazard Reduction Grants require applicants to demonstrate a strategic approach to address low-income neighborhoods having concentrated lead hazards for children. The grant requires the grantees to use evidence-based lead hazard control methods and meet cost-savings, productivity, and grant compliance benchmarks. The application assigns 13 points (out of 100) based on grantees’ past performance. Past research showing large returns on investment supported HUD’s decision to request a 26 percent increase in program funding for FY20.

• The Indian Housing competitive grant program was established to address issues of overcrowded and physically inadequate housing identified by a PD&R needs assessment completed in 2017, Housing Needs of American Indians and Alaska Natives in Tribal Areas.

8.3 Did the agency use its 5 largest competitive grant programs to build evidence? (e.g., requiring grantees to participate in evaluations)

• As a condition of grant award, all HUD competitive grantees are required to cooperate (p. 5) in any HUD-sponsored research or evaluation studies.

• The Continuum of Care program is supported by the National Homeless Data Analysis Project, which provides communities with resources to improve data collection and consistent reporting about individuals experiencing homelessness to support national Annual Homeless Assessment Reports.

• HUD Lead Paint grantees are required to integrate clearance testing of all housing units treated. Technical studies strengthen detection, evaluation, and control technologies, and rigorous evaluation has demonstrated the large return on investment related to children’s health from controlling lead hazards.

• All HUD-funded programs require recipients to submit, not less than annually, a report documenting achievement of outcomes under the purpose of the program and the work plan in the award agreement for accountability purposes and to build evidence of effective practices in the field.

8.4 Did the agency use evidence of effectiveness to allocate funds in any competitive grant program?

• HUD’s Housing Counseling Grant Program ($43m in FY19) provides counseling services to tenants and homeowners. One of the program’s main objectives is to “Distribute federal financial support to housing counseling agencies based on past performance.” As such, the program allocates seven points (out of 100) for past performance based on the “the positive impacts that an Applicant’s housing counseling services had on clients.” HUD scores this item based on its own performance records.

8.5 What are the agency’s 1-2 strongest examples of how competitive grant recipients achieved better outcomes and/or built knowledge of what works or what does not?

• Continuum of Care programs are the nation’s primary structure for assisting people experiencing homelessness. Over more than a decade, increased CoC effectiveness has been supported by Homeless Management Information Systems and evidence-based funding of increased permanent supportive housing. As a result, the estimated number of chronically homeless individuals in 2018 was 16 percent less than the number in 2010. Following the Interagency Council on Homelessness guidelines, 72 communities and 3 states have effectively ended veteran homelessness.

8.6 Did the agency provide guidance which makes clear that city, county, and state government, and/or other grantees can or should use the funds they receive from these programs to conduct program evaluations and/or to strengthen their evaluation capacity-building efforts?

• As a condition of grant award, all HUD competitive grantees are required to cooperate in any HUD-sponsored research or evaluation studies. Eligible activities using grant funds are limited to those allowed by law. HUD also provides technical assistance to strengthen performance management capacity.

Did the agency use evidence of effectiveness when allocating funds from its 5 largest non-competitive grant programs in FY19?
(Examples: Evidence-based funding set-asides; requirements to invest funds in evidence-based activities; Pay for Success provisions)

9.1 What were the agency’s 5 largest non-competitive programs and their appropriation amounts (and were city, county, and/or state governments are eligible to receive funds from these programs)?

• HUD’s budget contains three large formula grant programs for public housing authorities (PHAs): 1) the Public Housing Operating Fund ($4.6 billion in FY19); 2) the Public Housing Capital Grants ($2.7 billion); and 3) Housing Choice Voucher (HCV) Administrative Fees ($1.8 billion).

• Another two formula grant programs serve cities or tribes: 1) Community Development Block Grant Entitlement/Non-Entitlement ($3.3 billion in FY19); and 2) HOME Investment Partnerships ($1.4 billion).

9.2 Did the agency use evidence of effectiveness to allocate funds in largest 5 non-competitive grant programs? (e.g., Are evidence-based interventions/practices required or suggested? Is evidence a significant requirement?)

• Although the funding formulas are prescribed in statute, evaluation-based interventions are central to each program. HUD used evidence from a 2015 Administrative Fee study of the costs that high-performing PHAs incur in administering a HCV program to propose a new FY17 approach for funding Administrative Fees while strengthening PHA incentives to improve HCV outcomes by providing tenant mobility counseling.

• HUD’s funding of public housing is being radically shifted through the evidence-based Rental Assistance Demonstration (RAD), which enables accessing private capital to address the $26 billion backlog of capital needs funding. Based on demonstrated success of RAD, for FY19 HUD proposed removing the cap on the number of public housing developments to be converted to Section 8 contracts. HUD is also conducting a Rent Reform demonstration and a Moving To Work (MTW) demonstration to test efficiencies of changing rent rules and effects on tenant outcomes.

9.3 Did the agency use its 5 largest non-competitive grant programs to build evidence? (e.g., requiring grantees to participate in evaluations)

• Evidence-building is central to HUD’s funding approach through the use of prospective program demonstrations. These include the Public Housing Operating Fund’s Rental Assistance Demonstration (RAD), the Public Housing Capital Grants’ Rent Reform demonstration, and the Housing Choice Voucher program’s Moving To Work (MTW) demonstration grants. As Congress moved to expand MTW flexibilities to additional public housing authorities (PHAs), HUD sought authority to randomly assign cohorts of PHAs to provide ability to rigorously test specific program innovations.

• Program funds are provided to operate demonstrations through the HCV account, Tenant-Based Rental Assistance. These include the Tribal HUD-VA Supportive Housing (Tribal HUD-VASH) demonstration of providing permanent supportive housing to Native American veterans and the FSS-Family Unification Program demonstration that tests the effect of providing vouchers to at-risk young adults who are aging out of foster care.

9.4 Did the agency use evidence of effectiveness to allocate funds in any non-competitive grant program?

• No examples available.

9.5 What are the agency’s 1-2 strongest examples of how non-competitive grant recipients achieved better outcomes and/or built knowledge of what works or what does not?

• To address a severe backlog of capital needs funding for the nation’s public housing stock, the Rental Assistance Demonstration was authorized in 2011 to convert the properties to project-based Section 8 contracts to attract an infusion of private capital. The 2016 interim report on the RAD evaluation showed that conversions successfully obtained $2.2 billion of private funding, representing a 9:1 leverage ratio. Based on the successes, the limit on the number of public housing conversions was increased to 455,000 units in 2018, nearly half of the stock, and HUD proposed to eliminate the cap in FY19. Additionally, HUD extended the conversion opportunity to legacy multifamily programs through RAD 2.

9.6 Did the agency provide guidance which makes clear that city, county, and state government, and/or other grantees can or should use the funds they receive from these programs to conduct program evaluations and/or to strengthen their evaluation capacity-building efforts?

• Communities receiving HUD block grant funding through Community Development Block Grants, HOME block grants, and other programs are required to consult local stakeholders, conduct housing needs assessments, and develop needs-driven Consolidated Plans to guide their activities. They then provide Consolidated Annual Performance and Evaluation Reports (CAPERs) to document progress toward their Consolidated Plan goals in a way that supports continued community involvement in evaluating program efforts.

• HUD’s Community Development Block Grant program, which provides formula grants to entitlement jurisdictions, increases local evaluation capacity. Specifically, federal regulations (Section 24 CFR 507.200) authorize CDBG recipients (including city and state governments) to use up to 20% of their CDBG allocations for administration and planning costs that may include evaluation-capacity building efforts and evaluations of their CDBG-funded interventions (as defined in 507.205 and 507.206).

In FY19, did the agency shift funds away from or within any practice, policy, or program that consistently failed to achieve desired outcomes?
(Examples: Requiring low-performing grantees to re-compete for funding; removing ineffective interventions from allowable use of grant funds; incentivizing or urging grant applicants to stop using ineffective practices in funding announcements; proposing the elimination of ineffective programs through annual budget requests; incentivizing well-designed trials to fill specific knowledge gaps; supporting low-performing grantees through mentoring, improvement plans, and other forms of assistance; using rigorous evaluation results to shift funds away from a program)

10.1 Did the agency shift funds/resources away from ineffective practices or interventions used within programs or by grantees?

• In recent years HUD has reduced the role of transitional housing in favor of rapid re-housing as a cost-effective alternative for families experiencing homelessness that have low barriers and can benefit from shorter interventions.

• CDBG-DR (Disaster Recovery) is a large and growing program funded by emergency appropriations outside of HUD’s regular budgeting process. In FY 2018, HUD started promoting mitigation activities for disaster-prone communities, allocating $16 billion of the $28 billion in emergency disaster recovery funds for disaster mitigation in previously disaster-stricken communities. By investing in mitigation activities, rather than paying to rebuild existing infrastructure in its previous form, HUD shifted funds in order to help to break the cycle of publicly-funded rebuilding and repeated loss.

10.2 Did the agency shift funds/resources away from ineffective policies used within programs or by grantees?

• Evaluation of the Housing First model of rehousing chronically homeless individuals with serious mental illness supported a policy shift toward first achieving housing stability to provide a platform for social services. Based on such evidence, HUD continues to encourage the use of more cost-effective rapid rehousing approaches combined with increased permanent supportive housing that is integrated with mainstream services provided by HHS, VA, and others.

10.3 Did the agency shift funds/resources away from ineffective grantees?

• HUD grant programs typically provide for recapture of funds that are not committed in a timely fashion, or that remain unexpended after the limits. Effective management by grantees can be especially crucial for timely completion of complex housing development projects, such as with the Capital Fund for public housing and Housing Trust Fund for states. Such funds are reallocated to more effective grantees.

• Preference points used by competitive programs favor grantees that provide evidence of successful outcomes and strategies. The Continuum of Care program awards points that shift funds toward grant applications that have demonstrated better outcomes, that rank and fund better-performing projects, and that take over programs from small and struggling recipients. As noted in the notice of funding: “To encourage CoC mergers and mitigate the potential adverse scoring implications that may occur when a high performing CoC merges with one or more lower performing CoC(s), HUD will award up to 25 bonus points to CoCs that completed a merger…”

10.4 Did the agency shift funds/resources away from ineffective programs? (e.g., eliminations or legislative language in budget requests)

• No examples available.

10.5  Did the agency shift funds/resources away from consistently ineffective products and services?

• No examples available.