Research & Evaluation

Eagle helps clients improve population health and wellbeing through deep knowledge of public health, medical and social science theory; extensive program expertise; and sophisticated data analytics; and fluency with the methods for evaluating those programs.
Eagle’s researchers utilize advanced theory and methods to uncover the important issues and trends facing government decision-makers. Eagle’s evaluation scientists employ evidence-based practices to reveal the effectiveness of programs and policies at improving the lives of individuals, families, communities, and the nation.
Eagle offers the full lifecycle of research and evaluation services, from study design to dissemination. Our domain expertise encompasses behavioral health, chronic disease, COVID-19, gerontology, health services, military wellbeing, early childhood, education, and Health IT.

Capabilities

Eagle helps clients improve population health and wellbeing through deep knowledge of public health, medical and social science theory, extensive program expertise, sophisticated data analytics, and fluency with the methods for evaluating those programs.

Study Design

We use our subject matter expertise to choose the most powerful study design to answer clients’ questions—while staying within the available resources.

Data Collection

When designing a data collection strategy for our clients, we carefully consider the most appropriate mode of data collection given the study population, budget, and the available staff. This may include using surveys, interviews, focus groups, or observations, with data collected on-site or via the internet or telephone, as needed.

Qualitative Data Analysis

We assist government clients in understanding the experiences of those who use their programs and services by employing a range of analytical approaches, including content analysis, narrative analysis, grounded theory, and natural language processing.

Quantitative Data Analysis

We help clients draw reliable conclusions from their data through expert statistical analysis, modeling, forecasting, and simulation. We also offer a full range of predictive analytics.

Visualization

We help clients uncover the stories within their data by presenting them with clear, easy-to-understand data stories, infographics, interactive dashboards, and other visualizations.

Reporting

We provide clients with clear, comprehensive reports, peer-reviewed publications, white papers, and presentations to help them communicate their findings.

Technical Assistance

We promote the fidelity, consistency, and effectiveness of our evaluations by incorporating expert training and technical assistance.

Policy Analysis

To help clients assess and decide on complex policy issues, we conduct a range of policy analyses, including effectiveness, cost, cost-benefit, and cost-effectiveness analyses.

Cultural Competence

We prioritize sensitivity, respect, and inclusivity in all our research and evaluation processes to ensure that the voices of vulnerable individuals are heard. Our cultural competence practices align with the general statement on cultural competence for Domain 1 of the SAMHSA IDIQ.

Experience

Department of Health and Human Services (HHS), Substance Abuse and Mental Health Services Administration (SAMHSA)

  • Eagle analyzed the growth of telemedicine for mental healthcare in relation to the level of rurality/urbanicity and Health Professional Shortage Area designations before and after the pandemic. The findings showed that telemedicine has effectively met the care needs of both rural and urban areas with a shortage of mental health professionals.
  • In partnership with Mathematica, Eagle examined trends in access to Medication Assisted Treatment (MAT) for opioid use disorder by geography, facility type, and other major factors over time. The research revealed that MAT, the most effective treatment for opioid use disorder, is underutilized in many parts of the country.
  • We evaluated the feasibility of linking two data sources respectively containing facility- and patient-level healthcare information. The research community was interested in linking the two data sources and repeatedly asked SAMHSA to provide the linked data set, but our analysis found that the data sets could not be linked without compromising the accuracy and overall quality of the two data sets’ data. Based on our recommendations, SAMHSA decided not to link the data sources.
  • Eagle identified the factors that contribute to inconsistency in substance use and mental health treatment facilities’ completion of federal behavioral healthcare surveys from year to year. We used these findings to provide recommendations to SAMHSA on how to achieve consistent survey completion by facilities over time and maximize response rates.
  • We conducted a mixed methods study on mental health treatment facilities’ experiences completing a lengthy, quantitative annual survey and developed recommendations to improve the process and minimize item nonresponse. Our qualitative analysis of the thousands of comments from respondents was particularly helpful in informing the client’s policies for future surveys.

Department of Defense (DoD)

  • We studied the variation in the trajectories of clinical depressive symptoms among National Guard couples before, during, and after military deployment. Our research was based on social science theory, and it helped the Department of Defense (DoD) understand the nuances in the course of depression as it manifests across individual service members over time.

Success Stories

POLICY EVALUATION: The effects of timely follow-up care on psychiatric inpatient readmissions

The Need: A widely used quality measure recommends that psychiatric inpatients receive follow-up outpatient care within 30 days of discharge, but there was little evidence of the policy’s effectiveness. Federal decisionmakers wanted to know if this policy reduced readmissions among inpatients with serious mental illnesses (SMI), including major depression, bipolar disorder, and schizophrenia.

Our Response: Eagle created a multi-state, longitudinal data set of SMI inpatient and outpatient care over a six-year period. We compared demographic and clinical variables between patients who received timely outpatient care and those who did not. We also analyzed whether patients were less likely to be readmitted within six months if they received outpatient care within 30 days of discharge and used machine learning models to identify predictors of readmission.

Results: Our evaluation found that less than 10% of patients received the recommended outpatient care, while more than 15% of patients were readmitted within six months, with higher rates in state-operated programs. Patients with bipolar disorder or schizophrenia were no less likely to be readmitted if they received timely outpatient care, and patients with major depression were more likely to be readmitted if they received timely follow-up care. The results were poorest for state-operated programs, which seemed to discharge patients too soon.

Significance: The findings do not support the effectiveness of timely follow-up care in reducing readmissions. However, the lack of benefit may have been due to inadequate lengths of stay in state-operated facilities.

TREATMENT EFFECTIVENESS: Reducing homelessness with mental health treatment

The Need: Mental illness is common among homeless individuals and can prevent them from obtaining employment and income stability. Federal decisionmakers wanted to know if receiving mental health treatment impacted the housing status of patients experiencing homelessness.

Our Response: Using data from almost all states, Eagle created a four-year longitudinal data set of patient demographics, mental health treatment, and housing status at the end of each year. We analyzed whether patients who received more than one year of treatment were more likely to become housed within four years, compared to patients who received only one year of treatment.

Results: We found that the odds of becoming housed were significantly higher for all groups of patients who received more than one year of care, with the greatest improvements seen in patients with serious mental illness (major depression, bipolar disorder, or schizophrenia). In addition, the odds of becoming housed increased with each additional year of treatment received, with the odds of living in a private residence increasing 273% for patients who received four years of mental health treatment.

Significance: The findings suggest that regular mental health treatment is effective at reducing homelessness.

MILITARY WELLBEING AND HIGHER EDUCATION: Supporting Women Student Service Members/Veterans in Higher Education

The Need: Challenges such as military experience, trauma, mental health issues, and military sexual trauma can affect military-connected women’s academic functioning and satisfaction with their academic persistence. The Focus Forward program was designed to promote the persistence to graduation and career attainment of women service members engaged in higher education. In collaboration with a university, we aimed to determine the effectiveness of the program in achieving its short-term goals.

Our Response: We assessed participants’ knowledge of personal strengths, integration of their military and civilian identities, resume preparation, and networking skills.

Results: Within one year of starting the program, fellows demonstrated growth in their knowledge of personal strengths, identity integration, resume building, and networking skills.

Significance: The study highlighted the usefulness of targeted, evidence-informed support services for women service members and veterans engaged in higher education.

VULNERABLE POPULATIONS: Disparities in mental healthcare for persons experiencing homelessness

The Need: Accurate and stable diagnosis of serious mental illness (SMI)-related disorders benefits patients, providers, and the health service payment system. However, obtaining a repeatable diagnosis, known as diagnostic stability, is particularly challenging for homeless patients, who often lack continuity of healthcare and may present with complicating symptoms such as substance use and insomnia. SAMHSA wanted to understand these issues to guide providers in caring for this vulnerable and hard-to-treat population.

Our Response: We compared the diagnostic stability between homeless and privately-residing patients at one-year follow-up for each SMI-related disorder.

Results: We found that diagnostic stability was significantly lower among homeless patients than privately-residing patients with bipolar disorders, depressive disorders, and schizophrenic disorders.

Significance: The findings highlight the need for providers to consider factors that may hinder diagnostic stability among homeless patients.