Reprinted from JAMA Psychiatry http://archpsyc.jamanetwork.com/article.aspx?articleid=207801
THE HUD-VASH PROGRAM
Through an interagency agreement, HUD allocated funds for approximately 1000 housing vouchers for a program providing housing and case management assistance for literally homeless veterans with psychiatric or substance abuse problems or both.20 Participants were offered priority access to Section 8 housing vouchers administered by local housing authorities. These vouchers authorize payment of a standardized local fair-market rent (established by HUD using surveys of local rents) less 30% of the individual beneficiary’s income.
Professional staff of the VA’s Health Care for Homeless Veterans (HCHV) program,21 to which each experimental HUD-VASH program was linked, identified potential candidates for the program. To participate in the study, each veteran had to agree to a treatment plan involving further participation in case management and other specified services if randomized to either HUD-VASH or case management only. However, once assigned, retention of the voucher was not contingent on participation in treatment.
The case managers linked clients with the local housing authority and facilitated administrative access and use of the voucher. Case managers also eased the transition to independent living by helping clients (1) locate an apartment, (2) negotiate the lease (through face-to-face landlord meetings if the client wished), and (3) furnish and move into their new apartment. The case management model used in HUD-VASH was modified from the Assertive Community Treatment (ACT) model22 and encouraged at least weekly face-to-face contact, community-based service delivery, and more intensive involvement in crisis situations. Case managers, most of whom were experienced social workers and nurses, were also encouraged to provide substance abuse and employment counseling and to facilitate linkage with other VA services. Teams in this study consisted of 3 case managers and an allocation of 50 vouchers, allowing maximal caseloads of 25:1, including 25 case management– only clients. Training and monitoring were conducted with written materials and through conference calls, case reviews, and evaluation forms. There were no on-site monitoring visits.
STUDY DESIGN AND DATA COLLECTION
Eligibility was determined from administrative intake forms documenting housing and clinical status at the time of the initial outreach assessment by the HCHV program.21 After providing written informed consent and completing baseline assessments, 460 veterans were randomly assigned through a telephone call to the central evaluation staff, who identified the next assignment from a deck of cards specific to each site. Veterans were assigned to (1) HUD-VASH (case management plus vouchers) (n = 182), (2) case management only (n = 90), or (3) standard care (n = 188), which consisted of short-term broker case management as provided by HCHV program outreach workers. The randomization was weighted so that half as many veterans were assigned to case management alone as to the other 2 groups. In the case management– only condition, case managers were to provide the same intensity of services as in the HUD-VASH condition and were encouraged to use whatever housing resources could be obtained for their clients. Neither veterans nor staff could be masked to group assignment.
Baseline and follow-up assessment interviews every 3 months were conducted by trained evaluation assistants. The VASH clinicians used (1) structured forms to document their efforts to assist their clients in obtaining their vouchers and apartments and (2) quarterly structured summaries of case management services.
The study took place at VA medical centers in San Francisco, Calif (n= 107); San Diego, Calif (n = 91); New Orleans, La (n = 165); and Cleveland, Ohio (n = 97). Veterans were eligible if they were literally homeless at the time of outreach assessment (ie, living in a homeless shelter or on the streets), had been homeless for 1 month or longer, and had received a diagnosis of a major psychiatric disorder (schizophrenia, bipolar disorder, major affective disorder, or posttraumatic stress disorder) or an alcohol or drug abuse disorder or both.
All veterans provided written informed consent to participate in the study, and the protocol was approved by the human investigations committee at each medical center. Veterans received $20 after each interview.
Recruitment for the study took place between June 1, 1992, and December 31, 1995. During this time, 3489 veterans contacted through outreach at the 4 sites met minimal eligibility criteria, and 460 (13.2%) gave written informed consent to participate in the study. The major reason most veterans did not participate was limited program capacity.
Comprehensive intake data from all HCHV program participants allow detailed comparison of participants and nonparticipants. Compared with other eligible veterans, those who joined the study were slightly younger (42.0 vs 42.8 years, t = 2.2637; P<.03), were more likely to be female (4.2% vs 1.8%, χ21 =11.3; P<.001), were more likely to be African American (64% vs 57%, χ21 = 7.1; P<.008), and had slightly fewer nights of literal homelessness in the 60 days before intake (26.1 vs 27.1 nights, t =2.8; P<.01), but they had a greater likelihood of past hospitalization for drug abuse (49.8% vs 39.5%, χ21 = 17.3; P<.001). They were more likely to have been admitted to the residential treatment component of the HCHV program(29.1% vs 10.9%, χ21 = 119.7; P<.001). Participants thus showed evidence of more severe illness on some measures and greater involvement in treatment.
Demographic and Clinical Characteristics
Data were obtained on current sociodemographic characteristics, duration of the current episode of homelessness, and housing status during the 90 days before each interview. We recorded the number of days in the previous 90 that the client spent in each of 11 different types of housing. The primary outcome measures were the number of days housed in the previous 90 (ie, sleeping in an apartment, room, or house of one’s own or of a family member or friend) and the number of days homeless in the previous 90 (ie, sleeping in an emergency shelter; a substandard, single-room occupancy hotel; or outdoors, on the sidewalk, or in a park, abandoned building, automobile, truck, or boat).
Psychiatric, alcohol, and drug problems were assessed using specific items and composite scores from the Addiction Severity Index.23 Psychological distress was measured using the Brief Symptom Index.24 Diagnoses were based on the working clinical diagnoses of the case management teams. Quality of life was evaluated using selected subscales from the Lehman Quality of Life Interview.25
Among those who were housed, the quality of their residence was further assessed using one scale that addressed positive characteristics of the residence (eg, safety, proximity to shopping, affordability, adequate size, and privacy) and another that measured housing problems (eg, pests, broken windows, neighborhood crime, and plumbing problems).26
Social support was measured by the number of people in 9 different categories to whom the veteran reported feeling close, an index of the total frequency of contacts with these people, and the average number of types of people who would help with a loan, with transportation, or in an emotional crisis.27,28
Five kinds of indicators were used to compare services provided to each treatment group during the study. First, computerized VA workload data were used to measure contacts with the HUD-VASH or HCHV programs. Second, the nature of the therapeutic alliance was measured by a 5-item rating scale completed separately by the clinician (Cronbach α = .85) and the veteran (Cronbach α= .86) using a modified version of the Working Alliance Inventory.29,30 Veterans’ assessments of their alliance with their case managers were self-administered in private and mailed in a separate envelope. Third, data were obtained on the use of Section 8 housing vouchers in all 3 groups and on the initial housing search for those assigned to the HUD-VASH group using a structured activities questionnaire. Fourth, data from the quarterly case manager summaries were used to compare case management services provided to each group. Finally, we measured use of regular VA mental health outpatient services (ie, beyond those from HUD-VASH or HCHV program staff).
ASSESSMENT OF HEALTH CARE COSTS
The VA health care costs were estimated by multiplying the number of units of service consumed by each patient by the estimated unit cost of each type of service using VA cost data from fiscal year 1996 and methods developed previously.31 Unit costs of the HUD-VASH case management were estimated separately using more detailed data on program expenditures and services delivered during a sample year (1996) when the program was fully operational.32
The VA health service utilization data were derived from the VA’s comprehensive national workload data systems: the Patient Treatment File for inpatient care, the Outpatient Care File for outpatient care, and program monitoring data on the delivery of residential treatment through VA contracts with community agencies.
VA Unit Costs
Unit costs for VA inpatient, residential care, and outpatient treatment were estimated on the basis of data from the VA’s Cost Distribution Report, which is a facility-by-facility accounting record that identifies total expenditures and unit costs associated with VA inpatient and outpatient health care services nationwide.31
Non-VA Health Costs
Utilization of non-VA services was documented through quarterly patient interviews, during which was recorded the use of non-VA medical and mental health inpatient, residential, and nursing home care; non-VA medical-surgical outpatient care; and non-VA mental health outpatient care. Non-VA unit costs were estimated from several sources, including analysis of costs in the 1998 MarketScan (The Medstat Group, Ann Arbor, Mich) data set33 and published studies34,35 that identify unit costs in large non-VA health care systems.
NON–HEALTH CARE COSTS
Non–health care costs were also evaluated and used to estimate costs from the perspective of governmental agencies or taxpayers and of society as a whole (total resource consumption). Interview data documented the number of days spent in shelter beds or in jail or prison, cash transfer payments (eg, VA benefits, Supplemental Security Income, and Social Security disability), earnings, and the cost of the Section 8 vouchers. Although cash transfer payments (including housing subsidies) were included in the evaluation of costs from the perspective of governmental agencies, only the administrative cost of these payments was included in societal cost estimates.36 Productivity (employment earnings) was also included in the societal cost estimate, as a negative cost.
Per diem estimates of the cost of shelter and jail days were based on published literature.19 The administrative cost of the Section 8 vouchers were estimated by multiplying the number of months each veteran received a Section 8 subsidy during 3-year follow-up by HUD’s estimated monthly administrative cost for the Section 8 program at each locality.37 The value of housing subsidies received by program participants was calculated by subtracting 30% of monthly income from reported monthly rent among clients who identified themselves as Section 8 beneficiaries. This figure was only included in the cost estimate from the perspective of the government.
First, we evaluated the effectiveness of the randomization by comparing baseline characteristics of clients randomly assigned to each of the 3 groups. Next, we compared housing procurement processes and delivery of case management services across the 3 groups to determine whether the intended differences in access to housing subsidies and case management were achieved.
Third, housing and clinical outcomes across the groups were compared. The follow-up periods selected for analysis were baseline and 6, 12, 18, 24, and 36 months, and all interviews conducted during each interval were included. Because we planned to compare the groups during 5 intervals after the baseline assessment, we used a repeated-measures with mixed-effects analytic strategy. This method was chosen to allow use of all available data from each participant during each follow-up interval. Using hierarchical linear modeling, we modeled random effects for each participant to adjust standard errors for the nonindependence of observations within participants.38 The repeated-measures mixed-effects model approach was chosen because it allows(1) comparison of each experimental group (HUD-VASH and case management only) with the standard care group at each specific point and (2) comparison of groups averaged across all points (ie, area under the curve). These analyses were conducted using PROC MIXED of SAS version 8.0 (SAS Institute Inc, Cary, NC), with α<.05.
Differential Follow-up Rates
Comparison across groups showed significant differences in follow-up rates within each assessment period from the 6-month assessment (χ22 = 10.3; P<.006) through the 3-year assessment (χ22 = 55.1; P<.001), with higher follow-up rates in the voucher plus case management group (146 [80%], 153 [84%], 142 [78%], 140 [77%], and 127 participants [70%] at 6 months and 1, 1.5, 2, and 3 years, respectively) than in the case management– only group (59 [66%], 62 [69%], 66 [73%], 55 [61%], and 43 participants [48%]); follow-up rates were even lower in the standard care group (126 [67%], 113[60%], 111 [59%], 92 [49%], and 75 participants [40%]). Two strategies were used to address the potential bias from data loss. First, patients actually followed up at each point were compared on baseline characteristics, and measures for which significant differences were identified were included as covariates in subsequent analyses.
Second, marginal structural modeling was used to inversely weight observations from patients on the basis of their likelihood of being followed up.39,40 In this approach, survival analysis is first used to model the likelihood that each observation for each participant will be available for analysis. The predicted probabilities are then used to inversely weight each observation so that available observations from clients with characteristics similar to those who were not followed up are given a greater weight. Since the results did not differ substantially across these analytic strategies, we present data from the analysis that adjusted for baseline measures that were significantly different among interviewed groups at any point and dichotomous dummy codes representing 3 of the 4 sites.
Incremental cost-effectiveness ratios were used to compare the cost-effectiveness of each of the experimental conditions with standard care41 from each of 4 cost perspectives: the VA, the total health care system (VA and non-VA), the government (or taxpayers), and society as a whole. The incremental cost-effectiveness ratio is the ratio of the difference between groups in costs to the difference in effectiveness. The 95% confidence interval (CI) of the combined incremental cost-effectiveness ratio was computed using the methods described by O’Brien et al.42 In addition, cost-effectiveness acceptability curves were constructed using recently published methods43 for analyzing net benefits44 in which the probability that benefits equal costs is plotted across a range of possible shadow prices for a day of independent housing.
These analyses require complete data for all participants during 3-year follow-up. Although complete data were available for VA health costs, some data were missing on housing outcomes and non-VA resource use. We used multiple regression models to impute missing data. First, a series of models were estimated in which housing outcomes or service use were the dependent variables and measures of baseline clinical status and dichotomous dummy-coded variables representing treatment group and time interval were the independent variables. These models were then used to generate estimated values of these dependent variables for each client at each point, which were used to impute missing data.
To examine longitudinal time trends in costs, we conducted a series of analyses of variance comparing VA health care costs among groups at 6-month intervals from the year before randomization to 3 years after. We used VA costs for longitudinal analysis because no data are missing and they account for 77% of all health costs (range, 73%-82% across groups).