2015 Beneficiary Satisfaction Survey Briefing Report to Employment - - PowerPoint PPT Presentation

This communication is printed, published or produced and disseminated at U.S. taxpayer expense.

2015 Beneficiary Satisfaction Survey Briefing Report to Employment Networks

October 2, 2017

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

37%

2

Presentation Overview

• Purpose
• Methods
• Results
• Quantitative
• Qualitative
• Conclusions
• Your EN Report Card
• Questions

3

Purpose

• Ticket to Work (Ticket) is a Social Security program for adults age 18 through 64 who receive Social

Security Disability Insurance (SSDI) benefits and/or Supplemental Security Income (SSI) payments due to disability or blindness.

• It offers these individuals choices for receiving employment services or other supports

necessary to obtain their vocational goal.

• Beneficiary Satisfaction Survey – conducted periodically to assess Ticket beneficiaries’ or their

representative payees’ satisfaction with their Employment Network (EN) services

• Gauge the needs and expectations of beneficiaries to effectively tailor the services being

provided by ENs.

• Provide satisfaction results via the program’s website to help beneficiaries make informed

choices in selecting an EN.

• Conduct a multi-method analytic approach to achieve the goals of the survey and provide a

comprehensive understanding of the beneficiaries’ experience with their ENs.

4

Methods

4

5

18 Months From Survey Development to Final Report

Survey Development (Feb – May 2016) Data Preparation (June – Aug 2016) Sampling (Sept 2016) Data Collection (Sept – Nov 2016) Data Analysis (Dec 2016 – Mar 2017) Report Writing (Dec 2016 – July 2017)

6

Quantitative Results

6

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

3%

7

Who Responded to the Survey?

• 6,342 beneficiaries representing 372 Employment Networks (ENs)
• Response rate of 21.4%

○ N is sufficient to address the analysis questions

Assigned Unassigned

Assigned: Beneficiaries who had assigned their Ticket to an EN for at least 3 months in 2015 Unassigned: Beneficiaries who had unassigned their Ticket in 2015

Assignment Status Receipt of EN Services

85% 15% Currently receiving services from an EN Used an EN in the past, but no longer Never received services from an EN

26% who “never received EN services” are likely due to misunderstanding of the survey question and those who had a Ticket, but never actually received services.

26% 37% 37%

8

EN Utilization

22.2

5 10 15 20 25

EN no longer in business Not happy with the job my EN found EN decided not to work with me Decided not to work at this time Transportation issues Worried about losing Medicare or Medicaid Worried about losing SSI or SSDI benefits Not happy with EN EN not able to find me a job Other Disability or health issues

PERCENT

Usual Communication Methods Reasons for Stopping Service with EN

1.8 3.3 3.4 3.8 4.8 5.8 7.5 9.6 12.8 20.0

41.1 30.0 28.7 22.3 20.6 16.7 5 10 15 20 25 30 35 40 45 Phone - I call my EN In-person Phone - EN calls me Online - I send e-mails to EN Online - My EN e-mails me Other

Common “Other” reasons: “I haven’t received Common “Other” methods: Postal mail, services in a while,” “I want to get a Ticket again.” text message, and fax machine.

5.1 6.5 7.0 10.4 11.1 12.6 12.7 14.0 16.0 22.2 22.5

9

Reasons for Selecting EN

Staff were nice, easy to talk to, and

26.8

knowledgeable Closest to where I live

EN Selection

Willing to address my employment needs Staff understood how to meet my disability- related need The EN contacted me 50.5 26.0 11.3 6.3 2.0 0.6 3.1 10 20 30 40 50 60 PERCENT Number of ENs

Number of ENs Contacted Before Selecting

1 2 3 4 5 6 or more Only choice where I live Recommended by caseworker or provider Recommended by another person Recommended by vocational rehabilitation counselor Only EN that returned my call Other reason Recommended by another person who receives benefits

Majority of beneficiaries did not contact

10 20 30 PERCENT

any ENs before choosing one.

Common “Other” reasons: EN reputation, ENs accepting new clients, additional benefits such as incentives, and specific services, classes, and/or skills training offered.

My EN's ability to help me reach my financial goals My EN's understanding about types of jobs I want

10

Satisfaction with EN Experiences

• Assigned and Unassigned beneficiaries

had significantly different satisfaction results.

• Assigned beneficiaries reported

higher levels of satisfaction for every experience, and unassigned beneficiaries reported higher levels

• f dissatisfaction for every

experience.

• Given that unassigned beneficiaries

likely unassigned their Ticket because they weren’t happy with their EN in some way, these results make sense.

25.9 25.6 24 24.3 21.3 22.8 20.7

10.6

38.3 30.7 32.3 30.9 32.2 26.6 26.3 22.1 35.8 43.7 43.7 44.8 46.4 50.6 52.9 67.2

20 40 60 80 100

My EN's ability to help me reach my employment goals The information my EN gave me about resources to reach my employment goals The information my EN gave me about available jobs in community Overall How my EN supported me How my EN respected me

PERCENT

Dissatisfied Neither Satisfied My EN's understanding about types

• f jobs I want

My EN's ability to help me reach my financial goals

Satisfaction with EN Experiences

11

Goals for Money Earned

Fewer beneficiaries agreed that they wanted to replace all their benefits, compared to wanting to replace some of their benefits

37.1 24.4 39.7 41.0 23.2 34.6 5 10 15 20 25 30 35 40 45 Replace Some Replace All PERCENT Agree Neither Disagree

Goals to Replace Some/All Benefits Goals, by Assignment Status

Replace some* Replace all 32.6 21.8 24.9 10 20 30 PERCENT 37.9 40 Assigned Unassigned *Significant at .05 level

Assigned beneficiaries were more likely to want to replace some/all of their benefits than unassigned beneficiaries.

Other Location

12

Evaluation of EN Characteristics

• 7 of the 11 characteristics were rated as

majority “good” by beneficiaries.

• Comparing “bad” ratings from 2014 to 2015,

most improvement in: business hours, staff support, how quickly staff returned phone calls

• “Good” Other examples: EN helpfulness/

friendliness, support/encouragement, personal attention, resourcefulness

• “Bad” Other examples: Understanding of job

needs, provision of job listings, keeping the beneficiary informed, equipment needs (computers, headsets)

35.6 44.2 45.4 47.6 55.3 57 58.5 60.8 62.6 68.3 73 8.6 28.9 27.6 29.2 27.4 26.4 27.5 27.4 23.7 25.2 18.9 55.8 26.9 27 23.2 17.3 16.6 14 11.8 13.7 6.5 8

20 40 60 80 100

Local job information Job search help Time waiting for follow-up services How quickly staff returned phone calls

• r emails

Staff support Staff knowledge Information provided in accessible formats Business hours Physical accessibility

PERCENT

Good Fair Bad My EN's understanding about types of jobs I want My EN's ability to help me reach my financial goals

Evaluation of EN Characteristics

Services Wanted, Received, and Hoped For

• Most wanted: Help finding a job and

job planning

• Most received: Help understanding

how having a job can affect benefits and building a resume

• Most hoped for: Help getting

services from other organizations and help learning a skill

13

3.6 23.8 25.1 23.1 31.7 27.0 25.7 27.7 22.6 24.4 1.3 26.1 17.4 36.5 16.9 28.4 18.6 32.6 30.8 4.0 28.3 25.7 33.9 31.1 30.3 37.8 30.5 29.6 36.4 10 20 30 40

Other Job interview preparation Help getting accommodations Help understanding how having a job could affect my benefits Help getting services from other

• rganizations

Coaching on how to perform job duties with my disability/disabilities Help finding a job Help learning a skill Help building my résumé Job planning

PERCENT

Wanted Received Hope to receive

22.3

Services Wanted, Received, and Hoped for

14

Beneficiary Employment

4.4 8.2 11.3 24.5 16.8 11.2 4.5 16.8 2.3 5 10 15 20 25 30 1-5 6-10 11-15 16-20 21-25 26-30 31-35 36-40 41+ Percent

Hours worked per week

47% employed, 53% unemployed (no change from the 2014 survey)

Hourly Wage

0.9 57.0 30.6 7.1 3.2 1.2 10 20 30 40 50 60 $1-5 $6-10 $11-15 $16-20 $21-30 $31+ PERCENT Hourly Wage

Of those who work:

• Over three-quarters work 30 hours/week or less
• ACA defines full-time work as 30 hours or

more per week

• Average hourly wage is $11.99
• Over half made $6–10 per hour

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

15

Annual Salary & SGA/TWL

8.9 27.1 33.7 17.2 6.1 4.2 2.8 10 20 30 40 $1-5,000 $5,001-10,000 $10,001-15,000 $15,001-25,000 $25,001-35,000 $35,001-50,000 $50,000 and up Percent

Annual Salary

Less than SGA, 87.0% At least SGA, 13.0%

SGA

Less than TWL, 75.2% At least TWL, 24.8%

TWL

A vast majority are not engaging in SGA and are On average, employed beneficiaries made not making enough to meet the TWL amount. $15,555 per year

• 87% make less than SGA ($1,130/month)
• 75.2% make less than TWL ($810/month)

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

16

Yes 35.1% No 64.9% Yes 39.2% No 60.8%

ENs Help Respondents to Get/Keep their Current Job

Get Current Job Keep Current Job

The majority of beneficiaries did not receive help from their EN in getting their current job. Slightly more indicated that their EN helped them keep their current job, but still a majority did not receive this assistance from their EN.

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

17

Working 30 hours/week or more

Ever worked ≥ 30 hrs./week

Yes 64.4% No 35.6%

Interest in working ≥ 30 hrs./week

Yes 56.3% No 43.7%

Majority have ever worked ≥ 30 hours in their life, but only slightly more than half have interest in still doing so.

2.7 4.4 5.6 8.1 10.1 10.6 13.5 13.6 14.3 20.3 28.4

10 20 30 Problems with supervisor or coworker Terminated or laid off from previous job Lack of reliable transportation Only work I could find I want to see if I can work, so I'm starting off with a few hours Only work that is available Afraid of losing Medicare or Medicaid benefits Afraid of losing SSI or SSDI benefits Physical accessibility reason Other Health or disability reasons PERCENT

Reasons for not working ≥ 30 hrs./week

Examples of “Other” reasons: school/training, over age 65 (lost eligibility), volunteering instead of/in addition to working, self-employment efforts, caring for a family member

18

Qualitative Results

18

19

Qualitative Results

• Beneficiaries were asked to add any additional information they

wanted to share with SSA.

• Response rate of 47.6%
• Qualitative analysis resulted in 23 themes that were organized into 4

categories with implications for ENs

• Positive EN experiences
• Challenges with ENs
• SSA/Ticket program considerations
• Beneficiary-specific comments

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

20

Positive EN Experiences

• Communication*
• Employment Placement*
• Staff Support*
• Training for Beneficiaries*
• Navigating Ticket Program and Benefits*

*Also emerged as challenges Many positive comments around staff willingness to help beneficiaries, responsiveness to beneficiary contact attempts, and assistance in getting beneficiaries setup with the Ticket program. Other positive comments were job placement or job skills-specific, where beneficiaries were able to

• btain a job or have begun taking

steps toward that end, as a result

• f the Ticket program.

“The Ticket to Work program really helped me when I needed it. It prepared me for my job and my SSDI benefits that would be affected.” “I love my Employment Network they helped me to find an awesome job and restore my pride. I love everyone for all they have done for me.” “My employment network helped me find training which led to me finding a job. They continue to give me support.”

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

21

Challenges With ENs

• Communication*
• Employment Placement*
• Staff Support*
• Training for Beneficiaries*
• Confusion About Ticket Program and Benefits*
• EN Location
• Lack of Jobs With Disability

Accommodations

• Perceived Staff Competence

*Also emerged as positive EN experiences “I am in a wheelchair. Looking for jobs that do not require mobility…and I was denied… The jobs I was offered were all physical like maintenance and kitchen work.”

Many negative comments also focused around staff – their lack of support or knowledge – as well as beneficiaries’ discontent with their job-related outcomes. Several commented that their EN didn’t find them appropriate jobs, or any jobs at all. There was also confusion around the logistics of the Ticket program – how to enroll, responsibilities of beneficiaries, how to keep your assignment, etc.

“After those first 6 months I never got another call from the company…I’ve tried to find another company but haven’t been able to get a return call from any other nearby agencies.”

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

22

SSA/Ticket Program Considerations

• Fear of Losing Government Benefits
• SSA Rules and Regulations
• Ticket Program Design
• Ticket Program Goals

“I joined The Ticket too work program as Therapy for my disabilities and was surprised [to]find that SSDI wanted to discontinue my benefits so…I had to quit also so not too lose my benefits.” Several considerations around specifics of the Ticket program itself or SSA-related questions. Many wanted to work (or work more), but were concerned about losing their benefits and didn’t feel comfortable with the knowledge they were receiving about this topic. Also some confusion around the design of the Ticket program, from beneficiary enrollment to ongoing involvement. “[I] find it easier to find employment on my own; too many limitations with this program.” “I was not working a job that gave me a better life than what I would get by staying on SSD. In fact, after taxes I was making less money.”

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

23

Beneficiary-Specific Comments

• Not Linked to EN Satisfaction
• Criminal History
• Disability/Health Limitations
• Linked to Employment Satisfaction
• Current Employment Satisfaction
• Transportation Needs

Some beneficiaries faced challenges finding work due to the beneficiary’s criminal history, as well as difficulty in working due to health or disability limitations. Several beneficiaries reported that they are currently working and are happy with the EN services they

• received. Others commented that

lack of reliable transportation has caused issues related to EN services or job placement. “I am a good hard worker and want to work and the criminal activity is behind me by several years. I even have a diploma as a Medical Office Professional and can’t get a job in the medical field.” “I enjoy my employment, because it gives me a lot of willpower to carry on because of the fact that I have a terminal, but manageable illness.” “I couldn’t find a job so I just gave up—the transportation was not available—there’s no job on the bus line.”

24

Conclusions

24

25

Conclusions

A comprehensive synthesis of the qualitative and quantitative data resulted in the following conclusions:

• While satisfaction with EN staff’s treatment of beneficiaries is high, satisfaction with

the actual job-related services provided by the ENs needs to be addressed including:

• Understanding of the beneficiary’s unique job needs based on their disabilities,

and job placement opportunities that fit within those needs.

• Confusion exists around the Ticket program design, implementation, and rules and

regulations.

• Disconnect exists on goals of program, as only 37% of respondents reported wanting

to replace some of their SSA benefits; even less (24%) reported wanting to replace all of their benefits

26

Your EN Report Card

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

27

EN Report Card Methodology

• A breakout analysis was performed on all ENs for which a minimum of 10

assigned beneficiaries responded to calculate an “Index Score”

• A total of 142 ENs met this threshold
• Index Scores were calculated for each of the eight satisfaction questions

included in the survey:

8 Satisfaction Questions Overall Satisfaction Information EN Gave About Available Jobs in the Community EN Understanding Information EN Gave about Resources/Organizations to Help Reach Employment Goals EN Support EN Ability to Help Reach Employment Goals EN Respect EN Ability to Help Reach Financial Goals

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

28

EN Report Card Methodology

• Beneficiaries rated each of the eight satisfaction measures as “Satisfied”

(value = 3), “Neither Satisfied nor Dissatisfied” (value = 2), or “Dissatisfied” (value = 1)

• Then, we determined the total average score for a satisfaction measure across

all ENs by calculating the mean of the response values (1, 2, or 3) for all beneficiaries who answered that question. The result is the “all EN average.”

• Next, we determined the average score for a satisfaction measure for an

individual EN by calculating the mean of the response values (1, 2, or 3) only for beneficiaries from that EN. The result is the “individual EN average.”

• And then, we calculated the relative percentage between the all EN average

and the individual EN average to determine what percentage above or below the “all EN average,” the “individual EN average” was positioned.

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

29

EN Report Card Methodology

• We translated this percentage into an index score with 100 as its base.
• The Index Score of each satisfaction question indicates the relative position of

an EN compared to all other ENs.

• The number of responses associated with each EN had no bearing on the

index score.

• The Index Score was calculated using the same method as in past

satisfaction surveys

30

EN Report Card Reporting: Index Scores

• The average of all beneficiaries across all ENs is 100.0. Index scores

lower than 100.0 indicate lower levels of satisfaction, while scores above 100 indicate higher levels of satisfaction.

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

31

EN Report Card Reporting: Index Scores

• Here’s how to interpret an EN’s Index Scores for each of the eight satisfaction

questions:

• If the “individual EN average” equals the “all EN average,” then its index

score is 100.

• If the “individual EN average” is 10 points above the “all EN average,” then its

index score is 110.

• If the “individual EN average” is 5 points below the “all EN average,” then its

index score is 95.

• If the individual EN’s Index Score is greater than or equal to 110, then the score is

labeled EXCELLENT. If it’s greater than 90 and less than 110, then the score is labeled

• GOOD. If the EN’s Index Score is less than or equal to 90, then the score is labeled

FAIR.

zyxwvutsrqponmlkjihgfedcbaZYXWVUTSRQPONMLKJIHGFEDCBA

32

“Customer Satisfaction” Data Table in Find Help

Satisfaction Indicator (“Satisfaction with…”) EN Score National Average Compared to National Average

How my EN supported me 2.55 2.36 Good How my EN respected me 2.71 2.59 Good The information that my EN gave me about available jobs in my community 2.26 2.24 Good The information that my EN gave me about other resources or organizations to help me reach my employment goals 2.32 2.23 Good My EN’s ability to help me reach my employment goals 2.36 2.22 Good My EN’s ability to help me reach my financial goals 2.32 2.14 Good My EN’s understanding about the types of jobs I want 2.43 2.28 Good Overall satisfaction with my EN * 2.50 2.32 Good

This comparison is based on the National Average Index Score of 100. The scale is:

• Excellent
• Good
• Fair

* Overall satisfaction with EN was asked as a separate question and is not a total of the

• ther 7 satisfaction indicators.

33

Questions?