BUILDING A CROWDSOURCING PROGRAM at FDA FROM MARKET RESEARCH TO A - - PowerPoint PPT Presentation

building a crowdsourcing program at fda
SMART_READER_LITE
LIVE PREVIEW

BUILDING A CROWDSOURCING PROGRAM at FDA FROM MARKET RESEARCH TO A - - PowerPoint PPT Presentation

BUILDING A CROWDSOURCING PROGRAM at FDA FROM MARKET RESEARCH TO A SYSTEMATIC PROGRAM Mark Ascione and Kim Taylor Office of Strategic Programs (OSP) Center for Drug Evaluation and Research (CDER) Food & Drug Administration (FDA) Heres


slide-1
SLIDE 1

BUILDING A CROWDSOURCING PROGRAM at FDA

FROM MARKET RESEARCH TO A SYSTEMATIC PROGRAM

Mark Ascione and Kim Taylor Office of Strategic Programs (OSP) Center for Drug Evaluation and Research (CDER) Food & Drug Administration (FDA)

slide-2
SLIDE 2

Here’s what we want you to learn from

  • ur story

Leveraging standardized tools for consistent results and increased bandwidth Building excitement around and engagement in a crowdsourcing program Increasing value through program improvements

2 3 1

slide-3
SLIDE 3

Today’s agenda

Developing tools to support increased demand

1

Leveraging early successes to establish best practices Using learning from all of you to increase value Introducing CDER Crowdsourcing

2 3 4

Q&A

5

slide-4
SLIDE 4

Our program was designed to enhance intra-center communication and engagement

  • Idea sparked by Partnership for Public

Service’s “Effective Internal Communication Strategies Workshop”

  • Crowdsourced the federal ideation

community in designing program

  • Operate as part-time crowdsourcers
  • Focus on short-term challenges (two

weeks) sponsored by senior leaders

Background Execution Goals

  • Drive ideas and information up to senior

leadership

  • Engage employees
slide-5
SLIDE 5

Our early success was driven by thoughtful challenges, communications and incentives

E a r l y S u c c e s s

Example: Office Retreat Planning Challenge

  • Objective: Better understand
  • ffice issues to tee up

discussion at upcoming all-

  • ffice retreat
  • Incentive: Office-wide one-

hour early dismissal

  • Participation rate: 80%
  • Tangible results: New office

newsletter, recognition process and recurring All- Hands events

  • Engaged executive sponsors and

moderators

  • Focused topics with specific questions

appropriate for crowdsourced conversation

  • Interested target audiences
  • Developed by communications professional
  • Consider target audience
  • Include multiple modes and touch points
  • Targeted toward participation and/or quality

idea generation

  • Include creative approaches appropriate for

federal space Challenges Communications Incentives

slide-6
SLIDE 6

Increased demand and limited resources required standardized tools for continued success

C o n t i n u e d S u c c e s s

Example: Pharmacy Preceptor Challenge

  • Objective: Solicit ideas to

enable more pharmacy preceptor students at CDER

  • Incentive: Three four-hour

time-off awards

  • Evaluation criteria: Focus,

timeliness, feasibility, popularity

  • Tangible results: overhauled

preceptor program process

  • Outlines critical components of each

challenge: challenge objective, audience, questions, incentive, etc.

  • Informs planning conversations with

executive sponsor and moderator

  • Contains suggested communication

touch-points customizable to challenge

  • Spans pre and post challenge, including

communication of any implementation plans

  • Contains moderator best practices

Initiation Template Communication Plan Template Moderator Checklist

slide-7
SLIDE 7

Going forward we plan to evaluate our program, connect with customers, and increase value

Value “Stages” implementation and enhanced community design Customer Connection Marketing segmentation and SharePoint integration Evaluation Focus groups, interviews and surveys

slide-8
SLIDE 8

We will leverage our core strength by evaluating CDER Crowdsourcing!

P E I S

Program Evaluation and Implementation Staff

  • Consists of five expert

evaluators (three of whom form Crowdsourcing team)

  • Conducts and

manages major internal evaluations related to key programs and initiatives

  • Objective: Understand mindset of different

user segments, how to drive engagement, and how to enhance CDER crowdsourcing

  • Method: Surveys/Focus groups
  • Objective: Learn what went well, what we

can improve, and impact of implemented ideas

  • Method: Surveys/Focus groups
  • Objective: Discuss value of crowdsourcing,

how to increase presence within CDER, and impact of implemented ideas

  • Method: Interviews

Crowd Moderators Sponsors

slide-9
SLIDE 9

We will utilize the “lurker” metric as we increase user engagement

U s e r s e g m e n t s

  • Lurkers: Users who sign

in to a challenge but do not participate

  • Participants: Users

who submit ideas, comment, or vote

  • nce or twice but no

more

  • Super-crowdsourcers:

Users who log in repeatedly and participate more than twice

Engagement

Super crowdsourcer Participant Lurker

slide-10
SLIDE 10

We will bring crowdsourcing to where staff does its work …SharePoint!

O p e n i d e a t i o n

  • CDER’s large staff (5000)

and bureaucratic culture not ripe for open ideation

  • Offices are smaller (500-

1000) and have staff dedicated to employee feedback

  • Several offices are

developing message board / suggestion box processes

  • SharePoint integration

brings power of IdeaScale right to users

IdeaScale SharePoint

slide-11
SLIDE 11

We plan to implement IdeaScale’s “Stages” to help ideas move past ideation

S t a g e s

  • Ideate: Generate ideas

and comments from staff

  • Staff assessment:

Adjudicate ideas through voting (currently combined with ideate)

  • Sponsor assessment:

Review and elevate top ideas using ReviewScale (currently done offline)

  • Action: Choose ideas

for implementation, consideration, and parking lot

Ideate Staff Assessment Sponsor Assessment Action

slide-12
SLIDE 12

We will create one area on

  • ur site for crowdsourcing

resources

“ A b o u t u s ”

  • Our crowdsourcing

story

  • Top-line statistics (# of

challenges, # of ideas, etc.)

  • Infographics

(overview and how- to) About us

Provides background on the program and includes infographics

Past challenges

Includes previous challenge questions, ideas, votes, and comments

How to participate

Describes how to submit an idea, vote, comment and participate confidentially

Host a challenge

Educates potential sponsors and moderators about how to host a challenge

slide-13
SLIDE 13

QUESTIONS?

  • Kim Taylor:

Kimberly.Taylor@fda.hhs.gov

  • Mark Ascione:

Mark.Ascione@fds.hhs.gov

  • Emily Ewing:

Emily.Ewing@fda.hhs.gov

slide-14
SLIDE 14

Appendix

slide-15
SLIDE 15
slide-16
SLIDE 16
slide-17
SLIDE 17

RESPOND

ENGAGE

Thank users for idea submissions (comments as warranted) Ask for elaboration of ideas or comments on submissions that are vague or about which you are curious Specify in a responding comment if an idea submission is already in practice

MERGE + DIVERGE

Merge ideas that are similar Diverge comments that contain new ideas into their own separate ideas Send relevant ideas to SMEs and ask them to engage on the crowdsourcing site

SEND

MONITOR

NOTIFY

Notify crowdsourcing team of any inappropriate or negative submissions

TRANSFER

Transfer off-topic or irrelevant ideas to the “additional input” category

Version 2.0 | 6/1/2018

checklist //

C D E R C R O W D S O U R C I N G M O D E R A T O R S

slide-18
SLIDE 18

CDER CROWDSOURCING CHALLENGE INITIATION TEMPLATE

SPONSOR MODERATOR(S) DATES AUDIENCE INCENTIVE

What senior leader sees the need for and will act as the face of this challenge? Who will monitor submissions to make sure they are appropriate and respond to idea submissions? When do you want to hold the challenge? Which office(s) are you targeting? Is there an incentive for this challenge (based on participation, idea quality, or some other factor)?

OBJECTIVES

What do you plan to get out of this challenge? How will the results be used?

Insert your response here Insert your response here Insert your response here Insert your response here Insert your response here This list should contain bullets of your overall goals for this challenge.

slide-19
SLIDE 19

CDER CROWDSOURCING CHALLENGE INITIATION TEMPLATE

INTRO PARAGRAPH DRAFT QUESTIONS

What is this challenge about? Why is it important? Ask open-ended questions that require non-’yes’/’no’ answers

This paragraph will be posted on the home page of the crowdsourcing website and will be used to develop communications plan. Please keep your paragraph to 200 words or less. Limit yourself to 3-5 questions and provide titles for each question.

slide-20
SLIDE 20

CDER CROWDSOURCING CHALLENGE INITIATION TEMPLATE

OPTIONAL FEATURES

Please highlight below the features you would like to utilize for your challenge.

  • Executive sponsor photo on crowdsourcing site and in communications
  • Turn off voting
  • Turn off down voting
  • Close voting until the last week of challenge after ideas have been reviewed and selected
  • Add profile questions to track demographics
  • Incentive for challenge participation
  • Send final challenge report to participants
  • Using stages
  • Posting flyers for challenge promotion