Performance Measurements and Benchmarking Establishing Performance - - PowerPoint PPT Presentation

MCSAP Conference and Grants Training – 2017

Performance Measurements and Benchmarking

Establishing Performance Based Activities

MCSAP Conference and Grants Training – 2017

• Performance Benchmarks – Background

– FAST Requirements – DATA Act, GAO Report, etc. – OMB Super Circular

• Performance Standards, Measurements & Benchmarks

– Overview of PSMB results – Next Steps

• Metrics and Grants

– Safety Objectives and 49CFR350 – Role in CVSP development

• Wrap-up

2

Topics

MCSAP Conference and Grants Training – 2017

Performance Standards

The Background

MCSAP Conference and Grants Training – 2017

• SEC. 5101. GRANTS TO STATES § 31102 (b)

– … (4) assessing and improving statewide performance by setting program goals and meeting performance standards, measures, and benchmarks

• SEC. 5101. GRANTS TO STATES § 31102 (c)(2)

– …The Secretary shall approve a State plan if the Secretary determines the plan is adequate to comply with the requirements of this section, and the plan – (A) implements performance-based activities; …

4/7/2017 4

Performance Standards- FAST Act

MCSAP Conference and Grants Training – 2017

• 1993, GPRA - designed to improve program management

throughout the Federal government.

• 2010, GPRA Modernization Act (GPRAMA) a more defined

performance framework by defining a governance structure and by better connecting plans, programs, and performance information.

• GAO Report – Commercial Motor Carriers: DOT is Shifting to Performance-Based

Standards to Assess Whether Carriers Operate Safely (11-03-97, GAO/RCED-98-8)

“In April 1997, consistent with the Government Performance and Results Act of 1993, the Office of Motor Carriers began using performance based data through its Safety State Measurement System to identify carriers with the worst highway safety records. Complete data on accidents, roadside inspections, and compliance reviews that the states submit to the OMC are key to implementing performance-based criteria.” “OMC recently announced that it will work with the states to develop performance-based Commercial Vehicle Safety Plans…”

• MCSAP Statute and Part 350 have had the requirement since at

least 1996 (49 CFR 350, Appendix A.2.d.)

4/7/2017 5

Performance Standards - Drivers

MCSAP Conference and Grants Training – 2017 4/7/2017 6

Performance Standards - The Convergence of Data Analytics

Whitehouse.gov

MCSAP Conference and Grants Training – 2017

• Federal Motor Carrier Safety Administration (FMCSA) promotes safe

commercial motor vehicle (CMV) (large truck and bus) operations through education, innovation, regulation, enforcement, financial assistance, partnerships, and full accountability to reduce crashes, injuries, and fatalities on our Nation’s roadways involving CMVs.

• MAP-21, The Moving Ahead for Progress in the 21st Century Act (P.L

112-141), July 6, 2012 provides an increase in safety funds available for the Highway Safety Improvement Program along with a focus on performance-based programming. Implementation of MAP-21 will rely heavily on increased partnership across State agencies with DOT.

• What next? FAST Act and the consolidation of programs and an effort to

put more control in the State management of performance.

4/7/2017 7

Performance Standards

Department of Transportation FY 14-15 Priority Goal: Reduce the rate of roadway fatalities

MCSAP Conference and Grants Training – 2017

GAO Issues Revised Standards for Internal Control Government Brief, Business, Industry & Government Team, By Kristen A. Kociolek, October 15, 2014 Policymakers and program managers are continually seeking ways to improve accountability in achieving an entity’s mission. A key factor in improving accountability in achieving an entity’s mission is to implement an effective internal control system. An effective internal control system helps an entity adapt to shifting environments, evolving demands, changing risks, and new priorities. Kristen is a member of the AICPA Government Performance and Accountability Committee.

4/7/2017 8

Performance Standards – Standards for Internal Control

MCSAP Conference and Grants Training – 2017

• GAO Issues Revised Standards for Internal Control

On September 10, 2014 GAO issued its revision of Standards for Internal Control in the Federal Government. You can read the press release here. The 2014 revision will supersede GAO/AIMD-00-21.3.1, Standards for Internal Control in the Federal Government: November 1999.

• Legislative Authority

Federal Managers' Financial Integrity Act (FMFIA) requires that federal agency executives periodically review and annually report on the agency's internal control systems. FMFIA requires the Comptroller General to prescribe internal controls standards. These internal control standards, first issued in 1983, present the internal control standards for federal agencies for both program and financial management.

• State, Local, and Not-For-Profit Applicability

The Green Book may also be adopted by state, local, and quasi-governmental entities, as well as not-for-profit organizations, as a framework for an internal control system.

4/7/2017 9

Performance Standards – The Standards for Internal Control

MCSAP Conference and Grants Training – 2017

• Subpart A, Definitions
• § 200.61 Internal controls. Internal controls means a

process, implemented by a non-Federal entity, designed to provide reasonable assurance regarding the achievement of objectives in the following categories:

(a) Effectiveness and efficiency of operations; (b) Reliability of reporting for internal and external use; and (c) Compliance with applicable laws and regulations.

4/7/2017 10

Performance Standards – Compliance Under 2 CFR

MCSAP Conference and Grants Training – 2017

Subpart D, Post Award Requirements, Standards for Financial and Program Management § 200.303 Internal controls. The non-Federal entity must:

(a) Establish and maintain effective internal control over the Federal award that provides reasonable assurance that the non-Federal entity is managing the Federal award in compliance with Federal statutes, regulations, and the terms and conditions of the Federal award. These internal controls should be in compliance with guidance in ‘‘Standards for Internal Control in the Federal Government’’ issued by the Comptroller General of the United States and the ‘‘Internal Control Integrated Framework’’, issued by the Committee of Sponsoring Organizations of the Treadway Commission (COSO). (b) Comply with Federal statutes, regulations, and the terms and conditions of the Federal awards. (c) Evaluate and monitor the non-Federal entity’s compliance with statute, regulations and the terms and conditions of Federal awards. (d) Take prompt action when instances of noncompliance are identified including noncompliance identified in audit findings. (e) Take reasonable measures to safeguard protected personally identifiable information and other information the Federal awarding agency or pass-through entity designates as sensitive or the non- Federal entity considers sensitive consistent with applicable Federal, state and local laws regarding privacy and obligations of confidentiality.

4/7/2017 11

Performance Standards – Compliance Under 2 CFR

MCSAP Conference and Grants Training – 2017

Standards for Internal Control in New York State Government “Most people think that internal controls are about accounting for funds. While that is true, it is a myth that internal controls are only about money. Internal controls are also about demonstrating the value of an entity’s programs and reporting its accomplishments. Internal controls are about protecting assets, information, and employees, and enabling the organization to make the best

• decisions. Internal controls are ingrained in all business systems and functions.

They are essential to ensuring that an organization is functioning effectively and

• efficiently. Government managers should be able not only to account for funds

spent on a program, but also to assess the value of the program and measure its

• accomplishments. An effective system of internal control can give managers the

means to provide accountability for their programs, as well as the means to obtain reasonable assurance that the programs they direct meet established goals and

• bjectives.”

OFFICE OF THE NEW YORK STATE COMPTROLLER, Thomas P. DiNapoli, State Comptroller, March 2016 Standards for Internal Control in New York State Government

4/7/2017 12

Performance Standards – State Perspectives on Internal Control

MCSAP Conference and Grants Training – 2017

FMCSA Grants

4/7/2017 13

Performance Standards – Moving Ahead

MCSAP Conference and Grants Training – 2017

• PSMBs were called for in The Moving Ahead for Progress in the 21st Century Act (MAP-

21)

• The PSMB project involved the establishment of a joint Federal-State working group to

analyze available data and develop performance standards, measures, and benchmarks (PSMB) for MCSAP-related grants.

• The PSMB Working Group started working in 2014.
• PSMB subgroups were given the charge of determining which data elements or

measurement approaches should be used, rather than a single office within FMCSA establishing standards unilaterally.

4/7/2017 14

Performance Standards and Benchmarking Group

MCSAP Conference and Grants Training – 2017

• New Entrant
• Border Enforcement
• Crash Statistics
• Outreach and Public Education
• Carrier Investigation
• Inspection
• Special Emphasis
• Traffic Enforcement

4/7/2017 15

PSMB subgroups

MCSAP Conference and Grants Training – 2017

• FMCSA will establish a catalog of the performance standards that

relate to the program components; measurement methodologies and criteria

• Each State and Local partner and FMCSA can use the catalog and

guidance on how to establish incremental benchmarks to use when tracking progress towards goals, in evaluating performance and activities;

• This catalog will become an appendix to the MCSAP

Comprehensive Policy and will serve as the single point of reference for FMCSA-prescribed programmatic evaluation criteria.

16

PSMB Final Goal

MCSAP Conference and Grants Training – 2017

• States volunteer to conduct demonstration project (by

sending an e-mail to thomas.liberatore@dot.gov)

• Will include all 8 components of the PSMB effort
• Will apply these statistical approaches to real world scrutiny
• Will lead to final methodological proposal for review and

comment

17

PSMB Next Steps

MCSAP Conference and Grants Training – 2017

• Don’t wait on FMCSA to finish the PSMB work
• Many of you already use your own data or the data on

A&I

• Let’s look at the information you already develop and

report in the CVSP and Quarterly Progress Reports

18

Quantifying Performance

MCSAP Conference and Grants Training – 2017

• Corresponds with 350.201 (the 29 MCSAP participation

requirements)

• State agency goal or mission
• Program effectiveness summary of the prior year’s CVSP

activities (calendar, fiscal year or any 12 month period)

• Narrative describing how the program addresses the

national program elements (found in 350.109)

• Rationale for resource allocation decisions (where you

place personnel and why, plus the full time equivalents (FTE))

19

350.213- The CVSP includes?

MCSAP Conference and Grants Training – 2017

• Problem statement for each objective supported by data
• Performance objectives in quantifiable terms including a

measurable reduction in crashes and HM incidents

• Strategies used to achieve performance objectives
• Specific activities to achieve the strategies and

performance objectives

• Number of activities in each national program element

except data collection

20

350.213- The CVSP includes?

MCSAP Conference and Grants Training – 2017

• Quantifiable performance measures used to assist the

State in monitoring

• Description of the State’s method to monitor the plan
• Objective evaluation of last year’s plan identifying

problems discovered. Last year’s problems identified are addressed in this year’s CVSP as new projects or amended projects

• Budget that supports the execution of the CVSP

21

350.213- The CVSP includes?

MCSAP Conference and Grants Training – 2017

• CMV Crash data analytics – lead time indicators
• CMV Crash data dashboard – real time crash locations by

GIS coordinates on officers dashboard

• CVSP Crash reduction plans developed by each District

Captain using crash data. Each District plan is incorporated into the CVSP and monitored by the Division Captain who is responsible for the results

• CMV crash data analytics – University personnel look at

crash location and causation to develop action plans with the State to address high crash zones

22

Examples from States

MCSAP Conference and Grants Training – 2017

• Using DataQs submissions to identify Inspector training

needs

• Using A&I inspection data to identify Inspector training

needs

• Using A&I to evaluate traffic enforcement violations in

crash zones and compare with the top crash causation factors (e.g. following to closely, improper lane change, etc.) to ensure officers are targeting behaviors that cause crashes

23

Examples from States

MCSAP Conference and Grants Training – 2017 24

Thank You