Communicating Success Revisiting Key Performance Indicators Ryan Lanyon
Communicating Success Revisiting Key Performance Indicators Ryan Lanyon Manager, Smart Commute ACT Canada Sustainable Mobility Summit 2012 November 5, 2012 1 Agenda Introduction to Smart Commute and Metrolinx Brief History Start-up Indicators Foundation Indicators Growth Indicators Thanks to: Future Indicators
Mark Singh Kyle Kellam Next Steps Rebekah McGurran Meaghan Mendona Geoff Noxon 2 Introduction Metrolinx Established in 2006 Serving the GTHA 6.5 m people 30 municipalities Three operating divisions: GO Transit
Presto Airport Rail Link Two programs: Transit Procurement Initiative (TPI) Smart Commute 3 Introduction Smart Commute Established 2001, 2005 Serving the GTHA
Two-tiered structure Metrolinx 11 TMAs (soon to be 13) Metrolinx and municipal funding Some sponsors and private sector fees Focus on work trips 250+ workplaces 600,000+ commuters 4 Brief History Decade of Development Exploration Experimentation
Start-Up Foundation Growth
1999 Roundtable Breakfast and TMA Steering Committee 2000 2001 Black Creek Regional TMA 2002 2003 Smart Commute Black Creek 2004 404-7 2005 Smart Commute Assocation (SCA), Carpool Zone Mississauga, NE Toronto, North Toronto & Vaughan 2006 TMA Toolkit, Brampton-Caledon, Halton, Hamilton 2007 Central York, Durham, SC Awards 2008 Metrolinx, Toronto-Central
2009 Discount Transit Pass Programs 2010 Pearson Airport Area 2011 Workplace Program 2012 Scarborough 2013 Triple program impacts over 2009 5 5 Start-up Indicators Contribution Agreement with Transport Canada Awareness Participation Satisfaction Project Benefits
6 Start-up Indicators Awareness Measures Percentage of commuting population aware of Smart Commute services Percentage of commuting population aware of how to reach Smart Commute (e.g. 800 number, website) Number of commuters directly exposed to Smart Commute information by direct outreach efforts. Percentage of information requests received through various referral sources Percentage of regional and local commuters who currently use alternative modes for commuting Percentage of regional and local commuters who would be willing to try alternative modes for commuting Percentage of regional employers that are aware of Smart Commute Association
services; percentage of local employers that are aware of TMA services 7 Start-up Indicators Participation Measures Commuters Number of commuter requests for various services offered/supported by Smart Commute Association or the individual TMA Number of participants in Smart Commute Associations Project or the TMAs services. Number of ridematch applicants and number per 1,000 commuters Percentage of applicants who use ridematch information sent to them. Employers Number of employer requests for information and assistance Number of employer clients participating in Smart Commute Associations services (e.g.,
transportation fairs) Number of regional employers implementing worksite TDM measures (parking spaces saved, bike storage and shower/change facilities added, site design) 8 Start-up Indicators Satisfaction Measures Commuters Percentage of users who rate various Project services as excellent or very good overall Percentage of users who request improvements in Project services
Percentage of ridematch applicants who receive ridematches Commuter ratings on service quality features (e.g. time to obtain assistance, convenience of service access/availability, accuracy/quality of information provided) Employers Percentage of employers that rate various Smart Commute Association or TMA services as excellent or very good overall Employer ratings on service quality features (e.g. time to obtain assistance, usefulness of information and products provided, knowledge and expertise of outreach staff) 9 Start-up Indicators Project Benefits Measures Percentage and number of applicants placed in rideshare modes after receiving Smart Commute Association or TMA services or information
(continued and temporary/trial placements) Average vehicle trips reduced per placement Number of daily vehicle trips reduced by commuters who received Smart Commute Association or local TMA services Number of VKT reduced by commuters who received Smart Commute Association or local TMA services Number of tonnes of emissions reduced by commuters who received Smart Commute Association or local TMA services (also reported by mode) 10 Start-up Indicators Challenges Indicators negotiated and created with little experience Too many indicators Competing priorities
Did not paint a clear picture Usefulness of some indicators Measurement feasibility and accuracy What defined success? 11 Foundation Indicators Opportunity to Reflect on Experiences Collaborative approach Refinement of Start-up Indicators
Built on foundation provided Elimination of unnecessary indicators Simplification of indicators Alignment with existing data collection Used only for monitoring, not evaluation Short-term usage 12 Growth Indicators Triple Productivity in Three Years (2010-11 to 2012-13) Number of employers active in the program Number of commuters at active employers Number of registered online users
13 Growth Indicators 14 Growth Indicators Challenges Indicators mandated by Metrolinx senior management Partner and stakeholder reaction Isolating productivity to what is within our influence Regional and TMA-level accountability Focus on recruitment and retention
KPI too narrow Drive to have users register and stay registered Concerns around how to attract users, rather than incent behaviour change 15 Future Indicators Return to a collaborative approach; pro-active Sharper refinement based on greater experience Balance of needs for Metrolinx, Smart Commute, stakeholders Distinguishing between regional, TMA, employer What funders need to know What delivery agents need to know What employers need to know
16 Future Indicators Challenges Regionalize the KPI Measure depth of engagement at workplaces Measure the efficiency of program delivery Isolate performance from the broader context Inputs vs. outputs Balancing measurement accuracy with resources Indicators vs. collected data 17 Future Indicators Reach
Breadth and depth of engagement Satisfaction Client satisfaction with programs & services Impact Change in commuter behaviour Cost-effectiveness Efficiency in investment 18 Future Indicators Workplace Program Measuring Depth Programming Promotions Tracking SOV reductions Investments and incentives
Infrastructure and trip-end facilities 19 Designation: -Gold Workplace -Silver Workplace -Workplace Future Indicators Reach Employer Reach # of SC Workplaces Gold Workplaces Silver Workplaces Workplaces Proportion of all GTHA employers engaged (%)
Commuter Reach # of Commuters at Active SC employers Gold Workplaces Silver Workplaces Workplaces Proportion of all GTHA commuters engaged (%) 20 Future Indicators Reach (cont) Carpool Zone Participation # of active registrants # of new registrants (per year) Program Awareness
% of Commuters in GTHA aware of Smart Commute 21 Future Indicators Satisfaction Employer Satisfaction Employers rating services as very good or excellent (%) Value for money Understanding of needs Staff knowledge and expertise
Usefulness of information and products Time to obtain assistance Employer Retention Retention of employers from preceding year (%) Carpool Zone Satisfaction Online registrants rating the service as very good or excellent 22 Future Indicators Impact Change in vkt Set a baseline vkt through census survey Mandatory minimum response rate for extrapolation Alternatively assume non-responders to be SOVs
Assumptions: Average commute distance from a more rigourous data source Average number of days spent commuting Conduct a follow-up Calculate vkt in a similar manner Assume most vkt is reduced in first year and then held steady until survey Tally all surveys for a regional vkt reduction 23 Future Indicators Cost-effectiveness Service Efficiency Service efficiency index Inputs:
# of Employers Depth of engagement by workplace designation # of Employees Cost of Services Leveraged Funding TMA funding leveraged from non-MX sources (%) From non-MX sources From non-government sources 24 Future Indicators Cost-effectiveness
Service Efficiency Methodology Employer Size Factors: 50 and 499 employees = 1.0 500 and 4,999 employees = 2.0 5,000 and 49,999 employees = 3.0 50,000 and 499,000 employees = 4.0 Designation weightings:
Gold Workplace = 6 Silver Workplace = 4 Workplace = 1 Employer Efficiency Formula: [(1 + log10(Employer AA size/50)) * Employer AA designation weight] TMA Service Efficiency Formula: (Employer AA Efficiency + Employer BB Efficiency + Employer CC Efficiency) Cost of Services 25 Next Steps Finalize future KPI
Determine exact methodology for calculating vkt Ensure buy-in from management and municipalities Restrategize around revised KPI Five-year strategic and operational plan Realign efforts to focus on new KPI 26 Ryan Lanyon Manager, Smart Commute 416-874-5933 [email protected] 27
Schmidt spent more time writing All of Me than almost any other of his compositions . Also was the first song he preformed after releasing his album. Second most popular song he has ever produced. Notes: All of Me, is...
The same is true of netball, football, and any other team sport you can imagine. What is the 'Looking Glass Self'? What is the 'I' and the 'Me'. What is the Play Stage and the Game Stage? Erving Goffman.
Outline Questions for Noodletools : Compare these questions to your own and add/delete questions if applicable.. I. Topic=Amazon Rainforest (mining, animals, deforestation, etc.) What is your topic? Describe it. Where is the topic taking place within the Amazon Rainforest? What...
* 1995 Pack et al. Fall-asleep crashes fatality rate (1.4%) not far different from alcohol intoxication crashes (2.1%) Powel 2001 60 pts "sleep deprivation vs EtOH intoxication" + driving performance impairment comparable [one night total deprivation or 7 nites (2H...
Energy systems. Learning outcomes: All are able to demonstrate understanding of the relationship between OBLA and VO. 2 maxMost are able to explain the third energy system (the aerobic system) Some can analyse the causes of fatigue. Some can apply...
Risk adjustment is a method of determining and balancing the cost associated with caring for patients with varying severities of illness. Risk Adjustment is current trend in payment methodology for leveling the payment and cost of caring for a patient...
Unit 2 - The Cell Chapter 7, II ~ A Tour of the Cell Other membranous organelles, I Mitochondria • quantity in cell correlated with metabolic activity; •cellular respiration; •double membranous (phospholipid); •cristae/matrix; •intermembrane space; •contain own DNA Other membranous...
Ready to download the document? Go ahead and hit continue!