Wednesday, November 1

If you aren't piloting, you aren't doing your job, and you probably won't keep your job

I have seen hundreds of situations in the last 36 months where a training group had a training experience they said they wanted to consider for implementation.

  • Around 25% did not evaluate the program at all.
  • Around 25% evaluated the program by skimming it.
  • Around 20% evaluated the program by asking a few friends to skim it.
  • Around 20% evaluated the program by putting in a small group and asking the participants if they liked it.
  • Finally, around 10% evaluated the program by putting a group through it and measuring the effectiveness.
As a bit of a post script, when re-engaging training professionals a year later, there seem to be a significant correlation between those who didn't pilot well and those that did not have a job. Those most afraid of bothering people with a pilot became those most vulnerable.

3 comments:

Dave Lee said...

I'll trust your data on the unemployeed nature of non-piloters, but I can support the fact that piloting almost always finds problems and areas for improvement.

Throughout my career I've managed easily 100+ pilot sites for everything from companion workbooks, to textbooks, software activities, and full LMS's. What type of breakdowns do pilots catch? Here are 5 (well 5 1/2) of the biggies:

1) False assumptions about content progression. Your SME's already know the answers and have created short cuts for themselves that make them more efficient. The SME's you have review the content now the same shortcuts. When key steps are missing, learners don't get it you have a breakdown that needs to be fixed.

1b) Jargon, jargon, jargon. SME's use it. Learners don't know it. Facilitators often don't know it. (which can undermine the facilitator's credibility.

2) False assumptions about pacing. That activity you thought would take 15 minutes requires 12 minutes of setup and then runs for 30 minutes.

3) Ideas that looked good on paper but just didn't fly in real context. That team activity using hula hoops to teach Venn Diagrams sounded great but what a flop. The older employees spent time showing the younger employees how a hula hoop works.

4) Great activity once the facilitator figured out the poor facilitator guides.

5) Lack of clear link between activities and the learning objectives. Invariably signaled by an insistent raised hand follow by "why would we want to do this anyway?"

Now when would you want these things to happen? In a quiet pilot test that hardly anyone knows about or at company-wide launch with ever line manager on hand wondering why their people are here instead of on the job?

tigerjim14 said...

This is a great post here...I could not agree more! Dedicated to inform the public on relative issues. Keep up the great work!

Anonymous said...

runescape money runescape gold runescape money runescape gold wow power leveling wow powerleveling Warcraft Power Leveling Warcraft PowerLeveling buy runescape gold buy runescape money runescape items runescape gold runescape money runescape accounts runescape gp dofus kamas buy dofus kamas Guild Wars Gold buy Guild Wars Gold lotro gold buy lotro gold lotro gold buy lotro gold lotro gold buy lotro gold runescape money runescape power leveling runescape money runescape gold dofus kamas cheap runescape money cheap runescape gold Hellgate Palladium Hellgate London Palladium Hellgate money Tabula Rasa gold tabula rasa money lotro gold buy lotro gold Tabula Rasa Credit Tabula Rasa Credits Hellgate gold Hellgate London gold dofus kamas buy dofus kamas 血管瘤 肝血管瘤 音乐剧 北京富码电视 富码电视 富码电视台 7天酒店 7天连锁酒店 7天连锁 自清洗过滤器 过滤器 压力开关 压力传感器 流量开关 流量计 液位计 液位开关 温湿度记录仪 风速仪 可燃气体检测仪 wow power leveling wow powerleveling Warcraft PowerLeveling Warcraft Power Leveling World of Warcraft PowerLeveling World of Warcraft Power Leveling runescape power leveling runescape powerleveling
runescape money runescape gold wow power leveling 棕榈树
eve isk
eve online isk
eve isk
eve online isk