You Want Numbers?

You want metrics data? I got yer data right here....

Check out PubSub's new LinkRank Service, measuring inbound and outbound links to any site! Check out Learning Circuit's LinkRank...

Blogging by the Numbers

Not everything that can be counted counts,
and not everything that counts can be counted.

- Albert Einstein

One of the projects in the revitalization of Learning Circuits Blog has been the establishment of a data set for capture and resulting metrics. I thought it might be of interest to share our work in this area.

The purpose for creating a trackable set of data and corresponding metrics is simply so that we will have an idea as to whether or not we are succeeding at this blogging venture. Simple purpose however, does not necessarily lead to simple implementation. Two factors contributed to this being a rather extensive task. 1) We had no data collection practice in place and 2) Blogger, for all it’s great features, just doesn’t collect data for their free users.

A couple free stat counters (thanks SiteMeter and Bravenet) and several tracking spreadsheets that border on frightening later, we have a solid set of raw data. The data fall into two categories – "Activity" which I use to refer to the posting and comments by the Blog Squad and guest and "Traffic" which refers to the comings and goings of all of you good folks. The raw data we are capturing include:

ACTIVITY	TRAFFIC
Posts Author Date posted Comments Comment Author Date Commented Days with no Activity	Unique Visits Page Views First time visitors Returning Visitors Referring Page Entry Page Exit Page

From these raw data we can build metrics to determine in what direction, if any at all, LCB is moving and why. There are all the ratio metrics (i.e., posts per day, comments per post, zero activity days per month, etc.) and comparison metrics (i.e., August was twice as active as July, or Clark leads all Blog Squad members in postings).

Some metrics which seem to be emerging as meaningful as everything settles in are:

1. Total Activity - Posts and Comments are weighted equally. It seems a better representation of the work that is being done by the Blog Squad.
2. Unique Visitors vs. Returning Visitors - This metric seems to indicate that we are drawing people in, but not doing enough to have them return in a timely fashion.
3. Total Activity vs. Page Views - this has been the real surprise thus far. The ratio of page views per post or comment seems to have settled in at just aound 24-26 pages. Including in August when Activity tripled and page view followed suit. It's too early to know if this consistency is a coincidence or whether it will turn out to be a dynamic predictive metric.

I have a couple of questions I'd love to get some response from the LCB community on regarding metrics:

1. Are there metrics you see as key when looking at a blog?
2. Do you know of any benchmarking sources for blog metrics?
3. Do you track metrics for your blog? If so, care to swap info with LCB?
   

Authoring Simulation Genres: Game based models

One well-understood simulation genre is game-based models.

With the goal of “making learning fun,” students engage familiar and entertaining games such as Wheel of Fortune®, solitaire, or memory, with important pieces of linear or task-based content replacing trivia or icons. More diagnostic than instructional, game-based models nonetheless might be the technique of choice by traditional educators and training groups looking to quickly goose their reputation, student satisfaction, and even effectiveness. Game-based models also introduce, in the purest possible way, game elements that all educational simulation designers will need to understand in the near future.

Some authoring environments include:
• EGames Generator
• Games2Train


Do people know of other toolkits for game-based models?

Authoring Simulation Genres: Interactive Spreadsheets

One well-understood simulation genre is interactive spreadsheets.

Interactive spreadsheets focus on abstract business school issues such as supply chain management, product lifecycle, accounting, and general cross-functional business acumen. Students allocate finite resources along competing categories at successive turn-based fixed intervals, and each time they watch their results play out on dense graphs and charts.
This is often done in a multi-player or team-based environment and often with facilitators.
The subtlety, unpredictability, and variability make them appropriate for training b-school students and high-potential supervisors through the direct reports to the CEO. They are often the cornerstones of multi-day programs to align a fractured department or organization by building shared knowledge and understanding.

• One authoring tool is:
  Forio Broadcast

Another is straight Excel.

Do people know of other great authoring tools?

Authoring Simulation Genres: Branching Stories

One well-understood simulation genre is branching stories.

In branching stories, students make multiple-choice decisions along an ongoing sequence of events around what to say to another person in a given situation. The decisions impact the evolution of the story, ultimately terminating in either successful or unsuccessful outcomes.
Their ease of use, ease of deployment, and content style make them highly appropriate for entry-level salespeople, call center representatives, freshmen, customer-facing retail positions, and entry-level managers. Any high-turnover position should be trained, although not exclusively, using branching stories. Companies like WILL Interactive have advanced the genre to handle more moral and complex situations, making them also appropriate for higher-level employees as well.

One tool kit is:

• Scenariation

You can also use PowerPoint, although it is tricky.

Do people know of other branching story toolkits?

The New Ways of Describing the Life of Content

I have been thinking a lot about life of content. A few different dichotimies seem to frame conversations.

Staged vs. Organic: Staged events are one-shot. They need significant pre-establisehd processes and project management. They use up a lot of advertising and communication. Organic approaches are more incremental. They use small layers to build up over time. Television shows are staged, but they can also evolve over many episodes, as writers seize onto relationships, or the right directors are found. Organic content lifecycles stress less perfection up front, and often more feedback.

Transient vs. Peristent : This refers to the size of the window of availability, and/or the timeliness of the material. Even these can be fuzzy. DVDs should be totally persistant, for example, but advertising and shelf-issues make them somewhat more transient. Also, as with movies, there is a perceived success (being number one on the charts) that drives more success, and more of a staged content approach. DVDs can also make movies a bit more incremental, with multiple versions available. Computer games can also have various versions, and can also be patched on the fly.

Controlled vs. Community: Is their one-voice shapping this or multiple? Mods for computer games can subvert a very controlled piece of content. From a formal learning perspective, I always ask, how can I get the training group out of the way? How can the community generate the content, and how can the training group help that?

Is there a "so what" for these lenses? I think so. One is to recognize our own biases, and accept that these biases might be interfering with coversations with sponsors. The other is to recognize the current trends that are pushing towards incremental, persistant, and community, and selectively embrace some of this approach, and challenge others.

Small Pieces Loosely Joined

When talking about platforms, the opposite of a single monolothic approach (cough Windows cough) is small pieces loosely joined.

We in the learning industries seem to be good at the

• big pieces, tightly joined (degrees, curricula, certification)
• small pieces, tightly joined (LMS delivered content)
• small pieces, not at all joined (Google, mapquest, bubble help)

So what is it that can/does loosely join our small pieces of learning? Is it our PC? Outlook? The database in Google Desktop Search? Our PDA? Is it our organization? Our annual review? Our interests? Company? Career? Industry?

At the end of our day, it is probably summed up in our résumé, our sometimes forced, often surprising organization of the things we have demonstratably done. But at the heart, we are our own platform, constantly both writing to others and being written upon, vastly ranging in productivity and capability.

I would say for the last few hundred years we have probably been Human 2.0. And we are getting close, I think, to v3.

The Lowly Binder

It is a symbol of "old school" training. It is a joke, a dinosaur, a deep sea anchor. It wastes billions of dollars of office space in filing cabinets and cubicle shelving. It is THE TRAINING BINDER.

We, the Learning Cognoscenti, "prove" our own credentials to the world by openly disdaining them. '"What to do about the classroom binder?' you ask. 'How about dig a deep hole and throw them all in!'" Then we chuckle and drink our cognac and tighten our smoking jacket.

And yet...

And yet there are still a lot of them being produced. Almost every classroom experience, and there are a lot of them, support the activity with binders. Conferences do as well, I need hardly mention. I, as dutifully as the next speaker, turn in my PowerPoint stack for inclusion in whatever event I am part. I also helped build a workbook to support rolling out next-gen-sim Virtual Leader, which significantly increased usage.

So it might be worth a few moments to, with appropriate humility and pragmastism, reflect upon what are some of the tools available that can make for a good binder.

• Removable overviews, often in the front sleeve, that provide context and a map to what comes next, or a tip/cheat sheet to an application
• PDFs of the entire thing, available for download and storage; perhaps zipped files as well
• Copies of slides, for people to take notes in real time
• Color paper and/or tabs, to distinguish between sections and aid navigation ("Go to the yellow section...")
• Biographies, with face shots and contact information, of speakers and ideally participants, at the back
• Appendixes, with articles and links
• CD-ROMS, with editable versions of slides and spreadsheets
• Clear cover and spine, with subject and dates
• Three-ring rather than bound, so that users can dynamically and deliberately add or subtract what they want or don't want.

What are your thoughts? What should or should not be part of a well constructed binder?

Emotionally Evocative Design

For many an Instructional Designer, design follows analysis, with its main function being to identify all the important things that need to go into a course. Its end product is a curriculum, syllabus, or blue-print to build the learning module on.

Next comes development, which adds content so as to give depth to the end product of design. This mainly consists of the adding of "information."

Context is also used to add a third dimension to the design puzzle -- layers of activity so that the learners gain a variety of viewpoints, thus allowing them to gain experience with the information in a relative safe manner.

This mix of design, development, and context should theoretically help the learners build their knowledge and skill bases. Yet this combination often fails because it leaves one important piece of the puzzle -- Emotionally Evocative Design.




While Instructional Designers normally are quite good at ensuring that the important parts that build content, such as objectives and outcomes, are entered into the design equation; the total design process needs a second layer that captures the emotions of the learners so that they actually want to engage with or use the content.

Engagement does not have to be that complex as it is simply a means of inviting the learners' emotions into the environment. Emotions are the reason that we do anything -- without them we would simply be walking zombies. Emotions are what adds zest to life...to include learning

For example, a good metaphor invites the user to reflect over the information by asking her to relate the new information with a past experience. A good picture invites two senses into the mix. A problem begs for closure. A critical piece of information delivered "just-in-time" is utter relief.

What have you done lately to ensure that your designs are emotionally evocative?

CNN vs Education

I would not suggest that higher education institutions need to operate like CNN, but I find it fascinating to read Elliot Masie's observations of how CNN dealt with the flow of content and information in the wake of Hurrican Katrina. In CNN Newsroom in the Midst of Katrina - "Rapid Development... Content Objects... Learning Implications":

There were some incredible learnings and observations as I quietly watched the news gathering and assembly process and interviewed the Learning team at CNN. Many of these items relate directly to how organizations will be assembling content in the near future.

The question is, will educational institutions be one of these organizations? Below I have take some of Maise's descriptions of CNN and put them besides a gross and likely over generalized observation of higher education. Yes, there are numerous exceptions and counter examples to every one of my points, but as a whole, when you read how CNN operates and put it besides how your higher education institution operates, the contrast should be rather vivid.

CNN: Content From Multiple and Unconventional Sources: The nature of content in journalism is changing dramatically as media flows from non-traditional sources. CNN calls an aspect of this "Citizen Journalism" as they receive pictures and video feeds from digital cameras and even mobile phones.

Education: Content comes nearly solely from instructors and "trusted" resources like publishers and "refereed" journals. And content from students would be suspect or written off as "cute" but not useful. Is there an equivalent of Citizen Teaching? If so it is the inspired individual's effort of placing a tutorial or resource on the net.

CNN: Content To Multiple Formats: As content was created in the CNN newsroom, it flowed to multiple formats. Content started as video feeds, became streamed video, text on the website and even a mention for a scroll at the bottom of the screen. Each piece of content was "tagged" as it came into the newsroom, timecoded, meta-tags were added with context and it could be viewed by CNN staff around the world in low-res format. The concept was to see each media object as being highly reusable and redeployable.

Education: Content primarily text, email, PDFs to print, hand coded/Dreamweaver-ed HTML, and.. PowerPoint. Media objects are un-reusable, un-findable, and in-redeployable and would never be available in this kind of time frame.

CNN: Digital News Gathering: The footprint and format for news production is changing radically as the size and mobility of equipment evolves radically. I watched newsfeeds coming from CNN reporters using satellite phones (after the cell network dropped). They were even feeding content that was edited on laptops in the field using Final Cut Pro.

Education: Not much to compare except we do not think of educational content being created "in the field" or with portable devices... while some are moving towards laptops in place of desktop computers, they are used primarily in the same vein (a laptop on the desk). We still print a lot of material or offload this printing to students-- we do not "think"/"operate" primarily in digital, and much of what we do digital is the digital equivalent of print.

CNN: Content Repository: CNN operates a content and media repository that is quite impressive. The content objects are viewable, editable and sharable. Key levels of data is kept for how each object is being used and deployed. Digital Rights Management is tracked, to honor the appropriate use of each media object. I was struck by how easily every CNN staff person could access and work with this content repository.

Education: This is still a dream, despite years of wrangling over "Learning Objects" and the construction of numerous "repositories" few if any that have the features described above AND as much content. Data on the use of objects is absent and DRM is spotty.

CNN: Rapid Development: While CNN clearly has a breaking news model, it was fascinating to watch this process in action, including use of templates, collaborative and team-based editing and content refinement, focus on content ethics, standards and legal/compliance issues. I witnessed a team of professionals, drawn from a wide set of backgrounds, deeply focused on producing content that had value for viewers and the hurricane's victims.

Education: The development cycle is measured in months, perhaps as summer projects or long term grants, and is pretty much an individual cottage industry. Editing and development are solo projects.

So, how does Big U stand up to CNN?

Education's Collage of Subsidies

I was thinking about Clark's (the other Clark) post on learner rights, and then the broader learner marketplace. One of the trickiest parts about development of formal learning programs is that so often, the learner only has a passing relationship to the cost of the program.

Enterprise Programs
Enterprise training programs are paid by the corporate, military, or government sponsor.
Their goal is usually some form of increased productivity or compliance.
Any vendor has to care more about meeting the need of the training director, and the training sponsor, than the end learner. Even the cost of the classroom environment is often subsidized by the enterprise, penalizing e-learning, outside classes, or even training outsourcing.

K-12
K-12 schools are paid by taxpayers, not the students of course, nor even the family directly (even a family that goes on lavish vacations, for example, gets their education subsidized (of course if they have big houses, they also big property taxes; and then there is the private school whammy, where they are paying twice...).).
As we are also learning, some K-12 activity is subsidized by softdrink/juice/water vendors, that come with strings attached.
There are grant based subsidies that push activities in one way or another.
And there are more subtle subsidies, making land cheaper than building supplies or technology, relative to other building projects.
Of course, K-12 is required of students, which can be considered an anti-subsidy (increasing its cost to the student, while still interfering with the learner marketplace). Lack of choice for any required course also represents an anti-subsidy.

College
In many college situation, parents significantly augment the students' ability pay at least some of the tuition. Then there are scholarships for academic, sports, and music. And scholarships, like other endowments, are subsidies by alums, meaning schools have to do things specifically to keep alums happy (while students do certain things to meet the criteria of the scholarship), not necessarily in the interest of the students. Furthermore, the reason many students put up with schools is to get better jobs, which means that while corporations might subsidize universities directly, they anti-subsidize them indirectly. Any certification, in fact, is anti-subsidized by the organization that values it

These are just a few examples from the K-12, Higher Ed, and enterprise world. But every subsidy and anti-subsidy distorts the natural value proposition away from the learner marketplace. Getting over all of these will be our challenge for the decade.

Note: This blog post has been subsidized by my most recent book, Learning By Doing. New book review here.

Learning By Doing: The Tool is the Curriculum

I had to brush up on project management skills over the weekend. Did I read a book on it? No. Did I find someone who was an expert to help me? Nope.

I downloaded Open Workbench (for me, a random open-source project management tool) and played around with it. I mapped a few of my projects into it. I looked through all of the fields and capabilities. And I learned quite a bit, including what I needed to know, plus a lot more that I am very happy to know.

For those home schoolers looking at math curricula, I would suggest Microsoft Excel. Whatever math you need to know, you can learn by getting deeper and deeper into Excel functionality, including calculus, logic, probability, graphing, and more.

And you can learn by trial and error, by playing, by experimenting. You discover the material; you own the material.

There are plenty of dangers, I suppose, with having the tool be the curriculum. I am sure academics will be happy to tell me what they are. But increasingly mature software tools represent philosophies, approaches, domain knowledge that is much more sophisticated than the more trivial linear experts. And you can learn by doing.

My grandparents learned to read by picking up the classics out on the farm. Bach learned music, I am told, by copying over the sheets of the great composers before him. I wonder if today's tools will be considered tomorrow's classics?

E-Learning: Channels, piracy, and outrage PLUS REGULATIONS

I was thinking about the earlier post on how will we know we are getting close to e-learning meeting its potential. I thought of one more, slightly further down the road.

• Regulations! How long until we see- WARNING - Use this e-learning product only as specified. It is against the law to use this e-learning product in a manner inconsistent with its labeling.

A Snapshot of Leadership Training in the Enterprise

As part of rolling out Virtual Leader, it has been interesting for me to note how most organization who do formal leadership training actually do formal leadership training.

I think most terms are self-evident, but external coaching can also include board involvement. Tools include focused performance support.

The high-potential focus is not exclusive of other manager and supervisor leadership training.

MoodleQuest

I am going spend about two weeks now trying to get up to speed on Moodle, ultimately evaluating it (against other tools) for implementation.

I will share relevant data and artifacts as I come across them.

All comments are appreciated.

How to Optimize Organizational Learning (inclusive)

Here is the newest version of my accord chart. I continue to update it based on experiences in the field and even comments from this blog! It is a snapshot of all of our various options, the arrows in our quiver. Each have trade-offs; each have benefits. The question is no longer, which to do, but more, can you do each at the right time?
As with the consolidation chart, I am happy to send it anyone who emails me for it.

How to Optimize Organizational Learning

Recently while researching on the web on topics regarding Communities of Practice I came across the Community Intelligence Labs (CoIL) website. While it seems the website may have gone the way of the Brontosaurus (the last update was in Feb 2004), it's resources are still there for the viewing. Among them I found the following 14 guidelines on How to Optimize Organizational Learning written by Entienne Wenger in the summer of 1996 for the Healthcare Forum Journal.

I thought it would be interesting to get your take on these 14 guidelines now 9 years after Wenger first penned them. He introduced these guidelines with:

Here are the 14 guidelines to help you work with rather than
against the inner logic of organizational learning:

1. View learning as work and work as learning. Recognize learning in all it's forms in order to find ways to nurture it and connect it across the organization.
2. Count on the informal. That is where work gets done.
3. If there is a learning problem, look for patterns of social participation and exclusion.
4. Keep learning as close to practice as possible.
5. Treat Communities of practice as assets.
6. View individuals as members of communities of practice, not by stereotyping them, but by honoring the meaningfulness of their participation.
7. Encourage the formation and deepening of communities of practice by legitimizing the work of pulling them together and valuing the informal learning they facilitate.
8. Manage boundaries between communities of practice as opportunities for learning.
9. Expect transformations, misunderstandings, and reinterpretations when people, artifacts and information cross boundaries of practice.
10. Value the work of brokering learning among communities; it often does not look like work.
11. Be attuned to the emergence of new practices at boundaries.
12. View the organization as a constellation of interconnected practices.
13. Put communities of practice in charge of their learning, recognizing that they need access to other practices in order to proceed.
14. Make sure that the organizational apparatus is in the service of practices, and not the other way around.

Obviously, 9 years later, some of these guidelines are either proven or widely held to be true. I also would say it's obvious that communities of practice are thriving that would indicate that these guidelines are being well applied. Some of the questions I'd like to hear the LCB community chime in on are:

• Are any of these guidelines obsolete, or impractical to such a level, that they should be struck from the list?
• Which among the 14 are the "pain points" that corporate cultures just can't seem to get past, thus blocking the emergence of vibrant CoP's?
• What's the role of collaborative technologies to date in this? Have they helped nudge along a very slow adaptation process? Or have then hindered the effort?
  

What's your most memorable conference experience?

What makes for a conference event with impact? I'm building a list of what works best. What's your most memorable conference event? What delivered the most learning impact? I'll tell you mine; tell us yours in the Comment section.

About ten years ago I attended a presentation by the late Gordon MacKenzie, the "creative enigma" of Hallmark Cards and author of the wonderful book Orbiting the Giant Hairball. He hung a clothesline across the stage at the annual meeting of the Instructional Systems Association and pinned a dozen doodles on 8 1/2" x 11" sheets to it. The audience would point to a sheet, and Gordon would tell its story.

Gordon's stories were heartfelt, emotional, and often very personal. At times it felt like the story was telling Gordon rather than the other way around. Gordon broke into tears several times, and tears streamed down our faces in empathy.

Gordon shared another story with me over lunch. When he started a day with no particular objective, he would tell himself that during the day he was going to be amazed. And he always was. Gordon taught me a lot about feelings and sensitivity to others that day. He also demonstrated that the sequence of stories in a presentation isn't always that important. I mourned when his widow told me Gordon had passed on.

"Grand Theft Auto"ing Edutainment Software

Grand Theft Auto might be the worst game series ever, in part because it might be the best.

Almost Microsoft-esque, GTA has absorbed and fairly seamlessly integrated different genres, including driving games, first person shooters, fighting games, gambling games, flight simulators, adventure games, shopping games, role-playing games and more.

My last post got me thinking. What if one took a GTA approach to edutainment software?

What if you could create, over multiple iterations, an experience that both included a generic, open ended "real-world" environment with specific missions to increase capability?

You could absorb:

• Pet simulators/trainers
• Lemonade stands
• Tycoon (designing interactive environments, such as store layouts and zoos)
• Designing (building cars for example, or new computers, or new clothes. If a persistant world had a GTA visual structure, you could see hundreds of people wearing your trendy new hats as you walked down the street)
• Business building/ logistics
• Logic games
• Historic settings
• Physics simulators
• Medical/vet
• Ecosystems/genetic engineering
• Space
• Traffic Emergencies
• Negotiation simulators

Trucks, dinosaurs, dating, status, shopping, horses could co-exist in different locations as themes and reward structures. Mathematics could exist as an optional but useful lens. Multi-player and massively multiplayer would of course be optional key components. Building up business empires, huge houses, and vast social networks could start at humble activities, including creating great inventions and training animals. You could call in favors, as well as generate more and more people loyal to you. You could live in the stables, or labs, or making deliveries, or in the operating room as well if you wanted to.

I'm in!

Once a Booming Market, Educational Software for the PC Takes a Nose Dive

The New York Times ran a piece today about the crash and burn of the K-12 educational software market. To sum it up, once thriving, large, and profitable, the industry niche today, well, isn't.

I would be the first in line to buy any software that helps my son. But my own personal experiences with the "edutainment" software was pretty unsatisfying.

There were technical problems, production problems, quality problems, and just uninspired content problems.

Often times, the game elements, such as the cartoon rabbit, were not things to which my son could relate.

The general genre itself, the adventure game punctuated by unrelated, abstract exercises, was pretty uninteresting, especially as it was disconnected from the content.

I wrote in Learning By Doing:

Some game-based models use adventures games, a genre popularized in early computer games. Students traverse maps, either from a firstperson or top-down perspective, looking for pieces of information or performing small tasks to unlock new parts of the map.
This was the model for much of the early educational software. ("Sure Miss Squirrel, you can come into our clubhouse filled with toys and friends. But you have to identify these right four physics laws to get the key." There are also corporation examples, that might read, "Sure Mrs. Squire, you can come into the executive board room filled with stock options and great contacts, but you have to identify the right four ethics policies to get the key.")

Besides the reasons mentioned by the article, Will Wright has mentioned one reason he thought was the problem: increasingly kids are choosing/buying software, not adults.

Here are some reflections I have for the edutainment marketplace:

• Interactivity is key, with real interactivity around the learning, not just propping up workbook style content.
• Math is especially tricky. Math doesn't exist on its own! Math is itself a pedagogical layer, a set of tools to better understand the world. It needs to be handled as such, not futher burried under more and more layers of abstractions.
• Game elements need to be specific to the user. Anything interjected to be "fun" has to be able to be modded very easily, almost like skins. Any student should have a choice of at least five different themes.
• Building rewards creativity, and gives more ownership to the player.

Having said all of that, edutainment needs to be re-invented. As with American car manufacturers after producing Gremlins and Falcolns, there is so little credibility that a new brand has to emerge.