You Want Numbers?

You want metrics data? I got yer data right here....

Check out PubSub's new LinkRank Service, measuring inbound and outbound links to any site! Check out Learning Circuit's LinkRank...

Blogging by the Numbers

Not everything that can be counted counts,
and not everything that counts can be counted.

- Albert Einstein

One of the projects in the revitalization of Learning Circuits Blog has been the establishment of a data set for capture and resulting metrics. I thought it might be of interest to share our work in this area.

The purpose for creating a trackable set of data and corresponding metrics is simply so that we will have an idea as to whether or not we are succeeding at this blogging venture. Simple purpose however, does not necessarily lead to simple implementation. Two factors contributed to this being a rather extensive task. 1) We had no data collection practice in place and 2) Blogger, for all it’s great features, just doesn’t collect data for their free users.

A couple free stat counters (thanks SiteMeter and Bravenet) and several tracking spreadsheets that border on frightening later, we have a solid set of raw data. The data fall into two categories – "Activity" which I use to refer to the posting and comments by the Blog Squad and guest and "Traffic" which refers to the comings and goings of all of you good folks. The raw data we are capturing include:

ACTIVITY	TRAFFIC
Posts Author Date posted Comments Comment Author Date Commented Days with no Activity	Unique Visits Page Views First time visitors Returning Visitors Referring Page Entry Page Exit Page

From these raw data we can build metrics to determine in what direction, if any at all, LCB is moving and why. There are all the ratio metrics (i.e., posts per day, comments per post, zero activity days per month, etc.) and comparison metrics (i.e., August was twice as active as July, or Clark leads all Blog Squad members in postings).

Some metrics which seem to be emerging as meaningful as everything settles in are:

1. Total Activity - Posts and Comments are weighted equally. It seems a better representation of the work that is being done by the Blog Squad.
2. Unique Visitors vs. Returning Visitors - This metric seems to indicate that we are drawing people in, but not doing enough to have them return in a timely fashion.
3. Total Activity vs. Page Views - this has been the real surprise thus far. The ratio of page views per post or comment seems to have settled in at just aound 24-26 pages. Including in August when Activity tripled and page view followed suit. It's too early to know if this consistency is a coincidence or whether it will turn out to be a dynamic predictive metric.

I have a couple of questions I'd love to get some response from the LCB community on regarding metrics:

1. Are there metrics you see as key when looking at a blog?
2. Do you know of any benchmarking sources for blog metrics?
3. Do you track metrics for your blog? If so, care to swap info with LCB?
   

Authoring Simulation Genres: Game based models

One well-understood simulation genre is game-based models.

With the goal of “making learning fun,” students engage familiar and entertaining games such as Wheel of Fortune®, solitaire, or memory, with important pieces of linear or task-based content replacing trivia or icons. More diagnostic than instructional, game-based models nonetheless might be the technique of choice by traditional educators and training groups looking to quickly goose their reputation, student satisfaction, and even effectiveness. Game-based models also introduce, in the purest possible way, game elements that all educational simulation designers will need to understand in the near future.

Some authoring environments include:
• EGames Generator
• Games2Train


Do people know of other toolkits for game-based models?

Authoring Simulation Genres: Interactive Spreadsheets

One well-understood simulation genre is interactive spreadsheets.

Interactive spreadsheets focus on abstract business school issues such as supply chain management, product lifecycle, accounting, and general cross-functional business acumen. Students allocate finite resources along competing categories at successive turn-based fixed intervals, and each time they watch their results play out on dense graphs and charts.
This is often done in a multi-player or team-based environment and often with facilitators.
The subtlety, unpredictability, and variability make them appropriate for training b-school students and high-potential supervisors through the direct reports to the CEO. They are often the cornerstones of multi-day programs to align a fractured department or organization by building shared knowledge and understanding.

• One authoring tool is:
  Forio Broadcast

Another is straight Excel.

Do people know of other great authoring tools?

Authoring Simulation Genres: Branching Stories

One well-understood simulation genre is branching stories.

In branching stories, students make multiple-choice decisions along an ongoing sequence of events around what to say to another person in a given situation. The decisions impact the evolution of the story, ultimately terminating in either successful or unsuccessful outcomes.
Their ease of use, ease of deployment, and content style make them highly appropriate for entry-level salespeople, call center representatives, freshmen, customer-facing retail positions, and entry-level managers. Any high-turnover position should be trained, although not exclusively, using branching stories. Companies like WILL Interactive have advanced the genre to handle more moral and complex situations, making them also appropriate for higher-level employees as well.

One tool kit is:

• Scenariation

You can also use PowerPoint, although it is tricky.

Do people know of other branching story toolkits?

The New Ways of Describing the Life of Content

I have been thinking a lot about life of content. A few different dichotimies seem to frame conversations.

Staged vs. Organic: Staged events are one-shot. They need significant pre-establisehd processes and project management. They use up a lot of advertising and communication. Organic approaches are more incremental. They use small layers to build up over time. Television shows are staged, but they can also evolve over many episodes, as writers seize onto relationships, or the right directors are found. Organic content lifecycles stress less perfection up front, and often more feedback.

Transient vs. Peristent : This refers to the size of the window of availability, and/or the timeliness of the material. Even these can be fuzzy. DVDs should be totally persistant, for example, but advertising and shelf-issues make them somewhat more transient. Also, as with movies, there is a perceived success (being number one on the charts) that drives more success, and more of a staged content approach. DVDs can also make movies a bit more incremental, with multiple versions available. Computer games can also have various versions, and can also be patched on the fly.

Controlled vs. Community: Is their one-voice shapping this or multiple? Mods for computer games can subvert a very controlled piece of content. From a formal learning perspective, I always ask, how can I get the training group out of the way? How can the community generate the content, and how can the training group help that?

Is there a "so what" for these lenses? I think so. One is to recognize our own biases, and accept that these biases might be interfering with coversations with sponsors. The other is to recognize the current trends that are pushing towards incremental, persistant, and community, and selectively embrace some of this approach, and challenge others.

Small Pieces Loosely Joined

When talking about platforms, the opposite of a single monolothic approach (cough Windows cough) is small pieces loosely joined.

We in the learning industries seem to be good at the

• big pieces, tightly joined (degrees, curricula, certification)
• small pieces, tightly joined (LMS delivered content)
• small pieces, not at all joined (Google, mapquest, bubble help)

So what is it that can/does loosely join our small pieces of learning? Is it our PC? Outlook? The database in Google Desktop Search? Our PDA? Is it our organization? Our annual review? Our interests? Company? Career? Industry?

At the end of our day, it is probably summed up in our résumé, our sometimes forced, often surprising organization of the things we have demonstratably done. But at the heart, we are our own platform, constantly both writing to others and being written upon, vastly ranging in productivity and capability.

I would say for the last few hundred years we have probably been Human 2.0. And we are getting close, I think, to v3.

The Lowly Binder

It is a symbol of "old school" training. It is a joke, a dinosaur, a deep sea anchor. It wastes billions of dollars of office space in filing cabinets and cubicle shelving. It is THE TRAINING BINDER.

We, the Learning Cognoscenti, "prove" our own credentials to the world by openly disdaining them. '"What to do about the classroom binder?' you ask. 'How about dig a deep hole and throw them all in!'" Then we chuckle and drink our cognac and tighten our smoking jacket.

And yet...

And yet there are still a lot of them being produced. Almost every classroom experience, and there are a lot of them, support the activity with binders. Conferences do as well, I need hardly mention. I, as dutifully as the next speaker, turn in my PowerPoint stack for inclusion in whatever event I am part. I also helped build a workbook to support rolling out next-gen-sim Virtual Leader, which significantly increased usage.

So it might be worth a few moments to, with appropriate humility and pragmastism, reflect upon what are some of the tools available that can make for a good binder.

• Removable overviews, often in the front sleeve, that provide context and a map to what comes next, or a tip/cheat sheet to an application
• PDFs of the entire thing, available for download and storage; perhaps zipped files as well
• Copies of slides, for people to take notes in real time
• Color paper and/or tabs, to distinguish between sections and aid navigation ("Go to the yellow section...")
• Biographies, with face shots and contact information, of speakers and ideally participants, at the back
• Appendixes, with articles and links
• CD-ROMS, with editable versions of slides and spreadsheets
• Clear cover and spine, with subject and dates
• Three-ring rather than bound, so that users can dynamically and deliberately add or subtract what they want or don't want.

What are your thoughts? What should or should not be part of a well constructed binder?

Emotionally Evocative Design

For many an Instructional Designer, design follows analysis, with its main function being to identify all the important things that need to go into a course. Its end product is a curriculum, syllabus, or blue-print to build the learning module on.

Next comes development, which adds content so as to give depth to the end product of design. This mainly consists of the adding of "information."

Context is also used to add a third dimension to the design puzzle -- layers of activity so that the learners gain a variety of viewpoints, thus allowing them to gain experience with the information in a relative safe manner.

This mix of design, development, and context should theoretically help the learners build their knowledge and skill bases. Yet this combination often fails because it leaves one important piece of the puzzle -- Emotionally Evocative Design.




While Instructional Designers normally are quite good at ensuring that the important parts that build content, such as objectives and outcomes, are entered into the design equation; the total design process needs a second layer that captures the emotions of the learners so that they actually want to engage with or use the content.

Engagement does not have to be that complex as it is simply a means of inviting the learners' emotions into the environment. Emotions are the reason that we do anything -- without them we would simply be walking zombies. Emotions are what adds zest to life...to include learning

For example, a good metaphor invites the user to reflect over the information by asking her to relate the new information with a past experience. A good picture invites two senses into the mix. A problem begs for closure. A critical piece of information delivered "just-in-time" is utter relief.

What have you done lately to ensure that your designs are emotionally evocative?

CNN vs Education

I would not suggest that higher education institutions need to operate like CNN, but I find it fascinating to read Elliot Masie's observations of how CNN dealt with the flow of content and information in the wake of Hurrican Katrina. In CNN Newsroom in the Midst of Katrina - "Rapid Development... Content Objects... Learning Implications":

There were some incredible learnings and observations as I quietly watched the news gathering and assembly process and interviewed the Learning team at CNN. Many of these items relate directly to how organizations will be assembling content in the near future.

The question is, will educational institutions be one of these organizations? Below I have take some of Maise's descriptions of CNN and put them besides a gross and likely over generalized observation of higher education. Yes, there are numerous exceptions and counter examples to every one of my points, but as a whole, when you read how CNN operates and put it besides how your higher education institution operates, the contrast should be rather vivid.

CNN: Content From Multiple and Unconventional Sources: The nature of content in journalism is changing dramatically as media flows from non-traditional sources. CNN calls an aspect of this "Citizen Journalism" as they receive pictures and video feeds from digital cameras and even mobile phones.

Education: Content comes nearly solely from instructors and "trusted" resources like publishers and "refereed" journals. And content from students would be suspect or written off as "cute" but not useful. Is there an equivalent of Citizen Teaching? If so it is the inspired individual's effort of placing a tutorial or resource on the net.

CNN: Content To Multiple Formats: As content was created in the CNN newsroom, it flowed to multiple formats. Content started as video feeds, became streamed video, text on the website and even a mention for a scroll at the bottom of the screen. Each piece of content was "tagged" as it came into the newsroom, timecoded, meta-tags were added with context and it could be viewed by CNN staff around the world in low-res format. The concept was to see each media object as being highly reusable and redeployable.

Education: Content primarily text, email, PDFs to print, hand coded/Dreamweaver-ed HTML, and.. PowerPoint. Media objects are un-reusable, un-findable, and in-redeployable and would never be available in this kind of time frame.

CNN: Digital News Gathering: The footprint and format for news production is changing radically as the size and mobility of equipment evolves radically. I watched newsfeeds coming from CNN reporters using satellite phones (after the cell network dropped). They were even feeding content that was edited on laptops in the field using Final Cut Pro.

Education: Not much to compare except we do not think of educational content being created "in the field" or with portable devices... while some are moving towards laptops in place of desktop computers, they are used primarily in the same vein (a laptop on the desk). We still print a lot of material or offload this printing to students-- we do not "think"/"operate" primarily in digital, and much of what we do digital is the digital equivalent of print.

CNN: Content Repository: CNN operates a content and media repository that is quite impressive. The content objects are viewable, editable and sharable. Key levels of data is kept for how each object is being used and deployed. Digital Rights Management is tracked, to honor the appropriate use of each media object. I was struck by how easily every CNN staff person could access and work with this content repository.

Education: This is still a dream, despite years of wrangling over "Learning Objects" and the construction of numerous "repositories" few if any that have the features described above AND as much content. Data on the use of objects is absent and DRM is spotty.

CNN: Rapid Development: While CNN clearly has a breaking news model, it was fascinating to watch this process in action, including use of templates, collaborative and team-based editing and content refinement, focus on content ethics, standards and legal/compliance issues. I witnessed a team of professionals, drawn from a wide set of backgrounds, deeply focused on producing content that had value for viewers and the hurricane's victims.

Education: The development cycle is measured in months, perhaps as summer projects or long term grants, and is pretty much an individual cottage industry. Editing and development are solo projects.

So, how does Big U stand up to CNN?

Education's Collage of Subsidies

I was thinking about Clark's (the other Clark) post on learner rights, and then the broader learner marketplace. One of the trickiest parts about development of formal learning programs is that so often, the learner only has a passing relationship to the cost of the program.

Enterprise Programs
Enterprise training programs are paid by the corporate, military, or government sponsor.
Their goal is usually some form of increased productivity or compliance.
Any vendor has to care more about meeting the need of the training director, and the training sponsor, than the end learner. Even the cost of the classroom environment is often subsidized by the enterprise, penalizing e-learning, outside classes, or even training outsourcing.

K-12
K-12 schools are paid by taxpayers, not the students of course, nor even the family directly (even a family that goes on lavish vacations, for example, gets their education subsidized (of course if they have big houses, they also big property taxes; and then there is the private school whammy, where they are paying twice...).).
As we are also learning, some K-12 activity is subsidized by softdrink/juice/water vendors, that come with strings attached.
There are grant based subsidies that push activities in one way or another.
And there are more subtle subsidies, making land cheaper than building supplies or technology, relative to other building projects.
Of course, K-12 is required of students, which can be considered an anti-subsidy (increasing its cost to the student, while still interfering with the learner marketplace). Lack of choice for any required course also represents an anti-subsidy.

College
In many college situation, parents significantly augment the students' ability pay at least some of the tuition. Then there are scholarships for academic, sports, and music. And scholarships, like other endowments, are subsidies by alums, meaning schools have to do things specifically to keep alums happy (while students do certain things to meet the criteria of the scholarship), not necessarily in the interest of the students. Furthermore, the reason many students put up with schools is to get better jobs, which means that while corporations might subsidize universities directly, they anti-subsidize them indirectly. Any certification, in fact, is anti-subsidized by the organization that values it

These are just a few examples from the K-12, Higher Ed, and enterprise world. But every subsidy and anti-subsidy distorts the natural value proposition away from the learner marketplace. Getting over all of these will be our challenge for the decade.

Note: This blog post has been subsidized by my most recent book, Learning By Doing. New book review here.