Friday, October 28

Virtual Classrooms and Virtually No Data

Face it, you generally can't see participants' faces in a virtual classroom. How do we know they are getting it? Are they sorting spam in Outlook or checking sports scores on ESPN.com?

I remember quite a few high school & university classes where class participation counted as a relatively significant part of your grade. With most integrations between virtual classrooms and LMS products it seems like launching the classroom URL is about all that is tracked.

This strikes me as the equivalent of getting credit for class participation by walking in the door. At least some of the better integrations check to see that you didn't walk out (leave the meeting URL) before the end of class, or they have a "completion" threshold of some sort (percentage of meeting actual duration or, minimum time spent in the session).

A good virtual classroom facilitator or instructor draws the participants into the content with chat, pools, simulations and all sorts of interactions. But how is that tracked or managed? I might remember the names of a few active participants, but then again eyewitness reports aren't always accurate. Allowing the instructor or leader to enter a value or description for participation after the fact is a start, but how about automating this too.

Could we have integrations that reported back individual's quiz scores from virtual classes, a participation status indicated by upstream VoIP, frequencies or length of chat entries, questions asked?

Is anybody else wanting more real data from virtual classes? What data would you want and how would you foresee using these reports?

Tom King, Macromedia

2 comments:

Ben Watson said...

Tom,

The biggest problem I have with virtual classrooms is that for the most part they carry over all the mentality associated with the traditional classroom. In particular this notion that my learning starts when I enter and stops when I leave. (honestly, the best virtual classroom events have gotten me thinking after the event is over and I have had a chance to think about it / apply it)

Another thing I don't like is this focus on measuring engagement. Based on my experience at SmartForce/SkillSoft less than 10% of learners are visibly active/engaged (often closer to 1%) but the reality is that 90%+ are lurkers, i.e. they are learning from the interplay between the active participants. Good luck measuring that.

What I want from a virtual classroom is the ability to interact before and after the classroom event so that my learning is continuous. To me a virtual classroom is simply another way of engaging a learner and should be part of an overall tool box of options.

Who cares if I multitask during a virtual classroom? I rarely give a task my undivided attention, I like my "continuous partial attention". It helps me fit my learning into my work (workflow learning?).

I would rather see a permanent, always open, virtual classroom with asynchronous collaboration options (knowledge bases, discussion forums, etc) in addition to the synchronous 'meeting' options. I guess I would call it a 'collaborative learning environment'; not a static 'virtual classroom' for it is all about blending and remixing what already exists.

FYI - I am of the firm belief that data reporting has had a devestating impact on eLearning, especially dumb reports around logins and completion rates. Maybe it's because we measure the wrong things, maybe it is because learning is not easily measured. Hopefully we won't see this happen with virtual classrooms.

Peter Isackson said...

I have to agree with Ben. Learning and data should be seen as two entities with a very low rate of correlation. But because data can be assessed and managed, laziness pushes us to consider it a valid substitute for the appraisal of learning.

In a truly collaborative context, the result can be seen in the quality of output rather than the rate of attendance, the number of fulfilled tasks, the amount of study or the number of questions answered correctly. Where I come from, we tend to call the desired result "operational skills" rather than "acquired knowledge".

True professionals can intelligently approach even those questions about which they know little and arrive at creative and appropriate solutions. Their primary skill is framing and reframing, not displaying knowledge. They possess the ability to construct critical interpretation and engage in actions that tend towards solutions. Learners can work constructively in the same way... if they are allowed to! This achievement may be difficult to measure "objectively" (i.e. by an impersonal process), but anyone with truly operational skills can appreciate the same with little difficulty or ambiguity.

In any case, attendance is clearly NOT a significant operational skill.