How do you measure the value of informal learning?

March 13, 2011
For this month’s BQ, this post speaks addresses the needs of instructional designers, media developers, trainers, and anyone who seeks to incorporate social networks, free infrastructure, and open source applications to improve human performance and operational effectiveness.

To answer this month’s BQ, I will:
•    Reveal the meaning of informal learning
•    Identify free immediately available informal learning applications
•    Tell you how to measure informal learning
•    Demonstrate how to measure informal learning
•    Show you the results and relate what I learned

To begin before you can ask how to measure informal learning, you must first answer what do I measure? To do so, you must be clear on the meaning of informal learning.


What is informal learning? What do you measure?
You cannot answer the ‘how’ to measure question without determining ‘what’ to measure first. Many times I am asked by peers, educators, and people interested in the transfer of knowledge and skills the question, what is ‘informal learning’. For the ‘how’ question, most often responses do not point to actual examples. Instead, you hear informal learning relates to social media, its potential value, and users prefer it. Oddly, no one can provide a real instance.

To state it simply, informal learning is anything you use, any means you take, and anyway anyone helps you to acquire knowledge, develop skills, and learn. For the over 50 crowd, you maybe asking yourself, ‘do you mean informal learning relates to things like books, phone conversations, and a co-worker showing me how to do my job?’ While texters and tweeters are likely texting and tweeting, ‘u mean informal lrnin is like TW, txt, IM, n Facebook?’
Well, the answer to both questions is an affirmative ‘Yes’ with conviction. To state clearly the meaning, informal learning relates to any means, technology, and technique used to learn outside of a classroom or formal learning event. In essence any means you use to teach yourself, develop, and problem solve through human intervention and media constitute an informal learning experience.

What makes informal learning different from formal learning is the following distinction: informal learning is not driven by a company led initiative, educational institution, sponsored training event, formal coaching, courseware provider, learning management systems, SaaS, etc. These are the instruments and infrastructure of formal learning. However, this is also where the definition between formal and informal learning become blurry, because in an unstructured way, informal learning leverages and consists of formal learning instruments, infrastructure, and people too.
What are informal learning examples?

To provide examples, I will now identify some common and immediately available free ways to introduce informal learning into a company, educational institution, and non-profit organization.
•    IM particularly among remote and field professionals
•    Twitter real time availability, product changes, needs, and events
•    Integrate Uudutu with Facebook or Linked In as an ad-hoc LMS and authoring tool
•    Release animation, video, and presentations to YouTube
•    Share files and collaborate through Google Docs

By following my suggestions, you can immediately make the sharing and restricting of knowledge available to everyone or specific groups for free. For those of you who are involved in training, professional development, teaching, and coaching as a bonus, I am now going to identify a few free formal learning tools you can use to deliver informal learning.
•    Model courses with chat, wikis, and blogs
•    Dimdim virtual classes with voice over IP
•    Skype coaching and mentoring relationships

Through these methods, you reduce the cost of your infrastructure and operational platform to free and you are likely to begin to see the value of informal learning.

When leveraging free and open source technologies, applications, and platforms, the question how do you measure the value of free is a question best answered by the companies who provide the services.

For measuring the ‘learning’ of informal learning, I would like to share a story before I show you how. In the 1990’s I had the distinct privilege to work with Jakob Nielsen at Sun on the usability for the initial sun dot com website. During the study, Jakob would say after three studies you are splitting hairs of standard deviation.
I find this an enlightening perspective and key for making real time decisions in the field. By applying his guideline when you ask a question to three subjects, you gain sufficient information to spot a trend and take action. To take more time to observe, record, and measure, fails to add value, when you could take corrective action instead.
To answer how to measure informal learning the answer is to ask a few top performers.
What do you ask? Who do you ask?

As an easy example, let’s look at a specific case study. To provide background, suppose you observe a high performing team use a free and immediately available instant messaging application throughout the day. From your observation, you wonder perhaps by making IM standard on company equipment people might informally learn, thereby, becoming more effective on the job.
To assess your hypothesis, you decide to ask those engaging in the activity as opposed to asking the whole company. Please consider, why would you ask those who do not use it? What could they tell you about the value? In this case, does more input help you to see the trend and benefits? Next before you ask the team, inform them of the intent so they can think about the topic beforehand. In other words, ask them for quality thinking rather than impulse responses. Finally, conduct a qualitative discussion and include a short survey to help summarize findings and set a course of action.


What questions do you ask?

For your questions, make them behavior based. In this case, how someone ‘feels’ about their experience is meaningless. Focus on behaviors. Using this approach, I asked five operational professionals about their use of IM at work and how it specifically relates to learning on the job. The remainder of this post provides the questions, responses, and corrective actions.

Survey Questions

How important is IM to performing your daily work?
1) Not Important; 2) A little important; 3) Somewhat important; 4) Important; 5) Critically important

How much do you use IM in a day?
1) One to two times a day; 2) Four to five times a day; 3) One to two times an hour 4) More than eight to ten times a day; 5) Too many to count)

Related to learning, how many new or critical items do you learn in a day through IM?
1) One to two a day; 2) Three to four a day; 3) Five to eight a day; 4) More than eight a day; 5) Too many to count)

In terms of learning and acquiring information you need to perform your work on a daily basis, what tool is most important? Pick one. 1) eMail; 2) IM 3) Phone; 4) Intranet homepage; 5) Training

How would you describe the receipt of IMs throughout the day?
1) Highly Disruptive; 2) Disruptive; 3) A little disruptive; 4) Not disruptive; 5) Part of daily workflow.

As you see from the analysis, focus on the mode. The average is meaningless.


What do you learn and what actions do you take?

As you can see from the data and analysis, IM is a critical frequently used tool by staff to learn on the job. The big ‘ahah’ moment for me was the revelation the need for training organically drives informal learning and professionals value formal learning.

As a trainer and performance consultant, I could not be happier nor have a better instrument to discuss with decision makers how to make best use of people, technology, resources, time, and training.
As a final point by adding two subjects after three studies did I learn something new or did the additional information reinforce an already established trend? In the case described like many, your course of action remains the same while the collection of more information takes you away from the critical corrective actions to take. In conjunction with this approach, I am generally perplexed by learning professionals who indicate a level 3 transfer of training evaluation is too time consuming to perform. IMHO, I do not believe this to be true while it missing the true indicator of training and developments value to an organization.
Read the actual survey responses and assessment of the data.

Leave a Reply