Friday, August 26, 2016
So here we are at the start of a VR journey that has only just begun. The first raft of consumer devices are out there – from cardboard and cheap plastic goggles that work with smartphones to headsets with external sensors, hand control devices and 3D audio. Facebook bought Oculus Rift, Samsung are in there, PSVR is in the gaming market. Vive are out there. Google have a platform. Everyone’s playing with this stuff.
I have experienced dozens and dozens of different VR experiences. Let me tell you that all of them have blown my mind. Immersive VR repeatedly reminds me of the power of technology, not as something we use as tools and machines, but to truly interact our minds. The recent example of VR to rewire the brains of paraplegics so that they gain some control over their previously inert limbs is a glimpse of a future where the brain itself can be changed for the better by technology. That is the game we as learning professionals are in – changing minds for the better.
If you are in the learning game, you should at least be aware of its possibilities as it delivers some unique and exciting learning opportunities. It is clear that VR delivers some marvellous benefits in terms of attention (full-on), motivation (exciting), experience (experiential learning), learn by doing (often ignored), context (real) and therefore transfer and retention. But what can you do if you want to use VR to educate or train? Here’s a quick, practical primer.
1. Immersive photographs
The Mars Rover is a good example, where you feel as though you are there on Mars. Geographical and geological locations that benefit from this 360 degree view. Try these free locations, such as the Temple of Karnak in Luxor - some are astounding. This 360 view of the inside of the International Spacestation is great. One can set learning experiences around this image – ask learners to find stuff, annotate, explain things. One can also do the inside of a building, vehicle, whatever. Here’s one of the Supreme Court in the US.
360 degree photos have been around on Facebook for years. You simply shoot a panorama on your smartphone, open the Facebook app and post to share the photo. You can either turn with your finger or move the phone. To take thinks into proper VR, with newer Samsung phones you’ll see an icon tat says ‘View in VR’. Touch that, insert your phone into a Gear VR headset and you’ll see it in fully immersive VR. This is a great way to get started, play with ideas or prototype.
2. Immersive moving video
You place a 360 degree camera, such as a Ricoh (£300) in the middle of the space you want to video. This could be inside a vehicle, in a classroom, office, gallery, hospital ward, outside location – anywhere. Press play and you will have a full 360 world of action recorded, not in as high a definition as the photographs shot in option 1, but usable video.
I’ve seen this used for teacher training, where an entire lesson is captured and used to train that teacher through feedback or as exemplary lessons for other teachers. Hundreds of short 360 degree videos have been shot and are replayable through VR. News items from the BBC are plentiful on YouTube. Not for profits have high impact, emotionally charged videos on climate change, refugee issues and endangered species. In education I can’t think of few subjects that would NOT benefit from the use of VR. Maths, physics, chemistry, biology, drama, geography, history, languages, business, design, art, vocational subjects and soft skills can all potentially gain from the creation of 3D videos.
This approach is great for attitudinal training, where an issue, incident and shift in viewpoints is needed. The fact that the learner has no choice but to be fully attentive means you can hold them and deliver emotional impact with the learning outcome that results in high retention. I can remember, in details most of the VR experiences I have had that tried to deliver this type of attitudinal shift, from being shown retreating glaciers, rare White Rhinos, refugees landing in Greece, the guy who was going blind. There are dozens of these on YouTube. I love this one on Pluto's icy surface. This one on Mars. You can illustrate the inside of a an aircraft cockpit, even stand with David Attenborough, as dinosaurs come up to you. VR 360 degree videos are of use when you want to show motion or real people doing things in a space. They are cheap and relatively easy to shoot.
3. Immersive graphic worlds
Sometimes, rather than simply point a camera and shoot a still or moving image, you have to create a world with graphics. This frees you to create anything your imagination comes up with. It may at the tiny, even molecular level, an ideal hospital ward, or an impossible environment, such as the deep ocean or out in space, in a war zone, inside a nuclear reactor, out in the solar system, on Mars being near a black hole.
The advantage of created graphic environments is the ability to allow navigation, as well as the creation of other graphic entities within that world, objects, avatars and so on. It is a manipulable world of bits. You will need to build your world using a 3D graphics package, like 3D studio, then use a tool such as Unity, Unreal or Stingray to build your experience into a functional 3D learning experience, deliverable on specified devices. So you've created a world but where do you go from here? Interactivity. This is where things get trickier.
4, Navigate through worlds
If you want to string still or video environments together to give the user choices on where to go, either by cuts, fade down/fade back up, as used in video or, if in a graphically created world, moving through the environment. This can be done by simply looking at options and pressing a directional pad on the headset (Samsung Gear) or with a controller (Oculus comes with a Microsoft Xbox controller). This greatly expands your possibilities in learning. You give the learner choices and can create levels, progress, games. Navigation is possible but involves programing and hte creation of instructions and a usable user interface. It can be done but needs professional design and coding.
Hotspots for pop-up explanations, explanatory videos appearing, graphics or audio is pretty straight forward, as you;re mapping hotspots to the sphere or world you've created in VR. You need to identify hotspots as hotspots. never confuse the learner with hotspots that look as though they're interactive but they are not. One can imagine learning scenarios that get the learner to actually click on things they need to find and identify. One can embed PowerPoint, videos, animations, whatever.
6. Manipulate objects
The next level is to allow learners to manipulate objects. This can be useful in doing experiments, building things, maintenance tasks and so on. I’ve put together pipes and components inside a nuclear reactor, lifted and used safety equipment on an oil rig, grabbed a clip floating around in space to clip myself to the Space Station, before floating around on the outside, used hand controllers to shoot things in games and pull over protection shields. This is all possible with hand controllers, which have buttons and triggers for grabbing, releasing, shooting etc. This is tricky with mobile VR as they tend not to have hand controllers, all manipulation having to take place through the touchpad on your headset. But Oculus and Vive have separate controllers which allow both hands to do things in 3D spaces, as well as grab and so on. Haptic feel is also coming.
7. Avatar interaction
Avatars (human-like characters) can be created and programmed to move within 3D created VR worlds. This allows learning interaction with pre-programmed avatars. Doctors with patients, sales people with potential customers, managers with employees and so on. This is fine for pre-set encounters and directed training, useful in sales, management and soft skills. You have to cope with interaction, either through preselected text options (your questions, requests etc) and their replies. Beyond this lies speech, spoken by you the learner and the avatars you interact with. One interesting aspect of experiential learning is tutor directions and interventions. This can be fed live into your ears by audio or the tutor can be an avatar within the created world. I’ve seen this work well in technical training on oil rigs. There is also the future possibility of tutor support through chatbots. I've been experimenting with these AI-driven chatbots and they can be trained to respond to questions, requests for action etc.
8. Multilplayer interaction
Take all of the above and more, in worlds that are real, created, built and where you can meet, communicate and interact with your friends, business colleagues, customers, teachers whoever. It is no accident that Facebook bough Oculus Rift for an eye watering figure of $2.3 billion. They know that this is where social media is going. The real world is 3D, social media is currently 2D. Fully populated worlds, you can enter, play in and do things beyond our current imagination are coming. It’s frighteningly exciting. This may sound outlandish and way down the line but complete learning environments like this are already on with VR, multiplayer and tutor intervention.
9. Create things
In the outer limits of VR, you can already create your own 3D paintings and sculptures and walk through them. I like these applications. You can choose your brush, create images in a 3D space, stand back look at it, walk through it, stand inside it. This open world, where you create what you want, has an allure that has attracted millions of young people in Minecraft, who do this as a matter of routine. When they can climb inside these worlds a new level of creative effort will have been realised.
10. Hybrid real/virtual VR
You can go on rollercoasters where you wear VR but experience all the thrills of a real rollercoaster with extreme G-forces. What you see through VR is a rollercoaster in a created graphics world, in space, wherever. The VR experience is synched with the actual ride. You can even choose which virtual world to experience and now interact in space battles, at, for example Six Flags theme park. They use Samsung Gear headsets and can enhance the experience even on the oldest of rollercoasters.
I’ve sat in chairs that vibrate as I unlocked the air lock on the International Space Station and used my hands to grip onto handles to pull myself out. If you want to explore, walk, even run, through 3D worlds, a rig in which you stand and walk without moving can also be purchased. The VIVE headset allows you to walk about within a pre-defined cube, made visible from within. External cameras have been added to some headsets. In the porn industry all sorts of ‘objects’ can be used in the simulated of 3D sex (teledildonics). The line between the real and virtual is indeed blurring. It’s AR (Augmented Reality) meets VR (Virtual Reality) meets RR (real reality).
Couple of things to remember with VR. By all means play around with the medium but it needs careful thought and planning. Know what you want to do, choose the right level and understand the limits of the tools you use to capture and create experiences, as well as the tools used to create navigation, interaction and manipulation within those worlds. You have to understand where to place cameras (sitting or standing) and be careful with lighting. Audio is also important. Understand also, the resolution of the final output. It may say HD but the resolution on video will be far worse than that on stills from the same camera. The cost of coding a VR experience is open-ended. Then there’s your target devices – mobiles (selected), Oculus, Vive…. All of these? Remember that these are not easy to deliver via an LMS and SCORM is tricky.
The first three options (1,2,3)allow you to simply create learning ‘experiences’ that are linear and directed. They are cheap to produce, unless you want exotic locations. The next three (4,5,6) allow the learner more freedom with fuller forms of interaction, other than simply looking around. From 7 onwards, things get complex and expensive. If you're interested in doing a project contact me.
Tuesday, August 23, 2016
7 FAILS in the way we practice 'failure' in learning
Failure, as we know, is a fundamental part of learning which I have explored elsewhere. Yet failure, in practice, is often used in learning to hinder rather than help learning. It too often becomes defined in practice as a deficit technique, rather than a formative feature of progress. Here’s seven examples of how failure can fail learners.
1. Language of failure
Far too much emphasis is put on final, summative assessment, at the expense of formative assessment, confusing and importing summative habits into formative processes. The summative language of ‘pass’ and ‘fail’ is a mutually exclusive opposition that makes little sense in formative assessment. We take a dualist attitude and transfer it, mistakenly, back across to the entire process of learning. Too many teachers and online learning programmes default to the language of failure, rather than the language of learning. The fact that you have yet to know or master something is a state of ‘not yet knowing’ not failure. Yet the red pen culture and lack of knowledge about feedback, deliberate practice, memory and the role of failure in all learning is endemic.
2. Language of gifted and talented
My heart sinks when I hear parents use these terms about their kids. Even worse, are schools and teachers, who should know better, using a whole raft of terms associated with these fixed ability terms. Attributing success to ‘talent’, ‘ability’ and being ‘gifted’ is disturbing from a head teacher or teacher. You don’t have to be a Dweck freak to realise how destructive this language is in learning. It fixes attributes and therefore demotes effort and practice. It also gives learners a get out clause. Even the learners who succeed with high marks stop at the pass mark and ignore the remainder. The rest, if they are branded as failures (not talented or gifted) will make less effort and many will drop out
3. Hands-up anyone
A good example of awful teaching practice is the ‘hand up anyone’ technique, beautifully exposed in Ferris Beuller “….anyone, anyone.”. The teacher asks a question. This is good as it forces the learners to try to recall the answer but only the ones who know the answer put their hands up and the rest feel deflated. The introverts are excluded, tehre's not enough time for true reflection. It makes no sense. The process of learning needs to be kept positive at this level, not some lazy ritual where people are embarrassed, even castigated for not yet knowing.
4. Whole task assessment
Rather than create, active, effortful learning experiences, where failure is part of the learning process, we set whole tasks and simply repeat those tasks. You don’t learn to write by simply writing, you learn hundreds, indeed thousands of small rules around spelling, sentence structures, punctuation, style and so on. It’s lots of tiny acts of failure and correction that lead to success. The ‘whole language’ movement, for example, led to decades of bad teaching and poor literacy, as it failed to recognise the role of failure in the learning process. Whole task teaching and assessment is the route to self-doubt and failure.
The ‘essay’ is a lazy and vastly overused form of assessment. A Professor of Pharmacology once complained to me that her University forced her to set essays for her Pharmacology students, which she found ridiculous. Smart students simply memorise essays for exams, so they are far from being an adequate form of summative assessment. Hand written essays encourage this as it is difficult to engage in critical writing, which always involves redrafting, structural change and rewriting. Waiting for weeks (the norm) to get a grade back (with scant feedback) on an essay, is a ridiculous form of formative assessment. I’ve lost count of the number of times I’ve come across parents writing essays for their kids at school and, unbelievably, University. Then there’s the simple fact that you can buy them. Encouragecheating and you’ll get cheats.
6. Marking as end-point
Unnecessary marking is another technique that confuses summative with formative assessment. Professor Black rightly criticises teachers for being over-zealous with marking, when they should be promoting learning. His advice is to drop marking during the formative learning as it does more harm than good. Let’s say a few get a pass by crossing some threshold, let’s say x%. Even with these learners this will act as an end point, leaving 100-x% of the knowledge or skills absent. That’s not good in healthcare, where that 100-x% can kill. It also demotivates those who ‘fail’, so that more damage is done to the whole cohort. For a more detailed account of why marking sucks, see here.
7. Deficit model
The education system is too often seen in terms of a deficit model, a dangerous conceit. Structurally it is layered like rock and the learner has to punch up through these layers while many fail to punch through at each stage. This deficit model, where the system is always failing, with failed schools and failed standards, pushes politicians and professionals towards a deficit model that defines the domain, policy and practice. The glass is always half empty as the language of failure is allowed to dominate. League tables, winners and losers , do little other than promote a culture of failure.
Failure is the end point for too many in this process. To promote and see ‘failure’, not as a means to an end (learning) but an end in itself, is to misunderstand its fundamental role in learning and memory. Sifting, sorting and ultimately abandonment, is to fail to understand the true joy of education and learning. For too many the end point is being branded as a failures. Turn this on its head and see failure as a state of becoming and you turn a fixed entity into a dynamic process and opportunity.
Friday, August 12, 2016
5 levels of FAILURE used to succeed in learning
Zero to One, warns us about fetishizing failure. He hates the old mantra about entrepreneurs having to fail to succeed. Failure, he thinks, can hurt those that fail, as well as the collateral damage that failed businesses bring – job losses, people not getting paid, suppliers with unpaid bills and bankruptcies. He has a point but in the learning process, failure that is limited to the individual, is most certainly a good thing. This blog is called PlanB, in recognition that we have a lot to learn from failure. In fact, it is an essential and, some would argue, necessary condition for learning.
This may be a bit non-PC but what made Europe a dominant force in culture, commerce and science, was the critical thinking that developed in Ancient Greece. This continued, with a long Dark Ages interlude, when religion all but extinguished this mode of thought, to the development of the scientific method and the idea that all knowledge should be seen as subject to scrutiny, tested, and even then still open to future challenge. Quine applied this to all knowledge. It has held us in good stead.
Learning through failure
Learning is cognitive improvement. It is all about moving on from one mental state to another that improves performance. These small steps forward are, in fact, built on many of small failures. You learn to drive a car by adjusting thousands of small acts of over-steering, going too fast, too slow, taking the wrong line on the road, braking too hard…. You learn by building on many, many small acts of failure. Learning to write means making lots of spelling, punctuation and stylistic errors, eventually getting there over many years. The feedback loop try-fail-learn-repeat lies at the heart of the learning process. Unfortunately there is often a fear of failure in education and training, sometimes even a blame culture around failure. As an antidote to this, here are five levels of failure that one can use when learning or designing learning experiences.
Level 1. Failure recognition
We have all experienced those small, sometimes big, sometimes catastrophic experiences of failure, even humiliation. The teacher that told you that you’d never amount to anything, the exam failure and so on. Actual failure is compounded by the fact that the learning game is soaked in the language of innate ability not development and learning. From ‘Gifted’ children to ‘Talent management’, professionals use the bizarre language of fixed ability, often without realising the consequences.
The first step on the failure curve, therefore, is to recognise and encourage what Dweck calls a growth mindset. This does NOT mean endless praise, which can seem inauthentic and get counterproductive. It does mean encouraging learners to strive for improvement and, importantly, not let failure be the road-block it so often is at school or in other areas of human endeavour. The simple recognition that failure is normal, happens to everyone, and, when seen as the natural step towards improvement, can be turned from a negative to a positive, is a mainstay of good teaching and learning.
Level 2. Tons of tiny steps
Mathew Syed’s book Black Box Thinking draws on many examples of successful learning through failure. One stands out. When David Brailsford announced in 2009, that Team Sky would win the Tour de France ‘within five years’ no one took him seriously. Within three years Bradley Wiggins became the first Brit to win the race. Sure, he had a goal but that is never enough. A focus on ‘leadership’ and ‘goals’ is never enough. It is all about what Brailsford calls ‘marginal gains’, tons of tiny steps, all adding up to bigger success.
In any learning domain this is all about breaking things down into their constituent parts, mastering identifiable competences, and getting them right. So much education and training remains aloof in high levels of abstraction, hazy platitudes and generalities. What is often needed is attention to detail. This is now commonplace in sports’ training but not so common in education and L&D. It should be. In teacher training, for example, far more attention should be paid to specific things one can do to improve your performance as a teacher, through mentors or video captured performance and feedback. If it’s about actual practice, lectures on learning theory are not enough, deliberate practice and improvement really do matter.
Level 3. Deliberate practice
Anders Ericsson studied the role of practice in sport, music, medicine and other domains, where learners move from being novices to experts. He identified several characteristics that distinguish ‘deliberate’ from simple ‘repeated’ practice. First, concentrate, as there is no real learning without attention. Second, break down the task or skill into its constituent parts, so that one you build positively on failure at this micro level, rather than get discouraged by massive failure at the macro level. Third, focus on feedback from failure, either by yourself or by a coach or teacher, as conquering many small failures is the engine at the heart of learning. Fourth, increase the challenge to accelerate the rate of progress. Push yourself beyond your comfort zone, accept the fact that you will fail but embrace this as the price you pay for progress. This is called deliberate practice and upward trajectory based on overcoming failure.
Level 4. Catastrophic failure
Let’s up the stakes once more. Safe failure in dangerous or lethal tasks is the most obvious examples of failure as a means to a good end. Pilots can crash and burn on flight simulators. Doctors can train on surgery and other simulators without harming or killing patients. Emergency service personnel can deal with fire and other incidents without anyone getting hurt or dying. Why do all pilots do simulator training? They go down with the plane. Maybe we should see most, if not all competences, in that light. We should be allowed to push ourselves and accept that safe, catastrophic failure is a force for good.
Simulations, boosted by cheap, consumer price AR and VR will happen over the next decade or two. This will bring realistic, contextualised, learn by doing, attentive learning, that allows as much failure as is necessary for effective and speedy learning, way beyond most classroom training. This is a fantastic opportunity to push learning away from its current theoretical bias towards more realistic practice and success – real performance.
Level 5. Reboot
Let’s push this to one more level. An even stronger form of failure is Reboot failure, where you identify failure, stop and send the learner back to the start of a level or learning experience. This is the secret sauce in successful gaming. You shoot away, get killed, get sent back to the start of the level and try again. Why is this such a successful and addictive feature of gameplay? It’s all to do with accelerating learning. You, in effect, learn how to learn. Being subjected to failure checks your progress (constant assessment), sends you back (repetition) and motivates you to try again with greater effort or knowledge (learning). It’s a virtuous cycle.
This is the one feature of gamification that I like – Reboot failure. Forget all of that Pavlovian froth – collecting emeralds, silver coins and running around pac-man mazes, and focus on risk-reward failure within levels. Allow the learners to try things and fail. But when they fail, the equivalent of being killed in a shoot ‘em up’ game, send them back to the start of the level to start again. Don’t be scared to punish failure as it not only delivers repeated practice but the learner comes back eager to overcome that failure.
There is even a games’ genre that takes this Reboot failure to another level – survival games. In No Man’s Sky, procedurally generated, never-ending, you explore a vast universe of planets but if you die, you get reset back to the start of the game. Get the right balance between challenge, failure and success and you have a multi-million dollar game or a brilliant learning experience.
The most spectacular successes in human progress have been grounded in the recognition of failure. From the critical thinking of the Greeks – pre-Socratics, Socrates, Plato, Aristotle and others – through to the scientific revolution, where we see the world as something to be subjected to challenge, testing and falsification. Potential failure or falsification has led to astounding advances in art, medicine, engineering and technology.
The airline industry is an admirable example of the relentless pursuit of safety and quality through learning from failure. It’s in their DNA. If only that attitude and process could be applied to education and training. Yet the opposite seems to true. We wallow in the world of gifted programmes, summative assessment for selection, lectures, essays, talent management…. The world of learning is a failure factory, not in the positive sense of learning from failure, second chances and progress but one of selection, road blocks, disappointment, discouragement and real failure. As professionals, we seem to have lost our critical faculties, stuck in a time warp of old theory and models that were never verified in the first place; lectures, hands-up anyone, Maslow, Myers-Briggs, Learning Styles, Piaget, NLP, Kirkpatrick. This is not good enough. It introduces certainty where there is nothing but ideological belief and unverified theory and practice. We need to think critically and see failure as part of what it is to learn.
Thiel P. (2015) Zero to one. Virgin Books
Syed M. (2016) Black Box Thinking
Ericsson, K. Anders, Krampe, Ralf Th. and Tesch-Romer. Clemens (1993) The Role of Deliberate Practice in the Acquisition of Expert Performance. Psychological Review, Vol. 100. No. 3, 363-406
Ericsson K.; Prietula, Michael J.; Cokely, Edward T. (2007). "The Making of an Expert". Harvard Business Review (July–August 2007).
Ericsson, Anders K.; Roring, Roy W.; Nandagopal, Kiruthiga (2007). "Giftedness and evidence for reproducibly superior performance". High Ability Studies.
Sunday, August 07, 2016
Little tribute to Tim Berners-Lee from the learning community
He wrote his proposal in 1989, redistributed it in 1990, when it was accepted and the first website, at CERN, was up and running in 1991. With three simple standards HTML (write your letter), HTTP (delivery of your letter) and URLs (postal address), he invented a way to use the internet to publish, distribute, send and receive information. This, for learning, was an invention on a par with writing and the printing press.
Web of people
From the very start the web was used to share academic knowledge and collaborate on learning and research. It was, in effect, a knowledge sharing network. Berners-Lee understood that he was creating a web of people, connecting people and so creating a social effect. Beyond this his vision was also of the intelligent analysis of the data that the web creates. He looks forward to the emergence of a true semantic web, which should make this possible. In this sense the web, for Berners-Lee is always a work in progress.
Enables online learning
Without the World Wide Web there would be no search, web content such as Wikipedia, open educational resources, online learning, online games, online book stores or social media. With the humble hyperlink it changed forever the way content is written and read. We can move through content, drill down into content, get help and learn in a way that was difficult with largely flat, linear media. Of course, media other than just text was shared as images, audio, animation, video and now 3D worlds became available.
An important principle for Berners-Lee, is Open Educational Resources. Berners-Lee favours Net Neutrality and defends the position that the web should not be controlled by companies or governments. Some open educational initiatives have become major forces with hundreds of millions of learners using their services, such as Wikipedia, Khan Academy, YouTube and MOOCs. The promise of free at the point of delivery learning has already emerged with new business models, new forms of delivery and new forms of pedagogy.
Billions are online and almost all learners who are online use the web to find things out or to enhance their learning. We have seen the web evolve from websites to knowledge bases (such as Wikipedia), rich media (YouTube), self-paced online learning and social collaboration. Artificial intelligence through adaptive learning promises to make further advances in personalising learning and VR will bring us a new medium for learning. We are only at the start of a process where new forms of learning and pedagogies will emerge.
Saturday, August 06, 2016
40,000 years ago media was 3D - it’s making a comeback
human and animalform. Our ancestors’ brains understood that re-presentations of the 3D world should be in 3D. Even cave paintings took full advantage of the rock shapes and contours. Recent research shows that cave art may not have been ‘art’ at all but a more utilitarian tool – the cave as classroom, where we learned to be a predator and avoid being prey. It would seem that the images had real narrative purpose, showing the actual appearance and hunting habits of animals.
Media is 2D
Around 3400 years ago, we invented writing upon 2D surfaces; clay, papyrus, bark and paper. This form of semantic communication has remained a powerful, flexible and dominant force. Painted art on 2D walls and 2D canvases also came to dominate domestic decoration and art.
Writing, print, black and white photography, colour photography, black and white movies, colour movies, back and white television, colour television, green computer screens, colour computer screens, HD TVs, tablets, mobiles – what do they all have in common? They are all 2D.
TVs have been getting bigger and with higher and higher fidelity. Indeed we are capable of creating fidelity beyond that which the human eyes can perceive. This attempts to recreate 3D immersion in 2D by increasing the field of view. It also increases suspension of disbelief by bringing the immersion of cinema into the home. It is 2D that strives to be 3D.
The battle of mobile screens has led to a battle between mobile and tablet screens. Phone screens got smaller, then bigger, eventually settling on screens large enough to satisfy viewing needs but small enough to hold in the hand and put in one’s pocket. Phablets essentially outgunning the tablet market. All of this has taken place within the 2D paradigm of flat screens. Even the 3D worlds in gaming, where players moved around within 3D environments were still 2D.
McLuhan and Baudrillard reflected on these changes and the shift from the real to simulated and hyperreal. As the consumerist world of communications, art, entertainment and work become more virtual, we spent more and more time in this new realm. Yet it has remained stubbornly 2D. This is because the technology we invented and made available was limited. Black and white photography, movies and TV existed, not because we loved the ability of that medium to represent essence – we had it because silver nitrate and limited broadcast technology was all we had.
Yet human perception is 3D
But here’s the thing, contrast our media with the reality of human perception. We see in 3D, we hear in 3D. We have stereoscopic vision and hearing which creates enough data for the brain to recreate 3D worlds. Our two eyes recombine two separate data feeds to recreate in out brains the 3D world we perceive. Our two ears are subtly folded to create sound shadows and catch the exact location of sounds in 3D space. We feel, smell and touch in 3D, even our balance and sense of direction are in 3D. Consciousness itself is in 3D. Yet our media are still largely 2D.
Media now 3D
This brings us to the here and now, and a break point in the evolution of media, where AR and VR have emerged as viable consumer technologies. Our media now match perception and deliver data that can be seen as we see the real word – in 3D. We can now experience 3D media that truly match our human needs and 3D consciousness. That word ‘experience’ is important, as it is the distribution, decentralisation and democratisation of ‘experience’ that is now possible. So let’s explore the several levels and layers of re-presentation in 3D media.
AR (Augmented Reality) as 3D
Augmented reality retains our 3D vision as its backdrop. It places 3D images into that already perceived 3D world. This can get quite complex, as it blends different realities. Pokemon Go, for example, presents layers of reality. Let’s start with consciousness (in itself a complete re-presentation of reality), it recreates the real world as you walk around in that 3D world looking for your virtual Pokemon. There’s maps (an idealised mapped representation of reality) where you as a represented avatar walk around and encounter Pokemon and Pokemon stops and gyms. The camera view (a photographic representation of reality), Pokemon and all the other imagery (superimposed upon the other realities), are all eventually framed back within your conscious view of these realities. And don’t forget the internet (itself a created reality) and GPS (a created dynamic co-ordination path within both the virtual and real). One could add a social reality. It all comes back to your conscious mind simply bringing them together into one conscious, blended reality. This is heady stuff.
The augmentation that Microsoft’s Hololens brings is to create 3D holograms within your perceived 3D world. These appear as if being there and are even persistent – if you come back, they’re still there. With Magic Leap, based in Florida, the direct projection of 3D images onto your retinas brings with it remarkable levels of seamless 3D augmented realism. It’s potential is reflected in the largest round of C-funding ever, at $793.5 million. It has raised $1.4 billion and is valued at $4.5 billion – and here’s the rub – it doesn’t have a single developer company or customer. This is a pure tech play. What Magic Leap does is play to the 3D capabilities of the brain and present data that allows the brain to seamlessly integrate projected images from the goggles to your retina. This is blended reality is s like “dreaming with your eyes open”.
VR (Virtual Reality) is 3D
VR offers total 3D immersion, where you really feel as though you are in another place – that feeling is called ‘presence’. It gives you full 3D experiences of games, entertainment, locations, imaginary worlds, education, training – anything – in your mind. It is this access to 3D ‘experiences’ that makes it so compelling. When you first try VR, you get an ‘aha’ feeling. Wow – I’m in a created world that feels real. That’s because ALL media you’ve ever tried before has been in 2D. We’re so used to being presented with 2D print and screens, that it comes as a real shock to see the represented worlds as they really should be – in 3D. The epiphany is that we can now experience experiences as they are meant to be experienced.
Within a $99 Samsung Gear or even cheaper Google Cardboard device, we can use our mobile phones to deliver full, stereoscopic, 3D virtual experiences that make you feel as though you are in a different reality . These can be captured as 360 degree photographs, 360 degree video or completely built as graphic worlds. We can move around within these 3D worlds, manipulate build and take apart objects within these worlds, even interact with other virtual (created avatars or avatars that are real people. We can even hybrid the experience by, for example, sitting in a real roller coaster feeling the real G-forces, while seeing a thrill ride that goes through space. The only limit to what can be built and experienced is our imaginations.
Like AR, the thing about VR and 3D experience is that it speaks to our brains in a way that resonates physically, emotionally, even sub-consciously. It literally becomes an extension of consciousness, where all distance is eliminated. In my own field, education and training, the promise of full attention (a necessary condition for all learning), emotional input (important in learning), learning by doing (very important in learning), relevant context (proven efficacy in learning), transfer (proven in flight sims) and therefore faster learning with higher retention and recall, is now a reality or virtually a reality.
New media rarely completely knocks out old media and we have had 40,000 years of evolved 2D media from cave art to screens. But are we at the start of a new era, where media actually deliver what our brains expect – 3D realities, artificial realities, mixed realities, augmented realities, virtual realities? This is as it should be. We are being taken back inside the cave and out of the dark emerge 3D images that are exciting, terrifying and new.
Sound is 3D
Interestingly, radio, talkies and the telephone are of another order, as they are not flat and were 3D from the start, with a different technological trajectory. Music as distributed on wax, vinyl, tape, CD and now digital was always disseminated in 3D, for the obvious reason that the physics of sound is 3D – it can only be 3D.
Sculpture and iconoclasm
Another exception to the dominance of 2D media was sculpture, embraced by most ancient civilisations. In China, Egypt, Greece and Rome, 3D sculpture was revered. Only Islam rejected sculpture and representation of animal and human form as idolatrous. Yet 3D representations have always eventually become objects of idolatrous or sexual suspicion. Throughout history iconoclasts have repeatedly attacked 3D representation. Worship of 3D objects was seen as idolatry in Judaism, Christianity and Islam. All of these religions would periodically purge themselves of 3D representation. Perhaps the most sustained attack on 3D representation is in Islam. Although not in the Quran it has roots in the removal idols from the Ka'ba in Mecca and Islamic iconoclasm has endured to this day, extending to the destruction of ancient sites by the Saudis and ISIS. In Christianity, the Byzantines and then the Protestant Reformation saw the widespread destruction of statuary. Iconoclastic riots were common throughout the 16th century in Europe. They invoked the Commandment that forbade graven (sculpted) images. Political movements, both Communism and capitalists have also destroyed each others’ statuary. The symbolic toppling of Saddam Hussein’s statue was a memorable moment in the Gulf War.