Intelligent Virtual Agents for Education and Training: Opportunities and Challenges - Abstract - IEEE Paper

Virtual Agents: “The current darling of the media,” says Forrester (I believe they refer to my evolving relationships with Alexa), from simple chatbots to advanced systems that can network with humans. Currently used in customer service and support and as a smart home manager. Sample vendors: Amazon, Apple, Artificial Solutions, Assist AI, Creative Virtual, Google, IBM, IPsoft, Microsoft, Satisfi.

Abstract. Interactive virtual worlds provide a powerful medium for experiential learning. Intelligent virtual agents can cohabit virtual worlds with people and facilitate such learning as guides, mentors, and teammates. This paper reviews the main pedagogical advantages of animated agents in virtual worlds, discusses two key research challenges, and outlines an ambitious new project addressing those challenges.

1 Introduction

Interactive virtual worlds provide a powerful medium for experiential learning. Navy personnel can become familiar with the layout and operation of a ship to which they will be assigned before they ever set foot on it. History students  can learn about ancient Greece by walking its streets, visiting its buildings, and interacting with its people. Biology students can learn about anatomy and physiology through adventures inside the human body. The range of worlds that people can explore and experience is unlimited, ranging from factual to fantasy, set in the past, present, or future.

Our goal is to enrich such worlds with intelligent virtual agents – autonomous, animated agents that support face-to-face interaction with people in these environments  in a variety of roles. Existing virtual worlds such as military simulations and computer games often incorporate virtual agents with varying degrees of intelligence. However, the ability of these characters to interact with human users

is usually very limited; most typically, users can shoot at them and they can shoot back. Those characters that support more collegial interactions, such as in children’s educational software, are typically very scripted, and offer human users no ability to carry on a dialogue. In contrast, we envision virtual agents that cohabit virtual worlds with people and support face-to-face dialogues situated in those worlds, serving as guides, mentors, and teammates. We call this  new generation of computer characters animated pedagogical agents [1].

2 Roles for Animated Pedagogical Agents

Research on animated pedagogical agents can draw on a long history of work in computer-aided learning, especially work on intelligent tutoring systems [2]. However, as discussed in detail by Johnson, Rickel, and Lester [1], animated pedagogical agents offer a variety of distinct new capabilities:

Best Mobile Phones under 30,000 Rs in India
Best Flagship Smartphones to Buy in India

Interactive Demonstrations. A simulated mockup of a student’s real work environment, coupled with an animated agent that inhabits the virtual world, provides new opportunities for teaching the student how to perform tasks in that environment. Perhaps the most compelling advantage is that the agent can demonstrate physical tasks, such as operation and repair of equipment. For example, Rickel and Johnson’s Steve agent [3,4] cohabits a 3D mockup of a US Navy ship with students, and can demonstrate procedures while providing spoken commentary describing his objectives and actions. Steve can respond to interruptions from students and use planning to adapt the demonstration to the state of the virtual world, thereby providing more interactive demonstrations than alternatives such as video.

Navigational Guidance. When a student’s work environment is large and complex, such as a ship, one of the primary advantages of a virtual mockup is to teach the student where things are and how to get around. In this context, animated agents are valuable as navigational guides, leading students

around and preventing them from becoming lost. For example, as Steve demonstrates tasks, he uses a path planning algorithm to lead students around a complicated shipboard environment. Similarly, the WhizLow agent [5] uses path planning to guide students through the internals of a virtual computer.

Gaze and Gesture as Attentional Guides. To draw students’ attention to a specific aspect of a chart, graphic or animation, tutoring systems make use of many devices, such as arrows and highlighting by color. An animated agent, however, can guide a student’s attention with the most common and natural methods: gaze and deictic (e.g., pointing) gestures. For example, Steve uses gaze and pointing gestures to draw a student’s attention to objects in the virtual world as well as to people and agents who are responsible for the next step in a task. Among other animated agents, Presenter Jack [6] is notable for its broad repertoire of deictic gestures, and Cosmo [7] is notable for its sophisticated criteria for choosing deictic gestures and accompanying referring expressions.

Nonverbal Feedback. One primary role of a tutor is to provide feedback on a student’s actions. In addition to providing verbal feedback, an animated agent can also use nonverbal communication to influence the student, which allows more varied degrees of feedback than earlier tutoring systems. For example, the Adele agent [8,9] nods or smiles to indicate agreement with a student’s actions, presents a look of puzzlement when the student makes an error, and shows pleasant surprise when the student finishes a task, while Herman the Bug [10] sometimes congratulates students by cartwheeling across the screen.