Rob’ autism Project: Social Learning at the Center of the Therapy Volume 2 - Issue 2

Sophie Sakka*, Re Nald Gaboriau

• Central Nantes, Laboratory for Numerical Sciences of Nantes (LS2N), France

Received: September 04, 2019;   Published: September 23, 2019

Corresponding author: Sophie Sakka, Central Nantes, Laboratory for Numerical Sciences of Nantes (LS2N), France

DOI: 10.32474/ARME.2019.02.000132

Abstract PDF

Abstract

Rob Autism project was started in 2014, it aims at setting a therapy support for ASD (Autistic Spectrum Disorder) teenagers based on robot and culture mediation. The center of the therapy support is on the notion of liance, the capacity to build and destroy links with the environment. Three points were addressed to restore liance: individual, collective and social liance. The results show a redefinition of the subjects’identity and their legitimacy as members of the society. The experiments were organized in 20 working sessions of 1 hour and involve six teenagers and three robots; the program is concluded with a final public show restituting a robotic play designed during the working sessions. Up to now, four groups were studied (24 participants in total).

Keywords: Rob ‘Autism; Autistic Spectrum Disorder; Social Learning; Robot; Programming

Introduction

Using robots as mediators in therapy support of children and teenagers with Autism Spectrum Disorder (ASD) has been performed for five decades. The first official project was lead by Dautenhahn and Werry in the 90s 5, with the Aurora project. They propose four robotic platform to study the behavior of ASD children facing a robot, from the observation that ASD candidates had a natural attraction to electronics (tablets, computers, robots or other machines). Dautnhahn and Werry developed the robot companion paradigm (Robots as so- cial actors), where the robot has preprogrammed functions to behave like a human being. It was used to solicit an interlocutor to perform a given exercise concluded by congratulations (successful answer) or request to do again (wrong answer). The12 Insert Authors’ Names here robot would either face a single child (communication skills), or a group of children (reproduction of movements).The companion robot paradigm was accepted by all researchers since then, and companies develop software to improve it, to make it behave more like a human being, like a friend, a tutor, a teacher or a protector. The studies focused on different aspects of the exercises to be performed to improve specific skills like communication, behavior, motion, and so on [1-5]. Many researchers also studied the aspect this companion robot should have (humanoid vs. non humanoid), should it need articulated language or just generate sounds, lights and color. Several robots were designed specifically for these studies, which could not be reproduced by other researchers (nor approaches compared, as a consequence) because the robots were not easy to obtain, expensive and needed specific robotic skills. In 2009, Softbank Robotics released the NAO robot which allowed many research teams to work with an affordable robots, the robots being the same in the different team (the effect of the appearance of the robot disappeared and comparison of the approaches were more realistic). New exercises and solicitations for the ASD children were developed to improve therapy support based on the companion robot [1,3]. Still, no current answer exists to the following question: how do we replace the robot by a human being and obtain the same reactivity with the human than with the robot?
The rob’ Autism project was initiated in 2014, proposing an alternative to the companion robot paradigm. We have called it the extension robot paradigm, in which the ASD person programs the robot to make it act on his / her behalf. The operator becomes at the same time actor and spectator of his actions, remaining protected behind the (rigid) body of the robot. In this approach, the operator is not solicited by the robot, but becomes an acting person in the world. This paper addresses the definition of a social individual to explain a possible approach to use robotic mediation in therapy support for ASD teenagers, pointing that the robot allows modifications of an individual but does not operate the modifications. Then the therapy must be worked out carefully to accompany the subject to rebuild his identity. Section 2 presents the specific points of an individual definition that are used to design the therapy support, section 3 describe the context of the experiments. Section 4 focuses on the three levels of liance that are worked out during the 21 workshops of the program, sections 5 analyzes the obtain results.

Between the Self and the World

An individual is that which exists as a distinct entity within a group. It is a person separate from other people and possessing his or her own needs or goals, rights and responsibilities. Insert Paper’s Title here 3 The frontier between the self, defined according one’s frame, and the world (the non-self), results from a complex process which relies on several notions such as envelop, linking / delinking dynamics, otherness and selfhood. This frontier is changing, in the sense that it can be affected by external events (the making, adaptation to the environment) or by internal events (the being, search for balance). It guarantees a subtle and fragile balance of the individual, as it is permanently crossed by a flow of information from outside-in and from inside-out. The frontier leads to the definition of identity, which is studied by numerous research areas: psychoanalyze, psychology, sociology, anthropology, semiotic, and so on. The individual can be separate in two parts: a part that constitutes it, and a part that allows it to (inter)act. Its constitution includes two elements [1] a fixed, rigid and structured frame leading to a set of unchangeable rules, knowledge and certainties [2] a strong-dynamics built from a numerouscollisions between realities and imaginary, with a role of balancing regulation (sensibility). The permanent movement is essential as it has a balancing regulation role (the sensitivity). In this dual situation fixed frame / dynamic motion, the constituted individual evolves in a half-space: mobile and immobile, at the same time.

Interaction is based on perception and involves the notion of liance. It is a balance between reliance and defiance. The reliance is the capability to establish links with the world (transduction). It is said that it cannot happen without mediation. dalliance is defined as “non-reliance”, and groups the capabilities to destroy links with the world. The permanent flow of data between the individual and the world is regulated by his/her capability to voluntarily create and destroy links with other social individuals or objects. The rules constituting the frame can evolve with the equilibrium between reliance and defiance, it is the social learning process. This learning is made difficult in case the reliance deliance events are too numerous, in which situation a rigid frame cannot exist; and this too permanent dynamics generates anguish. The notion of temporality is also important here, as the notion of time can only be difined with an adequate identity definition, i.e. in a rigid, existing and identified frame. A robot is defined in a half-space from its design: inanimate and animate, at the same time. Real and imaginary, at the same time. A robot is a mediator, it connects to the individual through his / her capability of reliance. Moreover, the robot offers a simplified, rigid framework in which some actions can be performed, which is reassuring for the user, who feels safe when interacting. The robot is particularly adapted for ASD support, but the robot does not itself operate the change: it takes the operator / interlocutor in its halfspace with the feeling of safety that frees the capability of social learning. When the operator / interlocutor is in the half-space, he / she can be reached by a therapy. The work we proposed in the Rob’Autism project, and present thereafter, consists in rebuilding the liance to redefine an individual with ASD, from face-to-face discussions to group and society acknowledgment as a person.

Rob’Autism Project

Rob’Autism is a multidisciplinary project linking medical, social sciences, arts, robotics and computer sciences fields. It was born in 2014, and is a collaboration Nantes (school of engineering) and the non-profit organization
Robots! (Robotics and arts). The project is supported by private organizations mentioned in the thanks section. Rob’Autism consists in a therapy support for teenagers with ASD; it lasts 21 weeks and it is based on voluntary interactions between participants and the world using robots as prostheses in communication. The participants program the robots to make them do or say whatever they want, within the techno- logical limits of the machine. The robotics and non-robotics workshops, as well as global organization and observed results of the program, were described in [4-10].

The Robot Extension Paradigm

Rob’Autism program proposes several originalities in comparison to classical ap- proaches, among them the fact that the robot is not used as a companion. The robot is not making any pre-programmed solicitation to a participant, but the par- ticipants program it and use it as an extension of themselves. through the robot, they can safely interact with the world. They do not behave as followers, but as actors. They transfer their own creativity into the world, and their creativity, their contribution to the world is recognized by the world. This allows them to differentiate what acknowledge (the world) and what generates (the self), identify the parties and have a better view of their frontier. When the companion robot paradigm pre- vents from closing on oneself, the extension robot paradigm offers an opening to the world, redefining identity and replacing the individual at the center of the action. As previously mentioned, the robot does not operate the change in the operator / interlocutor. But the use of a robot is fundamental (particularly humanoid robot), as it places the operator / interlocutor in a halfspace in which he / she can be more easily reached by a therapy.

Subjects and Material

24 subjects, distributed in four groups of six ASD teenagers aged from 11 to 16 years old participated in this experiment: one group per year from 2014 to 2018 (21 boys and 3 girls). All subjects’ parents gave written informed consent before entering the study. The 24 subjects had some ability to read and write, but not all of them were going to school for their education. They were all familiar with the use of a computer. Some had met the other participants and workshop staff before the program started, and some discovered them at the first session of the program. There was no selection on a specific kind of autism. During one complete program All the robotic sessions happen in the same room, and all the non-programming sessions happen in a same room too, but different from the programming sessions. The program uses 3 humanoid robots NAO from Softbank Robotics. The robots are programmed by the subjects using the software interface Choregraphe, which is the classical programming interface sold with the robots (i.e., no specific software was used for the programming). Let us remark that that software language is English whereas the participants were not familiar with this language. For each session, except from the 6 participants, 5 people attended (always the same during one complete program): three nurses who helped the participants focusing on the exercises and shared the discovering with them. The nurses had a quick training on using the robot before the programs started. One animator was also present in the room: either a robot specialist for the programming sessions, or a sound specialist for the nonprogramming sessions. At last, one supervisor led the program and attended all the sessions (programming and nonprogramming), dealt with the families, organized the operational part of the whole experimental program.

Program organization

Rob’Autism is organized in 20 sessions of 1 hour each, once in a week, and is concluded by a public show where external people are invited. The 20 sessions alternate 10 non-robot programming and 10 robot programming work sessions 10. Here, a framework was defined and strictly respected during the complete program: A music was played before a session starts (always the same), another at the end of the session, the positions of the tables, chairs and robots were always the same when the teenagers arrived, the working staff was unchanged for the whole program.

Robot Mediation Used to Improve Reliance

Communicating is a bilateral notion which supposes to give something (object, feeling, etc.) to someone and accept something from someone. It is the first sign of recognition as an individual by the outside world and a classical way of exchanging with others, according to one’s definition of others. It cannot be performed when the function allowing to identify the self and the other is damaged, such as in ASD. As a consequence, the capability of communicating can be a way to evaluate the ASD subjects’ evolution. During a program, communicating with others is dealt with on three different levels of liance: individual, group and social links will be worked out. We will describe in what follows each level of liance and their respective observed impact on the subjects’ behaviors [6].

Individual liance

Individual liance takes place with a person, an object (animated, such as a com- panion robot) or an animal. It consists of both giving to and receiving from. As mentioned previously, the participants work by binomials, so each can program the robot during a given time (not fixed, according to the needed time to complete the exercise) and have to let their binomial also program the robot until they have finished their exercise. The subjects here never exchange anything with the robot itself, as it is considered as a machine and not as an intentional character. During the Rob’Autism program, the interaction with the world focuses on people and not on machines, the robot is a tool allowing a protected way to tell and do things to the others [7-9]. Alternate programming of the robot is at first a difficult exercise. The reactions, not to share, are such as holding firmly the keyboard or the mouse, while screaming or protesting loudly, then become sending black and disagreeing looks. A first behavior is not to give any interest on what the binomial is programming, trying to get the attention by doing something else that needs attention from the nurse, holding the nurse from helping the binomial to complete his/her programming. Programming the robot is closely linked to catching another’s attention and focus. Letting the binomial program generates a stress that cannot be controlled by the teenager alone.
This situation can be controlled if the exercises are short enough and can be performed several times in a session: when the teenager is allowed to go back to programming quickly enough, the acceptance of individual exchange starts while the stress lowers. The experimentation shows that 10 minutes exercises (each teenager goes back three times to programming the robot during one session) is a good com- promise between long enough exercise for elaboration, focus time of the programmer and attention delay of the waiting binomial. An evolution of the behaviors between the binomials is observed.

In the Case of Three Programming Exercises Each Per Session:

a) At first, the other binomial is rejected and attention of the nurse is at- tempted to be monopolized. Observed stress reduction can be seen within the very first robotic session, when the participant is allowed to program again once the binomial has finished.
b) Curiosity to what the binomial is programming appears between the second and the fourth robotic sessions, depending on the participant. A progression was observed (same for all the participants) in showing curiosity: first is an attention to what the other has programmed by watching the robot perform only at the end of an exercise; second is to look directly what the other is programming on the computer, during the exercise; third is to interact with Insert Paper’s Title here [7] the binomial and help realizing the performance: give opinion, give advice, explain programming, help with insuring robot security, help with holding the robot in a configuration while the binomial is registering it.
c) Complete acceptance of relying on the other is observed from the third robotic session for all the subjects (no stress observed with the guaranty to program again in a short time).
d) The progression in individual liance continues over the 20 workshop of the pro- gram, we mentioned the main observation points of the changes. The binomials positioning for individual liance is side by side, the center of attention of the liance is the computer and the robot. Individual liance impacts two factors for a partic- ipant: from pleasuring the self to accepting delayed pleasure of the self; and from monopolizing one’s attention despite one other to generating an interaction with one other. In the experiment, the framework is insured: when the participants enter the room, they find the three work stations in the same configuration, the present people are always the same, the manner the exercises are explain do not change. The only changes are the content of the exercises and the working pairs (different binomial at each session). This helps constituting the group as a sort of mini society. Individual liance is the most difficult one, as it needs constant contact and attention in time, which is a demanding exercise for the ASD participants.

Group Liance

At the end of each exercise, the three binomials are asked to stop all activities to show and watch what was performed. Each demonstration is concluded by group applause. Two types of communication exercises are realized with the robot: tell / show the others about something personal, including programming abilities, or address someone in particular with a personal message. The robot is allowed any kind of message, including the use of “forbidden words” (slang and insults) 10. Here, the timing is important: whatever time is needed to conclude the exercise, the groups have to wait until the three are ready to perform all the demonstrations. Group liance takes place faster than individual liance, as the attention is more localized in time. The teenager can work his/her program in a hidden way from the others, and shows his/her demonstration during a short time to the others, still hidden as the group is watching the robot (not the programmer). This focus in time makes it easier to handle than individual liance, which takes place in a constant manner during the complete session.

The Effects of Group Liance can be Observed from the very first Robotics Session:

First exercise the participant programs the robot because and how it is asked. (then shows to others, then applause from others, then watch others’ programming, then waiting time for the binomial to do the exercise) Second exercise the participant programs the robot because he/she has curiosity on how the robot behaves [8- 11]. (then shows to others, then applause from others, then watch others’ programming, then waiting time for the binomial to do the exercise) Third exercise the participant programs the robot for the others and expects the positive reaction from the others [11- 13]. The positive reactions appear through the applause, which validates the creation of the participant. This process transforms the participant through the robot mediator from a person to an individual. He/She is identified by the group as a contributor of the mini-society built in the framework of this project. The participant discovers the pride of showing to others and to exist as a social being. During the first robotics session, the candidates try the robot. they are still shy, they do not know what will please the others. They can only get their attention using the robotic mediation. From the third robotics session, they know each other much better and start communicating with the others through the robot actions. Its sayings will mention points that interest another participant (who is in another binomial), or their robot will answer another robot which had previously talked. The positioning for group liance is each binomials facing the two other binomials, the center of attention are the respective robots. The effects of group liance address the notion of intentionality: the participants learn how to project themselves in time to please other beings in order to be pleased by their reactions. The group is never changed during the complete program. Its constitution is fixed, part of the frame. Some people may be accidentally missing, participant or accompanying person, but no new face is included in the program until the social liance.

Social Liance

Social liance is performed only once, at the end of the program: the artistic creation realized during the sessions is publicly shown. The public consists in around thirty persons: they may be known or unknown by the participants, but they all represent “the outside world”. A scene is set up for the show, on which the programmed robot will make its performance. The six participants sit directly in front of the scene, facing it. The rest of the public sits in the back of the six participants. It is the first time that the participants discover their complete show: before that event, they had seen only small parts of it while they were creating it. As a consequence, they discover their creation at the same time as the public, which generates a situation of shared experience in this new assembly. Discovering the global result of putting together known small actions leads to living an experience that is at the same time known and unknown. The separate small actions are recognized, but take a complete new appearance when all put together, the story shows a new meaning. More than the meaning, the participants [9] realize at this moment the extent of their small contributions. The ascertainment of the greatness of their creation is directly linked to the pride of what they have made. The link leads to realizing how to generate something great step by step, which is equivalent to making a projection in time. Let us recall here that time projection is particularly lacking with the participants, at the beginning of Rob’Autism pro- gram. They hardly can focus more than five minutes in the first robot programming session, whereas they can focus the complete hour after four robotics session 10. As a consequence of the small concentration time they have, time projection cannot happen. At the end of the program, their mind is equipped with longer concentration times. They can start making time projections. This is done at the show restitution.

Acknowledgment from the public follows the show, with applause. This acknowledgment validates their being part of the society: they are officially recognized as contributors by unknown people who represent “the outside world”; it is equivalent to an initiatory introduction in the society (recognition by the pairs). From this moment on, they can exist as social individuals, and not only as unidentified persons.
Following the show, questions from the public are answered by the accompanying people of the project. During the questions, the participants stay sit and listen. They do not interfere even though there is no doubt they are present in the scene. Social liance is concluded by a drink and cakes served at a table and shared by all: public, participants and accompanying people. The participants stay voluntarily in the room and depending on the group mix with the society or stick together, reforming the group for talking and laughing.

Analysis and Complementary Results

The results observed in the Rob’Autism program are qualitative. Up to now, they are based on observation (the working sessions are monitored) and the program is adapted to the tracked group of teenagers. Four groups of six teenagers with ASD were studied, all of them reacted the same way as described in the previous section. We give precisions here to points of the program that are important for obtaining the results: Before a working session starts: The beginning of the working session are organized in a way so that all the participants enter the room together. This means a meeting space is organized outside the room, where the participants arrive one by one until the group is complete. During this meeting, where the parents do not attend (they drop their child and go), the group can be formed again, and they start communicating together. Once they have all arrived, they knock at the working session door, and the animator opens the door. The participants enter one by one in the room, greeting the animator [10] (always the same) by saying hello and shaking hands. The group meeting outside the room allows the participants to enter the room already in the group mode, so they can focus immediately on the mini society. Vocal synthesis: there is a strong temptation to play with the vocal synthesis, not in the objective to communicate with the world. Putting letters in random order and testing the sounds coming out of the robot, improving the shape of the sound until the operator is satisfied with it and ready to make it listen to the others. Here, playing with the vocal synthesis or using it to communicate (with organized sentences carrying a voluntary message) has very different effect on the participant. No progress is made with the playing. They enjoy it, it makes them work, they look for the group sharing moment, but they do not improve voluntary communication. Training of the accompanying staff: During the robotics working session, three nurses are dedicated to helping the participants to focus on the exercises, one per binomial. So three people work in each binomial. The accompanying staff was slightly trained during 12 hours before the program starts, on how to program the robot. The accompanying staff is not competent in robotics (medical staff mainly), and the training allows two situations: a) autonomy for the simplest requests: where to find the “say” box for example, how to connect the boxes, how to send the signal to the robot, and so on. In this situation the link between participants and accompanying per- son is based on pedagogy (individual sharing). b) need for help for the more advanced functions, in that case participants and accompanying person are in the same situation and have to deal with the unknown situation (group sharing). Applause the applause stabilizes the working progress of the liance, it generates satisfaction and pleasure that become an objective for the participant. Applause represents the acknowledgment of the individual’s contribution to the group / society. The contribution is Rob’Autism context is never discussed nor oriented. As an example, one of the candidate only wanted to make the robot count endless, taping 1 and making the robot say it, then taping 2 and let the robot say it, then taping 3 and let the robot say it, and so on. The participant’s contribution in this case is to make the robot count up to ten, then generate the applause. After some time, the participant accepted to use the vocal synthesis for other formulations with sentences, and communicate with different intentions than just making the robot count. The effects of Rob ’Autism program on the participant concern an appeasement of the participants’ anguish. The common observation concerned the concentration times and voluntary communication. The concentration time evolved from 5 consecutive minutes in the first robotic workshop to the complete hour after four robotic workshop. Nevertheless, the working time during one robotic workshop was limited to 45 minutes, leaving 15 minutes for playing: making the robot talk and communicating with the others through the robot. The voluntary communication starts taking place when the group is formed and identified by the participants, when they have been able to observe the others, see their contribution and showed their own contribution to them. Not using the robot to give personal impression starts in average at the seventh robotic session. Concerning the anguish, each participant has different symptoms: crises, mutilation, mutism and other isolation behaviors, screaming, absence, mirror behaviors, search for containment, and so on. Evaluation was performed with the participation of the parents of the participants, who observed a reduction in the manifestations of anguish at home or outside the home 10 [14].

Conclusion

The robot extension paradigm was addressed in this paper, in a complex experiment with ASD teenagers. The participants programmed the robot, using it as a prosthesis in communication, and were accompanied to improve stabilizing liance with the world. To do so, they first identified their self and differentiated it from the world (the not self), rebuilding their frontier to the world and redefining their identity. Three levels of liance were worked out: individual liance, dealing directly with another participant over time; group liance that allowed identifying one’s contribution to the group and acknowledging recognition of the contribution from the group; and social liance with the restitution of the robot play that was realized during the 20 workshops of the program in front of an external public, and concluded with applause. The results observed on the participants show longer concentration times, better voluntary communication and appeasement of the anguish (decreased symptoms, such as self-mutilation, mutism, screaming, absence, mirror behaviors, and so on). The next step of this study is to perform a regular evaluation of the participants to quantify their evolution.
Rob’Autism project results from a collaboration between Centrale Nantes and the non-profit organization Robots! Human, material and financial supports to this project were also provided by the following institutions and companies: RFI OIC, Faculty of speech specialists (Hospital of Nantes), Softbank Robotics, Soprabanking, Foundation Terre Plurielle, ADN’Ouest, Lion’s Club, AG2R La Mondiale, Caisse d’Epargne and EPSI.

References

1. ET Bekele, U Lahiri, AR Swanson, JA Crittendon, ZEWarren et al. (2013)A step towards developing adaptive robot-mediated intervention architecture (ARIA) for children with autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering21(2): 289-299.
2. Aude Billard, Ben Robins, Jacqueline Nadel,Kerstin Dautenhahn (2007) Building robota, a mini-humanoid robot for the rehabilitation of children with autism. RESNA Assistive Technology Journal 19(1):37-49.
3. JohnJohnCabibihan, HifzaJaved, Marcelo Ang Jr,Sharifah Mariam Aljunied (2013) Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. International Journal of Social Robotics 5(4): 593-618.
4. TChaltiel, RGaboriau, SSakka, LSarfaty, ABarreau, et al. (2017)L’enfant, les robots et les crans, chapter Chapter 7 - Un robot en institution pour adolescents autistes:Une aventure collective pp. 167-197. Dunod.
5. Kerstin Dautenhahn (1999) Robots as social actors: Aurora and the case of autism. In Third International Cognitive Technology Conference.
6. Paul Dickerson, Ben Robins,Kerstin Dautenhahn (2013) Where the action is: A conversation analytic perspective on interaction between a humanoid robot, a co-present adult and a child with an ASD. Interaction Studies14(2).
7. RGaboriau, S Sakka (2017) Le robot commemediateurtherapeutique:Une experience aupres de jeunesautistes. Ttralogiques22:249-262.
8. Ben Robins, Paul Dickerson, Penny Stribling,Kerstin Dautenhahn (2004) Robot-mediated joint attention in children with autism: A case study in robot-human interaction. Interaction studies 5(2): 161-198.
9. SSakka, RGaboriau (2017) A robotic puppet master application to ASD therapeutic support. International Journal of Mechanical, Aerospace, Industrial, Mechatronic and Manufacturing Engineering 11(8):1483-1491.
10. Sophie Sakka, Renald Gaboriau, Jean Picard, Edwina Redois, GwenaelleParchantour et al. (2016)Robautism: How to change autistic social skills in 20 weeks. In 5^thInternational Workshop on Medical and Service Robots.
11. Shamsuddin Syamimi, YussofHanafiah, Ismail Luthffi, HanapiahFazah Akhtar, Mohamed Salina et al. (2012) Initial response of autistic children in human-robot interaction therapy with humanoid robot NAO. In International Colloquium on Signal Processing and its Applications.
12. Adriana Tapus, AndreeaPeca, Amir Aly, Cristina Pop, Lavinia Jisa, et al. (2012) Children with autism social engagement in in- teraction with nao, an imitative robot: A series of single case experiments. Interaction Studies 13(3).
13. Iain Werry, Kerstin Dautenhahn (2007) Modeling Biology: Structures, Behaviors, EvolutionVienna Series in Theoretical Biology, chapter Human-Robot Interaction as a Model for Autism Therapy: An Experimental Study with ChildrenwithAutism. pp. 283-299.