Design a Cloud-enabled Humanoid Robot

Application System to Assess the ABA Learning for

Autistic Children
2022 International Conference on Intelligent Education and Intelligent Research (IEIR) | 979-8-3503-4562-9/22/$31.00 ©2022 IEEE | DOI: 10.1109/IEIR56323.2022.10050050

1st Ziyuan Wang 2nd Yiwei Chen

Hubei Key Laboratory of Smart Internet Technology Hubei Key Laboratory of Smart Internet Technology
School of Electronic Information and Communications School of Electronic Information and Communications
Huazhong University of Science and Technology Huazhong University of Science and Technology
wangzyprince@hust.edu.cn yiweichen@hust.edu.cn

3rd Xiaojun HeiB

Hubei Key Laboratory of Smart Internet Technology
School of Electronic Information and Communications
Huazhong University of Science and Technology
heixj@hust.edu.cn

Abstract—In recent years, the autistic community has grad- of pervasive developmental disorders. Autism is often
ually attracted widespread social attention. There are currently thought of as a physical disorder caused by genetic or
many difficulties with autism therapies in China, such as high other factors that manifest themselves early in life, and
market demand, high prices and inadequate autism teachers.
With the continuous development of robotics and digital many people have contributed their whole lives to cure
therapeutics in recent years, there are increasingly potentials this disorder.
of applying robotics to the autism companionship therapy. There are still many difficulties facing the treatment of
To demonstrate this possibility, in this paper we apply the people with autism worldwide, mainly including:
cloud-based robotics and the practical needs of the Applied- a) High market demand: According to statistics,
Behaviour-Analysis (ABA) learning for autistic children, and
design an a cloud-enabled humanoid robot application system, more than 1 in 59 children in the U.S. have autism
in order to reduce the teacher’s workload. We look into a typical spectrum disorder, and more than 10 million people
ABA therapy process for autistic children to recognize fruits. in China are waiting for treatment. The current global
Following the Childhood Autism Rating Scale and the Autism interventions, which are still primarily educational and
Behavior Checklist, we design an autistic child digital model physical, require a great deal of human and material
to interact with this humanoid robot system in a Robotic
Development Kit (RDK) virtual environment. This digital resources, and these intervention treatments are often
autistic children model mimes autistic children behaviors. Our heavy and repetitive in terms of simple movements and
humanoid robot is en-powered with cloud-enabled intelligence language.
to recognize the emotions and actions of this digital autistic b) Inadequate teacher: Autism treatment requires
child in order to assess the cognitive progress in his ABA professional autism teachers. Currently, only about
learning. This robotic system also implements various functions
to enable interaction with this digital autistic child, such 100,000 people are engaged in the education of children
as automatic navigation, intelligent voice interaction, visual with autism in China, and less than 1,000 of them are able
recognition and robotic dance. Our system prototype demon- to provide high-quality services, i.e., hold internationally
strates the feasibility of designing a cloud-enabled humanoid certified teacher qualifications for children with autism.
robot to accompany autistic children. On average, a teacher of children with autism has to deal
Index Terms—Autism Spectrum, Enabled Humanoid
Robots, Neural Network, Object Detection with more than 1,000 students, and there is a huge talent
gap for teachers of children with autism.
I. Introduction c) High medical cost: Autism requires treatment
from the start of the disease and for the rest of their
The autism spectrum disorder, also known as autism or
lives, which carries a significant medical cost. A child
autistic disorder, is one of the representative disorders
with autism needs more than ten hours of professional
This work has been in part supported by the NSFC under rehabilitation sessions per week, and each session costs
Grant No. 61972172, the Industry-University Cooperation Program 300-600 RMB; if a child with autism needs to go to school
of Ministry of Education under Grant No. 202102151011 and the
College Student Innovation Project under Grant No. S202210487083.
normally like a normal child, a ”shadow teacher” is needed
to guide and accompany him/her at all times. The cost of
a ”shadow teacher” is about 10-20,000 per month, which
979-8-3503-4562-9/22/$31.00 ©2022 IEEE is a big financial burden for an average family, and even

December 18th to 20th, 2022

Authorized licensed use limited to: UNIVERSIDADE FEDERAL DE SAO CARLOS. Downloaded on March 01,2024 at 13:33:33 UTC from IEEE Xplore. Restrictions apply.
IEEE IEIR2022-International Conference on Intelligent Education and Intelligent Research
Page 114
one parent needs to quit their job and act as a ”shadow (ATEC, Autism Treatment Evaluation Checklist [6], Psy-
teacher” themselves. choeducational Profile for Autistic Children (PEP) [7] et
In recent years, robotics based on artificial intelligence al. In 1990, the Chinese version of PEP-R was revised
has been applied to the early intervention of children with and released according to the Psychological Education
autism, and has achieved certain results in the early diag- Assessment Scale for Autistic Children (PEP), and the
nosis, monitoring and recording of children with autism, third version of R-PEP-3 was released in 2005. At the same
emotional recognition, social skills, behavior modification time, the main treatment method for autism at home and
and game interaction. Growing digital twin technology is abroad is the application of behavioral analysis therapy
expected to accelerate the robot application development for the autistic children [8]. Oivar Lovaas et al. from the
to provide efficient technical path, is expected to use University of California, Los Angeles applied ABA to the
the robot technology more accurate early diagnosis of treatment of autistic children as early as the 19th century
autism, and various sensors available real-time perception [9], and showed through research in 1987 that, 47% of
of children’s learning process, and carry out training and children made great progress after undergoing a lot of
evolution of evolution is based on the number of twin ABA training [10].
simulation platform, analysis of children learning status With the development of artificial intelligence and
and evaluation of learning effect, Dynamic programming robotics, it has been impacting autism intervention, kinds
adjusts the control of the companion robot and rec- of autism treatment methods and robotic applications
ommends learning methods and resources based on the have emerged. For example, in the early 1976, some
artificial intelligence algorithm, effectively demonstrating scholars have tried to apply the robotic technology to
the application prospect of intelligent learning companion intervene therapy for children with autism [11]. In other
robot for autistic children. scholars’ studies, in 1992, Adarraga et al. proposed to
In addition, because of the personal privacy issues, we apply artificial intelligence technology to autism diagno-
encountered many difficulties in collecting real behav- sis and developed a knowledge-based autism diagnosis
ioral data of autistic children. There are few public or system. By 2012, a software tool for Autism Diagnostic
semi-public databases about autistic children’s teaching Interview-Revised (ADI-R) has been widely used around
worldwide. If one wants to build a database of autistic the world. In 2015, due to the rapid development of
children, he/she has to contribute significant manpower robotics, a number of scholars have been interested in
and resources. In order to improve the data reuse rate of the application of robotics in autism intervention. Studies
autistic children and reduce the experimental cost of data have shown that autistic children are more interested
acquisition in future experiments, we used the Childhood in robots with human-like appearance than the nor-
Autism Rating Scale (CARS) [1] and the Autism Behavior mal person. In 2008, Duqueete et al. concluded in the
Checklist (ABC) [2] as the main reference basis and experiment that autistic children are more inclined to
designed an Autistic Child Digital Model. imitate the behaviors and actions of robots than normal
Based on Cloud Ginger XR-1 robot of Cloudminds and children, which provides enlightenment and possibility for
Cloud Robot Development Kit (RDK), combined with the the application of robot technology in the intervention
Cloud robot technology, in this paper we design a cloud- treatment of autistic children. Warren et al. [11] explored
enabled robot system for individual training classes for in details the possibility and effect of robot interaction
autistic children. By robot with autistic children know with autistic children in 2015. In 2017, a team [12]
fruit game as an example, the robot through the process from Massachusetts Institute of Technology (MIT) used
of autistic children, a professor at the fruit to feedback the NAO robot to analyze the emotions and actions of
action produced by autistic children, resulting in autistic autistic children, who accomplished various tasks with
children’s interest, and successfully teach children with different difficulties, and proposed a method to quantify
autism accurate understanding of fruit, improve the ability the external manifestations of autistic children. In 2018,
of autistic children perceive the external environment and Rudovic et al. [13], collected multi-modal information such
things. as facial expression, body posture, voice and heartbeat of
children with autism in a multi-modal way, and proposed
II. Related Work the Personalized Perception of Affect Network (PPA-
Studies on autism have been carried out since very early NET) for the diagnosis and intervention of each autism,
days. In 1943, Leo Kanner [3], an American psychologist, which provide individually customized solutions for each
first proposed the concept of autism and described the child. In 2019, Shi et al combined machine learning and
clinical symptoms of autism in detail. Since then, scientists self-supervised learning methods to design a dialog-based
have been working to quantify the scale of autism so personalized machine learning model [14] for each student
that it can be effectively treated and evaluated. Among by using a small amount of single individual’s data and a
them, the most representative ones are the ABC (Autism large amount of other individuals’ data. Compared with
Behavior Checklist) [4], the CARS (Child Autism Rating the non-personalised machine learning network, the effect
Scale) [5], and the Autism Treatment Assessment Scale of the network is significantly improved.

December 18th to 20th, 2022

Authorized licensed use limited to: UNIVERSIDADE FEDERAL DE SAO CARLOS. Downloaded on March 01,2024 at 13:33:33 UTC from IEEE Xplore. Restrictions apply.
IEEE IEIR2022-International Conference on Intelligent Education and Intelligent Research
Page 115
All the above studies have shown that the combination
of artificial intelligence, robotics and other technologies
is expected to achieve new breakthroughs in the research
and treatment of autism in the near future.
III. System Design
A. Applied-Behaviour-Analysis
The Applied Behavior Analysis [2] (ABA) is an Applied
form of Behavior Analysis. Behavioral analysis is a science
that studies behavior, behavioral changes and influencing
factors. Applied behavioral analysis is a science that
applies the results obtained from behavioral analysis to
understand the functional relationship between behavior
and environment. Two other forms of behavioral analysis
are radical behaviorism and experimental behavior anal-
ysis.
ABA decomposes the target tasks (i.e. teaching knowl-
edge, skills, behaviors, habits, etc.) into a series of
smaller or relatively independent steps in a certain way
and sequence, and then adopt appropriate reinforcement
methods. Gradually train each small step in the order
determined by the task breakdown until the child has Fig. 1: Teaching Preparing Stage
mastered all the steps and is finally able to complete the
task independently and apply the knowledge and skills he
has learned in other situations. C. Robot Teaching Stage
ABA has been widely accepted among healthcare pro-
fessionals and used in many schools and treatment clin- After the child appearing, the robot formally enters
ics. ABA encourages positive behaviours and discourages the Robot Teaching State. First, the robot will determine
negative behaviours to improve a variety of skills. The whether this teaching is the first time. If it is the first
child‘s progress is tracked and measured. What‘s more, it time, it will broadcast the corresponding language signal
is often used to treat children with autism, ABA therapy through the robot’s text-to-speech (TTS) capability node,
helps to improve social behavior at home, school and using the speech-to-text technology, indicating the start
in the community. It follows the cycle below: 1)nega- of teaching. If not, it go directly to the next step. Then
tive behavior; 2)analysis and intervention; 3)monitoring; the robot raises its hand to show the autistic child all
4)continued intervention therapy. kinds of fruits and teaches the autistic child to recognize
In the following, we will divide the ABA robot accompa- all kinds of fruits through speech. After the teaching is
nying autistic children teaching process into three stages, completed, the robot prompting autistic child to go to
including the robot teaching preparation stage, the robot a specific place in the room to find the fruit. If, after a
teaching stage, and the recognition stage. period of time, the autistic child does not respond or does
not start looking for fruit, the robot plays random actions
B. Robot Teaching Preparation Stage or music to attract the autistic child’s attention and re-
As shown in Fig. 1, when the parent or someone else enters teaching mode to teach the autistic child about the
sends a teaching start command to the robot, the robot types of fruit and to prompt the autistic child to look for
enters the teaching ready state. The start command is fruit again. If the autistic child is detected to start looking
achieved by the parent or other person sending a voice for fruits, the robot automatically enters the recognition
signal to the robot. The robot navigates to the designated state and waits for the autistic child to look for fruits.
teaching location and waits for the child with autism
to appear. The robot invokes face recognition ability to D. Recognition Stage
identify whether the child is approaching or not, mainly After the robot entering the recognition state, it will
by capturing the features of the approaching face. Then wait for the autistic child to look for the fruit. When
it will be uploaded in the cloud and will match the face of the waiting time exceeds 1 minute, the robot remind the
the autistic child recorded in the database. If no autistic autistic child to look for the fruit again; if the waiting time
child’s face is recognized or the face is not an autistic child, exceeds 2 minutes, the robot will perform actions to re-
the recognition fails. When the recognition is successful, capture the autistic child’s attention and then will go back
the robot enters the Robot Teaching State and teaching to the robot teaching state. If the waiting time does not
begins. exceed the time limit and the child take back the fruit, the

December 18th to 20th, 2022

Authorized licensed use limited to: UNIVERSIDADE FEDERAL DE SAO CARLOS. Downloaded on March 01,2024 at 13:33:33 UTC from IEEE Xplore. Restrictions apply.
IEEE IEIR2022-International Conference on Intelligent Education and Intelligent Research
Page 116
IV. System Implementation
Compared with some existing robots (Nao, Keepon,
Rbota), HARIX Cloud Ginger XR-1 has some superior
advantages. For example, HARIX Cloud Ginger XR-1 is
much easier to program and design its motions with a
graphic programming method while Nao [15] ought to
be programmed using C, C++, Python which could be
difficult for those researchers who do not program very
much. What‘s more, Ginger‘s moving ability makes it
possible to have a more flexible teaching environment
for autistic child, which Keepon [16] couldn‘t achieve
although it could also analyse child‘s mental situation
through cameras. And compared with a doll-shaped mini-
humanoid robot, which is caleed Robota [17], Ginger
has a more human-shaped appearance and more flexible
behavior which can create a better transition between
therapy and autistic children‘s real lives. In addition,
Cloudminds provides a more complete cloud-enabled robot
operating system and simulation interface, which provides
favorable conditions for robot application development.
Above all,we present the specific implementation of this
system based on a development environment of cloud-
enabled robot in this section.
A. Development Environment of Cloud-enabled Robot
Fig. 2: Robot Teaching Stage 1) Ginger XR-1: Cloudminds Ginger XR-1 integrates
intelligent cloud-based HARIX, including machine vision,
natural language interaction, intelligent grasping, cloud-
robot will call the cloud algorithm to determine whether
based 3D maps and other capabilities. The whole body is
the fruit that the child gets back is correct. If the fruit is
equipped with 34 smart flexible joints (SCA) for dancing,
incorrect, it will randomly play actions or perform verbal
grasping, etc. Ginger is equipped with a three-level control
encouragement, and then re-request the autistic child to
platform of robot control unit, central control unit, and
find the fruit and wait for the autistic child to find the
electronic control unit on the humanoid robot body,
fruit again. If the fruit is correct, the child is rewarded for
integrated with 2D camera, 3D camera, ultrasonic radar,
the action and praised for the autistic child. Up to this
inertial navigation, LIDAR, stereo speakers, microphone
point, the entire teaching process is over.
array, gas, particle cliff detection and other rich sensors.

Fig. 4: Cloud Ginger XR-1

Fig. 3: Recognition Stage The software systems consist of three parts: HARIX in
the cloud, APP in the robot RCU and development client.

December 18th to 20th, 2022

Authorized licensed use limited to: UNIVERSIDADE FEDERAL DE SAO CARLOS. Downloaded on March 01,2024 at 13:33:33 UTC from IEEE Xplore. Restrictions apply.
IEEE IEIR2022-International Conference on Intelligent Education and Intelligent Research
Page 117
HARIX cloud part provides AI service engine, vision game engine. In this way, developers do not need a
processing, natural language processing recognition, robot background in robotics or embedded systems, but can
skill management, map management and other services focus on application development, adding new skills to
connected to Ginger robot RCU through VBN, providing the robot and performing the intended tasks.
cloud service functions for the robot, and logging into the
cloud through the website to complete the corresponding
The website can be used to login to the cloud to com-
plete the corresponding settings and functions. The local
development client is the RDK development environment,
and the corresponding Ginger robot program is written
in RDK to realize the function control of the robot. the
(a) Map Editor (b) Blueprint Editor
APP in RCU is mainly the UE4 client, connected to the
cloud VBN tool, connected to some service APP of CCU,
etc.
2) Robot Develop Platform: Robot Develop Platform
(RDP) is a coordinated processing system deployed in
the intelligent brain cloud, including skills, roles, map
management and visual recognition model training. Each (c) Dance Movement Editor (d) Simulator
RDP system is matched with the simulation system and
physical robots through corresponding accounts. Programs Fig. 7: Robotics Development Kit
written on the client RDK need to be set up as a new skill
through the RDP and associated with the physical robot.
B. System Implementation Summary
This system focuses on interactive teaching of autis-
tic children through robot teaching autistic children to
recognize fruits in order to help autistic children better
perceive and understand the outside world. The expected
Fig. 5: Robot Develop Platform(RDP) implementation scenario is a one-to-one robot to autistic
children teaching session in a home. The robot is mainly
3) Smart Voice: Smart Voice intelligent voice platform used to teach the autistic child through movement as
is used to generalize and set up the corpus of voice well as verbal communication. The robot uses games with
dialogues. By analyzing text and voice commands that the autistic child as a form of teaching. The robot uses
are not exactly the same or similar, the robot recognizes small games to teach the autistic child to recognize four
the real intention of the command and then completes the common fruits in life, including apple, orange, mango and
corresponding operation. watermelon.

Fig. 6: Smart Voice

4) Robotics Development Kit(RDK): RDK is a cloud-

based intelligent robotics innovation open service platform Fig. 8: Mainly Teaching Scene
development kit based on the cloud-based brain oper-
ating system HARIX, including game blueprint editor, As shown in Fig. 8, we created a virtual classroom
scene map editor, motion dance editor, robot operator, environment in RDK suitable for teaching children with
and robot emulator. The RDK platform simplifies the autism at home. In addition, we implement a digital
application development process by encapsulating lower- character standardized model for autistic children based
level AI services and skills (such as robot system cali- on the Childhood Autism Rating Scale (CARS) [1] and
bration, dialogue management, inverse kinematics, and the Autism Behavior Checklist (ABC) [18] as a research
path planning algorithms) with the help of a proven subject for autistic children.

December 18th to 20th, 2022

Authorized licensed use limited to: UNIVERSIDADE FEDERAL DE SAO CARLOS. Downloaded on March 01,2024 at 13:33:33 UTC from IEEE Xplore. Restrictions apply.
IEEE IEIR2022-International Conference on Intelligent Education and Intelligent Research
Page 118
C. Autistic Children Model Implementation
1) Map Implementation: The main map is the home
of an autistic child, and the main teaching scenario is in
the living room and dining room . We created a virtual
classroom environment in RDK for teaching autistic chil-
dren at home. The environment mainly includes living
room, dining room, bathroom, corridor, bedroom and
checkroom. The table in the middle of the dining room has
various kinds of fruits, which are used to teach children
with autism to recognize fruit scenes, and the chairs in
front of the table are set with triggers, so that children Fig. 11: Function Of The Autistic Child Digital Model
with autism can sit down automatically after colliding
with the chairs. If the autistic child’s palm collides with
the fruit on the table, the fruit will be directly attached 3) Function Implementation: As shown in Fig. 11,
to the child’s hand. the design of Autistic child digital model is based on
the descriptions of specific behavioral characteristics of
autistic children in the Childhood Autism Rating Scale
(CARS) [1] and the Autism Behavior Checklist (ABC)
[18], as well as the videos on the Internet about autistic
children. The autistic child model will randomly match to
two walking states during walking, including the normal
(a) dining room (b) living room walking state and the toe walking state, which is a
common sign of having moderately specific body posture
dysfunction in children with autism. In addition, People
with autism always repeat stereotypical behavioral pat-
terns, which include prolonged body swaying, preferring
to rotate themselves for long periods of time, and seeking
(c) hallway (d) cloakroom and storage direct physical stimulation from the outside world.
room
Thus, while the autistic child character is standing still,
Fig. 9: Virtual Map i.e., when the robot is not speaking to him or performing
actions, the autistic child repeats the following three
2) Figure Implementation: As shown in Fig. 10, the actions at random every 60 seconds, including rocking
virtual figure of an autistic child was designed according back and forth, flinging his hands from side to side,
to the image of a real child, simulating a real human and spinning in place and hammering his head. Each
being, with the appearance designed as a cartoon image movement lasted 10 seconds and could not be interrupted
of a child, wearing a red hat, green clothes and blue pants during this period. During this time, the child with autism
as well as brown shoes, with a total of 64 bones designed closes himself or herself off to any outside information.
throughout the body, all of which can be freely rotated. The expression of a person with severe or moderate
autism is often characterized by a dull, expressionless face.
When he or she communicates with people, he or she
often does not make normal eye contact with them. All
of these behaviors are designed in the character model
of autistic children, as shown by the fact that autistic
children always have a constant facial expression, always
have a dull facial expression, always look at one place when
talking to people, and do not make eye contact with the
speaker (robot).
V. System Evaluation
A. Function Evaluation
The main purpose of the system functional testing is
to test whether each function of the system achieves the
expected function and whether the expected results can be
obtained effectively, and whether there are any problems
Fig. 10: Appearance Of The Autistic Child Digital Model associated with the test results. Based on the functionality
of our system, the following two test cases were designed.

December 18th to 20th, 2022

Authorized licensed use limited to: UNIVERSIDADE FEDERAL DE SAO CARLOS. Downloaded on March 01,2024 at 13:33:33 UTC from IEEE Xplore. Restrictions apply.
IEEE IEIR2022-International Conference on Intelligent Education and Intelligent Research
Page 119
TABLE I: Fruit Recognition Model Training Performance Metrics
Training Set Accuracy Validation Set Accuracy Average Recognition Delay Cloud Algorithm Delay
96.1% 93.3% 89ms 280ms

TABLE II: Robot Command Orders Average Delay

Order Name Teaching Preparing Stage Teaching Stage Recognition Stage Dance or Play Music Fruit Recognize
Average Delay(ms) 645 787 678 453 1005

Each test operation and the expected objectives to be the robot’s voice interaction time with the simulated robot
achieved as well as the test results were recorded in through the RDP platform and the average time of voice
detail.The first test case is the Robot Function Test, signal processing by the SmartVoice platform, as well as
mainly for the Cloud Ginger XR-1 robot in this system in the average time delay for the robot to execute each
the simulation side of the function test. In this case, several instruction. The average time for the robot to interact
tests are conducted: fundamental robot function test, like with the simulated robot using smart voice through the
login in robot, robot navigation, robot daily dialogue and RDP platform was 273 ms, and the average time delay
robot interaction function test like robot enter teaching for the SmartVoice processing of the voice signal was
preparing stage and recognition stage, robot attracts the about 144 ms. The time delay for the robot to carry out
attention of autistic children, robot‘s reaction to both instructions in RDK was almost always below 1 s, which
the wrong and right fruit. All of the functions above are was in line with the system design requirements, where
normal in our tests. Besides, when autistic child get the the time delay for carrying out each instruction is shown
wrong fruit back over 3 times, the robot will enter the in the following table.
teaching stage again. The second one is the autistic child
VI. Conclusion
model function test. In this case, autistic child‘s walking
movement, standing still, interruption when doing the In recent years, the expanding group of autistic patients
movement, walking along the track, grabbing, showing and is getting more and more attention from the society.
putting down the fruit are tested. All the results are in Currently, there are many problems in the field of autism
line with the design expectations, performing with typical medical care, such as high market demand, high cost,
autistic feature. and inadequate autism teachers. Combining the current
rapid development of robotics and digital medicine, in
B. Performance Evaluation this paper we design a robotic system based on RDK for
The performance of this system is mainly reflected in autistic children to interact with robots, focusing on a
two aspects, one is the fruit recognition model performance classroom scenario in which robots teach autistic children
testing, and the other is the performance of the connection to recognize fruits. The robot interacts with the autistic
between the cloud and the robot side. child in a gamification approach, designing relevant move-
1) Fruit Recognition Model Performance: The model ments and dances, as well as a dialogue function with
performance of fruit recognition is mainly reflected in two the autistic child. The system takes advantage of the fact
aspects, one is the recognition accuracy of the model, that robots are simple, predictable and non-intimidating
which is the training performance of the model in the compared to humans, and are more likely to attract the
training process, as shown in Table. I, this includes the attention of autistic children. The system uses robots as
following four indicators, model training set accuracy, the main body of companion-assisted autism medical care,
validation set accuracy, average recognition time delay demonstrating the feasibility and vision of using cloud-
and Cloud Algorithm Delay. based robots for autistic children’s paramedical scenarios,
On the other hand, the model algorithm needs to make a and the possibility of eventually implementing autism-
docker image to package and uploaded to the cloud. When assisted companion medical robots. A background scenario
we use the algorithm, we need to send a command to the of an individual training classroom for autistic children
cloud for invoking the cloud algorithm. The system was and a digital standard character model of an autistic child
tested by calling the cloud algorithm locally, and several are also designed to provide a basis for subsequent research
calls were made at different times of the day using different on robots for autistic children.
images. The time meets the requirements. In the future, we plan to extend this system in the
2) Cloud-enabled Robot Performance: The cloud-based following directions:
robot requires a lot of interaction between the robot and • Setting up more application scenarios: This system is
the cloud brain. Therefore, the interaction between the only designed to face the simple application scenario
robot and the cloud brain is an important criterion for of recognizing fruits for autistic children, and more
the performance test of the cloud robot. Due to the application scenarios can be designed for the assisted
limitation of the RDP platform, this system only tests treatment of autistic children in the future.

December 18th to 20th, 2022

Authorized licensed use limited to: UNIVERSIDADE FEDERAL DE SAO CARLOS. Downloaded on March 01,2024 at 13:33:33 UTC from IEEE Xplore. Restrictions apply.
IEEE IEIR2022-International Conference on Intelligent Education and Intelligent Research
Page 120
• Improve the interaction of autistic children: The [8] J. O. Cooper, T. E. Heron, W. L. Heward et al., Applied
robot’s judgment of the autistic child’s expression or behavior analysis. Pearson/Merrill-Prentice Hall Upper Saddle
River, NJ, 2007.
action in the system can be used to achieve real- [9] O. I. Lovaas, J. P. Berberich, B. F. Perloff, and B. Schaeffer,
time feedback on the robot’s emotion to the child. “Acquisition of imitative speech by schizophrenic children,”
Through reinforcement learning algorithm recommen- Science, vol. 151, no. 3711, pp. 705–707, 1966.
[10] O. I. Lovaas, “Behavioral treatment and normal educational
dation learning algorithm, the robot’s interaction and intellectual functioning in young autistic children.” Journal
response to autistic children is continuously strength- of consulting and clinical psychology, vol. 55, no. 1, p. 3, 1987.
ened, and the robot’s action and language morphology [11] S. Weir and R. Emanuel, Using LOGO to catalyse communica-
tion in an autistic child. University of Edinburgh. Department
are dynamically adjusted to achieve a fully intelligent of Artificial Intelligence, 1976.
companion medical robot for autistic children. [12] O. Rudovic, J. Lee, L. Mascarell-Maricic, B. W. Schuller, and
• Combining digital twin Combining the technology of R. W. Picard, “Measuring engagement in robot-assisted autism
therapy: a cross-cultural study,” Frontiers in Robotics and AI,
digital twin to explore a more efficient development vol. 4, p. 36, 2017.
path for companion robots for autistic children and [13] O. Rudovic, J. Lee, M. Dai, B. Schuller, and R. W. Picard,
train more powerful and intelligent companion robots. “Personalized machine learning for robot perception of affect
and engagement in autism therapy,” Science Robotics, vol. 3,
References no. 19, p. eaao6760, 2018.
[14] Z. Shi, T. R. Groechel, S. Jain, K. Chima, O. Rudovic, and
[1] E. Schopler, R. J. Reichler, and B. R. Renner, The childhood M. J. Matarić, “Toward personalized affect-aware socially as-
autism rating scale (CARS). WPS Los Angeles, 2010. sistive robot tutors for long-term interventions with children
[2] J. Cooper, T. Heron, and W. Heward, “Applied behavior with autism,” ACM Transactions on Human-Robot Interaction,
analysis (& pearson new international ed.),” Essex, England: 2021.
Pearson Education Limited, 2014. [15] S. Shamsuddin, L. I. Ismail, H. Yussof, N. I. Zahari, S. Bahari,
[3] L. Kanner et al., “Autistic disturbances of affective contact,” H. Hashim, and A. Jaffar, “Humanoid robot nao: Review of
Nervous child, vol. 2, no. 3, pp. 217–250, 1943. control and motion exploration,” in 2011 IEEE international
[4] F. R. Volkmar, D. V. Cicchetti, E. Dykens, S. S. Sparrow, J. F. conference on Control System, Computing and Engineering.
Leckman, and D. J. Cohen, “An evaluation of the autism behav- IEEE, 2011, pp. 511–516.
ior checklist,” Journal of autism and developmental disorders, [16] K. Hideki and M. P. Michalowski, “Keepon,” International
vol. 18, no. 1, pp. 81–97, 1988. Journal of Social Robotics, vol. 1, no. 1, pp. 3–18, 2009.
[5] E. Schopler, R. J. Reichler, R. F. DeVellis, and K. Daly, [17] A. Billard, B. Robins, J. Nadel, and K. Dautenhahn, “Building
“Toward objective classification of childhood autism: Childhood robota, a mini-humanoid robot for the rehabilitation of children
autism rating scale (cars).” Journal of autism and developmental with autism,” Assistive Technology, vol. 19, no. 1, pp. 37–49,
disorders, 1980. 2007.
[6] B. Rimland and S. M. Edelson, “Autism treatment evaluation [18] F. R. Volkmar, D. V. Cicchetti, E. Dykens, S. S. Sparrow, J. F.
checklist,” Journal of Intellectual Disability Research, 1999. Leckman, and D. J. Cohen, “An evaluation of the autism behav-
[7] E. Schopler, M. Lansing, R. Reichler, and L. Marcus, “Psy- ior checklist,” Journal of autism and developmental disorders,
choeducational profile,” Journal of Autism and Developmental vol. 18, no. 1, pp. 81–97, 1988.
Disorders, 1979.

December 18th to 20th, 2022

Authorized licensed use limited to: UNIVERSIDADE FEDERAL DE SAO CARLOS. Downloaded on March 01,2024 at 13:33:33 UTC from IEEE Xplore. Restrictions apply.
IEEE IEIR2022-International Conference on Intelligent Education and Intelligent Research
Page 121