1. Introduction

Rhythmic activities involve a combination of music, rhythm, and movement, which is intended to relax the mind and body. Today’s teenagers show their creativity through hip-hop, and visually impaired individuals learn physical activities through touch and spoken instructions. For the visually impaired, spoken instructions are often unclear, and physical contact may cause discomfort to the other party. This study aimed to develop an airflow-vibrator-motivated facility and assess exercise behaviors. Although these technologies have gradually matured, there is no appropriate combination of teaching materials to form a reasonable price auxiliary tool. Thus, there is a gap connecting the technology elements and assistive teaching need of motivated facility and assessing exercise behaviors in school. The research work’s contribution extends assistive technology of tactile perception to vibration and airflow, the combination of IoT devices, and the haptic feedback with a better user experience.

The combination of technology and design through tactile perception has been used to reduce the rhythmic learning difficulties of visually impaired children. Vibro-tactile feedback enhancement for orientation and obstacle avoidance can be obtained through the use of discreet actuators and obstacle detector sensors [1]. It provides frequent indications of useful dynamic information, such as level of proximity or distance. People can also perceive rhythm through tactile senses. In [2], the author described rhythm combined with frequency and amplitude to systematically produce 84 distinguishable tactile stimuli icons to help user perceptual tactile rhythm. In [3], the author presented unimodal and cross-modal rhythm perception with auditory, tactile, and visual modalities. Based on these findings, auditory and tactile modalities are suitable for presenting rhythmic information. Many assistive technologies have been successfully applied in education. Games that require physical activity are commonly used in rehabilitation to help restore physical function and balance [4]. Rehabilitation at home can reduce medical expenses and treatment time.

Visually impaired individuals often exhibit limited ability to participate in physical activities. Studies have found that visually impaired individuals take 50% longer to complete tasks with equivalent accuracy compared to sighted participants [5]. In [6], the author explained that movement is often limited by requiring outside guidance, fear of injury, and ridicule of others. However, when visually impaired individuals participated in bowling and tennis video games with vibration and sound prompts, they exhibited improved mood and satisfaction following the activities.

Further study by [7] demonstrate the feasibility of real-time sensory substitution as a cost-effective approach for making gesture-based video games. In [8], the author presented a method of real-time video analysis to detect the presence of a particular visual cue as a cost-effective approach for making gesture-based video games. Users can express meaning in terms of vibration intensity or frequency, and haptics is also used for control, such as feedback when tapping an electronic pet. Feedback can be used as a reminder of movements in different parts, and studies have found that tactile sensation improves user engagement [9]. In [10], the author described the evaluation of human-computer interaction with the decision-making model. The information flow is helpful to monitor the effectiveness of the collective’s activities. In [11], the author presented ubiquitous touch interaction with haptic feedback and show the movable world object supplies an accurate detection through a user study.

Figure 1: Design flow and applications for visually impaired

2. Detail Design

With the shrinking of the original size of the Internet of Things and the popularization of open platforms, researchers try to present the above ideas from the sensors and control components available on the market. This assistive tool involved a software development platform in performing human-machine interaction. Figure 1 indicates the design flow and possible applications for the visually impaired. First, start from the technical side, let those components be combined into the required system, and then look at the field of application and the idea of what kind of teaching materials teachers need. After the experiment in school, we extended it to the visually impaired (on the right) to apply it to different situations in their school life. The development platform and detailed parts in the design also show below.

As seen in Figure 2a, the vent is placed under the table for interacting with the user. The seated height of the participant was adjustable so that their legs remained at the same height as the air outlet so that their body could fully sense the airflow. As seen in Figure 2b, a large air compressor accommodates more compressed air.  Figure 2b, top right shows the electromagnetic valve that controls the air outlet; bottom right shows the outlet of an exhaust pipe, which was an elongated slit that adjusted to the subject’s position

Figure 2: Prototype of airflow assisted design: (a) use state; (b) compressor and spout.

A Phidget PC-USB interface board are utilized to provide an on/off sequence. As displayed in Figure 3, the interface control program has a graphical user interface using Microsoft Visual Studio with the current state of each switch. The button with a dark background indicates that it is currently activated. The text area on the right-hand side lists the sequence of the switching actions. A user can also manually control the switching by directly clicking on the buttons.

Figure 3. Software interface layout design with a set of the airflow outlet and timing control.

The total weighing of the vibrator is below 150 grams through circuit miniaturization to prevent additional loading during bodily movement. The vibrator is located in the skin contact surface with adjustable belts for proper contact. The control command could turn on/off the vibrator at a specific time. Each sensor module uses a 4.2 V lithium battery power. The vibrator is connected to a low-speed I/O pin on the sensor node to extend the module’s usage. The ZigBee interface is IP-Link5501, and the sensor node is IP-Link1223. An NPDU consists of a network header and a network payload. The Network header contains frame control of 2 octets, routing fields of 6 octets, and data payload NSDU (network service data unit) of variable length.

3. Collocation and Research of Auxiliary methods

After observing a class of visually impaired children, the researchers proposed tactile perception to assist in directing physical activity. Engineer submitted proposals for the method of tactile prompting: vibration and airflow through software control. For example, during the design of the airflow, the compressor air was controlled by the researchers. In addition, musical elements ensemble games required the synchronization of music with a pair of wireless vibrators.

Research simulated visual impairment with blindfolds and made detailed adjustments to the study after testing the design setup. Participants were expected to don vibrators on their left and right hands and beat a tambourine with the help of cues from the vibrators (Figure 4).

Figure 4: Wireless vibrators on group’s dual hand and wireless control software

Vibration and airflow cues complement each other: Vibration is more subtle and can prompt specific body movement. In contrast, airflow is more generally perceivable and can direct a wide range of group or individual actions. Experiments show that vibration stimulates activity, while the sound field of an airstream can indicate the direction of the movement. This study attempts to implement the combination of vibration and airflow into teaching: first by using vibration cues to instruct physical movement and airflow cues to guide group activity. Second, apply these findings to educate visually impaired individuals in physical activities.

3.1. Method

Although there are many ways to combine assistive technology and applications, this study selected vibration /airflow solutions for testing. We tested two separate groups:

(1) Children between the ages of 9 and 10 without visual impairments. These children had previously participated in rhythmic activities.

(2) Visually impaired children with general amblyopia or total blindness between the ages of 9 and 12 whose physical ability was close to that of children with normal vision. However, careful observation revealed that these students exhibited limited balance and restricted movement. Their teacher reported hoping to guide them to participate in more activities.

Eight independent wireless vibration modules were set up in a system where a single host controlled the signal. When configured between different people in the same group, they could be guided by the system to work together. The advantage of wireless communication was that subjects using it could move freely.

3.2. Group teaching

During group teaching, the participants were given different individual activities through the prompts of the vibrator after being linked. As a result, the following ideas were put forward to expand applications in teaching.

Activity 1: Ensemble: increase children’s concentration and sensitivity to rhythm through touch. The teacher instructed children to strike a tambourine when they felt a vibration, forming an ensemble rhythm.

Activity 2: AB mode: Increase children’s focus through two touching ways: A for a single short vibration and B for continuous vibration. Participator performs specific actions corresponding to the different vibrations when prompted at irregular intervals (Figure 5). The goal was to see if the child could concentrate and distinguish between different vibration patterns.

Figure 5: Simultaneous A/B vibration test

Activity 3: Consecutive A/B: Promote children’s memory through tactile sensation and movement; promote children’s creativity through music activities. One child was instructed to perform a motion after feeling the vibration signal. The next child was asked to imitate the action of the previous child in response to the vibration signal and then add a second motion to the sequence. A third child was then asked to perform the two last moves and add a third motion when they felt the vibration signal.

The jet sound is received through tactile and auditory cues. Air rhythmic action is the primary stimulus, supplemented by vibrations to prompt a change in the application, such as in the three activities above. When the child’s back was facing the airflow, the study design proposed a scenario of moving a wall to instruct the child to push the invisible wall with his hand so that the child could express creativity during movement. When vibration is combined with airflow, large-scale airflow can guide actions in large areas to different locations, such as forward or backward ambulation, walking around a circle, or pointing in a specific direction.

3.3. Evaluation Criteria and Behavioral Assessment Tools

Researcher used the following pediatric behavioral assessment tools:

(1) Wenlan Adaptation Behavior Scale

The Wenlan Adaptation Behavior Scale (Motor Skills Section), which is used to measure motor skills development in children, was completed at baseline. The test was conducted to track the changes in the children’s physical ability.

(2) Observation and Evaluation Record Form for children’s activities

The observation records of children were also used as a research tool for participatory observation research. Under controlled circumstances, systematic observations of phenomena or individual behaviors were based on established research goals. Observers recorded the performance of the assessed scores in writing. Finally, observer made an objective explanation of the phenomenon or individual behavior. The observation variables are defined as following: week-activity-participant-vibration mode; so that notation (1-2-M-B) means teaching week-1, playing activity-2, with participant-M, using continuous vibration mode-B.

The content of the observation table was divided into structured assessments, supplemented by available text supplements, and recorded by observing the child’s reaction and concentration during movement. The teacher adopted a Likert five-point scoring method for evaluation. A score of 5 indicated frequent behavior observed during the activity. Three observers rated the frequency of each behavior according to the situation described in the items and finally calculated the average result of the observation score. The evaluation items included:

(1)     Movement:

Students respond to vibrator and waves limbs. Uses instrument correctly according to instructions and beats instrument in time to the music. Responds to and moves limbs according to airflow position.

(2)     Attention

Senses change in vibration mode. Uses hearing and physical perception to pay attention and engage to the airflow.

(3)     Emotion

The student expresses positive emotion through spoken words, movement, and expression. Maintains positive feeling throughout the airflow process.

(4)     Creativity

Moves in varied ways in the designated space according to the vibration prompts. Swings body and waves limbs were corresponding to airflow.

4. Results and Discussion

During the first week, observers found that the children were unsure how to react to the vibrators, which caused them to respond incorrectly. In addition, they did not make prominent movements in response to a prompt for free creative actions because of personal shyness. In the second week, the teacher added explanations, introduced movement simulation, and carried out group activities using airflow stimulation. We found that children exhibited significantly different motion feedback and even actively added new motions. The research found that children were integrated into the activity as a whole and actively with new movements in third week. Further details are as follows:

4.1. Week 1 record

Activity 1: Ensemble

1. The children were unfamiliar with the vibrator and could not accurately sense it. Therefore, the performance is not apparent (1-1-M-A) (Figure 6).
2. The children immediately shook their arms when both hands were shaken and showed their arms after a pause (1-1-M-B).
3. After putting on the vibrator, the children kept looking at it. They started by making fists and frowning. (1-1-A-B).
4. The children smiled before the start of the activity. They laughed and jumped on the tambourines. (1-1-E-B)
5. The children beat the tambourines correctly from beginning to end, maintaining a single movement pattern without change. (1-1-E-B)

Figure 6: Student behaviors of three events: (a) ensemble, (b) A/B mode, and (c) A/B two-point.

Activity 2: A/B mode

1. The children could not clearly distinguish between A and B vibration patterns and demonstrated a high rate of action repetition. (1-2-M-A)
2. A small number of children could make the correct movements but could not clearly distinguish patterns when making inappropriate movements (1-2-M-B).
3. When the children wore the vibrator, they frowned and looked at it. They appeared quiet during the activity. (1-2-A-B)
4. Most of the children smiled during the activity, indicating excitement to participate. (1-2-E-B)
5. The child exhibited no change in movement and remained in place. (1-2-C-B)

Activity 3: Consecutive A/B

The instructions were not followed precisely and the children required cueing from the instructor. (1-3-M-A)

Some children responded quickly to the signal, but some needed a reminder from their peers. (1-3-M-B) (Figure 6b)

The children devoted themselves to the activity and paid attention to their vibrator. One child said, “Yeah, I’m here,” immediately after the vibration. (1-3-A-B)

The children exhibited happiness through expression and body language engaged in the activity. (1-3-E-B)

The movements of the children were almost all performed while running (one was jumping). (1-3-C-B)

In response to wearing a wireless vibrator, the children’s overall body language was stiff, and their movement rarely changed. While some exhibited creativity and came up with their ideas, most of them repeated the body movements of others. Some children could focus on the vibrator on their wrists and listen to the instructions of the tester. The children looked nervous but exhibited no negative emotions.

4.2. Week 2 record

Activity 1: Ensemble

Under the instructor’s guidance, the children’s ability to tap to the music improved, but the tapping method did not change. (2-1-C-A)

The children correctly responded to changes in vibration and beat the tambourines at the same time. When only one vibrator signaled them, most of them only moved one hand. (2-1-M-B)

The children were fully involved in the activity and could primarily respond correctly. (2-1-A-B)

The children laughed frequently and waved their arms, exhibiting positive emotions. (2-1-E-B)

The children’s movements were primarily similar, and the rhythm was maintained when the facilitator participated. (2-1-C-B) (Figure 7a)

Activity 2: AB Mode

The children were still unable to create new movements because of nervousness. (2-2-C-A)

The children were able to perform AB mode correctly and respond to different prompts. (2-2-M-B)

The children demonstrated a high willingness to participate. Children were smiling and showing positive emotions during activities. (2-2-E-B)

They maintained the same movement with no other limb movements demonstrated. (2-2-C-B)

Figure 7. Week 2 study: (a) ensemble (b) A/B two-point

Activity 3: Consecutive A/B

The children correctly judged the location of the airflow. (2-3-M-A) (Figure 7b)

The children made significant progress in focus and emotions. (2-3-A, E-A)

There were innovative movements. (2-3-C-A)

Most children could feel the vibration correctly, but one child happily ran around with the airflow; others only had physical movements. (2-3-M-B)

All of the children could focus on activities and follow the action. (2-3-A-B)

The children demonstrated a high willingness to participate and showed positive emotions throughout. (2-3-E-B)

The children exhibited 3 distinct movements; in addition to the original running and jumping, they were also sidestepping. (2-3-C-B)

After the participating children donned wireless vibrators and performed activities with airflow, the children occasionally need to be reminded by the testers. The overall magnitude of limb response became more significant, and the children were willing to cooperate actively to maintain rhythm (Figure 8). After growing familiar with the rhythm, they could focus on vibrational changes and react instantly. When students improved their sense of familiarity, the children had a high willingness to participate, showing positive emotions and smiling.

4.3. Week 3 record

The child feels the way of shaking (3-1-M-A). They demonstrated a high rate of repetitive movements, so they did not perform well in creativity. (3-1-C-A)

Most children could correctly sense the vibration pattern, but a few children exhibited slightly longer reaction times. (3-1-M-B)

The children engaged in the activity. (3-1-A-B)

The children demonstrated a high willingness to participate, showing positive emotions. (3-1-E-B)

Two children performed somersaults, swinging the tambourine from top to bottom. (3-1-C-B)

The children’s energetic performance improved significantly, and they could respond to tactile vibrators, and air prompts. They were also more engaged in activities. During the exercise, they cooperated with a high degree of concentration. The children talked naturally with the tester and performed activities with enthusiasm; at the end, they actively asked when they could participate again (Figure 8). During this period, children engaged in simple competition with their peers, so more innovative actions appeared, such as somersaults and trunk rotations. Overall, the children’s performance in terms of limb movement, concentration, emotions, and creativity improved significantly.

Figure 8: Week 3 study: (a) airflow + vibrator, (b) ensemble and creative move

4.4. Overall Performance

Haptic perception successfully assisted motor skill development in children. In terms of overall performance, children have noticeably changed four goals: movement, attention, emotion, and creativity. The children’s performance may have been limited due to unfamiliarity with the tester in the early stages. In the middle period, movements and expressions changed gradually as the children grew more comfort. The sensation of airflow was highly attractive to the children. When airflow appears in activities, the children seemed excited and did not want to leave so they could continue playing.

Figure 9: Children’s overall performance during three weeks

The quantitative data (using the SPSS) of three class activities are analyzed and discussed below. Figure 9 shows the evaluation results using the Likert five-point scoring method. The children’s overall performance in the later period improved, not only in more physical movements but also in the position of the vibrator and airflow. New actions appeared, such as somersaults. After the activity, the children asked if they could participate again, indicating that the activity was attractive. With appropriate syllabus design, children can demonstrate positive physical and emotional merges.

5. Applications for Visually Impaired Children

Visually impaired children with general amblyopia or total blindness, aged between 9-12 years, possess physical ability close to children with regular sight. Before the activity, however, observation revealed that the visually impaired students exhibited limited balance ability. In addition, they usually exhibit restricted movement. Because of safety concerns, the teacher gave them many constraints; for example, they had to go in pairs one by one when they went outside the classroom. When the teacher asked them to stand on one foot, they appeared unstable and eager to reach for the objects next to them for support. Their movements were less varied, and they presented with stiff posture. The teacher hoped to guide them to engage in more activities using the assistive devices. The wireless module lifted the distance limitation of the vibration connection and enabled prompts in a larger area covered by radio waves signal.

The scenario items included: ensemble, orientation, and passing ball, shoot the ball, and follow the change of airflow. Firstly, a small-scale practice using the previous ensemble and A/B game. The students were given different instruments instructed to tap the instrument through music and vibrator cues. The vibrator cues provided a rhythmic tacit understanding of ensemble participation. Visually impaired students have a good sense of sound and rhythm, students successfully played with musical instruments and A/B game.

Figure 10: Schematic drawing of the large-scale activities: vibrator cues provided walking instructions, students improved pitching in a different location.

Secondly, six pupils with amblyopia in special schools participated in four kinds of large-scale movement play for 60 minutes (Figure 10).

(1) Orientation: Vibrator cues provided directions and walking instructions. Testers used vibrators to indicate right and left to help children reach their destination.

(2) Passing practice: Vibrator cues are used to instruct children to throw a ball back and forth and improve communication efficiency. The vibrator provided the teacher’s instructions to pass and receive the ball.

(3) Shooting: We set up a wireless sounder on the basketball frame. When students receive the ball and want to shoot, they start the sound source with the wireless switch and judge the direction base on the sound from target area. We found that wireless signals give students more confidence in pitching.

(4) Quickly follow the change of airflow: promptly change the air outlet point, the participants rotate the body to monitor the airflow to form a continuous action creatively.

Sometimes students were out of sync during orientation due to the speed and circuit delay. After practicing, the students knew exactly how to pass and asked for the ball from the team classmate (Figure 11). Wireless switches give students more confidence in pitching it. The participants swap the posture to follow the airflow successfully. Through the interview, the vibrator and gas clues enabled more interaction in large-scale exercises. Wireless technology and moving gas help remote instructions to assist users in interacting with other people, guidance from teacher, and the signal in the surrounding environment.

Figure 11. Large scale activities: (a) vibrator cues provided walking instructions in orientation practice (children walk along the path), (b) students pass the ball according to vibration cues, (c) students improved pitching in a different location, and (d) dynamic gas clues for posture following.

6. Conclusion

The aim of the research is developing tactile perception assistive technology to activate the physical activity of children. The design method utilized vibration and airflow actuators and a control system to provide proper feedback and acknowledgment. The study also developed the corresponding instructions and introducing context to promote children’s participation. Three weeks evaluation process conducted through field practice in elementary school with guiding activities. Both explaining hints and free creative activities were encouraged in classroom group cooperation activities. They were also able to recognize different vibration patterns that respond to the corresponding limb movements. The field testing results indicated children’s positive responses and spontaneous creative actions. Their overall performance in exercise, attention, emotion, and creativity improved significantly during the evaluation instructions. The research extends assistive technology of tactile perception to vibration and airflow. The combination of new IoT devices of wireless sensor networks and remote control of haptic feedback are helpful to visually impaired individuals, which support better instruction in the future classroom.

1. G. Ghiani, B. Leporini, F. Paternò, “Vibrotactile feedback to aid blind users of mobile guides,” Journal of Visual Languages & Computing, 20(5), 305-317, 2009, doi: 10.1145/1409240.1409306.
2. D. Ternes, K. E. Maclean, “Designing large sets of haptic icons with rhythm,” in 2008 International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, 199-208, Heidelberg, 2008, doi: 10.1007/978-3-540-69057-3_24.
3. M. Jokiniemi, R. Raisamo,J. Lylykangas, “Crossmodal rhythm perception,” in International Workshop on Haptic and Audio Interaction Design, 111-119, Heidelberg, 2008, doi: 10.1007/978-3-540-87883-4_12.
4. B. Lange, C.Y. Chang, E. Suma, B. Newman, “Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor,” in 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 1831-1834, 2011, doi: 10.1109/IEMBS.2011.6090521.
5. S.A. Douglas, S. Willson, “Haptic comparison of size (relative magnitude) in blind and sighted people,” in 2007 Proceedings of the 9th international ACM SIGACCESS Conference on Computers and Accessibility, 83-90, 2007, doi: 10.1145/1296843.1296859.
6. D. Morelli, E. Folmer, “Real-time sensory substitution to enable players who are blind to play video games using whole body gestures,” Entertainment Computing, 5(1), 83-90, 2014, doi: 10.1016/j.entcom.2013.08.003.
7. D. Morelli, J. Foley, E. Folmer, “VI-Tennis: a vibrotactile/audio exergame for players who are visually impaired,” in 2010 Proceedings of the Fifth International Conference on the Foundations of Digital Games, 147-154, 2010, doi: 10.1145/1822348.1822368.
8. L. Zoccolillo, D. Morelli, M. Iosa, “Video-game based therapy performed by children with cerebral palsy: a cross-over randomized controlled trial and a cross-sectional quantitative measure of physical activity,” Eur J. Phys Rehabil Med, 51(6), 669-76, 2015, doi: 10.14288/hfjc.v10i1.225.
9. J. Yim, N. Graham, “Using games to increase exercise motivation,” in 2007 Proceedings of Conference on Future Play, 166-173, 2007, doi: 10.1145/1328202.1328232.
10. A.H. Hoppe, F. Marek, F.v.d Camp, R. Stiefelhagen, “Extending movable surfaces with touch interaction using the virtualtablet: an extended view,” Adv. Sci. Technol. Eng. Syst. J., 5(2), 328-337, 2020, doi: 10.25046/aj050243.
11. N. Bakanova, A. Bakanov, T. Atanasova, “Modelling human-computer interactions based on cognitive styles within collective decision-making,” Adv. Sci. Technol. Eng. Syst. J., 6(1), 631-635, 2021, doi: 10.25046/aj060169.

(Click Here)