Null Hypothesis: Older individuals who have difficulty with coordination will have the same alternative to touch preference as younger individuals that experience anxiety under stressful conditions. 

Alternate Hypothesis: Individuals over the age of 45 who have difficulty with coordination will prefer the speech recognition mode of interaction when playing Connect 4 and Checkers. Gesture tracking requires an extensive amount of movement to convey a move in the game; hence, speech could be considered an easier alternative for the older generation as it would not require the individuals to have perfect coordination or move extensively. Younger individuals, however, will prefer the gesture tracking alternative, as it can be interpreted as more interactive and comfortable. The younger generation is growing up with these new and advanced technologies, such as Iphones, in which gestures/ hands are used extensitvley. With more use to using gestures, younger people will prefer the method they have more experience with. 

The modern world is growing more digitized.¹ Portable devices, such as computers and smartphones, and the use of high-speed internet are growing in popularity around the world.¹ As the daily use of technology is increasing, it is important to consider the diversity of populations using computer software applications. More individuals are aging, and as individuals age, they grow more susceptible to illnesses and disabilities that affect their abilities.² This could include visual, hearing, movement, and cognitive/ language impairments. With the probability of these impairments increasing as people age, it is more likely that the use of additional assistance will be needed.² 

Assistance can come in multiple forms, including assistive technology such as features in software applications (e.g., zoom in). Specifically, assistive technology is defined as, “any item, piece of equipment, or product system...that is used to increase, maintain, or improve functional capabilities of individuals with disabilities.”³ It is crucial to consider the individuals, from the population of more than 8 billion, that are diagnosed or consider themselves to have health restrictions, disabilities, or disorders that limit their abilities to use and interact with computer software applications.⁴ For instance, in the United States of America, it is recorded that approximately 11 million people have visual impairments, more than 15 million people have some form of hearing impairment, and 1.9 million people have some type of language and learning disability.² Globally, it is estimated that 1.3 billion people in the world have disabilities in 2023, which is roughly 16% of the world’s population.⁵ This statistic represents the multitude of individuals that may have difficulty interacting with computer applications in the future.⁵ It is projected that in the year 2050, 4.9 billion people in the world will have neurological disorders.⁴ These disorders can include dyslexia, autism spectrum disorders (ASD), dyspraxia, and attention deficit hyperactivity disorder (ADHD).⁶ Such disorders could limit mobility and verbal abilities, making it difficult for these individuals to complete tasks on computer applications without accessible features. 

Computer software applications are not necessarily built with the level of accessibility that individuals with neurological disabilities need. Accessibility, in this context, is defined as the, “ability of [a] product and environment to be used by [a variety of] people.”² These issues can be especially difficult for those with motor or visual impairments.⁷ Solutions such as voice commands, movement controls, eye tracking, and sign language can be implemented quickly and affordably within applications to improve the accessibility. For example, individuals suffering from Parkinson's disease, which causes tremors and spasms, could use speech commands to perform tasks that they are not able to complete, like commanding a movement through diction, in video games. Non-speaking autistic individuals who have difficulty with motor skills could also play video games by using sign language or eye and head movement. In these examples, the individual’s mobility range is a factor that plays a role in the accessibility of applications. The range of motion can determine whether one is able to interact with an application to the full extent. It is vital to make computer applications universally accessible. This can be accomplished by understanding the variety of alternatives that can be integrated into computer softwares, such as speech recognition, gesture tracking, and head movement tracking. 

Psychologists established that 80% of communication interaction between individuals involves nonverbal signals, including gesture and head movements, highlighting the prevalence of these forms of communication in human interaction.⁸ Given this understanding, it is logical to integrate various forms of interaction within computer software applications such as tracking gestures, head movements, and even speech. With the tracking of individuals' behaviours, emotions, and movements while they use a computer application, applications can become more interactive and entertaining. For example, in interactive games like a boxing game, the user becomes more engaged, and, most likely, more entertained if the user imitated the fighting in real life.⁸ These games and applications would use gestures and head movement tracking to improve the user interface and interaction. With more computer software applications incorporating a conglomeration of tracking interactions, accessibility would improve, leading to an enhanced user experience. 

User experience is linked to the evolving concept of usability, and human-computer interaction (HCI).⁹ HCI is about understanding users' experience with interactive technologies.⁹ Familiar terms with HCI are user-friendly design and user interface design. From the study of user interaction with the computer software application in this experiment, insights can be gained about how to make applications more user-friendly, entertaining, and accessible. 

Computer applications are still not universally accessible. Although there has been relevant data and research in the field of HCI and the accessibility of computer software applications for those without and with disabilities affecting their motor and verbal skills, there has not been enough action in these fields to have drastically impacted the computer software applications to be fully accessible for these individuals.^9,10 Therefore, in this investigation, several alternatives to touch: speech recognition, eye and head movement tracking, and sign language tracking will be tested and implemented in the games, Connect 4 and Checkers, to evaluate the efficacy of alternative control methods. From the data on user’s preferences during the experiment, alternatives to touch that could be integrated within computer software applications will be identified. These alternatives could have the potential to enhance applications accessibility for those with disabilities and those without. This evaluation aims to provide further understanding on how to enhance user experience, user interface, and accessibility of computer software applications for all. The short term objective of this experiment is to develop a website that incorporates varying complexity of games and control methods. The website will implement methods of control: speech recognition and gesture tracking, for Connect 4 and Checkers games. A future extension would be to evaluate more complex games to ascertain the impact of increased levels of difficulty on preferred human-computer interaction modes. The ultimate objective of this study is to understand, how a person’s age and disabilities impact their preference for alternative options to touch when playing video games? By answering this question, we strive to understand what mode of interaction should be integrated into computer software applications to make it more accessible and improve user interface in the future. 

The manipulated variables are the different interaction methods: speech recognition and gesture tracking, the individual’s age group and disabilities.

The responding variable is the individual's preferred mode of control (speech recongition or gesture tracking). One’s preference could explain under which scenario certain individuals prefer or are restricted to a certain mode of control. 

The controlled variables are the type of game the individual is playing (Connect 4 or Checkers), difficulty and complexity of the game, and the number of times the user plays (a total of twelve times, two times per each game and alternative method). 

The confounding variables in this experiment include individual’s personal preference of Checkers or Connect 4, user’s engagement (i.e., clicking random spots just to complete the game; without a goal of winning), the user’s cognitive abilities and experience playing Connect 4 and Checkers, biological sex, and if the individual enjoys playing video games.

Website Overview:

The website was coded in JavaScript using the react framework, and tailwind CSS was used for the styling on all pages.^11,12 React JavaScript “reacts” to the user interface (UI) and rerenders and updates the website page. Jotai was the library used to track all the information in the website.¹³  The website was built with assistance from ChatGPT and ClaudeAI.^14,15

Website Login:

Two functions were programmed for the welcome/login page; handleDocumentOpen, and handleSubmit. In order to continue to the next page of the website, the user must agree to the terms and read the ethics form, ensuring the consent form was filled. The handleSubmit function verifies that the user fills in all categories (username, email, consent) by using the alert function. After all fields are filled, the user is prompted to the instruction video.

Instruction Video:

The instruction video was built using QuickTime Player (Mac screen recording ), and a text to speech platform.¹⁶ The video page has two buttons available (skip and continue). The user can press either one of these buttons and they will be redirected to the main screen. The line of code used to program this is: onClick={() => navigate('/control-selection')}. The variable, navigate, is a function, useNavigate(), which routes the user to different pages. Control-selection is the route to the home page.

Website Choosing Game and Method:

The user was able to choose the game and control method. The function renderProgressBar, was responsible for the progress bar located on the bottom of the page. This function fills the bar based on the user's progress. This was determined by the total number of games completed divided by the total number of games required multiplied by 100. User data was saved in local storage using atoms; however, the users were asked not to reload their pages as data may be lost. To prevent this, recorded data was emailed to the researchers after each game was completed. The Take Survey button routed the user to a google form: https://docs.google.com/forms/d/e/1FAIpQLSfTifHQwaAPx-3sFtevV9KAC8lMG6Whkr8_PYKZYI7mNoyQpQ/viewform?usp=header. 

Connect 4 and Checkers Game Board: 

The Connect 4 game board (Figure 1) is a map that has seven columns and six rows. The colours used were different shades of blue. 

 

The Checkers game board is a map that has eight columns and eight rows (Figure 2). The colours used were red, black, white, and blue. “If statements” were used to determine the position of the draught pieces. The red user’s draughts are located on the first three rows, while the black draughts are located on the last three rows. When a user chooses a draught to move, the function isHighlighted, was called. This function highlighted the available positions yellow. 

Connect 4 Game Logic: 

The user competed against a computer player, and always had the first move. Whenever a player placed a disc into a column, the player number alternated. This ensured that the colour of the disc always matched the player (red for player 1, yellow for player 2). If the move was invalid (i.e., the player placed a disc in a full column), the message “Invalid Move” was displayed, and the user was forced to redo their move (Figure 3). A new board was generated with the new disc, if the move was valid. 

 

 

Whenever the user chose a column, within range, a checkWin function was called. This function checks if four same coloured discs are next to each other in horizontal, vertical, or diagonal placements (Figure 4). The game did not end until the user or the computer won and the message “You Win” was displayed. 

 

Functions that calculated possible moves and wins were used to determine where the computer player could place the disc. The computer player had a 40% chance of playing a random move. This ensures that the game is not too difficult to win. To determine where to place the disc, one function gets available columns by returning the empty spots on the board. In another function, these empty spots were used as parameters to determine the possible wins by evaluating possible horizontal, vertical, and diagonal win positions (Figure 4). A points feature was also used to motivate the users to play their best. These points were determined by whether the user wins (plus 1000 points), and by how many moves they took. The max score is 2000. The min score is 0. 

Checkers Game Logic: 

Similar to Connect 4 Game Logic, the user always played first and uses the black draughts. To win checkers, all of the opponents' draughts had to be captured. The computer player had a 40% chance of playing a random move. When the user chose a draught, possible moves were highlighted in yellow (Figure 5). If the draught is a queen, the backwards diagonal left and right moves are highlighted as well. Also, the column and row of the chosen draught were returned as a map. When the user moves the draught, the matrix was updated with the new position of the draught. 

 

Speech Recognition for Connect 4 and Checkers: 

The speech recognition Google API library was used.¹⁷ The speech recognition language used was ‘en-US’ (American English). Two separate files were programmed for speech recognition for Checkers and Connect 4. A number map in Connect 4 was used to ensure that speech recognition recognized alternate ways that a column number can be interpreted as. When the user said a valid column number, a disc appeared above the corresponding column. The word “drop” was the command detected to place the disc in the correct position. The same functions were utilized as described in the Checkers Logic, but inputting different parameters into the functions. A popup alerted the user to use Chrome or Edge if the browser used was not compatible, as determined by a function. In Checkers, two maps were used to determine any alternative ways the speech recognition could recognize the letters and numbers. The Checkers board’s letters were converted to numbers to determine the row of the map/matrix. The first word is the letter (row), and the second word is the number (column). To move a draught, the user said the draught’s current position and the intended final position. Only legal moves were executed; any other commands were ignored. 

Gesture Tracking for Connect 4: 

The Google API library used were 2D hand recognition tracking, and camera recognition.^18,19 The screen was divided into the columns in the Connect 4 game. When the user moved their hand left or right, the coin moved accordingly across the columns. Landmark 4 (thumb tip) and landmark 8 (index finger tip) were utilized so when the user touched their index finger and thumb (act of pinching), the coin dropped in the chosen column (Figure 6).²⁰ The maximum number of hands detected was set to one by ‘maxNumHands”. The camera quality was decreased to increase frame performance. 

Gesture Tracking for Checkers: 

The gesture tracking for checkers used the same APIs as Connect 4, and was a separate file from gesture tracking for Connect 4. The hand movement to control a draught was the same as Connect 4 (Figure 7). The function, getGridPosition, returns the row and column of the draught chosen in an array. When the user confirms the position, the array is updated with the new move. 

Game Over Page:

When the game is completed the user is taken to a page which displays whether the user won or lost. “Emailjs” was the library used to send an email of the user data to the researcher.²¹ The data included game type (checkers or connect 4), control type (classical, speech, gesture), number of moves, total time, hover count, drop count, hover times, and total hover time.

Recieving Participants and Data:

The game was sent to a group of participants who played these games with the alternative modes of control. Once the users completed the games, they completed the survey where I collected my data. 

The graph and chart depict the number and percentage of particpants in each age range. There is more representation in the age group 11-18; however, all age groups are represented. 

Overall, more people preferred speech recognition compared to gesture tracking. 

Individuals with ADHD, Difficulty with focus/ attention, reading, writing, and anxiety prefer gesture tracking while those who had none of the following conditions and difficulty with coordination preferred speech recognition. 

Participants who identified with having anxiety and difficulty with focus, reading, or writing preferred gesture tracking. When the participants did not identify with anxiety, but had difficulty with focus, reading, or writing, they preferred different modes of control. 

61% of the participants preferred speech recognition. This could be because speech is integrated into our daily lifestyles through technologies (i.e., Alexa, Google, Siri) and communicating with one another. Since this mode of control is the alternative most individuals are used to, it makes sense why speech recognition was preferred by the majority of the participants. Individuals in the 11-18 age group preferred gesture tracking, which could be due to the younger generation growing up with more advanced technologies and applications that require the use of gestures (i.e, scrolling on an Iphone). This data shows that the gesture alternative is more comfortable for the younger participants than speech. Participants 19 years old and plus preferred speech recognition because as an individual ages, they grow more susceptible to disease and disabilities such as mobility difficulties, which would result in speech being an easier and more accessible alternative for this age group. Those who had identified having difficulty with focus/ attention preferred speech and gesture equally. This shows that there is no clear correlation between this condition and preferred mode of control. Participants who identified with anxiety, difficulty with reading, writing, and attention deficit hyperactivity disorder (ADHD) preferred gesture tracking. Interestingly, 100% of the ADHD individuals preferred gesture tracking. The data shows that people who have ADHD need to be visually engaged in order to focus and use an application effectively (to its full extent). When people have a difficulty with writing, they have a difficulty in conveying themselves with speech; therefore, gesture tracking proved to be a more preferred alternative for these individuals. People with anxiety most likely preferred gesture tracking because they have a difficulty in conveying themselves with words. Additionally, those who identified with coordination/ movement difficulty and those who did not identify with any of the following conditions preferred speech recognition. It makes sense why people with coordination difficulty would prefer speech, as it is their preferred alternative rather than gestures. Those who had anxiety and difficulty with focus, reading, or writing all preferred gesture tracking. This shows that people with anxiety are not able to express themselves with words without difficulty; thus this explains their preferred mode of gesture recognition. In contrast, those who had no anxiety and difficulty with focus all preferred speech. This could be because people who have no anxiety are more comfortable using speaking as their communication method. Participants who had no anxiety and difficulty with reading preferred gesture tracking equally to speech, depicting no clear correlation between these conditions and their preferred mode of controls. However, those who had no anxiety and difficulty with writing all preferred gesture tracking because when one has a difficulty in expressing themselves on paper (writing), they most likely have a difficulty in expressing themselves with speech. This claim is supported from my data. 

Younger individuals, specifically between the ages 11-18, preferred the gesture tracking mode of control. Alternatively, older partipants, over the age of 18, preferred the speech recognition alternative.

People who identitfied with the conditions: attention deficit hyperactivity disorder (ADHD), anxiety, difficulty with reading, or difficulty with writing, preferred gesture tracking. Those who had coordination difficuly and identified with none of the limitations/ disabilities, preferred speech recongition while playing the game Checkers and Connect 4. 

My alternate hypothesis was supported by this data. 

 

The conclusions from this study can be used to understand what alteratives to touch could be integrated into computer applications and technologies for more accessibility and greater user experience in the future. This research can be applied in various fields, such as banking, medicine, computer science, and gaming. Alternative modes of control, including speech recognition, gesture, head, eye tracking and more, could be used for security and when completing work related tasks. Also, by experimenting with different alternatives people prefer to touch, we can further recognize what can be integrated into future video games, websites, VR games/ tools, and softwares. 

A  source of error could be the sampling error. A larger sample size would show a stronger correlation between the different age groups and conditions people identified with, and their preferred mode of control/ interaction. Another source of error is a limitited representation of participants from each age group. There could be also interface errors and, depending on the user's experience with software bugs, errors, and inefficiencies could have affected their preferred mode of control. 

REFERENCES

1. Mehta VK, Deb PS, Rao DS. Application of computer techniques in medicine. Med J Armed Forces India. 1994;50(3):215-218. doi:10.1016/S0377-1237(17)31065-1

2. Vanderheiden GC. Making software more accessible for people with disabilities: a white paper on the design of software application programs to increase their accessibility for people with disabilities. SIGCAPH Newsl. 1993;(47):2-32. doi:10.1145/155824.155826

3. Jean Crosetto Deitz YS. 16 - Accessing Play Through Assistive Technology. In: L. Diane Parham LSF, ed. Play in Occupational Therapy for Children (Second Edition). Second Edition. Mosby; 2008:395-412. doi:10.1016/B978-032302954-4.10016-9

4. Lei J, Gillespie K. Projected global burden of brain disorders through 2050 (P7-15.001). Neurology. 2024;102(17_supplement_1). doi:10.1212/wnl.0000000000205009

5. Disability. World Health Organization. March 7, 2022. Accessed October 25, 2024. https://www.who.int/news-room/fact-sheets/detail/disability-and-health

6. Hirtz D, Thurman DJ, Gwinn-Hardy K, Mohamed M, Chaudhuri AR, Zalutsky R. How common are the “common” neurologic disorders? Neurology. 2007;68(5):326-337. doi:10.1212/01.wnl.0000252807.38124.a3

7. Smith K. Universal life: the use of virtual worlds among people with disabilities. Univers Access Inf Soc. 2012;11(4):387-398. doi:10.1007/s10209-011-0254-8

8. Devyatkov V, Alfimtsev A. Human-computer interaction in games using computer vision techniques. In: Business, Technological, and Social Dimensions of Computer Games. IGI Global; 2011:146-167. doi:10.4018/978-1-60960-567-4.ch010

9. Carroll JM, ed. Creating a design science of human-computer interaction. In: Creating a design science of human-computer interaction,. Vol 5, Issue 1. ; 1993:3-12. doi:10.1016/0953-5438(93)90022-L

10. Brown M, Anderson SL. Designing for disability: Evaluating the state of accessibility design in video games. Games Cult. 2021;16(6):702-718. doi:10.1177/1555412020971500

11. React. Accessed February 13, 2025. https://react.dev/

12. Installing with Vite - installation. Accessed February 13, 2025. https://tailwindcss.com/docs/installation

13. useAtom — Jotai, primitive and flexible state management for React. Jotai. Accessed February 13, 2025. https://jotai.org/docs/core/use-atom

14. Claude. Accessed February 19, 2025. https://claude.ai/new

15. ChatGPT. Accessed February 19, 2025. https://chatgpt.com/c/67ae376d-b3e0-8013-9a1f-8d4f53ab691f

16. Narakeet - easily create voiceovers and narrated videos using realistic text to speech! Narakeet. Accessed February 13, 2025. https://www.narakeet.com/app/text-to-audio/?projectId=8737c987-05a3-4776-a015-326aabc771db

17. Using the Web Speech API. MDN Web Docs. Accessed February 19, 2025. https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API

18. Camera Recognition. Accessed February 19, 2025. https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js

19. 2D Hand Recognition Tracking. Accessed February 19, 2025. https://cdn.jsdelivr.net/npm/@mediapipe/hands/hands.js

20. Hand landmarks detection guide. Google AI for Developers. Accessed February 19, 2025. https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker

21. Send email directly from your code. EmailJS. Accessed February 19, 2025. https://www.emailjs.com/

I would like to express my gratitude to my mentor Mr. Travis Dow, Departmenet of Electricial and Software Engineering, from the University of Calgary, and my Applied Science Project professor Dr. Beatriz Garcia-Diaz from Webber Academy.