We hypothesize that genetic algorithms implemented into the Connect 4 AI model will perform slightly better than the non-genetic algorithms model. We think that the genetic algorithms model will win more games against a normally-trained model.

Connect Four

Connect Four has a 6 vertical by 7 horizontal board where players take turns dropping different colour pieces. The objective of the game is to be the first to form a horizontal, vertical, or diagonal line of four of your own colour pieces. In the case that the board fills up without either player connecting four in a row, then it is considered a tie. Invented by Howard Wexler, and was first sold by Milton Bradley in 1974. The game is actually very complicated because there are 4,531,985,219,092 (about 4.5 trillion) possible situations the board can be in. Making the game very difficult and not so simple.

Neural Networks

Neural networks are a simplified computer model of neurons in the brain.  A neural network processes an input through several layers of neurons and emits an output.  The processing is controlled by numeric weights on each neuron, which control how the neuron activates other neurons in the network.

The neural network can recognize patterns in the input through the connections in the neurons.  AIs for games usually use neural networks because humans also look for patterns in the shape of a game board to decide what moves to make. 

How To Apply Neural Networks to Connect Four

• The input to the network is the state of the board (where pieces are placed on the board, and whose turn it is).
• The output of the network is a number for each possible move. The highest-numbered move is where the neural net recommends playing.

Training a Neural Network

For a game like Connect Four, a neural network can learn how to play well if it is taught which moves are good and which are bad.  Teaching the network uses a process called Reinforcement Learning, where we modify the weights in the network.

When we know a move is good, we update the weights so that this move is more likely to be played in the future if the inputs are similar.  When we know a move is bad, we update the weights so that this move is less likely to be played in the future.  This means that good moves get rewarded over time and bad moves get penalized.

We also need to play random moves on occasion, to experiment with new approaches.  This is similar to how a human might experiment with new ideas when learning to play a new game.

Training a Connect Four neural net traditionally involves playing the model against itself over many games, and teaching the model good moves and bad moves as it wins and loses games.  

Genetic Algorithms

Genetic Algorithms are a method to optimize a solution by randomly modifying parameters. Genetic algorithms were inspired and modeled around natural selection, where genes in living things randomly mutate over time, and gradually the living things adapt to fit their environment as the individuals with the best fit tend to reproduce more.

We can think of a set of parameters as “genes”.  Random modifications to the parameters can be “mutations”.  Parameters with the best fit are mutated to produce new parameters, while parameters that do not fit are discarded.  Like with natural selection, we expect the population of parameters to gradually adapt to fit better.

Genetic Algorithms For Neural Networks

For neural networks, the weights on the neurons are the “genes”.  We can produce mutations by randomly modifying the weights, and a model is more “fit” if it wins more games versus other models.

To implement a genetic algorithm, this means we need to maintain a population of models and have them compete against one another.  

Setup Of Genetic Algorithms

1. Create a population of neural network models
2. Each model plays games against every other model
3. Rank the models by the number of games won
4. Remove some of the worst-performing models
5. Refill the population by cloning and mutating the remaining models to get back to the original size
6. Repeat from step 2

Manipulative
Controlled
Responding
The manipulative variable is the type of training, with genetic algorithms or not. One model will have genetic algorithms implemented, and another one will not.
The controlled variable is the engine and the amount of time used for training both models. Both models play connect 4 and both models will receive the same type of code.
The responding variable is the level of skill each model has after training and how much progress it has made. That is the genetic algorithms model and the non-genetic algorithms model.

The materials used in our project were mainly composed of computers, as our project was based on AI and ML. Those materials include:

• The server running the AI has an AMD Ryzen 5 5600X, 32GB of RAM, more than a terabyte of storage, all paired with AMD’s most powerful and advanced graphics card, powered with AI capabilities, released on March 6th, 2025, and that is the AMD Radeon RX 9070XT.
• Other materials we used were apps, such as Microsoft Visual Studio Code, Google Chat, Google Docs, Microsoft DevHub, GitHub, Git, and other websites and minor apps.
• We used Python as our coding language as well as using PyTtorch for building the AI
• used AMD ROCm software through the PyTorch library to build neural network models

Procedure

First, we implemented the rules of Connect Four in a Python program, and allowed neural network models to choose moves to play in a game.

Our procedure then compares genetic algorithm training against normal training without genetic algorithms. Normal training uses neural networks with reinforcement learning. Genetic algorithms however use genetic algorithms on top of normal training

Comparison method

To judge whether one model is stronger than another, we have them play 2000 games against one another.  We consider the model that wins more games to be stronger.  We check the strongest normal and genetic models against one another periodically.

Reinforcement Learning method

We train models by playing a series of 100 Connect Four games between one model and another.  Humans are not involved in training or supervising the process.

After each game, we consider the last winning move to be “good” because it won the game.  The loser’s last move is considered “bad” because it did not prevent the loss.  On the winning side, we also choose 5 random moves from the list of moves the model played and consider those “good” as well, because they should have led to the winning situation.

Genetic Algorithm Process

We have chosen these parameters for our genetic algorithm:

• The population size is 40 neural network models
• After all of the models play each other, the bottom half are removed
• Fill the population back to 40 with copies of the remaining models, mutating 1% of weights
• Mutation is done by multiplying weights by a random number between -2.0 and 2.0

Normal Training process

Normal training only maintains one neural network model, and continuously plays it against itself.

Training Time

Both normal and genetic training systems were run in parallel together, and we periodically compared the top genetic model with the normal to see which was stronger.

Training ran continuously for 20 days.

Raw Data

Subjective Observations

• Both the genetic and normal AI models usually played in the same column
• They would move to another column if the first one was full
• Neither seemed to react to block a winning move

By the end of training, the genetic model won slightly more games over the normal model.  This result validates our hypothesis to some extent.

However, by observing the moves that the models played, neither one seemed to understand how to win the game.  They mostly played in the same column, not really reacting to an opponent’s moves.

Despite that, the genetic model still performed slightly better than the normal model.  We do not fully understand why this is the case, since both models always seemed to play in the same places.

As a result, we are not very convinced that our hypothesis is actually true.  It is hard to conclude that genetic algorithms are definitely better than normal training, but more investigation is justified.

If genetic algorithms are able to achieve better results in the same amount of training time, then they could help reduce the amount of resources needed to get a result.  This would mean we can save electricity and time developing AI. Genetic algorithms work when fitness of models can be easily measured.  When the fitness can be measured easily, this technique could be applied.

We wrote the code for the Connect Four game engine and training process.  There might be bugs in the code due to our inexperience, that were preventing training from working effectively.  There might also be other programming techniques that we did not use that would perform better.

• We tried our best to choose good genetic algorithm parameters, such as the size of the population, the number of games in a series, the mutation rate, and so on.  It’s possible that better choices for these parameters would give better results.

• Our time and compute power was limited during this experiment. We trained for 20 days on one computer.

• The neural network parameters include the size and number of layers, and the types of computation that each layer performs.  There are many possible choices for the shape of the network, and we might have chosen a shape that was not very effective for learning Connect Four.

References GeeksforGeeks. (2016\, June 14). Minimax Algorithm in Game Theory | Set 1 (Introduction). GeeksforGeeks. https://www.geeksforgeeks.org/dsa/minimax-algorithm-in-game-theory-set-1-introduction/ GeeksforGeeks. (2017a, June 29). Genetic Algorithms. GeeksforGeeks. https://www.geeksforgeeks.org/dsa/genetic-algorithms/ GeeksforGeeks. (2017b, August 21). Introduction to Convolution Neural Network. GeeksforGeeks. https://www.geeksforgeeks.org/machine-learning/introduction-convolution-neural-network/ GeeksforGeeks. (2017c, September 8). What is Artificial Intelligence(AI)? GeeksforGeeks. https://www.geeksforgeeks.org/artificial-intelligence/what-is-artificial-intelligence-ai/ GeeksforGeeks. (2019\, January 14). ML | Monte Carlo Tree Search (MCTS). GeeksforGeeks. https://www.geeksforgeeks.org/machine-learning/ml-monte-carlo-tree-search-mcts/ Leung, S. (n.d.). Genetic Algorithms [Personal communication]. PyGAD - Python Genetic Algorithm! — PyGAD 3.2.0 documentation. (n.d.). Pygad.readthedocs.io. https://pygad.readthedocs.io/ Świechowski, M., Godlewski, K., Sawicki, B., & Mańdziuk, J. (2021). Monte Carlo Tree Search: A Review of Recent Modifications and Applications. ArXiv:2103.04931 [Cs]. https://arxiv.org/abs/2103.04931 Wikipedia. (2019, February 18). Artificial Intelligence. Wikipedia; Wikimedia Foundation. https://en.wikipedia.org/wiki/Artificial_intelligence Wikipedia Contributors. (2019, November 16). Connect Four. Wikipedia; Wikimedia Foundation. https://en.wikipedia.org/wiki/Connect_Four Zvornicanin\, E. (2022\, April 11). Convolutional Neural Network vs. Regular Neural Network | Baeldung on Computer Science. Www.baeldung.com. https://www.baeldung.com/cs/convolutional-vs-regular-n

We, Marcus Leung and Jayden Yin, would personally like to thank all the people who helped us throughout this project, including our supportive parents and our teacher, Ms. Lai. Thank you to Westmount Charter for creating and managing our school fair. Also, to the people who helped set up the Calgary Youth Science Fair event, and to Dr. Adriana Klassen, Moon Kim, Xiaomeng Wang, Jan Brigden, and Franklin Koch who helped judge our project and citations. Once again, special thanks to everyone who helped us and to all those involved in the Calgary Youth Science Fair project.