Machine Learning

Early 2019 I took an online course on machine learning (available on Udemy here). Upon completion I decided to try and build a neural network capable of playing a game I created a while back. Check out the results here!

I suggest you first check out the video in the top of this post, then get back here if you want to know more. The model is based on training a neural network using reinforcement learning (specifically using a technique called Q-learning). Described in more depth in the below pseudocode:

  1. Initate random weights for the neural network’s hidden layer.
    These weights determine which action the agent should take depending on the input state.
  2. Check available line of sight in 5 directions.
    Specifically the five colored raycasts in the figure, indicating how far away any blocking object is located.
  3. Take action based on input state.
    1. Turn left
    2. Do nothing
    3. Go right
  4. Assign reward.
    1. If crashed = negative reward
    2. If alive = minor positive reward
    3. If successfully passed a tree = positive reward
  5. Log memory.
    Save the input state, which action was taken and the associated reward.
  6. If alive = repeat step 2-6.
  7. If crashed = train the neural network using backpropagation and experience replay.
    Reduces the likelihood of the network committing the same action that resulted in a crash, while simultaneously reinforcing the behaviour that resulted in a positive rewards.

 

The game is built in Unity and is written in C#. No external machine learning libraries were used.