Max Score:
20
Episode Iteration:
9236
RL Method Name:
Monte Carlo
SnakeLength:
1
Grid Dimensions:
[ 4, 4 , 4]
Number of Obstacles:
0
Controls:
Enable Autorotation
Enable Orbit Controls
Start AI
Start AI
Building Snake 3D videogame
As a brief intro, we are going to explain how we recreate the classic Snake to the actual Snake 3D that you might previosly saw. Explaining our implementation of Snake 3D's logic and how we manage to achive this awesome recreation.

Building the Game was a very simple part, because all we needed to do it was follow the logic of snake 2D, and to transform this base logic in a tree dimensional space. Building the graphics was something also nothing complicated because we used the three.js library that allow to us make whatever kind of 3D model that we want, in a simple lines of code. Also as you have seen the graphics are not to much.
We start creating our class Snake that you can see clicking below
The main methods that the snake has are the move(), changeDirection() and checkSnakeState(). With the move() method allows to the snake to move smoothly around the space acording to the direction that was selected. Involves five main atributes that are speed, current direction including the direction of each part of the tail, chosen direction, snake position, the translation error . It also prevents to change the direction if it is not a valid position to be changed, therefore the snake only can move acoording to the grid. Whereas if it is a valid position change the current direction to the chosen one, and also change the user controls, because the direction that the user might chose is in base of his perspective and not in perspective of the entire enviornment.
The Apple Object is a very simple class that you can inspect clicking below
Every time that apple has been captured it is deleted and it is instantiated with a random position avoiding the snake's position and the obstacles's positons.
We also have implemented some obstacles that the functionality that provide is in the case that the snake is in the same position of any obstacle, the snake will loose inmediatly. The implementation you can see it clicking below.
May 25th 2020

We are two students from CETYS University that are currently studying computational science in fourth semester. This platform was a final project of computer graphics course, but we wanted to develop more than an oriented graphic project. On a side note, currently we are reading about temporal difference learning, so on the future would not be a surprise that this method, to solve finite Markov Decision Processes, will be on this platform.
Developer Team:
Gerardo Hernández Meneses
AI and Game Programmer
Pablo Diaz Ochoa
Game Programmer
Building AI
Here we shall explain how we applied one of the most common introductory methods on reinforcement learning to solve a goal-directed learning task using a Monte Carlo method. All the knowledge acquired was recovered from the book at below.

The main point of most of the methods that have been developed on reinforcement learning field, it is that an certain agent must be capable to learn the dynamic of the environment which is confronting in base of the interaction with the environment itself. This is one of tree paradigms of machine learning.
The Snake 3D AI scenario yields a sitatuation that involves take a decision(action) in base of the situtation(state) about the current learning agent is. Being aware about each action the agent could take, could raise on future different states and different signals(rewards). This give us an overview about the Markov Decision Proccess, which is a mathematical framework that describe general equations to solve this problem of having this sequential interaction base on discrete time steps. MDPs is an abstraction of goal-directed learning with only tree main aspects, that are states
In this case we have a finite Markov Decision Process which involves the quantity of states that an learning agent can confront is less than inifnity, this same caracteristic aplies to the set of actions that an agent could perfom, and to the set of rewards that an agent could obtain. Thus a way to solve this finite MDPs we prefered to implement Monte Carlo Methods. Even Though other alterntives as a temporal diference learning can apply.
Monte Carlo Methods are a way to solve the sequential action selection that we have previously mentioned, when we do not have the state-transition probabilities. Whereas one of the best atritubtes that Monte Carlo methods have, it is the fact they only need experience's samples to learn. It learns at the end of each episodic step, and it is base on the MDP framework. Therefore we are going to use a set of States, Actions and Rewards to make this experience's samples. First we needed to determine the value of an action
At this point we are going to explain and describe how we estructure our states, actions, and rewards, that are implemented in this Snake 3D AI. We thought it could be right to set-up the state structure by the snake position followed by the apple position, and also adding the current direction of the snake. Initially if we analyze the environment it is easy to see that we count with
The set of action the agent can perform is represented as
As an instance about how the state's structure is
Start AI