Back To Index

I wanted to know one thing going into this project, is it even worth it to try injecting randomness into environments?

For my custom environment, because the Model has to learn how to tackle the environment, it’s interesting to see how the Model adapts to the Dynamic Environment, but what about a pre-made Environment with an already existing solution?

For my picked Environment that was already made, I chose Cartpole.

This environment has a Model learn how to balance a Pole on top of a Cart, without the Cart going to an end condition, which are moving too fast, going too far off screen, or letting the Pole tip over.

The environment awards the Model for every frame it runs, and stops giving rewards for an episode when an end condition is met.

Non-deterministic Element chosen

For Cartpole, I decided to change the Environment code, so that every now and then, the speed at which the Pole tips over would greatly increase, meaning that the Model would have to try and catch the Pole without letting it fall over.

Expectation vs Reality

When I first began to work on this section of my Project, I had presumed that the Model would overreact when my non-deterministic element was introduced.

However, once I introduced randomness into the environment, the exact opposite happened.

When I increased the Speed of the Pole so it fell faster, the Model simply began moving in the correct direction to catch the Pole.

I had assumed that the Model would get caught off-guard and the episode would end prematurely, but instead the Model was easily able to catch the Pole and continue onward.

My reasoning for this is that the Mode has already learned what movements to do at every Angle and Position for the Pole and Cart, so increasing one aspect simply moved the Model into a different State it had already explored and solved.

I am glad I tried this out, as it gave me more of an idea on how best to approach my own Custom Environment.