Back To Index

Background

For Agent Based Modelling, Reinforcement Learning is used to teach a Model how to approach a solution for an Environment. For example, driving around a track or attempting to drive up a steep hill. Typically these Environments would have a set of straightforward actions and rules, allowing the Agent in the Environment to efficiently reach the end goal of completing the task given to it, accumulating rewards effectively.

These Agents use algorithms, a set list of actions and gain rewards in order to determine what their end goal is. I wish to look into adding randomness to see how it affects their overall approach to learning the best way to maximize reward gain.

By introducing randomness into these otherwise static environments, I hope to gain new knowledge on why exactly randomness isn’t used in most Environments as a standard, and hope to see if I can find a reason as to why or why not randomness should be used in all Environments.

However, randomness is not a new concept to Reinforcement Learning, as randomness can be used in some scenarios to make Agents try alternative paths. For example, imagine an environment where the Agent can pick two doors. The Left Door gives +1 reward, and the right door only has a 10% chance to give +100 reward, otherwise it gives 0 reward. If the Agent picks the Right Door and gets nothing, then picks the Left Door, it will assume that picking the Left Door is always the correct choice, despite the fact the Agent could be getting +100 reward from picking the Right Door.

Motivation

While Randomness is standard in a few different Environments, I want to understand why Dynamic Environments aren’t used more often as opposed to Static Environments.

This is where non-deterministic elements play their part. I want to create my own Custom Environment so that I can create my own Dynamic Environment and observe the effect it has on an Agent Based System. While doing this, I hope to better understand what happens under the hood in a model, as well as try to propose a solution to Dynamic Environments.