TORCS-Self-Driving-Agent

Introduction

The project involves building of an intelligent game playing bot for the TORCS Racing environment.

Languages and libraries used:

Python

• Tensorflow

• Keras

Approach:

In order to train an AI Bot there can be different approaches but we opted to apply behavioral cloning and see how our agent mimics the behaviour seen in the data. There were two possibilities to collect the data i.e. either we play and collect the data or we find a perfect player online from which we can extract the data. We were able to find a perfect open-source agent named Ahoora being developed in Java therefore opted for this option. Game data (offline) was generated by running Ahoora on 30 different races, comprising 3 laps each. Resulting data was dumped in a csv format which was around 10 GB.

Preprocessing:

We had data for 30 races in every track. For feeding it to neural network, 73 sensor values (features) and 5 actuator values (labels) were seperated out. Labels consisted of accleration, brake, clutch, steer angle and gear. As gear can be changed according to the speed so gear was not predicted. Training data consisted of first 29 races of every track and Validation data consisted of last race of every track. The data was normalized and then stored the means and standard deviations which will be used in making a test example.

Experiments:

Training data consisted of more than 11 million examples and validation data consisted of 2 million examples. Initially made a small neural network containing three hidden layers of 512, 256, 128 neurons. Model was under-fitting since the number of parameters to learn were too low. After 30 epochs, the loss became constant and the model was behaving average when tested. Tried a deeper neural network having eight layers with more neurons and this model converged nicely.

Data Generation:

1. data_generation is the folder having all the necessary files for data generation. data_generation/Ahura is the agent from which data is extracted.
2. Place data_generation/temp/temp to /data_generation/.
3. Change directory to data_generation using cd data_generation.
4. Run python data_generation_server.py. For this, we do not need to activate any environment. Default environment will be okay. It will run TORCS and then click on every first option i.e. Race, Quick Race, New Race. It will then start waiting for the client to connect ;)
5. Run python data_generation_client.py. You will see the client will connect and race will start. When the race ends, the data is generated in ./data/.

Training:

Training material is present in model/Model.ipynb. It contains integration with sample data. Similarly one can change this notebook according to the data being generated.

How to run trained TORCS agent

Pre-requisite

1. Anaconda python installed. In case anaconda is not installed, follow Anaconda Installation and install anaconda.
2. Clone the repository and import the environment by running conda env create -f torcs_env.yml.
3. Activate the enviornment i.e. conda activate torcs_env.
4. Client.py is the file which tries to connect with the server. Run client.py and make sure no error comes.

How to run the server

1. torcs_server is the folder which is required for running the server. This server was downloaded from an open source github repository. The exe file named wtorcs.exe is the main server file which will be used to listen from client. Run wtorcs.exe file in torcs_server folder and then click on Race.
2. It will show multiple types of races. The one normally used for testing is Quick race and clicking it will show the option of New Race and Configure Race.
3. First we need to configure the players therefore click on Configure Race. Choose the track and then click Accept.
4. There should be one server named scr_server 1 in selected or Not selected tab. This is the client name which will be run therefore make sure to select the scr_server 1.
5. Other cars can be added to see the performance of the agents in presence of other agents.
6. Click on accept and then Quick Race Options will pop up. Set the options according to the requirements and click on Accept.
7. Then Click on New Race and enjoy seeing the agent race ;-)

Demo

Torcs.mp4