45 laps of qualifying session with 20 degrees of weather and optimum track condition, sampled with 10hz. Test is made with the setup of Ohne Speed in Ford Mustang GT3(2024). In track Hungaroring.
After getting healthy results with XGBoost the examinations of the features showed that most dominant feature were the temperature of the brakes, also must indicate that setup's brake bias was 48.5 heavily relying on rear brakes. Also worth noticing that the track itself is a rear limited braking heavy track. So when we take all of things happening on the track, rather than showing "Fast drivers are fast" by indicating the average speed feature, this project managed to showed that the main key is to be fast in this circuit is essencialy braking technique and efficiency.
This project attempts to measure driver performance in narrow conditions, which means the car's setup, fuel, tire degradation, and weather are kept constant throughout the data. My main driving factor was the gatherability of real MoTeC data from Assetto Corsa Competizione, which would help me to work with real-world examples.
I first trained the Random Forest Regressor and got a result of ~0.059 MSE (RMSE 0.243, about 0.24 seconds off). I found that the average speed was the dominant feature (~0.77).
Second, I tried XGBoost and got a result of ~0.046 MSE (RMSE ~0.21, about 0.21 seconds off). However, the shocking result was the importance of the temperature of the rear right brake as a feature. Without brake temperature data, the XGBoost model performed roughly the same as the RFG model.
Below, I've tried to explain my thinking and technical process. Fixed Variables
For simplicity and to focus on driver performance, some variables were fixed: fuel, tire degradation, weather, and, most importantly, the car's setup.
The data was completely gathered from my 82-minute-long qualifying session in Assetto Corsa Competizione with the Ford Mustang GT3 and ohnespeed's setup. Over about 45 laps, my times varied between 1:43.9 and 1:48.0, which roughly covered 101% to 107% of alien laps (according to Ohne Speed's ACC LFM laptimes spreadsheet). The first struggle was to understand the .csv file collected from the MoTeC i2 pro software.
Lap beacons were bugged, which meant that telemetry data didn't show when the car passed the start/finish line. Another issue was cleaning the data from out-lap telemetry. This happens when a car has just exited the pit for a qualifying lap or is cooling down after a qualifying lap. Out-laps occur because, in a qualifying session, a car starts a bit behind the start/finish line to gain straight-line speed. This wasn't a big issue; I just skipped some rows to find where the actual lap started.
For the first issue, even though the lap beacon wasn't working, the last column, Time, was collecting the lap time data, but with a minor problem. Since the data was encoded at 10Hz, every second had to be counted ten times, as every portion of a second reflected the car's current situation. However, second 0 was counted five times in every lap, and I'm not sure why. After handling these issues, I managed to compile the laptimes and the lap counts.
Sidenote: As you can see in the code, there is almost no visualization other than the first lap. The main reason was that the i2 software itself is a visualization tool for telemetry, and I mostly used i2 for that purpose.
Of course, I was expecting the average speed to be a dominant feature, but I wanted to try my luck with other features. I created features for throttle and brake efficiency, full throttle time, and coasting time.
Initially, I thought the first two were directly related to TC and ABS, but having TC and ABS as features themselves would be enough. So, I used the formula G-Force/Throttle and the same for the brake. The last two were intuitive. time_at_full_throttle was a "more gas, more speed" approach, and coasting_time is more related to the efficiency of the brake and gas. In a lap, good drivers usually don't waste their pedals. They generally do minimal braking or gas rather than no brake or gas. They are always on the input, making minimal changes on corners to get the fastest cornering speed. (This technique is known as Trail Braking). As you can see in the code, I've tried many features from the MoTeC data.
In the research "A Machine Learning Approach for Modeling and Analyzing of Driver Performance in Simulated Racing", Extreme Gradient Boost was the best model in this scenario. So, to verify this, I compared it to the Random Forest Regressor. XGBoost performed better with the same features. But the main difference was that the rear right brake temperature was a more impactful feature than the average speed, which was unexpected. After testing the model without brake temperature data, I found that for the Extreme Gradient Boost model, brake temperature is critical, as the MSE changes from ~0.04 to 0.0579.
Resources: A Machine Learning Approach for Modeling and Analyzing of Driver Performance in Simulated Racing
Ohne Speed setups and Youtube Channel can be found here: Ohne Speed SpreadSheet | Ohne Speed Youtube
written by a human