The Race is on! Sort of…
So I’ve been diving in deeper to AWS Deep Racer, pun not intended… and I’ve managed to work through a handful of tutorials. I’ve built 4 different models to race with so far.
My first model, Mark-1, currently holds the best record on lap time, of my 4 models, for the October race sitting at position 737 of 791 total racers. The October 2020 track looks like this:
The Mark-1 model used a very basic reward function that simply trained the model to attempt to follow the yellow dashed line on the center track. It’s a default reward function that reinforces the model attempting to stay towards the center of the track. It doesn’t get you far… but it does get the job done. Total race time: 08:29.214
With the Mark-2 model I tried to just build off of the first iteration. I added some additional logic to keep the model from trying to zigzag too much on the track as it worked towards finding the best center line. I also made a classic rookie mistake and implemented a bug into the reward function where instead of incrementing the reward value when the model stayed on track, I was just applying a default… this model didn’t even rank.
With the Mark-3 model, still containing the defect from Mark-2, I played around with the Hyperparameters of the reinforcement model. As a part of the training system you can do more than just modify the reward function that trains the model. You can also adjust and modify the batch size, number of epochs, learning rate, and a handful of other Hyperparameters for training the model. Since I still had the bug in my reward function from the previous model, this model didn’t rank either.
On to Mark-4… I caught my defect.. ugh… and fixed that up. I also decided that at this point I was ready to move beyond the tutorials and really start adjusting the models. So the first thing I did was take a look at what the competition was doing… Here’s a look at the number 1 model currently sitting at the top of the October qualifier…
Well shit… I’ve got a ways to go to compete with this model. I mean this model is flying around the track. Mine looks like a fucking drunk wall-e….
So I’ve got some work to do but the current AWS Deep Racer app is down. As soon as it comes back I’m refocusing on speed first. I need to Lightning McQueen this bad boy if I’m going to stand a chance of even ranking in the top 100 for the October Qualifier.
Check back with me next week and we’ll see if I can get this little guy going!