Tag Archives: Machine Learning

AWS Deep Racer: Part I

So after my little experiment with Predicting Stock Prices with Python Machine Learning I came across the AWS Deep Racer League. A machine learning racing league built around the concept of a little single camera car that uses Reinforcement Learning to navigate a race track. I’ve decide to sign up and Compete in upcoming the October race and share what I learn along the way.

photo cred: aws

Let me give a little bit of background on how the Deep Racer works and the various pieces. There are two ways to train your little race car, virtually and physically. If you visit the Deep Racer site you can purchase a 1/18th scale race car that you can then build physical tracks for, train, and race on. There are also various models you can purchase with a couple of different input sensors so you can do things like teach the car to avoid obstacles or actually race the track against other little cars.

This little guy has 2 cameras for depth detection and a back-mounted sensor for detecting cars beside/behind it

The good news is you don’t actually have to buy one of these models to race or even compete. You can do all of the training and racing virtually via the AWS console. You also get enough resources to start building your first model and training it for free!

Now let’s get into how this actually works. What AWS has done is built a system that does most of the heavy complex machine learning for you. They offer you the servers and compute power to run the simulations and even give you video feedback on how your model is navigating the track. It becomes incredibly simple to get setup and running and you can follow this guide to get started.

Advertisements

When you get setup you’ll be asked to build a reward function. A reward function contains the programming logic you’ll use to tell your little race car if it’s doing it’s job correctly. The model starts out pretty dumb. It basically will do nothing or drive randomly, forward, backwards, zig-zags…. until you give it some incentive to follow the track correctly.

This is where the reward function comes in. In the function you provide the metrics for how the model should operate and you reward it when it does the job correctly. For example, you might reward the model for following the center line of the race track.

On each iteration of the function you’ll be handed some arguments on the model’s current status. Then you’ll do some fancy work to say, “okay, if the car is dead center reward it with 1 point but if it’s too far to the left or the right only give it a half point and then if it’s off the track give it zero.”

The model tests and evaluates where the best value is based on the reward function

The model will then run the track going around, trying out different strategies and using that reward function to try and earn a higher reward. On each lap it will start to learn that staying in the middle of the track offers the highest, most valuable reward.

At first this seems pretty easy… until you start to realize a couple of things. The model truly does test out all types of various options which means it may very well go in reverse and still think it’s doing a good job because you’ve only rewarded it for staying on the center line. The model won’t even take into account speed in this example because you’re not offering a reward for it.

Advertisements

As you start to compete you’ll find that other racers are getting even more complex like finding ways to identify how to train the model to take the inside lane of a race track or how to block an upcoming model that is attempting to pass it.

On the surface, the concepts and tools are extremely easy to pick up and get started with, however the competition and depth of this race are incredible. I’m looking forward to building some complex models over the next couple of weeks and sharing my results. If you’re interested in a side hobby check out the AWS Deep Racer League and maybe we can race against each other!

Predicting Stock Prices with Machine Learning in Python: Part I

Over the last few weeks I’ve been keying away at building an application that can analyze stock prices and make use of Python Machine Learning libraries to predict stock prices.

This is the first part of a series of diving into machine learning and building this application. I’ve uploaded the entire project thus far to my personal GitHub repo at: https://github.com/Howard-Joshua-R/investor

I invite anyone and everyone to take a look at the project, fork it, add to it, point out where I’m doing something stupid, and build it with me! If you help, you’re more than welcome to use it for your own advantage.

photo cred: Shahadat Rahman

For this first post, I’ll walk through what I’ve built so far and how the meat and potatoes work.

If you drill into the directories and find the ‘spiders’ folder you’ll find the ‘lstm.py’ file. This particular spider is using Scrapy and an LSTM model to predict the stock price of any stock ticker you pass to it. Let’s take a look at the first piece of this tool, the scraper:

    def start_requests(self):                       

        ticker = getattr(self, 'ticker', None)      
        if (ticker is None):
            raise ValueError('Please provide a ticker symbol!')

        logging.getLogger('matplotlib').setLevel(logging.WARNING) 
        logging.getLogger('tensorflow').setLevel(logging.WARNING)  

        apikey = os.getenv('alphavantage_apikey')                   
        url = 'https://www.alphavantage.co/query?function=TIME_SERIES_DAILY
               &symbol={0}&apikey={1}&outputsize=full'.format(ticker, apikey) 

        yield scrapy.Request(url, self.parse) 

This first function uses Scrapy to reach out to AlphaAdvantage and pull down stock information in JSON format. AlphaAdvantage provides fantastic stock data on open and close prices over the last decade or longer. All it requires is that your register with them to obtain an APIKey. Best part, it’s’ free!

Now let’s break down each piece.

def start_requests(self):                       

        ticker = getattr(self, 'ticker', None)      
        if (ticker is None):
            raise ValueError('Please provide a ticker symbol!')

Here we define the name of our first function ‘start_requests(self)’ and allow Scrapy to know where to start our spider. From their we look to grab the ‘ticker’ argument which is what we are passing through to tell the spider what stock data to collect. I’ve currently tested TSLA = Tesla, AMZN = Amazon, and TGT = Target. Simply providing the ticker in the ‘main.py’ file is enough to set the target ticker. The final 2 lines here simply validate you’ve passed in the ticker argument.

photo cred: Sigmund

The next two lines suppress logs from two of the libraries we’ll use later for predicting our model. MatPlotLib is used to plot points on a graph and TensorFlow is used to help us implement the LSTM training model.

logging.getLogger('matplotlib').setLevel(logging.WARNING) 
logging.getLogger('tensorflow').setLevel(logging.WARNING)  

The following lines set our AlphaAdvantage APIKey and the URL we’re going to hit for our stock data. In this case you’ll want to store your AlphaAdvantage APIKey in your environment variables on your machine under the name ‘alphavantage_apikey’.

apikey = os.getenv('alphavantage_apikey')                   
url = 'https://www.alphavantage.co/query?function=TIME_SERIES_DAILY
&symbol={0}&apikey={1}&outputsize=full'.format(ticker, apikey)

The final piece kicks off the Scrapy spider and once it completes passes the results to our parse function. We provide our newly built URL, which contains our target ticker and APIKey, and our parse function.

yield scrapy.Request(url, self.parse) 

So far just this piece uses a Scrapy spider to reach out to AlphaAdvantage and download stock data in JSON format. In the next piece I will dive more into the parsing and building the machine learning model.

In the meantime feel free to jump out to my GitHub Repo and read through the comments of the lstm.py file. I’ve attempted to include as many notes as I could and left some open ended questions as well. If you have any feedback I’d be more than happy to discuss! If you’re feeling brave and want to submit your own pull request please do!