How I build the serverless auto-trading system using Tensorflow and GCP

The auto-trading system consists of 2 parts:

1. The machine learning model to predict the stock future price
2. The serverless automated workflow to execute the ML prediction and then place buying order based on the predicted price.

For part 1, I use Tensorflow to build the model. The model consists of :

• 1 layer of Conv1D with kernel_size=5,
• 2 layers of LSTM with return_sequences=True,
• 3 layers Dense layers to produce the final stock price output of output shape of 1.

Before we use train the model, we need to preprocess the data.

1. Use the python package “yfinance” to download the 4-year historical price of a stock.
2. Scale the price by dividing the price with the initial price, so that the data point is the increase/decrease of the price to exclude the effect of the magnitude of difference stocks.
3. Compute the moving window of 30-day of historical price

As the usual practice, we split the data by 80–20 for training and testing. Finally, we use the trained model to predict the future price for coming 5 days.

For part 2, I use GCP to build the serverless automated pipeline as follows. Please note that all components are serverless, no VM is needed. That means the running cost is based on the actual usage not pre-allocation.

Cloud Scheduler → Cloud Build Trigger → Cloud ML → Cloud pubsub → Cloud function → Alpaca API

1. Cloud Scheduler — it will call the http endpoint of the cloud build trigger REST API to run the ML training and prediction according to the schedule you specified.
2. Cloud build REST API expose the http endpoint for cloud scheduler to call: uri=https://cloudbuild.googleapis.com/v1/projects/${PROJECT_ID}/triggers/${TRIGGER_ID}:run
3. When the Cloud build trigger is invoked, it will retrieve the tensorflow python code for github, and then run gcloud command to submit the Cloud ML training job. (gcloud ai-platform jobs submit training …). Cloud ML will execute your code: the trainer/task_simple_model.py.
4. The python code will download the data, perform ML training, predict future price for coming 5 days.
5. The python code will repeat the training/prediction for all NASDAQ-100 stocks. And then select the best stock to buy based on the Mean Absolute Error (MAE), Accuracy, and predicted price increase. And finally send the buying recommendation to the pubsub topic.
6. The Cloud function will be triggered by the buying recommendation pubsub topic. It will then call Alpaca to place the order.
7. The buying recommendation pubsub message consists of the stock code to buy, and the predicted price increase% in 5 days. When the Cloud function get this message, it will place the bracket order at market price with take-profit = last closed price * predicted increase%, and with stop-loss at the drop% as the same as predicted increase%

This trading system is just for self-learning, not for any investment advise. In fact, I found that although the MAE is not bad, but the accuracy for price rise/drop is just about 50%. That means the ML model is just performing similar to tossing a coin. Haha, it is not a surprise, because how can I make a profit so easily by just a toy ML model!!.

My code can be found in my GitHub repo.