Can Azure Drive Your Car – I got the idea after making my GA code in PowerShell and then saw George Hotz self driving car
( https://www.youtube.com/watch?v=KTrgRYa2wbI ). I thought, I might be able to make mine own self driving car using Azure Machine Learning. So I started thinking and came up with the idea to create a top-down car game which can be controlled by keyboard and web requests. Since I did not know anything about game programming, I turned to my friend Andreas who helped me get started.

*My favorite car in GTA 1

The first problem control the car.

I wanted both stepless throttle and steering. To get a smooth steering which also could be predicted by Azure, a binary on-off steering would definitely fail.

So by using a steering interval from -3 to 3 and always slowly resetting the value to 0. I get a smooth return of the wheels. An as long as a press left or right key I will add or subtract a value to the steering value.

The next thing was to apply this force differently depending on the current velocity of the car. Because I don’t want the car to be able to turn when not it’s moving and also not be able to turn as quickly at a greater velocity.

 
if ((Speed + 16.1f) > 0)
 {
 ZeroSteer = (1 / (Speed +16.1f));
 } else
 {
 ZeroSteer = 0 ;
 }
transform.rotation *= Quaternion.AngleAxis(steering * 8f * (curspeed.magnitude * 0.5f) * ZeroSteer, Vector3.forward);

The variable Speed is not my throttle nor the velocity of the car. The speed is calculated a similar way as steering to make acceleration smooth. The Speed is calculated by subtracting the Break Power from the Acceleration power. Both are calculated so that you have a greater acceleration and deceleration at a lower velocity that at a higher. The break power is two times more effective to get it more realistic, problem to solve here is that the car accelerate faster when moving backwards, but that’s a later problem.

BreakPower = Mathf.Sqrt(Mathf.Pow((((BreakLvl / 100) / (forwardVelocity.magnitude + 1)) / 0.1f), 2));
AccPower = Mathf.Sqrt(Mathf.Pow((((AccLvl / 100) / (forwardVelocity.magnitude + 1))), 2));
Speed = (AccPower - BreakPower);

Last but not least I wanted to create a drift feeling in the car. And found this online. The constant 0.931f is just a value which I found suitable for my project. This can be a variable to simulate different frictions on different surfaces.

forwardVelocity = transform.up * Vector2.Dot(rig2D.velocity, transform.up);
rightVelocity = transform.right * Vector2.Dot(rig2D.velocity, transform.right);
rig2D.velocity = forwardVelocity + rightVelocity * 0.931f;

Another problem is the fact that this is a top-down game and the 2d environment is a platform game. So gravity doesn’t pull the car in the right direction nor does friction affect the car object because it’s in midair.

This is solved by constantly applying a fake friction and ignoring the gravity on the object.

rig2D.angularVelocity *= 0.9f;

And finally we apply the force to the object.

rig2D.AddForce(transform.up * Speed * 80.8f); 

The map

The map is basically just a sprite I found on google. In version 2 this will be generated.

By adding polygon colliders to the map sprite I can now trigger on edges of the road.

At first I used short raycasts as shown in the picture below, to measure the distance from the car to the edges of the road.

This resulted in the driver to be half blind. And the result wasn’t satisfying, so I added some more raycasts and change the length of them to always hit a collider.

Saving the data.

In azure ML I’m using a supervised learning method which means that I need to learn Azure to predict results from a dataset. Like Geohotz say in the video. We don’t learn how to drive by complex rules, we learn to drive by copy how others drive.

So to learn Azure ML to drive I have to drive the car myself. While I’m driving I also recording all the data from the raycasts and the car’s velocity and throttle and this give me a dataset to train the ML.

This is an example of the data. The data is saved for each frame and then saved to a csv file.

distfwrd,distfr,distfr1,distfr2,distfl,distfl1,distfl2,distbr,distbl,distbac,distRight,distLeft,AccLvl,BreakLvl,FrwdVel,righVel,stering
19.26596,1,2.715403,2.185843,0.9999999,6.960348,5.049075,5.858468,5.858468,18.06574,2.096169,4.498649,37.87161,0,2.208083,0.2627529,1.961354
20.12596,1,2.749578,2.190165,0.9999999,6.81405,5.008363,5.971817,5.971817,18.51581,2.083464,4.500871,38.54557,0,2.295704,0.2895483,1.961354
20.76004,1,2.788377,2.196275,0.9999999,6.670733,4.96831,6.095795,6.095795,18.93248,2.07175,4.504496,39.21952,0,2.204741,0.2928745,1.961354
21.37745,0.9999999,2.825896,2.202798,0.9999999,6.548989,4.934487,6.214087,6.214087,19.15173,2.062054,4.509322,39.94371,0,2.295997,0.3147297,2.019289
22.02164,1,2.870029,2.21175,1,6.424896,4.899772,6.346921,6.346921,19.39728,2.053388,4.515352,40.61779,0,2.20856,0.3130144,2.073215
22.4229,1,2.917815,2.22259,1,6.308402,4.867023,6.484819,6.484819,19.65132,2.04653,4.522227,41.29173,0,2.202325,0.3352932,2.127131
22.91497,0.9999999,2.980162,2.24242,0.9999999,6.187024,4.828842,6.62499,6.62499,19.87586,2.047292,4.524438,41.9657,0,2.40659,0.3622047,2.181048
23.44104,1,3.046011,2.259682,1,6.066928,4.795052,6.801198,6.801198,20.19656,2.043279,4.534757,42.63966,0,2.309924,0.3618215,2.234965
23.77752,1,3.117147,2.279232,1,5.955251,4.763781,6.98578,6.98578,20.37699,2.041283,4.546332,43.31364,0,2.226528,0.3587796,2.288884
24.13478,1,3.194163,2.301162,1,5.850802,4.73473,7.179958,7.179958,20.55269,2.041247,4.559137,43.98758,0,2.155112,0.3539713,2.342799
24.51509,1,3.277806,2.325611,1,5.752548,4.707642,7.385282,7.385282,20.73804,2.043134,4.573185,44.66159,0,2.258624,0.3779971,2.39672

Azure Machine Learning

I’m using tree types of learning algorithms to find the best suitable. I’m using three types of learning algorithms to find the best suitable. In my case the “Boosted Decision Tree Regression” gave me the best result

Models used

Boosted Decision Tree Regression – best driving preformance

Nerual Network Regression – lowest error in the predicted result, but made the car began to move like a snake.

Linear regression – good but not as good as the Decision Tree.

I tested a lot more algorithms but this three is the ones I find the best.

So how do you know which one to use? Start by using the cheat sheet
https://azure.microsoft.com/en-us/documentation/articles/machine-learning-algorithm-cheat-sheet/ and then test, test, test.

Here is an example of a visitation of one of the Decision Trees in my trained model. You don’t have to understand It and that’s the beauty of Azure ML.

After creating the experiment, I added a filter to the output to only send the predicted steering value.

The final predicted experiment looks like this.

Web service problem

Unity is using Mono .net 2.0 so using the nice C# example code from Azure web service didn’t work as it was, so I had to rewrite it.

String jsonString = "JSON input data"
const string apiKey = "[Key]"; // Replace this with the API key for the web service
Debug.Log(jsonString);
WWWForm form = new WWWForm();
form.AddField("name", "value");
string url = "https://europewest.services.azureml.net/workspaces/[id]/services/[id]/execute?api-version=2.0&details=true";
var headers = new Dictionary<string, string>();

var encoding = new System.Text.UTF8Encoding();
headers.Add("Authorization", "Bearer " + apiKey);

headers.Add("Content-Type", "application/json");
WWW www = new WWW(url, encoding.GetBytes(jsonString), headers);
result = www;

JSON deserialization problem

Next problem was to desterilize the respons data. I found out the hard way that multi dementional JSON objects with data types other than ints and strings wasn’t supported. So by using some ugly regex I was able to fetch the data from the web request result.

var t2 = Regex.Replace(result.text, ".*Values.*\["", String.Empty);
var t3 = Regex.Replace(t2, ""].*", String.Empty);

float t4 = float.Parse(t3);

Request time problem

The web request takes too long to do every frame so the solution I used is to run it as constantly and just send the response to the steering of the car. It didn’t work at all, it was too slow. Then I moved the experiment to a datacenter geographically closer to me and happiness filled my body.

It’s not perfect but it works. In the video above you can see the car flash in red. That’s every time a web request is done. You get the point, steering is not applied every frame so we went from a blind driver to a drunk driver.

To make the car drive like this I have recorded 269683 lines of measurements. approx 36 Mb of data and 1h drive time. The car also runs at a limited constant speed but is learned in various speeds. More learning data will improve the driving and I hope it will be flawless in the next version.

In this example Azure is only predicting the steering value. In the next step I will predict acceleration and breaking values.

If I was able to create this, imagine what you can make from you data.

Stay tunes for the next episode. @MeapaX