Get in touch with us!

CADYC (Can Azure Drive Your Car) Part1

Can Azure Drive Your Car – I got the idea after making my GA code in PowerShell and then saw George Hotz self driving car
( ). I thought, I might be able to make mine own self driving car using Azure Machine Learning. So I started thinking and came up with the idea to create a top-down car game which can be controlled by keyboard and web requests. Since I did not know anything about game programming, I turned to my friend Andreas who helped me get started.

clip_image001*My favorite car in GTA 1

The first problem control the car.

I wanted both stepless throttle and steering. To get a smooth steering which also could be predicted by Azure, a binary on-off steering would definitely fail.

So by using a steering interval from -3 to 3 and always slowly resetting the value to 0. I get a smooth return of the wheels. An as long as a press left or right key I will add or subtract a value to the steering value.


The next thing was to apply this force differently depending on the current velocity of the car. Because I don’t want the car to be able to turn when not it’s moving and also not be able to turn as quickly at a greater velocity.

if ((Speed + 16.1f) > 0)
 ZeroSteer = (1 / (Speed +16.1f));
 } else
 ZeroSteer = 0 ;
transform.rotation *= Quaternion.AngleAxis(steering * 8f * (curspeed.magnitude * 0.5f) * ZeroSteer, Vector3.forward);

The variable Speed is not my throttle nor the velocity of the car. The speed is calculated a similar way as steering to make acceleration smooth. The Speed is calculated by subtracting the Break Power from the Acceleration power. Both are calculated so that you have a greater acceleration and deceleration at a lower velocity that at a higher. The break power is two times more effective to get it more realistic, problem to solve here is that the car accelerate faster when moving backwards, but that’s a later problem.

BreakPower = Mathf.Sqrt(Mathf.Pow((((BreakLvl / 100) / (forwardVelocity.magnitude + 1)) / 0.1f), 2));
AccPower = Mathf.Sqrt(Mathf.Pow((((AccLvl / 100) / (forwardVelocity.magnitude + 1))), 2));
Speed = (AccPower - BreakPower);

Last but not least I wanted to create a drift feeling in the car. And found this online. The constant 0.931f is just a value which I found suitable for my project. This can be a variable to simulate different frictions on different surfaces.

forwardVelocity = transform.up * Vector2.Dot(rig2D.velocity, transform.up);
rightVelocity = transform.right * Vector2.Dot(rig2D.velocity, transform.right);
rig2D.velocity = forwardVelocity + rightVelocity * 0.931f;

Another problem is the fact that this is a top-down game and the 2d environment is a platform game. So gravity doesn’t pull the car in the right direction nor does friction affect the car object because it’s in midair.

This is solved by constantly applying a fake friction and ignoring the gravity on the object.

rig2D.angularVelocity *= 0.9f;

And finally we apply the force to the object.

rig2D.AddForce(transform.up * Speed * 80.8f); 

The map

The map is basically just a sprite I found on google. In version 2 this will be generated.


By adding polygon colliders to the map sprite I can now trigger on edges of the road.


At first I used short raycasts as shown in the picture below, to measure the distance from the car to the edges of the road.


This resulted in the driver to be half blind. And the result wasn’t satisfying, so I added some more raycasts and change the length of them to always hit a collider.



Saving the data.

In azure ML I’m using a supervised learning method which means that I need to learn Azure to predict results from a dataset. Like Geohotz say in the video. We don’t learn how to drive by complex rules, we learn to drive by copy how others drive.

So to learn Azure ML to drive I have to drive the car myself. While I’m driving I also recording all the data from the raycasts and the car’s velocity and throttle and this give me a dataset to train the ML.

This is an example of the data. The data is saved for each frame and then saved to a csv file.


Azure Machine Learning

I’m using tree types of learning algorithms to find the best suitable. I’m using three types of learning algorithms to find the best suitable. In my case the “Boosted Decision Tree Regression” gave me the best result

Models used

Boosted Decision Tree Regression – best driving preformance

Nerual Network Regression – lowest error in the predicted result, but made the car began to move like a snake.

Linear regression – good but not as good as the Decision Tree.

I tested a lot more algorithms but this three is the ones I find the best.

So how do you know which one to use? Start by using the cheat sheet and then test, test, test.


Here is an example of a visitation of one of the Decision Trees in my trained model. You don’t have to understand It and that’s the beauty of Azure ML.


After creating the experiment, I added a filter to the output to only send the predicted steering value.


The final predicted experiment looks like this.


Web service problem

Unity is using Mono .net 2.0 so using the nice C# example code from Azure web service didn’t work as it was, so I had to rewrite it.

String jsonString = "JSON input data"
const string apiKey = "[Key]"; // Replace this with the API key for the web service
WWWForm form = new WWWForm();
form.AddField("name", "value");
string url = "[id]/services/[id]/execute?api-version=2.0&details=true";
var headers = new Dictionary<string, string>();

var encoding = new System.Text.UTF8Encoding();
headers.Add("Authorization", "Bearer " + apiKey);

headers.Add("Content-Type", "application/json");
WWW www = new WWW(url, encoding.GetBytes(jsonString), headers);
result = www;

JSON deserialization problem

Next problem was to desterilize the respons data. I found out the hard way that multi dementional JSON objects with data types other than ints and strings wasn’t supported. So by using some ugly regex I was able to fetch the data from the web request result.

var t2 = Regex.Replace(result.text, ".*Values.*\["", String.Empty);
var t3 = Regex.Replace(t2, ""].*", String.Empty);

float t4 = float.Parse(t3);

Request time problem

The web request takes too long to do every frame so the solution I used is to run it as constantly and just send the response to the steering of the car. It didn’t work at all, it was too slow. Then I moved the experiment to a datacenter geographically closer to me and happiness filled my body.

It’s not perfect but it works. In the video above you can see the car flash in red. That’s every time a web request is done. You get the point, steering is not applied every frame so we went from a blind driver to a drunk driver.

To make the car drive like this I have recorded 269683 lines of measurements. approx 36 Mb of data and 1h drive time. The car also runs at a limited constant speed but is learned in various speeds. More learning data will improve the driving and I hope it will be flawless in the next version.

In this example Azure is only predicting the steering value. In the next step I will predict acceleration and breaking values.

If I was able to create this, imagine what you can make from you data.

Stay tunes for the next episode. @MeapaX


  1. Peter Kellner

    video is missing. was hoping to see demo.

    • Jon Jander

      Hi Peter the video works now.


Submit a Comment

Your email address will not be published. Required fields are marked *

Starkare tillsammans

Bolag inom både publik och offentlig sektor efterfrågar en alltmer kostnadseffektiv, platsoberoende och säker digital arbetsplats. Därför går nu Altitude 365 och Uclarity samman och bildar ett gemensamt specialistbolag.
Fortsätt på Altitude 365Kolla in Exobe

Altitude 365 + Uclarity – Årets Modern Work Partner!

Vinnaren är ett bra exempel på hur en “Born in the Cloud” ständigt förbättrar sitt erbjudande, arbetar nära och proaktivt med Microsoft för att hjälpa kunderna på deras digitaliseringsresa. Plus att vi på Microsoft verkligen ser fram mot den nya bolags-konstellationen (Altitude 365 + Uclarity) för ett “Starkare Tillsammans”.

Uclarity och Altitude 365 - Starkare tillsammans

Uclarity är specialister på digitala möten, telefoni, kontaktcenter och digitalt arbetssätt. Altitude 365 är specialister på säkerhet, mobilitet och hur bolag kan optimera resan till Microsoft365. Nu gör vi gemensam sak och bildar bolag tillsammans.

– Pandemin har tydliggjort behoven av en modern digital arbetsplats och vi har diskuterat ett samgående med Altitude 365 under en längre tid. Våra kunder har behov av specialistkompetens och tillsammans blir vi en ledande specialist inom Digital Workplace-området, säger Niklas Olsson Hellström, VD Uclarity AB.

Tommy Clark, Partner, Altitude 365, kommenterar:
– Inget bolag köper det andra utan båda bolagen får lika stora delar i det nya bolaget. Vår ledstjärna är att vi blir starkare tillsammans och att vi kan hjälpa våra kunder under hela deras resa.
Målet med sammanslagningen är att kunna hjälpa kunder med både teknik och effektiva arbetssätt.

– Det är då våra kunder får önskad effekt av sin investering i den digitala arbetsplatsen, säger Niklas Olsson Hellström.

Båda bolagen har svenska och internationella kunder från både privat och offentlig sektor. Sammanslagningen resulterar i en organisation på 50+ anställda baserade i Stockholm, Örebro och Göteborg.

För frågor, vänligen kontakta;
Tommy Clarke, Partner, Altitude 365 AB, 0703-593854,
Niklas Olsson Hellström, VD, Uclarity AB, 0734-198016,

Fortsätt på Altitude 365Kolla in Exobe