Building your first neural network self driving car in Python

Leave a comment Posted by Security Dude on July 31, 2016

1. Get RC Car

2. Learn to drive it

3. Take apart car to see controllers and wireless controller

4. Soldering Iron and Multimeter to determine positive and negative and circuits firing

Testing – Link Mac to Arduino to Wireless Controller

5. Need Arduino board and cable

6. Install software and load Arduino program onto board

7. Install pygame and serial

8. python carDriving.py to test soldering and driving by keyboard

Screen Shot 2016-07-31 at 10.28.15 AM

Testing – Capturing image data for training dataset

IMG_0319-1024x683

On the first iteration of the physical devices, I mounted the disassembled Logitech C270/Raspberry Pi on the car with a coat hanger that I chopped up and modified to hold the camera. I pointed it down so it could see the hood and some of the “road”. The webcam captures video frames of the road ahead at ~24 fps.

I send the captured stream across the wifi network back to my MacBookPro using python server implementation using basic sockets.

On my MacBookPro laptop computer, I run another client python program to connect to Raspberry Pi using basic sockets. I take the stream color stream 320×240 and down sample and grayscale video frames for preprocessing into a numpy matrix.

Wirelessly stream video and capture using opencv2 and slice into jpeg, preprocess and reshape numpy and feed array into with key press data as label.

Testing – First Build of Car with components

Testing – Convert 240×240 into greyscale

57600 input neurons

Take 2 : Using PiCamera and stream images to Laptop

Take 2 -Load new Arduino Sketch and change PINS

Take 2 – Stream Data from Pi to Laptop

Train Neural Network with train.pkl

Converted numpy data to pickle and then use it for training python simple 3 layer neural network. 65536 neurons for input layer, 1000 neurons for hidden layer and 4 output neurons. Forward, None, Left, and Right.

BigSnarf blog