Building your first neural network self driving car in Python
July 31, 2016
Posted by on
1. Get RC Car
2. Learn to drive it
3. Take apart car to see controllers and wireless controller
4. Soldering Iron and Multimeter to determine positive and negative and circuits firing
Testing – Link Mac to Arduino to Wireless Controller
5. Need Arduino board and cable
6. Install software and load Arduino program onto board
7. Install pygame and serial
8. python carDriving.py to test soldering and driving by keyboard
Testing – Capturing image data for training dataset
On the first iteration of the physical devices, I mounted the disassembled Logitech C270/Raspberry Pi on the car with a coat hanger that I chopped up and modified to hold the camera. I pointed it down so it could see the hood and some of the “road”. The webcam captures video frames of the road ahead at ~24 fps.
I send the captured stream across the wifi network back to my MacBookPro using python server implementation using basic sockets.
On my MacBookPro laptop computer, I run another client python program to connect to Raspberry Pi using basic sockets. I take the stream color stream 320×240 and down sample and grayscale video frames for preprocessing into a numpy matrix.
Wirelessly stream video and capture using opencv2 and slice into jpeg, preprocess and reshape numpy and feed array into with key press data as label.
Testing – First Build of Car with components
Testing – Convert 240×240 into greyscale
Take 2 : Using PiCamera and stream images to Laptop
Take 2 -Load new Arduino Sketch and change PINS
Take 2 – Stream Data from Pi to Laptop
Train Neural Network with train.pkl
Converted numpy data to pickle and then use it for training python simple 3 layer neural network. 65536 neurons for input layer, 1000 neurons for hidden layer and 4 output neurons. Forward, None, Left, and Right.
Check predictions of Neural Network
Test driving car via key press
Test driving car via prediction
Test trained Neural Network with live camera data…enjoy!
- Deep Learning
- Computer Vision
- Vehicle Dynamics
- Mapping (SLAM)
- Sensors & Fusion
- Safety Systems and Ethics
ReportStyleDocumentaton build RC custom