definition programmer

Advertisements

Steps to deep learning

1. Clean Dataset
2. Choose an appropriate model
3. Choose an activation function
4. Choose a cost function
5. Iterate to Optimize

GPS RTK

Obsessed with my Kaggle score – it’s just a number

“a state in which someone thinks about something constantly or frequently especially in a way that is not normal”

Three models grid search on 2 ubuntu boxes with a GTX1070 each

First model I used was a ConvNet (CNN) with multiple Recurrent (RNN) layers on top.

Apparently, this is a common structure NN in speech. The CCN–>RNN tries to exploit temporal relationships within the data. In addition with the 2D structure of the spectrograms.

Imagine for each vertical line in the spectrogram as a timestep, the RNN is an attempt to  model the relationships between those timesteps.

The second model, I ripped out the RNN and replaced it with LSTM.  The third model, I tried to increase the RNN layers.

The fourth model combines all the electrodes in batches of 64. Fifth model is straight MLP in Tensorflow and Sixth model is straight CNN Tensorflow.

AWS p2.xlarge is a pain, just to try out a pretrained VGG model need 12gb GPU

Bad Data – Leaderboard hacking

http://blog.kaggle.com/2017/01/10/seizure-prediction-competition-3rd-place-winners-interview-gareth-jones/

NHTSA Federal HAV Policy

http://www.nhtsa.gov/nhtsa/av/pdf/Federal_Automated_Vehicles_Policy.pdf

Driving a simulator

View at Medium.com

http://cs231n.stanford.edu/reports2016/112_Report.pdf

http://arxiv.org/pdf/1608.01230v1.pdf

http://download.visinf.tu-darmstadt.de/data/from_games/

http://motherboard.vice.com/read/video-games-artificial-intelligence-realistic-teach-ai-what-the-world-looks-like

http://cs231n.stanford.edu/reports2016/123_Report.pdf

http://cs224d.stanford.edu/lectures/CS224d-Lecture7.pdf

Deep Learning in a Nutshell: Reinforcement Learning

Fusing Sensors to Make a Smarter Driving Experience

http://www.cv-foundation.org/openaccess/content_cvpr_2016_workshops/w3/papers/Gurghian_DeepLanes_End-To-End_Lane_CVPR_2016_paper.pdf

https://github.com/commaai/research/blob/master/DriveSim.md

Steering by Flashcards

“To remove a bias towards driving straight the training data includes a higher proportion of frames that represent road curves”

 “to build a CNN to do lane following we only select data where the driver was staying in a lane and discard the rest. We then sample that video at 10 FPS.”

CNN for:

• Object-detection
• Segmentation
• Human pose estimation
• Video classification
• Object tracking
• Superresolution

 

Image Links:

1. https://arxiv.org/abs/1604.07316
2. http://www.cv-foundation.org/openaccess/content_cvpr_2016_workshops/w3/papers/Gurghian_DeepLanes_End-To-End_Lane_CVPR_2016_paper.pdf
3. https://www.ptgrey.com/case-study/id/10846
4. http://net-scale.com/doc/net-scale-dave-report.pdf
5. http://repository.cmu.edu/cgi/viewcontent.cgi?article=2874&context=compsci
6. http://www.cv-foundation.org/openaccess/content_cvpr_2016_workshops/w3/papers/Gurghian_DeepLanes_End-To-End_Lane_CVPR_2016_paper.pdf
7. https://drive.google.com/a/bench.co/file/d/0B9raQzOpizn1TkRIa241ZnBEcjQ/view
8. https://culurciello.github.io/tech/2016/06/04/nets.html
9. https://github.com/commaai/research/blob/master/SelfSteering.md
10. https://research.googleblog.com/2016/08/improving-inception-and-image.html
11. https://research.googleblog.com/2016/08/tf-slim-high-level-library-to-define.html
12. https://github.com/tensorflow/models/blob/master/slim/deployment/model_deploy.py
13. http://download.visinf.tu-darmstadt.de/data/from_games/
14. https://github.com/tensorflow/models/tree/master/slim#fine-tuning-a-model-from-an-existing-checkpoint
15. https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/slim
16. https://github.com/tensorflow/models/blob/master/slim/README.md
17. https://github.com/tensorflow/models/blob/master/slim/slim_walkthough.ipynb
18. http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43022.pdf
19. http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43442.pdf
20. http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44903.pdf
21. http://arxiv.org/pdf/1409.1556.pdf
22. https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
23. https://arxiv.org/abs/1512.03385

Resources:

1. https://github.com/rwightman/tensorflow-litterbox
2. https://github.com/tensorflow/models
3. https://github.com/tensorflow/models/tree/master/slim
4. https://github.com/facebook/fb.resnet.torch
5. https://github.com/rbgirshick/py-faster-rcnn/
6. https://github.com/KaimingHe/deep-residual-networks
7. https://github.com/daijifeng001/mnc
8. https://github.com/facebookresearch/multipathnet
9. https://github.com/jcjohnson/neural-style
10. https://www.quora.com/How-does-deep-residual-learning-work
11. http://videolectures.net/deeplearning2016_montreal/
12. http://academictorrents.com/details/743c16a18756557a67478a7570baf24a59f9cda6
13. http://cs231n.github.io/
14. http://www.deeplearningbook.org/
15. http://cilvr.nyu.edu/doku.php?id=deeplearning:slides:start
16. http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html
17. https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/