BigSnarf blog

Infosec FTW

Category Archives: Tools

Robots learning from humans

mvTCN

We propose a self-supervised approach for learning representations entirely from unlabeled videos recorded from multiple viewpoints. This is particularly relevant to robotic imitation learning, which requires a viewpoint-invariant understanding of the relationships between humans and their environment, including object interactions, attributes and body pose. We train our representations using a triplet loss, where multiple simultaneous viewpoints of the same observation are attracted in the embedding space, while being repelled from temporal neighbors which are often visually similar but functionally different. This signal encourages our model to discover attributes that do not change across viewpoint, but do change across time, while ignoring nuisance variables such as occlusions, motion blur, lighting and background. Our experiments demonstrate that such a representation even acquires some degree of invariance to object instance. We demonstrate that our model can correctly identify corresponding steps in complex object interactions, such as pouring, across different videos with different instances. We also show what are, to the best of our knowledge, the first self-supervised results for end-to-end imitation learning of human motions by a real robot.

https://sermanet.github.io/tcn/

Click to access 1704.06888.pdf

https://sermanet.github.io/imitation/

https://research.googleblog.com/2017/07/the-google-brain-residency-program-one.html

Self driving car Operating System – SDCOS

Pose detection for better pedestrian detection

Synthetic data for simulation

Compression techniques for deep learning

Left to train the Right – Stereo Camera – Monocular inference

Ego Motion from Video

Predicting deep into the future with segmentation

Screen Shot 2017-03-24 at 9.04.14 AM

Click to access 1703.07684.pdf

https://github.com/MarvinTeichmann/MultiNet

Click to access 1612.07695.pdf

https://github.com/e-lab/ENet-training

Self driving car LIDAR and camera download

Screen Shot 2017-03-23 at 11.06.36 PM

GOTURN tracking

ezgif-2-95cb54bb47ezgif-2-0f24362942

Hand annotating first image on Kitti Sequence 21

Screen Shot 2017-03-17 at 4.06.52 PM

http://davheld.github.io/GOTURN/GOTURN.html

https://github.com/davheld/GOTURN

Click to access GOTURN.pdf

https://github.com/Auron-X/GOTURN_Training_Toolkit

https://handong1587.github.io/deep_learning/2015/10/09/tracking.html

https://github.com/Guanghan/ROLO

http://guanghan.info/projects/ROLO/

Click to access 1703.01289.pdf

##Convolutional Neural Networks for Visual Tracking
The main goal of our GSoC project was an implementation of GOTURN tracker in OpenCV library.

Original paper is located here: http://davheld.github.io/GOTURN/GOTURN.pdf

Short GOTURN tracker overview: http://davheld.github.io/GOTURN/GOTURN.html

Our implementation of GOTURN comes in two parts:

  1. GOTURN tracker imlementation in OpenCV Tracking API. Pretrained GOTURN model is required.
  2. GOTURN tracker training toolkit, for pretraining GOTURN with custom parameters.

Also we uploaded a pretrained (according to the original paper) GOTURN model to opencv_extra repository, so anyone can use it straightly without time-consuming pretraining procedure.

##OpenMax layer implementation for tinyDNN
OpenMax layer implementation for tinyDNN has started as additional activity during the last week of GSoC. Current implementation includes only a utility function for Mean Activation Vector with the following MR EVT calibration using Weibull fiting in libMR.
For complete OpenMax implementation two are still need to be done:

  1. Function for calculation Weibull fiting for all classes in particular dataset
  2. Actual OpenMax layer implementation (now there is empty OpenMax layer skeleton) to calculate a final OpenMax score

##List of all commits during GSoC 2016:

##Convolutional Neural Networks for Visual Tracking
The main goal of our GSoC project was an implementation of GOTURN tracker in OpenCV library.

Original paper is located here: http://davheld.github.io/GOTURN/GOTURN.pdf

Short GOTURN tracker overview: http://davheld.github.io/GOTURN/GOTURN.html

Our implementation of GOTURN comes in two parts:

  1. GOTURN tracker imlementation in OpenCV Tracking API. Pretrained GOTURN model is required.
  2. GOTURN tracker training toolkit, for pretraining GOTURN with custom parameters.

Also we uploaded a pretrained (according to the original paper) GOTURN model to opencv_extra repository, so anyone can use it straightly without time-consuming pretraining procedure.

##OpenMax layer implementation for tinyDNN
OpenMax layer implementation for tinyDNN has started as additional activity during the last week of GSoC. Current implementation includes only a utility function for Mean Activation Vector with the following MR EVT calibration using Weibull fiting in libMR.
For complete OpenMax implementation two are still need to be done:

  1. Function for calculation Weibull fiting for all classes in particular dataset
  2. Actual OpenMax layer implementation (now there is empty OpenMax layer skeleton) to calculate a final OpenMax score

##List of all commits during GSoC 2016: