BigSnarf blog

Infosec FTW

Category Archives: Tools

Robots learning from humans


We propose a self-supervised approach for learning representations entirely from unlabeled videos recorded from multiple viewpoints. This is particularly relevant to robotic imitation learning, which requires a viewpoint-invariant understanding of the relationships between humans and their environment, including object interactions, attributes and body pose. We train our representations using a triplet loss, where multiple simultaneous viewpoints of the same observation are attracted in the embedding space, while being repelled from temporal neighbors which are often visually similar but functionally different. This signal encourages our model to discover attributes that do not change across viewpoint, but do change across time, while ignoring nuisance variables such as occlusions, motion blur, lighting and background. Our experiments demonstrate that such a representation even acquires some degree of invariance to object instance. We demonstrate that our model can correctly identify corresponding steps in complex object interactions, such as pouring, across different videos with different instances. We also show what are, to the best of our knowledge, the first self-supervised results for end-to-end imitation learning of human motions by a real robot.

Self driving car Operating System – SDCOS

Screen Shot 2017-07-05 at 8.14.44 AMScreen Shot 2017-07-05 at 8.14.22 AM.png

Pose detection for better pedestrian detection

Synthetic data for simulation

Compression techniques for deep learning

Left to train the Right – Stereo Camera – Monocular inference

Ego Motion from Video

Predicting deep into the future with segmentation

Self driving car LIDAR and camera download

Screen Shot 2017-03-23 at 11.06.36 PM

GOTURN tracking