BigSnarf blog

Infosec FTW

Category Archives: Tools

Unsupervised Reinforcement Learning



Click to access 1611.05397.pdf

Data Pipeline visualized


Image Retrieval Using Deep Learning


We propose a novel approach for instance-level image retrieval. It produces a global and compact fixed-length representation for each image by aggregating many region-wise descriptors. In contrast to previous works employing pre-trained deep networks as a black box to produce features, our method leverages a deep architecture trained for the specific task of image retrieval. Our contribution is twofold: (i) we leverage a ranking framework to learn convolution and projection weights that are used to build the region features; and (ii) we employ a region proposal network to learn which regions should be pooled to form the final global descriptor. We show that using clean training data is key to the success of our approach. To that aim, we use a large scale but noisy landmark dataset and develop an automatic cleaning approach. The proposed architecture produces a global image representation in a single forward pass. Our approach significantly outperforms previous approaches based on global descriptors on standard datasets. It even surpasses most prior works based on costly local descriptor indexing and spatial verification1

Click to access 1604.01325.pdf


Screen Shot 2018-05-21 at 12.42.48 PMScreen Shot 2018-05-21 at 12.42.41 PMScreen Shot 2018-05-21 at 12.42.35 PMScreen Shot 2018-05-21 at 12.42.28 PM5

Clustering photos for labels

  1. Histograms of RGB
  2. KMeans Histograms
  3. Autoencoder KMeans
  4. Unsupervised Deep Embedding for Clustering Analysis (DEC)

I have 50000 photos that I would like to label for training a classifier. I guess I could represent each image by raw pixels or RGB values but how do I divide them into K groups in terms of inherent latent semantics? Solutions 1, 2 and 3.

The traditional way, you first extract feature vectors according to domain-specific knowledge and then use a clustering algorithm on the extracted features.

My colleague said I had to use deep learning so I researched it and found DEC. A unified framework which can directly cluster images with linear performance. This new category of clustering algorithms using Deep Learning is typically called Deep Clustering.

From the paper:

Clustering is central to many data-driven application domains and has been studied extensively in terms of distance functions and grouping algorithms. Relatively little work has focused on learning representations for clustering. In this paper, we propose Deep Embedded Clustering (DEC), a method that simultaneously learns feature representations and cluster assignments using deep neural networks. DEC learns a mapping from the data space to a lower-dimensional feature space in which it iteratively optimizes a clustering objective. Our experimental evaluations on image and text corpora show significant improvement over state-of-the-art methods.

Click to access 1511.06335.pdf

Screen Shot 2018-05-15 at 10.06.49 PM

Screen Shot 2018-05-15 at 10.18.34 PM

Click to access ICONIP17-DCEC.pdf

SICK LiDAR for xmas fun

Denoising AutoEncoder

Screen Shot 2017-12-13 at 11.51.27 PMScreen Shot 2017-12-13 at 11.51.20 PMScreen Shot 2017-12-11 at 3.57.33 PM

Screen Shot 2017-12-08 at 3.49.33 PM.png




import os
import torch
from torch import nn
from torch.autograd import Variable
from import DataLoader
from torchvision import transforms
from torchvision.datasets import MNIST
from torchvision.utils import save_image
if not os.path.exists('./mlp_img'):
def to_img(x):
x = x.view(x.size(0), 1, 28, 28)
return x
num_epochs = 20
batch_size = 128
learning_rate = 1e-3
def plot_sample_img(img, name):
img = img.view(1, 28, 28)
save_image(img, './sample_{}.png'.format(name))
def min_max_normalization(tensor, min_value, max_value):
min_tensor = tensor.min()
tensor = (tensor min_tensor)
max_tensor = tensor.max()
tensor = tensor / max_tensor
tensor = tensor * (max_value min_value) + min_value
return tensor
def tensor_round(tensor):
return torch.round(tensor)
img_transform = transforms.Compose([
transforms.Lambda(lambda tensor:min_max_normalization(tensor, 0, 1)),
transforms.Lambda(lambda tensor:tensor_round(tensor))
dataset = MNIST('./data', transform=img_transform, download=True)
dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=True)
class autoencoder(nn.Module):
def __init__(self):
super(autoencoder, self).__init__()
self.encoder = nn.Sequential(
nn.Linear(28 * 28, 256),
nn.Linear(256, 64),
self.decoder = nn.Sequential(
nn.Linear(64, 256),
nn.Linear(256, 28 * 28),
def forward(self, x):
x = self.encoder(x)
x = self.decoder(x)
return x
model = autoencoder().cuda()
criterion = nn.BCELoss()
optimizer = torch.optim.Adam(
model.parameters(), lr=learning_rate, weight_decay=1e-5)
for epoch in range(num_epochs):
for data in dataloader:
img, _ = data
img = img.view(img.size(0), 1)
img = Variable(img).cuda()
# ===================forward=====================
output = model(img)
loss = criterion(output, img)
MSE_loss = nn.MSELoss()(output, img)
# ===================backward====================
# ===================log========================
print('epoch [{}/{}], loss:{:.4f}, MSE_loss:{:.4f}'
.format(epoch + 1, num_epochs,[0],[0]))
if epoch % 10 == 0:
x = to_img(img.cpu().data)
x_hat = to_img(output.cpu().data)
save_image(x, './mlp_img/x_{}.png'.format(epoch))
save_image(x_hat, './mlp_img/x_hat_{}.png'.format(epoch)), './sim_autoencoder.pth')

  • model that predicts  – “autoencoder” as a feature generator
  • model that predicts  – “incidence angle” as a feature generator

Screen Shot 2017-12-09 at 1.16.48 PM

List and Dicts to Pandas DF

Vision stuff TODO reading

NLP Neural Networks