BigSnarf blog

Infosec FTW

Apache Spark Streaming and AWS Kinesis integration in version 1.1.0

Me learning Scala, Akka, Spark, JVM, Intellij, sbt, Java, maven, ant, Functional programming – Meat grinding knob turner

Screen Shot 2014-09-29 at 11.58.29 AM

I thought this photo was really cool, then I found a better one



OpenSOC Machine Learning

Screen Shot 2014-09-26 at 3.26.58 PM Screen Shot 2014-09-26 at 3.23.34 PM Screen Shot 2014-09-26 at 3.22.05 PM Screen Shot 2014-09-26 at 3.20.43 PM

Self Hosted Maven repo on S3

s3cmd mb s3://www.example.mavenrepo
s3cmd ws-create s3://www.example.mavenrepo
mkdir com
cd com
mkdir amazonaws
cd amazonaws/
mkdir amazon-kinesis-connector
cd amazon-kinesis-connector
mkdir 1.0.0
cd com/amazonaws/amazon-kinesis-connector/1.0.0/
s3cmd -P sync /home/ubuntu/com/amazonaws/amazon-kinesis-connector/1.0.0 s3://www.example.mavenrepo/snapshots/com/amazonaws/amazon-kinesis-connector/1.0.0/

"AWS Snapshots" at ""


Finding attackers with Neo4J

Monitoring JVM


Scala REPL in Notebook

Screen Shot 2014-08-26 at 10.29.31 PM

Scalaz in the REPL

First run sbt:


Then issue these commands:

set scalaVersion := “2.10.0”
set libraryDependencies += “org.scalaz” %% “scalaz-core” % “7.0.0”
set initialCommands += “import scalaz._, Scalaz._”
session save

Algebird Monoids for IP Addresses and counts in Scala

import com.twitter.algebird.Operators._

case class IPRecord(val ipAddress: String, val number: Int) extends Ordered[IPRecord] {
 def compare(that: IPRecord): Int = {
   val c = this.number - that.number
   if (c == 0) this.ipAddress.compareTo(that.ipAddress) else c

val oneOneOneOne = IPRecord("", 67391)
val twoTwoTwoTwo = IPRecord("", 48013573)
val threeThreeThreeThree = IPRecord("", 6470)
val fourFourFourFour = IPRecord("", 731)

val topIPAddress: Max[IPRecord] = Max(oneOneOneOne) + Max(twoTwoTwoTwo) + Max(threeThreeThreeThree) + Max(fourFourFourFour)
assert(topIPAddress.get == twoTwoTwoTwo)

Get every new post delivered to your Inbox.

Join 46 other followers