BigSnarf blog

Infosec FTW

Apache Spark Streaming and AWS Kinesis integration in version 1.1.0

Me learning Scala, Akka, Spark, JVM, Intellij, sbt, Java, maven, ant, Functional programming – Meat grinding knob turner

Screen Shot 2014-09-29 at 11.58.29 AM

I thought this photo was really cool, then I found a better one

8812567121_25d8d9ae53_o

real-mountain-biking-225395

OpenSOC Machine Learning

Screen Shot 2014-09-26 at 3.26.58 PM Screen Shot 2014-09-26 at 3.23.34 PM Screen Shot 2014-09-26 at 3.22.05 PM Screen Shot 2014-09-26 at 3.20.43 PM

Self Hosted Maven repo on S3

s3cmd mb s3://www.example.mavenrepo
s3cmd ws-create s3://www.example.mavenrepo
mkdir com
cd com
mkdir amazonaws
cd amazonaws/
mkdir amazon-kinesis-connector
cd amazon-kinesis-connector
mkdir 1.0.0
cd com/amazonaws/amazon-kinesis-connector/1.0.0/
s3cmd -P sync /home/ubuntu/com/amazonaws/amazon-kinesis-connector/1.0.0 s3://www.example.mavenrepo/snapshots/com/amazonaws/amazon-kinesis-connector/1.0.0/



"AWS Snapshots" at "http://www.example.mavenrepo.s3.amazonaws.com/snapshots"

			

Finding attackers with Neo4J

Monitoring JVM

BuS2MHPCMAAm-fk

Scala REPL in Notebook

Screen Shot 2014-08-26 at 10.29.31 PM

Scalaz in the REPL

First run sbt:

sbt

Then issue these commands:

set scalaVersion := “2.10.0”
set libraryDependencies += “org.scalaz” %% “scalaz-core” % “7.0.0”
set initialCommands += “import scalaz._, Scalaz._”
session save
console

Algebird Monoids for IP Addresses and counts in Scala

import com.twitter.algebird.Operators._


case class IPRecord(val ipAddress: String, val number: Int) extends Ordered[IPRecord] {
 def compare(that: IPRecord): Int = {
   val c = this.number - that.number
   if (c == 0) this.ipAddress.compareTo(that.ipAddress) else c
 }
}


val oneOneOneOne = IPRecord("1.1.1.1", 67391)
val twoTwoTwoTwo = IPRecord("2.2.2.2", 48013573)
val threeThreeThreeThree = IPRecord("3.3.3.3", 6470)
val fourFourFourFour = IPRecord("4.4.4.4", 731)

val topIPAddress: Max[IPRecord] = Max(oneOneOneOne) + Max(twoTwoTwoTwo) + Max(threeThreeThreeThree) + Max(fourFourFourFour)
assert(topIPAddress.get == twoTwoTwoTwo)
Follow

Get every new post delivered to your Inbox.

Join 46 other followers