BigSnarf blog

Infosec FTW

Category Archives: Tools

Security ChatBots

Identify compromised logins

A collection of data models for real-time analysis, behaviour analysis, and artificial intelligence (AI) to quickly predict between valid and malicious user activity

CloudTrail provides an audit trail of the API activity in your AWS Environment. In order to maintain compliance with one of the many auditing standards, you need to implement continuous monitoring and demonstrate the ability to provide evidence when needed.

wolf-in-sheeps-clothing-1

 

Use behavioural patterns and build an identity profile for each user

  1. Sign in from unknown locations algorithm – Approximations and comparisons for each login against the population logins reachability score. Kinda like graph random walks:

2.  Impossible travel or logins too close together algorithm – Create likelihood of logins comparison of two places. Another was is geo-distance algos:

  • Login location 1 east coast
  • Login location 2 west coast
  • Create high, medium, low and output prediction

Screen Shot 2016-05-11 at 9.27.11 AM

3. Credential leak algorithm – check if AWS leaked

4. Sign-in anonymous IP or Tor algorithm – check if logins match ip address blacklist

5. Malware alerts correlated to antivirus or phishing campaigns opening algo – check if person machine compromised blacklist or blacklist dns lately

Screen Shot 2016-05-11 at 5.03.27 PM

6. Rolling 47 day window of ip address and location history using InfluxDB and CQ

7. Rolling 47 day window of Browser and OS history simple rolling window

8. Keep counters history on logins and failures for each user -tdigest or list of counts per user

9. Keep IP address history for each user and correlate different user logins from untrusted sources. Alert on flag on two logins, fail on three logins from untrusted

10. Locations time browser os for password resets – correlated to

11. Was this you email/slack message? Using feedback loop with security slackbot in validating admin and superuser logins to reduce 2nd factor flags/emails and updating models with data to create trusted user data profile

 

Small set labelled data – Bayesian Convolutional Neural Networks

Tensorflow Facial Key Points

Screen Shot 2016-05-06 at 12.02.36 AM

Anomaly Detection Python T-Digest

https://www.usenix.org/system/files/conference/hotcloud14/hotcloud14-vallis.pdf

Screen Shot 2016-05-01 at 12.16.13 AM

Parameterized anomaly detection settings

 

Event correlation is a technique for making sense of a large number of events and pinpointing the few events that are really important in that mass of information. This is accomplished by looking for and analyzing relationships between events.

Cloudtrail Dashboards

awsServiceEventTrackingScreenshot

Are Overview Reports helpful?

  • Top 10 Events
  • Top 10 Locations
  • Top 10 Access Keys
  • Top 10 Services
  • Top 10 Ip Addresses
  • Unauthorized Access

Activity reports helpful?

  • List of Instances
  • Errors
  • Instance Activities

Audit reports helpful?

  • List Users
  • List Keys
  • Access Keys Used
  • Locations Used
  • User activities
  • User patterns

Machine Learning and Data Mining helpful?

Get Moar Data or Tough Luck

ml_map.png

scikit-learn.org/stable/_static/ml_map.png

TensorFlow and Kaggle

Screen Shot 2016-04-17 at 12.02.46 AM

Screen Shot 2016-04-17 at 12.04.35 AM

TensorFlow explained

Anomaly detection with Bayesian networks

Anomaly detection, also known as outlier detection, is the process of identifying data which is unusual. I have been using basic python Markov Chains or more complex python MCMC.

https://www.quantstart.com/articles/Markov-Chain-Monte-Carlo-for-Bayesian-Inference-The-Metropolis-Algorithm

Anomaly detection can also be used to detect unusual time series. Bayesian networks are well suited for anomaly detection, because they can handle high dimensional data, which humans find difficult to interpret.

One typical way we can use data visualizations to identify some anomalies and these are clearly visible by plotting individual variables. More often anomalies are far more subtle, and are based on the interaction of many variables.

detect

Screen Shot 2016-04-10 at 9.12.48 AMScreen Shot 2016-04-10 at 9.07.25 AM

Here is a nice notebook on python mcmc:

I haven’t read the previous blog post on FFT. There are lots of time series analysis.

An interesting method for detection of patterns is using “Shape Search”:

Screen Shot 2016-04-10 at 8.58.20 AM

 

But I think there are interesting things using signal processing as well for AD like Median Filter.

http://docs.scipy.org/doc/scipy-0.16.0/reference/generated/scipy.ndimage.filters.median_filter.html

https://github.com/bugra/pydata-sv-2014

http://probcomp.csail.mit.edu/bayesdb/satellites-notebook.html

Follow

Get every new post delivered to your Inbox.

Join 53 other followers