BigSnarf blog

Infosec FTW

Category Archives: Tools

NHTSA Federal HAV Policy

Driving a simulator

View story at Medium.com

http://cs231n.stanford.edu/reports2016/112_Report.pdf

http://arxiv.org/pdf/1608.01230v1.pdf

http://download.visinf.tu-darmstadt.de/data/from_games/

http://motherboard.vice.com/read/video-games-artificial-intelligence-realistic-teach-ai-what-the-world-looks-like

http://cs231n.stanford.edu/reports2016/123_Report.pdf

http://cs224d.stanford.edu/lectures/CS224d-Lecture7.pdf

Deep Learning in a Nutshell: Reinforcement Learning

http://www.cv-foundation.org/openaccess/content_cvpr_2016_workshops/w3/papers/Gurghian_DeepLanes_End-To-End_Lane_CVPR_2016_paper.pdf

https://github.com/commaai/research/blob/master/DriveSim.md

Self Driving RC – Open Source

Bunch of RC Robot videos

Using Inception v3 Tensorflow for MNIST

Modern object recognition models have millions of parameters and can take weeks to fully train. Transfer learning is a technique that shortcuts a lot of this work by taking a fully-trained model for a set of categories like ImageNet, and retrains from the existing weights for new classes. In this example we’ll be retraining the final layer from scratch, while leaving all the others untouched. For more information on the approach you can see this paper on Decaf.

Though it’s not as good as a full training run, this is surprisingly effective for many applications, and can be run in as little as 75 minutes on a laptop, without requiring a GPU. The data I used is from Kaggle MNIST dataset.

Let’s reshape the train.csv data from Kaggle with this script to jpegs

Screen Shot 2016-07-19 at 8.24.47 PM

Script to convert train.csv to images in python

 

Let’s move the data to the proper folders

 

These are screenshots of the re-trained Inception v3 model

Screen Shot 2016-07-19 at 1.59.35 PM

Screen Shot 2016-07-19 at 2.02.52 PM

 

Re – Training ModelScreen Shot 2016-07-19 at 8.21.16 PM

Using the re-trained model to do MNIST prediction

Links:

Audit Security in AWS

When Should You Perform a Security Audit?

You should audit your security configuration in the following situations:

  • On a periodic basis. You should perform the steps described in this document at regular intervals as a best practice for security.
  • If there are changes in your organization, such as people leaving.
  • If you have stopped using one or more individual AWS services. This is important for removing permissions that users in your account no longer need.
  • If you’ve added or removed software in your accounts, such as applications on Amazon EC2 instances, AWS OpsWorks stacks, AWS CloudFormation templates, etc.
  • If you ever suspect that an unauthorized person might have accessed your account.

General Guidelines for Auditing

As you review your account’s security configuration, follow these guidelines:

  • Be thorough. Look at all aspects of your security configuration, including those you might not use regularly.
  • Don’t assume. If you are unfamiliar with some aspect of your security configuration (for example, the reasoning behind a particular policy or the existence of a role), investigate the business need until you are satisfied.
  • Keep things simple. To make auditing (and management) easier, use IAM groups, consistent naming schemes, and straightforward policies.

Review Your AWS Account Credentials

Take these steps when you audit your AWS account credentials:

  1. If you’re not using the root access keys for your account, remove them. We strongly recommend that you do not use root access keys for everyday work with AWS, and that instead you create IAM users.
  2. If you do need to keep the access keys for your account, rotate them regularly.

Review Your IAM Users

Take these steps when you audit your existing IAM users:

  1. Delete users that are not active.
  2. Remove users from groups that they don’t need to be a part of.
  3. Review the policies attached to the groups the user is in. See Tips for Reviewing IAM Policies.
  4. Delete security credentials that the user doesn’t need or that might have been exposed. For example, an IAM user that is used for an application does not need a password (which is necessary only to sign in to AWS websites). Similarly, if a user does not use access keys, there’s no reason for the user to have one. For more information, see Managing Passwords for IAM Users and Managing Access Keys for IAM Users in the IAM User Guide guide.

    You can generate and download a credential report that lists all IAM users in your account and the status of their various credentials, including passwords, access keys, and MFA devices. For passwords and access keys, the credential report shows how recently the password or access key has been used. Credentials that have not been used recently might be good candidates for removal. For more information, see Getting Credential Reports for your AWS Account in the IAM User Guide guide.

  5. Rotate (change) user security credentials periodically, or immediately if you ever share them with an unauthorized person. For more information, see Managing Passwords for IAM Users and Managing Access Keys for IAM Users in the IAM User Guide guide.

Review Your IAM Groups

Take these steps when you audit your IAM groups:

  1. Delete unused groups.
  2. Review users in each group and remove users who don’t belong. See Review Your IAM Users earlier.
  3. Review the policies attached to the group. See Tips for Reviewing IAM Policies.

Review Your IAM Roles

Take these steps when you audit your IAM roles:

  1. Delete roles that are not in use.
  2. Review the role’s trust policy. Make sure that you know who the principal is and that you understand why that account or user needs to be able to assume the role.
  3. Review the access policy for the role to be sure that it grants suitable permissions to whoever assumes the role—see Tips for Reviewing IAM Policies.

Review Your IAM Providers for SAML and OpenID Connect (OIDC)

If you have created an IAM entity for establishing trust with a SAML or OIDC identity provider, take these steps:

  1. Delete unused providers.
  2. Download and review the AWS metadata documents for each SAML provider and make sure the documents reflect your current business needs. Alternatively, get the latest metadata documents from the SAML IdPs that you want to establish trust with and update the provider in IAM.

Review Your Mobile Apps

If you have created a mobile app that makes requests to AWS, take these steps:

  1. Make sure that the mobile app does not contain embedded access keys, even if they are in encrypted storage.
  2. Get temporary credentials for the app by using APIs that are designed for that purpose. We recommend that you use Amazon Cognito to manage user identity in your app. This service lets you authenticate users using Login with Amazon, Facebook, Google, or any OpenID Connect (OIDC)–compatible identity provider. You can then use the Amazon Cognito credentials provider to manage credentials that your app uses to make requests to AWS.

    If your mobile app doesn’t support authentication using Login with Amazon, Facebook, Google, or any other OIDC-compatible identity provider, you can create a proxy server that can dispense temporary credentials to your app.

Review Your Amazon EC2 Security Configuration

Take the following steps for each AWS region:

  1. Delete Amazon EC2 key pairs that are unused or that might be known to people outside your organization.
  2. Review your Amazon EC2 security groups:
    • Remove security groups that no longer meet your needs.
    • Remove rules from security groups that no longer meet your needs. Make sure you know why the ports, protocols, and IP address ranges they permit have been allowed.
  3. Terminate instances that aren’t serving a business need or that might have been started by someone outside your organization for unapproved purposes. Remember that if an instance is started with a role, applications that run on that instance can access AWS resources using the permissions that are granted by that role.
  4. Cancel spot instance requests that aren’t serving a business need or that might have been made by someone outside your organization.
  5. Review your Auto Scaling groups and configurations. Shut down any that no longer meet your needs or that might have been configured by someone outside your organization.

Review AWS Policies in Other Services

Review the permissions for services that use resource-based policies or that support other security mechanisms. In each case, make sure that only users and roles with a current business need have access to the service’s resources, and that the permissions granted on the resources are the fewest necessary to meet your business needs.

Monitor Activity in Your AWS Account

Follow these guidelines for monitoring AWS activity:

  • Turn on AWS CloudTrail in each account and use it in each supported region.
  • Periodically examine CloudTrail log files. (CloudTrail has a number of partners who provide tools for reading and analyzing log files.)
  • Enable Amazon S3 bucket logging to monitor requests made to each bucket.
  • If you believe there has been unauthorized use of your account, pay particular attention to temporary credentials that have been issued. If temporary credentials have been issued that you don’t recognize, disabletheir permissions.
  • Enable billing alerts in each account and set a cost threshold that lets you know if your charges exceed your normal usage.

Tips for Reviewing IAM Policies

Policies are powerful and subtle, so it’s important to study and understand the permissions that are granted by each policy. Use the following guidelines when reviewing policies:

  • As a best practice, attach policies to groups instead of to individual users. If an individual user has a policy, make sure you understand why that user needs the policy.
  • Make sure that IAM users, groups, and roles have only the permissions that they need.
  • Use the IAM Policy Simulator to test policies that are attached to users or groups.
  • Remember that a user’s permissions are the result of all applicable policies—user policies, group policies, and resource-based policies (on Amazon S3 buckets, Amazon SQS queues, Amazon SNS topics, and AWS KMS keys). It’s important to examine all the policies that apply to a user and to understand the complete set of permissions granted to an individual user.
  • Be aware that allowing a user to create an IAM user, group, role, or policy and attach a policy to the principal entity is effectively granting that user all permissions to all resources in your account. That is, users who are allowed to create policies and attach them to a user, group, or role can grant themselves any permissions. In general, do not grant IAM permissions to users or roles whom you do not trust with full access to the resources in your account. The following list contains IAM permissions that you should review closely:
    • iam:PutGroupPolicy
    • iam:PutRolePolicy
    • iam:PutUserPolicy
    • iam:CreatePolicy
    • iam:CreatePolicyVersion
    • iam:AttachGroupPolicy
    • iam:AttachRolePolicy
    • iam:AttachUserPolicy
  • Make sure policies don’t grant permissions for services that you don’t use. For example, if you use AWS managed policies, make sure the AWS managed policies that are in use in your account are for services that you actually use. To find out which AWS managed policies are in use in your account, use the IAMGetAccountAuthorizationDetails API (AWS CLI command: aws iam get-account-authorization-details).
  • If the policy grants a user permission to launch an Amazon EC2 instance, it might also allow the iam:PassRoleaction, but if so it should explicitly list the roles that the user is allowed to pass to the Amazon EC2 instance.
  • Closely examine any values for the Action or Resource element that include *. It’s a best practice to grantAllow access to only the individual actions and resources that users need. However, the following are reasons that it might be suitable to use * in a policy:
    • The policy is designed to grant administrative-level privileges.
    • The wildcard character is used for a set of similar actions (for example, Describe*) as a convenience, and you are comfortable with the complete list of actions that are referenced in this way.
    • The wildcard character is used to indicate a class of resources or a resource path (e.g.,arn:aws:iam::account-id:users/division_abc/*), and you are comfortable granting access to all of the resources in that class or path.
    • A service action does not support resource-level permissions, and the only choice for a resource is *.
  • Examine policy names to make sure they reflect the policy’s function. For example, although a policy might have a name that includes “read only,” the policy might actually grant write or change permissions.

Hyper Parameter Optimization and AutoML

Data Implants will widen gap between rich and poor

Screen Shot 2016-06-16 at 9.41.54 AM.png

Gradient Descent Algorithms – Auto ML

Distributed Tensorflow