AWS setup links

Setting up an instance with a Jupyter notebook:

https://docs.aws.amazon.com/mxnet/latest/dg/setup-jupyter-configure-server.html

Cost explorer (use per tag option):

https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/cost-explorer-what-is.html

(or use the ec2 usage calculator - both for compute time and volume storage)

Deep Learning AMIs: https://docs.aws.amazon.com/dlami/latest/devguide/options.html

Resizing Volumes on AWS: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/console-modify.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/recognize-expanded-volume-linux.html

Configure Boto3

conda install boto3 or pip install boto3

Run aws configure and fill in the requested information.

Open your ~/.aws/credentials file and rename [default] to the profile name of your preference.

Running spot instances

Starting a spot instance from command line. First need to create a .json spec file:

spot_spec.json

{
  "ImageId": "ami-e580c79d",
  "KeyName": "dssg2018-ddd-keypair",
  "SecurityGroupIds": [ "sg-01c4ea7e" ],
  "SubnetId": "subnet-972d2ddf",
  "InstanceType": "t2.micro",
  "Placement":{
    "AvailabilityZone": "us-west-2a"
  }
}

Start instances:

aws ec2 request-spot-instances --instance-count 1 --type "one-time" --spot-price 0.2 --launch-specification file://~/AWS/spot_spec.json --profile DSSG2018-ddd-team

Check all active instances:

aws ec2 describe-spot-instance-requests  --filters "Name=state,Values=active" --profile DSSG2018-ddd-team

Calncel request:

aws ec2 cancel-spot-instance-requests --spot-instance-request-ids sir-vfag96kp  --profile DSSG2018-ddd-team

Cancelling request does not terminate the instances. The instances need to be cancelled separately!!!