New SSM Automation Document - Create Image (#20)

* New SSM Automation Document - Create AMI Added new automation document - create amazon machine image (ami). * Delete create_document.py No longer required with updated document. * Update Design.md Replaced existing steps with action aws:createImage and re-ordered the numbers. * Update schema.json Remove LambdaAssumeRole parameter as Lambda function is no longer required and update Document description. * Update aws-CreateImage.json Remove LambdaAssumeRole Parameter Role. Remove Cloudformation and Lambda function steps with Automation action aws:createImage. Update Document description. * Delete CreateImageCFTemplate.yml CloudFormation template no longer required. * Delete create_image.py Lambda function no longer required. * Update test_document.py Update document location. * Update TestTemplate.yml Update stack description. * New Automation Document - DeleteImage
awslabs · Jul 26, 2018 · c94bcb3 · c94bcb3
1 parent e6fdb29
commit c94bcb3
Show file tree

Hide file tree

Showing 12 changed files with 744 additions and 0 deletions.
diff --git a/Documents/Automation/CreateImage/Design/Design.md b/Documents/Automation/CreateImage/Design/Design.md
@@ -0,0 +1,23 @@
+# Create an Amazon Machine Image (AMI)
+
+## Notes
+
+Create an Amazon Machine Image (AMI) from an Amazon EC2 instance
+
+## Document Design
+
+Refer to schema.json
+
+Document Steps:
+1. aws:createImage - Creates a new AMI from an Amazon EC2 instance
+   * Inputs:
+     * InstanceId: {{InstanceId}} - The ID of the Amazon EC2 instance.
+
+## Test script
+
+Python script will:
+  1. Create a test stack with the automation assumed role and Amazon EC2 instance
+  2. Execute automation document create an Amazon Machine Image (AMI) of the Amazon EC2 instance
+  3. Verify that the Amazon Machine Image is created
+  4. Deregister the Amazon Machine Image and delete associated Snapshot.
+  5. Clean up test stack
diff --git a/Documents/Automation/CreateImage/Design/schema.json b/Documents/Automation/CreateImage/Design/schema.json
@@ -0,0 +1,17 @@
+{
+  "description": "Creates a new Amazon Machine Image (AMI) from an Amazon EC2 instance",
+  "schemaVersion": "0.3",
+  "assumeRole": "{{ AutomationAssumeRole }}",
+  "parameters": {
+    "InstanceId": {
+      "type": "String",
+      "description": "(Required) The ID of the Amazon EC2 instance."
+    },
+    "AutomationAssumeRole": {
+      "type": "String",
+      "description": "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf. ",
+      "default": ""
+    }
+  },
+  "mainSteps": []
+}
diff --git a/Documents/Automation/CreateImage/Documents/aws-CreateImage.json b/Documents/Automation/CreateImage/Documents/aws-CreateImage.json
@@ -0,0 +1,29 @@
+{
+  "description": "Creates a new Amazon Machine Image (AMI) from an Amazon EC2 instance",
+  "schemaVersion": "0.3",
+  "assumeRole": "{{ AutomationAssumeRole }}",
+  "parameters": {
+    "InstanceId": {
+      "type": "String",
+      "description": "(Required) The ID of the Amazon EC2 instance."
+    },
+    "AutomationAssumeRole": {
+      "type": "String",
+      "description": "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf. ",
+      "default": ""
+    }
+  },
+  "mainSteps": [
+{
+    "name": "createImage",
+    "action": "aws:createImage",
+    "maxAttempts": 3,
+    "onFailure": "Abort",
+    "inputs": {
+        "InstanceId": "{{ InstanceId }}",
+        "ImageName": "{{ InstanceId }}_{{automation:EXECUTION_ID}}",
+        "NoReboot": true
+    }
+}
+  ]
+}
diff --git a/Documents/Automation/CreateImage/Makefile b/Documents/Automation/CreateImage/Makefile
@@ -0,0 +1,34 @@
+#
+# Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of this
+# software and associated documentation files (the "Software"), to deal in the Software
+# without restriction, including without limitation the rights to use, copy, modify,
+# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+TARGET_DIR = "./Output"
+
+documents: targetdir createdocuments
+	@echo "Done making documents"
+
+targetdir:
+	@echo "Making $(TARGET_DIR)"
+	mkdir -p ./Output
+
+createdocuments:
+	python ./Setup/create_document.py > ./Output/aws-CreateImage.json
+
+test: documents
+	python -m unittest discover Tests
+
+clean:
+	@echo "Removing $(TARGET_DIR)"
+	@rm -rf ./Output
diff --git a/Documents/Automation/CreateImage/Tests/CloudFormationTemplates/TestTemplate.yml b/Documents/Automation/CreateImage/Tests/CloudFormationTemplates/TestTemplate.yml
@@ -0,0 +1,69 @@
+#
+# Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of this
+# software and associated documentation files (the "Software"), to deal in the Software
+# without restriction, including without limitation the rights to use, copy, modify,
+# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+---
+AWSTemplateFormatVersion: '2010-09-09'
+Description: Test stack for Create a new Amazon Machine Image (AMI)
+Outputs:
+  InstanceId:
+    Description: Instance Id
+    Value:
+      Ref: Instance0
+  AutomationAssumeRoleName:
+    Description: Automation Assume Role Name
+    Value: !Ref AutomationAssumeRole
+  AutomationAssumeRoleARN:
+    Description: Automation Assume Role ARN
+    Value: !GetAtt AutomationAssumeRole.Arn
+Parameters:
+  AMI:
+    Description: AMI ID for instances.
+    Type: String
+  INSTANCETYPE:
+    Description: AMI Instance Type (t2.micro, m1.large, etc.)
+    Type: String
+  UserARN:
+    Description: user ARN
+    Type: String
+Resources:
+  AutomationAssumeRole:
+    Type: "AWS::IAM::Role"
+    Properties:
+      AssumeRolePolicyDocument:
+        Version: "2012-10-17"
+        Statement:
+          - Effect: "Allow"
+            Principal:
+              Service:
+                - "lambda.amazonaws.com"
+                - "ssm.amazonaws.com"
+            Action:
+              - "sts:AssumeRole"
+          - Effect: "Allow"
+            Principal:
+              AWS: !Ref UserARN
+            Action:
+              - "sts:AssumeRole"
+      ManagedPolicyArns:
+        - "arn:aws:iam::aws:policy/AdministratorAccess"
+  Instance0:
+    Type: AWS::EC2::Instance
+    Properties:
+      ImageId: !Ref AMI
+      InstanceType: !Ref INSTANCETYPE
+      Tags:
+        - Key: Name
+          Value: Test Create a new Amazon Machine Image (AMI)
diff --git a/Documents/Automation/CreateImage/Tests/test_document.py b/Documents/Automation/CreateImage/Tests/test_document.py
@@ -0,0 +1,181 @@
+#
+# Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of this
+# software and associated documentation files (the "Software"), to deal in the Software
+# without restriction, including without limitation the rights to use, copy, modify,
+# merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
+# PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+#!/usr/bin/env python
+
+import os
+import sys
+import logging
+import unittest
+
+import ConfigParser
+import boto3
+import json
+
+import time
+
+DOC_DIR = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
+REPOROOT = os.path.dirname(DOC_DIR)
+
+# Import shared testing code
+sys.path.append(
+    os.path.join(
+        REPOROOT,
+        'Testing'
+    )
+)
+sys.path.append(os.path.join(
+    DOC_DIR, "Documents/Lambdas"
+))
+sys.path.append(
+    os.path.abspath(os.path.join(
+        os.path.dirname(os.path.realpath(__file__)),
+        "lib/"
+    ))
+)
+import ssm_testing  # noqa pylint: disable=import-error,wrong-import-position
+
+CONFIG = ConfigParser.ConfigParser()
+CONFIG.readfp(open(os.path.join(REPOROOT, 'Testing', 'defaults.cfg')))
+CONFIG.read([os.path.join(REPOROOT, 'Testing', 'local.cfg')])
+
+REGION = CONFIG.get('general', 'region')
+PREFIX = CONFIG.get('general', 'resource_prefix')
+
+WINDOWS_AMI_ID = CONFIG.get('windows', 'windows2016.{}'.format(REGION))
+INSTANCE_TYPE = CONFIG.get('windows', 'instance_type')
+LINUX_AMI_ID = CONFIG.get('linux', 'ami')
+LINUX_INSTANCE_TYPE = CONFIG.get('linux', 'instance_type')
+
+SSM_DOC_NAME = PREFIX + 'create-image'
+CFN_STACK_NAME = PREFIX + 'create-image'
+TEST_CFN_STACK_NAME = PREFIX + 'create-image'
+
+logging.basicConfig(level=CONFIG.get('general', 'log_level').upper())
+LOGGER = logging.getLogger(__name__)
+logging.getLogger('botocore').setLevel(level=logging.WARNING)
+
+boto3.setup_default_session(region_name=REGION)
+
+iam_client = boto3.client('iam')
+s3_client = boto3.client('s3')
+sns_client = boto3.client('sns')
+sts_client = boto3.client('sts')
+
+
+def verify_role_created(role_arn):
+    LOGGER.info("Verifying that role exists: " + role_arn)
+    # For what ever reason assuming a role that got created too fast fails, so we just wait until we can.
+    retry_count = 12
+    while True:
+        try:
+            sts_client.assume_role(RoleArn=role_arn, RoleSessionName="checking_assume")
+            break
+        except Exception as e:
+            retry_count -= 1
+            if retry_count == 0:
+                raise e
+
+            LOGGER.info("Unable to assume role... trying again in 5 sec")
+            time.sleep(5)
+
+
+class DocumentTest(unittest.TestCase):
+    def test_update_document(self):
+        cfn_client = boto3.client('cloudformation', region_name=REGION)
+        ssm_client = boto3.client('ssm', region_name=REGION)
+
+        ssm_doc = ssm_testing.SSMTester(
+            ssm_client=ssm_client,
+            doc_filename=os.path.join(DOC_DIR,
+                                      'Documents/aws-CreateImage.json'),
+            doc_name=SSM_DOC_NAME,
+            doc_type='Automation'
+        )
+
+        test_cf_stack = ssm_testing.CFNTester(
+            cfn_client=cfn_client,
+            template_filename=os.path.abspath(os.path.join(
+                DOC_DIR,
+                "Tests/CloudFormationTemplates/TestTemplate.yml")),
+            stack_name=TEST_CFN_STACK_NAME
+        )
+
+        LOGGER.info('Creating Test Stack')
+        LOGGER.info('AMI:' + LINUX_AMI_ID)
+        LOGGER.info('Instance Type:' + LINUX_INSTANCE_TYPE)
+        test_cf_stack.create_stack([
+            {
+                'ParameterKey': 'AMI',
+                'ParameterValue': LINUX_AMI_ID
+            },
+            {
+                'ParameterKey': 'INSTANCETYPE',
+                'ParameterValue': LINUX_INSTANCE_TYPE
+            },
+            {
+                'ParameterKey': 'UserARN',
+                'ParameterValue': sts_client.get_caller_identity().get('Arn')
+            }
+        ])
+        LOGGER.info('Test Stack has been created')
+
+        # Verify role exists
+        role_arn = test_cf_stack.stack_outputs['AutomationAssumeRoleARN']
+        verify_role_created(role_arn)
+        try:
+            LOGGER.info("Creating automation document")
+            self.assertEqual(ssm_doc.create_document(), 'Active')
+
+            instance_id = test_cf_stack.stack_outputs['InstanceId']
+
+            execution = ssm_doc.execute_automation(
+                params={'InstanceId': [instance_id],
+                        'AutomationAssumeRole': [role_arn]})
+            self.assertEqual(ssm_doc.automation_execution_status(ssm_client, execution, False), 'Success')
+
+            LOGGER.info('Create a new Amazon Machine Image has been initiated')
+
+            ec2_resource = boto3.resource('ec2')
+            ec2client = boto3.client('ec2')
+            instance = ec2_resource.Instance(instance_id)
+
+            imageName = instance_id + '_' + execution
+            describeImage = ec2client.describe_images(Filters = [ { 'Name': 'name','Values': [ imageName ], } ] )
+            ImageId = describeImage['Images'][0]['ImageId']
+
+            # Test Amazon Machine Image exists
+            waiter = ec2client.get_waiter('image_available') 
+            waiter.wait(ImageIds=[ImageId])
+            LOGGER.info("All Tests Successful, will clean up now")
+
+            #Get SnapshotId from Amazon Machine Image
+	    imageData = ec2_resource.Image(ImageId)
+	    SnapshotId = imageData.block_device_mappings[0]['Ebs']['SnapshotId']
+            LOGGER.info("Got the SnapshotId from the Amazon Machine Image for the clean up process.")
+        finally:
+            try:
+                ssm_doc.destroy()
+                ec2client.deregister_image(ImageId=ImageId)
+                ec2client.delete_snapshot(SnapshotId=SnapshotId)
+            except Exception:
+                pass
+
+            test_cf_stack.delete_stack()
+
+
+if __name__ == '__main__':
+    unittest.main()
diff --git a/Documents/Automation/DeleteImage/Design/Design.md b/Documents/Automation/DeleteImage/Design/Design.md
@@ -0,0 +1,25 @@
+# "Delete Amazon Machine Image (AMI) and all associated snapshots."
+
+## Notes
+
+Deletes the specified Amazon Machine Image (AMI) and all related snapshots.
+
+## Document Design
+
+Refer to schema.json
+
+Document Steps:
+1. aws:deleteImage - Deletes the Amazon Machine Image (AMI) and all related snapshots.
+   * Inputs:
+     * ImageId: {{ImageId}} - The ID of the Amazon Machine Image (AMI).
+
+
+## Test script
+
+Python script will:
+  1. Create a test stack with the automation assumed role and Amazon EC2 Instance.
+  2. Create an Amazon Machine Image (AMI) from the Amazon EC2 Instance.
+  3. Execute automation document to delete the specified Amazon Machine Image (AMI) and associated Snapshot.
+  3. Deregister the Amazon Machine Image (AMI) and delete associated Snapshot
+  4. Verify that the Amazon Machine Image (AMI) and associated Snapshot has been deleted
+  5. Clean up test stack