feature: add dynamodb, fix docs (#440)

Author: ran-isenberg

.github/workflows/serverless-service.yml

@@ -35,7 +35,7 @@ jobs:
           pip install -r lambda_requirements.txt
           npm install -g aws-cdk
       - name: Configure aws credentials
-        uses: aws-actions/configure-aws-credentials@v1-node16
+        uses: aws-actions/configure-aws-credentials@v1
         with:
           aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
           aws-secret-access-key: ${{ secrets.AWS_SECRET_KEY }}
@@ -53,7 +53,12 @@ jobs:
         run: |
           make complex
       - run: echo "🖥️ The workflow is now ready to test your code on the runner."
-      - name: Unit and Integration tests
+      - name: Unit tests
+        run: |
+          make unit
+      - name: Deploy to AWS
+        run: make deploy
+      - name: Integration tests
         run: |
           make pipeline-tests
       - name: Codecov
@@ -64,8 +69,6 @@ jobs:
           fail_ci_if_error: true # optional (default = false)
           verbose: false # optional (default = false)
           token: ${{ secrets.CODECOV_TOKEN }}
-      - name: Deploy to AWS
-        run: make deploy
       - name: Run E2E tests
         run: make e2e
       - name: Destroy stack

Makefile

@@ -35,7 +35,7 @@ integration:
 e2e:
 	pytest tests/e2e  --cov-config=.coveragerc --cov=service --cov-report xml
 
-pr: deps yapf sort pre-commit complex lint lint-docs unit integration e2e
+pr: deps yapf sort pre-commit complex lint lint-docs unit deploy integration e2e
 
 yapf:
 	yapf -i -vv --style=./.style --exclude=.venv --exclude=.build --exclude=cdk.out --exclude=.git  -r .

Pipfile

@@ -26,6 +26,8 @@ setuptools = ">=65.5.1"
 
 [packages]
 aws-lambda-powertools= {extras = ["all"],version = "*"}
+mypy-boto3-dynamodb = "*"
+cachetools = "*"
 
 [requires]
 python_version = "3.9"

Pipfile.lock

@@ -17,22 +17,43 @@ This project can serve as a template for new Serverless services - CDK deploymen
 **[📜Documentation](https://ran-isenberg.github.io/aws-lambda-handler-cookbook/)** | **[Blogs website](https://www.ranthebuilder.cloud)**
 > **Contact details | ran.isenberg@ranthebuilder.cloud**
 
-## Elevate Your Handler's Code
 
-What makes an AWS Lambda handler resilient, traceable and easy to maintain? How do you write such a code?
+## **The Problem**
 
-The project is a template project that is based on my AWS Lambda handler cookbook blog series that I publish in [ranthebuilder.cloud](https://www.ranthebuilder.cloud) and attempt to answer those questions.
+Starting a Serverless service can be overwhelming. You need to figure out many questions and challenges that have nothing to do with your business domain:
 
-This project provides a working, open source based, AWS Lambda handler skeleton Python code including DEPLOYMENT code with CDK and a pipeline.
+- How to deploy to the cloud? What IAC framework do you choose?
+- How to write a SaaS-oriented CI/CD pipeline? What does it need to contain?
+- How do you handle observability. Logging, tracing, metrics
+- How do you handle testing?
+- What makes an AWS Lambda handler resilient, traceable, and easy to maintain? How do you write such a code?
+
+
+## **The Solution**
+
+This project aims to reduce cognitive load and answer these questions for you by providing a skeleton Python Serverless service template
+
+that implements best practices for AWS Lambda, Serverless CI/CD, and AWS CDK in one template project.
+
+### Serverless Service - The Order service
+
+- This project provides a working orders service where customers can create orders of items.
+
+- The project deploys an API GW with an AWS Lambda integration under the path POST /api/orders/ and stores data in a DynamoDB table.
 
-The project deploys an API GW with an AWS Lambda integration under the path POST /api/service/.
+### **Features**
 
-The AWS Lambda handler embodies Serverless best practices and has all the bells and whistles for a proper production ready handler.
+- Python Serverless service with a recommended file structure.
+- CDK infrastructure with infrastructure tests and security tests.
+- CI/CD pipelines based on Github actions that deploys to AWS.
+- Unit/integration and E2E tests.
+- The AWS Lambda handler embodies Serverless best practices and has all the bells and whistles for a proper production ready handler.
 
+The GitHub template project can be found at [https://github.com/ran-isenberg/aws-lambda-handler-cookbook](https://github.com/ran-isenberg/aws-lambda-handler-cookbook).
 
 
 ## CDK Deployment
-The CDK code create an API GW with a path of /api/service which triggers the lambda on 'POST' requests.
+The CDK code create an API GW with a path of /api/orders which triggers the lambda on 'POST' requests.
 
 The AWS Lambda handler uses a Lambda layer optimization which takes all the packages under the [packages] section in the Pipfile and downloads them in via a Docker instance.
 
@@ -45,13 +66,14 @@ Each utility is implemented when a new blog post is published about that utility
 
 The utilities cover multiple aspect of a production-ready service, including:
 
-1.  [Logging](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-elevate-your-handler-s-code-part-1-logging)
-2.  [Observability: Monitoring and Tracing](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-elevate-your-handler-s-code-part-2-observability)
-3.  [Observability: Business Domain Metrics](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-elevate-your-handler-s-code-part-3-business-domain-observability)
-4.  [Environment variables](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-environment-variables)
-5.  [Input validation](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-elevate-your-handler-s-code-part-5-input-validation)
-6.  [Features flags & dynamic configuration](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-part-6-feature-flags-configuration-best-practices)
-7.  [Start Your AWS Serverless Service With Two Clicks](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-part-7-how-to-use-the-aws-lambda-cookbook-github-template-project)
+- [Logging](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-elevate-your-handler-s-code-part-1-logging)
+- [Observability: Monitoring and Tracing](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-elevate-your-handler-s-code-part-2-observability)
+- [Observability: Business KPIs Metrics](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-elevate-your-handler-s-code-part-3-business-domain-observability)
+- [Environment Variables](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-environment-variables)
+- [Input Validation](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-elevate-your-handler-s-code-part-5-input-validation)
+- [Dynamic Configuration & feature flags](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-part-6-feature-flags-configuration-best-practices)
+- [Start Your AWS Serverless Service With Two Clicks](https://www.ranthebuilder.cloud/post/aws-lambda-cookbook-part-7-how-to-use-the-aws-lambda-cookbook-github-template-project)
+- [CDK Best practices](https://github.com/ran-isenberg/aws-lambda-handler-cookbook)
 
 
 I've written 3 of the mentioned utilities (parser, feature flags and environment variables) and donated two of them, the [parser](https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/parser/) and [feature flags](https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/feature_flags/) to [AWS Lambda Powertools](https://awslabs.github.io/aws-lambda-powertools-python/latest/).

README.md

@@ -36,8 +36,8 @@ def __init__(self, scope: Construct, id_: str, environment: str, service_name: s
 
         self.config_app = appconfig.CfnApplication(
             self,
-            id=service_name,
-            name=service_name,
+            id=f'{id_}{service_name}',
+            name=f'{id_}{service_name}',
         )
         self.config_env = appconfig.CfnEnvironment(
             self,

cdk/my_service/service_stack/configuration/configuration_construct.py

@@ -1,14 +1,15 @@
 SERVICE_ROLE_ARN = 'ServiceRoleArn'
 LAMBDA_BASIC_EXECUTION_ROLE = 'AWSLambdaBasicExecutionRole'
 SERVICE_ROLE = 'ServiceRole'
-
+TABLE_NAME = 'orders'
+TABLE_NAME_OUTPUT = 'DbOutput'
 APIGATEWAY = 'Apigateway'
-GW_RESOURCE = 'service'
+GW_RESOURCE = 'orders'
 LAMBDA_LAYER_NAME = 'common'
 API_HANDLER_LAMBDA_MEMORY_SIZE = 128  # MB
 API_HANDLER_LAMBDA_TIMEOUT = 10  # seconds
 POWERTOOLS_SERVICE_NAME = 'POWERTOOLS_SERVICE_NAME'
-SERVICE_NAME = 'Example'
+SERVICE_NAME = 'Orders'
 METRICS_NAMESPACE = 'my_product_kpi'
 POWERTOOLS_TRACE_DISABLED = 'POWERTOOLS_TRACE_DISABLED'
 POWER_TOOLS_LOG_LEVEL = 'LOG_LEVEL'

cdk/my_service/service_stack/constants.py

@@ -1,5 +1,6 @@
 import my_service.service_stack.constants as constants
 from aws_cdk import CfnOutput, Duration, RemovalPolicy, aws_apigateway
+from aws_cdk import aws_dynamodb as dynamodb
 from aws_cdk import aws_iam as iam
 from aws_cdk import aws_lambda as _lambda
 from aws_cdk import aws_logs
@@ -9,47 +10,66 @@
 
 class ApiConstruct(Construct):
 
-    def __init__(self, scope: Construct, id_: str) -> None:
+    def __init__(self, scope: Construct, id_: str, appconfig_app_name: str) -> None:
         super().__init__(scope, id_)
 
-        self.lambda_role = self._build_lambda_role()
+        self.db = self._build_db(id_)
+        self.lambda_role = self._build_lambda_role(self.db)
         self.common_layer = self._build_common_layer()
-
         self.rest_api = self._build_api_gw()
         api_resource: aws_apigateway.Resource = self.rest_api.root.add_resource('api').add_resource(constants.GW_RESOURCE)
-        self.__add_post_lambda_integration(api_resource, self.lambda_role)
+        self.__add_post_lambda_integration(api_resource, self.lambda_role, self.db, appconfig_app_name)
+
+    def _build_db(self, id_prefix: str) -> dynamodb.Table:
+        table_id = f'{id_prefix}{constants.TABLE_NAME}'
+        table = dynamodb.Table(
+            self,
+            table_id,
+            table_name=table_id,
+            partition_key=dynamodb.Attribute(name='order_id', type=dynamodb.AttributeType.STRING),
+            billing_mode=dynamodb.BillingMode.PAY_PER_REQUEST,
+            point_in_time_recovery=True,
+            removal_policy=RemovalPolicy.DESTROY,
+        )
+        CfnOutput(self, id=constants.TABLE_NAME_OUTPUT, value=table.table_name).override_logical_id(constants.TABLE_NAME_OUTPUT)
+        return table
 
     def _build_api_gw(self) -> aws_apigateway.LambdaRestApi:
         rest_api: aws_apigateway.LambdaRestApi = aws_apigateway.RestApi(
             self,
             'service-rest-api',
             rest_api_name='Service Rest API',
-            description='This service handles /api/service requests',
+            description='This service handles /api/orders requests',
             deploy_options=aws_apigateway.StageOptions(throttling_rate_limit=2, throttling_burst_limit=10),
         )
 
         CfnOutput(self, id=constants.APIGATEWAY, value=rest_api.url).override_logical_id(constants.APIGATEWAY)
         return rest_api
 
-    def _build_lambda_role(self) -> iam.Role:
+    def _build_lambda_role(self, db: dynamodb.Table) -> iam.Role:
         return iam.Role(
             self,
             constants.SERVICE_ROLE,
             assumed_by=iam.ServicePrincipal('lambda.amazonaws.com'),
             inline_policies={
                 'dynamic_configuration':
                     iam.PolicyDocument(statements=[
-                        iam.PolicyStatement(actions=['appconfig:GetLatestConfiguration', 'appconfig:StartConfigurationSession'], resources=['*'],
-                                            effect=iam.Effect.ALLOW),
+                        iam.PolicyStatement(
+                            actions=['appconfig:GetLatestConfiguration', 'appconfig:StartConfigurationSession'],
+                            resources=['*'],
+                            effect=iam.Effect.ALLOW,
+                        )
                     ]),
+                'dynamodb_db':
+                    iam.PolicyDocument(
+                        statements=[iam.PolicyStatement(actions=['dynamodb:PutItem'], resources=[db.table_arn], effect=iam.Effect.ALLOW)]),
             },
             managed_policies=[
                 iam.ManagedPolicy.from_aws_managed_policy_name(managed_policy_name=(f'service-role/{constants.LAMBDA_BASIC_EXECUTION_ROLE}'))
             ],
         )
 
     def _build_common_layer(self) -> PythonLayerVersion:
-
         return PythonLayerVersion(
             self,
             'CommonLayer',
@@ -58,22 +78,23 @@ def _build_common_layer(self) -> PythonLayerVersion:
             removal_policy=RemovalPolicy.DESTROY,
         )
 
-    def __add_post_lambda_integration(self, api_name: aws_apigateway.Resource, role: iam.Role):
+    def __add_post_lambda_integration(self, api_name: aws_apigateway.Resource, role: iam.Role, db: dynamodb.Table, appconfig_app_name: str):
         lambda_function = _lambda.Function(
             self,
             'ServicePost',
             runtime=_lambda.Runtime.PYTHON_3_9,
             code=_lambda.Code.from_asset(constants.BUILD_FOLDER),
-            handler='service.handlers.my_handler.my_handler',
+            handler='service.handlers.create_order.create_order',
             environment={
                 constants.POWERTOOLS_SERVICE_NAME: constants.SERVICE_NAME,  # for logger, tracer and metrics
                 constants.POWER_TOOLS_LOG_LEVEL: 'DEBUG',  # for logger
+                'CONFIGURATION_APP': appconfig_app_name,  # for feature flags
+                'CONFIGURATION_ENV': constants.ENVIRONMENT,  # for feature flags
+                'CONFIGURATION_NAME': constants.CONFIGURATION_NAME,  # for feature flags
+                'CONFIGURATION_MAX_AGE_MINUTES': constants.CONFIGURATION_MAX_AGE_MINUTES,  # for feature flags
                 'REST_API': 'https://www.ranthebuilder.cloud/api',  # for env vars example
                 'ROLE_ARN': 'arn:partition:service:region:account-id:resource-type:resource-id',  # for env vars example
-                'CONFIGURATION_APP': constants.SERVICE_NAME,
-                'CONFIGURATION_ENV': constants.ENVIRONMENT,
-                'CONFIGURATION_NAME': constants.CONFIGURATION_NAME,
-                'CONFIGURATION_MAX_AGE_MINUTES': constants.CONFIGURATION_MAX_AGE_MINUTES,
+                'TABLE_NAME': db.table_name,
             },
             tracing=_lambda.Tracing.ACTIVE,
             retry_attempts=0,
@@ -84,5 +105,5 @@ def __add_post_lambda_integration(self, api_name: aws_apigateway.Resource, role:
             log_retention=aws_logs.RetentionDays.ONE_DAY,
         )
 
-        # POST /api/service/
+        # POST /api/orders/
         api_name.add_method(http_method='POST', integration=aws_apigateway.LambdaIntegration(handler=lambda_function))

cdk/my_service/service_stack/service_construct.py

@@ -14,7 +14,7 @@ def get_stack_name() -> str:
     try:
         username = os.getlogin().replace('.', '-')
     except Exception:
-        username = 'main'
+        username = 'github'
     print(f'username={username}')
     try:
         return f'{username}-{repo.active_branch}-{SERVICE_NAME}'
@@ -30,5 +30,5 @@ def __init__(self, scope: Construct, id: str, **kwargs) -> None:
         # This construct should be deployed in a different repo and have its own pipeline so updates can be decoupled
         # from running the service pipeline and without redeploying the service lambdas. For the sake of this template
         # example, it is deployed as part of the service stack
-        self.dynamic_configuration = ConfigurationStore(self, 'dynamic_conf'[0:64], ENVIRONMENT, SERVICE_NAME, CONFIGURATION_NAME)
-        self.lambdas = ApiConstruct(self, 'Service'[0:64])
+        self.dynamic_configuration = ConfigurationStore(self, f'{id}dynamic_conf'[0:64], ENVIRONMENT, SERVICE_NAME, CONFIGURATION_NAME)
+        self.lambdas = ApiConstruct(self, f'{id}Service'[0:64], self.dynamic_configuration.config_app.name)

cdk/my_service/service_stack/service_stack.py

@@ -7,7 +7,7 @@
 
 setup(
     name='aws-lambda-handler-cookbook',
-    version='1.0',
+    version='2.2',
     description='CDK code for deploying an AWS Lambda handler that implements the best practices described at https://www.ranthebuilder.cloud',
     classifiers=[
         'Intended Audience :: Developers',
@@ -25,6 +25,6 @@
         'aws-cdk-lib>=2.0.0',
         'constructs>=10.0.0',
         'cdk-nag>2.0.0',
-        'aws-cdk.aws-lambda-python-alpha==2.50.0-alpha.0',
+        'aws-cdk.aws-lambda-python-alpha==2.54.0-alpha.0',
     ],
 )