Application Cost Monitoring provides granular AWS spend tracking.
Using the AWS Cost and Usage report this system will parse the report and feed the line items to a place where you can perform analysis. This service only handles the billing report ingestion, parsing, and publishing to SNS. Additional publisher services need to be deployed. eg.
Read why we released Application Cost Monitoring as independent nanoservices as opposed to a single application along with our thoughts on using nanoservices to build serverless applications:
Read through the entire documentation first. There is information in setting up the billing report that may influence your deployment.
You will perform the folowing actions:
- Deploy the application
- Create the billing report
- Setup bucket policy (AWS Application Repository only)
This service supports both Serverless Framework and AWS Serverless Application Repository.
Clone this repository and deploy using Serverless Framework.
$ npm install -g serverless
$ npm install
$ serverless deploy -v
This application is available in the AWS Serverless Application Repository. Follow the directions there if you wish to deploy from AppRepo.
- aws-adm-${stage}-BillingReportS3BucketName: Name of S3 Bucket where billing reports will be delivered
- aws-adm-${stage}-BillingRecordsSnsTopicArn: ARN of SNS topic that data publishers are to connect to.
- aws-adm-${stage}-AdmStateS3BucketName: Name of the S3 bucket where state is stored during and between runs.
Setup the AWS Billing cost and usage report. This will deliver billing reports to the configured S3 bucket up to three times a day. This service will create the S3 bucket for you when it is deployed. Get the aws-adm-${stage}-BillingReportS3BucketName stack export after deploying and
Follow the AWS instructions for turning on the Cost and Usage Report. Select hourly or daily report data depending on the granularity of data you need (and can afford considering the potential size).
Additional cost insight can be found by using cost allocation tags. Enable cost allocation tags in the AWS console if desired and activate any appropriate user defined tags.
To see what data is in the report, refer to the AWS documentation for cost and Usage Report Details. You decide the tags to track before you deploy Application Cost Monitoring. Changing the tags tracked in billing reports will cause some line item's to change their ID. Depending on how you are performing your analysis this may not be an issue. If you are using AWS Athena to query the data then this will result in a schema change that will break querying. You will also have to deal with duplicate line item data in the dataset unless you purge all previous data. See the SCHEMA_CHANGE_HANDLING variable for more information.
SCHEMA_CHANGE_HANDLING: Set the desired behavior for how to handle a change in the billing report schema being detected. If using CloudFormation or AWS SAM, set this parameter to one of the values below. If using Serverless Framework, set value as an environmental variable. Choose the correct option for you after reading below. The default value is CONTINUE.
Changing tags on billing reports will alter the report schema which can:
- break downstream analysis systems dependent on the schema.
- result in item duplication as AWS will generate a new line item ID and this system has to continually reprocess 1st of month on every run.
- make tracking an item across the month difficult due to change in ID.
Options:
- ERROR: Error out line item writer. Must remove schema state file or remove tags to continue processing.
- CONTINUE: Just continue processing.
- RECONCILE: Reprocess the entire report.
This service only handles billing report ingestion and parsing. It requires an additional publisher service to be useful. Available ones are: