feat(stepfunctions): add support for custom WriterConfig fields for ResultWriter in Distributed Map (#33772) and (#33601)

[vishalsatam]

feat(stepfunctions): Create a new ResultWriterV2 class to support WriterConfig in ResultWriter object in Distributed Map

Closes #33772 and #33601.

Reason for making this change:
A new property WriterConfig was added under ResultWriter property of Distributed Map.
WriterConfig contains OutputType and Transformation

Ref: https://docs.aws.amazon.com/step-functions/latest/dg/input-output-resultwriter.html#input-output-resultwriter-field-contents

ItemReader also supports "JSONL" as the InputType. It's currently missing in the enum.
Ref: https://docs.aws.amazon.com/step-functions/latest/dg/input-output-itemreader.html#itemreader-field-contents

Changes:

• Created a copy of ResultWriter as ResultWriterV2 to avoid introducing a breaking change to ResultWriter.
• Consumers will need to set feature@aws-cdk/aws-stepfunctions:useDistributedMapResultWriterV2 to true in the context.
• Create new class WriterConfig and add writerConfig property under ResultWriterV2 in Distributed Map.
• Add OutputType and Transformation properties under WriterConfig.
• Add Warning when ResultWriter is empty.
• Add the missing JSONL type which is a part of InputType ItemReader of Distributed Map.
• Permissions - No additions. But removed the permission to put object when just writerConfig is specified.
• Also added the missing JSONL from InputType in the ItemReader of Distributed Map.
• Add a unit test and an integ test.

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[aws-cdk-automation]

(This review is outdated)

[aws-cdk-automation]

(This review is outdated)

[vishalsatam]

aws-cdk-automation says add a README , but I don't think this needs one.

Dismissing outdated PRLinter review.

[aws-cdk-automation]

(This review is outdated)

[vishalsatam]

Exemption Request

Need to add this to allowed-breaking-changes.txt.

# bucket property will become optional after StepFunctions introduced writerConfig.
# Either writerConfig or the bucket and prefix needs to be specified
changed-type:aws-cdk-lib.aws_stepfunctions.ResultWriter.bucket

Example: below is perfectly valid now

      resultWriter: new sfn.ResultWriter({
        writerConfig: {
          outputType: sfn.OutputType.JSONL,
          transformation: sfn.Transformation.COMPACT,
        },
      }),

[shikha372]

Hi @vishalsatam , Thank you contributing this pull request, we don't accept any breaking changes to the stable library module, please see below how changing a prop type from required to optional can break customers

https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md#breaking-changes

what's recommended in such cases is we add a new type with v2 and mark the existing one as deprecated.

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

Pull request has been modified.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.98%. Comparing base (74cbe27) to head (e9d1024).
Report is 29 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #33831   +/-   ##
=======================================
  Coverage   83.98%   83.98%           
=======================================
  Files         120      120           
  Lines        6976     6976           
  Branches     1178     1178           
=======================================
  Hits         5859     5859           
  Misses       1005     1005           
  Partials      112      112           

Flag	Coverage Δ
suite.unit	83.98% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
packages/aws-cdk	∅ <ø> (∅)
packages/aws-cdk-lib/core	83.98% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[shikha372]

Thank you @vishalsatam for adding comprehensive integration and unit test, just few minor comments to address

[vishalsatam]

@shikha372 , Thank you for your patience with this pull request. Really appreciate the review and the comments..

Pull request has been modified.

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[mergify]

This pull request has been removed from the queue for the following reason: pull request branch update failed.

The pull request can't be updated.

You should update or rebase your pull request manually. If you do, this pull request will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[shikha372]

@Mergifyio update

[mergify]

update

❌ Mergify doesn't have permission to update

For security reasons, Mergify can't update this pull request. Try updating locally.
GitHub response: refusing to allow a GitHub App to create or update workflow .github/workflows/enum-auto-updater.yml without workflows permission

Pull request has been modified.

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: 721f58b
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.