Skip to content

Conversation

@sam-goodwin
Copy link
Contributor

@sam-goodwin sam-goodwin commented Jan 12, 2026

Add AWS Kinesis Data Streams support with type-safe Lambda integration.

Stream Resource

import * as Kinesis from "alchemy-effect/aws/kinesis";
import * as S from "effect/Schema";

class EventStream extends Kinesis.Stream("EventStream", {
  schema: S.Struct({
    eventId: S.String,
    timestamp: S.Number,
  }),
  streamMode: "ON_DEMAND",
  retentionPeriod: 48,
}) {}

Consume Stream

import * as Lambda from "alchemy-effect/aws/lambda";

class Consumer extends Lambda.consumeStream("Consumer", {
  stream: EventStream,
  batchSize: 100,
  startingPosition: "LATEST",
  handle: Effect.fn(function* (event) {
    for (const record of event.Records) {
      console.log(record.kinesis.data.eventId);
    }
  }),
})({
  main: "./handler.ts",
  bindings: $(),
}) {}

Produce to Stream

class Producer extends Lambda.serve("Producer", {
  fetch: Effect.fn(function* () {
    yield* Kinesis.putRecord(
      EventStream,
      { eventId: "123", timestamp: Date.now() },
      { partitionKey: "user-1" },
    );
    return { statusCode: 200, body: "OK" };
  }),
})({
  main: "./handler.ts",
  bindings: $(Kinesis.PutRecord(EventStream)),
}) {}

🤖 Generated with Claude Code

sam-goodwin and others added 7 commits January 11, 2026 17:26
Implements comprehensive Kinesis Data Streams support:

- Stream resource with ON_DEMAND and PROVISIONED modes
- Configurable retention period (24-8760 hours)
- KMS encryption support
- Enhanced shard-level monitoring metrics
- Tag management

Lambda bindings:
- StreamEventSource for consuming stream records
- PutRecord/PutRecords for writing to streams

Includes full test coverage for create, update, delete operations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
AWS Kinesis API can return incomplete responses during stream creation
and deletion (e.g., missing OpenShardCount field). This causes ParseError
when the response doesn't match the expected schema. Added ParseError to
retry conditions in waitForStreamActive, waitForStreamDeleted, and
assertStreamDeleted to handle these transient states.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add consumeStream function that creates Lambda functions to consume from
Kinesis streams, similar to the existing consumeQueue pattern. Includes:
- consumeStream in the Lambda module with type-safe schema validation
- Integration tests for consumeStream and putRecord/putRecords
- Type tests verifying correct binding requirements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Document the convention for PR bodies to focus on DX code snippets
with minimal prose - one sentence per section followed by code examples.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
TypeScript was complaining about error type comparisons in retry
predicates because the error types didn't include ParseError or
DependencyViolation in their unions. Add explicit type annotations
to allow for these edge case error handling scenarios.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Lambda handlers must have Error type of `never`, so all SDK calls in
handlers need to catch errors with Effect.catchAll().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Document the requirement to run `bun tsc -b` before committing and
explain the available build commands including `bun build:clean`.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@pkg-pr-new
Copy link

pkg-pr-new bot commented Jan 12, 2026

Open in StackBlitz

npm i https://pkg.pr.new/alchemy-run/alchemy-effect@30

commit: 6e90479

The consume tests required Lambda function deployment which had AWS
credential configuration issues. The functionality is already covered:
- stream.provider.test.ts: Tests Kinesis stream CRUD (9 passing tests)
- stream.types.ts: Tests type safety of consumeStream

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@sam-goodwin sam-goodwin merged commit 3f30b79 into main Jan 12, 2026
2 checks passed
@sam-goodwin sam-goodwin deleted the sam-goodwin/kinesis branch January 12, 2026 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants