chore: init preview deployments & integration tests#655
Open
charlietlamb wants to merge 16 commits intodevfrom
Open
chore: init preview deployments & integration tests#655charlietlamb wants to merge 16 commits intodevfrom
charlietlamb wants to merge 16 commits intodevfrom
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Tip New to Tusk? Learn more here. View check history
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
railway.json
Outdated
| "environments": { | ||
| "pr": { | ||
| "deploy": { | ||
| "startCommand": "chmod +x /app/scripts/preview/start.sh && /app/scripts/preview/start.sh" |
Contributor
There was a problem hiding this comment.
/app/scripts/preview/start.sh does not exist in the repository
Prompt To Fix With AI
This is a comment left during a code review.
Path: railway.json
Line: 15:15
Comment:
`/app/scripts/preview/start.sh` does not exist in the repository
How can I resolve this? If you propose a fix, please make it concise.![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
3 issues found across 9 files
Confidence score: 2/5
- The preview environment will likely fail because
railway.jsonpoints startCommand to/app/scripts/preview/start.sh, which is missing in the repo, so deployment would break. - There are notable security concerns:
scripts/preview/setupTestOrgCI.tslogs API key data and.github/workflows/integration-tests.ymlpipes a remote install script directly into sudo bash. - Pay close attention to
railway.json,scripts/preview/setupTestOrgCI.ts,.github/workflows/integration-tests.yml- deployment failure and CI secret/supply‑chain risks.
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="scripts/preview/setupTestOrgCI.ts">
<violation number="1" location="scripts/preview/setupTestOrgCI.ts:22">
P2: Avoid logging the API key (even partially) in CI logs; this can leak credentials.</violation>
</file>
<file name=".github/workflows/integration-tests.yml">
<violation number="1" location=".github/workflows/integration-tests.yml:28">
P2: Avoid piping a remote install script directly into sudo bash without verification; it exposes the workflow to supply‑chain/script tampering risks.</violation>
</file>
<file name="railway.json">
<violation number="1" location="railway.json:15">
P1: The startCommand references `/app/scripts/preview/start.sh` but this file does not exist in the repository. The preview environment deployment will fail when Railway attempts to execute this non-existent script. Either add the missing `start.sh` file or update the startCommand to use an existing script.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
|
🚅 Deployed to the autumn-pr-655 environment in autumn
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by cubic
Automates PR preview deployments to Railway and runs integration tests against the preview URL. Includes Stripe webhook setup/cleanup, test org provisioning, and test config updates to support SERVER_URL.
New Features
Refactors
Written for commit 0602951. Summary will update on new commits.
Greptile Overview
Greptile Summary
This PR establishes automated preview deployments and integration testing for pull requests. When a PR is opened or updated, the system deploys a preview environment on Railway, waits for it to become healthy, then runs integration tests against it.
Improvements
mainordevbranchesSERVER_URLenvironment variableKey Components
preview-deploy.ymlorchestrates Railway deployment and health checksintegration-tests.ymlruns test groups in parallel and reports results to PRConfidence Score: 2/5
start.shscript that will cause preview environment deployments to fail. The workflow and script logic are otherwise sound, but this blocking issue needs to be resolved before merge.railway.json- it references a non-existent start scriptImportant Files Changed
Sequence Diagram
sequenceDiagram participant Dev as Developer participant GH as GitHub participant Preview as preview-deploy.yml participant Railway as Railway participant IntTest as integration-tests.yml participant Stripe as Stripe API Dev->>GH: Push to PR GH->>Preview: Trigger workflow (on PR open/sync) Preview->>Railway: Deploy preview environment Railway->>Railway: Build Docker image Railway->>Railway: Run start.sh (pr env) Railway-->>Preview: Return service domain Preview->>Preview: Wait for /health endpoint (30 attempts) Preview->>GH: Comment with preview URL Preview->>IntTest: Trigger integration-tests.yml IntTest->>IntTest: Run test-group-1 (Balances) IntTest->>IntTest: Run test-group-2 (Attach) in parallel IntTest->>Stripe: Setup webhook via setupPreviewWebhook.ts Stripe-->>IntTest: Return webhook ID & secret IntTest->>IntTest: Store encrypted secret in DB IntTest->>IntTest: Run tests against SERVER_URL IntTest->>Stripe: Cleanup webhook via cleanupPreviewWebhook.ts IntTest->>GH: Post test results as comment