Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updated sample_data import script, docs updated, githhub action to test #3146

Merged

Conversation

gautam-divyanshu
Copy link
Member

@gautam-divyanshu gautam-divyanshu commented Feb 5, 2025

What kind of change does this PR introduce?

Issue Number:

Fixes #2841

Snapshots/Videos:
image
image
image

If relevant, did you update the documentation?

Summary

Does this PR introduce a breaking change?

Checklist

CodeRabbit AI Review

  • I have reviewed and addressed all critical issues flagged by CodeRabbit AI
  • I have implemented or provided justification for each non-critical suggestion
  • I have documented my reasoning in the PR comments where CodeRabbit AI suggestions were not implemented

Test Coverage

  • I have written tests for all new changes/features
  • I have verified that test coverage meets or exceeds 95%
  • I have run the test suite locally and all tests pass

Other information

Have you read the contributing guide?

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Added a new job for importing sample data in the GitHub Actions workflow.
    • Introduced a new section in the documentation for importing sample data into the database.
  • Chores

    • Enhanced environment configuration management by adding a new dependency for environment variable management.
  • Bug Fixes

    • Improved data insertion logic with validation checks for user and organization IDs.

Copy link

coderabbitai bot commented Feb 5, 2025

Warning

Rate limit exceeded

@gautam-divyanshu has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 9 minutes and 30 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 97d2249 and 8bf1d50.

📒 Files selected for processing (2)
  • .github/workflows/pull-request.yml (1 hunks)
  • test/routes/graphql/gql.tada.d.ts (1 hunks)

Walkthrough

This PR introduces a new job named Import-Sample-Data to the GitHub Actions workflow, which runs on an Ubuntu environment. The job includes steps for checking out the repository, creating a .env file for the Talawa API, building a non-production Docker image, and importing sample data into the Docker container. Additionally, the package.json file has been updated to include the dotenv dependency, and the loadSampleData.ts file has been modified to utilize dotenv for loading environment variables and to implement error checks in the data insertion process.

Changes

File(s) Change Summary
.github/workflows/pull-request.yml Added new job Import-Sample-Data that checks out the repo, creates a .env file, builds a Docker image, and runs a command for data import.
package.json Added the dotenv dependency ("dotenv": "^16.0.3") in the dependencies section.
src/utilities/loadSampleData.ts Integrated dotenv to load environment variables and introduced error handling in insertCollections to ensure userIds and organizationIds are populated.
docs/docs/docs/getting-started/installation.md Added a new section titled "Import Sample Data" with instructions for importing sample data into the database.

Assessment against linked issues

Objective Addressed Explanation
Create an equivalent sample database for importation by the setup script (#2841)

Possibly related PRs

  • Fixed bugs to ensure proper flow of prompts and remove repetitive prompts #2845: The changes in the main PR, which introduce a new job for importing sample data into a Docker container, are related to the modifications in the retrieved PR that enhance data importation logic and error handling in the setup process.
  • Update push.yml #2900: The changes in the main PR, specifically the addition of a job to import sample data and the creation of a .env file, are related to the modifications in the retrieved PR, which also involves creating a .env file for the Talawa API testing environment.
  • fixed loadSampleData.ts and added tests  #2807: The changes in the main PR, which involve adding a job to import sample data and utilizing the dotenv package, are related to the modifications in the retrieved PR that focus on enhancing the functionality for loading and verifying sample data.

Suggested reviewers

  • palisadoes

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 815c423 and 5394a43.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (3)
  • .github/workflows/pull-request.yml (1 hunks)
  • package.json (1 hunks)
  • src/utilities/loadSampleData.ts (3 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/pull-request.yml

[error] 326-326: no new line character at the end of file

(new-line-at-end-of-file)

🔇 Additional comments (4)
src/utilities/loadSampleData.ts (2)

1-2: LGTM! Environment variables configuration added.

The dotenv configuration is correctly implemented at the start of the file.


152-156: LGTM! Added data integrity checks.

The error checks ensure that users and organizations exist before inserting posts and events, preventing potential data integrity issues.

Also applies to: 180-184

package.json (1)

20-20: Verify dotenv package version.

The current version (^16.0.3) might not be the latest. Let's check for any updates or security advisories.

✅ Verification successful

🌐 Web query:

What is the latest version of dotenv npm package?

Length of output: 2215


🏁 Script executed:

#!/bin/bash
# This command queries the npm registry for the latest published version of the "dotenv" package.
npm view dotenv version

Length of output: 32


dotenv version update verification

The latest version of the dotenv npm package is 16.4.7. Your package.json specifies "^16.0.3", which allows minor/patch upgrades within the major version 16. Consequently, a fresh installation should resolve to 16.4.7 unless the lock file prevents it.

.github/workflows/pull-request.yml (1)

315-326: LGTM! Sample data import test job added.

The job configuration is well-structured and follows the project's patterns for Docker-based testing.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 326-326: no new line character at the end of file

(new-line-at-end-of-file)

.github/workflows/pull-request.yml Outdated Show resolved Hide resolved
@gautam-divyanshu gautam-divyanshu removed the request for review from palisadoes February 5, 2025 18:24
coderabbitai[bot]
coderabbitai bot previously approved these changes Feb 5, 2025
@gautam-divyanshu
Copy link
Member Author

@palisadoes most of the action are failing because of that corepack issue

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5394a43 and c91ed6d.

📒 Files selected for processing (1)
  • docs/docs/docs/getting-started/installation.md (1 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.17.2)
docs/docs/docs/getting-started/installation.md

297-297: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Above

(MD022, blanks-around-headings)


301-301: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)


303-303: Trailing spaces
Expected: 0 or 2; Actual: 1

(MD009, no-trailing-spaces)


303-303: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)


304-304: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)

docs/docs/docs/getting-started/installation.md Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
docs/docs/docs/getting-started/installation.md (1)

298-299: 🧹 Nitpick (assertive)

Ensure Proper Markdown Heading Spacing

The newly added "#### Import Sample Data" heading should be surrounded by blank lines (both before and after) to comply with markdown lint guidelines (MD022) and improve overall readability.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c91ed6d and bac6512.

📒 Files selected for processing (1)
  • docs/docs/docs/getting-started/installation.md (1 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.17.2)
docs/docs/docs/getting-started/installation.md

302-302: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)


304-304: Trailing spaces
Expected: 0 or 2; Actual: 1

(MD009, no-trailing-spaces)


304-304: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)


305-305: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)

⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: Analyse Code With CodeQL (typescript)
🔇 Additional comments (1)
docs/docs/docs/getting-started/installation.md (1)

302-306: 🧹 Nitpick (assertive)

Close Fenced Code Blocks and Add Surrounding Blank Lines

The fenced code blocks for the shell commands in steps 2 and 3 are missing their closing triple backticks. Additionally, per MD031, these code blocks should be surrounded by blank lines. Please add a closing fence after the command in each block. For example:

@@ -303,0
+   ```

and

@@ -306,0
+   ```

This will ensure proper formatting of the shell command examples.

Likely invalid or redundant comment.

🧰 Tools
🪛 markdownlint-cli2 (0.17.2)

302-302: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)


304-304: Trailing spaces
Expected: 0 or 2; Actual: 1

(MD009, no-trailing-spaces)


304-304: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)


305-305: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)

coderabbitai[bot]
coderabbitai bot previously approved these changes Feb 5, 2025
@palisadoes
Copy link
Contributor

  1. Please fix the conflicting file.
  2. The broken corepack pull request errors have been fixed

Copy link

codecov bot commented Feb 6, 2025

Codecov Report

Attention: Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.

Project coverage is 39.71%. Comparing base (6233dca) to head (8bf1d50).
Report is 1 commits behind head on develop-postgres.

Files with missing lines Patch % Lines
src/utilities/loadSampleData.ts 0.00% 12 Missing ⚠️
Additional details and impacted files
@@                 Coverage Diff                  @@
##           develop-postgres    #3146      +/-   ##
====================================================
- Coverage             39.72%   39.71%   -0.02%     
====================================================
  Files                   455      455              
  Lines                 33541    33553      +12     
  Branches                407      407              
====================================================
  Hits                  13324    13324              
- Misses                20217    20229      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

coderabbitai[bot]
coderabbitai bot previously approved these changes Feb 6, 2025
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

♻️ Duplicate comments (1)
docs/docs/docs/getting-started/installation.md (1)

298-298: 🧹 Nitpick (assertive)

Markdown Heading Formatting

The heading "#### Import Sample Data" should be preceded (and ideally followed) by a blank line for better readability and to comply with markdown lint guidelines (MD022). Please insert a blank line before this heading.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b05562d and 09e3b19.

📒 Files selected for processing (2)
  • .github/workflows/pull-request.yml (1 hunks)
  • docs/docs/docs/getting-started/installation.md (1 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.17.2)
docs/docs/docs/getting-started/installation.md

305-305: Fenced code blocks should be surrounded by blank lines
null

(MD031, blanks-around-fences)


306-306: Trailing spaces
Expected: 0 or 2; Actual: 1

(MD009, no-trailing-spaces)

🪛 YAMLlint (1.35.1)
.github/workflows/pull-request.yml

[error] 332-332: no new line character at the end of file

(new-line-at-end-of-file)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: Run tests for talawa api
  • GitHub Check: Analyse Code With CodeQL (typescript)

.github/workflows/pull-request.yml Outdated Show resolved Hide resolved
.github/workflows/pull-request.yml Outdated Show resolved Hide resolved
coderabbitai[bot]
coderabbitai bot previously approved these changes Feb 6, 2025
@palisadoes
Copy link
Contributor

@gautam-divyanshu
Copy link
Member Author

yes, it is related to docker

@palisadoes
Copy link
Contributor

None of the other GitHub actions use the Dev container env file. Use the ci env file

@gautam-divyanshu
Copy link
Member Author

But when I use Docker Compose, it will still be unhealthy.

@gautam-divyanshu
Copy link
Member Author

same error using reverted pr,
image

@gautam-divyanshu
Copy link
Member Author

I think it was working fine earlier.

.github/workflows/pull-request.yml Outdated Show resolved Hide resolved
.github/workflows/pull-request.yml Outdated Show resolved Hide resolved
@gautam-divyanshu
Copy link
Member Author

I am getting this error with upstream also; can you also try?
I used Palisadoes branch, not my forked
image

@gautam-divyanshu
Copy link
Member Author

why is it skipping?

@palisadoes
Copy link
Contributor

I think it's related to the API running health checks on all the other processes but not running it on itself. There is an explanation from @xoldd in the #talawa-apin slack channel

Is it hanging at that test? If it's not, then it's OK

@gautam-divyanshu
Copy link
Member Author

Ok, this is how it's supposed to be.

@palisadoes
Copy link
Contributor

https://palisadoes-foundation.slack.com/archives/CSWH4PN0M/p1737570828958939?thread_ts=1737527664.171939&cid=CSWH4PN0M

@palisadoes
Copy link
Contributor

This one requires a container statement for the job.

This one uses a GitHub action which probably does the same thing

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3f53fd8 and 8d79db1.

📒 Files selected for processing (1)
  • .github/workflows/pull-request.yml (1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.35.1)
.github/workflows/pull-request.yml

[error] 326-326: no new line character at the end of file

(new-line-at-end-of-file)

⏰ Context from checks skipped due to timeout of 90000ms (1)
🔇 Additional comments (1)
.github/workflows/pull-request.yml (1)

326-326: Verify Container Name Consistency
Double-check that the container name talawa-api-1 in the docker exec command matches the name defined by your Docker Compose configuration. Mismatched names might cause the command to fail unexpectedly.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 326-326: no new line character at the end of file

(new-line-at-end-of-file)

- name: Build talawa api non production environment docker image
run: docker compose build
- name: Import sample data
run: docker exec talawa-api-1 /bin/bash -c "pnpm run import:sample-data"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick (assertive)

Ensure Newline at End of File
A newline character is missing at the end of the file. Adding a newline will comply with YAML best practices and avoid linting errors.

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 326-326: no new line character at the end of file

(new-line-at-end-of-file)

Comment on lines 315 to 326
Import-Sample-Data:
name: Import Sample Data
runs-on: ubuntu-latest
steps:
- name: Checkout this repository
uses: actions/checkout@v4.2.2
- name: Create .env file for talawa api testing environment
run: cp ./envFiles/.env.ci ./.env
- name: Build talawa api non production environment docker image
run: docker compose build
- name: Import sample data
run: docker exec talawa-api-1 /bin/bash -c "pnpm run import:sample-data"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Container Startup Step Missing in "Import-Sample-Data" Job
Currently, the "Import-Sample-Data" job only runs the command docker compose build but does not start the container before executing the sample data import via docker exec. If the container is not running when the docker exec command is issued, the job will likely fail. Consider adding a step to start the container (e.g., using docker compose up -d) right after building the image.

Proposed diff:

-      - name: Build talawa api non production environment docker image
-        run: docker compose build
+      - name: Build and start talawa api container
+        run: |
+          docker compose build
+          docker compose up -d
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Import-Sample-Data:
name: Import Sample Data
runs-on: ubuntu-latest
steps:
- name: Checkout this repository
uses: actions/checkout@v4.2.2
- name: Create .env file for talawa api testing environment
run: cp ./envFiles/.env.ci ./.env
- name: Build talawa api non production environment docker image
run: docker compose build
- name: Import sample data
run: docker exec talawa-api-1 /bin/bash -c "pnpm run import:sample-data"
Import-Sample-Data:
name: Import Sample Data
runs-on: ubuntu-latest
steps:
- name: Checkout this repository
uses: actions/checkout@v4.2.2
- name: Create .env file for talawa api testing environment
run: cp ./envFiles/.env.ci ./.env
- name: Build and start talawa api container
run: |
docker compose build
docker compose up -d
- name: Import sample data
run: docker exec talawa-api-1 /bin/bash -c "pnpm run import:sample-data"
🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 326-326: no new line character at the end of file

(new-line-at-end-of-file)

@PalisadoesFoundation PalisadoesFoundation deleted a comment from github-actions bot Feb 6, 2025
@palisadoes
Copy link
Contributor

This one requires a container statement for the job.

This one uses a GitHub action which probably does the same thing

Did you see this?

@gautam-divyanshu
Copy link
Member Author

This one requires a container statement for the job.

This one uses a GitHub action which probably does the same thing

Did you see this?

yes, pushing the commits

@palisadoes
Copy link
Contributor

@palisadoes
Copy link
Contributor

The job ID would be Import-Sample-Data

@gautam-divyanshu
Copy link
Member Author

Can we merge this without GitHub action so that other guys can work? I will create a new PR for GitHub action.

@PalisadoesFoundation PalisadoesFoundation deleted a comment from github-actions bot Feb 6, 2025
@PalisadoesFoundation PalisadoesFoundation deleted a comment from github-actions bot Feb 6, 2025
@palisadoes
Copy link
Contributor

Comment out the GitHub action.
Then I'll merge

@PalisadoesFoundation PalisadoesFoundation deleted a comment from github-actions bot Feb 6, 2025
@palisadoes palisadoes merged commit 8dca2ac into PalisadoesFoundation:develop-postgres Feb 6, 2025
15 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ignore-sensitive-files-pr The contributor has a legitimate reason for editiing protected files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants