Fix SQLFluff test and derived tables copying

[alistairewj]

• SQLFluff is now on version 3.5.0, we were using 1.4.5 (!). There are some new rules that are inherited so new PRs will have to update SQL as needed (either indentation change or capitalizing a boolean/literal)
• The derived tables for v3.1 were not fully copied because bq ls paginates the first 50 in the shell script. it's now at 500. if we create more than 500 concept tables this bug will reemerge :)

[Copilot]

Pull Request Overview

This PR modernizes the SQLFluff linting configuration and tooling by upgrading to SQLFluff 3.5.0 and updating related dependencies. The changes include reformatting SQL files to comply with new linting rules and improving the GitHub Actions workflow.

• Updates SQLFluff configuration to use the new rule naming convention (e.g., capitalisation.keywords instead of L010)
• Upgrades GitHub Actions dependencies and Python version (3.7 → 3.12)
• Adds pagination support to BigQuery CLI commands in shell scripts

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
.sqlfluff	Modernized configuration with new rule naming scheme and structure
.github/workflows/lint_sqlfluff.yml	Updated action versions, Python version, SQLFluff version, and improved shell script logic
mimic-iv/concepts/medication/arb.sql	Reformatted CASE statement with consolidated OR conditions across multiple lines
mimic-iv/concepts/firstday/first_day_bg_art.sql	Reformatted JOIN ON clause to place ON keyword on its own line
mimic-iv/concepts/copy_concepts_to_versioned_schema.sh	Added -n 500 pagination flag to bq ls command

Comments suppressed due to low confidence (1)

mimic-iv/concepts/copy_concepts_to_versioned_schema.sh:16

• The bq ls command at line 16 should also include the -n 500 flag for consistency with line 30. Without pagination flags, bq ls may have a default limit that could miss tables if the dataset contains many tables. Apply the same pagination flag as used in the source dataset listing.

    for TABLE in `bq ls ${PROJECT_ID}:${TARGET_DATASET} | cut -d' ' -f3`;

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.