diff --git a/.eslintrc.json b/.eslintrc.json index 513b251a7..0dbac2149 100644 --- a/.eslintrc.json +++ b/.eslintrc.json @@ -13,7 +13,7 @@ "plugins": ["@malloydata/lint"], "rules": { "no-console": "warn", - "prettier/prettier": ["error", {"quoteProps": "preserve"}], + "prettier/prettier": ["off", {"quoteProps": "preserve"}], "sort-keys": "off", "no-duplicate-imports": "error", "no-restricted-imports": [ diff --git a/.prettierignore b/.prettierignore new file mode 100644 index 000000000..92efc87d8 --- /dev/null +++ b/.prettierignore @@ -0,0 +1,3 @@ +build/ +packages/malloy-malloy-sql/src/grammar +packages/malloy-syntax-highlight/**/*.monarch.ts \ No newline at end of file diff --git a/.vscode/tasks.json b/.vscode/tasks.json index 37c8f1f36..3afcee99c 100644 --- a/.vscode/tasks.json +++ b/.vscode/tasks.json @@ -10,7 +10,7 @@ "problemMatcher": ["$tsc-watch"], "group": "build", "label": "tsc-compile-watch", - "runOptions": { "runOn": "folderOpen" }, + "runOptions": {"runOn": "folderOpen"}, "presentation": { "reveal": "silent", "revealProblems": "onProblem", diff --git a/CHANGELOG.md b/CHANGELOG.md index 916dade6f..c46c0ff56 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,6 @@ # Change Log -_Breaking changes indicated with *_ + +_Breaking changes indicated with \*_ We will use this space to highlight major and/or breaking changes to Malloy. @@ -7,11 +8,10 @@ We will use this space to highlight major and/or breaking changes to Malloy. ### VS Code helper view windows relocated -To improve discoverability and reduce extra clicking around in VS Code, we've relocated the Help, Connections, and Schema View Windows into the Explorer View. They will only appear when a `.malloy` file is opened. +To improve discoverability and reduce extra clicking around in VS Code, we've relocated the Help, Connections, and Schema View Windows into the Explorer View. They will only appear when a `.malloy` file is opened. Screen Shot 2022-08-08 at 11 32 26 AM - ## v0.0.10 ### The apply operator is now ? and not : @@ -43,6 +43,7 @@ sourceName { where: fieldName ? value1|value2 } Prior to v0.0.9, lists of things were contained inside `[ ]`. Going forward, the brackets have been removed. Our hope is that this will be one less piece of punctuation to track, and will make it easier to change from a single item in a list to multiple without adding in brackets. For example, this syntax: + ```malloy query: table('malloy-data.faa.airports') -> { top: 10 @@ -64,6 +65,7 @@ query: table('malloy-data.faa.airports') -> { ``` Is now written: + ```malloy query: table('malloy-data.faa.airports') -> { top: 10 diff --git a/README.md b/README.md index 008ae63ad..be85566f3 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ # Malloy + Malloy is an experimental language for describing data relationships and transformations. It is both a semantic modeling language and a querying language that runs queries against a relational database. Malloy currently supports BigQuery and Postgres, as well as querying Parquet and CSV files via DuckDB. ### [Click here](https://github.dev/malloydata/try-malloy/airports.malloy) to try Malloy in your browser! @@ -9,16 +10,16 @@ Malloy is an experimental language for describing data relationships and transfo The easiest way to try Malloy is with our VS Code Extension, which provides a place to create Malloy models, execute queries, get help, and more. VS Code is a text editor and IDE (integrated development environment) that runs on your desktop or in your browser. A few ways to install the extension: -* [I already have VS Code](https://malloydata.github.io/documentation/setup/extension.html#using-the-malloy-extension-on-your-desktop) -* [I use BigQuery and Google Cloud](https://malloydata.github.io/documentation/setup/extension.html#using-the-malloy-extension-on-google-cloud-shell-editor). -* [I have a Github account and want to try Malloy on a `.csv` or `.parquet` file in a repository](https://malloydata.github.io/documentation/setup/extension.html#using-the-malloy-extension-on-github-dev). +- [I already have VS Code](https://malloydata.github.io/documentation/setup/extension.html#using-the-malloy-extension-on-your-desktop) +- [I use BigQuery and Google Cloud](https://malloydata.github.io/documentation/setup/extension.html#using-the-malloy-extension-on-google-cloud-shell-editor). +- [I have a Github account and want to try Malloy on a `.csv` or `.parquet` file in a repository](https://malloydata.github.io/documentation/setup/extension.html#using-the-malloy-extension-on-github-dev). ![show_run](https://user-images.githubusercontent.com/1093458/182458787-ca228186-c954-4a07-b298-f92dbf91e48d.gif) To get to know the Malloy language, follow [our Quickstart](https://malloydata.github.io/documentation/user_guides/basic.html). Note: The Malloy VSCode Extension tracks a small amount of anonymous usage data. You can opt out in the extension settings. - [Learn more](https://policies.google.com/technologies/cookies). +[Learn more](https://policies.google.com/technologies/cookies). ## Join the Community @@ -44,6 +45,7 @@ To report security issues please see our [security policy](https://github.com/ma Malloy is not an officially supported Google product. ## Syntax Example + Here is a simple example of a Malloy query: ```malloy @@ -57,6 +59,7 @@ run: bigquery.table('malloy-data.faa.flights') -> { ``` In SQL this would be expressed: + ```sql SELECT carrier, diff --git a/cloudbuild/build-test/test.sh b/cloudbuild/build-test/test.sh index dba38d9cd..10ab3b40b 100755 --- a/cloudbuild/build-test/test.sh +++ b/cloudbuild/build-test/test.sh @@ -4,6 +4,6 @@ set -euxo pipefail nix-shell --quiet --pure --keep PGHOST --keep PGDATABASE --keep PGUSER --command "$(cat < /dev/null & sleep 2) && (./cloudbuild/deploy/$TRIGGER_NAME.sh)" - timeout: "1800s" - secretEnv: ["NPM_TOKEN", "GITHUB_SSH_KEY"] + - '(/workspace/cloud_sql_proxy -enable_iam_login -instances=malloy-303216:us-central1:pg-test=tcp:5432 -log_debug_stdout > /dev/null & sleep 2) && (./cloudbuild/deploy/$TRIGGER_NAME.sh)' + timeout: '1800s' + secretEnv: ['NPM_TOKEN', 'GITHUB_SSH_KEY'] env: - - "BRANCH_NAME=$BRANCH_NAME" + - 'BRANCH_NAME=$BRANCH_NAME' volumes: - - name: "ssh" + - name: 'ssh' path: /root/.ssh - waitFor: ["git-ssh-key", "proxy-install"] + waitFor: ['git-ssh-key', 'proxy-install'] options: logging: CLOUD_LOGGING_ONLY - machineType: "E2_HIGHCPU_32" -timeout: "1800s" + machineType: 'E2_HIGHCPU_32' +timeout: '1800s' availableSecrets: secretManager: - versionName: projects/malloy-303216/secrets/npm-publish/versions/latest - env: "NPM_TOKEN" + env: 'NPM_TOKEN' - versionName: projects/malloy-303216/secrets/github-ssh-key/versions/latest - env: "GITHUB_SSH_KEY" + env: 'GITHUB_SSH_KEY' diff --git a/developing.md b/developing.md index a9463394c..8cc4cbf5a 100644 --- a/developing.md +++ b/developing.md @@ -11,7 +11,13 @@ npm install npm run build ``` -Use the [VS Code ESLint extension](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) for code formatting, or run `npm run lint --fix`. +Use the [VS Code ESLint extension](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) for highlighting linter errors, or run `npm run lint --fix`. + +Use the [VS Code Prettier extension](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode) for code formatting, or run `npm run prettier-fix`. +To auto-format files on save: + +1. Go to Settings > Workspace, search for "format on save", and turn "Editor: Format on Save" to on +2. Go to Settings > Workspace, search for "formatter", and set "Editor: Default formatter" to Prettier - Code formatter" The tests can be run via the [VS Code Jest Runner extension](https://marketplace.visualstudio.com/items?itemName=firsttris.vscode-jest-runner). Alternatively, use `npm run test`, or a specific file, like `npm run test test/nomodel.spec.ts`. diff --git a/package.json b/package.json index 45684901e..e12f011c8 100644 --- a/package.json +++ b/package.json @@ -31,6 +31,8 @@ "lint-build": "npm run build -w packages/malloy -w packages/malloy-lint", "lint": "npm run lint-build && gts lint", "lint-fix": "npm run lint-build && gts fix", + "prettier-fix": "prettier . --write", + "prettier-check": "prettier . --check", "test": "jest --runInBand", "test-bigquery": "MALLOY_DATABASE=bigquery jest --runInBand", "test-postgres": "MALLOY_DATABASE=postgres jest --runInBand", diff --git a/packages/malloy-interfaces/README.md b/packages/malloy-interfaces/README.md index 2cacae0fb..5f9171b75 100644 --- a/packages/malloy-interfaces/README.md +++ b/packages/malloy-interfaces/README.md @@ -1,3 +1,3 @@ # Malloy Interfaces -This package contains most of the main definitions of the interfaces that are relevant for packages that implement Malloy Connections. \ No newline at end of file +This package contains most of the main definitions of the interfaces that are relevant for packages that implement Malloy Connections. diff --git a/packages/malloy-syntax-highlight/.github/README.md b/packages/malloy-syntax-highlight/.github/README.md index 8a97c43d0..e267789ef 100644 --- a/packages/malloy-syntax-highlight/.github/README.md +++ b/packages/malloy-syntax-highlight/.github/README.md @@ -6,4 +6,4 @@ Currently, two other dialects of Malloy are supported in addition to the standar Thus, this package can be pulled into only the Malloy repos that need it and includes a test runner to verify the semantic and visual accuracy of syntax highlighting provided by our TextMate grammars. Additionally, the need to maintain a Monarch grammar for Malloy has necessitated the inclusion of both a script to generate Monarch grammars from ground truth TextMate grammars as well as infrastructure to test all Monarch grammars for parity with their TextMate counterparts. -Please refer to `DEVELOPING.md` for more information on using the tools introduced in this package. \ No newline at end of file +Please refer to `DEVELOPING.md` for more information on using the tools introduced in this package. diff --git a/packages/malloy-syntax-highlight/README.md b/packages/malloy-syntax-highlight/README.md index d1d4f7772..f7020ec2f 100644 --- a/packages/malloy-syntax-highlight/README.md +++ b/packages/malloy-syntax-highlight/README.md @@ -4,4 +4,4 @@ Malloy is an experimental language for describing data relationships and transfo Currently, two other dialects of Malloy are supported in addition to the standard syntax used in `.malloy` files: the Malloy notebook format (`.malloynb`) and Malloy SQL (`.malloysql`). The purpose of this `npm` package is to mitigate syncing issues across `@malloydata` repositories that depend upon syntax highlight files and to make these assets more easily available for future integrations. -This `npm` package does not include the source files used to develop these syntax highlighting files. If you are interested in using the existing infrastructure to modify, test, or generate these grammar files, please refer to the [malloy monorepo](https://github.com/malloydata/malloy/tree/main/packages/malloy-syntax-highlight) and see `DEVELOPING.md` for steps to do so. \ No newline at end of file +This `npm` package does not include the source files used to develop these syntax highlighting files. If you are interested in using the existing infrastructure to modify, test, or generate these grammar files, please refer to the [malloy monorepo](https://github.com/malloydata/malloy/tree/main/packages/malloy-syntax-highlight) and see `DEVELOPING.md` for steps to do so. diff --git a/packages/malloy-syntax-highlight/grammars/malloy-notebook/malloy-notebook.tmGrammar.json b/packages/malloy-syntax-highlight/grammars/malloy-notebook/malloy-notebook.tmGrammar.json index 6d49623fe..f2d42cae0 100644 --- a/packages/malloy-syntax-highlight/grammars/malloy-notebook/malloy-notebook.tmGrammar.json +++ b/packages/malloy-syntax-highlight/grammars/malloy-notebook/malloy-notebook.tmGrammar.json @@ -2,30 +2,30 @@ "name": "Malloy Notebook", "scopeName": "source.malloy-notebook", "patterns": [ - { "include": "#comments" }, - { "include": "#malloysql-malloy" }, - { "include": "#malloysql-sql" } + {"include": "#comments"}, + {"include": "#malloysql-malloy"}, + {"include": "#malloysql-sql"} ], "repository": { "malloysql-malloy": { "begin": ">>>malloy", "end": "(?=>>>)", "beginCaptures": { - "0": { "name": "entity.other.attribute.malloy-sql" } + "0": {"name": "entity.other.attribute.malloy-sql"} }, "name": "meta.embedded.block.malloysql.malloy", - "patterns": [{ "include": "source.malloy" }] + "patterns": [{"include": "source.malloy"}] }, "malloysql-sql": { "begin": "(>>>sql)(\\s*connection:.*?(?>>)", "beginCaptures": { - "0": { "name": "entity.other.attribute.malloy-sql" }, - "1": { "name": "entity.other.attribute.malloy-sql" }, - "3": { "name": "comment.line.double-slash" } + "0": {"name": "entity.other.attribute.malloy-sql"}, + "1": {"name": "entity.other.attribute.malloy-sql"}, + "3": {"name": "comment.line.double-slash"} }, "name": "meta.embedded.block.malloysql.sql", - "patterns": [{ "include": "source.malloy-sql" }] + "patterns": [{"include": "source.malloy-sql"}] } } } diff --git a/packages/malloy-syntax-highlight/grammars/malloy-sql/malloy-sql.tmGrammar.json b/packages/malloy-syntax-highlight/grammars/malloy-sql/malloy-sql.tmGrammar.json index 48c6100c2..bce9e9e23 100644 --- a/packages/malloy-syntax-highlight/grammars/malloy-sql/malloy-sql.tmGrammar.json +++ b/packages/malloy-syntax-highlight/grammars/malloy-sql/malloy-sql.tmGrammar.json @@ -1,28 +1,28 @@ { "name": "Malloy SQL", "scopeName": "source.malloy-sql", - "patterns": [{ "include": "#malloysql-sql" }], + "patterns": [{"include": "#malloysql-sql"}], "repository": { "malloysql-sql": { "beginCaptures": { - "0": { "name": "entity.other.attribute.malloy-sql" }, - "1": { "name": "entity.other.attribute.malloy-sql" }, - "3": { "name": "comment.line.double-slash" } + "0": {"name": "entity.other.attribute.malloy-sql"}, + "1": {"name": "entity.other.attribute.malloy-sql"}, + "3": {"name": "comment.line.double-slash"} }, "name": "meta.embedded.block.malloysql.sql", "patterns": [ - { "include": "#comments" }, + {"include": "#comments"}, { "begin": "%{", "end": "}%", "beginCaptures": { - "0": { "name": "punctuation.definition.malloy-sql" } + "0": {"name": "punctuation.definition.malloy-sql"} }, "endCaptures": { - "0": { "name": "punctuation.definition.malloy-sql" } + "0": {"name": "punctuation.definition.malloy-sql"} }, "name": "meta.embedded.block.malloy", - "patterns": [{ "include": "source.malloy" }] + "patterns": [{"include": "source.malloy"}] }, { "include": "source.sql" @@ -35,10 +35,10 @@ "begin": "/\\*", "end": "\\*/", "beginCaptures": { - "0": { "name": "punctuation.definition.comment.begin" } + "0": {"name": "punctuation.definition.comment.begin"} }, "endCaptures": { - "0": { "name": "punctuation.definition.comment.end" } + "0": {"name": "punctuation.definition.comment.end"} }, "name": "comment.block" }, @@ -46,7 +46,7 @@ "begin": "//", "end": "\\n", "beginCaptures": { - "0": { "name": "punctuation.definition.comment" } + "0": {"name": "punctuation.definition.comment"} }, "name": "comment.line.double-slash" }, @@ -54,7 +54,7 @@ "begin": "--", "end": "\\n", "beginCaptures": { - "0": { "name": "punctuation.definition.comment" } + "0": {"name": "punctuation.definition.comment"} }, "name": "comment.line.double-hyphen" } diff --git a/packages/malloy-syntax-highlight/grammars/malloy/malloy.tmGrammar.json b/packages/malloy-syntax-highlight/grammars/malloy/malloy.tmGrammar.json index c94875fb3..5d9318bbb 100644 --- a/packages/malloy-syntax-highlight/grammars/malloy/malloy.tmGrammar.json +++ b/packages/malloy-syntax-highlight/grammars/malloy/malloy.tmGrammar.json @@ -1,31 +1,31 @@ { "scopeName": "source.malloy", - "patterns": [{ "include": "#malloy-language" }], + "patterns": [{"include": "#malloy-language"}], "repository": { "malloy-language": { "patterns": [ - { "include": "#sql-string" }, - { "include": "#comments" }, - { "include": "#tags" }, - { "include": "#strings" }, - { "include": "#numbers" }, - { "include": "#keywords" }, - { "include": "#properties" }, - { "include": "#functions" }, - { "include": "#datetimes" }, - { "include": "#identifiers-quoted" }, - { "include": "#types" }, - { "include": "#constants" }, - { "include": "#timeframes" }, - { "include": "#identifiers-unquoted" } + {"include": "#sql-string"}, + {"include": "#comments"}, + {"include": "#tags"}, + {"include": "#strings"}, + {"include": "#numbers"}, + {"include": "#keywords"}, + {"include": "#properties"}, + {"include": "#functions"}, + {"include": "#datetimes"}, + {"include": "#identifiers-quoted"}, + {"include": "#types"}, + {"include": "#constants"}, + {"include": "#timeframes"}, + {"include": "#identifiers-unquoted"} ] }, "malloy-matched": { "begin": "{", "end": "}", "patterns": [ - { "include": "#malloy-matched" }, - { "include": "#malloy-language" } + {"include": "#malloy-matched"}, + {"include": "#malloy-language"} ] }, "malloy-in-sql": { @@ -33,8 +33,8 @@ "name": "source.malloy-in-sql", "end": "}%?", "patterns": [ - { "include": "#malloy-matched" }, - { "include": "#malloy-language" } + {"include": "#malloy-matched"}, + {"include": "#malloy-language"} ] }, "sql-string": { @@ -43,34 +43,28 @@ "begin": "\\b(select)(\\s*:\\s*)(\"\"\")", "end": "\"\"\"", "beginCaptures": { - "1": { "name": "keyword.control.select" }, - "3": { "name": "punctuation.sql-block.open" } + "1": {"name": "keyword.control.select"}, + "3": {"name": "punctuation.sql-block.open"} }, "endCaptures": { - "0": { "name": "punctuation.sql-block.close" } + "0": {"name": "punctuation.sql-block.close"} }, "name": "source.sql", - "patterns": [ - { "include": "#malloy-in-sql" }, - { "include": "source.sql" } - ] + "patterns": [{"include": "#malloy-in-sql"}, {"include": "source.sql"}] }, { "begin": "(\\b[A-Za-z_][A-Za-z_0-9]*)(\\s*\\.\\s*)(sql)(\\s*\\(\\s*)(\"\"\")", "end": "\"\"\"", "beginCaptures": { - "1": { "name": "variable.other" }, - "3": { "name": "keyword.control.sql" }, - "5": { "name": "punctuation.sql-block.open" } + "1": {"name": "variable.other"}, + "3": {"name": "keyword.control.sql"}, + "5": {"name": "punctuation.sql-block.open"} }, "endCaptures": { - "0": { "name": "punctuation.sql-block.close" } + "0": {"name": "punctuation.sql-block.close"} }, "name": "source.sql", - "patterns": [ - { "include": "#malloy-in-sql" }, - { "include": "source.sql" } - ] + "patterns": [{"include": "#malloy-in-sql"}, {"include": "source.sql"}] } ] }, @@ -79,27 +73,27 @@ { "match": "(?i)\\b(count)(\\s*\\()(distinct)", "captures": { - "1": { "name": "entity.name.function" }, - "3": { "name": "entity.name.function.modifier" } + "1": {"name": "entity.name.function"}, + "3": {"name": "entity.name.function.modifier"} } }, { "match": "(?i)\\b(AVG|COUNT|FIRST|FORMAT|LAST|LCASE|LEN|MAX|MID|MIN|MOD|NOW|ROUND|SUM|UCASE|TABLE|FROM|FROM_SQL|UNGROUPED)(\\s*\\()", "captures": { - "1": { "name": "entity.name.function" } + "1": {"name": "entity.name.function"} } }, { "match": "(?i)\\b([a-zA-Z_][a-zA-Z_0-9]*)(\\s*\\()", "captures": { - "1": { "name": "entity.name.function" } + "1": {"name": "entity.name.function"} } }, { "match": "(?i)\\b([a-zA-Z_][a-zA-Z_0-9]*)(!)(timestamp|number|string|boolean|date)?(\\s*\\()", "captures": { - "1": { "name": "entity.name.function" }, - "3": { "name": "entity.name.type" } + "1": {"name": "entity.name.function"}, + "3": {"name": "entity.name.type"} } } ] @@ -150,10 +144,10 @@ "begin": "/\\*", "end": "\\*/", "beginCaptures": { - "0": { "name": "punctuation.definition.comment.begin" } + "0": {"name": "punctuation.definition.comment.begin"} }, "endCaptures": { - "0": { "name": "punctuation.definition.comment.end" } + "0": {"name": "punctuation.definition.comment.end"} }, "name": "comment.block" }, @@ -161,7 +155,7 @@ "begin": "//", "end": "\\n", "beginCaptures": { - "0": { "name": "punctuation.definition.comment" } + "0": {"name": "punctuation.definition.comment"} }, "name": "comment.line.double-slash" }, @@ -169,7 +163,7 @@ "begin": "--", "end": "\\n", "beginCaptures": { - "0": { "name": "punctuation.definition.comment" } + "0": {"name": "punctuation.definition.comment"} }, "name": "comment.line.double-hyphen" } @@ -181,14 +175,14 @@ "match": "##\\n", "name": "string.quoted", "captures": { - "0": { "name": "string.quoted" } + "0": {"name": "string.quoted"} } }, { "begin": "#\"", "end": "\\n", "beginCaptures": { - "0": { "name": "punctuation.definition.comment" } + "0": {"name": "punctuation.definition.comment"} }, "name": "comment.line.double-slash" }, @@ -196,32 +190,32 @@ "match": "#\\n", "name": "string.quoted", "captures": { - "0": { "name": "string.quoted" } + "0": {"name": "string.quoted"} } }, { "begin": "#\\s", "end": "\\n", "beginCaptures": { - "0": { "name": "support.type.property-name.json" } + "0": {"name": "support.type.property-name.json"} }, "name": "comment.line.double-slash", - "patterns": [{ "include": "#tag-values" }] + "patterns": [{"include": "#tag-values"}] }, { "begin": "##\\s", "end": "\\n", "beginCaptures": { - "0": { "name": "support.type.property-name.json" } + "0": {"name": "support.type.property-name.json"} }, "name": "comment.line.double-slash", - "patterns": [{ "include": "#tag-values" }] + "patterns": [{"include": "#tag-values"}] }, { "begin": "#", "end": "\\n", "beginCaptures": { - "0": { "name": "string.quoted" } + "0": {"name": "string.quoted"} }, "name": "string.quoted" } @@ -231,10 +225,10 @@ "name": "support.type.property-name.json", "match": "(-)?((?:[^\\s=#]+)|(?:\"[^#]+\"))(?:\\s*(=)\\s*((?:[^\\s=#]+)|(?:\"[^#]+\")))?", "captures": { - "1": { "name": "keyword.control.negate" }, - "2": { "name": "support.type.property-name.json" }, - "3": { "name": "keyword.operator.comparison.ts" }, - "4": { "name": "string.quoted" } + "1": {"name": "keyword.control.negate"}, + "2": {"name": "support.type.property-name.json"}, + "3": {"name": "keyword.operator.comparison.ts"}, + "4": {"name": "string.quoted"} } } } @@ -245,34 +239,34 @@ "begin": "'", "end": "'", "beginCaptures": { - "0": { "name": "punctuation.definition.string.begin" } + "0": {"name": "punctuation.definition.string.begin"} }, "endCaptures": { - "0": { "name": "punctuation.definition.string.end" } + "0": {"name": "punctuation.definition.string.end"} }, "name": "string.quoted.single", - "patterns": [{ "include": "#escapes" }] + "patterns": [{"include": "#escapes"}] }, { "begin": "\"", "end": "\"", "beginCaptures": { - "0": { "name": "punctuation.definition.string.begin" } + "0": {"name": "punctuation.definition.string.begin"} }, "endCaptures": { - "0": { "name": "punctuation.definition.string.end" } + "0": {"name": "punctuation.definition.string.end"} }, "name": "string.quoted.double", - "patterns": [{ "include": "#escapes" }] + "patterns": [{"include": "#escapes"}] }, { "begin": "\"\"\"", "end": "\"\"\"", "beginCaptures": { - "0": { "name": "punctuation.definition.string.begin" } + "0": {"name": "punctuation.definition.string.begin"} }, "endCaptures": { - "0": { "name": "punctuation.definition.string.end" } + "0": {"name": "punctuation.definition.string.end"} }, "name": "string.quoted.triple" }, @@ -280,7 +274,7 @@ "begin": "(?i)[r|/]'", "end": "'", "name": "string.regexp", - "patterns": [{ "include": "#regex-escapes" }] + "patterns": [{"include": "#regex-escapes"}] } ], "repository": { @@ -288,14 +282,14 @@ "name": "constant.character.escape", "match": "\\\\(u[A-Fa-f0-9]{4}|.)", "captures": { - "0": { "name": "constant.character.escape" } + "0": {"name": "constant.character.escape"} } }, "regex-escapes": { "name": "constant.character.escape", "match": "\\\\.", "captures": { - "0": { "name": "constant.character.escape" } + "0": {"name": "constant.character.escape"} } } } diff --git a/packages/malloy-syntax-highlight/test/visual/index.html b/packages/malloy-syntax-highlight/test/visual/index.html index a04071264..487735978 100644 --- a/packages/malloy-syntax-highlight/test/visual/index.html +++ b/packages/malloy-syntax-highlight/test/visual/index.html @@ -37,40 +37,43 @@ -
+ +
- \ No newline at end of file + diff --git a/packages/malloy/src/dialect/functions/README.md b/packages/malloy/src/dialect/functions/README.md index 74fb30759..3599a11d3 100644 --- a/packages/malloy/src/dialect/functions/README.md +++ b/packages/malloy/src/dialect/functions/README.md @@ -1,3 +1,3 @@ # Functions -This directory defines default definitions of SQL functions which will work in most dialects. If it doesn't work in your dialect, override it. \ No newline at end of file +This directory defines default definitions of SQL functions which will work in most dialects. If it doesn't work in your dialect, override it. diff --git a/packages/malloy/src/lang/doc/ast.md b/packages/malloy/src/lang/doc/ast.md index 46fa597ad..ec4d6eeab 100644 --- a/packages/malloy/src/lang/doc/ast.md +++ b/packages/malloy/src/lang/doc/ast.md @@ -35,7 +35,6 @@ export class Explore implements Element, ExploreInterface { pipeline: Pipestage[] = []; ``` - ## Methods on an AST object In general, the convention is, if an AST object knows how to make @@ -84,15 +83,15 @@ or chained evaluation need to implement this. Do no quite have this factored right, but the "Realm" contains three pieces of data which many (but not all) AST nodes need to do translation. -* The schema for any referenced tables, to resolve table reference4s -* The model, to resolve explore name references -* A log method for reporting semantic errors during translation +- The schema for any referenced tables, to resolve table reference4s +- The model, to resolve explore name references +- A log method for reporting semantic errors during translation ## Field Space A `FieldSpace` is a symbol table roughly equivalent to a `StructDef`. You can ask a `FieldSpace` to look up a dotted path name, and you can -ask a `FieldSpace` for its `StructDef` +ask a `FieldSpace` for its `StructDef` A `SpaceField` is the set of wrapper classes for `FieldDef` objects and is what you will get back if you ask a `FieldSpace` to lookup a field. diff --git a/packages/malloy/src/lang/doc/experimental.md b/packages/malloy/src/lang/doc/experimental.md index 41bb17ff4..83769b293 100644 --- a/packages/malloy/src/lang/doc/experimental.md +++ b/packages/malloy/src/lang/doc/experimental.md @@ -24,5 +24,5 @@ If the experiment involves different code generation, you can generate an AST at Add yours to the list, make sure no one else is using yours. -* `compilerTestExperimentParse` -* `compilerTestExperimentTranslate` +- `compilerTestExperimentParse` +- `compilerTestExperimentTranslate` diff --git a/packages/malloy/src/lang/doc/param-internal.md b/packages/malloy/src/lang/doc/param-internal.md index f91b31f36..320bcad0e 100644 --- a/packages/malloy/src/lang/doc/param-internal.md +++ b/packages/malloy/src/lang/doc/param-internal.md @@ -4,23 +4,22 @@ Landmarks on the road to a "fully parameterized Malloy" would be. ## Phase 0 -1) Exploreable items can require filter definition to limit the size of a query -2) Explorable itmes can have an optional filter definition to limit the size of a query +1. Exploreable items can require filter definition to limit the size of a query +2. Explorable itmes can have an optional filter definition to limit the size of a query ## No assigned phase -* Measures with parameters -* Turtles with parameters -* Normal constant parameters at least at the outermost (model) scope, but possibly also at the explorable, measure, and turtle scopes. +- Measures with parameters +- Turtles with parameters +- Normal constant parameters at least at the outermost (model) scope, but possibly also at the explorable, measure, and turtle scopes. ### Possibly on the plan -* Because these are a LOT like functions, maybe there is a ways to write parameterized expressions at some level other than the field expressions, maybe even up at the level where we declare explorable items, then very likely these "functions" or paramterized expressions are declared and instantiated with the same gestures all of the above items use. - +- Because these are a LOT like functions, maybe there is a ways to write parameterized expressions at some level other than the field expressions, maybe even up at the level where we declare explorable items, then very likely these "functions" or paramterized expressions are declared and instantiated with the same gestures all of the above items use. ## Phase 0+ -It is not much additional work to move constant parameters into the phase zero, inside and explore, and so even though constant parameters are not in the MVP for parameters, they are part of the MVP design +It is not much additional work to move constant parameters into the phase zero, inside and explore, and so even though constant parameters are not in the MVP for parameters, they are part of the MVP design # Syntax @@ -118,7 +117,6 @@ Here's an attempt with that in mind. This would need parentheses around declarations in measures, though not things that have field lists. , with `is` in invocations to seperate that from the definition, since that is an "is" also. - ### has block, kind of like paramter lists ... define thingWithParams @@ -150,7 +148,6 @@ Also the "has" followed by a list really makes it feel like each element in the is ('project.schema.tableName' ... - ### Rejected for sure ... You could move the parameter declarations inside the explore, since they match the syntax for statements inside an explore head ... The good thing about this is they are physically declared in the same textual unit where the namespace the exist in is defined. (something which is not true for programming languages) @@ -270,10 +267,9 @@ I guess we could require, for someone who wants params and fields ... But that is never going to feel as natural as "all parameter blocks have the same grouping syntax around them" - ## Parentheses around parameter lists ... -If you want this to look like a function call, parentheses can placed around the parameters, this could be optional syntax. Commas between parameters will probably need to at least be optional because these look more like lists rather than commands and people will try and use commas. +If you want this to look like a function call, parentheses can placed around the parameters, this could be optional syntax. Commas between parameters will probably need to at least be optional because these look more like lists rather than commands and people will try and use commas. ### Is in define statements with parentheses @@ -300,7 +296,6 @@ This makes me thing there would be a different word, maybe this is where `requir uses optValue timestamp or @2003 ) - ### Invocation with parentheses Invocation with `is` ... seems to scan pretty naturally @@ -319,7 +314,7 @@ A reference to a value parameter will produce an expression fragment which looks path: string; }; - For condition valued parameters we need to be able to compile the condition to an expression, and then somehow apply the runtime value to that condition. +For condition valued parameters we need to be able to compile the condition to an expression, and then somehow apply the runtime value to that condition. define xflights has dep_filter timestamp condition @2003 @@ -342,6 +337,7 @@ Needs to make a structdef, and the default value of that filter needs to expand } } ``` + But then what does a compiled predicate expression look like, for for a default value, or for a query time provided value? Well, that is essentially a function which takes an argument and substitutes the passed in as the value of the argument. @@ -476,10 +472,9 @@ And when I am writing the structdef for the join to rename the join I can fill i The question is where are all the places that a named thing can be referenced -1) In a join -2) In a query head -3) In a define statement - +1. In a join +2. In a query head +3. In a define statement The bigger question is, for other things which can have parameters ( turtles and measures ) where do the parameter blocks sit for references and for declaration. In this case every reference is kind of a delcaration so copying the declaraiton into the reference isn't a problem, but maybe that isn't the case for the other parameterized things ? @@ -489,7 +484,7 @@ And mahybe we don't care until then? This morning the think that I need before I can start actually generating code for parameters is what constant predicate expressions look like as expervalues? - The problem is they have two types, they really are of the form `Predicate`, so probably they should be stored ... +The problem is they have two types, they really are of the form `Predicate`, so probably they should be stored ... type: "predicate"; baseType: AtomicFieldType; @@ -511,6 +506,7 @@ Ok yes this is a problem. I have this magic thing called "requestTranslation" wh PKW dep_time_filter is @2003 EKW flights | by_carrier DKW (dep_time_filter is @2003) | flights | by_carrier (flights dep_time_filter is @2003) | by_carrier + --- REMAINING TO DO ... diff --git a/packages/malloy/src/malloy.ts b/packages/malloy/src/malloy.ts index 3299b376e..09b3caece 100644 --- a/packages/malloy/src/malloy.ts +++ b/packages/malloy/src/malloy.ts @@ -91,6 +91,9 @@ export interface Loggable { error: (message?: any, ...optionalParams: any[]) => void; } +// bad formatting to test prettier in CI +const s = 's'; + export interface ParseOptions { importBaseURL?: URL; } diff --git a/test/README.md b/test/README.md index 089a0f968..08aff7b69 100644 --- a/test/README.md +++ b/test/README.md @@ -30,7 +30,7 @@ Setting up DuckDB: # Using the custom matcher for running queries -There is now a custom matcher, `malloyResultMatches` for running queries. The customer matcher makes it easy to write readable tests which need to look at query results, and produces useful output when the test fails to make it easier to develop tests or respond to the output of failing tests. +There is now a custom matcher, `malloyResultMatches` for running queries. The customer matcher makes it easy to write readable tests which need to look at query results, and produces useful output when the test fails to make it easier to develop tests or respond to the output of failing tests. ## Check for results in the first row of output @@ -54,9 +54,9 @@ import './util/db-jest-matchers'; This will check the following things. -* There is at least one row of data in the output. So `.malloyQueryMatches(rt, {})` will fail if the query returns no rows -* There are entries in that row for each key. `{}` matches any row -* The entries are equal, but it will error if the expected data as a number and the returned data is a string. +- There is at least one row of data in the output. So `.malloyQueryMatches(rt, {})` will fail if the query returns no rows +- There are entries in that row for each key. `{}` matches any row +- The entries are equal, but it will error if the expected data as a number and the returned data is a string. ## Accessing nested results @@ -79,11 +79,11 @@ example also shows passing a model instead of a runtime to the matcher. }); ``` - > [!WARNING] - > There is currently a ... feature ... where if the source code of a test - > contains the characters `nest:` and the runtime connection does not support nesting, - > the test passes without actually doing anything. There will better handling of - > this problem in the future, but if your test is mysteriously passing, this is why. +> [!WARNING] +> There is currently a ... feature ... where if the source code of a test +> contains the characters `nest:` and the runtime connection does not support nesting, +> the test passes without actually doing anything. There will better handling of +> this problem in the future, but if your test is mysteriously passing, this is why. ## Queries returning more than one row of data @@ -102,8 +102,8 @@ An array of match rows may can also be used, if the test needs to verify more th This will pass if .. -* There are exactly two rows of output. If there are not exactly two the matcher will fail. If the query makes more than two rows, you will need to add a `limit: 2` to allow the matcher to pass. -* Each row is matches the matching criteria, again `{}` means "pass if there is a row" +- There are exactly two rows of output. If there are not exactly two the matcher will fail. If the query makes more than two rows, you will need to add a `limit: 2` to allow the matcher to pass. +- Each row is matches the matching criteria, again `{}` means "pass if there is a row" ## Reading failure output @@ -235,7 +235,9 @@ The old template for a test looked something like ``` The actual matcher for a result is limited to an equality test, in the old pattern you would have written something using an existing matcher for a data value + ```TypeScript expect(result.data.patch(0, 'numThings').value).toBeGreaterThan(7); ``` -and if this is desirable, more work on the custom matcher would be needed to allow expressions like this to be written. \ No newline at end of file + +and if this is desirable, more work on the custom matcher would be needed to allow expressions like this to be written. diff --git a/test/data/duckdb/test.json b/test/data/duckdb/test.json index e63d37b65..c8c4105eb 100644 --- a/test/data/duckdb/test.json +++ b/test/data/duckdb/test.json @@ -1,3 +1,3 @@ { - "foo": "bar" + "foo": "bar" } diff --git a/tsconfig.json b/tsconfig.json index 362acdb35..832c0cd3c 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -17,4 +17,4 @@ "compilerOptions": { "downlevelIteration": true } -} \ No newline at end of file +}