Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preliminary compound datatype support #1991

Merged
merged 107 commits into from
Dec 4, 2024
Merged

Preliminary compound datatype support #1991

merged 107 commits into from
Dec 4, 2024

Conversation

mtoy-googly-moogly
Copy link
Collaborator

@mtoy-googly-moogly mtoy-googly-moogly commented Oct 31, 2024

All that refactoring leads to this ... arrays and records working in more places. Using the word "compound" data for this, saving the word "composite" for cube-like sources.

The WN for this is https://github.com/malloydata/whatsnext/blob/main/wns/WN-0015-arrays-and-records/WN-0015.md but this is an MVP level first step. The MVP is based on avoiding some of the most common reasons Meta users are having to drop into SQL when working with compound data

  • Can select: a compound computation
  • Can pass a compound computation to a function call
  • Literals exist for compound types
  • Some array functions in the standard, or presto library
  • A query can select an array or an expression that has an array type
  • There is an array literal
  • You can pass arrays to functions
  • There is a record literal
  • A record can be selected
  • A literal for an array of records
  • An array of records can be selected
  • Record in a source
  • Array in a source
  • Array of records in a source
  • Tested in all dialects
  • Maybe add split/join/contains for @lloydtabb
  • Merge some doc for record and array datatypes malloydata.github.io#202
  • Add tests for group_by: records and arrays

@mtoy-googly-moogly mtoy-googly-moogly changed the title Next stage of array datatype support Next stage of composite datatype support Nov 7, 2024
@mtoy-googly-moogly mtoy-googly-moogly changed the title Next stage of composite datatype support Next stage of compund datatype support Nov 12, 2024
@mtoy-googly-moogly
Copy link
Collaborator Author

Thanks @lloydtabb for the test templates for the grouping ...

      test('can build rec select with data from table', async () => {
        await expect(`
          # test.debug
          run: ${conName}.table('malloytest.state_facts') -> {
            select: airport_count, val is {airport_count}
            order_by: airport_count desc
            limit: 10
        }
        `).malloyResultMatches(runtime, {
          'airport_count': 1845,
          'val.airport_count': 1845,
        });
      });

      test('can build record group with data from table', async () => {
        await expect(`
          # test.debug
          run: ${conName}.table('malloytest.state_facts') -> {
            group_by: airport_count, val is {airport_count}
            order_by: airport_count desc
            limit: 10
        }
        `).malloyResultMatches(runtime, {airport_count: 1845});
      });

      test('can build array group with data from table', async () => {
        await expect(`
          # test.debug
          run: ${conName}.table('malloytest.state_facts') -> {
            group_by: airport_count, val is [airport_count]
            order_by: airport_count desc
            limit: 10
        }
        `).malloyResultMatches(runtime, {airport_count: 1845});
      });

@mtoy-googly-moogly mtoy-googly-moogly merged commit e07495f into main Dec 4, 2024
11 checks passed
@mtoy-googly-moogly mtoy-googly-moogly deleted the array-in-query branch December 4, 2024 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants