feat: estimate the cost of running a tag from web view

[ashish10alex]

This PR:

1. Introduces ability to estimate cost of running a Tag from compiled query web view
2. To estimate cost, we construct a full query as follows and perform dry run.
   - Table/View: Pre operation + Create or replaces a table/view statement + main query
   - Partitioned or clustered tables: Pre operations + main query
   - Incremental: Incremental pre operation query + Create or replaces a table/view statement + main query
   - Partitioned or clustered Incremental table: Incremental pre operation query + main query
   - Assertion & Operation : Main query

NOTE:
The cost estimation logic is constructed such that we can get the cost as close to actual where possible. e.g. if there is multiple transactions BigQuery dry run will not be able to estimate the cost. So, I'd appreciate any suggestions to make changes to the logic, either in this MR or as part of a future version

3. Sorts Tags when tags are stored for autocompletion and for dropdown selection in cost estimation web view tab
4. Adds cost of running a model's query in multiple currencies that can be changed by the user in vscode settings for the extension.
5. Adds support for Adding unit conversion depending on cost size #91

TODO

• Apply unit conversion in preview results tab
• Add tests for Adding unit conversion depending on cost size #91
• Add example image in README

Potential v2 features

1. Add option to include dependents / dependencies when running cost estimator for tag
2. Estimate cost of running a model with dependencies and dependents

[HampB]

Great work as always, @ashish10alex! 👏

I'm encountering an issue with several models:
Syntax error: Unexpected ";" at [8:1].

Here’s an example of the error:

---

Adding the ability to switch currencies would be a great improvement.

• This could be implemented as an extension setting with fixed options.
• A good starting point might include USD, EUR, and GBP.

For example, I generally keep the SEK <> EUR conversion in mind, but I’d have to look up the rate for GBP.

[HampB]

I think the issue is caused by this: it fails when a preOps already ends with a ;.

const preOpsQuery = curModel.preOps ? curModel.preOps.join("\n") + ";" : "";
const incrementalPreOpsQuery = curModel.incrementalPreOps 
    ? curModel.incrementalPreOps.join("\n") + ";" 
    : "";

[ashish10alex]

thanks @HampB! , I have added the fix. could you please do one more test to ensure it works now ?

[ashish10alex]

@HampB , I have added support for multiple currencies as well. I was going to do that in one of the next versions. But your suggestion has persuaded me to expedite it 🙂. Appreciate it if you could test It out

[HampB]

Nicely done, @ashish10alex! 🎉

There’s still an issue with semicolons (;) in incremental queries, especially when using statements inside when(incremental()...).

For example, the following query still breaks:

pre_operations {
  DECLARE date_checkpoint DATE DEFAULT (DATE("${dataform.projectConfig.vars.fullLoadStartDate}"));

  ${when(incremental(),
    `SET date_checkpoint = (DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY));`
  )}

  ${when(incremental(),
    `DELETE FROM ${self()} WHERE PostingDate >= date_checkpoint;`
  )}
}

If you dont know the obvious solution, i can look into it more later today. :-)

[ashish10alex]

@HampB , thanks, that was a good catch! should be fixed now

[HampB]

Looks good to me now!

For future development, as you've mentioned in the PR, it would be great to include the possibility of adding dependencies. Additionally, enabling the estimator to run based on a specific model (instead of a tag) along with its dependencies would be a nice enhancement.