[BUG]: DQX profiling job gone stale #72
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
Whenever I submit a request to profile my schema's as part of a data contract,
The initial alert I receive is: "Failed to start profiling ["detail":"Invalid profiling request"]
However, after a while the DQX job seems to be starting.
Unfortunately, it never completes and no background jobs seem to be running on the App (in the Databricks worksapce)
Instead, the message keeps on showing 'on the contract': _"DQX profiling in progress... Results will appear here when complete."
Expected Behavior
Trigger DQX job succesfully
Provide job status polling so the user knows the status of the job
Create a job monitoring & restarting pane (for app administrators)
Steps To Reproduce
1 Create a new contract
2 Infer a schema from the UC in the contract
3 Activate the DQX profiling task
4 Select the (inferred) schema and click OK
Cloud
Azure
Browser
Chrome
Version
latest
Relevant log output
"Failed to start profiling ["detail":"Invalid profiling request"]