Hierarchy patient counts timeout issue in BigQuery #523
olayemiibrahim
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
We have around 4 million hierarchies and we need to run counts for each of them and insert/update them back in the Concept table. The stored procedure for the hierarchy counts can not complete within 6 hours. 6 hours is the BigQuery limit to run a single query or stored procedure with no way to increase it.
We ended up adding parameters to the stored procedure and called the stored procedure about 50 times. So each of the stored procedures can complete within 6 hours. We ran about 10 in parallel at a time.
Has anyone figured out a more efficient way to complete this?
Beta Was this translation helpful? Give feedback.
All reactions