How to deal with boundary condition in CUDA OP2?

[lj-cug]

Dear Sir:
Another question: How should I deal with the boundary condition in OP2-CUDA? The branch divergence problem always exists in cuda code when there's many if-conditions, especially for hydrodynamic simulation cases.
Thanks
Li Jian

[reguly]

Hello, There is no ideal way of doing this. You can either include if conditions in a kernel that includes the boundary, and check based on the index passed in by op_arg_idx, or you can launch separate ops_par_loops for the boundary (see e.g. update_halo.cpp in apps/c/CloverLeaf). Which one will perform better very much depends on your application.But if you want to easily switch between different boundary conditions, I suggest going with separate ops_par_loops (even though they might end up being slightly slower). Best, Istvan

…

On 2022. Jan 28., at 13:06, lj-cug ***@***.***> wrote: Dear Sir: Another question: How should I deal with the boundary condition in OP2-CUDA? The branch divergence problem always exists in cuda code when there's many if-conditions, especially for hydrodynamic simulation cases. Thanks Li Jian — Reply to this email directly, view it on GitHub <#225>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJWVVN5RRAIJX64O4MDPBTUYKBDDANCNFSM5NASNFLA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.

[reguly]

Let me correct that (I mixed up OPS and OP2 here). For OP2, you can create sets which only include the boundary elements, and then do an op_par_loop only over those. Or you can create a dataset which flags which elements are on the boundary, and do the if conditions inside the kernel for an op_par_loop over the entire domain.

[lj-cug]

The first method seems to be better, the second method will cause warp divergence problem. I want to use volna-op2 to simulate flood inundation process. Boundary condition and source term in SWE are important. BaseMENT software developed by ETH seems to use OP2 library, which has been applied in many projects.
Thanks for your continuously developing OP2.

[reguly]

Please let us know if in your applications one of these approaches is prohibitively slow. In my experience these loops are not very expensive despite the warp divergence/inefficient memory accesses.