Fix: should not add emptyDir sizeLimit conf on executor pods if it is nil

[Cian911]

Purpose of this PR

Proposed changes:

This is a follow up fix from work already done in #2305 by @ChenYi015

This PR adds a check at submission on executors pods if an emptyDir sizeLimit is nil.

Change Category

• Bugfix (non-breaking change which fixes an issue)
• Feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that could affect existing functionality)
• Documentation update

Rationale

Following some further testing of an older SparkApplication template wherein the volumeName (see issue #2199 for more details) had not been changed, I noticed behaviour wherein only the driver pod would get created and no executor pods would spawn. I believe this is due to the same issue outlined in #2305 which would make sense, as the check was only added on the driver pod submission, and not the executors.

Checklist

• I have conducted a self-review of my own code.
• I have updated documentation accordingly.
• I have added tests that prove my changes are effective or that my feature works.
• Existing unit tests pass locally with my changes.

Additional Notes

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ChenYi015

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ChenYi015]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ChenYi015]

@Cian911 Thanks for fixing this.
/lgtm