Update the Supervisor endpoint to not restart the Supervisor if the spec was unmodified #17707
Conversation
…pec was unmodified
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
server/src/main/java/org/apache/druid/metadata/MetadataSupervisorManager.java
Fixed
Show fixed
Hide fixed
| @@ -166,6 +166,18 @@ public boolean createOrUpdateAndStartSupervisor(SupervisorSpec spec) | |||
| } | |||
| } | |||
|
|
|||
| public boolean wasSupervisorSpecModified(SupervisorSpec spec) | |||
There was a problem hiding this comment.
Please add a short javadoc.
Also, I find the method name to be slightly ambiguous.
Maybe rename to shouldUpdateSupervisor() as that expresses the intent of the usage of this method more clearly.
| public Response specPost( | ||
| final SupervisorSpec spec, | ||
| @Context final HttpServletRequest req, | ||
| @QueryParam("restartIfUnmodified") @DefaultValue("true") boolean restartIfUnmodified) |
There was a problem hiding this comment.
| @QueryParam("restartIfUnmodified") @DefaultValue("true") boolean restartIfUnmodified) | |
| @QueryParam("restartIfUnmodified") @DefaultValue("true") boolean restartIfUnmodified | |
| ) |
| Preconditions.checkNotNull(spec.getDataSources(), "spec.getDatasources()"); | ||
| synchronized (lock) { | ||
| Preconditions.checkState(started, "SupervisorManager not started"); | ||
| return metadataSupervisorManager.wasSupervisorSpecModified(spec); |
There was a problem hiding this comment.
I don't think we should need to call metadata store here.
- If the supervisor is active (running or suspended), it would already be in memory in the map
SupervisorManager.supervisors. We can compare with the version in memory to determine if the spec has been updated. - If the supervisor is not present in the map, that indicates that the supervisor is either terminated or doesn't exist at all. In both the cases, we should just return
true.
There was a problem hiding this comment.
Thanks for the review @kfaraz! I thought to do this in MetadataSupervisorManager because the comparison relies on the ObjectMapper in SQLMetadataSupervisorManager
Let me dig more and see how to get the ObjectMapper into SupervisorManager
| * @param SupervisorSpec spec being submitted | ||
| * @return whether the spec was modified | ||
| */ | ||
| boolean wasSupervisorSpecModified(SupervisorSpec spec); |
There was a problem hiding this comment.
We probably don't need this method. Please see the other comment.
|
Thank you for these changes, @aho135! I think we would benefit from a change where we check if the spec has changed. If it hasn't we still restart the supervisor, but do not go to the metadata store and add an unnecessary entry in the spec history. Otherwise, the flow remains unchanged. I think @kfaraz has suggested this as well. I also wanted to understand if the problem was with the metadata operations associated with it including an unneeded entry, or if the supervisor operation is also problematic. If it is just the first case, is a feature flag really needed? If you still believe that the supervisor operation is wasteful, and want to introduce a flag, please add the relevant docs in |
Thanks for the review @AmatyaAvadhanula! My original motivation for this change was to avoid unnecessary restarts of the Supervisor if possible. Our use case is that we maintain a repository of schemas and do periodic releases. It is often unclear which schemas were actually modified. We want to be able to submit them all, and just restart the Supervisors which had schema updates. This is so we can avoid the undesirable side effects of task restart, such as small segments. With this use case in mind, I think that having the feature flag does make sense. I will add an update in the relevant doc |
…rvisorManager, update Supervisor API ref
Description
This PR adds an optional query parameter called restartIfUnmodified to the /druid/indexer/v1/supervisor endpoint. The caller can optionally set restartIfUnmodified=false so that the supervisor is not restarted if the spec is unchanged. Multiple members of the community mentioned that they maintain their own scripts to check whether the spec has changed before submitting to the endpoint: https://apachedruidworkspace.slack.com/archives/C0303FDCZEZ/p1738017586080509
For those that rely on this endpoint for restarting the supervisor, the behavior remains unchanged as restartIfUnmodified defaults to true.
Release note
Adds an optional query parameter called restartIfUnmodified to the /druid/indexer/v1/supervisor endpoint. Callers can set restartIfUnmodified=false to not restart the supervisor if the spec is unchanged. Example:
curl -X POST --header "Content-Type: application/json" -d @supervisor.json localhost:8888/druid/indexer/v1/supervisor?restartIfUnmodified=false
Key changed/added classes in this PR
SupervisorResourceSupervisorManagerSupervisorResourceTestThis PR has: