Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add action input as parameters for tool execution in conversational agent #3200

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jngz-es
Copy link
Collaborator

@jngz-es jngz-es commented Nov 4, 2024

Description

Related Issues

Resolves #[Issue number to be closed when this PR is merged]
#3134

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…gent

Signed-off-by: Jing Zhang <jngz@amazon.com>
Comment on lines +475 to +476
toolParams.put("action_input", actionInput);
if (isJson(actionInput)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can actionInput be null here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will it be a problem if we put actionInput as null within the params? I can see that isJson will check for null, but wondering if line 475 can cause issue

Copy link
Member

@dbwiddis dbwiddis Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, @pyek-bot looking at MLToolSpec we'd get here:

Object value = parameterObjs.get(key);
try {
AccessController.doPrivileged((PrivilegedExceptionAction<Void>) () -> {
if (value instanceof String) {
parameters.put(key, (String) value);
} else {
parameters.put(key, gson.toJson(value));
}
return null;
});

and Gson.toJson internally uses Object.getClass() which may throw NPE...

which looks like it handles null.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, thanks for validating!

Copy link
Collaborator

@ylwu-amzn ylwu-amzn Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mingshl
Copy link
Collaborator

mingshl commented Nov 14, 2024

tests passed but failed in upload, not related to this code change. Approved.

Run actions/upload-artifact@v4
/usr/bin/docker exec  a6c6ef6ad38d2e9993d03ba2bdc50f2146f892a4a109fc9fecaf2c66802943f0 sh -c "cat /etc/*release | grep ^ID"
/__e/node[20](https://github.com/opensearch-project/ml-commons/actions/runs/11670434010/job/32954019963?pr=3200#step:8:21)/bin/node: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /__e/node20/bin/node)
/__e/node20/bin/node: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by /__e/node20/bin/node)
``'

@@ -472,6 +472,11 @@ public static Map<String, String> constructToolParams(
if (toolSpecConfigMap != null) {
toolParams.putAll(toolSpecConfigMap);
}
toolParams.put("action_input", actionInput);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

action_input looks like coupled with code logic. Change to more readable name ?

@reuschling
Copy link

reuschling commented Nov 18, 2024

The changes for AgentUtils look fine, but is AgentUtils used for conversational agents? The tool parameters are build inside MLConversationalFlowAgentRunner.getToolExecuteParams(MLToolSpec toolSpec, Map<String, String> params), i.e. here. There is no AgentUtils invocation. The only invocation of AgentUtils.‎‎constructToolParams I found is inside MLChatAgentRunner.

I showed a code proposal for getToolExecuteParams at #2977 (comment), the only difference is that there is no dedicated actionInput parameter, the actionInput is the "input" entry so far, which has to be temporarily stored inside a local variable.

I would also highly recommend to add the new "action_input" parameter to flow agents (i.e. MLFlowAgentRunner.getToolExecuteParams) also. There is no use of AgentUtils too. Of course there is the possibility with parameters.previous_tool_name.output, but tool specifications should act the same independent where they should be used, whether inside flow or conversational agents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants