Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PromptTemplate validation prevents using maps in templates #1428

Open
mands opened this issue Sep 28, 2024 · 1 comment
Open

PromptTemplate validation prevents using maps in templates #1428

mands opened this issue Sep 28, 2024 · 1 comment

Comments

@mands
Copy link

mands commented Sep 28, 2024

Follow-up from #631

Bug description
When passing a Map into a PromptTemplate as a param to iterate over, validation fails due to an erroneously detected missing input variable.

The same template and params work correctly when using StringTemplate directly.

Environment

  • Spring AI 1.0.0-SNAPSHOT
  • Java 22

Steps to reproduce

The following unit tests indicate the issue, with the first failing using PromptTemplate, and the second succeeding using StringTemplate directly.

  @Test
  public void testPromptTemplateWithMap() {
    var template = "{ myMap:{ k | Key: {k}, Value: {myMap.(k)} }}";
    Map<String, Object> myMap = Map.of("k1", "v1", "k2", "v2");
    var pt = new PromptTemplate(template, Map.of("myMap", myMap));
    var res = pt.render();

    assertTrue(
      res.equals("Key: k2, Value: v2 Key: k1, Value: v1 ") || res.equals("Key: k1, Value: v1 Key: k2, Value: v2 ")
    );
  }

  @Test
  public void testSTWithMap() {
    var template = "{ myMap:{ k | Key: {k}, Value: {myMap.(k)} }}";
    Map<String, Object> myMap = Map.of("k1", "v1", "k2", "v2");

    ST st = new ST(template, '{', '}');
    st.add("myMap", myMap);
    var res = st.render();

    assertTrue(
      res.equals("Key: k2, Value: v2 Key: k1, Value: v1 ") || res.equals("Key: k1, Value: v1 Key: k2, Value: v2 ")
    );
  }

The error from the first test is,

java.lang.IllegalStateException: Not all template variables were replaced. Missing variable names are [k]

	at org.springframework.ai.chat.prompt.PromptTemplate.validate(PromptTemplate.java:231)
	at org.springframework.ai.chat.prompt.PromptTemplate.render(PromptTemplate.java:124)
	at com.example.service.AIServiceTests.testPromptTemplateWithMap(AIServiceTests.java:53)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)

Expected behavior

The template should render and return the simple iteration over the map keys and values. To note, the same error occurs with List iteration (as per #631) if the same variable is used twice in the loop block.

Notes

I think it would be worth making validation optional, as otherwise it may end up a game of whack-a-mole trying to handle all the cases where StringTemplate is used for more than variable substitution.

For instance, we have a more-bespoke RAG-like use-case that would benefit from a complex prompt with conditionals and iterations.

Similarly, would be great to get #355 in as { and } are used by StringTemplate itself to indicate an anonymous template block for iteration, which makes the templates quite confusing to develop. (Minor point, but coming from Python, I find being able to look at StringTemplate docs and having working IDE syntax checking more useful that the passing similarly with Python templates that using curly braces brings).

Thanks for the great library!

@markpollack
Copy link
Member

thanks for reporting this, will investigate after the M3 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants