Skip to content

PromptTemplate validation prevents using maps in templates #1428

Closed
@mands

Description

@mands

Follow-up from #631

Bug description
When passing a Map into a PromptTemplate as a param to iterate over, validation fails due to an erroneously detected missing input variable.

The same template and params work correctly when using StringTemplate directly.

Environment

  • Spring AI 1.0.0-SNAPSHOT
  • Java 22

Steps to reproduce

The following unit tests indicate the issue, with the first failing using PromptTemplate, and the second succeeding using StringTemplate directly.

  @Test
  public void testPromptTemplateWithMap() {
    var template = "{ myMap:{ k | Key: {k}, Value: {myMap.(k)} }}";
    Map<String, Object> myMap = Map.of("k1", "v1", "k2", "v2");
    var pt = new PromptTemplate(template, Map.of("myMap", myMap));
    var res = pt.render();

    assertTrue(
      res.equals("Key: k2, Value: v2 Key: k1, Value: v1 ") || res.equals("Key: k1, Value: v1 Key: k2, Value: v2 ")
    );
  }

  @Test
  public void testSTWithMap() {
    var template = "{ myMap:{ k | Key: {k}, Value: {myMap.(k)} }}";
    Map<String, Object> myMap = Map.of("k1", "v1", "k2", "v2");

    ST st = new ST(template, '{', '}');
    st.add("myMap", myMap);
    var res = st.render();

    assertTrue(
      res.equals("Key: k2, Value: v2 Key: k1, Value: v1 ") || res.equals("Key: k1, Value: v1 Key: k2, Value: v2 ")
    );
  }

The error from the first test is,

java.lang.IllegalStateException: Not all template variables were replaced. Missing variable names are [k]

	at org.springframework.ai.chat.prompt.PromptTemplate.validate(PromptTemplate.java:231)
	at org.springframework.ai.chat.prompt.PromptTemplate.render(PromptTemplate.java:124)
	at com.example.service.AIServiceTests.testPromptTemplateWithMap(AIServiceTests.java:53)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)

Expected behavior

The template should render and return the simple iteration over the map keys and values. To note, the same error occurs with List iteration (as per #631) if the same variable is used twice in the loop block.

Notes

I think it would be worth making validation optional, as otherwise it may end up a game of whack-a-mole trying to handle all the cases where StringTemplate is used for more than variable substitution.

For instance, we have a more-bespoke RAG-like use-case that would benefit from a complex prompt with conditionals and iterations.

Similarly, would be great to get #355 in as { and } are used by StringTemplate itself to indicate an anonymous template block for iteration, which makes the templates quite confusing to develop. (Minor point, but coming from Python, I find being able to look at StringTemplate docs and having working IDE syntax checking more useful that the passing similarly with Python templates that using curly braces brings).

Thanks for the great library!

Activity

markpollack

markpollack commented on Oct 8, 2024

@markpollack
Member

thanks for reporting this, will investigate after the M3 release.

modified the milestones: 1.0.0-M3, 1.0.0-M4 on Oct 8, 2024
modified the milestones: 1.0.0-M4, 1.0.0-M5 on Nov 19, 2024
modified the milestones: 1.0.0-M5, 1.0.0-M6 on Dec 23, 2024
self-assigned this
on Jan 21, 2025
markpollack

markpollack commented on Jan 21, 2025

@markpollack
Member

Will be taking a look shortly. Thanks for your patience and the suggestion. Yes, perhaps validation is going a bit too far. The idea was to fail fast, but perhaps the trade off here isn't worth it.

17 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

    Development

    Participants

    @ilayaperumalg@markpollack@hopperd@mands@tzolov

    Issue actions

      PromptTemplate validation prevents using maps in templates · Issue #1428 · spring-projects/spring-ai