postman.setNextRequest support questions

[mwhipple]

👋 I'm currently looking to make use of this project and running into setNextRequest being unsupported and am hoping to get some information that could work through that gap. As a disclaimer I'm not particularly familiar with Postman and am in somewhat of a consultant role for this particular work, and so therefore am particularly curious about where the specific challenges are?

My quick read on the issue (inferring a lot) is that the Postman API is largely oriented around an event loop model where that method enqueues requests to be processed at some subsequent tick and that doesn't conform with the K6 sobek model where concurrency is done at the virtual user level, an event loop isn't supported, and intra-process concurrency can get hairy given that the VU runtime can't be safely shared across goroutines.

But some other questions (which come back particularly to lack of familiarity with Postman and how this method may be used) is whether processing of the requests could be handled more synchronously, either being issued at the time that setNextRequest is invoked or wired up such that entries are pulled out of the queue for some predefined events such as an inflight response completing (or some combination of those two along the lines of requests being sent immediately if the queue is empty and otherwise popping on some trigger and then invoking the next) so that in any case the requests could be interleaved within the existing flow rather than being concurrent.

[thim81]

hi @mwhipple

You are understanding it correctly.

There are some differences in the way Posmtan & K6 executes requests

Postman request execution:
Postman runs collections in a sequential manner, executing requests one after the other.
setNextRequest is a function in Postman that allows you to dynamically set which request to execute next based on certain conditions or responses from previous requests. This makes Postman's execution flow highly flexible and can create complex, conditional workflows.

k6 request execution:
k6 is a load testing tool focused on performance testing rather than functional testing. It executes scripts written in JavaScript, but its primary design is around defining a fixed set of scenarios and iterating over them.
k6 scripts are executed in a linear, sequential manner. While you can control loops and conditionals in the script, the execution flow is typically fixed (since you want to test performance and not functional) and is not as dynamic as Postman.
k6 does not have a built-in mechanism to dynamically change the next request to execute mid-run in the same flexible manner as Postman’s setNextRequest.

Also important to be aware of is the stateless nature of Load Testing:
k6 emphasizes statelessness and independence of each virtual user (VU) to ensure accurate performance testing. Introducing dynamic request changes could complicate the stateless design and impact the accuracy and repeatability of load tests.

Next there is the complexity of conditional flows:
Currently postman-to-k6 converts the Postman requests to K6 requests. The CLI analyses the Postman request (method, headers, tests, ... ) and maps it to a K6 request (method, headers, tests, ...).
Implementing a feature like setNextRequest would require going beyond mapping the request structure but also parse the "pre/post" scripts, to extract the setNextRequest and replicate the conditions around setNextRequest, to be able properly map/recreate the Postman setNextRequest behaviour. This would add a large complexity and potentially lead to different interpretation of the Posmtan setNextRequest logic, resulting in different behaviour of the Postman execution vs the K6 execution.

My typical recommendation is to use Postman for behaviour, schema & contract testing and use K6 only for load tests.
It is certainly OK to have 2 postman collections with the different purposes, and

An alternative approach is to use the separate parameter of the postman-to-k6 converter, which will results in the each request being a separate K6 request method.

export default function() {
  group("Accounts", function() {
    postman[Request]("Get all accounts");
    postman[Request]("Create a new account");
    postman[Request]("Get a specific account");
    postman[Request]("Update an account");
    postman[Request]("Delete an account");
  });
  group("Domain clusters", function() {
    postman[Request]("Get all domain cluster configurations");
    postman[Request]("Create a new domain cluster configuration");
    postman[Request]("Get a specific domain cluster configuration");
    postman[Request]("Update a domain cluster configuration");
    postman[Request]("Delete a domain cluster configuration");
  });

In the main script, you can model/puzzle the requests in the order as you want.
It does not automatically apply the Postman setNextRequest logic, but it gives you the flexibility to manually reshuffle requests, independent of the Postman collection order, while still using the Postman collection as source for converting into K6 requests.

[thim81]

@mwhipple Did this answer your questions? Or would you need more info? Let me know, so I can assist you.

[mwhipple]

I'm picking this up again today. My main practical focus is on enabling use of the converted tests so I'll be trying the separate route and may revisit this depending on how that and other alternatives play out (and should have more concrete resolution this week). Thanks!

[mwhipple]

I have enough information to proceed, I've used separate and overridden setNextRequest to issue the named request which was enough to let some tests pass but more importantly surfaced other unsupported features in the suite so I'm abandoning the pursuit an more automatic transform and run process (which thankfully also doesn't seem to be warranted) and going for a one time conversion where the wrinkles can be ironed out. Thanks for the input!