Skip to content

Commit

Permalink
bug!: fix updateCount handling (#110)
Browse files Browse the repository at this point in the history 
- introduce a pseudo-transaction which writes updates when a request
ends
- introduce a transactionManager for this
- move locking to transactionManager (and remove some unneeded locks)
- update tests as the updateCount is only increased once now

<!-- Please describe your pull request here. -->

## References

#102 

<!-- References to relevant GitHub issues and pull requests, esp.
upstream and downstream changes -->

## Submitter checklist

- [ ] Recommended: Join [WireMock Slack](https://slack.wiremock.org/) to
get any help in `#help-contributing` or a project-specific channel like
`#wiremock-java`
- [ ] The PR request is well described and justified, including the body
and the references
- [ ] The PR title represents the desired changelog entry
- [ ] The repository's code style is followed (see the contributing
guide)
- [ ] Test coverage that demonstrates that the change works as expected
- [ ] For new features, there's necessary documentation in this pull
request or in a subsequent PR to
[wiremock.org](https://github.com/wiremock/wiremock.org)

<!--
Put an `x` into the [ ] to show you have filled the information.
The template comes from
https://github.com/wiremock/.github/blob/main/.github/pull_request_template.md
You can override it by creating .github/pull_request_template.md in your
own repository
-->
  • Loading branch information
@dirkbolte
dirkbolte authored Dec 23, 2023
1 parent 14b062f commit af4474a
Show file tree
Hide file tree
Showing 23 changed files with 676 additions and 327 deletions.
20 changes: 18 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ The state is recorded in `serveEventListeners` of a stub. The following function
- to delete a selective property, set it to `null` (as string).
- `list` : stores a state in a list. Can be used to prepend/append new states to an existing list. List elements cannot be modified (only read/deleted).

`state` and `list` can be used in the same `ServeEventListener` (would count as two updates). Adding multiple `recordState` `ServeEventListener` is supported.
`state` and `list` can be used in the same `ServeEventListener` (would count as ONE updates). Adding multiple `recordState` `ServeEventListener` is supported.

The following parameters have to be provided:

Expand Down Expand Up @@ -803,7 +803,9 @@ For documentation on using these matchers, check the [WireMock documentation](ht

### Context update count match

Whenever the serve event listener `recordState` is processed, the internal context update counter is increased. The number can be used
Whenever a request with a serve event listener `recordState` or `deleteState` is processed, the internal context update counter is increased.
The update count is increased by one whenever there is at least one change to a context (so: property adding/change, list entry addition/deletion). Multiple
event listeners with multiple changes of a single context within a single request only result in an increase by one.
for request matching as well. The following matchers are available:

- `updateCountEqualTo`
Expand Down Expand Up @@ -1070,6 +1072,20 @@ body file with handlebars to ignore a missing property:
}
```

# Distributed setups and concurrency

This extension is at the moment not optimized for distributed setups or high degrees concurrency. While it will basically work, there are some limitations
that should be held into account:

- The store used for storing the state is on instance-level only
- while it can be exchanged for a distributed store, any atomicity assurance on instance level is not replicated to the distributed setup. Thus concurrent operations on different instances might result in state overwrites
- Lock-level is basically the whole context store
- while the lock time is kept small, this can still impact measurements when being used in load tests
- Single updates to contexts (property additions or changes, list entry additions or deletions) are atomic on instance level
- Concurrent requests are currently allowed to change the same context. Atomicity prevents overwrites but does not provide something like a transaction, so: the context can change while a request is performed

For any kind of usage with parallel write requests, it's recommended to use a different context for each parallel stream.

# Debugging

In general, you can increase verbosity, either by [register a notifier](https://wiremock.org/3.x/docs/configuration/#notification-logging)
Expand Down
8 changes: 7 additions & 1 deletion src/main/java/org/wiremock/extensions/state/StateExtension.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@
import org.wiremock.extensions.state.extensions.RecordStateEventListener;
import org.wiremock.extensions.state.extensions.StateRequestMatcher;
import org.wiremock.extensions.state.extensions.StateTemplateHelperProviderExtension;
import org.wiremock.extensions.state.extensions.TransactionEventListener;
import org.wiremock.extensions.state.internal.ContextManager;
import org.wiremock.extensions.state.internal.TransactionManager;

import java.util.Collections;
import java.util.List;
Expand Down Expand Up @@ -52,15 +54,18 @@ public class StateExtension implements ExtensionFactory {
private final StateTemplateHelperProviderExtension stateTemplateHelperProviderExtension;
private final RecordStateEventListener recordStateEventListener;
private final DeleteStateEventListener deleteStateEventListener;
private final TransactionEventListener transactionEventListener;
private final StateRequestMatcher stateRequestMatcher;

public StateExtension(Store<String, Object> store) {
var contextManager = new ContextManager(store);
var transactionManager = new TransactionManager(store);
var contextManager = new ContextManager(store, transactionManager);
this.stateTemplateHelperProviderExtension = new StateTemplateHelperProviderExtension(contextManager);
var templateEngine = new TemplateEngine(stateTemplateHelperProviderExtension.provideTemplateHelpers(), null, Collections.emptySet(), false);

this.recordStateEventListener = new RecordStateEventListener(contextManager, templateEngine);
this.deleteStateEventListener = new DeleteStateEventListener(contextManager, templateEngine);
this.transactionEventListener = new TransactionEventListener(transactionManager);
this.stateRequestMatcher = new StateRequestMatcher(contextManager, templateEngine);
}

Expand All @@ -69,6 +74,7 @@ public List<Extension> create(WireMockServices services) {
return List.of(
recordStateEventListener,
deleteStateEventListener,
transactionEventListener,
stateRequestMatcher,
stateTemplateHelperProviderExtension
);
Expand Down
209 changes: 113 additions & 96 deletions src/main/java/org/wiremock/extensions/state/extensions/DeleteStateEventListener.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@
import com.github.tomakehurst.wiremock.stubbing.ServeEvent;
import org.apache.commons.lang3.StringUtils;
import org.wiremock.extensions.state.internal.ContextManager;
import org.wiremock.extensions.state.internal.DeleteStateParameters;
import org.wiremock.extensions.state.internal.ResponseTemplateModel;
import org.wiremock.extensions.state.internal.StateExtensionMixin;
import org.wiremock.extensions.state.internal.api.DeleteStateParameters;
import org.wiremock.extensions.state.internal.model.ResponseTemplateModel;

import java.util.List;
import java.util.Map;
Expand Down Expand Up @@ -72,122 +72,139 @@ public void beforeResponseSent(ServeEvent serveEvent, Parameters parameters) {
"response", ResponseTemplateModel.from(serveEvent.getResponse())
);
var configuration = Json.mapToObject(parameters, DeleteStateParameters.class);
Optional.ofNullable(configuration.getList()).ifPresentOrElse(
listConfig -> handleListDeletion(listConfig, createContextName(model, configuration.getContext()), model),
() -> handleContextDeletion(configuration, model)
);
new ListenerInstance(serveEvent.getId().toString(), model, configuration).run();
}

private void handleContextDeletion(DeleteStateParameters configuration, Map<String, Object> model) {
if (configuration.getContext() != null) {
deleteContext(configuration.getContext(), model);
} else if (configuration.getContexts() != null) {
deleteContexts(configuration.getContexts(), model);
} else if (configuration.getContextsMatching() != null) {
deleteContextsMatching(configuration.getContextsMatching(), model);
} else {
throw createConfigurationError("Missing/invalid configuration for context deletion");
}
private String renderTemplate(Object context, String value) {
return templateEngine.getUncachedTemplate(value).apply(context);
}

private void deleteContexts(List<String> rawContexts, Map<String, Object> model) {
private class ListenerInstance {
private final String requestId;
private final DeleteStateParameters configuration;
private final Map<String, Object> model;

var contexts = rawContexts.stream().map(it -> renderTemplate(model, it)).collect(Collectors.toList());
contextManager.onEach(context -> {
if(contexts.contains(context.getContextName())) {
contextManager.deleteContext(context.getContextName());
ListenerInstance(String requestId, Map<String, Object> model, DeleteStateParameters configuration) {
this.requestId = requestId;
this.model = model;
this.configuration = configuration;
}

public void run() {
Optional.ofNullable(configuration.getList()).ifPresentOrElse(
listConfig -> handleListDeletion(listConfig, createContextName(configuration.getContext())),
this::handleContextDeletion
);
}

private void handleContextDeletion() {
if (configuration.getContext() != null) {
deleteContext(configuration.getContext());
} else if (configuration.getContexts() != null) {
deleteContexts(configuration.getContexts());
} else if (configuration.getContextsMatching() != null) {
deleteContextsMatching(configuration.getContextsMatching());
} else {
throw createConfigurationError("Missing/invalid configuration for context deletion");
}
});
}
}

private void deleteContexts(List<String> rawContexts) {

private void deleteContextsMatching(String rawRegex, Map<String, Object> model) {
try {
var regex = renderTemplate(model, rawRegex);
var pattern = Pattern.compile(regex);
contextManager.onEach(context -> {
if(pattern.matcher(context.getContextName()).matches()) {
contextManager.deleteContext(context.getContextName());
var contexts = rawContexts.stream().map(it -> renderTemplate(model, it)).collect(Collectors.toList());
contextManager.onEach(requestId, context -> {
if (contexts.contains(context.getContextName())) {
contextManager.deleteContext(requestId, context.getContextName());
}
});
} catch (PatternSyntaxException ex) {
throw createConfigurationError("Missing/invalid configuration for context deletion: %s", ex.getMessage());
}
}

private void deleteContext(String rawContext, Map<String, Object> model) {
contextManager.deleteContext(createContextName(model, rawContext));
}
private void deleteContextsMatching(String rawRegex) {
try {
var regex = renderTemplate(model, rawRegex);
var pattern = Pattern.compile(regex);
contextManager.onEach(requestId, context -> {
if (pattern.matcher(context.getContextName()).matches()) {
contextManager.deleteContext(requestId, context.getContextName());
}
});
} catch (PatternSyntaxException ex) {
throw createConfigurationError("Missing/invalid configuration for context deletion: %s", ex.getMessage());
}
}

private void handleListDeletion(DeleteStateParameters.ListParameters listConfig, String contextName, Map<String, Object> model) {
if (Boolean.TRUE.equals(listConfig.getDeleteFirst())) {
deleteFirst(contextName);
} else if (Boolean.TRUE.equals(listConfig.getDeleteLast())) {
deleteLast(contextName);
} else if (StringUtils.isNotBlank(listConfig.getDeleteIndex())) {
deleteIndex(listConfig, contextName, model);
} else if (listConfig.getDeleteWhere() != null &&
listConfig.getDeleteWhere().getProperty() != null &&
listConfig.getDeleteWhere().getValue() != null
) {
deleteWhere(listConfig, contextName, model);
} else {
throw createConfigurationError("Missing/invalid configuration for list entry deletion");
private void deleteContext(String rawContext) {
contextManager.deleteContext(requestId, createContextName(rawContext));
}
}

private Long deleteFirst(String contextName) {
return contextManager.createOrUpdateContextList(contextName, maps -> {
if (!maps.isEmpty()) maps.removeFirst();
logger().info(contextName, "list::deleteFirst");
});
}
private void handleListDeletion(DeleteStateParameters.ListParameters listConfig, String contextName) {
if (Boolean.TRUE.equals(listConfig.getDeleteFirst())) {
deleteFirst(contextName);
} else if (Boolean.TRUE.equals(listConfig.getDeleteLast())) {
deleteLast(contextName);
} else if (StringUtils.isNotBlank(listConfig.getDeleteIndex())) {
deleteIndex(listConfig, contextName);
} else if (listConfig.getDeleteWhere() != null &&
listConfig.getDeleteWhere().getProperty() != null &&
listConfig.getDeleteWhere().getValue() != null
) {
deleteWhere(listConfig, contextName);
} else {
throw createConfigurationError("Missing/invalid configuration for list entry deletion");
}
}

private void deleteLast(String contextName) {
contextManager.createOrUpdateContextList(contextName, maps -> {
if (!maps.isEmpty()) maps.removeLast();
logger().info(contextName, "list::deleteLast");
});
}
private void deleteFirst(String contextName) {
contextManager.createOrUpdateContextList(requestId, contextName, maps -> {
if (!maps.isEmpty()) maps.removeFirst();
logger().info(contextName, "list::deleteFirst");
});
}

private void deleteIndex(DeleteStateParameters.ListParameters listConfig, String contextName, Map<String, Object> model) {
try {
var index = Integer.parseInt(renderTemplate(model, listConfig.getDeleteIndex()));
contextManager.createOrUpdateContextList(contextName, list -> {
list.remove(index);
logger().info(contextName, String.format("list::deleteIndex(%d)", index));
private void deleteLast(String contextName) {
contextManager.createOrUpdateContextList(requestId, contextName, maps -> {
if (!maps.isEmpty()) maps.removeLast();
logger().info(contextName, "list::deleteLast");
});
} catch (IndexOutOfBoundsException | NumberFormatException e) {
logger().info(contextName, String.format("Unknown or unparsable list index: '%s' - ignoring", listConfig.getDeleteIndex()));
}
}

private void deleteWhere(DeleteStateParameters.ListParameters listConfig, String contextName, Map<String, Object> model) {
var property = renderTemplate(model, listConfig.getDeleteWhere().getProperty());
var value = renderTemplate(model, listConfig.getDeleteWhere().getValue());
contextManager.createOrUpdateContextList(contextName, list -> {
var iterator = list.iterator();
while (iterator.hasNext()) {
var element = iterator.next();
if (Objects.equals(element.getOrDefault(property, null), value)) {
iterator.remove();
logger().info(contextName, String.format("list::deleteWhere(property=%s)", property));
break;
}
private void deleteIndex(DeleteStateParameters.ListParameters listConfig, String contextName) {
try {
var index = Integer.parseInt(renderTemplate(model, listConfig.getDeleteIndex()));
contextManager.createOrUpdateContextList(requestId, contextName, list -> {
list.remove(index);
logger().info(contextName, String.format("list::deleteIndex(%d)", index));
});
} catch (IndexOutOfBoundsException | NumberFormatException e) {
logger().info(contextName, String.format("Unknown or unparsable list index: '%s' - ignoring", listConfig.getDeleteIndex()));
}
});
}
}

private String createContextName(Map<String, Object> model, String rawContext) {
var context = Optional.ofNullable(rawContext).filter(StringUtils::isNotBlank)
.map(it -> renderTemplate(model, it))
.orElseThrow(() -> new ConfigurationException("no context specified"));
if (StringUtils.isBlank(context)) {
throw createConfigurationError("Context cannot be blank");
private void deleteWhere(DeleteStateParameters.ListParameters listConfig, String contextName) {
var property = renderTemplate(model, listConfig.getDeleteWhere().getProperty());
var value = renderTemplate(model, listConfig.getDeleteWhere().getValue());
contextManager.createOrUpdateContextList(requestId, contextName, list -> {
var iterator = list.iterator();
while (iterator.hasNext()) {
var element = iterator.next();
if (Objects.equals(element.getOrDefault(property, null), value)) {
iterator.remove();
logger().info(contextName, String.format("list::deleteWhere(property=%s)", property));
break;
}
}
});
}

private String createContextName(String rawContext) {
var context = Optional.ofNullable(rawContext).filter(StringUtils::isNotBlank)
.map(it -> renderTemplate(model, it))
.orElseThrow(() -> new ConfigurationException("no context specified"));
if (StringUtils.isBlank(context)) {
throw createConfigurationError("Context cannot be blank");
}
return context;
}
return context;
}

private String renderTemplate(Object context, String value) {
return templateEngine.getUncachedTemplate(value).apply(context);
}
}
Loading

0 comments on commit af4474a

Please sign in to comment.