diff --git a/docs/drivers.md b/docs/drivers.md deleted file mode 100644 index 1a1db66..0000000 --- a/docs/drivers.md +++ /dev/null @@ -1,67 +0,0 @@ -# Driver support - -You may add integration with other http caches using the extension points provided by this bundle. - -The following extension points are available - - PurgeClient - - TagHandler - - FOS TagHandler - -If you write a new PurgeClient driver, you **must** also create a corresponding TagHandler and vice -versa. Creating a FOS TagHandler is optional. - - -## PurgeClient - -The PurgeClient is responsible for sending purge requests to the http cache when content is about to be invalidated. -The PurgeClient must implement EzSystems\PlatformHttpCacheBundle\PurgeClient\PurgeClientInterface and can be registered -with the following code in services.yml: - -``` -services: - ezplatform.http_cache_myhttpcachebundle.purge_client.myhttpcache: - class: EzSystems\PlatformMyHttpCacheBundle\PurgeClient\MyHttpCachePurgeClient - arguments: ['@ezplatform.http_cache.cache_manager'] - tags: - - {name: ibexa.cache.http.purge_client, purge_type: myhttpcache} -``` - -Any service which implements the PurgeClientInterface must be tagged with `ibexa.cache.http.purge_client` in -order to be registered as such. - -`purge_type` specifies what the value of the `ibexa.http_cache.purge_type` setting in `config/packages/ibexa.yaml` -should be in order to enable this driver. By default this is set using `%purge_type%` parameter. - - -## TagHandler - -The TagHandler is responsible for tagging responses with headers which the http cache recognizes. -The TagHandler must implement EzSystems\PlatformHttpCacheBundle\Handler\TagHandlerInterface and can be registered with -the following code in services.yml: - -``` - ezplatform.http_cache_myhttpcachebundle.tag_handler.myhttpcache: - class: EzSystems\PlatformMyHttpCacheBundle\Handler\MyHttpCacheTagHandler - tags: - - {name: ezplatform.http_cache.tag_handler, purge_type: myhttpcache} - -``` - -Any service which implements the TagHandlerInterface must be tagged with `ezplatform.http_cache.tag_handler` in order -to be registered as such. - -## FOS TagHandler - -The FOS Http cache bundle also has a TagHandler which is not used by eZ Platform except for one thing, the -`fos:httpcache:invalidate:tag` command. With this command you may explicitly invalidate cache by tag. - -Normally, you would not need to implement your own FOS TagHandler as the ezplatform-http-cache bundle ships with a -default one which uses the PurgeClient to invalidate the given tags. -If you need to write your own FOS TagHandler anyway, you may register it with the following code in services.yml: - -``` - ezplatform.http_cache_myhttpcachebundle.fos_tag_handler.myhttpcache: - class: EzSystems\PlatformMyHttpCacheBundle\Handler\MyHttpCacheFosTagHandler - tags: - - {name: ezplatform.http_cache.fos_tag_handler, purge_type: myhttpcache} -``` diff --git a/docs/extensibility.md b/docs/extensibility.md deleted file mode 100644 index f38c5f2..0000000 --- a/docs/extensibility.md +++ /dev/null @@ -1,71 +0,0 @@ -# Extending ezplatform-http-cache - -## Registering custom configuration -This package defines the `http_cache` configuration blocks of the ezplatform semantic configuration: - -```yaml -ezpublish: - system: - default: - http_cache: - purge_servers: [] -``` - -For extensions that require extra siteaccess aware configuration for HTTP cache features, -an extension points exists. Instead of registering a new Config Parser with the `ezpublish` -container extension, one should do so on the `ezplatform_http_cache` one. - -```php -class MyCustomBundle -{ - public function build(ContainerBuilder $container) - { - $cacheExtension = $container->getExtension('ibexa_http_cache'); - $cacheExtension->addExtraConfigParser(new CustomConfigParser()); - } -} -``` - -```php -namespace MyCustomBundle\DependencyInjection\ConfigResolver; - -use eZ\Bundle\EzPublishCoreBundle\DependencyInjection\Configuration\ParserInterface; - -class CustomConfigParser implements ParserInterface -{ - public function addSemanticConfig(NodeBuilder $nodeBuilder) - { - $nodeBuilder->addScalarNode('custom_setting')->end(); - } - - public function mapConfig(array &$scopeSettings, $currentScope, ContextualizerInterface $contextualizer) - { - if (isset($scopeSettings['foo'])) { - $contextualizer->setContextualParameter('http_cache.foo', $currentScope, $scopeSettings['foo']); - } - } -} -``` - -The `ParserInterface` implementation (`CustomConfigParser`) plays the same role than a regular one, with -two exceptions: -- in `addSemanticConfig()`, items are added to the `http_cache` node. -- in `mapConfig()`, the provided `$scopeSettings` contains the *contents* of the `http_cache` configuration key. - -With the example above, `bin/console config:dump-reference ezpublish` will contain: - -```yaml -ezpublish: - system: - siteaccess_name: - http_cache: - - # This is defined by ezplatform-http-cache - purge_servers: - - # Examples: - - http://localhost/ - - http://another.server/``` - # This is defined by the example - foo: ~ -``` diff --git a/docs/fos_http_cache.md b/docs/fos_http_cache.md deleted file mode 100644 index c058cc0..0000000 --- a/docs/fos_http_cache.md +++ /dev/null @@ -1,99 +0,0 @@ -# FOSHttpCacheBundle usage in eZ - -_This doc is aimed at core developers, and explains how this bundle integrates with FOSHttpCacheBundle internally._ - - -eZ Platform HttpCache _(this bundle)_ uses [FOSHttpCacheBundle 1.x][fos] in the following way: -- When `local` purge type is set, the system uses a custom Symfony Proxy tag-aware data store, and hence does not - use FOSHttpCacheBundle directly. However FOSHttpCache(Bundle) can still be used to manipulate responses, incl tagging. -- When `http` purge type is set, the system uses FOSHttpCache configured with varnish as proxy client. - -As HttpCache in eZ Publish/Platform predates FOSHttpCache, we have own abstractions for: - - purge clients: In this bundle exposed as `ezplatform.http_cache.purge_client` service - - App Cache: `EzSystems\PlatformHttpCacheBundle\AppCache` app cache class to extend from, this in turn extends - `FOS\HttpCacheBundle\SymfonyCache\EventDispatchingHttpCache`, however it provides own set of request-aware purgers and - own handling of user context hash which for BC reasons is called `X-User-Hash` in eZ. - - Since version 1.0, FOSHttpCache's `X-User-Context-Hash` header is used for user context hash. - -_Note: Once FOSHttpCache(Bundle) has full support for tagging, a major version of this bundle might be be refactored to more -directly reuse FOSHttpCache(Bundle). However in the meantime our abstractions have allowed us to ship features across -Symfony Cache and Varnish such as full tagging support not possible with plain FOSHttpCacheBundle._ - -Some further differences on how HttpCache behaves on specific features in this bundle can be found below. - -## Http cache clear -Varnish proxy client from FOSHttpCache lib is used for clearing cache when `http` purge client is configured.. -It sends, for each cache tag that needs to be expired, a `PURGE` request with a `key` header to the registered purge servers. - -For cache clearing to work properly, you need to use the VCL from the [ezplatform `doc/varnish` directory][varnish_doc]. - -## User context hash -[FOSHttpCacheBundle *User Context feature* is used][fos_user_context] is activated by default. - -As the response can vary on a request header, the base solution is to make the kernel do a sub-request in order to retrieve -the context (the **user context hash**). Once the *user hash* has been retrieved, it is injected into the original request -as the `X-User-Hash` header, making it possible to *vary* the HTTP response on this header: - -> The name of the [user hash header is configurable in FOSHttpCacheBundle][fos_user_context]. -> By default eZ Publish sets it to `**X-User-Hash**`. - -```php -setVary( 'X-User-Hash' ); -``` - -This solution is [implemented in Symfony reverse proxy (aka *HttpCache*)][fos_symfony_cache] -and is also accessible to [dedicated reverse proxies like Varnish][fos_varnish_cache]. - - -### How it works -Please refer to [FOSHttpCacheBundle documentation on how user context feature works][fos_user_context#how]. - -### User hash generation -Please refer to [FOSHttpCacheBundle documentation on how user hashes are being generated][fos_user_context#hashes]. - -eZ Platform already interferes in the hash generation process, by adding current user permissions and limitations. -One can also interfere in this process by [implementing custom context provider(s)][fos_user_context#providers]. - -### Varnish VCL -While the described behavior comes out of the box with Symfony reverse proxy, Varnish is also supported. Using the documented -[eZ Platform Varnish VCL][_doc]. - - -## Default options for FOSHttpCacheBundle defined in eZ -The following configuration is defined in eZ by default for FOSHttpCacheBundle. -You may override these settings, however for the configured headers changing them will break code both here as well as -in other parts of the ez ecosystem. - -```yaml -fos_http_cache: - proxy_client: - # "varnish" is used, even when using Symfony HttpCache. - default: varnish - varnish: - # Means http_cache.purge_servers defined for current SiteAccess. - servers: [$http_cache.purge_servers$] - - user_context: - enabled: true - # User context hash is cached during 10min - hash_cache_ttl: 600 - user_hash_header: X-User-Hash - tags: - header: xkey -``` - -[varnish_doc]: https://github.com/ezsystems/ezplatform/blob/master/doc/varnish -[fos]: http://foshttpcachebundle.readthedocs.org/ -[fos_user_context]: http://foshttpcachebundle.readthedocs.io/en/1.3/features/user-context.html -[fos_user_context#how]: http://foshttpcachebundle.readthedocs.io/en/1.3/features/user-context.html#how-it-works -[fos_user_context#providers]: http://foshttpcachebundle.readthedocs.io/en/1.3/reference/configuration/user-context.html#custom-context-providers -[fos_user_context_hashes]: http://foshttpcachebundle.readthedocs.io/en/1.3/features/user-context.html#generating-hashes -[fos_symfony_cache]: http://foshttpcachebundle.readthedocs.io/en/1.3/features/symfony-http-cache.html -[fos_varnish_cache]: http://foshttpcache.readthedocs.io/en/1.4/varnish-configuration.html diff --git a/docs/response_taggers.md b/docs/response_taggers.md deleted file mode 100644 index 452e2da..0000000 --- a/docs/response_taggers.md +++ /dev/null @@ -1,36 +0,0 @@ -# Response taggers API - -> added in ezpublish-kernel 6.8 - -ResponseTaggers will take a `Response`, a `ResponseConfigurator` and any value object, and will add tags to the Response -based on the value. - -## Example -This value tagger will add the `content-`, `location-` and `content-type-` tags to the -Response: - -```php -$contentInfoResponseTagger->tag($response, $configurator, $contentInfo); -``` - -In this example ContentInfo is gotten by a "delegator" which takes `ContentView` as input and passes the relevant value object from it to value taggers like the one above. _Keep on reading below for further info._ - -## The ResponseConfigurator -A `ResponseCacheConfigurator` configures an HTTP Response object: make the response public, add tags, set the shared max -age... It is provided to `ResponseTaggers` who use it to add the tags to the Response. - -The `ConfigurableResponseCacheConfigurator` (`ezplatform.view_cache.response_configurator`) will follow the `view_cache` -configuration, and only enable cache if it is enabled in the configuration. - -## Delegator and Value Taggers -Even though they share the same API, ResponseTaggers are of two types, reflected by their namespace: Delegator and Value. - -Delegator Taggers will extract another value, or several, from the given value, and pass it on to another tagger. For -instance, a `ContentView` is covered by both the `ContentValueViewTagger` and the `LocationValueViewTagger`. The first will -extract the `Content` from the `ContentView`, and pass it to the `ContentInfoTagger`. The second will extract the `Location`, -and pass it to the `LocationViewTagger`. - -## The Dispatcher Tagger -While it is more efficient to use a known tagger directly, sometimes you don't know what object you want to tag with. -The Dispatcher ResponseTagger will accept any value, and will pass it to every tagger registered with the service tag -`ezplatform.http_response_tagger`. diff --git a/docs/using_tags.md b/docs/using_tags.md deleted file mode 100644 index 4aa0696..0000000 --- a/docs/using_tags.md +++ /dev/null @@ -1,260 +0,0 @@ -# Using Tags - -Understanding tags is the key to making the most of `ezplatform-http-cache`. - -They work in a similar way as [persistence cache tags in eZ Platform v2](https://github.com/ezsystems/ezpublish-kernel/tree/7.0/doc/specifications/cache/persistence): -- A set of secondary keys set on every cache item, on top of "primary key" which in this case is the URI -- Like an index in a database it is typically used for anything relevant that represent the given cache item -- Used for cache invalidation - -It works across all supported proxies _(see ["drivers"](drivers.md))_ by eZ Platform: -- Symfony Proxy _(PHP based for single server usage, primarily for smaller web sites)_ -- [Varnish](https://varnish-cache.org/) with [xkey module (0.10.2+)](https://github.com/varnish/varnish-modules) _or_ [Varnish Plus](https://www.varnish-software.com/products/varnish-plus/) _(High performance reverse proxy)_ -- [Fastly](https://www.fastly.com/) _(High performance reverse proxy, originally based on Varnish, worldwide as a CDN, driver available in eZ Platform Enterprise)_ - -_In order to support several repositories on one installation, tags will be prefixed by -repository index in use. I.e. "0p1", where "0" is the repository prefix._ - -Varnish or Fastly are highly recommended for medium to large traffic needs. Besides being able to handle much more traffic, -supported for use in cluster setup, they also both support soft purge _(by tags)_, meaning they are able to serve stale -content while it's refreshed in the background on-demand, leading to more stable load on your backend. - - -## Tags in use in this bundle - -### Tags for Content responses - -- *Content*:\ - `c`: - _Used on anything that is affected by changes to content, on content itself as well as location and so on._ - -- *Content Version*:\ - `cv`: - _Used for clearing cache for content version list views, when *not* affecting the published content._ - -- *Content Type*:\ - `ct`: - _For use when content type changes affecting content of its type._ - -- *Location*:\ - `l`: - _Used for clearing all cache relevant for a given location._ - -- *Parent Location*:\ - For responses this represent the parent location, on purges it's used in 2 distinct ways:\ - `pl`: - _Used for clearing cache of all siblings of an location._\ - `pl`: - _Used for clearing cache of all the children of a location._ - -- *Path* _(all path elements of a location)_:\ - `p`: - _For operations that change the tree itself, like move/remove/(..)._ - -- *Relations*:\ - `r` & `rl`: - _For use when updates affect all their reverse relations. ezplatform-http-cache does not add this tag to responses - automatically, just purges on it if present, response tagging with this is currently done inline in template logic / views - where relation is actually used for rendering (when using ESI, if inline the Content own tags should be added to response instead, unless you are hitting tag header limits)._ - These differs from `content-` and `location-` by _only_ being purged when relation itself is removed or otherwise affected._ - -### Tags for Section responses - -- `s` : - _For use when section changes affecting section responses (i.e. REST)._ - - -### Tags for ContenType responses - -- `t` : - _For use when content type changes affecting content type responses (i.e. REST)._ - -- `tg` : - _For use when content type group changes affecting content type group responses (i.e. REST)._ - -### Misc internal tags - -- `ez-user-context-hash`: - _Internal tag used for tagging /_fos_user_context_hash to expire it on role & role assigment changes._ - -- `ez-invalidate-token`: - _Internal tag for use by token lookup when in token authentication mode (for setups where IP validation is not possible)._ - -- `ez-all`: - _Internal tag used for being able to clear all cache. Main use case is being able to expire (soft purge) all cache on - deployment of new versions of your installation which for instance changes representation / design dramatically._ - -## How Response tagging is done - - -### For Content View - -For Content View there is a dedicated response listener `HttpCacheResponseSubscriber` that triggers a set of [Response -taggers](docs/response_taggers.md) responsible for translating info from the objects involved in generating the view to -corresponding tags as listed above. These can be found in `src/ResponseTagger`. - - -### For responses with X-Location-Id - -For custom or Ibexa controllers _(like REST at the time of writing)_ still using `X-Location-Id`, a dedicated response -listener `XLocationIdResponseSubscriber` handles translating this to tags so the cache can be properly invalidated by -this bundle. It supports comma separated location id values which was only partially supported in earlier versions: - -```php - /** @var \Symfony\Component\HttpFoundation\Response $response */ - $response->headers->set('X-Location-Id', 123); - - // Alternatively using several location id values, requires ezplatform-http-cache to work across all supported proxies - $response->headers->set('X-Location-Id', '123,212,42'); -``` - -*NOTE: This is currently marked as Deprecated, and for rendering Ibexa content it is thus adviced to refactor to use Content -View. For other needs there is an FOS tag handler for Twig and PHP that can be used, see below for further info.* - - -### For custom needs using FOSHttpCache (tagging relations and more) - -For custom needs, including template logic for Ibexa content relations which is here used for examples, there are two ways -to tag your responses. - -#### Twig use - -For twig usage, you can make sure response is tagged correctly by using the following twig operator in your template: -```twig - {{ fos_httpcache_tag('r33') }} - - {# Or using array for several values #} - {{ fos_httpcache_tag(['r33', 'r44']) }} -``` - -See: http://foshttpcachebundle.readthedocs.io/en/1.3/features/tagging.html#tagging-from-twig-templates - - -However for relations, which you typically used prior to a ESI include for some content, rather use one of: -```twig - {# As of v0.9.3 two twig functions for relation use cases was added, both handling single and array values #} - {# First one is for relation(s) for Content, as shown by it's id #} - {{ ibexa_http_cache_tag_relation_ids(relation_content.id) }} - - {# Second one for relation locations, here shown using array of location id's #} - {{ ibexa_http_cache_tag_relation_location_ids(relation_location_ids) }} -``` - - -Alternatively if you have a location(s) that you render _inline_ & want invalidated on any kind of change: -```twig - {{ ibexa_http_cache_tag_location( location ) }} -``` - -TIP: Don't use `ibexa_http_cache_tag_location` when you are rendering a large amount of content/location items, it will cause tag - header to become to large. Consider using less tags with for instance `ez_http_tag_relation_(location_)ids`, and account for - possible stale cache by reducing cache ttl for the given response. - Also strongly consider to upgrade to ezplatform-http-cache 1.0 or higher which reduces cache tag size. - -#### PHP use - -For PHP usage, a few options exist _(autowirable classes of '@fos_http_cache.handler.tag_handler')_: -```php - /** - * Using low level response tagger to add tags manually. - * - * @var \FOS\HttpCache\ResponseTagger $responseTagger - */ - $responseTagger->addTags(['r33', 'r44']); - - /** - * Better option if you need to add Ibexa specific tags. - * - * @var \EzSystems\PlatformHttpCacheBundle\Handler\ContentTagInterface $responseTagger - */ - $responseTagger->addRelationTags([33, 44]); -``` - -See: http://foshttpcachebundle.readthedocs.io/en/1.3/features/tagging.html#tagging-from-code - -## How purge tagging is done (invalidation) - -This bundle uses Repository API Slots to listen to Signals emitted on repository operations, and depending on the -operation triggers expiry on a specific tag or set of tags. - -E.g. on Move Location signal the following tags will be purged: -```php -use EzSystems\PlatformHttpCacheBundle\Handler\ContentTagInterface; - - /** - * @param \eZ\Publish\Core\SignalSlot\Signal\LocationService\MoveSubtreeSignal $signal - */ - protected function generateTags(Signal $signal) - { - return [ - // The tree itself being moved (all children will have this tag) - ContentTagInterface::PATH_PREFIX . $signal->locationId, - // old parent - ContentTagInterface::LOCATION_PREFIX . $signal->oldParentLocationId, - // old siblings - ContentTagInterface::PARENT_LOCATION_PREFIX . $signal->oldParentLocationId, - // new parent - ContentTagInterface::LOCATION_PREFIX . $signal->newParentLocationId, - // new siblings - ContentTagInterface::PARENT_LOCATION_PREFIX . $signal->newParentLocationId, - ]; - } -``` - -All slots can be found in `src/SignalSlot` - -## Troubleshooting - -### Understanding tag response headers - -Given a header like `Xkey: 0c22 0ct2 (...)`, the following can be understood: -- `Xkey:` Configured tag header, by this bundle config. Understood by Varnish _(with xkey module)_, and by our enhanced version of Symfony HttpCache Proxy "AppCache". -- `0..`: Current repository index, set for you by this bundle. Done in order to support multi repository setups, it's the array key of the current repository is used _(and not the full identifier)_. -- `..c22`: Content id 22 -- `..ct2`: Content Type id 2 - -### Header limits - -Even if the tags are kept as short as possible, you might still encounter issue with tag header exceeding -limits in Varnis/Apache/Nginx/Fastly, stopping either caching or invalidation from working as expected. - -For handling these cases it's simplest to increase the limits on the service side if possible, and if not look at the -code involved to see if amount of tags can be reduced. - -#### Configuring Services - -*Varnish* config: -- [http_resp_hdr_len](https://varnish-cache.org/docs/6.0/reference/varnishd.html#http-resp-hdr-len) (default 8k, change to i.e. 32k) -- [http_max_hdr](https://varnish-cache.org/docs/6.0/reference/varnishd.html#http-max-hdr) (default 64, change to i.e. 128) -- [http_resp_size](https://varnish-cache.org/docs/6.0/reference/varnishd.html#http-resp-size) (default 23k, change to i.e. 96k) -- [workspace_backend](https://varnish-cache.org/docs/6.0/reference/varnishd.html#workspace-backend) (default 64k, change to i.e. 128k) - -*Fastly* has a `Surrogate-Key` header limit of *16kb*, this can *not* be changed. - -*Apache* has a [hard](https://github.com/apache/httpd/blob/5f32ea94af5f1e7ea68d6fca58f0ac2478cc18c5/server/util_script.c#L495) [coded](https://github.com/apache/httpd/blob/7e2d26eac309b2d79e467ef586526c10e0f226f8/include/httpd.h#L299-L303) limit of 8kb, so if you face this issue consider using nginx instead. - -*Nginx* has a default limit of 4k/8k when buffering responses from PHP-fpm/fast-cgi: -- https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size -- https://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffer_size - - -#### Limit tags header output by system - -Typical case with too many tags would be when inline rendering a lot of embed content object. -Normally the system will add all the tags for this content, to handle every possible scenario of updates to them. - -So if you embed hundreds of content on the same page _(i.e. in richtext, using relations, or using page builder)_, it will explode the tag usage. - -However if for instance you just display the content name, image attribute, and/or link, then it would be enough to: -- Just use `r` tag, or preferably the abstractions for it. -- Optionally: Set reduced cache TTL for the given view in order to reduce remote risk of subtree operations affecting the cached page - without correctly purging the view. - -If that is not an option, you can opt-in to set a max length parameter (in bytes) and corresponding ttl (in seconds): -```yaml -parameters: - # Warning, setting this means you risk losing tag information, risking stale cache. Here set below 8k: - ibexa.http_cache.tags.header_max_length: 7900 - # In order to reduce risk of stale cache issues, you should set a lower TTL here then globally (here set as 2h) - ibexa.http_cache.tags.header_reduced_ttl: 7200 -``` diff --git a/docs/varnish/vcl/varnish5.vcl b/docs/varnish/vcl/varnish5.vcl index f59c022..15819da 100644 --- a/docs/varnish/vcl/varnish5.vcl +++ b/docs/varnish/vcl/varnish5.vcl @@ -1,7 +1,9 @@ +// IMPORTANT: this is a copy of varnish6.vcl file kept for BC reasons and it will be removed in 5.0 +// make sure to apply changes both to this file and varnish6.vcl // Varnish VCL for: -// - Varnish 6.0 or higher (6.0LTS recommended, and is what we mainly test against) +// - Varnish 6.0LTS // - Varnish xkey vmod (via varnish-modules package 0.10.2 or higher, or via Varnish Plus) -// - eZ Platform 3.x or higher with ezplatform-http-cache (this) bundle +// // // Make sure to at least adjust default parameters.vcl, defaults there reflect our testing needs with docker. diff --git a/docs/varnish/vcl/varnish6.vcl b/docs/varnish/vcl/varnish6.vcl new file mode 100644 index 0000000..a5c8bbf --- /dev/null +++ b/docs/varnish/vcl/varnish6.vcl @@ -0,0 +1,338 @@ +// IMPORTANT: this file has its 1:1 copy named varnish5.vcl and kept for BC reasons, to be removed in 5.0 +// make sure to apply changes both to this file and varnish5.vcl +// Varnish VCL for: +// - Varnish 6.0LTS +// - Varnish xkey vmod (via varnish-modules package 0.10.2 or higher, or via Varnish Plus) +// +// +// Make sure to at least adjust default parameters.vcl, defaults there reflect our testing needs with docker. + +vcl 4.1; +import std; +import xkey; + +// For customizing your backend and acl rules see parameters.vcl +include "parameters.vcl"; + +// Called at the beginning of a request, after the complete request has been received +sub vcl_recv { + + // Set the backend + set req.backend_hint = ezplatform; + + // Add a Surrogate-Capability header to announce ESI support. + set req.http.Surrogate-Capability = "abc=ESI/1.0"; + + // Ensure that the Symfony Router generates URLs correctly with Varnish + if (req.http.X-Forwarded-Proto == "https" ) { + set req.http.X-Forwarded-Port = "443"; + } else { + set req.http.X-Forwarded-Port = "80"; + } + + // Trigger cache purge if needed + call ez_purge; + + // Don't cache requests other than GET and HEAD. + if (req.method != "GET" && req.method != "HEAD") { + return (pass); + } + + // Don't cache Authenticate & Authorization + // You may remove this when using REST API with basic auth. + if (req.http.Authenticate || req.http.Authorization) { + if (client.ip ~ debuggers) { + set req.http.X-Debug = "Not Cached according to configuration (Authorization)"; + } + return (hash); + } + + // Remove all cookies besides Session ID, as JS tracker cookies and so will make the responses effectively un-cached + if (req.http.cookie) { + set req.http.cookie = ";" + req.http.cookie; + set req.http.cookie = regsuball(req.http.cookie, "; +", ";"); + set req.http.cookie = regsuball(req.http.cookie, ";(eZSESSID[^=]*)=", "; \1="); + set req.http.cookie = regsuball(req.http.cookie, ";(ibexa-[^=]*)=", "; \1="); + set req.http.cookie = regsuball(req.http.cookie, ";[^ ][^;]*", ""); + set req.http.cookie = regsuball(req.http.cookie, "^[; ]+|[; ]+$", ""); + + if (req.http.cookie == "") { + // If there are no more cookies, remove the header to get page cached. + unset req.http.cookie; + } + } + + // Do a standard lookup on assets (these don't vary by user context hash) + // Note that file extension list below is not extensive, so consider completing it to fit your needs. + if (req.url ~ "\.(css|js|gif|jpe?g|bmp|png|tiff?|ico|img|tga|wmf|svg|swf|ico|mp3|mp4|m4a|ogg|mov|avi|wmv|zip|gz|pdf|ttf|eot|wof)$") { + return (hash); + } + + // Sort the query string for cache normalization. + set req.url = std.querysort(req.url); + + // Retrieve client user context hash and add it to the forwarded request. + call ez_user_context_hash; + + // If it passes all these tests, do a lookup anyway. + return (hash); +} + +// Called when a cache lookup is successful. The object being hit may be stale: It can have a zero or negative ttl with only grace or keep time left. +sub vcl_hit { + if (obj.ttl >= 0s) { + // A pure unadulterated hit, deliver it + return (deliver); + } + + if (obj.ttl + obj.grace > 0s) { + // Object is in grace, logic below in this block is what differs from default: + // https://varnish-cache.org/docs/5.2/users-guide/vcl-grace.html#grace-mode + if (!std.healthy(req.backend_hint)) { + // Service is unhealthy, deliver from cache + return (deliver); + } else if (req.http.cookie) { + // Request it by a user with session, refresh the cache to avoid issues for editors and forum users + return (miss); + } + + // By default deliver cache, automatically triggers a background fetch + return (deliver); + } + + // fetch & deliver once we get the result + return (miss); +} + +// Called when the requested object has been retrieved from the backend +sub vcl_backend_response { + + if (bereq.http.accept ~ "application/vnd.fos.user-context-hash" + && beresp.status >= 500 + ) { + return (abandon); + } + + // Check for ESI acknowledgement and remove Surrogate-Control header + if (beresp.http.Surrogate-Control ~ "ESI/1.0") { + unset beresp.http.Surrogate-Control; + set beresp.do_esi = true; + } + + // Make Varnish keep all objects for up to 1 hour beyond their TTL, see vcl_hit for Request logic on this + set beresp.grace = 1h; + + // Compressing the content + if (beresp.http.Content-Type ~ "application/javascript" + || beresp.http.Content-Type ~ "application/json" + || beresp.http.Content-Type ~ "application/vnd.ms-fontobject" + || beresp.http.Content-Type ~ "application/vnd.ibexa.api" + || beresp.http.Content-Type ~ "application/x-font-ttf" + || beresp.http.Content-Type ~ "image/svg+xml" + || beresp.http.Content-Type ~ "text/css" + || beresp.http.Content-Type ~ "text/plain" + ) { + set beresp.do_gzip = true; + } + + // Modify xkey header to add translation suffix + if (beresp.http.xkey && beresp.http.x-lang) { + set beresp.http.xkey = beresp.http.xkey + " " + regsuball(beresp.http.xkey, "(\S+)", "\1" + beresp.http.x-lang); + } +} + +// Handle purge +// You may add FOSHttpCacheBundle tagging rules +// See http://foshttpcache.readthedocs.org/en/latest/varnish-configuration.html#id4 +sub ez_purge { + // Retrieve purge token, needs to be here due to restart, match for PURGE method done within + call ez_invalidate_token; + + # Adapted with acl from vendor/friendsofsymfony/http-cache/resources/config/varnish/fos_tags_xkey.vcl + if (req.method == "PURGEKEYS") { + call ez_purge_acl; + + # If neither of the headers are provided we return 400 to simplify detecting wrong configuration + if (!req.http.xkey-purge && !req.http.xkey-softpurge) { + return (synth(400, "Neither header XKey-Purge or XKey-SoftPurge set")); + } + + # Based on provided header invalidate (purge) and/or expire (softpurge) the tagged content + set req.http.n-gone = 0; + set req.http.n-softgone = 0; + if (req.http.xkey-purge) { + set req.http.n-gone = xkey.purge(req.http.xkey-purge); + } + + if (req.http.xkey-softpurge) { + set req.http.n-softgone = xkey.softpurge(req.http.xkey-softpurge); + } + + return (synth(200, "Purged "+req.http.n-gone+" objects, expired "+req.http.n-softgone+" objects")); + } + + # Adapted with acl from vendor/friendsofsymfony/http-cache/resources/config/varnish/fos_purge.vcl + if (req.method == "PURGE") { + call ez_purge_acl; + + return (purge); + } +} + +sub ez_purge_acl { + if (req.http.x-invalidate-token) { + if (req.http.x-invalidate-token != req.http.x-backend-invalidate-token) { + return (synth(405, "Method not allowed")); + } + } else if (!client.ip ~ invalidators) { + return (synth(405, "Method not allowed")); + } +} + +// Sub-routine to get client user context hash, used to for being able to vary page cache on user rights. +sub ez_user_context_hash { + + // Prevent tampering attacks on the hash mechanism + if (req.restarts == 0 + && (req.http.accept ~ "application/vnd.fos.user-context-hash" + || req.http.x-user-context-hash + ) + ) { + return (synth(400)); + } + + if (req.restarts == 0 && (req.method == "GET" || req.method == "HEAD")) { + // Backup accept header, if set + if (req.http.accept) { + set req.http.x-fos-original-accept = req.http.accept; + } + set req.http.accept = "application/vnd.fos.user-context-hash"; + + // Backup original URL + set req.http.x-fos-original-url = req.url; + set req.url = "/_fos_user_context_hash"; + + // Force the lookup, the backend must tell not to cache or vary on all + // headers that are used to build the hash. + return (hash); + } + + // Rebuild the original request which now has the hash. + if (req.restarts > 0 + && req.http.accept == "application/vnd.fos.user-context-hash" + ) { + set req.url = req.http.x-fos-original-url; + unset req.http.x-fos-original-url; + if (req.http.x-fos-original-accept) { + set req.http.accept = req.http.x-fos-original-accept; + unset req.http.x-fos-original-accept; + } else { + // If accept header was not set in original request, remove the header here. + unset req.http.accept; + } + + // Force the lookup, the backend must tell not to cache or vary on the + // user context hash to properly separate cached data. + + return (hash); + } +} + +// Sub-routine to get invalidate token. +sub ez_invalidate_token { + // Prevent tampering attacks on the token mechanisms + if (req.restarts == 0 + && (req.http.accept ~ "application/vnd.ezplatform.invalidate-token" + || req.http.x-backend-invalidate-token + ) + ) { + return (synth(400)); + } + + if (req.restarts == 0 && (req.method == "PURGE" || req.method == "PURGEKEYS") && req.http.x-invalidate-token) { + set req.http.accept = "application/vnd.ezplatform.invalidate-token"; + + // Backup original http properties + set req.http.x-fos-token-url = req.url; + set req.http.x-fos-token-method = req.method; + + set req.url = "/_ibexa_http_invalidatetoken"; + + // Force the lookup + return (hash); + } + + // Rebuild the original request which now has the invalidate token. + if (req.restarts > 0 + && req.http.accept == "application/vnd.ezplatform.invalidate-token" + ) { + set req.url = req.http.x-fos-token-url; + set req.method = req.http.x-fos-token-method; + unset req.http.x-fos-token-url; + unset req.http.x-fos-token-method; + unset req.http.accept; + } +} + +sub vcl_deliver { + // On receiving the invalidate token response, copy the invalidate token to the original + // request and restart. + if (req.restarts == 0 + && resp.http.content-type ~ "application/vnd.ezplatform.invalidate-token" + ) { + set req.http.x-backend-invalidate-token = resp.http.x-invalidate-token; + + return (restart); + } + + // On receiving the hash response, copy the hash header to the original + // request and restart. + if (req.restarts == 0 + && resp.http.content-type ~ "application/vnd.fos.user-context-hash" + ) { + set req.http.x-user-context-hash = resp.http.x-user-context-hash; + + return (restart); + } + + // If we get here, this is a real response that gets sent to the client. + + // Remove the vary on user context hash, this is nothing public. Keep all + // other vary headers. + if (resp.http.Vary ~ "X-User-Context-Hash") { + set resp.http.Vary = regsub(resp.http.Vary, "(?i),? *X-User-Context-Hash *", ""); + set resp.http.Vary = regsub(resp.http.Vary, "^, *", ""); + if (resp.http.Vary == "") { + unset resp.http.Vary; + } + + // If we vary by user hash, we'll also adjust the cache control headers going out by default to avoid sending + // large ttl meant for Varnish to shared proxies and such. We assume only session cookie is left after vcl_recv. + if (req.http.cookie) { + // When in session where we vary by user hash we by default avoid caching this in shared proxies & browsers + // For browser cache with it revalidating against varnish, use for instance "private, no-cache" instead + set resp.http.cache-control = "private, no-cache, no-store, must-revalidate"; + } else if (resp.http.cache-control ~ "public") { + // For non logged in users we allow caching on shared proxies (mobile network accelerators, planes, ...) + // But only for a short while, as there is no way to purge them + set resp.http.cache-control = "public, s-maxage=600, stale-while-revalidate=300, stale-if-error=300"; + } + } + + if (client.ip ~ debuggers) { + // Add X-Cache header if debugging is enabled + if (obj.hits > 0) { + set resp.http.X-Cache = "HIT"; + set resp.http.X-Cache-Hits = obj.hits; + set resp.http.X-Cache-TTL = obj.ttl; + } else { + set resp.http.X-Cache = "MISS"; + } + } else { + // Remove tag headers when delivering to non debug client + unset resp.http.xkey; + unset resp.http.x-lang; + // Sanity check to prevent ever exposing the hash to a non debug client. + unset resp.http.x-user-context-hash; + } +} diff --git a/docs/varnish/vcl/varnish7.vcl b/docs/varnish/vcl/varnish7.vcl index b06ce90..3443e75 100644 --- a/docs/varnish/vcl/varnish7.vcl +++ b/docs/varnish/vcl/varnish7.vcl @@ -1,7 +1,7 @@ // Varnish VCL for: // - Varnish 7.1 or higher // - Varnish xkey vmod (via varnish-modules package 0.10.2 or higher, or via Varnish Plus) -// - eZ Platform 3.x or higher with ezplatform-http-cache (this) bundle +// // // Make sure to at least adjust default parameters.vcl, defaults there reflect our testing needs with docker.