Milestone 4 (#6)

* Start planning milestone 4

* Interceptor conce[t

* Update tests

* Add interceptor support

* remove the extra option

* With interceptors, the endpoints are optional

* Fix the flake8 errors

* Add roadmap items

* Make 'interceptors' default to emtpy tuple

* Move dummy interceptors module to tests directory

* Support 'endpoints' key being optional in services

* Pass separate Request and Response objects to interceptors

* Fix the optional 'endpoints' key support

* Override Application and ErrorHandler classes of tornado.web

Trigger the interceptors at the correct moment of request handling.

* Enable TestInterceptors class and fix some issues related to interceptors

* Fix the issues in tests_integrated

* Revert "Fix the issues in tests_integrated"

This reverts commit 50dcd77.

* Eliminate the need for 'is_error' and 'force_update' attributes in the Response object

* Fix test

* Make 'Request' object a full blown object that resembles all the properties of a HTTP request

* Implement the HTTP verbs other than GET and POST

* Remove 'is_response_str' criteria from 'useTemplating' logic

* Do some minor performance improvements by not creating 'Faker' and 'Compiler' instances each time

* Fix an IndexError

* Complete milestone

* Update the documentation

* Fix the issues in section headers

* Fix a link in README.md

* Fix README.md

* SEt version

Co-authored-by: M. Mert Yildiran <mehmet@up9.com>

.travis.yml

@@ -11,16 +11,16 @@ install:
   - travis_fold start "Build.Image" && docker build . -t mockintosh && docker image ls mockintosh && travis_fold end "Build.Image"
 script:
   - stty cols 120
-  - travis_fold start "Unit.Tests" && pytest tests -s -v && travis_fold end "Unit.Tests"
-  - travis_fold start "Integration.Tests" && docker run -d -p 8000-8010:8000-8010 -v `pwd`:/tmp mockintosh -v -l /tmp/server.log /tmp/tests_integrated/integration_config.json && sleep 5 && pytest -v --log-level=DEBUG tests_integrated/tests_integration.py && travis_fold end "Integration.Tests"
+  - travis_fold start "Unit.Tests" && pytest tests -s -v --log-level=DEBUG && travis_fold end "Unit.Tests"
+  - travis_fold start "Integration.Tests" && docker run -d -p 8000-8010:8000-8010 -v `pwd`/tests_integrated:/tmp/tests_integrated -e PYTHONPATH=/tmp/tests_integrated mockintosh -v -l /tmp/tests_integrated/server.log --interceptor=custom_interceptors.intercept_for_logging --interceptor=custom_interceptors.intercept_for_modifying /tmp/tests_integrated/integration_config.json && sleep 5 && pytest -v --log-level=DEBUG tests_integrated/tests_integration.py && travis_fold end "Integration.Tests"
   - flake8
 after_success:
   - codecov
 after_failure:
-  - travis_fold start "server_log" && ( cat server.log || echo No logfile) && travis_fold end "server_log" # the log from container
+  - travis_fold start "server_log" && ( cat tests_integrated/server.log || echo No logfile) && travis_fold end "server_log" # the log from container
 deploy:
   edge: true  # use latest v2 deploy
   provider: pypi
   #  skip_cleanup: true
   on:
-    tags: true
+    tags: true

Configuring.md

@@ -253,7 +253,7 @@ response:
   status: '403'
 ```
 
-as the status code in the response definition.
+as the status code in the response definition. If the status code is not specified it defaults to `200`.
 
 It's also possible to use templating in the `status` field like this:
 
@@ -293,13 +293,75 @@ globals:
           Cache-Control: no-cache
 ```
 
-### Request Object
+### Body
 
-The `request` object is exposed and can be used in places where the templating is possible. These are its attributes:
+The body can be direct response string:
+
+```yaml
+response:
+  body: 'hello world'
+```
+
+or a string that starts with `@` sign to indicate a separete template file:
 
-```text
-request.method
-request.path
-request.headers.<key>
-request.queryString.<key>
+```yaml
+response:
+  body: '@some/path/my_template.json.hbs'
 ```
+
+## Request Object
+
+The `request` object is exposed and can be used in places where the templating is possible. These are its attributes:
+
+#### `request.version`
+
+HTTP version e.g. `HTTP/1.1`, [see](https://tools.ietf.org/html/rfc2145).
+
+#### `request.remoteIp`
+
+The IP address of the client e.g. `127.0.0.1`.
+
+#### `request.request.protocol`
+
+The HTTP protocol e.g. `http` or `https`.
+
+#### `request.host`
+
+Full address of host e.g. `localhost:8001`.
+
+#### `request.hostName`
+
+Only the hostname e.g. `localhost`.
+
+#### `request.uri`
+
+URI, full path segments including the query string e.g. `/some/path?a=hello%20world&b=3`.
+
+#### `request.method`
+
+[HTTP methods](https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html). Supported verbs are:
+`HEAD`, `GET`, `POST`, `DELETE`, `PATCH`, `PUT` and `OPTIONS`.
+
+#### `request.path`
+
+The path part of the URI e.g. `/some/path`.
+
+#### `request.headers.<key>`
+
+A request header e.g. `request.headers.accept` is `*/*`.
+
+#### `request.queryString.<key>`
+
+A query parameter e.g. `request.queryString.a` is `hello world`.
+
+#### `request.body`
+
+The raw request body as a whole. Can be `str`, `bytes` or `dict`.
+
+#### `request.formData.<key>`
+
+The `POST` parameters sent in a `application/x-www-form-urlencoded` request e.g. `request.formData.param1` is `value1`.
+
+#### `request.files.<key>`
+
+The fields in a `multipart/form-data` request.

README.md

@@ -2,7 +2,7 @@
 
 ## About
 
-Mockintosh aims to provide usual mock service functionality with small resource footprint, making it friendly for
+Mockintosh aims to provide usual HTTP mock service functionality with small resource footprint, making it friendly for
 microservice applications. You can have tens of mocks at once, inside moderate laptop or single Docker container. Also,
 we have some additional ideas on how to make mocks simple and useful.
 
@@ -100,3 +100,55 @@ cat tests/configs/json/hbs/common/config.json | mockintosh
 
 Using `--quiet` and `--verbose` options the logging level can be changed.
 
+### Interceptors
+
+One can also specify a list of interceptors to be called in `<package>.<module>.<function>` format using
+the `--interceptor` option. The interceptor function get a [`mockintosh.Request`](#request-object) and
+a [`mockintosh.Response`](#response-object) instance. Here is an example interceptor that for
+every requests to a path starts with `/admin`, sets the reponse status code to `403`:
+
+```python
+import re
+
+def forbid_admin(req: Request, resp: Response):
+    if re.match(r'^\/admin.*$', req.path):
+        resp.status = 403
+```
+
+and you would specify such interceptor with a command like below:
+
+```bash
+mockintosh some_config.json --interceptor=mypackage.mymodule.forbid_admin
+```
+
+Instead of specifying a package name, you can alternatively set the `PYTHONPATH` environment variable
+to a directory that contains your interceptor modules like this:
+
+```bash
+PYTHONPATH=/some/dir mockintosh some_config.json --interceptor=mymodule.forbid_admin
+```
+
+#### Request Object
+
+The `Request` object is exactly the same object defined in [here](Configuring.md#request-object)
+with a minor difference: Instead of accesing the dictonary elements using `.<key>`,
+you access them using `['<key>']` e.g. `request.queryString['a']`.
+
+#### Response Object
+
+The `Response` object consists of three fields:
+
+##### Status
+
+`resp.status` holds the [HTTP status codes](https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html)
+e.g. `200`, `201`, `302`, `404`, etc.
+
+##### Headers
+
+`resp.headers` is a Python dictionary that you access and/or modify the response headers.
+e.g. `resp.headers['Cache-Control'] == 'no-cache'`
+
+##### Body
+
+The body can be anything that supported by [`tornado.web.RequestHandler.write`](https://www.tornadoweb.org/en/stable/web.html#tornado.web.RequestHandler.write)
+Which means the body can be `str`, `bytes` or `dict` e.g. `resp.body = 'hello world'` or `resp.body = {'hello': 'world'}`

Roadmap.md

@@ -29,7 +29,19 @@
 3. Response status code   
 4. response headers - global and local
 
-## Milestone 4
+## Milestone 4 - Complete
+
+1. Integration of custom "interceptor" 
+   - Object model for request and response
+   - Ability to provide python function/object to alter response
+   - Both successful and failed responses intercepted
+
+## Milestone N
+
+1. base64-encoded body strings, for binary responses
+1. SSL support
+
+## Milestone N
 
 1. Multi-response functionality
 1. Configuration-by-request
@@ -55,12 +67,6 @@
 - jsonSchema extraction from body
 - referencing multipart/urlencoded fields in matchers and templates
 
-## Milestone N
-
-1. Extensibility/integrations
-    - Object model for request and response
-    - Ability to provide python function/object to alter response
-    - Ability to provide python function/object to access the request/response notifications
 
 ## Milestone N
 
@@ -79,7 +85,13 @@
 
 # Backlog
 
-## Config Example
+## Ideas
+
+- `mockintosh --cli` to start interactive shell that would allow building the mock configuration interactively
+- `cat OpenAPI.json | mockintosh > mockintosh-config.yml`
+- mocks for gRPC servers
+
+## Config Ideas
 
 ```json5
 {
@@ -181,7 +193,3 @@
 }
 ```
 
-# Ideas
-
-- `mockintosh --cli` to start interactive shell that would allow building the mock configuration interactively
-- `cat OpenAPI.json | mockintosh > mockintosh-config.yml`

mockintosh/__init__.py

@@ -18,11 +18,12 @@
 
 from mockintosh import configs
 from mockintosh.exceptions import UnrecognizedConfigFileFormat
-from mockintosh.methods import _detect_engine, _nostderr
+from mockintosh.methods import _detect_engine, _nostderr, _import_from
 from mockintosh.recognizers import PathRecognizer, HeadersRecognizer, QueryStringRecognizer
 from mockintosh.servers import HttpServer
+from mockintosh.handlers import Request, Response  # noqa: F401
 
-__version__ = "0.0"
+__version__ = "0.1"
 __location__ = path.abspath(path.dirname(__file__))
 
 
@@ -64,6 +65,8 @@ def validate(self):
 
     def analyze(self):
         for service in self.data['services']:
+            if 'endpoints' not in service:
+                continue
             for endpoint in service['endpoints']:
                 endpoint['params'] = {}
                 endpoint['context'] = OrderedDict()
@@ -105,7 +108,19 @@ def get_schema():
     return schema
 
 
-def run(source, is_file=True, debug=False):
+def import_interceptors(interceptors):
+    imported_interceptors = []
+    if interceptors is not None:
+        if 'unittest' in sys.modules.keys():
+            tests_dir = path.join(__location__, '../tests')
+            sys.path.append(tests_dir)
+        for interceptor in interceptors:
+            module, name = interceptor[0].rsplit('.', 1)
+            imported_interceptors.append(_import_from(module, name))
+    return imported_interceptors
+
+
+def run(source, is_file=True, debug=False, interceptors=()):
     schema = get_schema()
 
     if 'unittest' in sys.modules.keys():
@@ -118,7 +133,7 @@ def run(source, is_file=True, debug=False):
 
     try:
         definition = Definition(source, schema, is_file=is_file)
-        http_server = HttpServer(definition, debug=debug)
+        http_server = HttpServer(definition, debug=debug, interceptors=interceptors)
     except Exception:
         logging.exception('Mock server loading error:')
         with _nostderr():
@@ -140,9 +155,12 @@ def initiate():
     ap.add_argument('-d', '--debug', help='Enable Tornado Web Server\'s debug mode', action='store_true')
     ap.add_argument('-q', '--quiet', help='Less logging messages, only warnings and errors', action='store_true')
     ap.add_argument('-v', '--verbose', help='More logging messages, including debug', action='store_true')
+    ap.add_argument('-i', '--interceptor', help='A list of interceptors to be called in <package>.<module>.<function> format.', action='append', nargs='+')
     ap.add_argument('-l', '--logfile', help='Also write log into a file', action='store')
     args = vars(ap.parse_args())
 
+    interceptors = import_interceptors(args['interceptor'])
+
     fmt = "[%(asctime)s %(name)s %(levelname)s] %(message)s"
     if args['quiet']:
         logging.basicConfig(level=logging.WARNING, format=fmt)
@@ -156,4 +174,4 @@ def initiate():
         handler.setFormatter(logging.Formatter(fmt))
         logging.getLogger('').addHandler(handler)
 
-    run(args['source'], debug=args['debug'])
+    run(args['source'], debug=args['debug'], interceptors=interceptors)