Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(testing): simulate multipart file upload #2141

Open
wants to merge 59 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
d37beaf
Enhancement #2124: added the samesite parameter to unset_cookie.
TigreModerata Jan 16, 2023
f89c31c
Added new option to documentation in cookies.srt
TigreModerata Jan 16, 2023
32daa63
Added tests, CookiesUnsetSameSite class and test_unset_cookies_samesi…
TigreModerata Jan 17, 2023
2b48c6b
Merge branch 'master' into #2124_Enhancement
TigreModerata Jan 17, 2023
6b9b198
Removed unused import logging from test_cookies
TigreModerata Jan 17, 2023
0a17b79
Reverted changes to docs/changes4.0.0.rst
TigreModerata Jan 18, 2023
187aa4f
Reverted changes to docs/changes4.0.0.rst
TigreModerata Jan 18, 2023
9d2fe9e
Added 'files' parameter to _simulate_request; Added tests for file up…
TigreModerata Feb 9, 2023
2b9d0e0
Fixed nested mixed request problems
TigreModerata Feb 9, 2023
835b062
clean up
TigreModerata Feb 10, 2023
3e28a94
updated docstrings
TigreModerata Feb 10, 2023
7920cdd
Fixes
TigreModerata Feb 10, 2023
72d54ee
Merge branch 'master' into SimulateMultipartFile#1010
TigreModerata Feb 10, 2023
ded1ac0
Fixes2
TigreModerata Feb 10, 2023
66d347b
Merge remote-tracking branch 'originTM/SimulateMultipartFile#1010' in…
TigreModerata Feb 10, 2023
574e667
Fixes3
TigreModerata Feb 10, 2023
ffa8d9f
Fixes4
TigreModerata Feb 10, 2023
eadb1f3
urllib3 in minitest requirements (?)
TigreModerata Feb 10, 2023
7493b92
minor fix 5
TigreModerata Feb 10, 2023
c49e369
minor fix 6
TigreModerata Feb 10, 2023
235ed3e
Removed urllib3; creating encoded bodystring in _encode_files.
TigreModerata Feb 12, 2023
94ae979
blued
TigreModerata Feb 12, 2023
c9d4f1b
formatting corrections
TigreModerata Feb 12, 2023
9a9e58d
Corrections after comments
TigreModerata Feb 12, 2023
608efb3
Added test_upload_fileobj
TigreModerata Feb 12, 2023
5c68868
typo fom-data
TigreModerata Feb 13, 2023
f052dca
removed conditional where object type bytes is not possible
TigreModerata Feb 19, 2023
b527de7
added fct testing null data value
TigreModerata Feb 19, 2023
1465241
another unneccesay if removed
TigreModerata Feb 19, 2023
e4cb4c9
Update falcon/testing/client.py
TigreModerata Mar 5, 2023
dba5eb9
Update falcon/testing/client.py
TigreModerata Mar 5, 2023
51ea53e
Update falcon/testing/client.py
TigreModerata Mar 5, 2023
8a66346
typo
TigreModerata Mar 5, 2023
3041583
check for datatype in json changed
TigreModerata Mar 5, 2023
77f350b
Corrections after code-review (part1)
TigreModerata Mar 5, 2023
4bc299a
blued
TigreModerata Mar 5, 2023
df37dbd
unnecessary import removed
TigreModerata Mar 5, 2023
2149b0c
Handling of new data parameter;
TigreModerata Mar 11, 2023
6f8b3ae
blued
TigreModerata Mar 11, 2023
632317c
utf-8 corrected
TigreModerata Mar 11, 2023
3b2edd8
more tests
TigreModerata Mar 11, 2023
bfa6a4c
test for string data
TigreModerata Mar 11, 2023
e1d9834
testing bool and float data types
TigreModerata Mar 11, 2023
514e936
blue & pep8
TigreModerata Mar 11, 2023
dc74456
Docstrings updated
TigreModerata Mar 12, 2023
6d8fd96
data string (not json) treated like body
TigreModerata Mar 26, 2023
eeba35d
Merge branch 'master' into SimulateMultipartFile#1010
vytas7 Jun 5, 2023
aef3e0d
Update 4.0.0.rst
vytas7 Jul 2, 2023
3c2b4fc
chore(requirements): remove extraneous whitespace from `mintest`
vytas7 Jul 3, 2023
8665a34
Merge branch 'master' into SimulateMultipartFile#1010
vytas7 Jul 11, 2023
1b7a207
Merge branch 'master' into SimulateMultipartFile#1010
vytas7 Jul 12, 2023
4572ee8
Merge branch 'master' into SimulateMultipartFile#1010
vytas7 Jul 19, 2023
62275cc
refactor: restore some stuff from master, temp remove 1 file for now
vytas7 Jul 19, 2023
5872d3d
Merge branch 'master' into SimulateMultipartFile#1010
vytas7 Dec 27, 2023
6845e5a
feat(testing): add a new parameter to simulate form
vytas7 Dec 29, 2023
93fad43
fix(testing): fix a regression wrt passing json to simulate_request
vytas7 Dec 29, 2023
3af2fcc
Merge branch 'master' into SimulateMultipartFile#1010
vytas7 Mar 3, 2024
6ae4d06
Merge branch 'master' into SimulateMultipartFile#1010
vytas7 May 7, 2024
6ede18d
Merge branch 'master' into SimulateMultipartFile#1010
vytas7 Aug 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 81 additions & 4 deletions falcon/testing/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,18 @@

import asyncio
import datetime as dt
import hashlib
import inspect
import json as json_module
import time
from urllib.parse import urlencode
from typing import Dict, Optional, Sequence, Union
import warnings
import wsgiref.validate

from falcon.asgi_spec import ScopeType
from falcon.constants import COMBINED_METHODS
from falcon.constants import MEDIA_JSON
from falcon.constants import MEDIA_JSON, MEDIA_MULTIPART, MEDIA_URLENCODED
from falcon.errors import CompatibilityError
from falcon.testing import helpers
from falcon.testing.srmock import StartResponseMock
Expand Down Expand Up @@ -90,7 +92,7 @@
or ``None`` if not specified.
max_age (int): The lifetime of the cookie in seconds, or
``None`` if not specified.
secure (bool): Whether or not the cookie may only only be
secure (bool): Whether or not the cookie may only be
transmitted from the client via HTTPS.
http_only (bool): Whether or not the cookie may only be
included in unscripted requests from the client.
Expand Down Expand Up @@ -442,6 +444,7 @@
content_type=None,
body=None,
json=None,
form=None,
file_wrapper=None,
wsgierrors=None,
params=None,
Expand Down Expand Up @@ -579,6 +582,7 @@
content_type=content_type,
body=body,
json=json,
form=form,
params=params,
params_csv=params_csv,
protocol=protocol,
Expand All @@ -602,6 +606,7 @@
headers,
body,
json,
form,
extras,
)

Expand All @@ -626,7 +631,7 @@
# NOTE(vytas): Even given the duct tape nature of overriding
# arbitrary environ variables, changing the method can potentially
# be very confusing, particularly when using specialized
# simulate_get/post/patch etc methods.
# simulate_get/post/patch etc. methods.
raise ValueError(
'WSGI environ extras may not override the request method. '
'Please use the method parameter.'
Expand Down Expand Up @@ -655,6 +660,7 @@
content_type=None,
body=None,
json=None,
form=None,
params=None,
params_csv=True,
protocol='http',
Expand Down Expand Up @@ -739,6 +745,9 @@
overrides `body` and sets the Content-Type header to
``'application/json'``, overriding any value specified by either
the `content_type` or `headers` arguments.
form (dict): A form to submit as the request's body
(default: ``None``). If present, overrides `body`, and sets the
Content-Type header.
host(str): A string to use for the hostname part of the fully
qualified request URL (default: 'falconframework.org')
remote_addr (str): A string to use as the remote IP address for the
Expand Down Expand Up @@ -777,6 +786,7 @@
headers,
body,
json,
form,
extras,
)

Expand Down Expand Up @@ -2143,8 +2153,70 @@
await self._task_req


def _encode_form(form: dict) -> tuple:
"""Build the body for a URL-encoded or multipart form.

This utility method accepts two types of forms: a simple dict mapping
string keys to values will get URL-encoded, whereas if any value is a list
of two or three items, these will be treated as (filename, content) or
(filename, content, content_type), and encoded as a multipart form.

Returns: (encoded body bytes, Content-Type header)
"""
form_items = form.items() if isinstance(form, dict) else form

Check warning on line 2166 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2166

Added line #L2166 was not covered by tests

if not any(isinstance(value, (list, tuple)) for _, value in form_items):
# URL-encoded form
return urlencode(form, doseq=True).encode(), MEDIA_URLENCODED

Check warning on line 2170 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2170

Added line #L2170 was not covered by tests

# Encode multipart form
body = [b'']

Check warning on line 2173 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2173

Added line #L2173 was not covered by tests

for name, value in form_items:
data = value
filename = None
content_type = 'text/plain'

Check warning on line 2178 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2176-L2178

Added lines #L2176 - L2178 were not covered by tests

if isinstance(value, (list, tuple)):
try:
filename, data = value
content_type = 'application/octet-stream'
except ValueError:
filename, data, content_type = value

Check warning on line 2185 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2181-L2185

Added lines #L2181 - L2185 were not covered by tests
if isinstance(data, str):
data = data.encode()

Check warning on line 2187 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2187

Added line #L2187 was not covered by tests
elif not isinstance(data, bytes):
# Assume a file-like object
data = data.read()

Check warning on line 2190 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2190

Added line #L2190 was not covered by tests

headers = f'Content-Disposition: form-data; name="{name}"'

Check warning on line 2192 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2192

Added line #L2192 was not covered by tests
if filename:
headers += f'; filename="{filename}"'
headers += f'\r\nContent-Type: {content_type}\r\n\r\n'

Check warning on line 2195 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2194-L2195

Added lines #L2194 - L2195 were not covered by tests

body.append(headers.encode() + data + b'\r\n')

Check warning on line 2197 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2197

Added line #L2197 was not covered by tests

checksum = hashlib.sha256()

Check warning on line 2199 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2199

Added line #L2199 was not covered by tests
CaselIT marked this conversation as resolved.
Show resolved Hide resolved
for chunk in body:
checksum.update(chunk)
boundary = checksum.hexdigest()

Check warning on line 2202 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2201-L2202

Added lines #L2201 - L2202 were not covered by tests

encoded = f'--{boundary}\r\n'.encode().join(body)
encoded += f'--{boundary}--\r\n'.encode()
return encoded, f'{MEDIA_MULTIPART}; boundary={boundary}'

Check warning on line 2206 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2204-L2206

Added lines #L2204 - L2206 were not covered by tests


def _prepare_sim_args(
path, query_string, params, params_csv, content_type, headers, body, json, extras
path,
query_string,
params,
params_csv,
content_type,
headers,
body,
json,
form,
extras,
):
if not path.startswith('/'):
raise ValueError("path must start with '/'")
Expand Down Expand Up @@ -2178,6 +2250,11 @@
headers = headers or {}
headers['Content-Type'] = MEDIA_JSON

elif form is not None:
body, content_type = _encode_form(form)
headers = headers or {}
headers['Content-Type'] = content_type

Check warning on line 2256 in falcon/testing/client.py

View check run for this annotation

Codecov / codecov/patch

falcon/testing/client.py#L2254-L2256

Added lines #L2254 - L2256 were not covered by tests

return path, query_string, headers, body, extras


Expand Down
39 changes: 39 additions & 0 deletions tests/test_media_multipart.py
Original file line number Diff line number Diff line change
Expand Up @@ -845,3 +845,42 @@ async def deserialize_async(self, stream, content_type, content_length):

assert resp.status_code == 200
assert resp.json == ['', '0x48']


def test_simulate_form(client):
resp = client.simulate_post(
'/submit',
form={
'checked': 'true',
'file': ('test.txt', b'Hello, World!\n', 'text/plain'),
'another': ('test.dat', io.BytesIO(b'1\n2\n3\n')),
},
)

assert resp.status_code == 200
assert resp.json == [
{
'content_type': 'text/plain',
'data': 'true',
'filename': None,
'name': 'checked',
'secure_filename': None,
'text': 'true',
},
{
'content_type': 'text/plain',
'data': 'Hello, World!\n',
'filename': 'test.txt',
'name': 'file',
'secure_filename': 'test.txt',
'text': 'Hello, World!\n',
},
{
'content_type': 'application/octet-stream',
'data': '1\n2\n3\n',
'filename': 'test.dat',
'name': 'another',
'secure_filename': 'test.dat',
'text': None,
},
]
9 changes: 9 additions & 0 deletions tests/test_media_urlencoded.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,12 @@ def test_urlencoded_form(client, body, expected):
headers={'Content-Type': 'application/x-www-form-urlencoded'},
)
assert resp.json == expected


@pytest.mark.parametrize(
'form', [{}, {'a': '1', 'b': '2'}, (('a', '1'), ('b', '2'), ('c', '3'))]
)
def test_simulate_form(client, form):
resp = client.simulate_post('/media', form=form)
assert resp.status_code == 200
assert resp.json == dict(form)
Loading