Skip to content

Commit

Permalink
initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
muslashwhy committed Jan 22, 2023
0 parents commit dfea92e
Show file tree
Hide file tree
Showing 10 changed files with 1,318 additions and 0 deletions.
50 changes: 50 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# These are some examples of commonly ignored file patterns.
# You should customize this list as applicable to your project.
# Learn more about .gitignore:
# https://www.atlassian.com/git/tutorials/saving-changes/gitignore

# Node artifact files
node_modules/
dist/

# Compiled Java class files
*.class

# Compiled Python bytecode
*.py[cod]

# Log files
*.log

# Package files
*.jar

# Maven
target/
dist/

# JetBrains IDE
.idea/

# Unit test reports
TEST*.xml

# Generated by MacOS
.DS_Store

# Generated by Windows
Thumbs.db

# Applications
*.app
*.exe
*.war

# Large media files
*.mp4
*.tiff
*.avi
*.flv
*.mov
*.wmv

109 changes: 109 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Request Curl

User-friendly wrapper for pycurl

## Installation
Use the package manager
[pip](https://pip.pypa.io/en/stable/)
to install request_curl.

```
pip install request_curl
```

## HTTP2
HTTP2 is disabled by default.

```python
import request_curl
s = request_curl.Session(http2=True)
r = s.get("https://www.example.com")
```

## Proxy Support
Proxy has to be formatted as a string.

```python
import request_curl
s = request_curl.Session()
r = s.get("https://www.example.com", proxies="ip:port:user:password")
```

## Content Decoding
```python
import request_curl
s = request_curl.Session(accept_encoding="br, gzip, deflate")
r = s.get("https://www.example.com", debug=True)
```

## Response Object

The response object behaves
similar to the one of the requests library.

```python
import request_curl
s = request_curl.Session()
r = s.get("https://www.example.com")

print(r)
print(r.status_code)
print(r.content)
print(r.text)
print(r.json)
print(r.url)
print(r.history)
```

## Cipher Suites
You can specify custom cipher suites as an array.

```python
import request_curl

cipher_suite = [
"AES128-SHA256",
"AES256-SHA256",
"AES128-GCM-SHA256",
"AES256-GCM-SHA384"
]
s = request_curl.Session(cipher_suite=cipher_suite)
r = s.get("https://www.example.com")
```

## Debug Request
If debug is set to True the raw input
and output headers will bre printed out.

```python
import request_curl
s = request_curl.Session()
r = s.get("https://www.example.com", debug=True)
```

## Custom Header
You can specify custom a customer header
as a dictionary.

```python
import request_curl
s = request_curl.Session()
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
}
r = s.get("https://www.example.com", headers=headers)
```

## Install with Curl-Impersonate
- https://github.com/lwthiker/curl-impersonate/blob/main/INSTALL.md
- sudo apt install build-essential pkg-config cmake ninja-build curl autoconf automake libtool
- ``sudo apt install -y libbrotli-dev golang build-essential libnghttp2-dev cmake libunwind-dev libssl-dev git python3-dev``
- git clone https://github.com/pycurl/pycurl.git
- sudo python3 setup.py install --curl-config=/usr/local/bin/curl-impersonate-chrome-config

```python
import pycurl
pycurl.version_info()
# (9, '7.84.0', 480256, 'x86_64-pc-linux-gnu', 1370063517, 'BoringSSL', 0, '1.2.11', ('dict', 'file', 'ftp', 'ftps', 'gopher', 'gophers', 'http', 'https', 'imap', 'imaps', 'mqtt', 'pop3', 'pop3s', 'rtsp', 'smb', 'smbs', 'smtp', 'smtps', 'telnet', 'tftp'), None, 0, None)
quit()
```
392 changes: 392 additions & 0 deletions poetry.lock

Large diffs are not rendered by default.

18 changes: 18 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
[tool.poetry]
name = "request_curl"
version = "0.0.1"
description = ""
authors = ["Notifysolutions <Ennis Blank, Mauritz Uphoff>"]

[tool.poetry.dependencies]
python = "^3.10"
requests = "^2.28.1"
Brotli = "^1.0.9"
pycurl = "^7.45.2"

[tool.poetry.dev-dependencies]
pytest = "^7.1.3"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
5 changes: 5 additions & 0 deletions request_curl/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD
from .api import get, options, post, put, patch, delete, head, request
from .sessions import Session

version = "0.0.1"
152 changes: 152 additions & 0 deletions request_curl/api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
"""
:copyright: (c) 2022 by Mauritz Uphoff.
"""
from .sessions import Session


def request(method, url, **kwargs):
"""Constructs and sends a :class:`Request <Request>`.
:param method: method for the new :class:`Request` object:
``GET``, ``OPTIONS``, ``HEAD``, ``POST``, ``PUT``, ``PATCH``, or ``DELETE``.
:param url: URL for the new :class:`Request` object.
:param params: (optional) Dictionary, list of tuples or bytes to send
in the query string for the :class:`Request`.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
:param json: (optional) A JSON serializable Python object
to send in the body of the :class:`Request`.
:param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`.
:param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`.
:param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload.
``file-tuple`` can be a 2-tuple ``('filename', fileobj)``, 3-tuple ``('filename', fileobj, 'content_type')``
or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content-type'`` is a string
defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers
to add for the file.
:param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth.
:param timeout: (optional) How many seconds to wait for the server to send data
before giving up, as a float, or a :ref:`(connect timeout, read
timeout) <timeouts>` tuple.
:type timeout: float or tuple
:param allow_redirects: (optional) Boolean. Enable/disable
GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD redirection. Defaults to ``True``.
:type allow_redirects: bool
:param proxies: (optional) String to the URL of the proxy.
:param verify: (optional) Either a boolean, in which case it controls whether we verify
the server's TLS certificate, or a string, in which case it must be a path
to a CA bundle to use. Defaults to ``True``.
:return: :class:`Response <Response>` object
:rtype: requests.Response
Usage::
>>> import request_curl
>>> req = request_curl.request('GET', 'https://httpbin.org/get')
>>> req
<Response [200]>
"""

# By using the 'with' statement we are sure the session is closed, thus we
# avoid leaving sockets open which can trigger a ResourceWarning in some
# cases, and look like a memory leak in others.
with Session() as session:
return session.request(method, url, **kwargs)


def get(url, params=None, **kwargs):
r"""Sends a GET request.
:param url: URL for the new :class:`Request` object.
:param params: (optional) Dictionary, list of tuples or bytes to send
in the query string for the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""

return request('get', url, params=params, **kwargs)


def options(url, **kwargs):
r"""Sends an OPTIONS request.
:param url: URL for the new :class:`Request` object.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""

return request('options', url, **kwargs)


def head(url, **kwargs):
r"""Sends a HEAD request.
:param url: URL for the new :class:`Request` object.
:param \*\*kwargs: Optional arguments that ``request`` takes. If
`allow_redirects` is not provided, it will be set to `False` (as
opposed to the default :meth:`request` behavior).
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""

kwargs.setdefault('allow_redirects', False)
return request('head', url, **kwargs)


def post(url, data=None, json=None, **kwargs):
r"""Sends a POST request.
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
:param json: (optional) json data to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""

return request('post', url, data=data, json=json, **kwargs)


def put(url, data=None, **kwargs):
r"""Sends a PUT request.
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
:param json: (optional) json data to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""

return request('put', url, data=data, **kwargs)


def patch(url, data=None, **kwargs):
r"""Sends a PATCH request.
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
:param json: (optional) json data to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""

return request('patch', url, data=data, **kwargs)


def delete(url, **kwargs):
r"""Sends a DELETE request.
:param url: URL for the new :class:`Request` object.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
"""

return request('delete', url, **kwargs)


Loading

0 comments on commit dfea92e

Please sign in to comment.