Merge pull request #199 from Preetwinder/PY3-documentation

PY3 Documentation changes

Author: sibiryakov

AUTHORS

@@ -2,4 +2,5 @@ Here is the list of the primary authors & contributors:
 
 * Javier Casas
 * Alexander Sibiryakov
-* Joris Rau
+* Joris Rau
+* Preetwinder Singh

README.md

@@ -19,6 +19,7 @@ the crawler to decide which pages to visit next, and capable of doing it in dist
 - Revisiting logic with RDBMS.
 - Optional use of Scrapy for fetching and parsing.
 - 3-clause BSD license, allowing to use in any commercial product.
+- Python 3 support.
 
 ## Installation
 

docs/source/topics/installation.rst

@@ -4,7 +4,7 @@ Installation Guide
 
 The installation steps assume that you have the following requirements installed:
 
-* `Python`_ 2.7
+* `Python`_ 2.7+ or 3.4+
 
 * `pip`_ and `setuptools`_ Python packages. Nowadays `pip`_ requires and
   installs `setuptools`_ if not installed.

docs/source/topics/overview.rst

@@ -16,6 +16,7 @@ downloading them from the web. Main features of Frontera are:
   :doc:`Graph Manager <graph-manager>`.
 * transparent transport layer concept (:term:`message bus`) and communication protocol,
 * pure Python implementation.
+* Python 3 support.
 
 
 .. _use-cases:

docs/source/topics/quick-start-distributed.rst

@@ -13,7 +13,7 @@ Prerequisites
 
 Here is what services needs to be installed and configured before running Frontera:
 
-- Python 2.7+
+- Python 2.7+ or 3.4+
 - Scrapy
 
 Frontera installation

frontera/contrib/middlewares/domain.py

@@ -15,7 +15,7 @@ class DomainMiddleware(Middleware):
     :attr:`Response.meta <frontera.core.models.Response.meta>` if is activated.
 
 
-    ``domain`` object will contains the following fields:
+    ``domain`` object will contain the following fields, with both keys and values as bytes:
 
     - **netloc**: URL netloc according to `RFC 1808`_ syntax specifications
     - **name**: Domain name

frontera/contrib/middlewares/fingerprint.py

@@ -54,6 +54,7 @@ class UrlFingerprintMiddleware(BaseFingerprintMiddleware):
     Fingerprint will be calculated from object ``URL``, using the function defined in
     :setting:`URL_FINGERPRINT_FUNCTION` setting.
     You can write your own fingerprint calculation function and use by changing this setting.
+    The fingerprint must be bytes.
 
     An example for a :class:`Request <frontera.core.models.Request>` object::
 
@@ -87,6 +88,7 @@ class DomainFingerprintMiddleware(BaseFingerprintMiddleware):
     Fingerprint will be calculated from object ``URL``, using the function defined in
     :setting:`DOMAIN_FINGERPRINT_FUNCTION` setting.
     You can write your own fingerprint calculation function and use by changing this setting.
+    The fingerprint must be bytes
 
     An example for a :class:`Request <frontera.core.models.Request>` object::
 

frontera/core/models.py

@@ -22,7 +22,8 @@ def __init__(self, url, method=b'GET', headers=None, cookies=None, meta=None, bo
         :param string method: HTTP method to use.
         :param dict headers: dictionary of headers to send.
         :param dict cookies: dictionary of cookies to attach to this request.
-        :param dict meta: dictionary that contains arbitrary metadata for this request.
+        :param dict meta: dictionary that contains arbitrary metadata for this request, the keys must be bytes and \
+        the values must be either bytes or serializable objects such as lists, tuples, dictionaries with byte type items.
         """
         self._url = to_native_str(url)
         self._method = to_bytes((method or b'GET').upper())
@@ -65,7 +66,8 @@ def meta(self):
         """
         A dict that contains arbitrary metadata for this request. This dict is empty for new Requests, and is usually
         populated by different Frontera components (middlewares, etc). So the data contained in this dict depends
-        on the components you have enabled.
+        on the components you have enabled. The keys are bytes and the values are either bytes or serializable objects \
+        such as lists, tuples, dictionaries with byte type items.
         """
         return self._meta
 

setup.py

@@ -31,6 +31,9 @@
         'Programming Language :: Python',
         'Programming Language :: Python :: 2',
         'Programming Language :: Python :: 2.7',
+        'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: 3.4',
+        'Programming Language :: Python :: 3.5',
         'Topic :: Internet :: WWW/HTTP',
         'Topic :: Software Development :: Libraries :: Application Frameworks',
         'Topic :: Software Development :: Libraries :: Python Modules',