Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
229 changes: 102 additions & 127 deletions README
Original file line number Diff line number Diff line change
@@ -1,175 +1,150 @@
-----
About
-----
# Django Picklefield

**django-picklefield** provides an implementation of a pickled object field.
Such fields can contain any picklable objects.
**Django Picklefield** provides an implementation of a pickled object field that can store any picklable Python object in a database field.

The implementation is taken and adopted from Django snippet #1694
<http://www.djangosnippets.org/snippets/1694/> by Taavi Taijala, which is in
turn based on Django snippet #513 <http://www.djangosnippets.org/snippets/513/>
by Oliver Beattie.
This package is based on [Django snippet #1694](http://www.djangosnippets.org/snippets/1694/) by Taavi Taijala, itself inspired by [Django snippet #513](http://www.djangosnippets.org/snippets/513/) by Oliver Beattie.

django-picklefield is available under the MIT license.
Django Picklefield is available under the MIT license.

## Table of Contents
- [Installation](#installation)
- [Usage](#usage)
- [Notes](#notes)
- [Original Author's Notes](#original-authors-notes)
- [Changelog](#changelog)
- [Feedback](#feedback)

-----
Usage
-----
## Installation

First of all, you need to have **django-picklefield** installed; for your
convenience, recent versions should be available from PyPI.
### 1. Install using pip

To use, just define a field in your model::
First, install the latest version of **django-picklefield** from PyPI using pip:

>>> from picklefield.fields import PickledObjectField
... class SomeObject(models.Model):
... args = PickledObjectField()
```bash
pip install django-picklefield
```

and assign whatever you like (as long as it's picklable) to the field::
### 2. Add to `INSTALLED_APPS`

>>> obj = SomeObject()
>>> obj.args = ['fancy', {'objects': 'inside'}]
>>> obj.save()
After installing, add `picklefield` to your `INSTALLED_APPS` in your Django settings:

```python
# settings.py

-----
Notes
-----
INSTALLED_APPS = [
# Other apps...
'picklefield',
]
```

If you need to serialize an object with a PickledObjectField for transmission
to the browser, you may need to subclass the field and override the
``value_to_string()`` method. Currently pickle fields are serialized as
base64-encoded pickles, which allows reliable deserialization, but such a
format is not convenient for parsing in the browser. By overriding
``value_to_string()`` you can choose a more convenient serialization format.
### 3. Setup Virtual Environment (Optional but Recommended)

--------------
Original notes
--------------
To avoid dependency conflicts and isolate the project, it's recommended to create a virtual environment before installing the package:

Here are the notes by taavi223, the original author:
```bash
# Create a virtual environment
python -m venv env

Incredibly useful for storing just about anything in the database (provided it
is Pickle-able, of course) when there isn't a 'proper' field for the job.
# Activate the virtual environment (Windows)
.\env\Scripts\activate

PickledObjectField is database-agnostic, and should work with any database
backend you can throw at it. You can pass in any Python object and it will
automagically be converted behind the scenes. You never have to manually pickle
or unpickle anything. Also works fine when querying; supports exact, in, and
isnull lookups. It should be noted, however, that calling QuerySet.values()
will only return the encoded data, not the original Python object.
# Activate the virtual environment (macOS/Linux)
source env/bin/activate

This PickledObjectField has a few improvements over the one in snippet #513.
# Install django-picklefield inside the virtual environment
pip install django-picklefield
```

This one solves the DjangoUnicodeDecodeError problem when saving an object
containing non-ASCII data by base64 encoding the pickled output stream. This
ensures that all stored data is ASCII, eliminating the problem.
### 4. Database Migrations

PickledObjectField will now optionally use zlib to compress (and uncompress)
pickled objects on the fly. This can be set per-field using the keyword
argument "compress=True". For most items this is probably not worth the small
performance penalty, but for Models with larger objects, it can be a real space
saver.
If you are adding the `PickledObjectField` to a model in an existing project, don’t forget to create and apply migrations:

You can also now specify the pickle protocol per-field, using the protocol
keyword argument. The default of 2 should always work, unless you are trying to
access the data from outside of the Django ORM.
```bash
# Create migrations
python manage.py makemigrations

Worked around a rare issue when using the cPickle and performing lookups of
complex data types. In short, cPickle would sometimes output different streams
for the same object depending on how it was referenced. This of course could
cause lookups for complex objects to fail, even when a matching object exists.
See the docstrings and tests for more information.
# Apply migrations
python manage.py migrate
```

You can now use the isnull lookup and have it function as expected. A
consequence of this is that by default, PickledObjectField has null=True set
(you can of course pass null=False if you want to change that). If null=False
is set (the default for fields), then you wouldn't be able to store a Python
None value, since None values aren't pickled or encoded (this in turn is what
makes the isnull lookup possible).
## Usage

You can now pass in an object as the default argument for the field without it
being converted to a unicode string first. If you pass in a callable though,
the field will still call it. It will not try to pickle and encode it.
To use the pickled object field, define it in your Django model:

You can manually import dbsafe_encode and dbsafe_decode from fields.py if you
want to encode and decode objects yourself. This is mostly useful for decoding
values returned from calling QuerySet.values(), which are still encoded
strings.
```python
from picklefield.fields import PickledObjectField
from django.db import models

Note: If you are trying to store other django models in the PickledObjectField,
please see the comments for a discussion on the problems associated with doing
that. The easy solution is to put django models into a list or tuple before
assigning them to the PickledObjectField.
class SomeObject(models.Model):
args = PickledObjectField()
```

Update 9/2/09: Fixed the value_to_string method so that serialization should
now work as expected. Also added deepcopy back into dbsafe_encode, fixing #4
above, since deepcopy had somehow managed to remove itself. This means that
lookups should once again work as expected in all situations. Also made the
field editable=False by default (which I swear I already did once before!)
since it is never a good idea to have a PickledObjectField be user editable.
You can assign any picklable object to the field:

-------
Changes
-------
```python
obj = SomeObject()
obj.args = ['fancy', {'objects': 'inside'}]
obj.save()
```

Changes in version 0.1.8
========================
## Notes

* Updated link to code repository.
If you need to serialize a `PickledObjectField` for transmission to the browser, you may need to subclass the field and override the `value_to_string()` method. By default, the field serializes as a base64-encoded pickle, which can be difficult to parse in a browser.

Changes in version 0.1.7
========================
For more convenient serialization, you can customize the `value_to_string()` method to output in a different format.

* Added `connection` and `prepared` parameters to `get_db_prep_lookup()` to
get rid of deprecation warnings in Django 1.2.
## Original Author's Notes

Changes in version 0.1.6
========================
Here are the original author's (Taavi Taijala) notes:

* Fixed South support (thanks aehlke@github).
- Useful for storing any picklable object in the database when there isn’t a specific field for it.
- **PickledObjectField** is database-agnostic, working across all backends.
- Automatically converts objects without manual pickling.
- Supports `exact`, `in`, and `isnull` lookups when querying.
- Values returned from `QuerySet.values()` will only contain encoded data, not the original Python object.

Changes in version 0.1.5
========================
### Improvements
This field includes several improvements over snippet #513:
- Solves the `DjangoUnicodeDecodeError` issue by base64 encoding the pickled output, ensuring all stored data is ASCII.
- Optionally compresses pickled objects using zlib (set `compress=True`).
- Supports custom pickle protocols using the `protocol` keyword argument.
- Addresses issues when using `cPickle` with complex data types.
- Supports `isnull` lookups properly with the default `null=True` for the field.

* Added support for South.
* Changed default to null=False, as is common throughout Django.
## Changelog

Changes in version 0.1.4
========================
### Version 0.1.8
- Updated link to code repository.

* Updated copyright statements.
### Version 0.1.7
- Added `connection` and `prepared` parameters to `get_db_prep_lookup()` to remove deprecation warnings in Django 1.2.

Changes in version 0.1.3
========================
### Version 0.1.6
- Fixed South support (thanks aehlke@github).

* Updated serialization tests (thanks to Michael Fladischer).
### Version 0.1.5
- Added support for South.
- Changed default to `null=False` to align with Django conventions.

Changes in version 0.1.2
========================
### Version 0.1.4
- Updated copyright statements.

* Added Simplified BSD licence.
### Version 0.1.3
- Updated serialization tests (thanks to Michael Fladischer).

Changes in version 0.1.1
========================
### Version 0.1.2
- Added Simplified BSD license.

* Added test for serialization.
* Added note about JSON serialization for browser.
* Added support for different pickle protocol versions (thanks to Michael
Fladischer).
### Version 0.1.1
- Added test for serialization.
- Added note about JSON serialization for the browser.
- Added support for different pickle protocol versions (thanks to Michael Fladischer).

Changes in version 0.1
======================
### Version 0.1
- First public release.

* First public release.
## Feedback

For more information and access to the code repository, visit the project’s [GitHub page](http://github.com/gintas/django-picklefield).

--------
Feedback
--------

There is a home page <http://github.com/gintas/django-picklefield>
with instructions on how to access the code repository.

Send feedback and suggestions to gintautas@miliauskas.lt .
If you have feedback or suggestions, feel free to send them to [gintautas@miliauskas.lt](mailto:gintautas@miliauskas.lt).