diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 000000000..4254797c5 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,14 @@ +## 0.4.0 - 2015-11-03 + +* Rewrite the read-only to prepare full sync +* Add unit tests +* GUI is currently broken + +## 0.3.0 - 2015-03-10 + +* Read-only version (not very reliable) +* GUI with Node-Webkit + +## Previous releases + +* Proof of concept diff --git a/README.md b/README.md index 36d2c4b33..ccdefa96c 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,11 @@ The Cozy desktop app allows to sync the files stored in your Cozy with your lapt and/or your desktop. It replicates your files on your hard drive and apply changes you made on them on other synced devices and on your online Cozy. +**Note**: the code is currently **alpha** quality and there is only a readonly +mode (files from the cozy are replicated to the local hard drive). But it's +moving fast and we plan to do a more stable release in the coming weeks. Stay +tuned! + ## CLI Install @@ -21,7 +26,7 @@ Then you can install cozy-desktop via NPM: Configure it with your remote Cozy - cozy-desktop add-remote-cozy http://url.of.my.cozy devicename /sync/directory + cozy-desktop add-remote-cozy https://url.of.my.cozy/ devicename /sync/directory Then start synchronization daemon: @@ -32,52 +37,6 @@ Other commands can be listed with cozy-desktop -h -## GUI running - -The Graphical User Interface requires Node Webkit to be launched. It should be -available in your dev dependencies. You can install it with the following -commmands: - - cd cozy-desktop - npm install - -Once done, you can launch nodewebkit in the current directory - - node_modules/.bin/nw - -**Note:** On Ubuntu 13.04+, Fedora 18+, ArchLinux and Gentoo, you will run -into a `libudev.so.0` issue. See -https://github.com/rogerwang/node-webkit/wiki/The-solution-of-lacking-libudev.so.0 -for more information. - -If you made a modification in the code and you want to recompile `.coffee` files, -run: - - node_modules/.bin/gulp scripts # Compiles backend files - cd client - npm install - npm install -g brunch - brunch build # Compiles client files - - -### GUI package building - -If you want to build the GUI package, you will need `rubygems` and the `fpm` -gem: - - sudo apt-get install ruby-dev build-essential # On Ubuntu/Debian - sudo gem install fpm - gulp build-gui-package - -To make a package for your platform, choose either: - - node_modules/.bin/gulp make-deb-32 - node_modules/.bin/gulp make-deb-64 - node_modules/.bin/gulp make-rpm-32 # require the 'npm' package installed - node_modules/.bin/gulp make-rpm-64 # require the 'npm' package installed - node_modules/.bin/gulp make-osx-app - - ## Hack To hack the synchronization backend, you can just edit the files under the @@ -86,8 +45,8 @@ To hack the synchronization backend, you can just edit the files under the ### Tests -[![Build -Status](https://travis-ci.org/cozy-labs/cozy-desktop.png?branch=master)](https://travis-ci.org/cozy-labs/cozy-desktop) +[![Build Status](https://travis-ci.org/cozy-labs/cozy-desktop.png?branch=master) +](https://travis-ci.org/cozy-labs/cozy-desktop) There are several levels of tests in cozy-desktop: @@ -124,6 +83,61 @@ DEFAULT_DIR=tmp mocha --compilers coffee:coffee-script/register tests/integratio ``` +## Limitations + +Cozy-desktop is designed to synchronize files and folders between a remote +cozy instance and a local hard drive, for a personal usage. We tried to make +it simple and easy. So, it has some limitations: + +- It's only a command-line interface and it works only on Linux for the + moment. We are working to improve this in the next weeks. + +- It's all or nothing for files and folders to synchronize, but we have on our + roadmap to add a mean to select which files and folders to synchronize. + +- Files and folders named like this are ignored: + - `.cozy-desktop` (they are used to keep internal state) + - `_design` (special meaning for pouchdb/couchdb) + +- It's not a particularly good idea to share code with cozy-desktop: + - `node_modules` can't be ignored for the moment and have tons of small files + - compiled code often has to be recompile to works on another environment + - git and other VCS are not meant to be share like this. You may lose your + work if you make changes on two laptops synchronized by cozy-desktop (it's + the same with + [dropbox](https://github.com/anishathalye/git-remote-dropbox#faq), + [google-drive](https://stackoverflow.com/questions/31984751/google-drive-can-corrupt-repositories-in-github-desktop), + [syncthing](https://forum.syncthing.net/t/is-putting-a-git-workspace-in-a-synced-folder-really-a-good-idea/1774), + etc.) + +- If the same file has been modified in parallel, cozy-desktop don't try to + merge the modifications. It will just rename of one the copies with a + `-conflict` suffix. It's the same for folders. + +- We expect a personal usage: + - a reasonable number of files and folders (< 1.000.000) + - a reasonable number of files per folder (< 10.000) + - a reasonable size for files (< 1 To) + - a reasonable size for files and folders path (< 1024 bytes) + - not too many changes + - etc. + +- The full sync directory must be on the same partition. + +- Large files must be uploaded/downloaded in one time (we are thinking about + making it possible to split a large file in several blocks for + download/upload). + +- Due to its nature, cozy-desktop needs resources: + - CPU, for checksums in particular + - RAM, to keep all the metadata in memory, and for nodejs libraries + - Disk, but the overhead from cozy-desktop is low + - Network bandwidth obviously + +- No advanced feature, like P2P replication between several cozy-desktop + instances. + + ## What is Cozy? ![Cozy Logo](https://raw.github.com/cozy/cozy-setup/gh-pages/assets/images/happycloud.png) diff --git a/bin/cli.coffee b/bin/cli.coffee index 4ac130e7a..897200a02 100755 --- a/bin/cli.coffee +++ b/bin/cli.coffee @@ -16,6 +16,7 @@ Please enter your password to register your device to 'your remote Cozy: callback err, password +# TODO make devicename optional, and use `hostname` for the default program .command 'add-remote-cozy ' .description 'Configure current device to sync with given cozy' diff --git a/doc/design.md b/doc/design.md new file mode 100644 index 000000000..90317a884 --- /dev/null +++ b/doc/design.md @@ -0,0 +1,45 @@ +Cozy-Desktop - Design +===================== + +The Cozy desktop app allows to sync the files stored in your Cozy with your laptop +and/or your desktop. It replicates your files on your hard drive and apply +changes you made on them on other synced devices and on your online Cozy. + + +Metadata workflow +----------------- + +Cozy-desktop is a nodejs app, written in coffeescript to be coherent with +other cozy codes. As its core, there is a pouchdb database used to keep +metadata about files and folders. + +On the remote cozy, the files and folders are saved in CouchDB. Cozy-desktop +listens to the changes feed of CouchDB to be notified of all the metadata +changes on the remote cozy. Those metadata are recopied in the local PouchDB. + +On the local filesystem, the synchronized folder is watched via chokidar. It's +a library that uses nodejs' watch powered by inotify/fsevents (with a fallback +on polling). With it, cozy-desktop is notified of all the changes on the file +system and can update the pouchdb with the new metadata. + +Cozy-desktop also uses a changes feed on the local PouchDB. It takes metadata +changes one by one, and apply them to the other side. For example, if the last +modification date of a folder has changed on the remote cozy, cozy-desktop +will update it on the local filesystem. + +If a new file is added, cozy-desktop will ask one side to provide a nodejs +readable stream, and the other side will pipe it to its destination: a file on +the local filesytem, or a binary document on the remote couchdb. + +**TODO:** + +- add a schema +- explain how we avoid loops of metadata updates +- explain when conflicts happen and how they are resolved +- explain the special trick for renames and moves + + +Documents schema +---------------- + +See [`backend/merge.coffee`](https://github.com/cozy-labs/cozy-desktop/blob/master/backend/merge.coffee#L15-L37) diff --git a/package.json b/package.json index 0a35eab3d..760147564 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "cozy-desktop", - "version": "0.3.0", + "version": "0.4.0", "description": "File Synchronization Client for Cozy Cloud", "main": "app.html", "window": {