Features, performances, security

This is an important upgrade that brings performance improvements, bugfixes and security fixes. Upgrade is recommanded.

Massive performance improvements

• Improve indexation speed, compute checksums in parallel, batch events in many services: indexation of 10k files now takes a couple of minutes.
• Fix DB prepared statements count going too high
• Use new index format for Bleve-based services (needs manual reindexation, see Upgrade instructions)
• Fix idle CPU usage: can be as low as 2% for main process, below 0.5% for sub-processes

Datasources

• Allow creating a local FS pointing to a Symbolic Link
• Fix Streaming on Encrypted Datasources
• Show specific status when starting datasources
• Prevent enabling Encryption with no Master Key
• Fix inter-datasource copy/move operations (especially between encrypted/clear datasources)

UX / APIs

• Easy Drag'n'drop between Workspaces and/or Cells. Dropping to a Cell shows a dialog to either move the file/folder to the Cell or add it as a new shared root.
• Copy/move task reports progress in UX
• Handle numeric sorting of Cells and Files. Add a quick filter on Cells.
• Diaporama supports left/right arrow keys for navigation, Esc for closing, fix sorting
• Add Markdown Preview in code editor
• Fix issue with thumbnails size on switching workspace
• Update VideoJS version
• Publish Share as Activity
• Additional parameters for upload requests timeout, for super-slow connections
• Add generic validation mechanism on Protos and apply automatically when reading pb entity in Rest API. Applied to DSName as an example.
• Fake synchronous API for WebDAV
• Reload Jobs automatically when a task is stuck in Pending Mode
• Allow unicode string usage in user attribute values (e.g. display name of users can use emoji).

Scheduler

• New Trigger Event on User modification
• Send groups deletion as background task, and pass a channel for handling objects as they are deleted
• Optimize events received by scheduler: add a NodesSelector at the Jobs level to avoid creating unnecessary tasks
• Scheduler actions: use TasksLogger instead of Logger

Cells Enterprise

• Implement simple per-group tenancy by adjusting users policies.
• Fix corner cases on LDAP migration
• BasePluginsBox was registered two times.
• Change heuristic for computing license stat

Other

• Vulnerabilities reported by 'Loginsoft Research' (CVEs to be submitted)
• Mailer: Sender control feature, Default sender display name, PR #58
• Better JSON serialization for exporting to ELK stack
• Add various subscribers for cleaning resources on user deletion. By default, personal folder is now renamed to a folder name with uuid.

Change log

You can find a summary of the change log here.

Massive performance improvements

This release provides a huge performance gain by fixing a piece of code that was basically hogging the CPU...
Upgrade quickly, your hardware will thank you!

Performances

• Fixed CPU consumption in Idle mode
• Fixed high number of SQL Prepared Statements, it should have an impact on the DB footprint
• Lowered BigCache initial memory as it is not necessary, instrument cache for publishing metrics: overall RAM footprint is much lower now

Configuration

• Removed some annoying limitations with networking at install:
  • Do not prevent start when "no private IP is found" (but show a warning instead)
  • Support Hostname in datasources to avoid having to restart them manually if IP is changing (typically on a container restart)
• Improve flexibility in defining internal/external URLs for reverse-proxy cases
• Updated go-sql plugin for MySQL 8 support
• Added new command-line to manage databases definitions, to avoid manual edition of pydio.json

Issues

• Fixed policies when using cells-ctl tool to create users
• New UX was breaking on Firefox 52. You should not be using that version, but some organizations do not have the choice
• Refactor common/utils package into specific sub-packages (to follow #golang best practices)
• Removed support for Consul for the moment, as it was half-baked
• [ED] XLSX/CSV export was broken in Firefox

Change log

You can find a summary of the change log here.

UX revamp, new features and performances

This important release greatly improves performances and provides new powerful features and many bugfixes.

UX Revamp

In our constant quest for clarity and simplicity, we have reworked the end-users interface to gain space and flatness.
The new home page focuses on the search engine, and otherwise displays recently accessed items, user bookmarks and eventually user Cells.

When inside a workspace, the top toolbar takes less space and is less present. Users can still revert to the previous style by changing the theme in their profile.

Performances

With Cells being rolled out more and more in production environment, we started to identify possible bottlenecks leading to scalability issues.

• We reworked the scheduler persistence layer (jobs vs. tasks vs tasks events), as it could lead to very long load time of the scheduler board, as well as huge memory consumption due to the underlying BoltDB storage
• Fixed People listing filtering (could lead to wrong pagination when filtering e.g. on shared users)
• Fixed an implicit time limit leading to incomplete archives when downloading a folder as Zip for more than 10 minutes
• Added a generic OperationUuid to the logs to list events for a particular operation
• [ED] New tooling for monitoring processes (see Enterprise Distribution section)

Using Cells behind a Proxy

Common configuration issues were reported when exposing Cells behind a reverse-proxy, linked most of the time to a misunderstanding of our "bind" vs. "external" URLS.
We reworked the install form to make it more explicit, and added some tricks in our own internal proxy to avoid "Signature Mismatch" errors on some S3 queries.
Installing behind a proxy should now be easier!

Enterprise Distribution New Features

Shares Audit Reports

Pydio Cells Enterprise now provides a dedicated tool to gather all information about shared data inside the application.
This report allows admins to browse all Cells, Workspaces and Public Links on all datasources, with an ability to filter
these resources by owner, filetypes, visibility, etc.

Admins can also list the users recent activity extracted from auditable logs directly inside the user editor in the admin Console.

DevOps Monitoring via Prometheus

We added support for exposing internal metrics to a Prometheus server, and we will publish it ready to be used on Grafana dashboard and imported.

This is really cool to monitor your installation (memory, goroutines, scheduler tasks running, etc), we will add more metrics as time goes by.

Migration Tool from Pydio 8 (Beta)

This new import tools allows you to create a brand new Cells installation from an existing Pydio 8 servers, by importing Users/Groups/Roles, mapping old workspaces to Cells datasources and workspaces, files metadata, shares, etc.

Given its number of features, every Pydio 8 install is unique, and we will need to crash-test this against real life usecases.
This tool is thus currently in BETA, and our paying customers are welcome to contact the support for guidance on the migration process (the import tool does not modify anything in the existing P8 installation).

Other fixes

• Prevent admin user from deleting herself or her own group
• Improved various search form issues: Filesize (add unit picker), Extension (needs a trailing dot), Date (cannot empty field), etc.
• Added missing GetUser() method declared in OpenAPI spec but not implemented
• Fixed UX permission issue when sharing a single file with "Edit" permission (editor was still opened as readonly)
• Fixed race condition in UX when loading Actions (empty 'New ...' submenu)
• Prevent multiple clicks when creating Cells on slow network
• Prevent ugly login dialog appearing on public links after a while
• Prevent re-opening of editors in a strange state after a logout
• Roles "Always Override" incorrectly wired (added a new column in DB)
• [ED] Fix various issues on Cells Console dashboard graphs
• [ED] Naming conflict breaking colors branding
• [ED] Add ability to edit Scheduler jobs frequency

Change log

You can find a summary of the change log here.

Bugfixes and Improvements

This release fixes a couple of annoying bugs in Pydio Cells:

• Fix an issue with encrypted datasource that could prevent accessing a file after updating it,
• Fix an issue in admin dashboard that could lead to overwriting an existing user,
• Fix a crash when an encrypted datasource references a non-existing master key,
• Prevent the ability to rename a file/folder with a "slash" in it,
• Fix a CSS resizing issue on Firefox,
• Prevent multiple read event when streaming a file,
• Added an option on SMTP server configuration to skip the certificate verification (for self-signed certificates on SMTP server).

Cells Enterprise specific fixes:

• Fix a permission issue in OnlyOffice plugin,
• Prevent multiple Audit events when streaming a file.

Update is done in-app or by just replacing the binary. Do not forget to setcap the new binary at restart if required (when using ports 80 or 443).

Change log

You can find a summary of the change log here.

Bugfixes and Improvements

With Cells 1.2.4, we are polishing our code and getting rid of hard angles!

Main Changes

• DataSources: we've worked on how they are configured, started and refreshed in a distributed environment.
• Background Jobs: better error reporting and monitoring from scheduler (directories and files sync).
• Upgrade: perform binary upgrade in a background task, including a nice progress bar in UX!
• Very long uploads: a token expiration could lead to a restart of the upload... Now refreshed properly every now and then :-)
• Optimizations: memory footprint is getting better every day, and our Docker image size has just been divided by two.

Upgrade is done in-app, see full changelog below.

We are progressing on translation, thanks to Patrick Polsterer and Christian Eberle for the updated German version! Every contributor is welcome on https://crowdin.com/project/pydio-cells to help.

Change log

You can find a summary of the change log here.

Hotfix for 1.2.2

This version fixes a startup issue that could randomly lead to "502 Bad Gateway" errors.

Please read full release note of 1.2.2 here

Change log

You can find a summary of the change log here.

Bugfixes and New Features

Pydio Cells 1.2.2 brings many new features and improvements!

Main changes

• We fixed a lot of small glitches to ease the deployment of Cells on distributed architecture:
  Clustering of the discovery infrastructure allows to start individual services on different nodes and register them automagically to the central gateway. Data Sources can also be started on various nodes to achieve data aggregation from distributed storages. The associated Admin > Services panel has been improved for better visualisation and supports filtering services by nodes.

• Frontend uploader has been almost entirely rewritten to use Multipart Upload for big files (file chunking and concurrent uploads of parts), improving performances and progress visualisation, and allowing to Pause/Resume long-running uploads. When uploading existing files and folders, options allow to either merge folders contents (while renaming files) or rename folders directly.

Enterprise Features

Cells enterprise provides great integrations with new third-party software and storages:

• OnlyOffice editor for MS Office Documents online and collaborative edition.
• New objects storage are supported: Azure Blob Storage and Google Cloud Storage

Other fixes and UX improvements

• Automatically create folder on storage when creating a datasource
• Fix Download using cached URL for streaming resources (mp3/mp4)
• Keyring is not necessary to be available on OS anymore. It is used if it's available, but we use our own sealed vault if not.
• Fix "New Office Document" action by embedding proper empty templates in application
• Fix "Sort" action in List display mode (was randomling appearing)
• Cells quick creation: when creating a Cell from scratch, one can directly create it without going through the steps of adding users and data.
• People list in admin Console is now directly displaying the "locked" status of a user
• Build is running for both go1.10.5 and go1.11.2
• Docker image embeds Root CA to avoid errors when checking for updates
• Fix issue with multiple emails sent to multiple users
• Fixed mp3 player minization
• Many fixes on error handling and formatting
• Audit Logs (Enterprise) homegenization

You can see the detailed changelog below.

Change log

You can find a summary of the change log here.

Bugfixes Release

Cells 1.2.1 provides bug fixes and improvements.

Change log

You can find a summary of the change log here.

Cells finally removes PHP-dependency

Bye PHP !

Cells now reached maturity by finally removing PHP fully. The pydio/cells-front repository is now deprecated and frontend assets are served directly by a dedicated web micro-service.

This leads to the web interface feeling more responsive than ever. With an average X5 time gained on each request, the real-time update of data through websocket, and long-running tasks sent to background in the server scheduler, users can focus on their work and handle their files and documents as if they were locally available.

Removing the external dependency to a php daemon has also greatly simplified installation of Pydio Cells on any server. Download the binary, set up an SQL database and you are ready to go!

A beautiful user experience

On top of the massive performance gain, many screens of the interface were polished to provide the user with a gorgeous experience: noticeably, embedded viewers (image diaporama, PDF viewer, text editor, etc.) as well as “public links” layouts were entirely reworked to maximize screen space usage.

The Administration Console was also streamlined for better readability: hiding “advanced” parameters by default makes it easier for the first usage and many other UX improvements were applied (role editor, top bar, etc).

Internationalization crowd-sourced (i18n)

i18n libraries formats were harmonized between backend and frontend, and are now publicly available on CrowdIn for the community to easily contribute to translations. See https://crowdin.com/project/pydio-cells

Changelog

You can see full changelog here: v1.0.4...v1.2.0

Bug Fix Release

Fix a few bugs and enhance handling of a few errors.

See Full Backend Changelog

• #c2599ca: Update coreos/dex TP. Also closes #15
• #1aaadda: Fix typo in error checking
• #5529392: Revert default code version
• #c73a6d9: Add more log on error
• #6814dd7: Test new color for grpc service names
• #660d01e: Fix migration corner case for fresh installs
• #522db2a: Fix failing test
• #f9fd16c: Fix init issue on empty DB, makes linter happy
• #de1e705: Enhance comments and error handling, add tests.
• #e0b7754: Add comments, enhance error messages to ease error solving.
• #701b0f4: Make linter happy
• #413e66d: Merge frontend changelog

See Full Front Changelog

• #e2de68a: Fix aws-sdk-php to last working version