From b8d7ed4c3d0ffeeaaa0f3a0561335626b8d29fff Mon Sep 17 00:00:00 2001 From: John Rouillard Date: Sun, 12 May 2024 18:19:09 -0400 Subject: [PATCH] doc: prep announcement.txt for 2.4.0beta1 Still have to add the classhelper web-component, but get the rest of the beta announcement ready. [skip travis] --- doc/announcement.txt | 322 +++++++++++++------------------------------ 1 file changed, 96 insertions(+), 226 deletions(-) diff --git a/doc/announcement.txt b/doc/announcement.txt index 21ffcc94..b9cd8d40 100644 --- a/doc/announcement.txt +++ b/doc/announcement.txt @@ -1,4 +1,4 @@ -I'm proud to release version 2.3.0 of the Roundup issue +I'm proud to release version 2.4.0 beta of the Roundup issue tracker. This release is a bugfix and feature release, so make sure to read `docs/upgrading.txt `_ to @@ -12,131 +12,150 @@ update the database schema version. Do this before you use the web, command-line or mail interface and before any users access the tracker. -You can download it with:: - - pip download roundup - -then unpack and test/install the tarball. Also:: +You can install it with:: pip install roundup -(preferably in a virtual environment) can be used. +(preferably in a virtual environment). To download it, use:: -Among the notable improvements from the 2.2.0 release are: + pip download roundup + +then unpack and test/install from the tarball. -* Dockerfile demo mode implemented. This allows quick evaluation as - well as the ability to spin up a configured tracker to customise. +Among the notable improvements from the 2.3.0 release are: -* SQLite backends can use WAL mode to reduce blocking between readers - and writers improving concurrent use. +* new classhelper component thanks to a team of students + from CS682 at U-Mass Boston. This fixes many issues with + the old classhelper. It is implemented as a web-component + and needs REST interface access. It will fall back to the + classic classhelper if REST is not available or if the + browser does not support web-components. -* Redis can be used for session database with SQLite and dbm - backends. Provides a major performance improvement. +* fix Windows Python installation using pip. It used to go + into an infinite loop during install or download. Also fix + installation of shared files (templates) so roundup-admin + can find them. -* roundup-mailgw can use OAUTH authentication to SMTP - server. (roundup-mailgw command line options changed as a result.) +* using ``@current_user`` as a value in a search URL for a + user property will use the current logged in user. Now you + can share searches like: "My issues" as "my" will become + the current logged in user. -* Postgres full text index can now be enabled. +* login failures to the REST/Xmlrpc interfaces ae now rate + limited. -* Modifications to in-reply-to threading when there are multiple - matches resulting in more predictable handling of messages. +* utf8mb4 is the default charset for MySQL. This requires + migrating your database using the mysql client. You can + choose to keep the older character set in config.ini. -* Many updates to documentation to make it scannable, useful and - work on mobile. +* PostgreSQL services defined in pg_service.conf can be + used. PostgreSQL schemas are supported to eliminate the + need for the roundup user to have database + creation/deletion privs. -* Admin documentation includes a section on setting up Content - Security Policy (CSP) to better secure your Roundup trackers. +* fix out of memory issue when importing larger trackers + into PostgreSQL. -* REST now allows rate limiting headers to be accessed by client - JavaScript. +* multiple roundup-admin improvements: display protected + properties (like creation date), better formatting of + output, command history. Also on windows, pyreadline3 is + supported to provide an editable interactive command line. -* Default number of rounds for PBKDF2 updated to 2M to account for - improvements in password crackers and CPU power. +* an experimental wsgi performance improvment in 2.3.0 is + now now the default and is opt-out. -* Support PBKDF2 with SHA512 for password storage to improve - resistance to password crackers. +* new template functions: utils.readfile and + utils.expandfile. Javascript that is included in the + Python core will be moved to external files and be able to + have values from Roundup substituted in the Javascript. -* Deprecate SSHA password hash function. +* allow content-type of a template to be set from inside the + template. This allows returning json or xml from a + template without a .json or .xml extention. -* roundup-admin reindex can be done in batches to manage load - incurred by reindexing. +* fix import/export on windows to use Unix style line + endings fixing export/import on Windows and making exports + portable across platforms. -* roundup-admin can list available templates and their installed - locations. This is useful when installing via pip or in a docker - container as supporting files are not stored in the usual locations - like /usr/share/roundup. +* various other Windows platform fixes including test suite + fixes. -* Crash fixes in detector handling +* sqlite version 1 and StructuredText support removed. -The file CHANGES.txt has a detailed list of feature additions and -bug fixes (53) for each release. The most recent changes from -there are at the end of this announcement. Also see the -information in doc/upgrading.txt. +The file CHANGES.txt has a detailed list of feature +additions and bug fixes (58) for each release. The most +recent changes from there are at the end of this +announcement. Also see the information in doc/upgrading.txt. -If you find bugs, please report them to issues AT roundup-tracker.org -or create an account at https://issues.roundup-tracker.org and open a -new ticket. If you have patches to fix the issues they can be attached -to the email or uploaded to the tracker. +If you find bugs, please report them to issues AT +roundup-tracker.org or create an account at +https://issues.roundup-tracker.org and open a new ticket. If +you have patches to fix the issues they can be attached to +the email or uploaded to the tracker. Upgrading ========= -If you're upgrading from an older version of Roundup you *must* follow -all the "Software Upgrade" guidelines given in the doc/upgrading.txt -documentation. +If you're upgrading from an older version of Roundup you +*must* follow all the "Software Upgrade" guidelines given in +the doc/upgrading.txt documentation. Note that you should run ``roundup-admin ... migrate`` for all your trackers to update the database schema version. Do this before you use the web, command-line or mail interface and before any users access the tracker. -Roundup requires Python 2 newer than version 2.7.12 or Python 3 newer -than or equal to version 3.6 for correct operation. (Python -3.4 or 3.5 may work, but are not tested.) Note that Python 2 support -is being removed from the CI platforms, so you should deploy new -trackers with Python 3 and plan on upgrading older trackers from Python -2 to Python 3. See the upgrade guide. +Roundup requires Python 2 newer than version 2.7.12 or +Python 3 newer than or equal to version 3.6 for correct +operation. (Python 3.4 or 3.5 may work, but are not tested.) +Note that Python 2 support is being removed from the CI +platforms, so you should deploy new trackers with Python 3 +and plan on upgrading older trackers from Python 2 to Python +3. See the upgrade guide. -To give Roundup a try, just download (directions above), unpack and run:: +To give Roundup a try, just download (directions above), +unpack and run:: python demo.py then open the url printed by the demo app. Release info and download page: + https://pypi.org/project/roundup/ + Source and documentation is available at the website: + https://www.roundup-tracker.org/ + Mailing lists - the place to ask questions: + https://sourceforge.net/p/roundup/mailman/ About Roundup ============= -Roundup is a simple-to-use and install issue-tracking system with -command-line, web and e-mail interfaces. It is based on the winning design -from Ka-Ping Yee in the Software Carpentry "Track" design competition. - -Note: Ping is not responsible for this project. The contact for this -project is rouilj at users.sourceforge.net. Use this address for -security or other sensitive issues. Development discussions occur on -the roundup-devel at lists.sourceforge.net mailing list. Tickets can -be opened at https://issues.roundup-tracker.org. +Roundup is a simple-to-use and install issue-tracking system +with command-line, web and e-mail interfaces. It is based on +the winning design from Ka-Ping Yee in the Software +Carpentry "Track" design competition. -Roundup manages a number of issues (with flexible properties such as -"description", "priority", and so on) and provides the ability to: +Roundup manages a number of issues (with flexible properties +such as "description", "priority", and so on) and provides +the ability to: (a) submit new issues, (b) find and edit existing issues, and (c) discuss issues with other participants. -The system facilitates communication among the participants by managing -discussions and notifying interested parties when issues are edited. One of -the major design goals for Roundup that it be simple to get going. Roundup -is therefore usable "out of the box" with any Python 2.7.2+ (or 3.6+) -installation. It doesn't even need to be "installed" to be operational, -though an install script is provided. +The system facilitates communication among the participants +by managing discussions and notifying interested parties +when issues are edited. One of the major design goals for +Roundup that it be simple to get going. Roundup is therefore +usable "out of the box" with any Python 3.6+ +installation. It doesn't even need to be "installed" to be +operational, though an install script is provided. It comes with five basic issue tracker templates @@ -146,161 +165,12 @@ It comes with five basic issue tracker templates * a jinja2 version of the devel template (work in progress) * a minimal skeleton -and supports four database back-ends (anydbm, sqlite, mysql and postgresql). +and supports four database back-ends (anydbm, sqlite, mysql +and postgresql). Recent Changes ============== -From 2.2.0 to 2.3.0 - -Fixed: ------- - -- Updated directions for verifying Roundup distribution using pgp. -- Dockerfile healthcheck fixed so it works when trackers are - specified on command line. Also cleanup of unneeded - packages. (John Rouillard) -- issue2551224 - Replace dbm db for sessions and otks when using - sqlite. New databases are created for session data (db-session) - and one time key data (db-otk). The data is ephemeral so no - need to migrate. (John Rouillard) -- issue2551223 - Timestamps are truncated in mysql and postgresql - for session and otk database tables. Modify db schema to use a - numeric type that preserves more significant figures. See - upgrading.txt for required steps. (John Rouillard) -- added more testing of BasicDatabase to support use of SQLite - for that purpose. Had to fix memory, rdbms and dbm edge cases - due to new tests. (John Rouillard) -- issue2551138 - roundup-server with ssl under python2 throws - traceback on socket close. Not sure how this got fixed, - but after fixing issue2551137 it was not an issue anymore. -- issue2551137 - roundup-server won't run with ssl under python3 - Fixed by using SocketIO and manually adding buffering io and - catching SSL.ZeroReturnError indicating SSL has been shut down. -- add caching header for text/javascript in addition to depricated - application/javascript. (John Rouillard) -- Enable postgres-fts: fix indexer-common::get_indexer so it returns a - postgresql-fts Test code paths in get_indexer. (John Rouillard) -- Fix Postgres native-fts, implement a two phase initialization of the - indexer. The native-fts one gets assigned after the database - connection is open. (John Rouillard) -- fix crash if postgresql native-fts backend is asked to index content - with null bytes. (John Rouillard) -- issue2551232 - modify in-reply-to threading when multiple matches - Change how in-reply-to threading works in the mailgw. If there is - more than one issue with a matching parent message, fall back to - subject matching. See upgrading.txt for details. (John Rouillard) -- issue2551195 - port scripts from optparse to argparse (Ralf Schlatterbeck) -- issue2551246 - mitigation, document how -u doesn't work for - roundup-admin. (John Rouillard) -- Document better that files in the template or static_files - directories accessed via @@file are available to any user with the - url. (John Rouillard) -- Fix final exception handler in roundup-server to send proper - Content-Length header to the client. (John Rouillard) -- Fix traceback if Origin header is missing. (John Rouillard) -- issue2551250: Fix sorting of detectors even if there are two with the - same name and priority (can happen if they are created in two - different files). -- Fix Traceback when a numeric order attribute is empty (Ralf - Schlatterbeck) -- Update some template schema files to assign Register permissions for the - Anonymous user. Replaces the old Create permission. (John Rouillard) -- Allow '*' and explicit origins in allowed_api_origins. Only return - 'Access-Control-Allow-Credentials' when not matching '*'. Fixes - security issue with rest when using '*'. (John Rouillard) -- issue2551263: In REST response expose rate limiting, sunset, allow - HTTP headers to calling JavaScript. (John Rouillard) -- issue2551257: When downloading an attached (user supplied file), - make sure that an 'X-Content-Type-Options: nosniff' header is sent. - (John Rouillard) -- issue2551252 - default number of rounds for PKDF2 password increased - to 2,000,000. (John Rouillard) -- issue2551251 - migrate/re-encrypt PBKDF2 password if stored - password used a smaller number of rounds than set in - password_pbkdf2_default_rounds. (John Rouillard) -- upgrade from jquery-3.5.1 to jquery-3.6.3. Update user.help.html - to new version. (John Rouillard) -- Dockerfile scanned with hadolint. Fixed multiple issues. (John Rouillard) -- fix crash due to invalid initialization/reset of configuration.py - option_validators. Crashed roundup-admin on second command if an - option_validator was added by a detector or extension. (John Rouillard) -- Dockerfile uses dumb-init to properly wait for child/zombie - processes. Defense against child process starting from detector - and becoming a zombie when its roundup-server instance exits. - (John Rouillard) -- Move installed frontend/Zope back to frontend/ZRoundup - directory. This better identifies the directory when copied into - the Zope framework. It also matches existing - documentation. (John Rouilard) -- Multiple fixes/updates for installation documentation. - Including docker shell/admin/demo mdoes. (John Rouillard) -- Invalid item identifiers passed to REST endpoint return a 404 - rather than a 400 error. E.G. /rest/data/issue/issue4 (rather - than .../issue/4). (John Rouillard) -- issue2551280 - sorted() method of MultilinkHTMLProperty is broken? - (Gabor Nagy report and fix; commit John Rouillard) - -Features: ---------- - -- Add warning about limited Python 2 support lifetime to install and - upgrading docs. (John Rouillard) -- Dockerfile supports demo mode for instant gratification - 8-). Also supports shell and admin mode (John Rouillard) -- Dockerfile build allows adding additional python packages via - pip, setting UID tracker is run under. (John Rouillard) -- issue2551140 - Added redis as a session and otk database for use - with anydbm and sqlite primary databases. (John Rouillard) -- issue2550559 - Pretty printing / formatting for Number types. - Added pretty(format='%0.3f') method to NumberHTMLProperty to - print numeric values. If value is None, return empty string - otherwise str() of value. (John Rouillard) -- sqlite native-fts backend now uses the stopwords list in config.ini - to filter words from queries. (Stopwords are still indexed so that - phrase/proximity searches still work.) (John Rouillard) -- sqlite databases use WAL mode when *created* to improve read - concurrency. Existing sqlite database still use rollback journal - mode. See upgrading.txt for details. (John Rouillard) -- issue2551233 - create new roundup-admin command "templates" list all - template names, location and descriptions. Should help find where - /usr/share/roundup/templates is buried during some install - mechanisms. Does not need a tracker home to run. (John Rouillard) -- Add OAuth authentication to the mailgw script. Now IMAPS can be used - with OAuth as required by several large cloud providers. Move command - line processing of the mailgw script to ``argparse``. Note that the - command line options of the mailgw have changed, see upgrading.txt for - details. (Ralf Schlatterbeck) -- issue2551243: schema-dump.py enhanced with anti-CSRF headers. Flake8 - cleanup and python2 support. (John Rouillard) -- issue2551253 - new password hash PBDKF2-SHA512 added. Not - available by default. Follow directions in upgrading document - to use. (John Rouillard) -- roundup-admin migrate command reports the schema version. -- issue2551262 - the mail gateway subject prefix now allows spaces - before/after prefix. Also allow spaces between classname and id - number in prefix designator. So "[ issue 23 ] subject" is parsed - like "[issue23] subject". (John Rouillard) -- [doc]: add section on implementing CSP for Roundup to admin - doc. (John Rouillard) -- issue2551265 - deprecate SSHA password hash method. Users using SSHA - passwords will have their passwords transprently upgraded to PBKDF2 - derived hash on next login. (John Rouillard) -- issue2551253 - Modify password PBKDF2 method to use SHA512. New - hash function using PBKDF2-SHA512 available. Will be default in - future. Directions for upgrading security by using it now is - included in upgrading.txt. (John Rouillard) -- issue2551275 - Allow configuring max_children in roundup-server. - When using roundup-server in fork mode, allow raising number of - forked children above the default of 40. (Original patch by Joseph - Myers, config settings and docs by John Rouillard.) -- roundup-admin genconfig does not need a tracker home to run. (John - Rouillard) -- issue2551190 - Allow roundup-admin reindex to work in - batches. Running roundup-admin -i ... reindex issue:1-1000 will - reindex the first 1000 issues while reporting any missing issues - in the range. Also completion progress is reported when indexing a - specific class. -- doc updates: add explanation for SQL code in 1.3.3->1.4.0 upgrade. - document schema table in rdbms backends and how to dump/extract - version from them. (John Rouillard) +From 2.3.0 to 2.4.0 + +TBA