v0.13.4

What's New?

Documentation Website 🎉

Context Mod now has a website generated with Just The Docs. This site will be generated and available locally if the host environment supports its build requirements. It can also be built and served separately (see docs).

A big re-organization of the docs occurred due to the way Jekyll works but overall things should be clearer and better organized now! The site even has local search available.

Docker-Compose Support

Context Mod is now fully tested and supported using docker-compose.

The docker-compose template enables one-line setup of MariaDB, Redis, Grafana, and Influx.

See the docker-compose usage documentation for more.

Full Changelog

Features

• (docs) Generate docs on startup or in docker build
• (docs) Improve jekyll dependencies to speed up docker image build
• (No Category) Add full docker-compose support and documentation
• (No Category) Add some author properties for templating
• (No Category) Implement templating for flair actions

Bug Fixes

• (database) Prevent usage of LIMIT in session storage driver when db backend is mysql/mariadb
• (No Category) Modify reddit entity parsing regex to allow subreddit names that have been replaced with placeholders
• (No Category) Fix including self activities in Recent Activity without filtering
• (flair) - Fix endpoint used when assigning flair from template id @rysie 🎉

Documentation

• (No Category) Initial implementation of just-the-docs
• (No Category) Fix detail expansion blocks
• (No Category) Use project README as index and fix more links
• (No Category) Add doc site development documentation
• (No Category) Add missing mermaid version
• (No Category) Add more filters documentation
• (No Category) Update docker compose instructions

Miscellaneous Tasks

• (No Category) Add excluded folders for idea
• (No Category) Exclude jekyll generated files from typescript compiler
• (No Category) Remove comments about wrong endpoints

Refactor

• (polling) Improve resilience for polling source parsing

v0.13.3

What's New?

Easier Shared Configs

Sharing parts of a config are not new but the mechanism has been improved to enable extracting Runs from full, unaltered configs.

This allows Subreddit A to reference the full-fat config for Subreddit B and get its Runs without any interaction required from Subreddit B. The only changes required for Subreddit B is adding a sharing property in their config to enable their config to be shared.

Comment as Subreddit

CM can now use the new remove/comment as subreddit moderator feature from reddit. This enables CM to invoke /u/MySubreddit-ModTeam to comment on a Submission or Comment with an arbitrary message. The only restriction being that the Activity being replied to must already be removed before replying.

This new feature is part of the Comment Action and can be used by specifying asModTeam: true.

Cookbook

The "subreddit-ready" documentation has been overhauled into a Cookbook of CM "recipes" detailing common uses of CM functionality. The new cookbook includes many more useful scenarios compared to the old documentation and will continue to be be improved!

Action Template Variables

Most Actions with folders in the components docs now include a section detailing all of the variables available for use in Action Templating. Look for the Template Variables section (example) in each action's folder for a table of variables with descriptions and examples.

Full Changelog

Features

• (cache) Add default prefix and shared option for interacting with cache
• (cache) Add force/shared options for getting content/wiki/url results from subreddit resources
• (comment) Implement commenting as subreddit
• (config) Add config fragments to shared cache
• (config) Refactor manager config updates to check hydrated structure
• (config) Implement ACL for full config sharing
• (docs) Add more cookbook recipies
• (filter) Add same regex/matching functionality for author flair filtering as item has
• (filter) Improve building filters from config

Bug Fixes

• (cache) Fix pre activities being wrongly reconstructed when retrieved from memory provider cache
• (cache) Fix more memory provider cache object reconstruction issues #123
• (filter) Re-add missing item is criteria to filter test
• (filter) Fix filtered usernote assignment when search type is current and no notes exist
• (filter) Fix usernote cutoff date comparison
• (filter) Re-add missing modaction filtering improvements
• (mhs) Fix examples missing criteria property
• (ui) Add workaround for removal reason copy-to-clipboard when in unsecure context
• (No Category) Fix how submission/comment snoowrap entities are generated from reddit thing ids

Documentation

• (No Category) Typo fix
• (No Category) Add template variable examples for most rules #121
• (No Category) Add cookbook recipe for monitoring popular submissions

Refactor

• (docs) Improve CM examples by using a "cookbook"
• (modnotes) Encapsulate filtering functionality in modnote class for future testing
• (No Category) Consolidate ttl values into well-defined object
• (No Category) Extract stat-related functionality from SubredditResources into its own class
• (No Category) Improve reuseability of SubredditResources
• (No Category) Move cache key prefix into CMCache

v0.13.2

What's New?

Mod/User Note Filtering Improvements

Criteria for filtering by userNotes or modActions in authorIs now has additional properties:

• referencesCurrentActivity filters notes based on whether the currently processing activity is associated with the note
• note allows matching note text content with a list of strings (case-insenstive, subset of) or regular expressions
• Both UserNote and ModNote Actions now have existingNoteCheck property which supersedes allowDuplicates
  • When true emulates allowDuplicates, when false all notes are always added
  • Otherwise, can be a UserNoteCritera or ModLogCriteria (for the respective action) -- if the criteria passes then the note is added

Real-time logging UI Improvements

• The "loading" icon in the real time logs in the dashboard is now actually connected to streaming status
• Better error handling and error visibility when streaming fails
  • Error is shown instead of loading icon, when it occurs
  • A link in the error message allows for manually restarting the stream

Full Changelog

Features

• (bot) Improve log wording for manager loading phase
• (filter) Add author flair item criteria
• (filter) Improve modAction filtering functionality
• (modnote) Implement existing note check before adding note to replace allowDuplicates
• (testing) Implement initial author filter tests
• (ui) Add favicon files
• (ui) Improve visibility and resilience for live log stream
• (usernote) Improve usernote filtering functionality

Bug Fixes

• (filter) Fix detecting empty filter when using 'replace' filter default behavior
• (filter) Fix mod action note filtering assignment
• (logs) Remove listeners from log stream event emitter before end of response to prevent write-after-end errors
• (No Category) Fix custom footer never loading
• (No Category) Attempt to decrease frequency of influxdb timeout errors
• (No Category) Further improvements for influxdb logging
• (No Category) Add missing else condition

Documentation

• (No Category) Fix malformed URLs

Refactor

• (recent) Log image parsing error with cause

v0.13.1

What's New?

History Activity Ratio #111

The new History Criteria ratio enables comparing two sets of activity (from two different sets of subreddits and/or windows) to determine the amount of activity in one set against the amount of activity in the other.

See the full documentation for usage and an example.

Full Changelog

• fix: Fix wiki location usage when getting from subreddit resources
• feat(history): Log failure results for easier rule tuning
• chore: Add patreon link

v0.13.0

What's New?

ModerateHateSpeech.com Toxic Context Detection

Integrates ModerateHateSpeech.com toxic content prediction model as a Rule. This provides a more accurate/targeted approach to detecting troll/toxic behavior than the Sentiment Rule and is trained specifically to work on reddit content.

Check the full MHS Rule docuementation to learn more.

Subreddit Onboarding

Adding a Subreddit to an existing bot now includes a "helper" page similar to Bot Onboarding where an operator can specify

• list of initial Guests
• a URL/wiki token to an initial config to save to the subreddit

If either option is specified a moderator of the subreddit must access the generated onboarding link and accept the invitation. This should reduce barrier to entry but consolidating common onboarding steps for moderators into one page.

Note: Onboarding feature requires database migration (minor) which is the reason for the minor version bump.

createdOn Item Criteria

New criteria allows filtering an Activity by the relative date it was created on. Example:

createdOn: wednesday # activity was created on a wednesday

createdOn: '* * 26 * *' # cron expression, activity was created on the 26th day of the month

This allows CM to enforce things like

only posts with [MLM] in title are allowed to be submitted on mondays

Full Changelog

Bug Fixes

• (client) Improve responses dependent on server information
• (filter) Fix copy-past typo for createdAt
• (recent) Empty viable activity list if processing activity is required to be reference but is not a valid type
• (No Category) Improve error detection for log transform
• (No Category) Check bot entity exists before getting invites
• (No Category) Set default invites to empty array on heartbeat
• (No Category) Fix subreddit filtering result when no subreddit criteria are present
• (No Category) Fix typo in default footer

Documentation

• (No Category) Update screenshots
• (No Category) Add general subreddit/moderator docs readme
• (No Category) Add docs for MHS toxic content prediction rule #110
• (No Category) Fix some typos

Features

• (filter) Add createdAt activity criteria matching
• (filter) Rename createdAt to createdOn
• (mhs) Add default confidence threshold
• (ui) Improve finished onboarding response
• (No Category) Util function for generating full wiki urls
• (No Category) Improve external resource fetching in subreddit resources
• (No Category) Add subredditExists function to snoowrap client
• (No Category) Implement rule that tests activity content against moderatehatespeech.com ML model using api #110

Miscellaneous Tasks

• (No Category) Add tests for item criteria

Refactor

• (manager) Improve wiki CRUD and onboarding initial config
• (No Category) Improve activity source parse and comparison

v0.12.2

Full Changelog

Features

• (author) AuthorIs 'name' criteria may be regular expression
• (ui) Improve guest expiration interface
• (ui) Add home logo
• (No Category) Update github actions to push to multiple registries

Bug Fixes

• (config) Use correct filter defaults data structure for runs
• (filter) Fix missed toLowerCase for named filter getter
• (ui) Fix typo in reddit status indicator switch

Documentation

• (No Category) Add GHCR image location to install docs

Refactor

• (ui) Replace websockets opStats with client delta polling #106

v0.12.1

What's New?

Templating got some much needed QOL updates in individual Rules as well as Action results now available in the {{actions}} object and as a markdown list of action results at {{actionSummary}}.

Full Changelog

Features

• (action) Implement Action templating data #104
• (message) Implement templating for to field so subreddit can be used generically with {{item.subreddit}}
• (template) Add subreddit breakdown to template data for Recent and History rule ({{rules.myrule.subredditBreakdown}})
• (template) Refactor templating to be more extensible and add some requested data
• (template) Add {{check}} to top-level template variables #87

Bug Fixes

• (image) Fix hash result test assertion
• (submission) Fix targets data type to accept array
• (template) Return empty string if no subreddits in breakdown

v0.12.0

What's New?

Big, new features for Operators and Moderators alike!

Guest Access

Guest Access allows moderators of a subreddit to specify any reddit user that may access their Subreddit's dashboard and configuration as if they were a moderator. This enables moderators to share temporary access to non-moderators and allows these guests to view and edit their subreddit's configuration. Editing is done via the bot and the guest who edited is noted in the wiki page edit reason.

• Only moderators (not operators) are allowed to add/modify guests
  • Access has a built-in expiration date that can be specified on add

Additionally, operators can specify users who should be given initial guest access (24 hours) on a bot invite. This could, for example, enable a more frictionless experience for new bots/moderators by allowing operators who have also built a config to "complete" the onboarding process for moderators.

Due to changes in database structure required to make Guests work correctly and securely existing bot invitations will be lost on migration (breaking change). If you have existing invitations you must re-create them after migration.

Arbitrary Comment/Submission

Previously CM could only create comments in response to the processing Activity, or its parent Submission.

Now CM can now create comments and submissions in arbitrary locations using targets. Check the Comment Action and Submission Action for how to use targets.

Simplified Initial CM Setup Process

When CM is started with no configuration visiting the web interface will display a new First Time Setup page where you can input the required info (reddit client and operator) for a minimum configuration and write the config with one button click.

With this change it is now possible to run CM without ever modifying the configuration file yourself.

Full Changelog

There were many other improvements and small features added in this release. Check out the full changelog below:

Features

• (action) Implement creating submissions #93
• (comment) Implement arbitrary comment targets #93
• (config) Allow wikiConfig override at subreddit-level
• (database) Implement automated backup for better-sqlite3 connection
• (docs) Simplify getting started operator instructions
• (image) Improve image normalization to make duplicate detection more powerful
• (image) Implement image flip detection in recent rule
• (remove) Implement removal reason id and note
• (ui) Implement saving config as guest mod
• (ui) Add guided tour for main dashboard
• (ui) Add initial guests to bot invite and rename Guest Mod to Guest
• (No Category) Add removal reason related endpoints to snoowrap client
• (No Category) Add initial setup wizard

Bug Fixes

• (auth) Fix default instance middleware accessibility check
• (auth) Fix subreddit normalization
• (auth) Fix subreddit normalization on client instance
• (config) Fix setting api friendly name
• (database) Handle invite migration when db is sqljs
• (database) Fix how new columns are added to existing table on invite migration
• (database) Remove table data check due to incompatibility with postgres
• (guest mod) Parse/clean name from user input before adding
• (image) Url variable name usage
• (manager) Fix missing valid config check before starting queue
• (ui) Fix constant reassignment
• (No Category) Remove debug statements for image processing
• (No Category) Cleanup guests and no-config scenario
• (No Category) Fix missing column type specifying for db session entity

Documentation

• (No Category) Remove removal reason footnote
• (No Category) Fix incorrect url for minimum config

Miscellaneous Tasks

• (No Category) Re-add clean username for guest due to merge
• (No Category) Bump version constant

Refactor

• (guest) Migrate invites to be owned by the server
  • BREAKING: Migrate invites to be owned by the server
• (image) Normalize image hash usage a bit more to reduce complexity
• (No Category) Simplify ACL for server/client users and introduce guest context

v0.11.5

What's New?

This release is mostly bug fixes, influx improvements, and QOL for operators.

Full Changelog

Bug Fixes

• (filter) Make source filter stricter by requiring exact value rather than "contains"
• (ui) Use correct auth middleware for api proxy endpoint
• (ui) Fix setting style for non-existing element on subreddit invite
• (ui) Fix missing bot parameter for delayed activity cancelling

Documentation

• (No Category) Add memory management notes

Features

• (config) Gate memory monitoring behind operator config option monitorMemory
• (docker) Add NODE_ARGS (ENV) variable in node run command to allow passing arbitrary args through docker ENV
• (influx) Add manager health metrics
• (influx) Add server memory metrics
• (server) Add server child logger to requests for easier logging
• (server) Better handling for subreddit invite CRUD
• (templating) Include activity id and title for all activities
• (templating) Render content for more actions and properties (Ban action props)
• (ui) Improve delayed activity cancel handling server-side
• (ui) Add Cancel All button on delayed activities popup
• (ui) Reduce data usage for live stats using response deltas better stream handling on the browser
• (ui) Real-time data usage improvements based on page visibility

Miscellaneous Tasks

• (schema) Update schema to reflect ban property max length changes

v0.11.4

What's New?

Partial Configurations

Partial Configurations allow parts of a subreddit configuration -- Runs, Checks, Rules, and Actions -- to be replaced with a URL or Reddit wiki resource. CM will retrieve the text response from the resource and use that as the configuration.

This enables consolidating common/shared configuration fragments between many subreddits as well as supporting usage of external URLs to store configuration.

Influx/Grafana

CM can now write time-series data to InfluxDB. It currently supports writing general statistics (events processed/seen, etc.) and api health for bots and individual subreddits as well as recording the names of checks/runs/actions triggered for each subreddit. This enables moderators, as well as data scientists, to track custom metrics in real time.

Additionally, there is a pre-built Grafana dashboard to showcase some of the data CM writes.

Report Tracking over Time

If a Subreddit is polling modqueue CM will automatically track all new/seen Activities and store the time new reports are made at. This enables filtering reports based on a time window in itemIs:

itemIs:
  - reports: '> 3 in 30 minutes'

A few caveat for this due to the way Reddit structures report data -- Activities with more than 1 existing reports before CM is running will have their existing report times "averaged" across the lifespan of the Activity IE A 1 hour old Activity with 3 existing reports will look like it has 1 report every 20 minutes.

Full Changelog

Bug Fixes

• (cache) Use correct cache key and improve external url cache re-use
• (database) Fix processing/queued time value
• (filter) Fix logic for assigning number of reports to compare
• (modnote) Use correct activity identifier for reddit_id
• (usernote) Fix regression with note types
• (No Category) Fix unnecessary subreddit fetch call on init

Documentation

• (config) Add partial configuration documentation
• (database) Add influx/grafana documentation
• (sentiment) Add sentiment documentation
• (usernote) Document usernote types in schema and in documentation

Features

• (ban) Implement templating for ban note/reason
• (bot) Implement better in-situ subreddit add/removal
• (config) Initial implementation for partial configs #76
• (config) Implement cache control for config fragments #76
• (config) Support config fragments as array of fragments #76
• (config) More error handling and internal improvements to config fragment parsing
• (database) Initial influx db implementation #84
• (database) Add api health measurement to influxdb
• (filter) Implement specifying report reason in report comparison
• (filter) Implement filtering by upvoteRatio for authorIs
• (report) Implement report tracking #78
• (report) Implement filtering reports in itemIs by time
• (No Category) Improve api health monitoring
• (No Category) Add more Activity data to templating data #94
• (No Category) Support parsing filename from non-raw GIST url
• (No Category) Support parsing filename from non-raw GIST url in editor

Refactor

• (database) Move Event creation to earlier in activity handling function for a more accurate processing time
• (logging) Improve log retention performance
• (report) Cleanup monitoring in manager