All notable changes to this project will be documented in this file.
Generated by Changy.
No changes.
- ff-209 - Uped priority of Reddit feeds discoverer "hack".
- ff-209 — Improved logging for feeds loading & discovering logic.
- ff-205 — fixed processing of quota errors in Google LLM provider.
- ff-206 — skip entries that caused temporary errors in the general LLM processor.
Operations required AFTER UPDATE:
- Run migrations
ffun migrate
Changes:
- ff-174 — Custom API entry points for OpenAI and Gemini clients.
- ff-175 — Export feeds to OPML file.
- ff-171 — Simplified rules creation:
- The rules creation form was moved to the tags filter.
- Rules now allow specifying excluded tags (will not apply if the news has any of them).
- Tags under the news caption now behave as tags in the tags filter.
- Button
[more]on the tags line under the news caption now opens the whole news. - Openned news items now always show all tags.
- Only a single news item now can be opened.
- Performance of the News page improved 2-3 times.
- ff-172 — Added feeds collection "Entrepreneurship & Startups". Also added cli commands
ffun estimates entries-per-day-for-collectionandffun estimates entries-per-day-for-feed - ff-199 — Fixed Reddit feeds discovery.
- ff-173 — Significantly improved feeds discovery. Also:
- Added types for URL values for better consistency control.
- GUI now will reload news and feeds after importing an OPML file.
- ff-156 — Further optimizations of
/api/get-last-entries, concentrated onontology.operations.get_tags_for_entries.
- ff-158 — Improved prompts to produce less tags of better quality.
- ff-159 — Fixed the exception in the API when no entries found.
- ff-153 — The model Gemini 1.0 pro is removed from fixtures because Google discontinued its support.
-
ff-155 — Performance improvements.
- use orjson instead of standard json lib for response serialization;
- changed response format of
/api/get-last-entriesand/api/get-entries-by-idsAPI to reduce the size of the response and speed up its generation.
- Track only user metrics with values != 0.
In this release, we added a bunch of tracking functionality to better understand how users interact with the app.
This analytics:
- is strongly for our internal use;
- uses only internal IDs — does not contain emails, urls, etc.
- in most cases does not exceed what we already have in logs and database — just a more convenient way to access the data.
Changes:
- Frontend dependencies upgraded.
- ff-141 — Track frontend events
news_link_opened,news_body_opened. - ff-103 — Added
in_collectionmarker tofeed_linkedandfeed_unlinkedevents. - ff-142 — CLI command to track aggregated metrics.
- ff-143 — Fixed a situation when, after removing a feed on the Feeds page, the feeds list was not updated.
- ff-145 — Track users' interest in feeds.fun social channels.
- ff-119 — Track OPML imports.
- ff-112 — Track subscriptions on feeds from collections.
- ff-131 — Track subscription dynamics on feeds.fun news channels.
- Added normalization to business event attributes.
- Updated frontend packages.
- ff-70 — Business events are written in logs.
- ff-68 — Tech metrics are written in logs. They can be filtered and processed by tools such as SplitBit or Vector and redirected to monitoring systems. Note that metrics are written at the INFO log level—if a higher log level is set, they will be hidden with all INFO logs.
- Changed the environment variables prefix for logs settings from
FFUN_toFFUN_LOGS_.
- Mark all ssl errors catched by httpx as expected.
- Connection error
[SSL: TLSV1_UNRECOGNIZED_NAME]added to the list of known "ok" errors for the feeds loader.
- Connection error
[SSL: CERTIFICATE_VERIFY_FAILED]added to the list of known "ok" errors for the feeds loader.
- ff-56 — Connection error
[SSL: WRONG_VERSION_NUMBER]added to the list of known "ok" errors for the feeds loader. - ff-51 — "No description" text for feeds without a description in the Feeds list.
- ff-52 — Fixed the
remove-deprecated-settingscommand, now it really removes only deprecated settings. - ff-50 — Unsubscribing from a feed now do not lead to excessive server requests due to cache invalidation.
- Removed dependency on
pkg_resources.
Operations required AFTER UPDATE:
- Run migrations
ffun migrate - Run
ffun user-settings remove-deprecated-settings - If you are using an OpenAI tag processor, specify a custom configuration file for tag processors with an API key. Details can be found in the updated README.md
- If you want to periodically clean your database from old entries, add the call
ffun cleaner cleanto your cron tasks. It is recommended.
Changes:
- gh-229 — Removed
Allow using OpenAI key for feeds in standard collectionsuser settings. It was confusing for users and not usable in the project's current state. Instead, you can specify an LLM API key to process collections (one per tag processor). By new logic, the service's owner is responsible for managing tags for all feeds in collections. - gh-245 — Implemented the universal LLM tag processor. Currently, it supports two API providers: OpenAI (ChatGPT) and Google (Gemini). Also:
- The configuration of tag processors was moved to a separate file; see README.md for actual details.
- The configuration of LLM models was moved to a separate file; see README.md for actual details.
- You can configure multiple LLM tag processors with different parameters, including prompts, API keys, etc.
- Token usage tracking has been replaced with cost tracking. User settings will convert automatically.
- Collections refactored. Now, they are configurable and empty by default. You can specify the path to the directory with collections configuration in
FFUN_FEEDS_COLLECTIONS_COLLECTION_CONFIGS. An example of configuration can be found in the./feeds_collections/collections. FFUN_META_MAX_ENTRIES_PER_FEEDsetting moved toFFUN_LIBRARY_MAX_ENTRIES_PER_FEED.
- gh-136 — Links to the Discord server and to the Reddit community.
- gh-24 — Link to the blog. Blog RSS is added to collections.
- gh-220 Removed Home link from internal pages.
- gh-221 Slighly increased width of the central column on the main page.
- gh-241 Optional plausible analytics for the frontend.
- gh-41 Build dev service images only once.
- gh-231 CLI tool and base of texts to measure quality of tag processors.
- gh-233 Improved prompt of OpenAI General processor. It should start to output more relevant tags.
- gh-234 (possibly) Fixed deadlocks in updating tags properties by
apply_tags_properties.
- gh-231 CLI tool and base of texts to measure quality of tag processors.
- gh-233 Improved prompt of OpenAI General processor. It should start to output more relevant tags.
- gh-234 (possibly) Fixed deadlocks in updating tags properties by
apply_tags_properties.
Configs migration: if you have turned on OpenAI ChatGPT processors, like FFUN_LIBRARIAN_OPENAI_CHAT_35_FUNCTIONS_PROCESSOR__ENABLED="True" or FFUN_LIBRARIAN_OPENAI_CHAT_35_PROCESSOR__ENABLED="True", remove their configs and add configs for OpenAI General Processor, like FFUN_LIBRARIAN_OPENAI_GENERAL_PROCESSOR__ENABLED="True"`, instead. The semantic of the configs is the same.
- gh-227 — Old OpenAI ChatGPT processors is replaced with OpenAI General Processor which utilizes power of GPT-4o-mini model.
Recommendation: Because GPT-4o-mini is much smarter than GPT-3.5-turbo, I recommend you to disable GPT function processor FFUN_LIBRARIAN_OPENAI_CHAT_35_FUNCTIONS_PROCESSOR__ENABLED="False" and use only standard GPT processor FFUN_LIBRARIAN_OPENAI_CHAT_35_PROCESSOR__ENABLED="False". In gh-227, one of the processors will be refactored, and one of them will be removed.
- gh-226 Switch used OpenAI model to gpt4o-mini
DB migration required: no
- gh-217 Fixed:
apply_tags_propertiesfails because of duplicated tags.
DB migration required: yes
- gh-214 Added not null constraint to the
o_tags_properties.tag_idcolumn. - gh-213 Fixed:
Native tagstags processor periodically fails with deadlock in the database.
Breaking changes:
- gh-68 Supertokens services upgraded to version 9.0.2. If you are using Supertokens, you should upgrade them manually (from 5.0.0 to 9.0.2) according to the instructions in https://github.com/supertokens/supertokens-core/blob/master/CHANGELOG.md
Changes: