From 8e1be924c897b7122482fc5136389359f7d13eb7 Mon Sep 17 00:00:00 2001 From: slatinsky <slatinsky.adam@gmail.com> Date: Mon, 9 Oct 2023 02:01:53 +0200 Subject: [PATCH] simplify thread export instructions in the docs --- README.md | 35 +++++------ docs/Exporting-forum-posts.md | 48 --------------- docs/Exporting-threads.md | 106 ---------------------------------- 3 files changed, 18 insertions(+), 171 deletions(-) delete mode 100644 docs/Exporting-forum-posts.md delete mode 100644 docs/Exporting-threads.md diff --git a/README.md b/README.md index b1274438..7a314ea2 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ Browse your [Discord chat exports](https://github.com/Tyrrrz/DiscordChatExporter ## Quick start (Windows) -1. Export your data from Discord using [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/tree/master/.docs). Your export should be in JSON format, disable `format markdown` option and enable `download assets` and `reuse assets` options. If exporting using a CLI - use `--media --reuse-media --markdown false --format Json` command line options. More information in [How to export data from Discord to view it in DCEF?](#supported-exports) +1. Export your data from Discord using [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/tree/master/.docs). Your export should be in JSON format, disable `format markdown` option and enable `download assets` and `reuse assets` options. If exporting using a CLI - use `--media --reuse-media --markdown false --format Json` command line options. Include threads and forum posts by adding `--include-threads All` option. More information in [How to export data from Discord to view it in DCEF?](#supported-exports) 2. Download the latest release from [releases page](https://github.com/slatinsky/DiscordChatExporter-frontend/releases) 3. Extract the archive 4. Move all your [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter) exports to `/exports/` folder ([supported exports](#supported-exports)). @@ -152,15 +152,27 @@ JSON exports are created using [DiscordChatExporter](https://github.com/Tyrrrz/D > - JSON export format (`--format Json`) is required > - Skipping markdown prerendering (`--markdown false`) is highly recommended, but not required > - Downloading assets (`--media --reuse-media`) is highly recommended, but not required +> - Including threads (`--include-threads All`) is recommended for a full server export -<details><summary><b>Partial export using GUI version of DiscordChatExporter</b></summary> +<details><summary><b>Incremental export using DiscordChatExporter-incrementalBackup wrapper</b></summary> + +I have created simple wrapper for DiscordChatExporter to simplify the process of incremental export for this frontend viewer. You declare what you want to export in a config file and the wrapper will automatically export only new messages since the last export. + +For more information and instructions how to use, see [slatinsky/DiscordChatExporter-incrementalBackup](https://github.com/slatinsky/DiscordChatExporter-incrementalBackup) + +</details> + +<details><summary><b>Full export using GUI version of DiscordChatExporter</b></summary> Make sure that export format is set to `JSON` and `Format markdown` is disabled. Optionally, you should also enable `Download assets`+`Reuse assets` option to download images, videos and other types of assets. +Recently `Show threads` option was added to the general settings of DiscordChatExporter - if you want to export threads. Note that fetching threads is slow and it may take a while to show them in the GUI. + +  -**NOTE**: You can't export threads and forums using GUI version of DCE. You need to use CLI version for that. +Note: Exporting using CLI version of DiscordChatExporter is actually easier if you want to mass export everything. </details> @@ -168,23 +180,11 @@ Make sure that export format is set to `JSON` and `Format markdown` is disabled. <details><summary><b>Full server export using CLI version of DiscordChatExporter</b></summary> -Export all accessible channels in a server (**without threads and forum posts**): +Export all accessible channels, threads and forum posts in a server: ```bash -DiscordChatExporter.Cli.exe exportguild --token DISCORD_TOKEN -g SERVER_ID --media --reuse-media --markdown false --format Json --output OUTPUT_FOLDER_PATH +DiscordChatExporter.Cli.exe exportguild --token DISCORD_TOKEN -g SERVER_ID --media --reuse-media --markdown false --format Json --include-threads All --output OUTPUT_FOLDER_PATH ``` -Export threads/forum posts (you can pass multiple thread/forum post ids): -```bash -DiscordChatExporter.Cli export --token DISCORD_TOKEN --media --reuse-media --markdown false --output OUTPUT_FOLDER_PATH --format Json --channel THREAD_ID_OR_FORUM_POST_ID_1 THREAD_ID_OR_FORUM_POST_ID_2 THREAD_ID_OR_FORUM_POST_ID_3 THREAD_ID_OR_FORUM_POST_ID_4 -``` - -Don't want to manually right click all threads/forums post and copy their ids? These helper scripts can help you: - -[Helper script to export archived threads in a channel](docs/Exporting-threads.md) - -[Helper script to export forum posts in a channel](docs/Exporting-threads.md) - - </details> @@ -313,6 +313,7 @@ And for other technologies used in this project - sveltekit, docker, nodejs, nvm - [Tyrrrz/DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter) - exporter for your Discord chats - [Roachbones/discordless](https://github.com/Roachbones/discordless) - real time man-in-the-middle exporter - [mlomb/chat-analytics](https://github.com/mlomb/chat-analytics) - analytics for your Discord chats +- [slatinsky/DiscordChatExporter-incrementalBackup](https://github.com/slatinsky/DiscordChatExporter-incrementalBackup) - incrementally export your Discord chats for DiscordChatExporter-frontend ## License GNU GENERAL PUBLIC LICENSE. See [LICENSE](LICENSE) for more details. diff --git a/docs/Exporting-forum-posts.md b/docs/Exporting-forum-posts.md deleted file mode 100644 index 933e8773..00000000 --- a/docs/Exporting-forum-posts.md +++ /dev/null @@ -1,48 +0,0 @@ -# Helper script to export forum posts in a channel - -Viewing forums is supported by this viewer, but exporting them manually is time consuming. You can use this helper script to generate command to download all forum posts in a forum channel automatically: - -### Steps -1. Open discord in browser (browser needs to be Chromium based - Chrome, Edge, Opera, Brave, Vivaldi, etc., not working in Firefox) -2. Navigate to channel with forum post list -3. press F12 and paste this script to the console: - -```js -len = 0 -ids = [] -previouseScrollTop = 0 - -function scrollToPosition(offset) { - scrollDiv = document.querySelector('div[class*="chat-"] > div > div > div[class*="scrollerBase-"]') - scrollDiv.scroll(0, offset) -} - -function captureIds() { - document.querySelectorAll('div[data-item-id]').forEach((e) => ids.push(e.dataset.itemId)) - ids = [...new Set(ids)] - if (ids.length > len) { - len = ids.length - console.log('Found', len, 'IDs') - } -} - -function printIds() { - console.log('DiscordChatExporter.Cli.exe export --token TOKEN --output "exports/forums" --format Json --media --reuse-media --markdown false --channel',ids.join(' ')) -} - -scrollToPosition(0) -interval = setInterval(() => { - scrollToPosition(scrollDiv.scrollTop + window.innerHeight / 3) - setTimeout(() => { - captureIds() - if (previouseScrollTop === scrollDiv.scrollTop) { - clearInterval(interval) - printIds() - } - previouseScrollTop = scrollDiv.scrollTop - }, 1000) -}, 1542) -``` - -4. script will scroll the page. At the the end, it will print command to the console, which allows you to export all forum posts in the forum channel -5. edit command printed in the console (`--format`, `--output` and `--token`) and export with CLI version of DiscordChatExporter \ No newline at end of file diff --git a/docs/Exporting-threads.md b/docs/Exporting-threads.md deleted file mode 100644 index 9cdc2e7e..00000000 --- a/docs/Exporting-threads.md +++ /dev/null @@ -1,106 +0,0 @@ -# Helper script to export forum posts in a channel - -Viewing threads is supported by this viewer, but exporting them manually is time consuming. You can use this helper script to generate command to download all archived threads in a channel automatically: - -### Steps -1. Open discord in browser (browser needs to be Chromium based - Chrome, Edge, Opera, Brave, Vivaldi, etc., not working in Firefox) -2. Navigate to channel with threads. Do not open thread list (if you opened it, refresh the page) -3. press F12 and paste this script to the console: - -```js -len = 0 -ids = [] -previouseScrollTop = 0 - -// interceptor https://stackoverflow.com/a/66564476 -if (window.oldXHROpen === undefined) { - window.oldXHROpen = window.XMLHttpRequest.prototype.open; - window.XMLHttpRequest.prototype.open = function(method, url, async, user, password) { - this.addEventListener('load', function() { - try { - var json = JSON.parse(this.responseText); - if (json.hasOwnProperty('threads')) { - // get ids - for (const thread of json.threads) { - ids.push(thread.id) - } - } - } - catch (e) { - console.log(e) - } - }); - return window.oldXHROpen.apply(this, arguments); - } -} -else { - console.log('OK, interceptor already exists') -} - - -function getSelector(name) { - if (name === 'thread-icon') - return document.querySelectorAll('div[class*="toolbar-"] > div[aria-label="Threads"]') - if (name === 'scroller-base') - return document.querySelectorAll('div[id*="popout_"] > div > div > div[class*="scrollerBase-"]') -} - -function clickThreadsIcon() { - if (getSelector('scroller-base').length > 0) { - console.log('OK, found scroller base (1)') - mainFunc() - } - else if (getSelector('scroller-base').length == 0 && getSelector('thread-icon').length > 0) { - getSelector('thread-icon')[0].click() - console.log('OK, clicked Threads Icon') - setTimeout(() => { - if (getSelector('scroller-base').length == 0) { - throw new Error('ERROR, could not find scroller-base') - } - else { - console.log('OK, found scroller base (2)') - mainFunc() - } - }, 1000) - } - else { - throw new Error('ERROR, could not find threads icon') - } -} -clickThreadsIcon() - -function scrollToPosition(offset) { - scrollDiv = getSelector('scroller-base')[0] - scrollDiv.scroll(0, offset) -} - -function captureIds() { - ids = [...new Set(ids)] - if (ids.length > len) { - len = ids.length - console.log('Found', len, 'IDs') - } -} - -function printIds() { - console.log('DiscordChatExporter.Cli.exe export --token TOKEN --output "exports/threads" --format Json --media --reuse-media --markdown false --channel',ids.join(' ')) -} - -function mainFunc() { - scrollToPosition(0) - interval = setInterval(() => { - scrollToPosition(scrollDiv.scrollTop + window.innerHeight / 3) - setTimeout(() => { - captureIds() - if (previouseScrollTop === scrollDiv.scrollTop) { - clearInterval(interval) - printIds() - } - previouseScrollTop = scrollDiv.scrollTop - }, 500) - }, 742) -} -``` - -4. script will scroll thread list. At the the end, it will print command to the console, which allows you to export all archived threads in the channel -5. edit command printed in the console (`--format`, `--output` and `--token`) and export with CLI version of DiscordChatExporter \ No newline at end of file