Skip to content

Commit

Permalink
add support for colornote
Browse files Browse the repository at this point in the history
  • Loading branch information
marph91 committed Oct 14, 2024
1 parent ef9594f commit 5f11c18
Show file tree
Hide file tree
Showing 11 changed files with 135 additions and 2 deletions.
1 change: 0 additions & 1 deletion docs/contributing/more_note_apps.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ Loose collection of note apps/messengers/wikis/formats that could be implemented
| [Calibre](https://calibre-ebook.com/) | - [doc](https://manual.calibre-ebook.com/en/faq.html#how-do-i-move-my-calibre-data-from-one-computer-to-another) <br>- [script](https://github.com/Mick2nd/Calibre-Import) | |
| [Capacities](https://capacities.io/) | [doc](https://docs.capacities.io/reference/export#export) | |
| [CintaNotes](http://cintanotes.com/) | [doc](http://cintanotes.com/help/#export) (text export) | |
| [Colornote](https://www.colornote.com/) | [script](https://github.com/shervinemami/colornote_to_joplin) | |
| [Confluence](https://www.atlassian.com/software/confluence) | - [doc](https://support.atlassian.com/confluence-cloud/docs/export-content-to-word-pdf-html-and-xml/#Export-a-space) <br>- [script](https://github.com/KkEi34/confluence-to-obsidian-plugin) | |
| [Clickup](https://clickup.com/) | - [doc](https://help.clickup.com/hc/en-us/articles/6310551109527-Task-data-export) <br>- [script](https://github.com/jordanbates/clickup_to_joplin) | project management |
| [Craft](https://www.craft.do/) | - [doc](https://support.craft.do/hc/en-us/articles/4418134683665-Exporting-documents-from-Craft) <br>- [script](https://github.com/coofdy/craft-to-obsidian) | |
Expand Down
13 changes: 13 additions & 0 deletions docs/formats/colornote.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
This page describes how to convert notes from ColorNote to Markdown.

## General Information

- [Website](https://www.colornote.com/)
- Typical extension: `.backup`

## Instructions

1. Export as described [at the website](https://www.colornote.com/faq-question/what-is-device-backup/)
2. [Install jimmy](../index.md#installation)
3. Convert to Markdown. Example: `jimmy-cli-linux colornote-20241013.backup --format colornote`
4. [Import to your app](../import_instructions.md)
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Export data from your app and convert it to Markdown. For details, click on the
||||||
| :---: | :---: | :---: | :---: | :---: |
| <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/3/3d/Anki-icon.svg/240px-Anki-icon.svg.png" style="height:100px;max-width:100px;"><br>[Anki](https://marph91.github.io/jimmy/formats/anki/) | <img src="https://bear.app/images/logo.png" style="height:100px;max-width:100px;"><br>[Bear](https://marph91.github.io/jimmy/formats/bear/) | <img src="https://raw.githubusercontent.com/CacherApp/cacher-cli/e241f06867dba740131db5314ef7fe279135baf6/images/cacher-icon.png" style="height:100px;max-width:100px;"><br>[Cacher](https://marph91.github.io/jimmy/formats/cacher/) | <img src="https://raw.githubusercontent.com/giuspen/cherrytree/c822b16681b002b8882645d8d1e8f109514ddb58/icons/cherrytree.svg" style="height:100px;max-width:100px;"><br>[CherryTree](https://marph91.github.io/jimmy/formats/cherrytree/) | <img src="https://avatars.githubusercontent.com/u/53916365?s=200&v=4" style="height:100px;max-width:100px;"><br>[Clipto](https://marph91.github.io/jimmy/formats/clipto/) |
| <img src="https://www.colornote.com/wp-content/uploads/2016/05/cropped-favicon.png" style="height:100px;max-width:100px;"><br>[ColorNote](https://marph91.github.io/jimmy/formats/colornote/) | | | | |
| <img src="https://seeklogo.com/images/D/day-one-logo-F4CA245C26-seeklogo.com.png" style="height:100px;max-width:100px;"><br>[Day&nbsp;One](https://marph91.github.io/jimmy/formats/day_one/) | <img src="https://images.saasworthy.com/dynalist_5288_logo_1576239391_xhkcg.jpg" style="height:100px;max-width:100px;"><br>[Dynalist](https://marph91.github.io/jimmy/formats/dynalist/) | <img src="https://upload.wikimedia.org/wikipedia/commons/b/b8/2021_Facebook_icon.svg" style="height:100px;max-width:100px;"><br>[Facebook](https://marph91.github.io/jimmy/formats/facebook/) | <img src="https://wavebox.pro/store2/store/0b46bf0a-107c-4fa2-a657-3df7412e3d3d.png" style="height:100px;max-width:100px;"><br>[FuseBase, Nimbus&nbsp;Note](https://marph91.github.io/jimmy/formats/fusebase/) | <img src="https://www.gstatic.com/images/branding/product/1x/docs_2020q4_96dp.png" style="height:100px;max-width:100px;"><br>[Google&nbsp;Docs](https://marph91.github.io/jimmy/formats/google_docs/) |
| <img src="https://www.gstatic.com/images/branding/product/1x/keep_2020q4_96dp.png" style="height:100px;max-width:100px;"><br>[Google&nbsp;Keep](https://marph91.github.io/jimmy/formats/google_keep/) | <img src="https://github.com/laurent22/joplin/blob/dev/Assets/LinuxIcons/128x128.png?raw=true" style="height:100px;max-width:100px;"><br>[Joplin](https://marph91.github.io/jimmy/formats/joplin/) | <img src="https://raw.githubusercontent.com/jrnl-org/jrnl/85a98afcd91ed873c0eceba9893c3ec424f201b8/docs_theme/img/logo.svg" style="height:100px;max-width:100px;"><br>[jrnl](https://marph91.github.io/jimmy/formats/jrnl/) | <img src="https://upload.wikimedia.org/wikipedia/commons/4/45/Notion_app_logo.png" style="height:100px;max-width:100px;"><br>[Notion](https://marph91.github.io/jimmy/formats/notion/) | <img src="https://upload.wikimedia.org/wikipedia/commons/1/10/2023_Obsidian_logo.svg" style="height:100px;max-width:100px;"><br>[Obsidian](https://marph91.github.io/jimmy/formats/obsidian/) |
| <img src="https://raw.githubusercontent.com/pbek/QOwnNotes/d89a597a28eeb16f57692ac121933b478f44bf07/src/images/icons/256x256/apps/QOwnNotes.png" style="height:100px;max-width:100px;"><br>[QOwnNotes](https://marph91.github.io/jimmy/formats/qownnotes/) | <img src="https://raw.githubusercontent.com/jendrikseipp/rednotebook/b2cefe5f321b21ab7ad855059f3c0496eb0830d2/rednotebook/images/rednotebook-icon/rn-256.png" style="height:100px;max-width:100px;"><br>[RedNotebook](https://marph91.github.io/jimmy/formats/rednotebook/) | <img src="https://raw.githubusercontent.com/Automattic/simplenote-electron/4a140a96545763c849b26a81a2e27ff67eaa68f0/lib/icons/app-icon/icon_256x256.png" style="height:100px;max-width:100px;"><br>[Simplenote](https://marph91.github.io/jimmy/formats/simplenote/) | <img src="https://avatars.githubusercontent.com/u/24537496?s=100" style="height:100px;max-width:100px;"><br>[Standard&nbsp;Notes](https://marph91.github.io/jimmy/formats/standard_notes/) | <img src="https://www.synology.com/img/dsm/note_station/notestation_72.png" style="height:100px;max-width:100px;"><br>[Synology Note&nbsp;Station](https://marph91.github.io/jimmy/formats/synology_note_station/) |
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ nav:
- Cacher: formats/cacher.md
- CherryTree: formats/cherrytree.md
- Clipto: formats/clipto.md
- ColorNote: formats/colornote.md
- Day One: formats/day_one.md
- Dynalist: formats/dynalist.md
- Facebook: formats/facebook.md
Expand Down
1 change: 1 addition & 0 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Export data from your app and convert it to Markdown. For details, click on the
||||||
| :---: | :---: | :---: | :---: | :---: |
| <img src="https://upload.wikimedia.org/wikipedia/commons/thumb/3/3d/Anki-icon.svg/240px-Anki-icon.svg.png" style="height:100px;max-width:100px;"><br>[Anki](https://marph91.github.io/jimmy/formats/anki/) | <img src="https://bear.app/images/logo.png" style="height:100px;max-width:100px;"><br>[Bear](https://marph91.github.io/jimmy/formats/bear/) | <img src="https://raw.githubusercontent.com/CacherApp/cacher-cli/e241f06867dba740131db5314ef7fe279135baf6/images/cacher-icon.png" style="height:100px;max-width:100px;"><br>[Cacher](https://marph91.github.io/jimmy/formats/cacher/) | <img src="https://raw.githubusercontent.com/giuspen/cherrytree/c822b16681b002b8882645d8d1e8f109514ddb58/icons/cherrytree.svg" style="height:100px;max-width:100px;"><br>[CherryTree](https://marph91.github.io/jimmy/formats/cherrytree/) | <img src="https://avatars.githubusercontent.com/u/53916365?s=200&v=4" style="height:100px;max-width:100px;"><br>[Clipto](https://marph91.github.io/jimmy/formats/clipto/) |
| <img src="https://www.colornote.com/wp-content/uploads/2016/05/cropped-favicon.png" style="height:100px;max-width:100px;"><br>[ColorNote](https://marph91.github.io/jimmy/formats/colornote/) | | | | |
| <img src="https://seeklogo.com/images/D/day-one-logo-F4CA245C26-seeklogo.com.png" style="height:100px;max-width:100px;"><br>[Day&nbsp;One](https://marph91.github.io/jimmy/formats/day_one/) | <img src="https://images.saasworthy.com/dynalist_5288_logo_1576239391_xhkcg.jpg" style="height:100px;max-width:100px;"><br>[Dynalist](https://marph91.github.io/jimmy/formats/dynalist/) | <img src="https://upload.wikimedia.org/wikipedia/commons/b/b8/2021_Facebook_icon.svg" style="height:100px;max-width:100px;"><br>[Facebook](https://marph91.github.io/jimmy/formats/facebook/) | <img src="https://wavebox.pro/store2/store/0b46bf0a-107c-4fa2-a657-3df7412e3d3d.png" style="height:100px;max-width:100px;"><br>[FuseBase, Nimbus&nbsp;Note](https://marph91.github.io/jimmy/formats/fusebase/) | <img src="https://www.gstatic.com/images/branding/product/1x/docs_2020q4_96dp.png" style="height:100px;max-width:100px;"><br>[Google&nbsp;Docs](https://marph91.github.io/jimmy/formats/google_docs/) |
| <img src="https://www.gstatic.com/images/branding/product/1x/keep_2020q4_96dp.png" style="height:100px;max-width:100px;"><br>[Google&nbsp;Keep](https://marph91.github.io/jimmy/formats/google_keep/) | <img src="https://github.com/laurent22/joplin/blob/dev/Assets/LinuxIcons/128x128.png?raw=true" style="height:100px;max-width:100px;"><br>[Joplin](https://marph91.github.io/jimmy/formats/joplin/) | <img src="https://raw.githubusercontent.com/jrnl-org/jrnl/85a98afcd91ed873c0eceba9893c3ec424f201b8/docs_theme/img/logo.svg" style="height:100px;max-width:100px;"><br>[jrnl](https://marph91.github.io/jimmy/formats/jrnl/) | <img src="https://upload.wikimedia.org/wikipedia/commons/4/45/Notion_app_logo.png" style="height:100px;max-width:100px;"><br>[Notion](https://marph91.github.io/jimmy/formats/notion/) | <img src="https://upload.wikimedia.org/wikipedia/commons/1/10/2023_Obsidian_logo.svg" style="height:100px;max-width:100px;"><br>[Obsidian](https://marph91.github.io/jimmy/formats/obsidian/) |
| <img src="https://raw.githubusercontent.com/pbek/QOwnNotes/d89a597a28eeb16f57692ac121933b478f44bf07/src/images/icons/256x256/apps/QOwnNotes.png" style="height:100px;max-width:100px;"><br>[QOwnNotes](https://marph91.github.io/jimmy/formats/qownnotes/) | <img src="https://raw.githubusercontent.com/jendrikseipp/rednotebook/b2cefe5f321b21ab7ad855059f3c0496eb0830d2/rednotebook/images/rednotebook-icon/rn-256.png" style="height:100px;max-width:100px;"><br>[RedNotebook](https://marph91.github.io/jimmy/formats/rednotebook/) | <img src="https://raw.githubusercontent.com/Automattic/simplenote-electron/4a140a96545763c849b26a81a2e27ff67eaa68f0/lib/icons/app-icon/icon_256x256.png" style="height:100px;max-width:100px;"><br>[Simplenote](https://marph91.github.io/jimmy/formats/simplenote/) | <img src="https://avatars.githubusercontent.com/u/24537496?s=100" style="height:100px;max-width:100px;"><br>[Standard&nbsp;Notes](https://marph91.github.io/jimmy/formats/standard_notes/) | <img src="https://www.synology.com/img/dsm/note_station/notestation_72.png" style="height:100px;max-width:100px;"><br>[Synology Note&nbsp;Station](https://marph91.github.io/jimmy/formats/synology_note_station/) |
Expand Down
1 change: 1 addition & 0 deletions requirements/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ enlighten==1.12.4
markdown==3.7
platformdirs==4.3.3
puremagic==1.27
pycryptodomex==3.21.0
pypandoc==1.13
python-frontmatter==1.1.0
pytodotxt==1.5.0
Expand Down
84 changes: 84 additions & 0 deletions src/formats/colornote.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
"""Convert ColorNote notes to the intermediate format."""

import hashlib
import io
import json
from pathlib import Path
import struct

from Cryptodome.Cipher import AES
from Cryptodome.Util.Padding import unpad

import common
import converter
import intermediate_format as imf
import markdown_lib.colornote
import markdown_lib.common


def decrypt(salt: bytes, password: bytes, ciphertext: bytes) -> bytes:
# decrypting is based on:
# https://github.com/olejorgenb/ColorNote-backup-decryptor/blob/61e105d6f13b2cd22b5141b6334bb098617665e1/src/ColorNoteBackupDecrypt.java
key = hashlib.md5(password + salt).digest()
iv = hashlib.md5(key + password + salt).digest()
decoder = AES.new(key, AES.MODE_CBC, iv)
return unpad(decoder.decrypt(ciphertext), 16)


class Converter(converter.BaseConverter):
accepted_extensions = [".backup"]

def __init__(self, config):
super().__init__(config)
self.password = config.password

def handle_wikilink_links(self, body: str) -> list[imf.NoteLink]:
# only internal links
# https://www.colornote.com/faq-question/how-can-i-link-a-note-with-another-note/
note_links = []
for _, url, description in markdown_lib.common.get_wikilink_links(body):
note_links.append(imf.NoteLink(f"[[{url}]]", url, description or url))
return note_links

def convert(self, file_or_folder: Path):
ciphertext = file_or_folder.read_bytes()
# TODO: Meaning of ciphertext[:28]?
plaintext = decrypt(
b"ColorNote Fixed Salt", self.password.encode("utf-8"), ciphertext[28:]
)

# TODO: Meaning of plaintext[:16]? Looks similar to the iv.
plaintext_stream = io.BytesIO(plaintext)
plaintext_stream.read(16)
while chunk_length_bytes := plaintext_stream.read(4):
# parse binary colornote format
# 4 bytes: chunk length
# chunk length bytes: json data
(chunk_length,) = struct.unpack(">L", chunk_length_bytes)
chunk_bytes = plaintext_stream.read(chunk_length)
note_json = json.loads(chunk_bytes.decode("utf-8"))

# actual conversion
# TODO: reminder, tags, ...
title = note_json["title"]
if title == "syncable_settings":
# TODO: needed?
# syncable_settings = json.loads(note_json["note"])
# print(syncable_settings)
continue
self.logger.debug(f'Converting note "{title}"')
note_imf = imf.Note(
title,
markdown_lib.colornote.colornote_to_md(note_json["note"]),
created=common.timestamp_to_datetime(note_json["created_date"] / 1000),
updated=common.timestamp_to_datetime(note_json["modified_date"] / 1000),
source_application=self.format,
original_id=title, # not "uuid", because the title is linked
latitude=note_json["latitude"],
longitude=note_json["longitude"],
)
if note_json["space"] == 16:
note_imf.tags.append(imf.Tag("colornote-archived"))
note_imf.note_links = self.handle_wikilink_links(note_imf.body)

self.root_notebook.child_notes.append(note_imf)
30 changes: 30 additions & 0 deletions src/markdown_lib/colornote.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""Convert ColorNote markup to Markdown."""

import re

import pyparsing as pp


list_re = re.compile(r"^(\[[ V]\] )", re.MULTILINE)


def list_():
def to_md(_, t): # noqa
match = t[0][0]
list_character = {"[ ] ": "- [ ] ", "[V] ": "- [x] "}[match]
return list_character

return pp.Regex(list_re, as_group_list=True).set_parse_action(to_md)


def colornote_to_md(body: str) -> str:
r"""
Main ColorNote markup to Markdown conversion function.
>>> colornote_to_md("[V] A\n[V] B")
'- [x] A\n- [x] B'
>>> colornote_to_md("[ ] Item 1\n[ ] Item 2\n[ ] Item 3")
'- [ ] Item 1\n- [ ] Item 2\n- [ ] Item 3a'
"""
markup = list_()
return markup.transform_string(body)
1 change: 1 addition & 0 deletions test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ set -e # fail if any command fails

export PYTHONPATH="$PYTHONPATH:src"
python -m unittest discover -v --buffer
# TODO: https://stackoverflow.com/questions/43463273/python-m-doctest-ignores-files-with-same-names-in-different-directories
python -m doctest src/*.py src/**/*.py
2 changes: 2 additions & 0 deletions test/test_convert.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ def setUp(self):
# https://stackoverflow.com/a/51197422/7410886
self.config = SimpleNamespace(
format=None,
password="1234", # TODO: only used at colornote for now
frontmatter=None,
global_resource_folder=None,
local_resource_folder=Path("."),
Expand Down Expand Up @@ -83,6 +84,7 @@ def compare_dirs(dir1: Path, dir2: Path):
[["bear/test_1/backup.bear2bk"]],
[["bear/test_2/backup-2.bear2bk"]],
[["cacher/test_1/cacher-export-202406182304.json"]],
[["colornote/test_1/colornote-20241014.backup"]],
[["cherrytree/test_1/cherry.ctb.ctd"]],
[["cherrytree/test_2/cherrytree_manual.ctd"]],
[["cherrytree/test_3/lab_report.ctd"]],
Expand Down

0 comments on commit 5f11c18

Please sign in to comment.