Skip to content
This repository has been archived by the owner on Jun 15, 2024. It is now read-only.

Commit

Permalink
refactor: rename notebook to notetree
Browse files Browse the repository at this point in the history
refactor: rename notebook to notetree
  • Loading branch information
doomspec committed Oct 9, 2023
1 parent 0dff970 commit 0410efc
Show file tree
Hide file tree
Showing 42 changed files with 315 additions and 308 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/analyze.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: build project tree📦
run: |
mkdir output
python -c "import evonote, json;from evonote.gui.notebook import get_json_for_treemap;from evonote.transform.module_to_notebook import get_notebook_for_module;notebook = get_notebook_for_module(evonote);tree = get_json_for_treemap(notebook.root);f = open('output/project_tree.json', 'w');json.dump(tree, f)"
python ./.github/workflows/gen_project_tree.py
- name: Deploy to image-data branch
uses: peaceiris/actions-gh-pages@v3
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/gen_project_tree.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import evonote, json
from evonote.gui.notetree import get_json_for_treemap
from evonote.transform.module_to_notetree import get_notetree_for_module
notetree = get_notetree_for_module(evonote)
tree = get_json_for_treemap(notetree.root)
f = open('output/project_tree.json', 'w');json.dump(tree, f)
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Typically, these tasks requires

## Why EvoNote

EvoNote is build for holding and operating knowledge bases (notebooks). EvoNote offers
EvoNote is build for holding and operating knowledge in the form of `note trees`. EvoNote offers

- A tree-based knowledge base framework
- Convenient tools for
Expand Down
2 changes: 1 addition & 1 deletion docs/development/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Knowledge storage
`Note`: The node of knowledge. It only contains the knowledge itself.

`Notebook`: The collection of the references to `Note` objects. It contains **the relationship among knowledge**. It includes parents, children and path in the tree structure. It also contains the indexings of the notes.
`Tree`: The collection of the references to `Note` objects. It contains **the relationship among knowledge**. It includes parents, children and path in the tree structure. It also contains the indexings of the notes.

## Indexing
`Indexing`: The class for storing the data of one indexing. It can return related notes when provided with queries. It must be interpreted by the `Indexer` class.
Expand Down
6 changes: 3 additions & 3 deletions evonote/agent/general.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
from evonote.notebook.notebook import Notebook
from evonote.notetree import Tree


class AgentState:
def __init__(self):
self.root_notebook = Notebook(
"root notebook that indexes all available notebooks")
self.root_notetree = Tree(
"root notetree that indexes all available notetrees")
self.objective_stack = []
self.logs = []
8 changes: 4 additions & 4 deletions evonote/agent/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ Satisfied?

Atomic operations:

1. Search in notebook
1. Search in notetree
2. Execute action in the retrieved note
3. Ask (and update in-prompt memory and notebook) (I doubt this can be removed from atomic)
3. Ask (and update in-prompt memory and notetree) (I doubt this can be removed from atomic)
4. Memo (update in-prompt memory)
5. Planing: Make a complex objective into atomic operations

Operation arguments:

1. Search in notebook: notebook, indexing, query, query_type: similarity | question
1. Search in notetree: notetree, indexing, query, query_type: similarity | question
2. Execute action in the retrieved note: note, action
3. Ask: question
4. Memo: content
Expand All @@ -44,7 +44,7 @@ Planning:

Search:

1. The search should be done with a special searching agent. The input is the searching objective. A special notebook
1. The search should be done with a special searching agent. The input is the searching objective. A special notetree
for how to search might be used.
2. The searching agent will break down the objective into further sub-objectives. The sub-objectives are finer and more
specific than the original objective.
Expand Down
15 changes: 7 additions & 8 deletions evonote/gui/notebook.py → evonote/gui/notetree.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@
from evonote.gui.utlis import hypenate_texts

if TYPE_CHECKING:
from evonote.notebook.note import Note
from evonote.notebook.notebook import Notebook
from evonote.notetree import Note, Tree

h_en = Hyphenator('en_US')

Expand Down Expand Up @@ -53,12 +52,12 @@ def get_json_for_treemap(root: Note):


def prepare_tree_parameters(root):
notebook = root.notebook
notetree = root.notetree
labels = []
parents = []
texts = []
ids = []
add_note_to_list(labels, parents, texts, ids, root, notebook)
add_note_to_list(labels, parents, texts, ids, root, notetree)
line_width = 40
for i in range(len(texts)):
if len(texts[i].strip()) == 0:
Expand All @@ -68,15 +67,15 @@ def prepare_tree_parameters(root):
return ids, labels, parents, texts


def add_note_to_list(labels, parents, values, ids, note: Note, notebook: Notebook):
def add_note_to_list(labels, parents, values, ids, note: Note, notetree: Tree):
i = 1
children = notebook.get_children_dict(note)
children = notetree.get_children_dict(note)
for key, child in children.items():
notepath = notebook.get_note_path(child)
notepath = notetree.get_note_path(child)
label = str(i) + ". " + key if len(children) > 1 else key
labels.append(label)
parents.append("/".join(notepath[:-1]))
values.append(child.content)
ids.append("/".join(notepath))
add_note_to_list(labels, parents, values, ids, child, notebook)
add_note_to_list(labels, parents, values, ids, child, notetree)
i += 1
2 changes: 1 addition & 1 deletion evonote/indexing/code_indexer.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from typing import List

from evonote.indexing.core import AbsEmbeddingIndexer, Indexing
from evonote.notebook.note import Note
from evonote.notetree import Note


class CodeParameterIndexer(AbsEmbeddingIndexer):
Expand Down
18 changes: 9 additions & 9 deletions evonote/indexing/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
from evonote.model.chat import Chat

if TYPE_CHECKING:
from evonote.notebook.note import Note
from evonote.notebook.notebook import Notebook
from evonote.notetree import Note
from evonote.notetree import Tree

from evonote.file_helper.cache_manage import save_cache, cached_function
from evonote.model.openai import get_embeddings
Expand Down Expand Up @@ -41,11 +41,11 @@ def remove_note(cls, note: Note):

class Indexing:
def __init__(self, notes: List[Note], indexer: Type[Indexer],
notebook: Notebook):
notetree: Tree):
self.notes_without_indexer: List[Note] = notes[:]
self.indexer: Type[Indexer] = indexer
self.data: Any = None
self.notebook = notebook
self.notetree = notetree

def add_new_note(self, note: Note):
self.notes_without_indexer.append(note)
Expand Down Expand Up @@ -223,7 +223,7 @@ class FragmentedEmbeddingIndexer(AbsEmbeddingIndexer):
def process_note_with_content(cls, notes: List[Note], indexing: Indexing,
):
notes_content = [note.content for note in notes]
notebook = indexing.notebook
notetree = indexing.notetree

new_src_list = []
new_weights = []
Expand All @@ -233,9 +233,9 @@ def process_note_with_content(cls, notes: List[Note], indexing: Indexing,
executor.map(process_sent_into_frags, notes_content)):
new_src = []
new_src.extend(frags)
note_path = notebook.get_note_path(note)
note_path = notetree.get_note_path(note)
if len(note_path) > 0:
new_src.append(notebook.get_note_path(note)[-1])
new_src.append(notetree.get_note_path(note)[-1])
new_src.append(note.content)

new_src_list.append(new_src)
Expand Down Expand Up @@ -274,13 +274,13 @@ def process_note_without_content(cls, notes: List[Note], indexing: Indexing,
def prepare_src_weight_list(cls, new_notes: List[Note], indexing: Indexing,
):

notebook = indexing.notebook
notetree = indexing.notetree
notes_with_content = []
notes_content = []
notes_without_content = []
for note in new_notes:
if len(note.content) == 0:
keywords_on_path = notebook.get_note_path(note)
keywords_on_path = notetree.get_note_path(note)
if len(keywords_on_path) != 0:
notes_without_content.append(note)
continue
Expand Down
21 changes: 0 additions & 21 deletions evonote/notebook/bookshelf.py

This file was deleted.

2 changes: 2 additions & 0 deletions evonote/notetree/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
from .tree import Tree
from .note import Note
12 changes: 6 additions & 6 deletions evonote/notebook/analysis.py → evonote/notetree/analysis.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
from evonote.notebook.notebook import Notebook
from evonote.notetree import Tree


def analyze_notebook_sparsity(notebook: Notebook):
def analyze_notetree_sparsity(notetree: Tree):
"""
Return the average number of children per note.
"""
notes = notebook.get_note_list()
notes = notetree.get_note_list()
max_children = -1
total_children = 0
n_non_leaf_notes = 0
Expand All @@ -16,7 +16,7 @@ def analyze_notebook_sparsity(notebook: Notebook):
total_children += n_children
max_children = max(max_children, n_children)
average_children = total_children / n_non_leaf_notes
heavy_notes = get_children_heavy_notes(notebook)
heavy_notes = get_children_heavy_notes(notetree)
print("Total_notes:", len(notes))
print("Average children per non-leaf note:", average_children)
print("Max children per note:", max_children)
Expand All @@ -27,11 +27,11 @@ def analyze_notebook_sparsity(notebook: Notebook):
print("Content:", note.content)
print("")

def get_children_heavy_notes(notebook: Notebook, min_children=10):
def get_children_heavy_notes(notetree: Tree, min_children=10):
"""
Return a list of notes with at least min_children children.
"""
notes = notebook.get_note_list()
notes = notetree.get_note_list()
heavy_notes = []
for note in notes:
n_children = len(note.children())
Expand Down
21 changes: 21 additions & 0 deletions evonote/notetree/bookshelf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
from typing import List

from evonote.notetree import Note
from evonote.notetree import Tree


class Bookshelf(Tree):
def __init__(self, root_content):
super().__init__(root_content)

def add_notetree_by_path(self, path: List[str], notetree: Tree):
note = get_note_for_notetree(notetree, self)
self.add_note_by_path(path, note)
return note


def get_note_for_notetree(notetree: Tree, default_notetree: Tree) -> Note:
note = Note(default_notetree)
note.resource.add_notetree(notetree, notetree.topic)
note.be("A notetree with root_content: " + notetree.topic)
return note
42 changes: 21 additions & 21 deletions evonote/notebook/note.py → evonote/notetree/note.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@
from typing import TYPE_CHECKING, Dict

if TYPE_CHECKING:
from evonote.notebook.notebook import Notebook
from evonote.notetree import Tree


class Note:
"""
A tree-like data structure that stores notebook
A tree-like data structure that stores notetree
usually for the direct summary of paragraphs
The relation of the items are mainly represented by the tree structure
Expand All @@ -18,18 +18,18 @@ class Note:
Notice that Note object can be indexed by embedding vectors because its
"""

def __init__(self, notebook: Notebook):
def __init__(self, notetree: Tree):
super().__init__()

# content is string no matter what _content_type is
self.content: str = ""
# The root note helps merge two notebook bases
self.notebook: Notebook = notebook
# The root note helps merge two notetree bases
self.notetree: Tree = notetree
# The resource is the data that is indicated by the note
self.resource: NoteResource = NoteResource()

def copy_to(self, notebook: Notebook):
new_note = Note(notebook)
def copy_to(self, notetree: Tree):
new_note = Note(notetree)
new_note.content = copy(self.content)
new_note.resource = copy(self.resource)
return new_note
Expand All @@ -39,13 +39,13 @@ def copy_to(self, notebook: Notebook):
"""

def note_path(self):
return self.notebook.get_note_path(self)
return self.notetree.get_note_path(self)

def parent(self):
return self.notebook.get_parent(self)
return self.notetree.get_parent(self)

def children(self) -> Dict[str, Note]:
return self.notebook.get_children_dict(self)
return self.notetree.get_children_dict(self)

def title(self) -> str:
note_path = self.note_path()
Expand All @@ -54,19 +54,19 @@ def title(self) -> str:
return note_path[-1]

def has_child(self, key: str):
return self.notebook.has_child(self, key)
return self.notetree.has_child(self, key)

"""
## Functions for adding children of note
"""

def add_child(self, key: str, note) -> Note:
self.notebook.add_child(key, self, note)
self.notetree.add_child(key, self, note)
return note

def new_child(self, key: str) -> Note:
note = Note(self.notebook)
self.notebook.add_child(key, self, note)
note = Note(self.notetree)
self.notetree.add_child(key, self, note)
return note

"""
Expand All @@ -79,12 +79,12 @@ def s(self, key) -> Note:
:param key: the key of the child note
:return:
"""
notebook = self.notebook
notetree = self.notetree
if isinstance(key, int) or isinstance(key, str):
children = notebook.get_children_dict(self)
children = notetree.get_children_dict(self)
if key not in children:
note = Note(notebook)
notebook.add_child(key, self, note)
note = Note(notetree)
notetree.add_child(key, self, note)
return note
return children[key]
else:
Expand Down Expand Up @@ -116,7 +116,7 @@ def __repr__(self):
class NoteResource:
def __init__(self):
self.resource = {}
# Possible types: Notebook, Note, Function, Class, Module
# Possible types: Tree, Note, Function, Class, Module
self.resource_type = {}

def has_type(self, resource_type):
Expand Down Expand Up @@ -158,8 +158,8 @@ def add_resource(self, resource, resource_type: str, key: str):
def add_text(self, text, key: str):
self.add_resource(text, "text", key)

def add_notebook(self, notebook, key: str):
self.add_resource(notebook, "notebook", key)
def add_notetree(self, notetree, key: str):
self.add_resource(notetree, "notetree", key)

def add_function(self, function, key: str):
self.add_resource(function, "function", key)
Expand Down
Loading

0 comments on commit 0410efc

Please sign in to comment.