Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove usage of fastbook #38

Open
5 of 12 tasks
jfthuong opened this issue Oct 11, 2021 · 0 comments
Open
5 of 12 tasks

Remove usage of fastbook #38

jfthuong opened this issue Oct 11, 2021 · 0 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@jfthuong
Copy link
Contributor

jfthuong commented Oct 11, 2021

Is your feature request related to a problem? Please describe.
Fastbook is a bit overkill for very few features and installation is sometimes buggy on some platforms (e.g. Kaggle AFAIK)

Describe the solution you'd like
Moving similar features to unpackai package

  • Setting contexts (see below) => could be in unpackai/__init__.py
from fastai.vision.all import *
# from fastai.torch_core import set_seed
# import torch

mpl.rcParams['savefig.dpi']= 200
mpl.rcParams['font.size']=12

set_seed(42)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
pd.set_option('display.max_columns',999)
np.set_printoptions(linewidth=200)
torch.set_printoptions(linewidth=200)
  • Function setup_colab (it just mounts Google Drive in Colab) (related to issue Simplify usage of Google Drive #36)
  • Function setup_book (useless => it just setup colab if we are using Colab)
  • Function gv
def gv(s): return graphviz.Source('digraph G{ rankdir="LR"' + s + '; }')
  • Function get_image_files_sorted
def get_image_files_sorted(path, recurse=True, folders=None):
    return get_image_files(path, recurse, folders).sorted()
  • Function search_images_bing (search images with Bing => we recommend DDG so don't need it)
  • Function search_images_ddg (we use another utility => don't need it)
  • Function plot_function
  • Function draw_tree
  • Function cluster_columns (dendogram)
  • Function image_cat (returns BytesIO object from an image of cat) => probably don't need it
  • Function image_bear (same but for a bear) => probably don't need it

Here is the part of the code copied from __init__.py of fastbook:

__version__ = "0.0.18"
import matplotlib as mpl, pkgutil, requests, time
from fastai.vision.all import *
from fastdownload import download_url
from pandas.api.types import CategoricalDtype
from scipy.cluster import hierarchy as hc
from io import StringIO, BytesIO
from urllib.error import URLError,HTTPError

try: from ipywidgets import widgets
except ModuleNotFoundError: warn("Missing `ipywidgets` - please install it")
try: import sentencepiece
except ModuleNotFoundError: warn("Missing `sentencepiece` - please run `pip install 'sentencepiece<0.1.90'`")
try: from nbdev.showdoc import *
except ModuleNotFoundError: warn("Missing `nbdev` - please install it")
try:
    import graphviz
    from sklearn.tree import export_graphviz
except ModuleNotFoundError: warn("Missing `graphviz` - please run `conda install fastbook`")

[...]

def plot_function(f, tx=None, ty=None, title=None, min=-2, max=2, figsize=(6,4)):
    x = torch.linspace(min,max)
    fig,ax = plt.subplots(figsize=figsize)
    ax.plot(x,f(x))
    if tx is not None: ax.set_xlabel(tx)
    if ty is not None: ax.set_ylabel(ty)
    if title is not None: ax.set_title(title)

def draw_tree(t, df, size=10, ratio=0.6, precision=0, **kwargs):
    s=export_graphviz(t, out_file=None, feature_names=df.columns, filled=True, rounded=True,
                      special_characters=True, rotate=False, precision=precision, **kwargs)
    return graphviz.Source(re.sub('Tree {', f'Tree {{ size={size}; ratio={ratio}', s))

def cluster_columns(df, figsize=(10,6), font_size=12):
    corr = np.round(scipy.stats.spearmanr(df).correlation, 4)
    corr_condensed = hc.distance.squareform(1-corr)
    z = hc.linkage(corr_condensed, method='average')
    fig = plt.figure(figsize=figsize)
    hc.dendrogram(z, labels=df.columns, orientation='left', leaf_font_size=font_size)
    plt.show()
@jfthuong jfthuong added the enhancement New feature or request label Oct 11, 2021
@jfthuong jfthuong added this to the UnpackAI Beta milestone Oct 11, 2021
jfthuong pushed a commit that referenced this issue Oct 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants