Machine Learning Things is a lightweight python library that contains functions and code snippets that I use in my everyday research with Machine Learning, Deep Learning, NLP.
I created this repo because I was tired of always looking up same code from older projects and I wanted to gain some experience in building a Python library. By making this available to everyone it gives me easy access to code I use frequently and it can help others in their machine learning work. If you find any bugs or something doesn't make sense please feel free to open an issue.
That is not all! This library also contains Python code snippets and notebooks that speed up my Machine Learning workflow.
Note:
- Update:
Feb 5, 2022Thank you all again for your support and kindness! This package is available on pypi now!pip install ml-things - Update:
July 16, 2021Thank you all for your support and kindness! As I promissed I will move this repo to pip install modules. - If I reach 100 stars I will release the first official version and add it to the pip install modules!
-
- Installation Details on how to install ml_things.
- Array Functions Details on the ml_things array related functions:
- Plot Functions Details on the ml_things plot related functions:
- Text Functions Details on the ml_things text related functions:
- Web Related Details on the ml_things web related functions:
-
Snippets: Curated list of Python snippets I frequently use.
-
Comments: Sample on how I like to comment my code. It is still a work in progress.
-
Notebooks Tutorials: Machine learning projects that I converted to tutorials and posted online.
-
Final Note: Being grateful.
This repo is tested with Python 3.6+.
It's always good practice to install ml_things in a virtual environment. If you guidance on using Python's virtual environments you can check out the user guide here.
You can install ml_things with pip from GitHub:
pip install git+https://github.com/gmihaila/ml_thingsOr from pypi:
pip install ml-thingsAll function implemented in the ml_things module.
Array manipulation related function that can be useful when working with machine learning.
pad_array [source]
Pad variable length array to a fixed numpy array. It can handle single arrays [1,2,3] or nested arrays [[1,2],[3]].
By default it will padd zeros to the maximum length of row detected:
>>> from ml_things import pad_array
>>> pad_array(variable_length_array=[[1,2],[3],[4,5,6]])
array([[1., 2., 0.],
[3., 0., 0.],
[4., 5., 6.]])It can also pad to a custom size and with cusotm values:
>>> pad_array(variable_length_array=[[1,2],[3],[4,5,6]], fixed_length=5, pad_value=99)
array([[ 1., 2., 99., 99., 99.],
[ 3., 99., 99., 99., 99.],
[ 4., 5., 6., 99., 99.]])batch_array [source]
Split a list into batches/chunks. Last batch size is remaining of list values. Note: This is also called chunking. I call it batches since I use it more in ML.
The last batch will be the reamining values:
>>> from ml_things import batch_array
>>> batch_array(list_values=[1,2,3,4,5,6,7,8,8,9,8,6,5,4,6], batch_size=4)
[[1, 2, 3, 4], [5, 6, 7, 8], [8, 9, 8, 6], [5, 4, 6]]Plot related function that can be useful when working with machine learning.
plot_array [source]
Create plot from a single array of values.
All arguments are optimized for quick plots. Change the magnify arguments to vary the size of the plot:
>>> from ml_things import plot_array
>>> plot_array([1,3,5,3,7,5,8,10], path='plot_array.png', magnify=0.1, use_title='A Random Plot', start_step=0.3, step_size=0.1, points_values=True, use_ylabel='Thid', use_xlabel='This')
plot_dict [source]
Create plot from a single array of values.
All arguments are optimized for quick plots. Change the magnify arguments to vary the size of the plot:
>>> from ml_things import plot_dict
>>> plot_dict({'train_acc':[1,3,5,3,7,5,8,10],
'valid_acc':[4,8,9]}, use_linestyles=['-', '--'], magnify=0.1,
start_step=0.3, step_size=0.1,path='plot_dict.png', points_values=[True, False], use_title='Title')
plot_confusion_matrix [source]
This function prints and plots the confusion matrix. Normalization can be applied by setting normalize=True.
All arguments are optimized for quick plots. Change the magnify arguments to vary the size of the plot:
>>> from ml_things import plot_confusion_matrix
>>> plot_confusion_matrix(y_true=[1,0,1,1,0,1], y_pred=[0,1,1,1,0,1], magnify=0.1, use_title='My Confusion Matrix', path='plot_confusion_matrix.png');
Confusion matrix, without normalization
array([[1, 1],
[1, 3]])
Text related function that can be useful when working with machine learning.
clean_text [source]
Clean text using various techniques:
>>> from ml_things import clean_text
>>> clean_text("ThIs is $$$%. \t\t\n \\ so dirtyyy$$ text :'(. omg!!!", full_clean=True)
'this is so dirtyyy text omg'Web related function that can be useful when working with machine learning.
download_from [source]
Download file from url. It will return the path of the downloaded file:
>>> from ml_things import download_from
>>> download_from(url='https://raw.githubusercontent.com/gmihaila/ml_things/master/setup.py', path='.')
'./setup.py'This is a very large variety of Python snippets without a certain theme. I put them in the most frequently used ones while keeping a logical order. I like to have them as simple and as efficient as possible.
| Name | Description |
|---|---|
| Read FIle | One liner to read any file. |
| Write File | One liner to write a string to a file. |
| Debug | Start debugging after this line. |
| Pip Install GitHub | Install library directly from GitHub using pip. |
| Parse Argument | Parse arguments given when running a .py file. |
| Doctest | How to run a simple unittesc using function documentaiton. Useful when need to do unittest inside notebook. |
| Fix Text | Since text data is always messy, I always use it. It is great in fixing any bad Unicode. |
| Current Date | How to get current date in Python. I use this when need to name log files. |
| Current Time | Get current time in Python. |
| Remove Punctuation | The fastest way to remove punctuation in Python3. |
| PyTorch-Dataset | Code sample on how to create a PyTorch Dataset. |
| PyTorch-Device | How to setup device in PyTorch to detect if GPU is available. |
These are a few snippets of how I like to comment my code. I saw a lot of different ways of how people comment their code. One thing is for sure: any comment is better than no comment.
I try to follow as much as I can the PEP 8 — the Style Guide for Python Code.
When I comment a function or class:
# required import for variables type declaration
from typing import List, Optional, Tuple, Dict
def my_function(function_argument: str, another_argument: Optional[List[int]] = None,
another_argument_: bool = True) -> Dict[str, int]
r"""Function/Class main comment.
More details with enough spacing to make it easy to follow.
Arguments:
function_argument (:obj:`str`):
A function argument description.
another_argument (:obj:`List[int]`, `optional`):
This argument is optional and it will have a None value attributed inside the function.
another_argument_ (:obj:`bool`, `optional`, defaults to :obj:`True`):
This argument is optional and it has a default value.
The variable name has `_` to avoid conflict with similar name.
Returns:
:obj:`Dict[str: int]`: The function returns a dicitonary with string keys and int values.
A class will not have a return of course.
"""
# make sure we keep out promise and return the variable type we described.
return {'argument': function_argument}This is where I keep notebooks of some previous projects which I turnned them into small tutorials. A lot of times I use them as basis for starting a new project.
All of the notebooks are in Google Colab. Never heard of Google Colab? 🙀 You have to check out the Overview of Colaboratory, Introduction to Colab and Python and what I think is a great medium article about it to configure Google Colab Like a Pro.
If you check the /ml_things/notebooks/ a lot of them are not listed here because they are not in a 'polished' form yet. These are the notebooks that are good enough to share with everyone:
| Name | Description | Links |
|---|---|---|
| 🍇 Better Batches with PyTorchText BucketIterator | How to use PyTorchText BucketIterator to sort text data for better batching. |      |
| 🐶 Pretrain Transformers Models in PyTorch using Hugging Face Transformers | Pretrain 67 transformers models on your custom dataset. | |
| 🎻 Fine-tune Transformers in PyTorch using Hugging Face Transformers | Complete tutorial on how to fine-tune 73 transformer models for text classification — no code changes necessary! | |
| ⚙️ Bert Inner Workings in PyTorch using Hugging Face Transformers | Complete tutorial on how an input flows through Bert. | |
| 🎱 GPT2 For Text Classification using Hugging Face 🤗 Transformers | Complete tutorial on how to use GPT2 for text classification. | |
Thank you for checking out my repo. I am a perfectionist so I will do a lot of changes when it comes to small details.
If you see something wrong please let me know by opening an issue on my ml_things GitHub repository!
A lot of tutorials out there are mostly a one-time thing and are not being maintained. I plan on keeping my tutorials up to date as much as I can.
🦊 GitHub: gmihaila
🌐 Website: gmihaila.github.io
👔 LinkedIn: mihailageorge
📬 Email: georgemihaila@my.unt.edu.com