tasknet
is an interface between Huggingface datasets and Huggingface Trainer.
tasknet
relies on task templates to avoid boilerplate codes. The task templates correspond to Transformers AutoClasses:
SequenceClassification
TokenClassification
MultipleChoice
Seq2SeqLM
(experimental support)
The task templates follow the same interface. They implement preprocess_function
, a data collator and compute_metrics
.
Look at tasks.py and use existing templates as a starting point to implement a custom task template.
Each task template has fields that should be matched with specific dataset columns. Classification has two text fields s1
,s2
, and a label y
. Pass a dataset to a template, and fill-in the mapping between the tempalte fields and the dataset columns to instanciate a task.
import tasknet as tn; from datasets import load_dataset
rte = tn.Classification(
dataset=load_dataset("glue", "rte"),
s1="sentence1", s2="sentence2", y="label") #s2 is optional
class hparams:
model_name='microsoft/deberta-v3-base' # deberta models have the best results (and tasknet support)
learning_rate = 3e-5 # see hf.co/docs/transformers/en/main_classes/trainer#transformers.TrainingArguments
tasks = [rte]
model = tn.Model(tasks, hparams)
trainer = tn.Trainer(model, tasks, hparams)
trainer.train()
trainer.evaluate()
p = trainer.pipeline()
p([{'text':x.premise,'text_pair': x.hypothesis}]) # HuggingFace pipeline for inference
Tasknet is multitask by design. model.task_models_list
contains one model per task, with shared encoder.
pip install tasknet
https://colab.research.google.com/drive/15Xf4Bgs3itUmok7XlAK6EEquNbvjD9BD?usp=sharing
jiant is another library comparable to tasknet. tasknet is a minimal extension of Trainer
centered on task templates, while jiant builds a Trainer
equivalent from scratch called runner
.
tasknet
is leaner and closer to Huggingface native tools. Jiant is config-based and command line focused while tasknet is designed for interative use and python scripting.
This code uses some part of the examples of the transformers library and some code from multitask-learning-transformers.
You can request features on github or reach me at damien.sileo@inria.fr
@misc{sileod22-tasknet,
author = {Sileo, Damien},
doi = {10.5281/zenodo.561225781},
month = {11},
title = {{tasknet, multitask interface between Trainer and datasets}},
url = {https://github.com/sileod/tasknet},
version = {1.5.0},
year = {2022}}