Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hugging Face Model Training #45

Merged
merged 66 commits into from
Jun 21, 2023
Merged

Hugging Face Model Training #45

merged 66 commits into from
Jun 21, 2023

Commits on May 26, 2023

  1. feat: added entity label file loading function

    @tieandrews
    tieandrews committed May 26, 2023
    Configuration menu
    Copy the full SHA
    281cb57 View commit details
    Browse the repository at this point in the history

Commits on May 29, 2023

  1. feat: script to coordinate training process

    @tieandrews
    tieandrews committed May 29, 2023
    Configuration menu
    Copy the full SHA
    79fd36f View commit details
    Browse the repository at this point in the history

  2. feat: bash script to run training and set params

    @tieandrews
    tieandrews committed May 29, 2023
    Configuration menu
    Copy the full SHA
    f289c36 View commit details
    Browse the repository at this point in the history

  3. feat: script to generate hf formatted data

    @tieandrews
    tieandrews committed May 29, 2023
    Configuration menu
    Copy the full SHA
    ca9c5fa View commit details
    Browse the repository at this point in the history

  4. docs: updated ner_training target name

    @tieandrews
    tieandrews committed May 29, 2023
    Configuration menu
    Copy the full SHA
    dd3696f View commit details
    Browse the repository at this point in the history

  5. docs: model training readme

    @tieandrews
    tieandrews committed May 29, 2023
    Configuration menu
    Copy the full SHA
    7f4ac68 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2023

  1. feat: automated utils to evaluate trained models

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    c64e778 View commit details
    Browse the repository at this point in the history

  2. feat: bash script to run evaluation on checkpoint

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    7361e91 View commit details
    Browse the repository at this point in the history

  3. bug: added _init_ to src to make module

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    26b20ce View commit details
    Browse the repository at this point in the history

  4. bug: token based evaluation same as entity based 

    Lists passed to funcion were modifying data outside the
function. This was causing the evaluation to be the same
for both token and entity based evaluation.
    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    8d69aa6 View commit details
    Browse the repository at this point in the history

  5. feat: added new return objects

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    2cb379a View commit details
    Browse the repository at this point in the history

  6. docs: updated docs to match new input format

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    eff149e View commit details
    Browse the repository at this point in the history

  7. feat: improved error checking and logging

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    df022a7 View commit details
    Browse the repository at this point in the history

  8. feat: cleaned up imports and constants

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    234aa3d View commit details
    Browse the repository at this point in the history

  9. feat: last imports cleaned

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    a5348f8 View commit details
    Browse the repository at this point in the history

  10. feat: added custom metrics to mlflow logging

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    0944ea3 View commit details
    Browse the repository at this point in the history

  11. feat: model training bash script

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    9cf2140 View commit details
    Browse the repository at this point in the history

  12. feat: basic notebook to use on colab to train

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    1a18d96 View commit details
    Browse the repository at this point in the history

  13. docs: initial commit

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    2357286 View commit details
    Browse the repository at this point in the history

  14. docs: results and models directory setup

    @tieandrews
    tieandrews committed May 30, 2023
    Configuration menu
    Copy the full SHA
    9ea5abe View commit details
    Browse the repository at this point in the history

Commits on May 31, 2023

  1. feat: removed unused label2id object

    @tieandrews
    tieandrews committed May 31, 2023
    Configuration menu
    Copy the full SHA
    9192ad8 View commit details
    Browse the repository at this point in the history

  2. feat: separated folder location setting

    @tieandrews
    tieandrews committed May 31, 2023
    Configuration menu
    Copy the full SHA
    0153d73 View commit details
    Browse the repository at this point in the history

  3. bug: setup individual copied objects for pred/true

    @tieandrews
    tieandrews committed May 31, 2023
    Configuration menu
    Copy the full SHA
    8e31500 View commit details
    Browse the repository at this point in the history

  4. bug: added defined max token length

    @tieandrews
    tieandrews committed May 31, 2023
    Configuration menu
    Copy the full SHA
    1ac2bbc View commit details
    Browse the repository at this point in the history

  5. bug: added quotes for paths with spaces in colab

    @tieandrews
    tieandrews committed May 31, 2023
    Configuration menu
    Copy the full SHA
    89be2df View commit details
    Browse the repository at this point in the history

  6. bug: fixed labelled file location with spaces

    @tieandrews
    tieandrews committed May 31, 2023
    Configuration menu
    Copy the full SHA
    4534ba2 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2023

  1. Merge remote-tracking branch 'origin/dev' into 22-fine-tune-allenai-s… 

    …pecter2-model-for-ner
    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    f2bf1ac View commit details
    Browse the repository at this point in the history

  2. feat: rename and update standard params

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    4cbfaaf View commit details
    Browse the repository at this point in the history

  3. docs: switch default data folder

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    5ca03dd View commit details
    Browse the repository at this point in the history

  4. docs: clean up and make better defaults

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    e7faccb View commit details
    Browse the repository at this point in the history

  5. feat: update to use new train/val/test data struct

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    fd60e9b View commit details
    Browse the repository at this point in the history

  6. feat: setting up __init__ files

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    24eb8fb View commit details
    Browse the repository at this point in the history

  7. feat: added final model name field

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    e40751c View commit details
    Browse the repository at this point in the history

  8. feat: added custom model logging at training end

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    00dd4bf View commit details
    Browse the repository at this point in the history

  9. feat: setup huggingface batch inference

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    fb534fa View commit details
    Browse the repository at this point in the history

  10. docs: init file setup

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    de94f40 View commit details
    Browse the repository at this point in the history

  11. bug: fixed batch prediction

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    2578015 View commit details
    Browse the repository at this point in the history

  12. feat: include max_samples by default for local run

    @tieandrews
    tieandrews committed Jun 1, 2023
    Configuration menu
    Copy the full SHA
    3f30515 View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2023

  1. feat: added better input checking from tests

    @tieandrews
    tieandrews committed Jun 2, 2023
    Configuration menu
    Copy the full SHA
    bb14221 View commit details
    Browse the repository at this point in the history

  2. tests: fixes to complete test suite

    @tieandrews
    tieandrews committed Jun 2, 2023
    Configuration menu
    Copy the full SHA
    8ea406b View commit details
    Browse the repository at this point in the history

  3. docs: added transformers requirements

    @tieandrews
    tieandrews committed Jun 2, 2023
    Configuration menu
    Copy the full SHA
    0b43a54 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2023

  1. bug: moved docopt to main

    @tieandrews
    tieandrews committed Jun 3, 2023
    Configuration menu
    Copy the full SHA
    31c2fc8 View commit details
    Browse the repository at this point in the history

  2. tests: initial basic tests

    @tieandrews
    tieandrews committed Jun 3, 2023
    Configuration menu
    Copy the full SHA
    c135b09 View commit details
    Browse the repository at this point in the history

  3. feat: colab train notebook autoreload used

    @tieandrews
    tieandrews committed Jun 3, 2023
    Configuration menu
    Copy the full SHA
    92df812 View commit details
    Browse the repository at this point in the history

  4. docs: delete model card of specter2

    @tieandrews
    tieandrews committed Jun 3, 2023
    Configuration menu
    Copy the full SHA
    da53411 View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2023

  1. tests: added tests for calculate/plot methods

    @tieandrews
    tieandrews committed Jun 6, 2023
    Configuration menu
    Copy the full SHA
    67088c4 View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2023

  1. feat: added stride to generating overlap windows

    @tieandrews
    tieandrews committed Jun 8, 2023
    Configuration menu
    Copy the full SHA
    ca10b58 View commit details
    Browse the repository at this point in the history

  2. feat: added early stopping with patience 5

    @tieandrews
    tieandrews committed Jun 8, 2023
    Configuration menu
    Copy the full SHA
    37fefaf View commit details
    Browse the repository at this point in the history

  3. feat: added warmup and seed

    @tieandrews
    tieandrews committed Jun 8, 2023
    Configuration menu
    Copy the full SHA
    5357087 View commit details
    Browse the repository at this point in the history

  4. bug: added load_best_model_at_end for early stop

    @tieandrews
    tieandrews committed Jun 8, 2023
    Configuration menu
    Copy the full SHA
    9521d9c View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2023

  1. bug: added control for when mlflow diasbled

    @tieandrews
    tieandrews committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    a9572c6 View commit details
    Browse the repository at this point in the history

  2. feat: updated preprocessing stride length

    @tieandrews
    tieandrews committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    babdb8b View commit details
    Browse the repository at this point in the history

  3. feat: updated grouped_entities to aggregationstrat

    @tieandrews
    tieandrews committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    50ac915 View commit details
    Browse the repository at this point in the history

  4. feat: baseline test results

    @tieandrews
    tieandrews committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    af0e2d4 View commit details
    Browse the repository at this point in the history

  5. docs: roberta finetune v3 results

    @tieandrews
    tieandrews committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    bc669ec View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2023

  1. feat: bert-multilanguage results

    @tieandrews
    tieandrews committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    386c058 View commit details
    Browse the repository at this point in the history

  2. feat: specter2 results

    @tieandrews
    tieandrews committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    b84e7ae View commit details
    Browse the repository at this point in the history

  3. docs: removed results folder

    @tieandrews
    tieandrews committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    7215a39 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2023

  1. docs: HF model training readme update

    @tieandrews
    tieandrews committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    129fac0 View commit details
    Browse the repository at this point in the history

  2. docs: update colab notebook

    @tieandrews
    tieandrews committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    1c617e2 View commit details
    Browse the repository at this point in the history

  3. feat: final roberta model training script

    @tieandrews
    tieandrews committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    743780b View commit details
    Browse the repository at this point in the history

  4. feat: automated baseline evaluation script

    @tieandrews
    tieandrews committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    81bc738 View commit details
    Browse the repository at this point in the history

  5. docs: labelsetudio setup guide

    @tieandrews
    tieandrews committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    8d99f29 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'dev' into 22-fine-tune-allenai-specter2-model-for-ner

    @tieandrews
    tieandrews authored Jun 21, 2023
    Configuration menu
    Copy the full SHA
    9211ba3 View commit details
    Browse the repository at this point in the history

  7. bug: docopt outside of main

    @tieandrews
    tieandrews committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    aa181eb View commit details
    Browse the repository at this point in the history

  8. bug: mermaid diagram not rendering in README

    @tieandrews
    tieandrews authored Jun 21, 2023
    Configuration menu
    Copy the full SHA
    fc50bd7 View commit details
    Browse the repository at this point in the history