Skip to content

@llm-jp llm-jp

Change the repository type filter

All

    Repositories list

    41 repositories

    • scripts

      Public
      Shell
      •
      Apache License 2.0
      •1•2•4•11•Updated Dec 25, 2024Dec 25, 2024

    • nlp2025-ws

      Public
      NLP2025ワークショップ「大規模言語モデルのファインチューニング技術と評価」
      Python
      •0•0•0•0•Updated Dec 25, 2024Dec 25, 2024

    • nii-sakura-cluster-configs

      Public
      Shell
      •0•3•0•0•Updated Dec 24, 2024Dec 24, 2024

    • Megatron-LM

      Public
      Python
      •
      Other
      •1•6•0•0•Updated Dec 19, 2024Dec 19, 2024

    • llm-jp-membership-inference

      Public
      This is the repository for llm jp membership inference attack.
      Apache License 2.0
      •0•0•0•0•Updated Dec 18, 2024Dec 18, 2024

    • tuning-competition

      Public
      CSS
      •0•0•0•0•Updated Dec 16, 2024Dec 16, 2024

    • tuning-competition-baseline

      Public
      Python
      •0•1•0•0•Updated Dec 16, 2024Dec 16, 2024

    • llm-jp-VILA

      Public
      Python
      •
      Apache License 2.0
      •0•8•0•0•Updated Dec 16, 2024Dec 16, 2024

    • awesome-japanese-llm

      Public
      日本語LLMまとめ - Overview of Japanese LLMs
      japanesegenerative-modeljapanese-languagelanguage-modelslanguage-modelgenerative-modelsmultimodalvision-and-languagevision-languagefoundation-models
      TypeScript
      •
      Apache License 2.0
      •31•1.1k•7•0•Updated Dec 16, 2024Dec 16, 2024

    • llm-jp-eval

      Public
      Python
      •
      Apache License 2.0
      •38•112•5•3•Updated Dec 12, 2024Dec 12, 2024

    • FastChat2

      Public
      An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
      Python
      •
      Apache License 2.0
      •4.6k•0•0•0•Updated Nov 25, 2024Nov 25, 2024

    • Toxicity_Filter_fastText

      Public
      A fastText-based classifier for removing toxic texts from corpora, trained on automatically generated labeled data.
      Python
      •0•0•0•0•Updated Nov 23, 2024Nov 23, 2024

    • text2dataset

      Public
      Easily turn large English text datasets into Japanese text datasets using open LLMs.
      Python
      •
      MIT License
      •1•17•6•0•Updated Nov 14, 2024Nov 14, 2024

    • Toxicity_Filter

      Public
      A DeBERTa-based toxicity filter trained on LLM-jp Toxicity Dataset
      Python
      •0•0•0•0•Updated Nov 5, 2024Nov 5, 2024

    • moe-recipes

      Public
      Shell
      •0•2•0•0•Updated Oct 24, 2024Oct 24, 2024

    • transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •27k•0•0•0•Updated Oct 14, 2024Oct 14, 2024

    • llm-leaderboard

      Public
      Project of llm evaluation to Japanese tasks
      Python
      •37•1•0•0•Updated Aug 11, 2024Aug 11, 2024

    • FastChat

      Public
      An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
      Python
      •
      Apache License 2.0
      •4.6k•0•0•0•Updated Aug 9, 2024Aug 9, 2024

    • experiments

      Public
      Issue-Only Pretrain Task Management Repository
      0•0•66•0•Updated Aug 8, 2024Aug 8, 2024

    • llm-jp-tokenizer

      Public
      Roff
      •
      Apache License 2.0
      •6•31•1•0•Updated Aug 7, 2024Aug 7, 2024

    • llmjp_topic_model

      Public
      Python
      •0•0•0•0•Updated Jul 19, 2024Jul 19, 2024

    • modelwg

      Public
      LLM-jp Model-WG Working Directory
      Shell
      •
      Apache License 2.0
      •0•6•0•4•Updated Jul 9, 2024Jul 9, 2024

    • DeepSpeed-MII

      Public
      MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
      Python
      •
      Apache License 2.0
      •175•0•0•0•Updated Jul 5, 2024Jul 5, 2024

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •5k•0•0•0•Updated Jun 21, 2024Jun 21, 2024

    • nii-geniac-gcp-lustre

      Public
      GENIAC用のLustreクラスタ構築スクリプト
      Jinja
      •0•0•0•0•Updated Jun 20, 2024Jun 20, 2024

    • nii-geniac-hpc-toolkit

      Public
      HCL
      •0•1•0•0•Updated Jun 13, 2024Jun 13, 2024

    • llm-jp-sft

      Public
      Python
      •
      Apache License 2.0
      •15•55•0•1•Updated Jun 13, 2024Jun 13, 2024

    • memorization-analysis

      Public
      Python
      •0•0•3•1•Updated Jun 7, 2024Jun 7, 2024

    • nii-geniac-gcp-maintenance

      Public
      Shell
      •0•0•0•0•Updated Jun 7, 2024Jun 7, 2024

    • open-llm-leaderboard-local

      Public
      Open LLM Leaderboard のローカル実行用スクリプト
      Python
      •1•3•0•1•Updated Feb 24, 2024Feb 24, 2024