Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] OPT training improvements #858

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

[WIP] OPT training improvements #858

wants to merge 18 commits into from

Commits on Jan 3, 2023

  1. add some dev essentials

    @zhisbug
    zhisbug committed Jan 3, 2023
    Configuration menu
    Copy the full SHA
    60fc3be View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2023

  1. finetuning works [skip ci]

    @zhisbug
    zhisbug committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    fd25c2e View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2023

  1. updates

    @zhisbug
    zhisbug committed Jan 19, 2023
    Configuration menu
    Copy the full SHA
    db589d9 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into hao-opt

    @zhisbug
    zhisbug committed Jan 19, 2023
    Configuration menu
    Copy the full SHA
    bb7aaf4 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2023

  1. some new updates

    @zhisbug
    zhisbug committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    8047611 View commit details
    Browse the repository at this point in the history

  2. dummy value update

    @zhisbug
    zhisbug committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    4b25289 View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2023

  1. switch to use broadcast [skip ci]

    @zhisbug
    zhisbug committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    fbb4621 View commit details
    Browse the repository at this point in the history

  2. expand the dataset size 10x

    @zhisbug
    zhisbug committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    bfdb2dd View commit details
    Browse the repository at this point in the history

  3. do not destroy collective group

    @zhisbug
    zhisbug committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    901a483 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2023

  1. 175B script for 32 node

    @zhisbug
    zhisbug committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    40ef377 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2023

  1. use real data for training instead of dummy

    @zhisbug
    zhisbug committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    45306eb View commit details
    Browse the repository at this point in the history

  2. opt init and change to 2048

    @zhisbug
    zhisbug committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    37b2d9b View commit details
    Browse the repository at this point in the history

Commits on Feb 3, 2023

  1. update

    @zhisbug
    zhisbug committed Feb 3, 2023
    Configuration menu
    Copy the full SHA
    6e8d4c2 View commit details
    Browse the repository at this point in the history

  2. reproduct bug

    @zhisbug
    zhisbug committed Feb 3, 2023
    Configuration menu
    Copy the full SHA
    a9f6402 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2023

  1. Fix output_sharding_dict

    @merrymercy
    merrymercy committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    ce155d4 View commit details
    Browse the repository at this point in the history

  2. add master copy distributed init

    @zhisbug
    zhisbug committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    a70e540 View commit details
    Browse the repository at this point in the history

  3. incorparate lianmin change

    @zhisbug
    zhisbug committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    e80cce0 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2023

  1. update opt finetuning weight init code

    @zhisbug
    zhisbug committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    256631b View commit details
    Browse the repository at this point in the history