Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Convert result of group by agg to pyarrow if input is pyarrow #58129

Open
wants to merge 38 commits into
base: main
Choose a base branch
from
Open

Convert result of group by agg to pyarrow if input is pyarrow #58129

wants to merge 38 commits into from
+112 −8

Commits on Apr 1, 2024

  1. Set preserve_dtype flag for bool type only when result is also bool

    Kei committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    9faa460 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. Update implementation to change type to pyarrow only

    Kei committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    969d5b1 View commit details
    Browse the repository at this point in the history

  2. Change import order

    Kei committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    66114f3 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. Convert numpy array to pandas representation of pyarrow array

    Kei committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    b0290ed View commit details
    Browse the repository at this point in the history

  2. Add tests

    Kei committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    20c8fa0 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into fix/group_by_agg_pyarrow_bool_numpy_same_type

    Kei committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    97b3d54 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. Change pyarrow to optional import in agg_series() method

    Kei committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    932d737 View commit details
    Browse the repository at this point in the history

  2. Seperate tests

    Kei committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    82ddeb5 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into fix/group_by_agg_pyarrow_bool_numpy_same_type

    Kei committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    d510052 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. Merge branch 'main' into fix/group_by_agg_pyarrow_bool_numpy_same_type

    Kei committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    62a31d9 View commit details
    Browse the repository at this point in the history

  2. Revert to old implementation

    Kei committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    a54bf58 View commit details
    Browse the repository at this point in the history

  3. Update implementation to use pyarrow array method

    Kei committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    64330f0 View commit details
    Browse the repository at this point in the history

  4. Update test_aggregate tests

    Kei committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    0647711 View commit details
    Browse the repository at this point in the history

  5. Move pyarrow import to top of method

    Kei committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    affde38 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. Update according to pr comments

    Kei committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    842f561 View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2024

  1. Merge branch 'main' into fix/group_by_agg_pyarrow_bool_numpy_same_type

    Kei committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    93b5bf3 View commit details
    Browse the repository at this point in the history

  2. Fallback convert to input dtype is output is all nan or empty array

    Kei committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    6f35c0e View commit details
    Browse the repository at this point in the history

  3. Strip na values when inferring pyarrow dtype

    Kei committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    abd0adf View commit details
    Browse the repository at this point in the history

  4. Update tests to check expected inferred dtype instead of inputy dtype

    Kei committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    bebc442 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2024

  1. Override test case for test_arrow.py

    Kei committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    bb6343b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into fix/group_by_agg_pyarrow_bool_numpy_same_type

    Kei committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    3a3f2a2 View commit details
    Browse the repository at this point in the history

  3. Empty commit to trigger build run

    Kei committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    6dc40f5 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. In agg series, convert to np values, then cast to pyarrow dtype, acco… 

    …unt for missing pyarrow dtypes
    Kei committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    4ef96f7 View commit details
    Browse the repository at this point in the history

  2. Update tests

    Kei committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    c6a98c0 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Update rst docs

    Kei committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    9181eaf View commit details
    Browse the repository at this point in the history

  2. Update impl to fix tests

    Kei committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    612d7d0 View commit details
    Browse the repository at this point in the history

  3. Declare variable in outer scope

    Kei committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    3b6696b View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. Update impl to use maybe_cast_pointwise_result instead of maybe_cast_… 

    …to_pyarrow_array
    Kei committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    680e238 View commit details
    Browse the repository at this point in the history

  2. Fix tests with nested array

    Kei committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    3a8597e View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. Update according to pr comments

    Kei committed May 2, 2024
    Configuration menu
    Copy the full SHA
    6496b15 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into fix/group_by_agg_pyarrow_bool_numpy_same_type

    Kei committed May 2, 2024
    Configuration menu
    Copy the full SHA
    712c36a View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. Preserve_dtype if argument is passed in, else don't preserve

    Kei committed May 7, 2024
    Configuration menu
    Copy the full SHA
    e1ccef6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into fix/group_by_agg_pyarrow_bool_numpy_same_type

    @undermyumbrella1
    undermyumbrella1 committed May 7, 2024
    Configuration menu
    Copy the full SHA
    0ce083d View commit details
    Browse the repository at this point in the history

  3. Update tests

    Kei committed May 7, 2024
    Configuration menu
    Copy the full SHA
    a1d73f5 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'fix/group_by_agg_pyarrow_bool_numpy_same_type' of githu… 

    …b.com:undermyumbrella1/pandas into fix/group_by_agg_pyarrow_bool_numpy_same_type
    Kei committed May 7, 2024
    Configuration menu
    Copy the full SHA
    57845a8 View commit details
    Browse the repository at this point in the history

Commits on May 12, 2024

  1. Remove redundant tests

    @undermyumbrella1
    undermyumbrella1 committed May 12, 2024
    Configuration menu
    Copy the full SHA
    fa257b0 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into fix/group_by_agg_pyarrow_bool_numpy_same_type

    @undermyumbrella1
    undermyumbrella1 committed May 12, 2024
    Configuration menu
    Copy the full SHA
    0a9b83f View commit details
    Browse the repository at this point in the history

  3. retrigger pipeline

    @undermyumbrella1
    undermyumbrella1 committed May 12, 2024
    Configuration menu
    Copy the full SHA
    139319a View commit details
    Browse the repository at this point in the history