Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-48031][SQL][FOLLOW-UP] Use ANSI-enabled cast in view lookup test #46614

Closed
wants to merge 1 commit into from

Conversation

HyukjinKwon
Copy link
Member

What changes were proposed in this pull request?

This PR is a followup of #46267 that uses ANSI-enabled cast in the tests. It intentionally uses ANSI-enabled cast in castColToType when you look up a view.

Why are the changes needed?

In order to fix the scheduled CI build without ANSI:

[info] - look up view relation *** FAILED *** (72 milliseconds)
[info]   == FAIL: Plans do not match ===
[info]    'SubqueryAlias spark_catalog.db3.view1                                                                                                                                                                                                                                                                                                                                                                          'SubqueryAlias spark_catalog.db3.view1
[info]    +- View (`spark_catalog`.`db3`.`view1`, ['col1, 'col2, 'a, 'b])                                                                                                                                                                                                                                                                                                                                                 +- View (`spark_catalog`.`db3`.`view1`, ['col1, 'col2, 'a, 'b])
[info]       +- 'Project [cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, col1, 0, 1) as int) AS col1#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, col2, 0, 1) as string) AS col2#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, a, 0, 1) as int) AS a#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, b, 0, 1) as string) AS b#0]      +- 'Project [cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, col1, 0, 1) as int) AS col1#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, col2, 0, 1) as string) AS col2#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, a, 0, 1) as int) AS a#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, b, 0, 1) as string) AS b#0]
[info]          +- 'Project [*]                                                                                                                                                                                                                                                                                                                                                                                                 +- 'Project [*]
[info]             +- 'UnresolvedRelation [tbl1], [], false        
[info] - look up view created before Spark 3.0 *** FAILED *** (452 milliseconds)
[info]   == FAIL: Plans do not match ===
[info]    'SubqueryAlias spark_catalog.db3.view2                                                                                                                                                                                                                                                                                                          'SubqueryAlias spark_catalog.db3.view2
[info]    +- View (`db3`.`view2`, ['col1, 'col2, 'a, 'b])                                                                                                                                                                                                                                                                                                 +- View (`db3`.`view2`, ['col1, 'col2, 'a, 'b])
[info]       +- 'Project [cast(getviewcolumnbynameandordinal(`db3`.`view2`, col1, 0, 1) as int) AS col1#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, col2, 0, 1) as string) AS col2#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, a, 0, 1) as int) AS a#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, b, 0, 1) as string) AS b#0]      +- 'Project [cast(getviewcolumnbynameandordinal(`db3`.`view2`, col1, 0, 1) as int) AS col1#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, col2, 0, 1) as string) AS col2#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, a, 0, 1) as int) AS a#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, b, 0, 1) as string) AS b#0]
[info]          +- 'Project [*]                                                                                                                                                                                                                                                                                                                                 +- 'Project [*]
[info]             +- 'UnresolvedRelation [tbl1], [], false                                                                                                                                                                                                                                                                                                        +- 'UnresolvedRelation [tbl1], [], false (PlanTest.scala:179)

Does this PR introduce any user-facing change?

No, the main change has not been released yet.

How was this patch tested?

Manually ran the tests after ANSI disabled.

Was this patch authored or co-authored using generative AI tooling?

No.

@srielau
Copy link
Contributor

srielau commented May 16, 2024

@HyukjinKwon Any idea why this did not trip the original PR?

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented May 16, 2024

because we don't run the tests without ANSI in PR builder (because of resource limitation). It only runs in the daily build defined here https://github.com/apache/spark/actions/workflows/build_non_ansi.yml

@HyukjinKwon
Copy link
Member Author

Merged to master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
4 participants