Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sqlp: add ability to handle different decimal operators #1050

Closed
jqnatividad opened this issue Jun 15, 2023 · 11 comments · Fixed by #1832
Closed

sqlp: add ability to handle different decimal operators #1050

jqnatividad opened this issue Jun 15, 2023 · 11 comments · Fixed by #1832
Assignees
Labels
enhancement New feature or request upstream there is an issue upstream blocking this issue

Comments

@jqnatividad
Copy link
Owner

          @jqnatividad I'm adding a problem, that I'm able to manage using duckdb.

In Italy, the floating separator is , and not the . If I use qsv to import OpenCoesione data, all the numeric fields are mapped as string (because we have the ,).

In duckdb I solve using decimal_separator=',' parameter.
If there is something similar in polar, it would be very useful to have it also in qsv.

image

Originally posted by @aborruso in #1047 (comment)

@jqnatividad jqnatividad added the enhancement New feature or request label Jun 15, 2023
@jqnatividad
Copy link
Owner Author

jqnatividad commented Jun 15, 2023

Unfortunately @aborruso , at the moment, this is not available in the current version of Polars.

There's an open issue for it though -
pola-rs/polars#6698

In the meantime, you can use apply operations thousands in your pipeline to convert the comma decimal separator to the dot decimal separator, before passing it to sqlp, and then convert it back to the comma decimal separator with the apply operations thousands command.

@aborruso
Copy link
Contributor

Thank you very much

@github-actions
Copy link

Stale issue message

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 22, 2023
@jqnatividad jqnatividad reopened this Aug 22, 2023
@github-actions
Copy link

Stale issue message

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Oct 30, 2023
@jqnatividad
Copy link
Owner Author

This is still open on polars repo...

@jqnatividad jqnatividad reopened this Oct 30, 2023
@jqnatividad jqnatividad added the upstream there is an issue upstream blocking this issue label Nov 2, 2023
Copy link

github-actions bot commented Jan 2, 2024

Stale issue message

@jqnatividad
Copy link
Owner Author

This remains an open issue in Polars...

@jqnatividad
Copy link
Owner Author

jqnatividad commented Apr 19, 2024

Glad to report that configurable decimal operators are now supported in Polars main!
pola-rs/polars#15774

Hopefully, it finds its way into polars-sql...

@jqnatividad
Copy link
Owner Author

Just opened a feature request issue asking for Polars SQL support

pola-rs/polars#15786

@jqnatividad
Copy link
Owner Author

With the --decimal-comma option, you can now read input data with floats using comma as a decimal separator instead of period.

Bear in mind that you'll need to specify an alternate delimiter as well (not the default comma delimiter)

@aborruso
Copy link
Contributor

Thank you very much

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request upstream there is an issue upstream blocking this issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants