Handling doubles with FFI seems to be locale dependent

[senolfeldmann]

Environment Information

• jruby 9.3.13.0 (2.6.8) 2023-11-02 09b6f22 OpenJDK 64-Bit Server VM 11.0.21+0 on 11.0.21+0 +jit [x86_64-darwin]
• Darwin Ubuntu-Karmic-Koala-910.local 22.6.0 Darwin Kernel Version 22.6.0: Tue Nov 7 21:48:06 PST 2023; root:xnu-8796.141.3.702.9~2/RELEASE_X86_64 x86_64

Other relevant info you may wish to add:

• fast_excel
• LANG=de_DE.UTF-8

Expected Behavior

require 'fast_excel'

wb = FastExcel.open
# print output filename
puts wb.filename
wb.close

The excel file is not malformed when using CRuby. When extract the xlsx file (which is just a zip archive) and inspecting xl/worksheets/sheet1.xml, we get the correct formatting of pageMargins the data:

<pageMargins left="0.7" right="0.7" top="0.75" bottom="0.75" header="0.3" footer="0.3"/>

Actual Behavior

When using the fast_excel to write excel files with JRuby malformed data is written into xl/worksheets/sheet1.xml:

<pageMargins left="0,7" right="0,7" top="0,75" bottom="0,75" header="0,3" footer="0,3"/>

---

The cause seems to be the system locale. I encountered the bug both under MacOS (LANG=de_DE.UTF-8) and Ubuntu Focal (LC_ALL=de_DE.UTF-8). In Germany, the default decimal seperator is "," and not ".".
When looking into the source code of fast_excel (which is an FFI binding for the C-library libxlsxwriter) and its underlying library, I could not find any references to the system locale at the relevant places.

When following the underlying function which writes the page margin data _worksheet_write_page_margins(lxw_worksheet *self) in fast_excel/libxlsxwriter/src/worksheet.c you end up at the line struct xml_attribute *lxw_new_attribute_dbl(const char *key, double value) in fast_excel/libxlsxwriter/src/xmlwriter.c which tells us that the data is stored and written with doubles as the underlying data type.

Please tell me if you need any more info or if I can improve this bug report.

[headius]

This is very peculiar! Looking at your source links, it seems like a normal double is passed through to the library's lxw_sprintf_dbl function, which eventually goes to emyg_dtoa (a third-party dtoa impl), which as far as I can tell only outputs "." for decimal separator.

I have to assume there's some double-to-string logic running on the Java side, and that logic must be picking up locale somewhere. Are you sure the doubles are being written via the code paths you linked?

I'm also confused; you say , is the proper decimal separator, and the JRuby output has that as the decimal separator, but that's incorrect?

[senolfeldmann]

I have to assume there's some double-to-string logic running on the Java side, and that logic must be picking up locale somewhere. Are you sure the doubles are being written via the code paths you linked?

I am not 100% sure yet. I just followed the code starting from the ruby-side of things and did not do any debugging. I did not have the time to dig deeper yet.

I'm also confused; you say , is the proper decimal separator, and the JRuby output has that as the decimal separator, but that's incorrect?

Maybe my wording was confusing, sorry for that. JRuby's output is wrong in this case, as the decimal seperator in this particular case should always be a ..
I just wanted to emphasize that in Germany, the decimal separator is ,. So something that costs 1.50 USD would cost 1,50 USD here. This is how I figured that there must be some string conversion going on which is locale dependent in JRuby, but which is should not be locale dependent as we can observe in the CRuby example.

[enebo]

@senolfeldmann I am curious if the problem is resolved by changing one or both of those env vars. No doubt we need to get to the bottom of it but knowing which one is causing it might help? It will likely give a temporary workaround if nothing else.