Performing multiple HDF5 operations "captures" cout,cerr

[brendenroberts]

Hi guys, on my system performing multiple HDF5 operations somehow "captures" cout and cerr, and they don't seem to point to the right buffers afterward. I tried this with the newest commit and v3.1.9 as well as C++17 and C++20. In all cases running the following example:

#include "itensor/all_basic.h"

int main(int argc, char *argv[]) {
    auto ofHandle0 = itensor::h5_open("test_0.h5",'w');
    close(ofHandle0);

    std::cout << "cout 0" << std::endl;
    std::cerr << "cerr 0" << std::endl;

    auto ofHandle1 = itensor::h5_open("test_1.h5",'w');
    close(ofHandle1);

    std::cout << "cout 1" << std::endl;
    std::cerr << "cerr 1" << std::endl;

    auto ofHandle2 = itensor::h5_open("test_2.h5",'w');
    close(ofHandle2);

    std::cout << "cout 2" << std::endl;
    std::cerr << "cerr 2" << std::endl;

    return 0;
    }

produces only

cout 0
cerr 0
cerr 1

Adding more flushes doesn't make a difference, and other iostreams don't seem to be affected.

[mtfishman]

Not sure why that would be, those are just wrappers around a call to the h5 library:

ITensor/itensor/util/h5/wrap_h5.hpp

Lines 43 to 47 in 167aaff

	h5::file inline
	h5_open(std::string const& name, char mode)
	{
	return h5::file(name.c_str(),mode);
	}

(which I think is just some version of https://github.com/TRIQS/h5, so maybe helpful to test the latest version of that library directly).

@emstoudenmire (or @Wentzell), any idea?

[Wentzell]

That's very strange. I have adjusted the example to use https://github.com/TRIQS/h5 directly

#include <h5/h5.hpp>

int main(int argc, char *argv[]) {
    auto ofHandle0 = h5::file("test_0.h5",'w');
    ofHandle0.close();

    std::cout << "cout 0" << std::endl;
    std::cerr << "cerr 0" << std::endl;

    auto ofHandle1 = h5::file("test_1.h5",'w');
    ofHandle1.close();

    std::cout << "cout 1" << std::endl;
    std::cerr << "cerr 1" << std::endl;

    auto ofHandle2 = h5::file("test_2.h5",'w');
    ofHandle2.close();

    std::cout << "cout 2" << std::endl;
    std::cerr << "cerr 2" << std::endl;

    return 0;
}

Both for the release branch 1.0.x and the development branch unstable do I get the expected output

cout 0
cerr 0
cout 1
cerr 1
cout 2
cerr 2

@brendenroberts Could you try the example above in your build environment?
The steps would be

• clone https://github.com/TRIQS/h5 to e.g. h5_src
• Add the example above in h5_src/test/c++/stream_issue.cpp
• Build and run the example with

mkdir h5_src/build && cd h5_src/build
cmake .. -DCMAKE_INSTALL_PREFIX=$PWD/install && make stream_issue
./test/c++/stream_issue

[brendenroberts]

Your direct example works as expected for me also.

[Wentzell]

Your direct example works as expected for me also.

Ok. In that case the issue should be related to either the TRIQS/h5 version
that is currently part of ITensor, or to how ITensor wraps the TRIQS/h5 functionality.

I would suggest to first update the version within ITensor and see if the problem persists.

[Wentzell]

@mtfishman @emstoudenmire Would you agree? If so, could either of you look into that?

[brendenroberts]

Thanks @Wentzell. The issue seems to be with the calls to close(). It solves the problem if instead I use the member function h5_object.close() (or whatever class it is). I think I got this io pattern from the unit test?

[Wentzell]

Ok. h5::object.close() should properly close the file handle. In the end it simply invokes the desctructor of the h5::object which internally will do the H5Idec_ref. I am not sure about what the close(..) in ITensor does.

[mtfishman]

Yeah, I'm trying to search the source code for the free function close(...) and I can't even find it. @emstoudenmire do you know its origin?

[mtfishman]

I guess we should remove it from the unit tests and docs in favor of h5::object.close() until we track down the issue with the free version.