Tests fail under alpha #711
Comments
|
Wow, I think your runner is literally too fast for doctest! Could you check if #712 fixes it? |
No, that does not do the trick unfortunately. However I highly doubt that being "too fast" is the cause - this is a CPU from 1999 running at 1GHz! And I ran it on machines I know to be much, much faster and did not see the issue there. I think a much more likely explanation has to do with clock resolution, it might be that the RTC cannot measure time that precisely. What units is the timeout decorator in? This machine runs with Again, if it's easier for you to just try it out on the machine yourself, let me know! |
|
Interesting, thank you for this very detailed report. The resolution is microseconds and the argument to the timeout decorator is in seconds, so in theory this shouldn't be a problem. ticks_t getCurrentTicks() {
timeval t;
gettimeofday(&t, nullptr);
return static_cast<ticks_t>(t.tv_sec) * 1000000 + static_cast<ticks_t>(t.tv_usec);
}The simplest solution is probably just using |
|
Does the current version do the trick for you? I've moved to |
Unfortunately still nope. |
|
Okay, implemented platform specific sleeping now since the standardized one wouldn't work. Could you test one last time if the branch works for you now? If not, then there's seriously something wrong somewhere! |
|
That seems to have gotten rid of the timeout issue, but introduced three new failures (at least I think they are new). Luckily these failures are deterministic at least. The output from the three failures is: |
|
Seems like it just exits unexpectedly at some point, assuming it just throws an exception, are you able to attach a debugger to figure out what's going on here? |
I'm not really sure tbh. But what I have done is gone ahead and provisioned you an account so that you can try it out yourself - you should be able to log in with your ssh key to saalvage [at] alphadev [dot] matoro [dot] tk. tmux, git, gdb, etc are all available. Let me know if you are able to log in and check it out. |
|
Hi @Saalvage , were you able to try logging into the account I provisioned above? This machine is still available so that you can check out the issue firsthand. |
|
Turns out this also affects HPPA (PA-RISC), at least the machine I just got. I am wondering if this is simply something that shows up on very slow hardware in general... |
|
After digging into this for quite a while I finally understand what is going on. My very first suspicion was correct - clock resolution is the problem. I used this snippet in order to get information about the clock resolution: https://gist.github.com/traversaro/e031b324dd278acbd033059e3604ed3f On x86, ARM, POWERPC it returns the following: But on alpha, it outputs: And HPPA is even less precise: Small patch I added for debugging: Then, I took a random sampling of test runtimes. On x86: But on alpha: Because of our clock precision, test runtimes are going to be rounded to the nearest millisecond (977 microseconds technically, but easier to just say 1 millisecond). Anything with a short enough runtime is just going to be rounded down to 0. It seems that tests which are expected to fail MUST |
|
Simply adding a sleep to the test that is expected to time out is sufficient to solve the timeout issues, no need for the test of #712. This is sufficient to get all tests passing on HPPA. As for the three remaining issues on alpha, it is indeed a segfault, but gdb is not really functional it seems...so it could be difficult to track down where the problem is. Seems to have jumped to the zero address somehow ( This seems to be as close as I can get to a backtrace by stepping through it until it errors like so: |
matoro
added a commit
to matoro/doctest
that referenced
this issue
Apr 15, 2023
Fixes doctest#711 under HPPA, and partially under alpha.
matoro
added a commit
to matoro/doctest
that referenced
this issue
Apr 15, 2023
Fixes doctest#711 under HPPA, and partially under alpha.
matoro
added a commit
to matoro/doctest
that referenced
this issue
Apr 15, 2023
Fixes doctest#711 under HPPA, and partially under alpha.
matoro
added a commit
to matoro/doctest
that referenced
this issue
Apr 15, 2023
Fixes doctest#711 under HPPA, and partially under alpha.
matoro
added a commit
to matoro/gentoo
that referenced
this issue
Apr 15, 2023
See: doctest/doctest#770 See: doctest/doctest#711 Bug: https://bugs.gentoo.org/874345 Signed-off-by: Matoro Mahri <matoro@users.noreply.github.com>
gentoo-bot
pushed a commit
to gentoo/gentoo
that referenced
this issue
Apr 26, 2023
See: doctest/doctest#770 See: doctest/doctest#711 Bug: https://bugs.gentoo.org/874345 Signed-off-by: Matoro Mahri <matoro@users.noreply.github.com> Closes: #30607 Signed-off-by: Sam James <sam@gentoo.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
I'm attempting to validate this package on alpha for Gentoo, but I'm getting (inconsistent) failures in the test suite related to the timeout decorator. Usually somewhere between 5 and 15 of the test cases will fail, the failures all look like this:
Steps to reproduce
Extra information
Downstream bug, along with a complete sample log, is here
I have real hardware available that can be used to debug if necessary! Let me know if this would be useful and I can provide access.
The text was updated successfully, but these errors were encountered: