[main-] fix rare duplication of replay cmds #2392
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When replaying command files, in rare circumstances, some commands can be run twice.
The cause is that
reload()
is called twice for the command file. Once insideeval_vd()
, and again aftereval_vd()
finishes:visidata/visidata/main.py
Line 49 in 3ad7a3d
visidata/visidata/main.py
Line 352 in 3ad7a3d
Both
reload()
calls can run at the same time in two different threads, becauseTableSheet.reload()
has the@asyncthread
decorator. Both threads add toself.rows
viaself.addRow(r)
inTableSheet.loader()
, so command rows can get added twice.The specific events that must happen to trigger the bug are:
reload()
call must runTableSheet.loader()
, past the line ofself.rows = []
visidata/visidata/sheets.py
Line 282 in 3ad7a3d
reload()
finishes adding rows, the secondreload()
call must runTableSheet.loader()
, past the line ofself.rows = []
reload()
call finishes adding rows, the first call continues adding rowsI made a pathological loader that triggers these conditions every time, by loading only two command rows per second : 5c97a84. To test with that loader, do
touch row_duplication.slow; vd -p dupes.slow
, and see that the dupes_vd sheet contains more than 5 commands. The loader only generates 5 commands, but in my tests, the demo code duplicates all of them, for a total of 10 commands.The affected versions of Visidata could be multiple versions since Sep 2019, as of 330117f. But on my system, I have not yet found a way to trigger it for replay files that are short (under 100 lines). For much longer files, it does happen every time. For example, it happens if I add ~5000 lines of comments to a short
.vdj
file. But for short files with no comments (~10 lines), I couldn't trigger it at all by adding CPU/IO/disk stress via the Ubuntustress
binary, in ~500 tries.