git workflow evmone coverage script

[winsvega]

🗒️ Description

git job to measure coverage of ported tests
if a new line introduced to converted_ethereum_tests, the script will run coverage and make an artifact

demonstration: #504

What does it do:

Evmone docker image

The main script is in this repo: https://github.com/winsvega/evmtest_coverage
Using ./dcoverage.sh build command a docker image of evmone in coverage mode is built.
As well as the coverage software is installed in the docker container.
A sanpshot of the container is currently hosted at http://retesteth.ethdevops.io/release/evmone.tar

Coverage script

(requires root to assign generated file permissions)

The script accpets 2 arguments.
testBASE - folder with the tests we had before coverage
testPATCH - folder with the tests we generated after converting existed .json tests fillers into .py

sudo ./dcoverage.sh --base=testBASE --patch=testPATCH [--driver=native|retesteth]

The folder ./coverage/evmone_coverage.sh is mounted inside the docker and execute instructions with evmone.
it automatically sorts state tests and blockchain tests and runs them on evmone using retesteth (t8n interface). (can be changed to evmone native test tools using the flag --driver=native, note that the current image support only retesteth as of now)

The script produces coverage report of testBase, testPatch and a diff of testPatch on top of testBase
Thus we can see what changed after we swithed from testBase to testPatch. The coverage diff output log is written to
./coverage/difflog.txt
Finally ./check.sh analyzes the log file and if it has the lines or function paths that lost it's coverage in testPatch version of the tests, it yells an error.
The result information is uploaded as a git artifact by the workflow script, so we can see what got lost.

🔗 Related Issues

Feauture

✅ Checklist

• All: Set appropriate labels for the changes.
• All: Considered squashing commits to improve commit history.
• All: Added an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
• Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

[marioevz]

Some comments.

[marioevz]

Suggested change

	- name: Checkout code
	uses: actions/checkout@v3
	with:
	ref: ${{ github.event.pull_request.head.ref }} # Checks out the PR branch
	fetch-depth: 0 # Necessary to fetch all history for diff
	- name: Checkout code
	uses: actions/checkout@v3

I think this action by default checks out the current PR's branch, so the ref is unnecessary.
Also, we seem to be doing this twice at the start of the steps? (Line 12).

[winsvega]

there must be 2 separate steps. one is to fetch branches, and enother one to restore cache.
never used the git cache in this context, but it supposed to keep the docker file and only update it when the hash file is changed.