ADR 083: Supporting out of band state sync #9651

cmwaters · 2022-11-01T10:17:49Z

This ADR is an implementation write up based around the following issue: #4642

The actual implementation can be found in this PR: #9541

PR checklist

Tests written/updated, or no tests needed
CHANGELOG_PENDING.md updated, or no changelog entry needed
Updated relevant documentation (docs/) and code comments, or no
documentation updates needed

* abci: clarify what abci stands for * link to abci type protos.

* abci: clarify connection use in-process * Update abci.md * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * invert abci explanations * lint++ * lint++ * lint++ * lint++ Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

…package name (#344)

* abci: points of clarification ahead of v0.1.0 * lint++ * typo * lint++ * double word score * grammar * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * pr feedback * wip * update non-zero status code docs * fix event description * update CheckTx description Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* Update supervisor_001_draft.md If the only node in the *FullNodes* set is the primary, that was just deemed faulty, we can't find honest primary. * Update supervisor_001_draft.md

* wip * wip * wip * remove comments in favor of gh comments * wip * udpates to language, should must etc * Apply suggestions from code review Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * remove tendermint cache description Co-authored-by: M. J. Fromberger <fromberger@interchain.io>

* add missed proto files * add abci changes * rename blockchain to blocksync * Update proto/tendermint/abci/types.proto Co-authored-by: Callum Waters <cmwaters19@gmail.com> Co-authored-by: Callum Waters <cmwaters19@gmail.com>

…ions Tendermint specification version 0.7.1

* [Rebased to v0.34.x] abci: PrepareProposal (#6544) * fixed cherry-pick * proto changes * make proto-gen * UT fixes * generate Client directive * mockery * App fixes * Disable 'modified tx' hack * mockery * Make format * Fix lint Co-authored-by: Marko <marbar3778@yahoo.com>

This test would fail if run with "go test -count=2" because it uses a fixed address and was not closing the server, so the subsequent run could not bind to the address. While closing the server is correct, it would probably be better if the API was able to report the bound address so that we could pass "localhost:0" for an anonymous port. But I am currently focusing on test cleanup, not ready to change any existing APIs.

* libs/pubsub/query: specify peg version in go.mod The code to generate the pubsub queries was dependent on an unspecified version of the peg tool. This brings peg into go.mod so it is on a fixed version. This should also enable dependabot to notify us of future updates to peg. The version of query.peg.go generated from the current version of peg correctly contains the special "Code generated by..." line to indicate to other tools that the file is automatically generated and should therefore be excluded from linters, etc. I removed the make target as there were no git grep results referencing "gen_query_parser"; directly running "go generate" is a reasonable expectation in Go projects. Now that "go run" is module aware, I would typically use "go run" inside the go:generate directive, but in this case we go build to a gitignore-d directory in order to work around the nondeterministic output detailed in pointlander/peg#129. * libs/pubsub/query: check error from (*QueryParser).Init() The newly generated peg code returns an error from Init(); the previous version was niladic. Co-authored-by: Sam Kleinman <garen@tychoish.com>

* Update CODEOWNERS to use teams Update the `CODEOWNERS` file to use the @tendermint/tendermint-engineering and @tendermint/tendermint-research teams as opposed to adding people one by one. This makes repository administration somewhat easier to manage, especially when onboarding/offboarding people. Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add Ethan as superuser Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Update Makefile with changes from #7372 Signed-off-by: Thane Thomson <connect@thanethomson.com> * Sync main GitHub config with master and update Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove unnecesary dot folders Signed-off-by: Thane Thomson <connect@thanethomson.com> * Sync dotfiles Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove unused Jepsen tests for now Signed-off-by: Thane Thomson <connect@thanethomson.com> * tools: remove k8s (#6625) Remove mintnet as discussed on team call. closes #1941 * Restore nightly fuzz testing of P2P addrbook and pex Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix YAML lints Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix YAML formatting nits Signed-off-by: Thane Thomson <connect@thanethomson.com> * More YAML nits Signed-off-by: Thane Thomson <connect@thanethomson.com> * github: fix linter configuration errors and occluded errors (#6400) * Minor fixes to OpenAPI spec to sync with structs on main Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove .github/auto-comment.yml - does not appear to be used Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add issue config with link to discussions Signed-off-by: Thane Thomson <connect@thanethomson.com> * Adjust issue/PR templates to suit current process Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove unused RC branch config from release workflow Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix wildcard matching in build jobs config Signed-off-by: Thane Thomson <connect@thanethomson.com> * Document markdownlint config Signed-off-by: Thane Thomson <connect@thanethomson.com> * Restore manual E2E test group config Signed-off-by: Thane Thomson <connect@thanethomson.com> * Document linter workflow with local execution instructions Signed-off-by: Thane Thomson <connect@thanethomson.com> * Document and fix minor nit in Super-Linter markdownlint config Signed-off-by: Thane Thomson <connect@thanethomson.com> * Update .github/ISSUE_TEMPLATE/bug-report.md Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com> * Update pull request template to add language around discussions/issues Signed-off-by: Thane Thomson <connect@thanethomson.com> * .golangci.yml: Deleted commented-out lines Signed-off-by: Thane Thomson <connect@thanethomson.com> * ci: Drop "-2" from e2e-nightly-fail workflow Signed-off-by: Thane Thomson <connect@thanethomson.com> * Address triviality concern in PR template Signed-off-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Sam Kleinman <garen@tychoish.com> Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>

Import the readme, contributing guidelines, code of conduct, security guide and releases guide. Format all of these documents, replacing references to `master` by references to `main`, and use Markdown link references instead of embedded links wherever it improves legibility. Signed-off-by: Thane Thomson <connect@thanethomson.com>

Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/substack/minimist/commit/7efb22a518b53b06f5b02a1038a88bd6290c2846"><code>7efb22a</code></a> 1.2.6</li> <li><a href="https://github.com/substack/minimist/commit/ef88b9325f77b5ee643ccfc97e2ebda577e4c4e2"><code>ef88b93</code></a> security notice for additional prototype pollution issue</li> <li><a href="https://github.com/substack/minimist/commit/c2b981977fa834b223b408cfb860f933c9811e4d"><code>c2b9819</code></a> isConstructorOrProto adapted from PR</li> <li><a href="https://github.com/substack/minimist/commit/bc8ecee43875261f4f17eb20b1243d3ed15e70eb"><code>bc8ecee</code></a> test from prototype pollution PR</li> <li>See full diff in <a href="https://github.com/substack/minimist/compare/1.2.5...1.2.6">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=minimist&package-manager=npm_and_yarn&previous-version=1.2.5&new-version=1.2.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/tendermint/tendermint/network/alerts). </details>

Bumps [github.com/bufbuild/buf](https://github.com/bufbuild/buf) from 1.4.0 to 1.7.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/bufbuild/buf/releases">github.com/bufbuild/buf's releases</a>.</em></p> <blockquote> <h2>v1.7.0</h2> <ul> <li>Support protocol and encoding client options based on content-type in Studio Agent.</li> <li>Add <code>--draft</code> flag to <code>buf push</code>.</li> <li>Add <code>buf beta registry draft {list,delete}</code> commands.</li> </ul> <h2>v1.6.0</h2> <ul> <li>Fix issue where <code>// buf:lint:ignore</code> comment ignores did not work for the <code>ENUM_FIRST_VALUE_ZERO</code> rule.</li> <li>Add <code>buf beta studio-agent</code> command to support the upcoming Buf Studio.</li> </ul> <h2>v1.5.0</h2> <ul> <li>Upgrade to <code>protoc</code> 3.20.1 support.</li> <li>Fix an issue where <code>buf</code> would fail if two or more roots contained a file with the same name, but with different file types (i.e. a regular file vs. a directory).</li> <li>Fix check for <code>PACKAGE_SERVICE_NO_DELETE</code> to detect deleted services.</li> <li>Remove <code>buf beta registry track</code>.</li> <li>Remove <code>buf beta registry branch</code>.</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/bufbuild/buf/blob/main/CHANGELOG.md">github.com/bufbuild/buf's changelog</a>.</em></p> <blockquote> <h2>[v1.7.0] - 2022-06-27</h2> <ul> <li>Support protocol and encoding client options based on content-type in Studio Agent.</li> <li>Add <code>--draft</code> flag to <code>buf push</code>.</li> <li>Add <code>buf beta registry draft {list,delete}</code> commands.</li> </ul> <h2>[v1.6.0] - 2022-06-21</h2> <ul> <li>Fix issue where <code>// buf:lint:ignore</code> comment ignores did not work for the <code>ENUM_FIRST_VALUE_ZERO</code> rule.</li> <li>Add <code>buf beta studio-agent</code> command to support the upcoming Buf Studio.</li> </ul> <h2>[v1.5.0] - 2022-05-30</h2> <ul> <li>Upgrade to <code>protoc</code> 3.20.1 support.</li> <li>Fix an issue where <code>buf</code> would fail if two or more roots contained a file with the same name, but with different file types (i.e. a regular file vs. a directory).</li> <li>Fix check for <code>PACKAGE_SERVICE_NO_DELETE</code> to detect deleted services.</li> <li>Remove <code>buf beta registry track</code>.</li> <li>Remove <code>buf beta registry branch</code>.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/bufbuild/buf/commit/028fdd557df6aebe93ae4682fc4be62b490a4e60"><code>028fdd5</code></a> Update to v1.7.0</li> <li><a href="https://github.com/bufbuild/buf/commit/c6a545ecc01f911022150fdc4f8e6862b80b83b9"><code>c6a545e</code></a> Run <code>make upgrade</code> (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1302">#1302</a>)</li> <li><a href="https://github.com/bufbuild/buf/commit/65503feaf39e9bf363d7f111585d0703d0a1e47a"><code>65503fe</code></a> allow Dockerfile.plugin or Dockerfile (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1301">#1301</a>)</li> <li><a href="https://github.com/bufbuild/buf/commit/075e294d6ada86703376a08a7650d2b2e916b53f"><code>075e294</code></a> add CLI support for repository draft commit (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1269">#1269</a>)</li> <li><a href="https://github.com/bufbuild/buf/commit/3d2ca26216612ea55a4525a6e24459281f2114ae"><code>3d2ca26</code></a> ModuleReferenceForString support reference with slash (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1298">#1298</a>)</li> <li><a href="https://github.com/bufbuild/buf/commit/72b51ab8713a0b5fc30bed32281eaad87377b1aa"><code>72b51ab</code></a> rename CuratedPlugin options to default_options (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1300">#1300</a>)</li> <li><a href="https://github.com/bufbuild/buf/commit/65c921b0b7600e5e888d23e2105c8128e164430f"><code>65c921b</code></a> rename opts/options to default_opts/default_options (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1299">#1299</a>)</li> <li><a href="https://github.com/bufbuild/buf/commit/0a8aec22c701d9d9b3b7f81c5a5fb38df6564cf5"><code>0a8aec2</code></a> BSR-379: document version can be optional (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1297">#1297</a>)</li> <li><a href="https://github.com/bufbuild/buf/commit/33033b8fc3a55dbb470ee85c4704225de7ee7ddd"><code>33033b8</code></a> BSR-369: update plugin dependencies to a struct (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1296">#1296</a>)</li> <li><a href="https://github.com/bufbuild/buf/commit/613283806d2df11b681c9e34017fd6f90ca2a521"><code>6132838</code></a> Add webhook delete and list commands to beta and fixup create (<a href="https://github-redirect.dependabot.com/bufbuild/buf/issues/1293">#1293</a>)</li> <li>Additional commits viewable in <a href="https://github.com/bufbuild/buf/compare/v1.4.0...v1.7.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/bufbuild/buf&package-manager=go_modules&previous-version=1.4.0&new-version=1.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

) Bumps [github.com/cosmos/gogoproto](https://github.com/cosmos/gogoproto) from 1.4.2 to 1.4.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/cosmos/gogoproto/releases">github.com/cosmos/gogoproto's releases</a>.</em></p> <blockquote> <h2>v1.4.3</h2> <h3>Bug Fixes</h3> <ul> <li><a href="https://github-redirect.dependabot.com/cosmos/gogoproto/pull/24">#24</a> Fix <code>CompactTextString</code> panics with nested Anys and private fields.</li> <li><a href="https://github-redirect.dependabot.com/cosmos/gogoproto/pull/14">#14</a> Fix <code>make regenerate</code>.</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/cosmos/gogoproto/blob/main/CHANGELOG.md">github.com/cosmos/gogoproto's changelog</a>.</em></p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/cosmos/gogoproto/commit/1c0272593e387aeb68b5470509513a0d22a7420e"><code>1c02725</code></a> fix: fix <code>CompactTextString</code> panics with nested Anys and private fields (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/24">#24</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/b5eb9e6f58ee8c176e79330b0ced07735130a908"><code>b5eb9e6</code></a> fix: fix <code>make regenerate</code> (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/14">#14</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/a14993478f40695898ed8a86931094b6656e8a5d"><code>a149934</code></a> chore(deps): bump bufbuild/buf-setup-action from 1.8.0 to 1.9.0 (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/23">#23</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/f8f123e02214c364f1464849fa1643b45e8f8928"><code>f8f123e</code></a> chore(deps): bump amannn/action-semantic-pull-request from 5.0.1 to 5.0.2 (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/22">#22</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/6884d250505275b9b560465b9424053f7d691afc"><code>6884d25</code></a> chore(deps): bump google.golang.org/grpc from 1.50.0 to 1.50.1 (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/21">#21</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/c0a5a75024187cd13419a04365953a8fdc99fd98"><code>c0a5a75</code></a> chore(deps): bump amannn/action-semantic-pull-request from 5.0.0 to 5.0.1 (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/20">#20</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/c2a18447f3586aa33b4e8fcba5cc1c31b3b93e8a"><code>c2a1844</code></a> chore(deps): bump amannn/action-semantic-pull-request from 4.6.0 to 5.0.0 (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/19">#19</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/aa8e9b1be6eb5738698d270114c93433a5421bc2"><code>aa8e9b1</code></a> chore(deps): bump google.golang.org/grpc from 1.49.0 to 1.50.0 (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/18">#18</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/adee73738b3c97131d3754293934324cc81d0ca8"><code>adee737</code></a> chore(deps): bump amannn/action-semantic-pull-request from 4.5.0 to 4.6.0 (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/17">#17</a>)</li> <li><a href="https://github.com/cosmos/gogoproto/commit/802691b05b6a770823dc471a704f82110a44f6f4"><code>802691b</code></a> chore(deps): bump bufbuild/buf-setup-action from 1.7.0 to 1.8.0 (<a href="https://github-redirect.dependabot.com/cosmos/gogoproto/issues/16">#16</a>)</li> <li>See full diff in <a href="https://github.com/cosmos/gogoproto/compare/v1.4.2...v1.4.3">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/cosmos/gogoproto&package-manager=go_modules&previous-version=1.4.2&new-version=1.4.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

#9729) Bumps [github.com/btcsuite/btcd/btcutil](https://github.com/btcsuite/btcd) from 1.1.2 to 1.1.3. - [Release notes](https://github.com/btcsuite/btcd/releases) - [Changelog](https://github.com/btcsuite/btcd/blob/master/CHANGES) - [Commits](btcsuite/btcd@btcutil/v1.1.2...btcutil/v1.1.3) --- updated-dependencies: - dependency-name: github.com/btcsuite/btcd/btcutil dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

….2 (#9730) Bumps [github.com/btcsuite/btcd/btcec/v2](https://github.com/btcsuite/btcd) from 2.3.1 to 2.3.2. - [Release notes](https://github.com/btcsuite/btcd/releases) - [Changelog](https://github.com/btcsuite/btcd/blob/master/CHANGES) - [Commits](btcsuite/btcd@btcec/v2.3.1...btcec/v2.3.2) --- updated-dependencies: - dependency-name: github.com/btcsuite/btcd/btcec/v2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix abci-cli help * add test * fix script * debugging CI * revert the changes for verifying CI * update pending log Co-authored-by: Sergio Mena <sergio@informal.systems>

Co-authored-by: Thane Thomson <connect@thanethomson.com>

noticed this place is still printing unreadable apphash, convert to hex string --- #### PR checklist - [ ] Tests written/updated, or no tests needed - [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [ ] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed

Since starting off as a wee validator, I've been mystified by the volume of p2p logspam, which often makes it impossible to monitor other tasks. Thus, routine p2p events, have been cast into the land of debug. --- #### PR checklist - [x] Tests written/updated, or no tests needed - [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [x] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed

## NOTE: this pr exclusively runs commands from the makefile found here This PR ONLY runs `make format` ... then `make mockery` Its purpose is to ensure that the review scope of other PR's, which changed .go files and thus triggered the linter that only runs conditionally, have smaller review scopes, and should be merged before: #9738 #9739 #9742 --- #### PR checklist - [x] Tests written/updated, or no tests needed - [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [x] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed

Fixes the tendermint OSS-Fuzz build by adjusting for some infra changes that were made in OSS-Fuzz. --- #### PR checklist - [ ] Tests written/updated, or no tests needed - [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [ ] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed

github-actions · 2022-11-27T00:18:55Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

) This change reduces the number of Precommit messages sent to peers by 50%. During the `ApplyNewRoundStepMessage`, we update the known state of the peer sending us the message. We set the value of `ps.PRS.Precommits` to nil in this method if the peer is entering a new height or round. https://github.com/tendermint/tendermint/blob/34ca3fb4741cdb205a4714d345218a0d5e9fefd0/consensus/reactor.go#L1368 We then assign `ps.PRS.LastCommit = ps.PRS.Precommits` if the peer is entering a new height only - this does not happen during just a round change. This therefore results in `ps.PRS.LastCommit` having the value `nil`. When the `LastCommit` bit field is seen as `nil` in the reactor, an empty bit field is initialized. https://github.com/tendermint/tendermint/blob/34ca3fb4741cdb205a4714d345218a0d5e9fefd0/consensus/reactor.go#L1273 The code responsible for gossiping votes from the previous height uses this `LastCommit` value and, seeing an empty `LastCommit` will resend every `Precommit` from the previous height since it lost the information it previously had detailing which precommits from the previous height the peer had. This can be seen in the code responsible for gossiping precommits from the previous height: https://github.com/tendermint/tendermint/blob/34ca3fb4741cdb205a4714d345218a0d5e9fefd0/consensus/reactor.go#L773 Where this code grabs the, previously `nil`, `LastCommit` bit field: https://github.com/tendermint/tendermint/blob/34ca3fb4741cdb205a4714d345218a0d5e9fefd0/consensus/reactor.go#L1204-L1212 --- #### PR checklist - [ ] Tests written/updated, or no tests needed - [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [ ] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed

williambanfield

This seems like a reasonable way to go about this but I'd be interested to know if there's an actual concrete use of this or a chain asking for this directly. Application developers can provide their own syncing protocols is a nice idea, but without first customers, it's very possible we are going to build something no one will use.

williambanfield · 2022-12-01T20:36:34Z

docs/architecture/adr-083-out-of-band-sync.md

+- Tendermint's block store has a height of 0 (i.e. it's empty and we're not overriding an existing instance. This check may be relaxed in the future) 
+- State sync is enabled and the config is valid (i.e it contains at least 2 rpc endpoints and a trusted header and hash)
+
+The bootstrap protocol will be run directly after the handshake. It (currently) does not require the p2p layer and will work by updating the `StateStore` and `BlockStore`. It should be performed instead of statesync, moving to either blocksync or consensus once completed.


I don't think I understand this protocol. Where are we getting the Commit from? Are we catching up so we have all of the blocks or are we dropping directly into consensus?

through the lightClientStateProvider (a.k.a. via RPC)

williambanfield · 2022-12-01T20:37:21Z

docs/architecture/adr-083-out-of-band-sync.md

+Allow applications to start with a bootstrapped state alongside an empty Tendermint instance using
+a subsystem of the state sync protocol. 


If I'm understanding correct, this is a protocol for Tendermint to sync to the height of the application. It doesn't actually require anything from the application other than for the application to report height higher than Tendermint's height?

Correct (although RPC endpoints in the statesync config do need to be configured)

cmwaters · 2022-12-02T10:27:00Z

This seems like a reasonable way to go about this but I'd be interested to know if there's an actual concrete use of this or a chain asking for this directly. Application developers can provide their own syncing protocols is a nice idea, but without first customers, it's very possible we are going to build something no one will use.

cc: @chillyvee @tac0turtle <- they probably have more context then myself. I think the basic use case in my head (which I think already exists) is that the SDK saves snapshots of state at a certain height. An operator could just send this to another machine, restore that state and then run an empty Tendermint instance which would then verify the block at that height

chillyvee · 2022-12-02T10:59:17Z

This is actually something we're interested in as a subject. Needing another server to restore simple state dumps adds complexity to restores. This is especially the case for chains which are rapidly losing nodes.

Being able to distribute a snapshot directory for a height without the underlying leveldb and bundling the light client results is valuable.

For example, recently we found that on multiple chains, possibly due to a common nginx issue, RPC servers were unable to respond to tendermint. Sysadmins could curl the results manually and supply via a file if the mechanism was available.

Also during debug, it is nice to be able to restore a particular snapshot rather than a randomly chosen one via p2p discovery.

tac0turtle · 2022-12-02T13:49:43Z

Nomic also asked about this back when state sync was written. I have a sync with them later today and can ask about it. On the sdk front we have attempted to do this feature without tendermint changes but weren't able to complete it. I tried doing it outside the binary but couldn't because of default tendermint. Generally I think close to 100% of node operators would use this feature.

faddat · 2022-12-05T11:19:59Z

We would use this at notional to make state sync easier for us and others, as part of cosmosia

https://GitHub.com/notional-labs/cosmosia

assafmo · 2022-12-05T11:36:10Z

Secret Network is very interested in this as well! Due to our hardware requirements it's harder to run nodes, therefore when a node fails it's harder to re-sync it. We've been also wanting to add a mode to allow getting a state diff for a block to allow applying that block by just verifying the headers and without actually executing it.

toschdev · 2022-12-05T14:58:50Z

I would find this useful

cmwaters and others added 30 commits August 3, 2021 11:14

evidence: add section explaining evidence (#324)

8029cf7

statesync: new messages for gossiping consensus params (#328)

72adbf9

rpc: update peer format in specification in NetInfo operation (#331)

1e985f6

Update supervisor_001_draft.md (#334)

4c3339a

core: text cleanup (#332)

e44ab95

abci: clarify what abci stands for (#336)

26ee62a

* abci: clarify what abci stands for * link to abci type protos.

proto: move proto files under the correct directory related to their …

41e6812

…package name (#344)

Update supervisor_001_draft.md (#333)

f793752

* Update supervisor_001_draft.md If the only node in the *FullNodes* set is the primary, that was just deemed faulty, we can't find honest primary. * Update supervisor_001_draft.md

light: update initialization description (#320)

e2a038e

proto: add tendermint go changes (#349)

fa2ccc8

* add missed proto files * add abci changes * rename blockchain to blocksync * Update proto/tendermint/abci/types.proto Co-authored-by: Callum Waters <cmwaters19@gmail.com> Co-authored-by: Callum Waters <cmwaters19@gmail.com>

Merge tag 'v0.7.1' of github.com:tendermint/spec into cal/spec-migrat…

1ad0683

…ions Tendermint specification version 0.7.1

fix mockery generation script (#9094)

a11b774

Merge branch 'main' into cal/spec-migrations

0323bd5

spec: migrate spec at v0.7.1. into main (#9098)

a7049ed

Fix unsafe-reset-all for working with default home (#9103)

18c7e3a

docs: migrate adrs and rfcs across from master (#9115)

e32b86b

build(deps): Bump bufbuild/buf-setup-action from 1.6.0 to 1.7.0 (#9131)

476f8fa

build(deps): Bump url-parse from 1.5.7 to 1.5.10 in /docs (#9135)

13da93a

chore: lint repo (use american english) (#9144)

49ec3b9

dependabot bot and others added 9 commits November 22, 2022 17:21

cli: fix abci help (#9717)

2d036c5

* fix abci-cli help * add test * fix script * debugging CI * revert the changes for verifying CI * update pending log Co-authored-by: Sergio Mena <sergio@informal.systems>

Delete release_notes.md (#9753)

3e76698

Co-authored-by: Thane Thomson <connect@thanethomson.com>

github-actions bot added the stale for use by stalebot label Nov 27, 2022

williambanfield and others added 2 commits November 28, 2022 20:13

Merge branch 'main' into cal/local-sync-adr

cfd663c

github-actions bot removed the stale for use by stalebot label Nov 29, 2022

williambanfield reviewed Dec 1, 2022

View reviewed changes

thanethomson added the S:wip Work in progress (prevents stalebot from automatically closing) label Dec 1, 2022

sergio-mena self-assigned this Dec 5, 2022

This was referenced Dec 21, 2022

State sync from local snapshot #9946

Closed

State sync from local snapshot: Understand problem, explore solutions #9947

Closed

sergio-mena removed their assignment Dec 21, 2022

This was referenced Dec 23, 2022

State sync from local snapshot: Understand problem, explore solutions cometbft/cometbft#28

Closed

State sync from local snapshot: Understand problem, explore solutions cometbft/cometbft#57

Closed

jaekwon force-pushed the main branch from a9feb1c to 64747b2 Compare January 5, 2023 21:18

cason mentioned this pull request Apr 18, 2023

State sync from local snapshot: Implementation cometbft/cometbft#29

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ADR 083: Supporting out of band state sync #9651

ADR 083: Supporting out of band state sync #9651

cmwaters commented Nov 1, 2022 •

edited by thanethomson

github-actions bot commented Nov 27, 2022

williambanfield left a comment

williambanfield Dec 1, 2022

cmwaters Dec 2, 2022

williambanfield Dec 1, 2022

cmwaters Dec 2, 2022

cmwaters commented Dec 2, 2022

chillyvee commented Dec 2, 2022

tac0turtle commented Dec 2, 2022

faddat commented Dec 5, 2022

assafmo commented Dec 5, 2022

toschdev commented Dec 5, 2022

		Allow applications to start with a bootstrapped state alongside an empty Tendermint instance using
		a subsystem of the state sync protocol.

ADR 083: Supporting out of band state sync #9651

Are you sure you want to change the base?

ADR 083: Supporting out of band state sync #9651

Conversation

cmwaters commented Nov 1, 2022 • edited by thanethomson

PR checklist

github-actions bot commented Nov 27, 2022

williambanfield left a comment

Choose a reason for hiding this comment

williambanfield Dec 1, 2022

Choose a reason for hiding this comment

cmwaters Dec 2, 2022

Choose a reason for hiding this comment

williambanfield Dec 1, 2022

Choose a reason for hiding this comment

cmwaters Dec 2, 2022

Choose a reason for hiding this comment

cmwaters commented Dec 2, 2022

chillyvee commented Dec 2, 2022

tac0turtle commented Dec 2, 2022

faddat commented Dec 5, 2022

assafmo commented Dec 5, 2022

toschdev commented Dec 5, 2022

cmwaters commented Nov 1, 2022 •

edited by thanethomson