Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invalid memory address or nil pointer dereference in (*SortedRowContainer).Add #52768

Closed
GaranR opened this issue Apr 19, 2024 · 10 comments · Fixed by #53237
Closed

invalid memory address or nil pointer dereference in (*SortedRowContainer).Add #52768

GaranR opened this issue Apr 19, 2024 · 10 comments · Fixed by #53237

Comments

@GaranR
Copy link

GaranR commented Apr 19, 2024

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

First execute the following valid.sql
valid.txt

Then a crash occurs when executing the error.sql below
error3.txt

2. What did you expect to see? (Required)

Expect no crashes

3. What did you see instead (Required)

runtime error: index out of range [0] with length 0
invalid memory address or nil pointer dereference

tidb.log:

[2024/04/17 11:57:26.208 +00:00] [ERROR] [shuffle.go:259] ["shuffle panicked"] [error="runtime error: index out of range [0] with length 0"] [stack="github.com/pingcap/tidb/pkg/executor.recoveryShuffleExec
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:259
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:388
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:914
runtime.goPanicIndex
	/usr/local/go/src/runtime/panic.go:114
github.com/pingcap/tidb/pkg/util/chunk.(*Column).AppendInt64
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/column.go:247
github.com/pingcap/tidb/pkg/util/chunk.(*Chunk).AppendInt64
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/chunk.go:489
github.com/pingcap/tidb/pkg/executor/aggfuncs.(*rank).AppendFinalResult2Chunk
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/aggfuncs/func_rank.go:65
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).produce
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:404
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:155
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:398"]
[2024/04/17 11:57:26.208 +00:00] [ERROR] [shuffle.go:259] ["shuffle panicked"] [error="runtime error: index out of range [0] with length 0"] [stack="github.com/pingcap/tidb/pkg/executor.recoveryShuffleExec
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:259
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:388
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:914
runtime.goPanicIndex
	/usr/local/go/src/runtime/panic.go:114
github.com/pingcap/tidb/pkg/util/chunk.(*Column).AppendInt64
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/column.go:247
github.com/pingcap/tidb/pkg/util/chunk.(*Chunk).AppendInt64
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/chunk.go:489
github.com/pingcap/tidb/pkg/executor/aggfuncs.(*rank).AppendFinalResult2Chunk
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/aggfuncs/func_rank.go:65
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).produce
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:404
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:155
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:398"]
[2024/04/17 11:57:26.208 +00:00] [ERROR] [shuffle.go:259] ["shuffle panicked"] [error="runtime error: invalid memory address or nil pointer dereference"] [stack="github.com/pingcap/tidb/pkg/executor.recoveryShuffleExec
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:259
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:388
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:914
runtime.panicmem
	/usr/local/go/src/runtime/panic.go:261
runtime.sigpanic
	/usr/local/go/src/runtime/signal_unix.go:861
github.com/pingcap/tidb/pkg/util/chunk.(*SortedRowContainer).Add
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/row_container.go:615
github.com/pingcap/tidb/pkg/executor.(*SortExec).fetchRowChunks
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/sort.go:210
github.com/pingcap/tidb/pkg/executor.(*SortExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/sort.go:117
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).fetchChild
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:211
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).getRowsInPartition
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:182
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:127
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:398"]
[2024/04/17 11:57:26.208 +00:00] [ERROR] [shuffle.go:259] ["shuffle panicked"] [error="runtime error: invalid memory address or nil pointer dereference"] [stack="github.com/pingcap/tidb/pkg/executor.recoveryShuffleExec
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:259
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:388
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:914
runtime.panicmem
	/usr/local/go/src/runtime/panic.go:261
runtime.sigpanic
	/usr/local/go/src/runtime/signal_unix.go:861
github.com/pingcap/tidb/pkg/util/chunk.(*SortedRowContainer).Add
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/row_container.go:615
github.com/pingcap/tidb/pkg/executor.(*SortExec).fetchRowChunks
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/sort.go:210
github.com/pingcap/tidb/pkg/executor.(*SortExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/sort.go:117
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).fetchChild
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:211
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).getRowsInPartition
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:182
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:127
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:398"]
[2024/04/17 11:57:26.208 +00:00] [ERROR] [shuffle.go:259] ["shuffle panicked"] [error="runtime error: index out of range [19] with length 0"] [stack="github.com/pingcap/tidb/pkg/executor.recoveryShuffleExec
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:259
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:388
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:914
runtime.goPanicIndex
	/usr/local/go/src/runtime/panic.go:114
github.com/pingcap/tidb/pkg/util/chunk.(*SortedRowContainer).GetSortedRowAndAlwaysAppendToChunk
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/row_container.go:637
github.com/pingcap/tidb/pkg/executor.(*SortExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/sort.go:133
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).fetchChild
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:211
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).getRowsInPartition
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:182
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:127
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:398"]
[2024/04/17 11:57:26.208 +00:00] [WARN] [session.go:2245] ["compile SQL failed"] [conn=408946062] [session_alias=] [error="runtime error: index out of range [0] with length 0"] [SQL="(check error.sql above)"]
[2024/04/17 11:57:26.209 +00:00] [INFO] [conn.go:1124] ["command dispatched failed"] [conn=408946062] [session_alias=] [connInfo="id:408946062, addr:10.0.2.1:49730 status:10, collation:utf8mb4_0900_ai_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="(check error.sql above)"] [txn_mode=PESSIMISTIC] [timestamp=449145745225285634] [err="runtime error: index out of range [0] with length 0
github.com/pingcap/tidb/pkg/executor.recoveryShuffleExec
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:258
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run.func1
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:388
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:914
runtime.goPanicIndex
	/usr/local/go/src/runtime/panic.go:114
github.com/pingcap/tidb/pkg/util/chunk.(*Column).AppendInt64
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/column.go:247
github.com/pingcap/tidb/pkg/util/chunk.(*Chunk).AppendInt64
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/util/chunk/chunk.go:489
github.com/pingcap/tidb/pkg/executor/aggfuncs.(*rank).AppendFinalResult2Chunk
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/aggfuncs/func_rank.go:65
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).produce
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:404
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/pipelined_window.go:155
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/internal/exec/executor.go:283
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run
	/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/tidb/pkg/executor/shuffle.go:398
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1650"]

4. What is your TiDB version? (Required)

+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_version()                                                                                                                                                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Release Version: v7.5.1
Edition: Community
Git Commit Hash: 7d16cc79e81bbf573124df3fd9351c26963f3e70
Git Branch: heads/refs/tags/v7.5.1
UTC Build Time: 2024-02-27 14:28:32
GoVersion: go1.21.6
Race Enabled: false
Check Table Before Drop: false
Store: tikv |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

We are the BASS team from the School of Cyber Science and Technology at Beihang University. Our main focus is on system software security, operating systems, and program analysis research, as well as the development of automated program testing frameworks for detecting software defects. Using our self-developed database vulnerability testing tool, we have identified the above-mentioned vulnerabilities in TiDB that may lead to database crashes.

@GaranR GaranR added the type/bug This issue is a bug. label Apr 19, 2024
@GaranR GaranR changed the title index out of range [0] with length 0 in (*Column).AppendInt64 (*SortedRowContainer).Add (*SortedRowContainer).GetSortedRowAndAlwaysAppendToChunk invalid memory address or nil pointer dereference in (*SortedRowContainer).Add Apr 19, 2024
@yibin87
Copy link
Contributor

yibin87 commented Apr 28, 2024

Reproduced with latest tidb:
Release Version: v8.2.0-alpha-6-g8f138d5113
Edition: Community
Git Commit Hash: 8f138d5
Git Branch: HEAD
UTC Build Time: 2024-04-22 08:00:01
GoVersion: go1.21.6
Race Enabled: false
Check Table Before Drop: false
Store: tikv

@yibin87
Copy link
Contributor

yibin87 commented Apr 28, 2024

/label affect-8.1

Copy link

ti-chi-bot bot commented Apr 28, 2024

@yibin87: The label(s) affect-8.1 cannot be applied. These labels are supported: fuzz/sqlancer, challenge-program, compatibility-breaker, first-time-contributor, contribution, good first issue, correctness, duplicate, proposal, security, ok-to-test, needs-ok-to-test, needs-more-info, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.5, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.5, needs-cherry-pick-release-8.1, affects-5.4, affects-6.1, affects-6.5, affects-7.1, affects-7.5, affects-8.1, may-affects-5.4, may-affects-6.1, may-affects-6.5, may-affects-7.1, may-affects-7.5, may-affects-8.1.

In response to this:

/label affect-8.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@yibin87
Copy link
Contributor

yibin87 commented Apr 28, 2024

/label affects-8.1

@yibin87
Copy link
Contributor

yibin87 commented Apr 28, 2024

/label affects-7.5

@xzhangxian1008
Copy link
Contributor

minimal reproduce sql:

select
    ref_15.c_wzmb0 as c0
from
    t_bhze93f as ref_15
where (EXISTS (
    select
        RANK() over win_fru73h9zyn as c8
    from
        t_xf1at0 as ref_16
    window win_fru73h9zyn as (partition by (ref_16.c__icnfdo_ is NULL), ref_16.c_ldqj5xa)));

@xzhangxian1008
Copy link
Contributor

tidb version: 09c8f96

Not an execution bug, but optimizer.

panic stack:

	/DATA/disk3/xzx/go/src/runtime/panic.go:114\ngithub.com/pingcap/tidb/pkg/util/chunk.(*Column).AppendInt64
	/DATA/disk3/xzx/tidb/pkg/util/chunk/column.go:258\ngithub.com/pingcap/tidb/pkg/util/chunk.(*Chunk).AppendInt64
	/DATA/disk3/xzx/tidb/pkg/util/chunk/chunk.go:505\ngithub.com/pingcap/tidb/pkg/executor/aggfuncs.(*rank).AppendFinalResult2Chunk
	/DATA/disk3/xzx/tidb/pkg/executor/aggfuncs/func_rank.go:64\ngithub.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).produce
	/DATA/disk3/xzx/tidb/pkg/executor/pipelined_window.go:403\ngithub.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).Next
	/DATA/disk3/xzx/tidb/pkg/executor/pipelined_window.go:154\ngithub.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/DATA/disk3/xzx/tidb/pkg/executor/internal/exec/executor.go:410\ngithub.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run
	/DATA/disk3/xzx/tidb/pkg/executor/shuffle.go:409\nruntime.goexit
	/DATA/disk3/xzx/go/src/runtime/asm_amd64.s:1650

After debug, we can find that the type of column appended by AppendFinalResult2Chunk is a variable length type, but we append it with a fixed length int64 variable which causes panic.

The appended chunk named as chk is actually the resultChk generated by PipelinedWindowExec at pipelined_window.go:L221. e.RetFieldTypes() determines the column attribute of resultChk. Window executor's RetFieldType is set by plannercore.PhysicalWindow.Schema() at builder.go:L4679 when window executor is built and the plannercore.PhysicalWindow is generated at expression_rewriter.go:L1044.

@xzhangxian1008
Copy link
Contributor

/sig planner

@ti-chi-bot ti-chi-bot bot added the sig/planner SIG: Planner label May 11, 2024
@xzhangxian1008
Copy link
Contributor

/remove-sig execution

@ti-chi-bot ti-chi-bot bot removed the sig/execution SIG execution label May 11, 2024
@winoros
Copy link
Member

winoros commented May 13, 2024

img_v3_02ar_8959d11b-7399-42f9-b5a7-42b6c76fbc6g
img_v3_02ar_caf1d1de-3c40-4f75-aa83-404dd4f8e0fg
image

The output columns of the top plan are directly passed as the parentUsedCols and then the parentUsedCols is modified during the pruning. So the output columns changed unexpectedly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants