Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add index blocked due to tidb schema version can not sync and caused that other ddl job always queueing after injection pd leader io delay #53073

Closed
Lily2025 opened this issue May 7, 2024 · 2 comments · Fixed by #53234
Assignees
Labels
affects-8.1 component/ddl This issue is related to DDL of TiDB. severity/major type/bug This issue is a bug.

Comments

@Lily2025
Copy link

Lily2025 commented May 7, 2024

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

1、enable tidb_enable_dist_task and global sort
2、run sysbench
3、add index for sbtest1 and sbtest2 concurrently
4、inject pd leader io delay 5s

operator logs:
the status of ddl job is not synced after 1h0m0s (now: 2024-05-06 07:53:23, jobId: 936, job type: add index /* ingest cloud /, state: running)
operatorLogs:
[2024-05-06 06:53:10] ###### start adding index
ALTER TABLE sbtest2 ADD INDEX index_test_1714949590002(c)
[2024-05-06 06:53:10] ###### wait for ddl job finish
[2024-05-06 07:53:23] ###### wait for ddl job finish timeout(1h0m0s)
select job_id, job_type, state from information_schema.ddl_jobs where query = 'ALTER TABLE sbtest2 ADD INDEX index_test_1714949590002(c)'
jobId: 936, job type: add index /
ingest cloud */, state: running

tidb logs:
tidb-0.tar.gz
tidb-1.tar.gz

2. What did you expect to see? (Required)

add index can success

3. What did you see instead (Required)

add index blocked and caused that other ddl job always queueing

4. What is your TiDB version? (Required)

./tidb-server -V
Release Version: v8.2.0-alpha
Edition: Community
Git Commit Hash: 600b2ed
Git Branch: heads/refs/tags/v8.2.0-alpha
UTC Build Time: 2024-05-01 11:46:22
GoVersion: go1.21.6
Race Enabled: false
Check Table Before Drop: false
Store: unistore

@Lily2025 Lily2025 added the type/bug This issue is a bug. label May 7, 2024
@Lily2025
Copy link
Author

Lily2025 commented May 7, 2024

/type bug
/assign tangenta

@tangenta
Copy link
Contributor

This issue affects v6.5, v7.1 and v7.5. However, the fix includes upgrade operation, which is not easy to cherry-pick backwards. Let's only fix this in v8.1.

@Lily2025 Lily2025 changed the title add index blocked and caused that other ddl job always queueing after injection pd leader io delay add index blocked due to tidb schema version can not sync and caused that other ddl job always queueing after injection pd leader io delay May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-8.1 component/ddl This issue is related to DDL of TiDB. severity/major type/bug This issue is a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants