-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Getting Lost connection to MySQL server during query (Mysql2::Error::ConnectionError) #117
Comments
Hey @nimir, sorry for the trouble! Could you let me know the output from this in your MySQL console? show global variables like 'max_connections'; And could you also copy your |
Not at all @rosa , and thank you and the team for the great work mysql> show global variables like 'max_connections';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 151 |
+-----------------+-------+
1 row in set (0.03 sec) My default: &default
adapter: mysql2
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
username: REDACTED
password: REDACTED
collation: utf8mb4_unicode_ci
development:
<<: *default
database: dmydb_development
test:
<<: *default
database: mydb_test
production:
<<: *default
host: <%= ENV["PRODUCTION_DB_HOST"] %>
database: mydb_production
username: <%= ENV["PRODUCTION_DB_USER"] %>
password: <%= ENV["PRODUCTION_DB_PASSWORD"] %> Interacting with the database works fine via the |
So strange. I haven't been able to reproduce this with your same configuration (except I'm running MySQL 8 in Docker via the 8.0.31 image). I imagine it matches this, but what do you get for the following MySQL settings?
|
Weird that I got same output: mysql> show global variables like 'net_read_timeout';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| net_read_timeout | 30 |
+------------------+-------+
1 row in set (0.07 sec)
mysql> show global variables like 'net_write_timeout';
+-------------------+-------+
| Variable_name | Value |
+-------------------+-------+
| net_write_timeout | 60 |
+-------------------+-------+
1 row in set (0.01 sec)
mysql> show global variables like 'connect_timeout';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| connect_timeout | 10 |
+-----------------+-------+
1 row in set (0.00 sec) |
I can add more debug code to |
Thank you 🙏 Let's try to see if we get anything useful in the development logs. Could you clear your |
Ok, here is the fresh log file:
|
Ahh, nothing useful there 😞 Thanks for trying that. I imagine you got this after starting fresh, but just in case: do you get any errors when running SolidQueue::Pause.all.pluck(:queue_name)
SolidQueue::ReadyExecution.where("queue_name LIKE ?", "dc_development_%").distinct(:queue_name).pluck(:queue_name) from your Rails console? |
No errors ❯ rails c
Loading development environment (Rails 7.1.2)
irb(main):001> SolidQueue::Pause.all.pluck(:queue_name)
=> []
irb(main):002> SolidQueue::ReadyExecution.where("queue_name LIKE ?", "dc_development_%").distinct(:queue_name).pluck(:queue_name)
=> [] |
Thanks for checking that 🙏 Ok, let's rule out forking here causing issues. Instead of starting Solid Queue via
could you start just a single worker like this? In a Rails console:
And then try enqueuing the job and see if you get the error. To stop it, just do:
|
Yes, starting the worker without forking worked like a charm and I didn't get any errors.
So, how can we debug the forking part to get more useful insight on the cause? |
Hey @nimir! So sorry for the silence and delay in replying after your last comment. I tried to come up with a good way to debug this but in the end, I had to work on other unrelated stuff and lost track of it. I'm quite puzzled about why this might be happening. Did you figure anything else on your own or are you still having the exact same problem? |
Hi @rosa , no worries and I appreciate your support I thought the issue might be with my version of MySQL so tried downgrading to v8.0 and also testing on fresh apps just to eliminate the possibility I am making something wrong in the configurations but I had no luck. Unfortunately I have limited time to work on this at the time! |
Hi @rosa
|
I found a work around this morning but I am not sure how sustainable it is since it gives a depreciation warning Setting
|
The Update: |
Decided to give solid_queue a try today and I'm running into the same issue that @nimir described. Env and Versions:Ruby 3.2.2
Looking at my development.log file I can see that I get about 200 SolidQueue::ReadyExecution Pluck commands after starting solid_queue (using 'bundle exec rake solid_queue:start') before I see the first Mysql2::Error::ConnectionError error. |
Yesterday, I also conducted some tests. In my case, when I extended the worker's polling interval period to seconds, the issue did not occur. I'm not sure of the exact cause, but I hope this helps someone :D development:
dispatchers:
- polling_interval: 1
batch_size: 500
workers:
- queues: "dc_development_*"
threads: 2
processes: 1
polling_interval: 1 # ⬅️ 0.1 to 1
Changed
|
@hahwul thanks for sharing the workaround, unfortunately it didn't work for me ... It just delays the error due to the extended polling interval i suppose! |
Experiencing something similar on ruby 3.2.2 and rails 7.1.3 with all of the above pointers tried
|
Just my two cents. I got this error on a M1 Pro machine, and this is what I did.
There are no more connection issues now. 🚀 |
Is there any fix yet? @rosa this seems to happen when starting solid queue but there are no Jobs or too few (< 5) Jobs to claim |
I haven't been able to reproduce this in any way, @brincau, so no, I don't have a fix for it, I'm afraid. |
I've seen the same thing: starting a solid queue is fine, but when running a rake command, it generates a MySQL connection error. The Trilogy adaptor seems to work better, but it has some issues when running GitHub Actions.
|
Env and Versions:
3.3.0
8.2.0 for macos13.5 on arm64 (Homebrew)
mysql2 V0.5.5
7.1.2
0.1.2
Solid_Queue Configurations:
config/solid_queue.yml
:Steps to reproduce:
bundle exec rake solid_queue:start
Solid_Queue Log Trace:
The text was updated successfully, but these errors were encountered: