[Bug]: Running app from Docker image doesn't work

[maximedupre]

Describe the bug

I'm using the default example of the "Getting started" docs.

maximedupre@max-mba mt-agents % docker run --rm -it -v $(pwd):/st-agents autogen_full_img:latest python /st-agents/src/main.py
Traceback (most recent call last):
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/transport/unixconn.py", line 27, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/autogen/.local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/util/retry.py", line 470, in increment
    raise reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/util/util.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/home/autogen/.local/lib/python3.11/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 996, in send
    self.connect()
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/transport/unixconn.py", line 27, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/api/client.py", line 213, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
                        ^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/utils/decorators.py", line 44, in inner
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/api/client.py", line 236, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/st-agents/src/main.py", line 10, in <module>
    with autogen.coding.DockerCommandLineCodeExecutor(work_dir="coding") as code_executor:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/autogen/coding/docker_commandline_code_executor.py", line 88, in __init__
    client = docker.from_env()
             ^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/client.py", line 94, in from_env
    return cls(
           ^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/autogen/.local/lib/python3.11/site-packages/docker/api/client.py", line 220, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

Steps to reproduce

main.py is as follows:

import os
import autogen
from autogen import AssistantAgent, UserProxyAgent
from dotenv import load_dotenv

load_dotenv()

llm_config = {"model": "gpt-4", "api_key": os.environ["OPENAI_API_KEY"]}

with autogen.coding.DockerCommandLineCodeExecutor(work_dir="coding") as code_executor:
    assistant = AssistantAgent("assistant", llm_config=llm_config)
    user_proxy = UserProxyAgent(
        "user_proxy", code_execution_config={"executor": code_executor}
    )

    # Start the chat
    user_proxy.initiate_chat(
        assistant,
        message="Plot a chart of NVDA and TESLA stock price change YTD. Save the plot to a file called plot.png",
    )

Model Used

gpt-4

Expected Behavior

No error :/

Screenshots and logs

No response

Additional Information

AutoGen: Using the latest docker image
Python: 3.11
OS: Sonoma 14.4.1 (23E224)
Related bugs:

• docker error with ollama and litellm running. This code runs fine with docker set to false.  #690
• Error with API Version #1020

[maximedupre]

I tried adding -v /var/run/docker.sock:/var/run/docker.sock to the docker command, but no success.

I don't think there is something wrong with my environment/docker config, because this command from another project works fine: docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock -v ./:/app/studio -v /Users/maximedupre/Desktop/programming/princeton-nlp/SWE-agent/keys.cfg:/app/keys.cfg -v /Users/maximedupre/Desktop/docker-volume:/app/trajectories/root sweagent/swe-agent-run:latest python run.py --image_name=sweagent/swe-agent:latest --model_name gpt4 --per_instance_cost_limit 10.00 --data_path studio/issue.md --repo_path studio --config_file config/default.yaml --skip_existing=False

[maximedupre]

I just tried it again and now it works for whatever reason. Maybe Docker update. Maybe reboot from macOS update fixed it 🤷🏻‍♂️

[maximedupre]

Nevermind, I had commented out the use of the DockerCommandLineCodeExecutor 😅 Still not working...

[maximedupre]

I tried downgrading Docker to 4.27.0 (135262), but still no success.

[ekzhu]

@jackgerrits Does this looks like a docker service update breaking the docker client? All PRs are blocked by similar error: https://github.com/microsoft/autogen/actions/runs/9183464819/job/25254224164

[maximedupre]

@ekzhu It might be related to this: docker/docker-py#3256