Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[🐛 Bug]: WebDriverException thrown with empty message when remote driver request returns an empty response with status code above 500 #15357

Open
takafumiokamoto opened this issue Mar 2, 2025 · 2 comments

Comments

@takafumiokamoto
Copy link

takafumiokamoto commented Mar 2, 2025

What happened?

This error occurs when Selenium attempts to connect to the remote driver, but the response is empty with a status code above 500. I think that the remote driver itself might not return such status codes, but when the client machine is behind a proxy or in a restricted network (e.g., in corporate environments). In such cases, the proxy server or network firewall may return an empty response instead of forwarding the remote driver's response, or the request might simply time out.

Why WebDriverException has empty message ?

in _request function in py/selenium/webdriver/remote/remote_connection.py (around line 455)

def _request(self, method, url, body=None):
~~~ omit ~~~
    if not any([x.startswith("image/png") for x in content_type]):
        try:
            data = utils.load_json(data.strip())
        except ValueError:
            if 199 < statuscode < 300:
                status = ErrorCode.SUCCESS
            else:
                status = ErrorCode.UNKNOWN_ERROR
            ## this ↓
            return {"status": status, "value": data.strip()}

The data.strip() method returns an empty string due to an empty response. Since the "status" is set to UNKNOWN_ERROR, the WebDriverException is selected, and the "value" field, which is empty, is assigned in the ErrorHandler around line 194 in py/selenium/webdriver/remote/errorhandler.py.

        if not value:
            value = response["value"]
        if isinstance(value, str):
            # this ↓
            raise exception_class(value)
        if message == "" and "message" in value:
            message = value["message"]

Core problem

This error is highly unlikely to occur because it only happens when an incorrect proxy setting is explicitly specified in Selenium.
By default, Selenium automatically picks up system-wide proxy settings if no proxy is specified in client_config.

However, on Windows, this issue is more likely to happen — here's why:
On Windows, system-wide proxy settings are configured via the Settings app (Internet Options) and stored in the registry, while Selenium only reads proxy settings from environment variables by default.
This behavior can be confusing because even though a proxy is configured system-wide, Selenium won't use it unless the environment variables are set. Without the proxy, driver request might return an empty response in corporate environments or restricted networks that require proxy usage.

What I would like to contribute

Error Message

Displaying an empty error message can be slightly inconvenient. It would be more helpful if the message included the status code, URL, and reason for the error.

Proxy Setting (Default Behavior)

On Windows, when the ProxyType is set to SYSTEM (the default value), it would be more convenient to automatically retrieve proxy settings from the registry. This can be achieved with the following simple code:

import winreg
from typing import Optional
from contextlib import suppress

def get_windows_proxy_url() -> tuple[Optional[str], Optional[str]]:
    """
    Get Windows Proxy Settings from Registry.

    Returns:
        tuple[Optional[str], Optional[str]]: 
            - First element: Proxy server (HTTP/HTTPS_PROXY equivalent)
            - Second element: Proxy override (NO_PROXY equivalent)
    """
    try:
        with winreg.OpenKey(winreg.HKEY_CURRENT_USER, r'Software\Microsoft\Windows\CurrentVersion\Internet Settings', 0, winreg.KEY_READ) as key:
            proxy_enable, _ = winreg.QueryValueEx(key, 'ProxyEnable')
            if proxy_enable == 0:
                return (None, None)
            _proxy = None
            _no_proxy = None
            with suppress(FileNotFoundError):
                _proxy, _ = winreg.QueryValueEx(key, 'ProxyServer')
            with suppress(FileNotFoundError):
                _no_proxy, _ = winreg.QueryValueEx(key, 'ProxyOverride')
            return (_proxy, _no_proxy)
    except FileNotFoundError:
        return (None, None)

If proposes are suite this project, I would like to create PR for this

How can we reproduce the issue?

1. setup proxy server and configure to return empty response
2. set `HTTP_PROXY` environment variable to proxy server address
3. run Selenium

Relevant log output

(selenium-test-3.13) PS C:\Users\admin\git\selenium-test> python .\main.py
Traceback (most recent call last):
  File "C:\Users\admin\git\selenium-test\main.py", line 6, in <module>
    driver = webdriver.Chrome()
  File "C:\Users\admin\git\selenium-test\.venv\Lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 45, in __init__
    super().__init__(
    ~~~~~~~~~~~~~~~~^
        browser_name=DesiredCapabilities.CHROME["browserName"],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        keep_alive=keep_alive,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\admin\git\selenium-test\.venv\Lib\site-packages\selenium\webdriver\chromium\webdriver.py", line 66, in __init__
    super().__init__(command_executor=executor, options=options)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\git\selenium-test\.venv\Lib\site-packages\selenium\webdriver\remote\webdriver.py", line 250, in __init__
    self.start_session(capabilities)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "C:\Users\admin\git\selenium-test\.venv\Lib\site-packages\selenium\webdriver\remote\webdriver.py", line 342, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\git\selenium-test\.venv\Lib\site-packages\selenium\webdriver\remote\webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "C:\Users\admin\git\selenium-test\.venv\Lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 196, in check_response
    raise exception_class(value)
selenium.common.exceptions.WebDriverException: Message:

Operating System

Windows 11

Selenium version

Python 4.29.0

What are the browser(s) and version(s) where you see this issue?

Google Chrome Version 133.0.6943.142 (Official Build) (64-bit)

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver-133.0.6943.141

Are you using Selenium Grid?

Yes

Copy link

github-actions bot commented Mar 2, 2025

@takafumiokamoto, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

@takafumiokamoto
Copy link
Author

I think this might be related to #15325

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant