fix(PD-disaggregation): Can not get local ip

[storyicon]

Motivation

While performing a PD-disaggregation deployment, I encountered an issue with the get_local_ip_by_remote function. From the source code, I observed that it first attempts to obtain the local IPv4 address by connecting to 8.8.8.8. If this fails, it silently falls back to connecting to 2001:4860:4860::8888 to retrieve the local IPv6 address.

The code comments state, "Doesn't need to be reachable," which holds true if the local host has public internet access (even if restricted by a WAF). However, in enterprise networks or private cloud environments (e.g., VPC), access to the public internet is typically blocked, with no gateway available for public internet connectivity. This leads to the issue shown above, where the function fails entirely.

Modifications

This PR introduces a fallback mechanism via hostname resolution to ensure the correct local IP is returned even in environments without public internet routing. It enhances the function's robustness in restricted networks while maintaining cross-platform compatibility and simplicity.

Now everything is working smoothly.

[gemini-code-assist]

Hello @storyicon, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello! Gemini here, providing a summary of this pull request to help everyone quickly understand the changes. This PR addresses an issue where the get_local_ip_by_remote function fails in network environments without public internet access. The original implementation attempted to determine the local IP by connecting to public Google DNS servers (8.8.8.8 and 2001:4860:4860::8888), which doesn't work in restricted networks like enterprise VPCs. This change introduces a fallback mechanism using hostname resolution (socket.gethostname and socket.gethostbyname) to find the local IP if the initial connection attempts fail. This makes the function more robust and reliable in various network configurations.

Highlights

• Fix Local IP Retrieval: The primary goal is to fix the get_local_ip_by_remote function which was failing in environments lacking public internet connectivity.
• Add Hostname Resolution Fallback: A new fallback method is added that attempts to get the local IP by resolving the machine's hostname if connecting to external IPs fails.
• Improve Network Robustness: The change enhances the function's ability to correctly identify the local IP in restricted or private network environments.

Changelog

• python/sglang/srt/utils.py
  • Added a new try...except block within get_local_ip_by_remote to attempt hostname resolution (socket.gethostname, socket.gethostbyname) as a fallback.
  • Included checks to ensure the resolved IP is not a loopback or unspecified address (127.0.0.1, 0.0.0.0).
  • Positioned the new fallback logic between the failed IPv4 attempt and the existing IPv6 attempt.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

Local IP sought,
Public route fails, a dead end,
Hostname finds the way.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

Thank you for this pull request! The motivation and problem description are very clear, and the addition of a fallback mechanism for obtaining the local IP address in environments without public internet access is a valuable improvement. The approach of using hostname resolution is a good idea to handle these specific network configurations.

Summary of Findings

• Robustness of hostname resolution: The current implementation uses socket.gethostbyname, which only resolves to IPv4 addresses. A more robust approach for selecting the local IP, especially in environments with mixed IPv4/IPv6 or IPv6-only configurations, would be to use socket.getaddrinfo to retrieve all associated addresses and select the most suitable one.
• Silent exception handling: The except Exception: pass block hides potential errors during hostname resolution, making debugging difficult. Logging the exception would provide valuable information.

Merge Readiness

The pull request addresses a valid issue and provides a functional fallback. However, the current implementation of the hostname resolution fallback has potential limitations regarding IPv6 and uses silent exception handling. I recommend addressing the high severity issue regarding the robustness of IP selection (potentially using getaddrinfo) and the medium severity issue regarding silent exceptions before merging. I am unable to approve the pull request directly; please have other reviewers review and approve this code before merging.

[gemini-code-assist]

The use of socket.gethostbyname here will only resolve the hostname to an IPv4 address. In environments where the hostname might be associated with both IPv4 and IPv6 addresses, or primarily with IPv6 addresses, this method might not return the most appropriate local IP for the disaggregation setup, or could fail if no IPv4 address is configured for the hostname. Would it be more robust to use socket.getaddrinfo? This function can retrieve address information for both IPv4 and IPv6 and allows for more flexible selection of a suitable address (e.g., preferring global unicast addresses over link-local or loopback).

[gemini-code-assist]

Catching a general Exception and silently passing (except Exception: pass) can hide the specific reason for failure during hostname resolution (e.g., DNS errors, network configuration issues). This makes debugging more difficult if this fallback mechanism doesn't work as expected. Could we consider logging the exception, perhaps as a warning, to provide more insight into potential problems?

Suggested change

	except Exception:
	pass
	except Exception as e:
	logger.warning(f"Failed to get local IP via hostname resolution: {e}")