After changing OLLAMA_HOST=0.0.0.0, ollama.chat() and ollama.generate() not work

[hkbb2014]

Before setting OLLAMA_HOST, ollama.generate() was working.
response = ollama.generate(model=model, prompt=prompt, stream=True)

After setting OLLAMA_HOST=0.0.0.0, ollama.generate() did not work at all.

But directly connecting to http://localhost:11434/api/generate without python work well.

How can I fix this?

Traceback (most recent call last):
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_transports\default.py", line 101, in map_httpcore_exceptions
yield
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_transports\default.py", line 250, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection_pool.py", line 256, in handle_request
raise exc from None
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection_pool.py", line 236, in handle_request
response = connection.handle_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection.py", line 101, in handle_request
raise exc
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection.py", line 78, in handle_request
stream = self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_sync\connection.py", line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_backends\sync.py", line 207, in connect_tcp
with map_exceptions(exc_map):
^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\contextlib.py", line 158, in exit
self.gen.throw(value)
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: [WinError 10049] 內容中所要求的位址不正確。

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "x:\pythonProjects\llmDemo\1_ollama\1_generate\generate_python.py", line 35, in
completion_with_ollama(prompt)
File "x:\pythonProjects\llmDemo\1_ollama\1_generate\generate_python.py", line 18, in completion_with_ollama
for chunk in response:
^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\ollama_client.py", line 163, in inner
with self._client.stream(*args, **kwargs) as r:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\contextlib.py", line 137, in enter
return next(self.gen)
^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 868, in stream
response = self.send(
^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 914, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 942, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 979, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_client.py", line 1014, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_transports\default.py", line 249, in handle_request
with map_httpcore_exceptions():
^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\contextlib.py", line 158, in exit
self.gen.throw(value)
File "X:\pythonProjects\py312_pytorch_cuda_v2\Lib\site-packages\httpx_transports\default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [WinError 10049] 內容中所要求的位址不正確。

[hkbb2014]

I print the request in httpx _client.py, it seems that it is making request to http://0.0.0.0:11434/api/generate

<Request('POST', 'http://0.0.0.0:11434/api/generate')>

I guess this is the reason, how can I specify the default host for for ollama.generate()

It seems that langChain's ChatOllama and OllamaLLM also have the same problem. How to fix this other than specifying other OLLAMA_HOST?

[nickdickinson]

I can confirm I have the same problem (windows).

I believe this is a bug but not sure why it does this. It is common to have OLLAMA_HOST set to 0.0.0.0 and that value is commonly used by other software such as Continue.dev in VSCode for code completion etc.

@hkbb2014 My workaround at the moment is to do the following in a script and to import that BEFORE importing ollama.
os.environ['OLLAMA_HOST'] = 'http://127.0.0.1:11434'
import ollama

A few ways to do this in a project:

• Modify the activate script of your venv to set your environment variable as above
• Use a .env file with conda
• Use a .env file with venv and the dotenv package... etc.

[hkbb2014]

@nickdickinson set OLLAMA_HOST before importing ollama works. It works well for langchain too, thanks!

[ParthSareen]

Are you also serving ollama with OLLAMA_HOST set?

export OLLAMA_HOST=...
ollama serve

[nickdickinson]

@ParthSareen Yes, but the global system env value is 0.0.0.0 which does not work in the Python package (at least on my Windows system). It does work when I explicitly set it to http://127.0.0.1:11434/

[ParthSareen]

Got it, thanks @nickdickinson. Will dig in

[ParthSareen]

@nickdickinson
This is expected behavior - probably surfaced with the httpx bump from a bit ago.

OLLAMA_HOST should not be 0.0.0.0 from the client side.

See:
#491 (comment)