chore: use modern type annotation syntax #419

akx · 2025-01-14T16:33:08Z

Using from __future__ import annotations makes futuristic annotations usable in any Python version.

This adds eval_type_backport as a dep for older Pythons so Pydantic can robustly evaluate these annotations.

ParthSareen · 2025-01-15T19:25:00Z

I believe I've tried this out before but it breaks the parsing logic when passing tools as function - could you verify?

akx · 2025-01-16T06:56:04Z

I believe I've tried this out before but it breaks the parsing logic when passing tools as function - could you verify?

Wouldn't there be a failing test here if that was the case? 😁

ParthSareen · 2025-01-16T06:58:01Z

@akx I would hope so HAHA. I'm pretty sure I had a bunch of tests for parsing logic but maybe it's fine. I'd still want to just run the example for a sanity check.

akx · 2025-01-16T07:21:28Z

@ParthSareen Yes, the example still works on various versions of Python. Love that it's nondeterministic though and sometimes the model decides to pass in strings instead of ints, and sometimes it apologizes about a confusing response.

This will require #422 to be merged first so CI can be green at all.

(ollama-python) ~/b/ollama-python (futuristic-types-py38 *) $ tox run
py38: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/9/ollama-0.0.0.tar.gz
py38: commands[0]> python --version
Python 3.8.19
py38: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': '3', 'b': '1'}
Function output: 31
Final response: The answer to the equation 3 + 1 is 4.
py38: OK ✔ in 4.78 seconds
py39: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/10/ollama-0.0.0.tar.gz
py39: commands[0]> python --version
Python 3.9.6
py39: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': 3, 'b': 1}
Function output: 4
Final response: The result of the calculation is 4.
py39: OK ✔ in 3.32 seconds
py310: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/11/ollama-0.0.0.tar.gz
py310: commands[0]> python --version
Python 3.10.16
py310: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': 3, 'b': 1}
Function output: 4
Final response: The answer to the math problem "three plus one" is 4.
py310: OK ✔ in 2.6 seconds
py311: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/12/ollama-0.0.0.tar.gz
py311: commands[0]> python --version
Python 3.11.11
py311: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': 3, 'b': 1}
Function output: 4
Final response: I apologize for the confusing response earlier. The correct answer to the question "What is three plus one?" is indeed 4.
py311: OK ✔ in 2.81 seconds
py313: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/13/ollama-0.0.0.tar.gz
py313: commands[0]> python --version
Python 3.13.1
py313: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': '3', 'b': '1'}
Function output: 31
Final response: The answer to three plus one is 4.

(ollama-python) ~/b/ollama-python (futuristic-types-py38 *) $ tox run
py38: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/14/ollama-0.0.0.tar.gz
py38: commands[0]> python --version
Python 3.8.19
py38: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': 3, 'b': 1}
Function output: 4
Final response: The result of the calculation 3 + 1 is indeed 4.
py38: OK ✔ in 3.11 seconds
py39: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/15/ollama-0.0.0.tar.gz
py39: commands[0]> python --version
Python 3.9.6
py39: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': '3', 'b': '1'}
Function output: 31
Final response: The answer to the math problem is 4.
py39: OK ✔ in 3.66 seconds
py310: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/16/ollama-0.0.0.tar.gz
py310: commands[0]> python --version
Python 3.10.16
py310: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': '3', 'b': '1'}
Function output: 31
Final response: The answer to 3 + 1 is 4.
py310: OK ✔ in 2.52 seconds
py311: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/17/ollama-0.0.0.tar.gz
py311: commands[0]> python --version
Python 3.11.11
py311: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': '3', 'b': '1'}
Function output: 31
Final response: The answer to the equation 3 + 1 is 4.
py311: OK ✔ in 7.85 seconds
py313: install_package> python -I -m pip install --force-reinstall --no-deps /Users/akx/build/ollama-python/.tox/.tmp/package/18/ollama-0.0.0.tar.gz
py313: commands[0]> python --version
Python 3.13.1
py313: commands[1]> python examples/tools.py
Prompt: What is three plus one?
Calling function: add_two_numbers
Arguments: {'a': 3, 'b': 1}
Function output: 4
Final response: The answer to the equation 3 + 1 is 4.

ParthSareen · 2025-01-16T07:28:07Z

@akx what model are you using? I find it funny that it does that and also a bit annoying.

Working on sampling in the new engine and so also thinking about improving tool calling as a whole through structured generation. Hopefully can have something soon which can improve this experience.

I'll check both the PRs out in the AM. Really appreciate these especially since I've been heads down on a few other areas!!

akx · 2025-01-16T07:31:46Z

@akx what model are you using? I find it funny that it does that and also a bit annoying.

llama3.2. Am on a train, so no chance to pull llama3.1...

ParthSareen · 2025-01-17T18:12:54Z

@akx we should be able to rebase and get this in :)

Actually - do we need the extra package - could we do without it? We try to keep the package very minimal with low external dependencies

akx · 2025-01-21T11:25:56Z

Actually - do we need the extra package - could we do without it?

It's marked to be installed only when the environment has an old Python.

I can elide it from this library's deps, but then users will need to install it on those versions by hand. 🤷

ParthSareen · 2025-01-21T17:49:31Z

Actually - do we need the extra package - could we do without it?

It's marked to be installed only when the environment has an old Python.

I can elide it from this library's deps, but then users will need to install it on those versions by hand. 🤷

I'm on the fence for this PR - to a degree I think it is good to have the old typing here as it forces us to think about those cases when working on any typing-related feature. I don't want to bring in unnecessary external dependencies unless it's vastly improving the developer experience. I'll think a bit more about it but honestly this PR improves our experience as devs not really anyone else's.

akx · 2025-01-22T06:15:34Z

to a degree I think it is good to have the old typing here as it forces us to think about those cases when working on any typing-related feature.

If you only have old typing here, then you're not thinking about new typing when working on any typing-related features. You can mix and match old and new typing in tests as you like 🤷

I don't want to bring in unnecessary external dependencies unless it's vastly improving the developer experience.

Based on https://pypistats.org/packages/ollama, about 6.5% of users would ever see the new dependency.

ParthSareen · 2025-01-22T17:43:58Z

to a degree I think it is good to have the old typing here as it forces us to think about those cases when working on any typing-related feature.

If you only have old typing here, then you're not thinking about new typing when working on any typing-related features. You can mix and match old and new typing in tests as you like 🤷

I don't want to bring in unnecessary external dependencies unless it's vastly improving the developer experience.

Based on https://pypistats.org/packages/ollama, about 6.5% of users would ever see the new dependency.

That's a fair point. Going to leave this open for now and come back to this. Thanks!

ParthSareen · 2025-02-12T19:45:36Z

Gave it more thought. I'm hesitant to bring new dependencies - especially unmaintained ones into the repo. Even with a small % of users, any regression would be an extremely poor experience. With the increase in usage for the repo, need to keep it as stable as possible and not have unintended changes.

Don't see the benefit of the upgrade vs. risk in this case. I do appreciate the back and forth and the PR.

akx · 2025-02-12T19:47:06Z

especially unmaintained ones

Where'd "unmaintained" come from?

ParthSareen · 2025-02-12T19:50:28Z

Sorry - probably not the best descriptor. More so less used dependency. What you linked and was added doesn't seem to be widely used which makes me hesitant to bring it in.

akx · 2025-02-12T19:54:49Z

More so less used dependency.

344k downloads per day. Approximately 4 times as much as ollama at 83k downloads per day, and that's also considering it's a backport library only required for old Python versions.

Anyway, it's alright, you do your decisions for your repo – but I'm just allergic to decisions made based on "vibes" 😄

ParthSareen · 2025-02-12T20:34:42Z

More so less used dependency.

344k downloads per day. Approximately 4 times as much as ollama at 83k downloads per day, and that's also considering it's a backport library only required for old Python versions.

Anyway, it's alright, you do your decisions for your repo – but I'm just allergic to decisions made based on "vibes" 😄

Good to know it's being used. I was basing it off the github. I'd still stand by the risk to reward not being worth it in this case. Either way, thanks for the PR and appreciate the contrasting opinion! 😁

akx force-pushed the futuristic-types-py38 branch from 7f896f0 to 1fb7b5f Compare January 16, 2025 07:01

chore: use futuristic type annotations

61e68f3

akx force-pushed the futuristic-types-py38 branch from 1fb7b5f to 61e68f3 Compare January 21, 2025 11:24

ParthSareen closed this Feb 12, 2025

chore: use modern type annotation syntax #419

chore: use modern type annotation syntax #419

Uh oh!

Conversation

akx commented Jan 14, 2025

Uh oh!

ParthSareen commented Jan 15, 2025

Uh oh!

akx commented Jan 16, 2025

Uh oh!

ParthSareen commented Jan 16, 2025

Uh oh!

akx commented Jan 16, 2025

Uh oh!

ParthSareen commented Jan 16, 2025

Uh oh!

akx commented Jan 16, 2025

Uh oh!

ParthSareen commented Jan 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

akx commented Jan 21, 2025

Uh oh!

ParthSareen commented Jan 21, 2025

Uh oh!

akx commented Jan 22, 2025

Uh oh!

ParthSareen commented Jan 22, 2025

Uh oh!

ParthSareen commented Feb 12, 2025

Uh oh!

akx commented Feb 12, 2025

Uh oh!

ParthSareen commented Feb 12, 2025

Uh oh!

akx commented Feb 12, 2025

Uh oh!

ParthSareen commented Feb 12, 2025

Uh oh!

Uh oh!

ParthSareen commented Jan 17, 2025 •

edited

Loading