Skip to content

[mypyc] feat: ForMap generator helper for builtins.map #19649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: master
Choose a base branch
from

Conversation

BobTheBuidler
Copy link
Contributor

@BobTheBuidler BobTheBuidler commented Aug 12, 2025

This PR adds a ForMap helper class for for loops over builtins.map which allows mypyc to maintain control of the function calling, so it can call native functions and primitive ops, and to avoid allocation of iterators in many cases.

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, map is a commonly used builtin.

s += x * 10 + y
return s

[file driver.py]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please merge all the run tests into a single test without a [file driver.py] section. Use assertions instead of [out] sections for checking things. This way tests will run faster and they will be easier to maintain.

r14 = box(int, _mypyc_map_arg_0)
r15 = box(int, _mypyc_map_arg_1)
r16 = unbox(int, r14)
r17 = unbox(int, r15)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The boxing followed by unboxing seems pointless. It would be great if we didn't need this (it might be a pre-existing issue though).

Copy link
Contributor Author

@BobTheBuidler BobTheBuidler Aug 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I'll take a look. Does mypyc currently perform any optimization over the IR before generating the actual C code? I looked briefly at the transform module, which seems like it might play some role here, but I couldn't figure out the entrypoint, or if that section even works as I think it does

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After looking, I'm not entirely sure what causes this.

The fake NameExpr that I created is using the correct rtype in the IR, and from reading the code in for_helpers I can't figure out why (or where) r14 thru r17 are being emitted.

What I see is that we create a CallExpr using the NameExpr directly as the input, which implies to me there wouldn't be any intermediate values involved in the actual call.

L6:
return s

[case testForMapThreeArgs]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The number of irbuild tests is perhaps a bit much? I wonder if 1-2 irbuild test cases would be sufficient, with the remaining cases only covered by run tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I picked 3 to keep, one-arg, 3-arg, and comprehension

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants