-
Notifications
You must be signed in to change notification settings - Fork 627
Use IterWithAttributes in the stalePartialBlockLastModifiedTime function if possible #12395
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Use IterWithAttributes in the stalePartialBlockLastModifiedTime function if possible #12395
Conversation
if bucket supports UpdatedAt IterOptionType.
I didn't update any tests, since existing test TestStalePartialBlockLastModifiedTime uses filesystem bucket which supports UpdatedAt option, so the existing test covers the new code. Please tell if I should add test or update existing one to cover the case when UpdatedAt option is not supported. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice, generally looks ok
Please tell if I should add test or update existing one to cover the case when UpdatedAt option is not supported.
is there a way to wrap the buckt implmentation with our own which returns empty SupportedIterOptions
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR. Both branches should be unit tested.
pkg/compactor/blocks_cleaner.go
Outdated
|
||
// If bucket supports UpdatedAt IterOptionType, use IterWithAttributes | ||
// to reduce the amount of object attributes calls. | ||
if slices.Contains(userBucket.SupportedIterOptions(), objstore.UpdatedAt) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be evaluated once upon construction of the blocks cleaner instead of every call. That would also provide a way to unit test both paths even if the test bucket supports this type of listing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you elaborate?
Do you want to add a new bool field to a BlocksCleaner struct, something like supportsUpdatedAt
, set it in the NewBlocksCleaner
function, and then use in the stalePartialBlockLastModifiedTime
function?
…n UpdatedAt IterOptionType is not supported by bucket
Applied suggestions from the review, updated test to also use a filesystem bucket wrapper which doesn't support UpdatedAt option. |
What this PR does
This PR updates stalePartialBlockLastModifiedTime function to use IterWithAttributes instead of Iter to reduce the amount of object attributes calls if bucket supports objstore.UpdatedAt IterOptionType.
Which issue(s) this PR fixes or relates to
Fixes #12359
Checklist
CHANGELOG.md
updated - the order of entries should be[CHANGE]
,[FEATURE]
,[ENHANCEMENT]
,[BUGFIX]
. If changelog entry is not needed, please add thechangelog-not-needed
label to the PR.about-versioning.md
updated with experimental features.