Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimizations on to_indices #227

Merged
merged 13 commits into from
Nov 28, 2021
Merged

Optimizations on to_indices #227

merged 13 commits into from
Nov 28, 2021

Conversation

Tokazama
Copy link
Member

@Tokazama Tokazama commented Nov 8, 2021

This is a performance optimized version of to_indices. I've provided the justification for most changes from Base.to_indices in the "# Extended help" section of the doc string. It also includes some minimal benchmarking to drive the point home. I might change a few minor bits of code once I get some sleep, but it can work for any of the indexing argument types in currently registered packages and can do so faster than Base.to_indices in all cases (as far as I'm aware). I'm sure there are parts I haven't documented or explained well, but I think it's a pretty decent start for now.

The biggest hang ups I anticipate are potentially breaking changes (still need to check if we can take away the 3 arg version of to_indices) and the two large @generated methods working behind the scenes (_splat_indices and __to_indices). I think we can solve any problems that might come up with regards to accommodating new indexing types using traits.

@codecov
Copy link

codecov bot commented Nov 8, 2021

Codecov Report

Merging #227 (66eaf05) into master (5c9681d) will increase coverage by 0.11%.
The diff coverage is 87.17%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     JuliaArrays/ArrayInterface.jl#227      +/-   ##
==========================================
+ Coverage   87.81%   87.93%   +0.11%     
==========================================
  Files          11       11              
  Lines        1789     1591     -198     
==========================================
- Hits         1571     1399     -172     
+ Misses        218      192      -26     
Impacted Files Coverage Δ
src/ranges.jl 90.51% <ø> (-0.79%) ⬇️
src/indexing.jl 83.87% <86.60%> (+7.37%) ⬆️
src/ArrayInterface.jl 85.87% <100.00%> (-0.71%) ⬇️
src/dimensions.jl 97.54% <100.00%> (+0.43%) ⬆️
src/size.jl 85.00% <0.00%> (-2.81%) ⬇️
src/axes.jl 92.08% <0.00%> (-2.07%) ⬇️
src/stridelayout.jl 85.62% <0.00%> (-1.74%) ⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c9681d...66eaf05. Read the comment docs.

@Tokazama Tokazama marked this pull request as ready for review November 19, 2021 01:57
@Tokazama
Copy link
Member Author

Tokazama commented Nov 24, 2021

@ChrisRackauckas and/or @chriselrod, I know the inner @generated code is a bit messy for a thorough review here, but is the overall approach and motivation for the unique approach clear at all through the docs?

If we're worried about breaking changes with this, then we should probably address some additional breaking changes before releasing 4.0 (like JuliaArrays/StaticArrayInterface.jl#10). I could make a project task list with some clear goals that we could agree on for this.

Copy link
Collaborator

@chriselrod chriselrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have time for a thorough review, but I'm not expecting any problems at a glance.

After a lot of testing I found out that we can just rely on
`ndims_index` for aligning axes in a generated function and then flatten
out tuples returned from `to_index`. This greatly simplifies the
explanation of internals and it also makes accomodating new indexing
types simpler.
@Tokazama Tokazama merged commit 70c135e into master Nov 28, 2021
@Tokazama Tokazama deleted the to_indices branch November 28, 2021 19:19
@Tokazama Tokazama restored the to_indices branch November 28, 2021 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants