Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BlockSparseArrays] Use new SparseArrayDOK type in BlockSparseArrays #1272

Merged
merged 34 commits into from
Dec 2, 2023
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
6401807
[NDTensors] Start SparseArrayDOKs module
mtfishman Nov 25, 2023
9e4142c
Reorganization
mtfishman Nov 26, 2023
0677dfa
Format
mtfishman Nov 26, 2023
995c3f9
Start making SparseArrayInterface module, to be used by SparseArrayDO…
mtfishman Nov 28, 2023
b696a7e
Update interface
mtfishman Nov 28, 2023
e984cbe
Add another test
mtfishman Nov 28, 2023
475ee31
Improved map
mtfishman Nov 28, 2023
36961f8
New SparseArrayDOKs using SparseArrayInterface
mtfishman Nov 28, 2023
25f8229
Fix namespace issues
mtfishman Nov 29, 2023
3b3c50d
One more namespace issue
mtfishman Nov 29, 2023
a20e819
More namespace issues
mtfishman Nov 29, 2023
d571bd9
Julia 1.6 backwards compatibility
mtfishman Nov 29, 2023
410df32
Use SparseArrayInterface in DiagonalArrays
mtfishman Nov 29, 2023
41c9816
Format
mtfishman Nov 29, 2023
420692a
Fix loading issue
mtfishman Nov 29, 2023
c6dcefe
Missing include, improve README
mtfishman Nov 29, 2023
2478166
[BlockSparseArrays] Start using SparseArrayDOK
mtfishman Nov 29, 2023
43929be
Small fixes
mtfishman Nov 30, 2023
f70a520
Merge branch 'main' into NDTensors_new_BlockSparseArrays
mtfishman Nov 30, 2023
41b92b8
Change SparseArray to SparseArrayDOK
mtfishman Nov 30, 2023
af5ff02
Format
mtfishman Nov 30, 2023
a1733f6
Temporarily remove broken tests
mtfishman Nov 30, 2023
1be1d00
Introduct AbstractSparseArray, start rewriting BlockSparseArray
mtfishman Dec 1, 2023
97f3df4
Move AbstractSparseArray to SparseArrayInterface
mtfishman Dec 1, 2023
32d375a
Improve testing and organization
mtfishman Dec 1, 2023
0ea3eee
DiagonalArrays reorganization and simplification
mtfishman Dec 1, 2023
f918aee
Get more BlockSparseArrays tests passing
mtfishman Dec 1, 2023
fc0ff14
Move arraytensor code to backup files
mtfishman Dec 1, 2023
8487d0c
Move arraystorage code to backup files
mtfishman Dec 1, 2023
fc9ff82
Try fixing tests
mtfishman Dec 1, 2023
b5b643d
Comment
mtfishman Dec 1, 2023
796f33d
Merge main
mtfishman Dec 1, 2023
4d1453d
Fix namespace issue
mtfishman Dec 1, 2023
1f04d9c
Remove arraytensor test
mtfishman Dec 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions NDTensors/src/arraystorage/blocksparsearray/storage/contract.jl
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# TODO: Change to:
# using .SparseArrayDOKs: SparseArrayDOK
using .BlockSparseArrays: SparseArray
# TODO: Define in `SparseArrayInterface`.
using ..SparseArrayDOKs: SparseArrayDOK

# TODO: This is inefficient, need to optimize.
# Look at `contract_labels`, `contract_blocks` and `maybe_contract_blocks!` in:
Expand Down Expand Up @@ -39,11 +38,11 @@ function default_contract_muladd(a1, labels1, a2, labels2, a_dest, labels_dest)
end

function contract!(
a_dest::SparseArray,
a_dest::SparseArrayDOK,
labels_dest,
a1::SparseArray,
a1::SparseArrayDOK,
labels1,
a2::SparseArray,
a2::SparseArrayDOK,
labels2;
muladd=default_contract_muladd,
)
Expand Down
5 changes: 3 additions & 2 deletions NDTensors/src/lib/BlockSparseArrays/src/BlockSparseArrays.jl
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,17 @@ using Dictionaries: Dictionary, Indices, getindices, set! # TODO: Move to `Spars
using LinearAlgebra: Hermitian
using SplitApplyCombine: groupcount

export BlockSparseArray, SparseArray
export BlockSparseArray # , SparseArray

include("defaults.jl")
include("tensor_product.jl")
include("base.jl")
include("axes.jl")
include("abstractarray.jl")
include("permuteddimsarray.jl")
include("blockarrays.jl")
# TODO: Split off into `SparseArraysExtensions` module, rename to `SparseArrayDOK`.
include("sparsearray.jl")
# include("sparsearray.jl")
include("blocksparsearray.jl")
include("allocate_output.jl")
include("subarray.jl")
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
module LinearAlgebraExt
using ...AlgorithmSelection: Algorithm, @Algorithm_str
using BlockArrays: BlockArrays, blockedrange, blocks
using ..BlockSparseArrays: SparseArray, nonzero_keys # TODO: Move to `SparseArraysExtensions` module, rename `SparseArrayDOK`.
using ..BlockSparseArrays: nonzero_keys # TODO: Move to `SparseArraysExtensions` module, rename `SparseArrayDOK`.
using ..BlockSparseArrays: BlockSparseArrays, BlockSparseArray, nonzero_blockkeys
using LinearAlgebra: LinearAlgebra, Hermitian, Transpose, I, eigen, qr
using SparseArrays: SparseArrays, SparseMatrixCSC, spzeros, sparse
Expand Down
10 changes: 7 additions & 3 deletions NDTensors/src/lib/BlockSparseArrays/src/LinearAlgebraExt/qr.jl
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
using ...SparseArrayDOKs: SparseArrayDOK

# Check if the matrix has 1 or fewer entries
# per row/column.
function is_permutation_matrix(a::SparseMatrixCSC)
Expand All @@ -6,7 +8,7 @@ end

# Check if the matrix has 1 or fewer entries
# per row/column.
function is_permutation_matrix(a::SparseArray{<:Any,2})
function is_permutation_matrix(a::SparseArrayDOK{<:Any,2})
keys = collect(Iterators.map(Tuple, nonzero_keys(a)))
I = first.(keys)
J = last.(keys)
Expand All @@ -17,7 +19,8 @@ function findnonzerorows(a::SparseMatrixCSC, col)
return view(a.rowval, a.colptr[col]:(a.colptr[col + 1] - 1))
end

function SparseArrays.SparseMatrixCSC(a::SparseArray{<:Any,2})
# TODO: Is this already defined?
function SparseArrays.SparseMatrixCSC(a::SparseArrayDOK{<:Any,2})
# Not defined:
# a_csc = SparseMatrixCSC{eltype(a)}(size(a))
a_csc = spzeros(eltype(a), size(a))
Expand All @@ -27,8 +30,9 @@ function SparseArrays.SparseMatrixCSC(a::SparseArray{<:Any,2})
return a_csc
end

# TODO: Is this already defined?
# Get the sparse structure of a SparseArray as a SparseMatrixCSC.
function sparse_structure(structure_type::Type{<:SparseMatrixCSC}, a::SparseArray{<:Any,2})
function sparse_structure(structure_type::Type{<:SparseMatrixCSC}, a::SparseArrayDOK{<:Any,2})
mtfishman marked this conversation as resolved.
Show resolved Hide resolved
# Idealy would work but a bit too complicated for `map` right now:
# return SparseMatrixCSC(map(x -> iszero(x) ? false : true, a))
# TODO: Change to `spzeros(Bool, size(a))`.
Expand Down
8 changes: 7 additions & 1 deletion NDTensors/src/lib/BlockSparseArrays/src/allocate_output.jl
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@ end
# SparseArray
#############################################################################

using ..SparseArrayDOKs: SparseArrayDOK

const SparseArrayLike{T,N,Zero} = Union{
SparseArrayDOK{T,N,Zero},PermutedDimsArray{T,N,<:Any,<:Any,SparseArrayDOK{T,N,Zero}}
}

# TODO: Maybe store nonzero locations?
# TODO: Make this backwards compatible.
# TODO: Add a default for `eltype` and `zero`.
Expand All @@ -100,7 +106,7 @@ Base.@kwdef struct SparseArrayStructure{ElType,Axes,Zero} <:
zero::Zero
end

function allocate(arraytype::Type{<:SparseArray}, structure::SparseArrayStructure)
function allocate(arraytype::Type{<:SparseArrayDOK}, structure::SparseArrayStructure)
# TODO: Use `set_eltype`.
return arraytype{structure.eltype}(structure.axes, structure.zero)
end
Expand Down
11 changes: 7 additions & 4 deletions NDTensors/src/lib/BlockSparseArrays/src/blocksparsearray.jl
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
using BlockArrays: block
## using ..SparseArrayDOKs: SparseArrayDOK

# Also add a version with contiguous underlying data.
struct BlockSparseArray{
T,N,A<:AbstractArray{T,N},Blocks<:SparseArray{A,N},Axes<:NTuple{N,AbstractUnitRange{Int}}
T,N,A<:AbstractArray{T,N},Blocks<:AbstractArray{A,N},Axes<:NTuple{N,AbstractUnitRange{Int}}
mtfishman marked this conversation as resolved.
Show resolved Hide resolved
} <: AbstractBlockArray{T,N}
blocks::Blocks
axes::Axes
Expand All @@ -26,7 +27,7 @@ function Base.reshape(a::BlockSparseArray, ax::Tuple{Vararg{AbstractUnitRange}})
## blocks_reshaped = reshape(blocks(a), blocklength.(ax))
blocktype_reshaped = set_ndims(blocktype(a), length(ax))
# TODO: Some other way of getting `zero` function?
blocks_reshaped = SparseArray(
blocks_reshaped = default_sparsearray_type()(
Dictionary{CartesianIndex{length(ax)},blocktype_reshaped}(),
blocklength.(ax),
BlockZero(ax),
Expand Down Expand Up @@ -113,7 +114,7 @@ end
## axes::Axes
## end

function BlockSparseArray(a::SparseArray, axes::Tuple{Vararg{AbstractUnitRange}})
function BlockSparseArray(a::AbstractArray, axes::Tuple{Vararg{AbstractUnitRange}})
A = eltype(a)
T = eltype(A)
N = ndims(a)
Expand All @@ -132,7 +133,9 @@ function BlockSparseArray(
map(block -> CartesianIndex(inttuple(block)), blocks)
end
cartesiandata = Dictionary(cartesianblocks, blockdata)
block_storage = SparseArray(cartesiandata, blocklength.(axes), BlockZero(axes))
block_storage = default_sparsearray_type()(
cartesiandata, blocklength.(axes), BlockZero(axes)
)
return BlockSparseArray(block_storage, axes)
end

Expand Down
3 changes: 3 additions & 0 deletions NDTensors/src/lib/BlockSparseArrays/src/defaults.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
using ..SparseArrayDOKs: SparseArrayDOK

default_sparsearray_type() = SparseArrayDOK
1 change: 1 addition & 0 deletions NDTensors/src/lib/BlockSparseArrays/test/Project.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
[deps]
BlockArrays = "8e7c35d0-a365-5155-bbbb-fb81a777f24e"
Compat = "34da2185-b29b-5c13-b0c7-acf172513d20"
NDTensors = "23ae76d9-e61a-49c4-8f12-3f1a16adf9cf"
Loading