Skip to content

Commit

Permalink
Rename to PtrArrays and support PtrArray constructor (#8)
Browse files Browse the repository at this point in the history
  • Loading branch information
LilithHafner authored May 8, 2024
1 parent 54ba925 commit 7597b56
Show file tree
Hide file tree
Showing 5 changed files with 118 additions and 94 deletions.
6 changes: 3 additions & 3 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "MallocArrays"
uuid = "bdde6022-96f5-457e-ae5c-df64ce731513"
name = "PtrArrays"
uuid = "43287f4e-b6f4-7ad1-bb20-aadabca52c3d"
authors = ["Lilith Orion Hafner <[email protected]> and contributors"]
version = "1.0.1"
version = "1.1.0"

[compat]
julia = "1"
Expand Down
37 changes: 20 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# MallocArrays
# PtrArrays

[![Build Status](https://github.com/LilithHafner/MallocArrays.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/LilithHafner/MallocArrays.jl/actions/workflows/CI.yml?query=branch%3Amain)
[![Coverage](https://codecov.io/gh/LilithHafner/MallocArrays.jl/branch/main/graph/badge.svg)](https://codecov.io/gh/LilithHafner/MallocArrays.jl)
[![PkgEval](https://JuliaCI.github.io/NanosoldierReports/pkgeval_badges/M/MallocArrays.svg)](https://JuliaCI.github.io/NanosoldierReports/pkgeval_badges/M/MallocArrays.html)
[![Build Status](https://github.com/LilithHafner/PtrArrays.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/LilithHafner/PtrArrays.jl/actions/workflows/CI.yml?query=branch%3Amain)
[![Coverage](https://codecov.io/gh/LilithHafner/PtrArrays.jl/branch/main/graph/badge.svg)](https://codecov.io/gh/LilithHafner/PtrArrays.jl)
[![PkgEval](https://JuliaCI.github.io/NanosoldierReports/pkgeval_badges/P/PtrArrays.svg)](https://JuliaCI.github.io/NanosoldierReports/pkgeval_badges/P/PtrArrays.html)
[![Aqua](https://raw.githubusercontent.com/JuliaTesting/Aqua.jl/master/badge.svg)](https://github.com/JuliaTesting/Aqua.jl)

Do you miss playing hide and seek with memory leaks? Do you find GC overhead problematic?
MallocArrays.jl can take you back to the good old days of manual memory management.
PtrArrays.jl can take you back to the good old days of manual memory management.
See also [Bumper.jl](https://github.com/MasonProtter/Bumper.jl) if you want to avoid GC
overhead and don't like hide and seek.

Expand All @@ -18,7 +18,7 @@ Example usage

```julia
julia> malloc(Int, 4)
4-element MallocArray{Int64, 1}:
4-element PtrArray{Int64, 1}:
1053122630
0
936098496
Expand All @@ -27,7 +27,7 @@ julia> malloc(Int, 4)
julia> free(ans)

julia> malloc(Int, 4, 4)
4×4 MallocArray{Int64, 2}:
4×4 PtrArray{Int64, 2}:
923300075 1046634192 1046634192 1046634408
0 120 124 152
0 0 0 0
Expand All @@ -39,7 +39,7 @@ julia> free(ans)
Benchmarks:

```julia
using MallocArrays
using PtrArrays
function f(n)
x = malloc(Int, n)
try
Expand Down Expand Up @@ -67,20 +67,23 @@ end

The whole package's source code is only about 44 lines (excluding comments and whitespace),
half of which is re-implementing Julia's buggy `Core.checked_dims` function.
[Read it here](https://github.com/LilithHafner/MallocArrays.jl/blob/main/src/MallocArrays.jl)
[Read it here](https://github.com/LilithHafner/PtrArrays.jl/blob/main/src/PtrArrays.jl)

## Alternatives

[Bumper.jl](https://github.com/MasonProtter/Bumper.jl) provides bump allocators which allow
you to manage your own allocation stack, bypassing the Julia GC.

[StaticTools.jl](https://github.com/brenhinkeller/) is a much larger package which provides, among other
things, a [MallocArray](https://brenhinkeller.github.io/StaticTools.jl/dev/#StaticTools.MallocArray)
type that behaves similarly to this package's MallocArray. As StaticTools.jl is meant to run
without the Julia runtime, it does not make use of features such as exceptions. Consequently, all
usages of StaticTools.MallocArray are implicitly annotated with `@inbounds` and out of bounds
[StaticTools.jl](https://github.com/brenhinkeller/) is a much larger package which provides,
among other things, a
[MallocArray](https://brenhinkeller.github.io/StaticTools.jl/dev/#StaticTools.MallocArray)
type that behaves similarly to PtrArray. As StaticTools.jl is meant to run without the Julia
runtime, it does not make use of features such as exceptions. Consequently, all usages of
`StaticTools.MallocArray` are implicitly annotated with `@inbounds` and out of bounds
accesses are UB instead of errors, StaticTools.MallocArray with invalid indices will return
a null pointer with size zero while MallocArrays will throw, etc. In general, StaticTools.jl's
MallocArray can be thought of as "unsafe" while MallocArrays.MallocArray is "safe" (though
memory leaks will still occur if you fail to call free)
a null pointer with size zero while PtrArrays will throw, etc. In general, StaticTools.jl's
`MallocArray` can be thought of as "unsafe" while `PtrArray` is "safe" (though memory leaks
will still occur if you call `malloc` and fail to call `free`)

[MallocArrays.jl](https://github.com/LilithHafner/PtrArrays.jl/tree/0b6dbdc012e1058b2b64d0f94863eff4120def85)
is the original name of this package. It is obsolete.
69 changes: 0 additions & 69 deletions src/MallocArrays.jl

This file was deleted.

90 changes: 90 additions & 0 deletions src/PtrArrays.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
module PtrArrays

export malloc, free, PtrArray

"""
PtrArray(ptr::Ptr{T}, dims::Int...; check_dims=true) <: AbstractArray{T}
Wrap a pointer in an `AbstractArray` interface conformant `PtrArray` using the standard
Julia memory order.
Validates that `dims` are non-negative and don't overflow when multiplied if `check_dims` is
true. Wierd things might happen if you set `check_dims=false` and use nagative or
overflowing `dims`.
!!! note
The Julia garbage collector is not able to track `Ptr`s, so the user is responsible for
ensuring that the memory pointed to by `ptr` through `ptr + prod(dims) - 1` remains
allocated throughout the time that the `PtrArray` is used and that mutable objects
stored in a `PtrArray` are prevented from garbage collection.
see also [`malloc`](@ref), [`free`](@ref)
"""
struct PtrArray{T, N} <: AbstractArray{T, N}
ptr::Ptr{T}
size::NTuple{N, Int}
function PtrArray(ptr::Ptr{T}, dims::Vararg{Int, N}; check_dims=true) where {T, N}
check_dims && checked_dims(sizeof(T), dims...; message=:PtrArray)
new{T, N}(ptr, dims)
end
end

# Because Core.checked_dims is buggy 😢
checked_dims(elsize::Int) = elsize
function checked_dims(elsize::Int, d0::Int, d::Int...; message)
overflow = false
neg = (d0+1) < 1
zero = false # of d0==0 we won't have overflow since we go left to right
len = d0
for di in d
len, o = Base.mul_with_overflow(len, di)
zero |= di === 0
overflow |= o
neg |= (di+1) < 1
end
len, o = Base.mul_with_overflow(len, elsize)
err = o | neg | overflow & !zero
err && throw(ArgumentError("invalid $message dimensions"))
len
end

"""
malloc(T::Type, dims::Int...) -> PtrArray{T, N} <: AbstractArray{T, N}
Allocate a new array of type `T` and dimensions `dims` using the C stdlib's `malloc`.
`T` must be an `isbitstype`.
This array is not tracked by Julia's garbage collector, so it is the user's responsibility
to call [`free`](@ref) on it when it is no longer needed.
"""
function malloc(::Type{T}, dims::Int...) where T
isbitstype(T) || throw(ArgumentError("malloc only supports isbits types"))
ptr = Libc.malloc(checked_dims(sizeof(T), dims...; message=:malloc))
ptr === C_NULL && throw(OutOfMemoryError())
PtrArray(Ptr{T}(ptr), dims..., check_dims=false)
end

"""
free(p::PtrArray)
Free the memory allocated by a [`PtrArray`](@ref) allocated by [`malloc`](@ref).
It is only safe to call this function on `PtrArray`s returned by `malloc`, and it is unsafe
to perform any opperation on a `PtrArray` after calling `free`.
"""
free(p::PtrArray) = Libc.free(p.ptr)

Base.size(p::PtrArray) = p.size
Base.IndexStyle(::Type{<:PtrArray}) = IndexLinear()
Base.@propagate_inbounds function Base.getindex(p::PtrArray, i::Int)
@boundscheck checkbounds(p, i)
unsafe_load(p.ptr, i)
end
Base.@propagate_inbounds function Base.setindex!(p::PtrArray, v, i::Int)
@boundscheck checkbounds(p, i)
unsafe_store!(p.ptr, v, i)
p
end

end
10 changes: 5 additions & 5 deletions test/runtests.jl
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
using MallocArrays
using PtrArrays
using Test
using Aqua

@testset "Code quality (Aqua.jl)" begin
Aqua.test_all(MallocArrays, deps_compat=false)
Aqua.test_deps_compat(MallocArrays, check_extras=false)
Aqua.test_all(PtrArrays, deps_compat=false)
Aqua.test_deps_compat(PtrArrays, check_extras=false)
end

@testset "Basics" begin
x = malloc(Int, 10)
@test x isa AbstractVector{Int}
@test x isa MallocArray
@test x isa PtrArray

x .= 1:10
@test x == 1:10
Expand All @@ -26,7 +26,7 @@ end
@test length(y) == 40
@test size(y) == (4, 10)
@test y isa AbstractMatrix{Complex{Float64}}
@test y isa MallocArray
@test y isa PtrArray

fill!(y, im)
@test all(z -> z === 1.0im, y)
Expand Down

2 comments on commit 7597b56

@LilithHafner
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/106439

Tip: Release Notes

Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.

@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.

Tagging

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v1.1.0 -m "<description of version>" 7597b5689c17d1be021e4ce6ad208daaeffc4ffe
git push origin v1.1.0

Also, note the warning: This looks like a new registration that registers version 1.1.0.
Ideally, you should register an initial release with 0.0.1, 0.1.0 or 1.0.0 version numbers
This can be safely ignored. However, if you want to fix this you can do so. Call register() again after making the fix. This will update the Pull request.

Please sign in to comment.