Tidy up for 0.8 (#228)

* Rename and move

* Remove redundant comments

* Shuffle and remove dead code

* Update NEWS

* Bump minor version

* Update news

* Bump docs deps

* Import things

* Tweak docs builder

* Improve test coverage

* Temporary performance bug fix

* stabilise getting_started example

* Finish off extended_mauna_loa upgrade

NEWS.md

@@ -6,6 +6,28 @@ between versions, and discuss new features.
 If you find a breaking change this is not reported here, please either raise an issue or
 make a PR to ammend this document.
 
+## 0.8.0
+
+### Breaking Changes
+
+This version contains some re-naming, specifically
+- `WrappedGP` -> `AtomicGP`,
+- `wrap` -> `atomic`, and
+- `CompositeGP` -> `DerivedGP`,
+which better reflect what these types / functions represent in the context of a `GPPP`.
+It's possible that you've never interacted with them, in which case there's nothing to
+worry about.
+
+Lots of code has been moved around in order to better organise everything.
+
+A method of `vcat` has been added to build `BlockData` from `GPPPInput`s. This can make
+your code look a bit nicer, so you might want to use it.
+
+Additionally, this package no longer depends explicitly upon Zygote or ZygoteRules.
+Instead, all AD rules that are needed use ChainRulesCore directly.
+
+Deprecations mentioned in the 0.7 release have also been dropped.
+
 ## 0.7.16
 - Deprecate `approx_posterior` in favour of `posterior`. This is being removed because it has been removed in AbstractGPs in favour of `posterior`. It will be entirely removed in the next breaking release.
 - Remove some redundant testing infrastructure and tidy up the file structure slightly.

Project.toml

@@ -1,29 +1,22 @@
 name = "Stheno"
 uuid = "8188c328-b5d6-583d-959b-9690869a5511"
-version = "0.7.18"
+version = "0.8.0"
 
 [deps]
 AbstractGPs = "99985d1d-32ba-4be9-9821-2ec096f28918"
 BlockArrays = "8e7c35d0-a365-5155-bbbb-fb81a777f24e"
 ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
-FillArrays = "1a297f60-69ca-5386-bcde-b61e274b549b"
 KernelFunctions = "ec8451be-7e33-11e9-00cf-bbf324bd1392"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
 MacroTools = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
 Reexport = "189a3867-3050-52da-a836-e630ba90ab69"
-Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
-Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
-ZygoteRules = "700de1a5-db45-46bc-99cf-38207098b444"
 
 [compat]
 AbstractGPs = "0.4, 0.5"
 BlockArrays = "0.15, 0.16"
 ChainRulesCore = "1"
-FillArrays = "0.7, 0.8, 0.9, 0.10, 0.11, 0.12"
 KernelFunctions = "0.9.6, 0.10"
 MacroTools = "0.4, 0.5"
 Reexport = "0.2, 1"
-Zygote = "0.6"
-ZygoteRules = "0.2"
 julia = "1.3"

docs/Project.toml

@@ -6,4 +6,4 @@ Stheno = "8188c328-b5d6-583d-959b-9690869a5511"
 [compat]
 Documenter = "0.27"
 julia = "1.6"
-Stheno = "0.7"
+Stheno = "0.8"

docs/make.jl

@@ -1,3 +1,9 @@
+using Pkg
+Pkg.develop(path=joinpath(@__DIR__, ".."))
+
+using Documenter
+using Stheno
+
 ### Process examples
 
 # Always rerun examples
@@ -35,8 +41,9 @@ else
     mkpath(EXAMPLES_OUT)
 end
 
+dev_command = "Pkg.develop(PackageSpec(; path=relpath(\"$(pkgdir(Stheno))\", pwd())));"
 
-let script = "using Pkg; Pkg.activate(ARGS[1]); Pkg.instantiate()"
+let script = "using Pkg; Pkg.activate(ARGS[1]); $dev_command Pkg.instantiate()"
     for example in example_locations
         if !success(`$(Base.julia_cmd()) -e $script $example`)
             error(
@@ -68,10 +75,6 @@ isempty(processes) || success(processes) || error("some examples were not run su
 
 
 ### Build documentation
-using Pkg
-Pkg.develop(path=joinpath(@__DIR__, ".."))
-using Documenter, Stheno
-
 DocMeta.setdocmeta!(
     Stheno,
     :DocTestSetup,

docs/src/input_types.md

@@ -31,7 +31,7 @@ For example, this means that when handling multi-dimensional inputs stored in a
 When constructing a GP whose domain is the real-line, for example when using a GP to solve some kind of time-series problem, it is sufficient to work with `Vector{<:Real}`s of inputs. As such, the following is completely valid:
 ```julia
 using Stheno: GPC
-f = wrap(GP(SqExponentialKernel()), GPC())
+f = atomic(GP(SqExponentialKernel()), GPC())
 x = randn(10)
 f(x)
 ```

docs/src/internals.md

@@ -1,9 +1,9 @@
 # Interfaces
 
 The primary objects in Stheno are some special subtypes of `AbstractGP`. There are three primary concrete subtypes of `AbstractGP`:
-- `WrappedGP`: an atomic Gaussian process given by wrapping an `AbstractGP`.
-- `CompositeGP`: a Gaussian process composed of other `WrappedGP`s and `CompositeGP`s, whose properties are determined recursively from the GPs of which it is composed.
-- `GaussianProcessProbabilisticProgramme` / `GPPP`: a Gaussian process comprising `WrappedGP`s and `CompositeGP`s. This is the primary piece of functionality that users should interact with.
+- `AtomicGP`: an atomic Gaussian process given by wrapping an `AbstractGP`.
+- `CompositeGP`: a Gaussian process composed of other `AtomicGP`s and `CompositeGP`s, whose properties are determined recursively from the GPs of which it is composed.
+- `GaussianProcessProbabilisticProgramme` / `GPPP`: a Gaussian process comprising `AtomicGP`s and `CompositeGP`s. This is the primary piece of functionality that users should interact with.
 
 Each of these types implements the [Internal AbstractGPs API](https://github.com/JuliaGaussianProcesses/AbstractGPs.jl).
 
@@ -13,7 +13,7 @@ This documentation provides the information necessary to understand the internal
 
 It is crucial for pseudo-point methods, and for the computation of marginal statistics at a reasonable scale, to be able to compute the diagonal of a given covariance matrix in linear time in the size of its inputs.
 This, in turn, necessitates that the diagonal of a given cross-covariance matrix can also be computed efficiently as the evaluation of covariance matrices often rely on the evaluation of cross-covariance matrices.
-As such, we have the following functions in addition to the AbstractGPs API implemented for `WrappedGP`s and `CompositeGP`s:
+As such, we have the following functions in addition to the AbstractGPs API implemented for `AtomicGP`s and `CompositeGP`s:
 
 | Function | Brief description |
 |:--------------------- |:---------------------- |
@@ -24,18 +24,18 @@ As such, we have the following functions in addition to the AbstractGPs API impl
 The second and third rows of the table only make sense when `length(x) == length(x′)`, of course.
 
 
-## WrappedGP
+## AtomicGP
 
-We can construct a `WrappedGP` in the following manner:
+We can construct a `AtomicGP` in the following manner:
 
 ```julia
-f = wrap(GP(m, k), gpc)
+f = atomic(GP(m, k), gpc)
 
 ```
 where `m` is its `MeanFunction`, `k` its `Kernel`. `gpc` is a `GPC` object that handles some book-keeping, and is discussed in more depth below.
 
-The `AbstractGP` interface is implemented for `WrappedGP`s in terms of the `AbstractGP` that they wrap.
-So if you want a particular behaviour, just make sure that the `AbstractGP` that you wrap has the functionality you want.
+The `AbstractGP` interface is implemented for `AtomicGP`s in terms of the `AbstractGP` that they atomic.
+So if you want a particular behaviour, just make sure that the `AbstractGP` that you atomic has the functionality you want.
 
 ### Aside: Example Kernel implementation
 
@@ -55,16 +55,16 @@ You should write
 ```julia
 # THIS IS GOOD. PLEASE DO THIS
 gpc = GPC()
-f = wrap(GP(mf, kf), gpc)
-g = wrap(GP(mg, kg), gpc)
+f = atomic(GP(mf, kf), gpc)
+g = atomic(GP(mg, kg), gpc)
 h = f + g
 # THIS IS GOOD. PLEASE DO THIS
 ```
 The rule is simple: when constructing GPs that you plan to make interact later in your program, construct them using the same `gpc` object. For example, DON'T do the following:
 ```julia
 # THIS IS BAD. PLEASE DON'T DO THIS
-f = wrap(GP(mf, kf), GPC())
-g = wrap(GP(mg, kg), GPC())
+f = atomic(GP(mf, kf), GPC())
+g = atomic(GP(mg, kg), GPC())
 h = f + g
 # THIS IS BAD. PLEASE DON'T DO THIS
 ```
@@ -75,7 +75,7 @@ The mistake here is to construct a separate `GPC` object for each `GP`. Hopefull
 
 ## CompositeGP
 
-`CompositeGP`s are constructed as affine transformations of `CompositeGP`s and `WrappedGP`s.
+`CompositeGP`s are constructed as affine transformations of `CompositeGP`s and `AtomicGP`s.
 We describe the implemented transformations below.
 You can add additional transformations -- see [Custom Affine Transformations](@ref) for an a worked example.
 
@@ -129,7 +129,7 @@ This particular function isn't part of the user-facing API because it isn't gene
 ## GPPP
 
 The `GaussianProcessProbabilisticProgramme` is another `AbstractGP` which enables provides a
-thin layer of convenience functionality on top of `WrappedGP`s and `CompositeGP`s, and is
+thin layer of convenience functionality on top of `AtomicGP`s and `CompositeGP`s, and is
 the primary way in which it is intended that users will interact with this package.
 
 A `GPPP` like this
@@ -140,11 +140,11 @@ f = @gppp let
     f3 = f1 + f2
 end
 ```
-is equivalent to manually constructing a `GPPP` using `WrappedGP`s and `CompositeGP`s:
+is equivalent to manually constructing a `GPPP` using `AtomicGP`s and `CompositeGP`s:
 ```julia
 gpc = GPC()
-f1 = wrap(GP(SEKernel()), gpc)
-f2 = wrap(GP(SEKernel()), gpc)
+f1 = atomic(GP(SEKernel()), gpc)
+f2 = atomic(GP(SEKernel()), gpc)
 f3 = f1 + f2
 f = Stheno.GPPP((f1=f1, f2=f2, f3=f3), gpc)
 ```

examples/custom_affine_transformations/script.jl

@@ -17,12 +17,12 @@ using Stheno
 # Suppose that, for some reason, you wish to implement the affine transformation of a single
 # process `f` given by `(Af)(x) = f(x) + f(x + 3) - 2`.
 # In order to define this transformation, first create a function which accepts `f` and
-# returns a `CompositeGP`:
-using Stheno: AbstractGP, CompositeGP, SthenoAbstractGP
+# returns a `DerivedGP`:
+using Stheno: AbstractGP, DerivedGP, SthenoAbstractGP
 
-A(f::SthenoAbstractGP) = CompositeGP((A, f), f.gpc)
+A(f::SthenoAbstractGP) = DerivedGP((A, f), f.gpc)
 
-# The first argument to `CompositeGP` contains `A` itself and any data needed to fully
+# The first argument to `DerivedGP` contains `A` itself and any data needed to fully
 # specify the process results from this transformation. In this case the only piece of
 # information required is `f`, but really any data can be put in this argument.
 # For example, if we wished to replace the translation of `-3` by a parameter, we could do
@@ -48,7 +48,7 @@ const A_args = Tuple{typeof(A), SthenoAbstractGP};
 Stheno.mean((A, f)::A_args, x::AbstractVector) = mean(f, x) .+ mean(f, x .+ 3) .- 2
 
 # The first argument here is _always_ going to be precisely the tuple of arguments passed
-# into the `CompositeGP` constructor above.
+# into the `DerivedGP` constructor above.
 # You can assume that you can compute any statistics of `f` that the AbstractGPs API
 # provides.
 

examples/differentiation/script.jl

@@ -22,9 +22,9 @@ import AbstractGPs: mean, cov, var
 
 using AbstractGPs: AbstractGP
 using AbstractGPs.TestUtils: test_internal_abstractgps_interface
-using Stheno: CompositeGP
+using Stheno: DerivedGP
 
-derivative(f::AbstractGP) = CompositeGP((derivative, f), f.gpc)
+derivative(f::AbstractGP) = DerivedGP((derivative, f), f.gpc)
 
 
 const deriv_args = Tuple{typeof(derivative), AbstractGP}

examples/extended_mauna_loa/script.jl

@@ -209,6 +209,14 @@ function optimize_loss(loss, θ_init; optimizer=default_optimizer, maxiter=1_000
     return unflatten(result.minimizer), result
 end
 
+function KernelFunctions.kernelmatrix(k::ConstantKernel, x::AbstractVector)
+    return fill(only(k.c), length(x), length(x))
+end
+
+function KernelFunctions.kernelmatrix(k::ConstantKernel, x::AbstractVector, y::AbstractVector)
+    return fill(only(k.c), length(x), length(y))
+end
+
 θ_opt, result = optimize_loss(nlml, init_params)
 
 # ## Plot the resulting model fit.

examples/getting_started/script.jl

@@ -129,8 +129,9 @@ using ParameterHandling
     l2 = positive(5.0),
     s2 = positive(1.0),
 
-    ## Observation noise variance -- we'll be learning this as well.
-    s_noise = positive(0.1),
+    ## Observation noise variance -- we'll be learning this as well. Constrained to be
+    ## at least 1e-3.
+    s_noise = positive(0.1, exp, 1e-3),
 )
 
 # We've used `ParameterHandling.jl`s `positive` constraint to ensure that all of the

src/Stheno.jl

@@ -1,68 +1,47 @@
 module Stheno
 
+    # Users generally need access to the functionality from both of these packages.
     using Reexport
+    @reexport using AbstractGPs
+    @reexport using KernelFunctions
 
-    using AbstractGPs
     using BlockArrays
     using ChainRulesCore
-    using FillArrays
-    @reexport using KernelFunctions
     using LinearAlgebra
     using Random
-    using Zygote
-    using ZygoteRules
 
     import Base.Broadcast: broadcasted
 
-    using AbstractGPs: AbstractGP, FiniteGP, GP
-    import AbstractGPs:
-        mean,
-        cov,
-        var,
-        mean_and_cov,
-        mean_and_var,
-        rand,
-        logpdf,
-        elbo,
-        dtc,
-        posterior,
-        marginals
+    using AbstractGPs: AbstractGP, FiniteGP
+    import AbstractGPs: mean, cov, var
 
     using MacroTools: @capture, combinedef, postwalk, splitdef
 
     const AV{T} = AbstractVector{T}
 
-    # Various bits of utility that aren't inherently GP-related. Often very type-piratic.
-    include(joinpath("util", "zygote_rules.jl"))
-    include(joinpath("util", "covariance_matrices.jl"))
-    include(joinpath("util", "block_arrays.jl"))
-    include(joinpath("util", "abstract_data_set.jl"))
-    include(joinpath("util", "proper_type_piracy.jl"))
-
-    # Supertype for GPs.
-    include("abstract_gp.jl")
-
-    # Atomic GP objects.
-    include(joinpath("gp", "gp.jl"))
-
-    # Composite GPs, constructed via affine transformation of CompositeGPs and GPs.
-    include(joinpath("composite", "composite_gp.jl"))
-    include(joinpath("composite", "cross.jl"))
-    include(joinpath("composite", "product.jl"))
-    include(joinpath("composite", "addition.jl"))
-    include(joinpath("composite", "compose.jl"))
-    # include(joinpath("composite", "gradient.jl"))
-    # include(joinpath("composite", "integrate.jl"))
-
-    # Gaussian Process Probabilistic Programme object which implements the AbstractGPs API.
+    # A couple of AbstractVector subtypes useful for expressing structure in inputs
+    # regularly found in GPPPs.
+    include("input_collection_types.jl")
+
+    # AbstractGP subtypes and associated utility.
+    include(joinpath("gp", "util.jl"))
+    include(joinpath("gp", "atomic_gp.jl"))
+    include(joinpath("gp", "derived_gp.jl"))
+    include(joinpath("gp", "sparse_finite_gp.jl"))
+
+    # Affine transformation library. Each file contains one / a couple of closely-related
+    # affine transformations. Consequently, the code in each file can be understood
+    # independently of the code in each other file.
+    include(joinpath("affine_transformations", "cross.jl"))
+    include(joinpath("affine_transformations", "addition.jl"))
+    include(joinpath("affine_transformations", "compose.jl"))
+    include(joinpath("affine_transformations", "product.jl"))
+
+    # AbstractGP subtype which groups together other AbstractGP subtypes.
     include("gaussian_process_probabilistic_programme.jl")
 
-    # Sparse GP hack to make pseudo-point approximations play nicely with Turing.jl.
-    include("sparse_finite_gp.jl")
-
-    include("deprecate.jl")
-
-    export wrap, BlockData, GPC, GPPPInput, @gppp
+    export atomic, BlockData, GPC, GPPPInput, @gppp
     export ∘, select, stretch, periodic, shift
-    export cov_diag, mean_and_cov_diag
+    export SparseFiniteGP
+
 end # module

src/composite/addition.jl → src/affine_transformations/addition.jl

@@ -7,7 +7,7 @@ Produces an AbstractGP `f` satisfying `f(x) = fa(x) + fb(x)`.
 """
 function +(fa::AbstractGP, fb::AbstractGP)
     @assert fa.gpc === fb.gpc
-    return CompositeGP((+, fa, fb), fa.gpc)
+    return DerivedGP((+, fa, fb), fa.gpc)
 end
 -(fa::AbstractGP, fb::AbstractGP) = fa + (-fb)
 
@@ -52,10 +52,10 @@ end
 
 
 #
-# Add a constant or known function to a GP -- just shifts the mean
+# Add a constant or known function to an AbstractGP -- just shifts the mean
 #
 
-+(b, f::AbstractGP) = CompositeGP((+, b, f), f.gpc)
++(b, f::AbstractGP) = DerivedGP((+, b, f), f.gpc)
 +(f::AbstractGP, b) = b + f
 -(b::Real, f::AbstractGP) = b + (-f)
 -(f::AbstractGP, b::Real) = f + (-b)