-
Notifications
You must be signed in to change notification settings - Fork 203
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds a pertubation advection open boundary matching scheme #3977
Open
jagoosw
wants to merge
55
commits into
main
Choose a base branch
from
jsw/pertubation-advection-obc
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+568
−81
Open
Changes from all commits
Commits
Show all changes
55 commits
Select commit
Hold shift + click to select a range
d992d81
fix FEOBC
jagoosw 2a374cb
x-boundary condition implemented
jagoosw babe922
changed relaxation a bit
jagoosw 32cd354
cleaning up + validation case
jagoosw 5a0285f
changed to pointwise
jagoosw fef6185
oops
jagoosw fa5a471
fixed open boundary filling + tests
jagoosw 29a3b4a
Merge branch 'main' into jsw/fix-feobc
jagoosw b55395c
generalised and fixed bug in left boundary
jagoosw 1a9fbec
bump patch
jagoosw e66f3d1
Formating
jagoosw dc73eba
Merge branch 'main' into jsw/fix-feobc
jagoosw 51c9938
test problem
jagoosw 94ce249
Merge branch 'main' into jsw/fix-feobc
jagoosw 6ff1bc7
bump patch
jagoosw b10f833
Merge remote-tracking branch 'origin' into jsw/pertubation-advection-obc
jagoosw a7cdcfd
Merge branch 'main' into jsw/fix-feobc
jagoosw 62e27b2
clean up
jagoosw a23adae
boundary normal velocity
jagoosw 5246c9a
oops
jagoosw a34c927
Rename
jagoosw 0df1abe
finish renaming
jagoosw ff0c79e
renamed
jagoosw 3257eec
kind of GPU frieldly
jagoosw 07cc471
typo
jagoosw 9ae0b66
bug
jagoosw ec9dd88
bug
jagoosw 9859c17
Merge branch 'main' into jsw/fix-feobc
jagoosw 0e02a2b
Merge branch 'main' into jsw/fix-feobc
jagoosw 9912b51
Merge remote-tracking branch 'origin/jsw/fix-feobc' into jsw/pertubat…
jagoosw 96495b1
allow forward euler integration
jagoosw c49ed26
fix adapt
jagoosw 842ef63
mistake
jagoosw 73cd923
changed a function name
jagoosw 57261e2
Merge branch 'main' into jsw/pertubation-advection-obc
jagoosw b961259
fix src/Oceanaingans conflicts (?)
jagoosw 9f30bbf
Merge branch 'main' into jsw/pertubation-advection-obc
jagoosw 85bf032
Merge branch 'main' into jsw/pertubation-advection-obc
tomchor 895d3f2
Merge branch 'main' into jsw/pertubation-advection-obc
tomchor 6bb4bbf
add x and y directions
jagoosw e412995
added test
jagoosw 549e4cd
Merge branch 'main' into jsw/pertubation-advection-obc
jagoosw dddb91a
Merge branch 'main' into jsw/pertubation-advection-obc
jagoosw 00d56cf
Apply suggestions from code review
jagoosw 0de469d
commented tests
jagoosw 270d26a
tidied up validation
jagoosw f20894e
fixed adapt
jagoosw ef4d4d7
fix validation case
jagoosw 2b15848
Update validation/open_boundaries/cylinder.jl
jagoosw d12414c
added plotting to validation
jagoosw 335926e
added explanation to docstring
jagoosw c3241ca
corrected validation script
jagoosw fa94dd1
Merge branch 'main' into jsw/pertubation-advection-obc
jagoosw 35f42b2
removed @allowscalars
jagoosw a13ffd2
Merge branch 'main' into jsw/pertubation-advection-obc
tomchor File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
242 changes: 242 additions & 0 deletions
242
src/BoundaryConditions/perturbation_advection_open_boundary_matching_scheme.jl
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,242 @@ | ||
using Oceananigans.Operators: Δxᶠᶜᶜ, Δyᶜᶠᶜ, Δzᶜᶜᶠ, Ax_qᶠᶜᶜ, Ay_qᶜᶠᶜ, Az_qᶜᶜᶠ | ||
|
||
""" | ||
PerturbationAdvection | ||
|
||
For cases where we assume that the internal flow is a small perturbation from | ||
an external prescribed or coarser flow, we can split the velocity into background | ||
and perturbation components. | ||
|
||
We begin with the equation governing the fluid in the interior: | ||
∂ₜu⃗ + u⃗⋅∇u⃗ = −∇P + F⃗, | ||
and note that on the boundary the pressure gradient is zero. | ||
We can then assume that the flow composes of mean (U⃗) and pertubation (u⃗′) components, | ||
and considering the x-component of velocity, we can rewrite the equation as | ||
∂ₜu₁ = -u₁∂₁u - u₂∂₂u₁ - u₃∂₃u₁ + F₁ ≈ - U₁∂₁u₁′ - U₂∂₂u₁′ - U₃∂₃u₁′ + F. | ||
|
||
Simplify by assuming that U⃗ = Ux̂, an then take a numerical step to find u₁. | ||
|
||
When the boundaries are filled the interior is at time tₙ₊₁ so we can take | ||
a backwards euler step (in the case that the mean flow is boundary normal) on a right boundary: | ||
(Uⁿ⁺¹ - Uⁿ) / Δt + (u′ⁿ⁺¹ - u′ⁿ) / Δt = - Uⁿ⁺¹ (u′ⁿ⁺¹ᵢ - u′ⁿ⁺¹ᵢ₋₁) / Δx + Fᵤ. | ||
|
||
This can not be solved for general forcing, but if we assume the dominant forcing is | ||
relaxation to the mean velocity (i.e. u′→0) then Fᵤ = -u′ / τ then we can find u′ⁿ⁺¹: | ||
u′ⁿ⁺¹ = (uⁿ + Ũu′ⁿ⁺¹ᵢ₋₁ - Uⁿ⁺¹) / (1 + Ũ + Δt/τ), | ||
|
||
where Ũ = U Δt / Δx, then uⁿ⁺¹ is: | ||
uⁿ⁺¹ = (uᵢⁿ + Ũuᵢ₋₁ⁿ⁺¹ + Uⁿ⁺¹τ̃) / (1 + τ̃ + U) | ||
|
||
where τ̃ = Δt/τ. | ||
|
||
The same operation can be repeated for left boundaries. | ||
""" | ||
struct PerturbationAdvection{VT, FT} | ||
backward_step :: VT | ||
inflow_timescale :: FT | ||
outflow_timescale :: FT | ||
end | ||
|
||
Adapt.adapt_structure(to, pe::PerturbationAdvection) = | ||
PerturbationAdvection(adapt(to, pe.backward_step), | ||
adapt(to, pe.inflow_timescale), | ||
adapt(to, pe.outflow_timescale)) | ||
|
||
function PerturbationAdvectionOpenBoundaryCondition(val, FT = Float64; | ||
backward_step = true, | ||
outflow_timescale = Inf, | ||
inflow_timescale = 300.0, kwargs...) | ||
|
||
classification = Open(PerturbationAdvection(Val(backward_step), inflow_timescale, outflow_timescale)) | ||
|
||
@warn "`PerturbationAdvection` open boundaries matching scheme is experimental and un-tested/validated" | ||
|
||
return BoundaryCondition(classification, val; kwargs...) | ||
end | ||
|
||
const PAOBC = BoundaryCondition{<:Open{<:PerturbationAdvection}} | ||
|
||
const BPAOBC = BoundaryCondition{<:Open{<:PerturbationAdvection{Val{true}}}} | ||
const FPAOBC = BoundaryCondition{<:Open{<:PerturbationAdvection{Val{false}}}} | ||
|
||
@inline function step_right_boundary!(bc::BPAOBC, l, m, boundary_indices, boundary_adjacent_indices, | ||
grid, u, clock, model_fields, ΔX) | ||
Δt = clock.last_stage_Δt | ||
|
||
Δt = ifelse(isinf(Δt), 0, Δt) | ||
|
||
ūⁿ⁺¹ = getbc(bc, l, m, grid, clock, model_fields) | ||
|
||
uᵢⁿ = @inbounds getindex(u, boundary_indices...) | ||
uᵢ₋₁ⁿ⁺¹ = @inbounds getindex(u, boundary_adjacent_indices...) | ||
|
||
U = max(0, min(1, Δt / ΔX * ūⁿ⁺¹)) | ||
|
||
pa = bc.classification.matching_scheme | ||
|
||
τ = ifelse(ūⁿ⁺¹ >= 0, pa.outflow_timescale, pa.inflow_timescale) | ||
|
||
τ̃ = Δt / τ | ||
|
||
uᵢⁿ⁺¹ = (uᵢⁿ + U * uᵢ₋₁ⁿ⁺¹ + ūⁿ⁺¹ * τ̃) / (1 + τ̃ + U) | ||
|
||
@inbounds setindex!(u, uᵢⁿ⁺¹, boundary_indices...) | ||
|
||
return nothing | ||
end | ||
|
||
@inline function step_left_boundary!(bc::BPAOBC, l, m, boundary_indices, boundary_adjacent_indices, boundary_secret_storage_indices, | ||
grid, u, clock, model_fields, ΔX) | ||
Δt = clock.last_stage_Δt | ||
|
||
Δt = ifelse(isinf(Δt), 0, Δt) | ||
|
||
ūⁿ⁺¹ = getbc(bc, l, m, grid, clock, model_fields) | ||
|
||
uᵢⁿ = @inbounds getindex(u, boundary_secret_storage_indices...) | ||
uᵢ₋₁ⁿ⁺¹ = @inbounds getindex(u, boundary_adjacent_indices...) | ||
|
||
U = min(0, max(-1, Δt / ΔX * ūⁿ⁺¹)) | ||
|
||
pa = bc.classification.matching_scheme | ||
|
||
τ = ifelse(ūⁿ⁺¹ <= 0, pa.outflow_timescale, pa.inflow_timescale) | ||
|
||
τ̃ = Δt / τ | ||
|
||
u₁ⁿ⁺¹ = (uᵢⁿ - U * uᵢ₋₁ⁿ⁺¹ + ūⁿ⁺¹ * τ̃) / (1 + τ̃ - U) | ||
|
||
@inbounds setindex!(u, u₁ⁿ⁺¹, boundary_indices...) | ||
@inbounds setindex!(u, u₁ⁿ⁺¹, boundary_secret_storage_indices...) | ||
|
||
return nothing | ||
end | ||
|
||
|
||
@inline function step_right_boundary!(bc::FPAOBC, l, m, boundary_indices, boundary_adjacent_indices, | ||
grid, u, clock, model_fields, ΔX) | ||
Δt = clock.last_stage_Δt | ||
|
||
Δt = ifelse(isinf(Δt), 0, Δt) | ||
|
||
ūⁿ⁺¹ = getbc(bc, l, m, grid, clock, model_fields) | ||
|
||
uᵢⁿ = @inbounds getindex(u, boundary_indices...) | ||
uᵢ₋₁ⁿ⁺¹ = @inbounds getindex(u, boundary_adjacent_indices...) | ||
|
||
U = max(0, min(1, Δt / ΔX * ūⁿ⁺¹)) | ||
|
||
pa = bc.classification.matching_scheme | ||
|
||
τ = ifelse(ūⁿ⁺¹ >= 0, pa.outflow_timescale, pa.inflow_timescale) | ||
|
||
τ̃ = Δt / τ | ||
|
||
uᵢⁿ⁺¹ = uᵢⁿ + U * (uᵢ₋₁ⁿ⁺¹ - ūⁿ⁺¹) + (ūⁿ⁺¹ - uᵢⁿ) * τ̃ | ||
|
||
@inbounds setindex!(u, uᵢⁿ⁺¹, boundary_indices...) | ||
|
||
return nothing | ||
end | ||
|
||
@inline function step_left_boundary!(bc::FPAOBC, l, m, boundary_indices, boundary_adjacent_indices, boundary_secret_storage_indices, | ||
grid, u, clock, model_fields, ΔX) | ||
Δt = clock.last_stage_Δt | ||
|
||
Δt = ifelse(isinf(Δt), 0, Δt) | ||
|
||
ūⁿ⁺¹ = getbc(bc, l, m, grid, clock, model_fields) | ||
|
||
uᵢⁿ = @inbounds getindex(u, boundary_secret_storage_indices...) | ||
uᵢ₋₁ⁿ⁺¹ = @inbounds getindex(u, boundary_adjacent_indices...) | ||
|
||
U = min(0, max(-1, Δt / ΔX * ūⁿ⁺¹)) | ||
|
||
pa = bc.classification.matching_scheme | ||
|
||
τ = ifelse(ūⁿ⁺¹ <= 0, pa.outflow_timescale, pa.inflow_timescale) | ||
|
||
τ̃ = Δt / τ | ||
|
||
u₁ⁿ⁺¹ = uᵢⁿ - U * (uᵢ₋₁ⁿ⁺¹ - ūⁿ⁺¹) + (ūⁿ⁺¹ - uᵢⁿ) * τ̃ | ||
|
||
@inbounds setindex!(u, u₁ⁿ⁺¹, boundary_indices...) | ||
@inbounds setindex!(u, u₁ⁿ⁺¹, boundary_secret_storage_indices...) | ||
|
||
return nothing | ||
end | ||
|
||
@inline function _fill_east_halo!(j, k, grid, u, bc::PAOBC, ::Tuple{Face, Any, Any}, clock, model_fields) | ||
i = grid.Nx + 1 | ||
|
||
boundary_indices = (i, j, k) | ||
boundary_adjacent_indices = (i-1, j, k) | ||
|
||
Δx = Δxᶠᶜᶜ(i, j, k, grid) | ||
|
||
step_right_boundary!(bc, j, k, boundary_indices, boundary_adjacent_indices, grid, u, clock, model_fields, Δx) | ||
|
||
return nothing | ||
end | ||
|
||
@inline function _fill_west_halo!(j, k, grid, u, bc::PAOBC, ::Tuple{Face, Any, Any}, clock, model_fields) | ||
boundary_indices = (1, j, k) | ||
boundary_adjacent_indices = (2, j, k) | ||
boundary_secret_storage_indices = (0, j, k) | ||
|
||
Δx = Δxᶠᶜᶜ(1, j, k, grid) | ||
|
||
step_left_boundary!(bc, j, k, boundary_indices, boundary_adjacent_indices, boundary_secret_storage_indices, grid, u, clock, model_fields, Δx) | ||
|
||
return nothing | ||
end | ||
|
||
@inline function _fill_north_halo!(i, k, grid, u, bc::PAOBC, ::Tuple{Any, Face, Any}, clock, model_fields) | ||
j = grid.Ny + 1 | ||
|
||
boundary_indices = (i, j, k) | ||
boundary_adjacent_indices = (i, j-1, k) | ||
|
||
Δy = Δyᶜᶠᶜ(i, j, k, grid) | ||
|
||
step_right_boundary!(bc, i, k, boundary_indices, boundary_adjacent_indices, grid, u, clock, model_fields, Δy) | ||
|
||
return nothing | ||
end | ||
|
||
@inline function _fill_south_halo!(i, k, grid, u, bc::PAOBC, ::Tuple{Any, Face, Any}, clock, model_fields) | ||
boundary_indices = (i, 1, k) | ||
boundary_adjacent_indices = (i, 1, k) | ||
boundary_secret_storage_indices = (i, 0, k) | ||
|
||
Δy = Δyᶜᶠᶜ(i, 1, k, grid) | ||
|
||
step_left_boundary!(bc, i, k, boundary_indices, boundary_adjacent_indices, boundary_secret_storage_indices, grid, u, clock, model_fields, Δy) | ||
|
||
return nothing | ||
end | ||
|
||
@inline function _fill_top_halo!(i, j, grid, u, bc::PAOBC, ::Tuple{Any, Any, Face}, clock, model_fields) | ||
k = grid.Nz + 1 | ||
|
||
boundary_indices = (i, j, k) | ||
boundary_adjacent_indices = (i, j, k-1) | ||
|
||
Δz = Δzᶜᶜᶠ(i, j, k, grid) | ||
|
||
step_right_boundary!(bc, i, j, boundary_indices, boundary_adjacent_indices, grid, u, clock, model_fields, Δz) | ||
|
||
return nothing | ||
end | ||
|
||
@inline function _fill_bottom_halo!(i, j, grid, u, bc::PAOBC, ::Tuple{Any, Any, Face}, clock, model_fields) | ||
boundary_indices = (i, j, 1) | ||
boundary_adjacent_indices = (i, j, 1) | ||
boundary_secret_storage_indices = (i, j, 0) | ||
|
||
Δz = Δzᶜᶜᶠ(i, j, 1, grid) | ||
|
||
step_left_boundary!(bc, i, j, boundary_indices, boundary_adjacent_indices, boundary_secret_storage_indices, grid, u, clock, model_fields, Δz) | ||
|
||
return nothing | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
using Adapt | ||
using Oceananigans: instantiated_location | ||
using Oceananigans.Fields: Center, Face | ||
using Oceananigans.AbstractOperations: GridMetricOperation, Ax, Ay, Az | ||
using Oceananigans.BoundaryConditions: BoundaryCondition, Open, PerturbationAdvection | ||
|
||
import Adapt: adapt_structure | ||
import Oceananigans.BoundaryConditions: update_boundary_condition! | ||
|
||
struct BoundaryAdjacentMean{V} | ||
value :: V | ||
|
||
BoundaryAdjacentMean(; value::BV = Ref(0.0)) where BV = new{BV}(value) | ||
end | ||
|
||
@inline (bam::BoundaryAdjacentMean)(args...) = bam.value[] | ||
|
||
Adapt.adapt_structure(to, mo::BoundaryAdjacentMean) = | ||
BoundaryAdjacentMean(; value = adapt(to, mo.value[])) | ||
|
||
const MOPABC = BoundaryCondition{<:Open{<:PerturbationAdvection}, <:BoundaryAdjacentMean} | ||
|
||
@inline boundary_normal_area(::Union{Val{:west}, Val{:east}}, grid) = GridMetricOperation((Face, Center, Center), Ax, grid) | ||
@inline boundary_normal_area(::Union{Val{:south}, Val{:north}}, grid) = GridMetricOperation((Center, Face, Center), Ay, grid) | ||
@inline boundary_normal_area(::Union{Val{:bottom}, Val{:top}}, grid) = GridMetricOperation((Center, Center, Face), Az, grid) | ||
|
||
@inline boundary_adjacent_indices(::Val{:east}, grid, loc) = (size(grid, 1), 1, 1), (2, 3) | ||
@inline boundary_adjacent_indices(val_side::Val{:west}, grid, loc) = (first_interior_index(val_side, loc), 1, 1), (2, 3) | ||
|
||
@inline boundary_adjacent_indices(::Val{:north}, grid, loc) = (1, size(grid, 2), 1), (1, 3) | ||
@inline boundary_adjacent_indices(val_side::Val{:south}, grid, loc) = (1, first_interior_index(val_side, loc), 1), (1, 3) | ||
|
||
@inline boundary_adjacent_indices(::Val{:top}, grid, loc) = (1, 1, size(grid, 3)), (2, 3) | ||
@inline boundary_adjacent_indices(val_side::Val{:bottom}, grid, loc) = (1, 1, first_interior_index(val_side, loc)), (2, 3) | ||
|
||
@inline first_interior_index(::Union{Val{:west}, Val{:east}}, ::Tuple{Center, <:Any, <:Any}) = 1 | ||
@inline first_interior_index(::Union{Val{:west}, Val{:east}}, ::Tuple{Face, <:Any, <:Any}) = 2 | ||
|
||
@inline first_interior_index(::Union{Val{:south}, Val{:north}}, ::Tuple{<:Any, Center, <:Any}) = 1 | ||
@inline first_interior_index(::Union{Val{:south}, Val{:north}}, ::Tuple{<:Any, Face, <:Any}) = 2 | ||
|
||
@inline first_interior_index(::Union{Val{:bottom}, Val{:top}}, ::Tuple{<:Any, <:Any, Center}) = 1 | ||
@inline first_interior_index(::Union{Val{:bottom}, Val{:top}}, ::Tuple{<:Any, <:Any, Face}) = 2 | ||
|
||
function update_boundary_condition!(bc::MOPABC, val_side, u, model) | ||
grid = model.grid | ||
loc = instantiated_location(u) | ||
|
||
An = boundary_normal_area(val_side, grid) | ||
(i, j, k), dims = boundary_adjacent_indices(val_side, grid, loc) | ||
total_area = CUDA.@allowscalar sum(An; dims)[i, j, k] | ||
Ū = sum(u * An; dims) / total_area | ||
|
||
bc.condition.value[] = CUDA.@allowscalar Ū[i, j, k] | ||
return nothing | ||
end |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jagoosw you still need to remove this file from here, right?
Is this only used for the perturbation OBC? If so, it can go in the same directory.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We never resolved this, the problem is that
AbstractOperations
are defined afterBoundaryConditions
so this can't go in the boundary moduleThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I see. I don't have a good answer here, but I don't think it can be in
/src
. @glwagner @simone-silvestri can you advise here?Also, is there a simple way to define these functions without
AbstractOperation
s?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be possible to write it without but it would not be trivial to do it for general grids and would end up reinventing the wheel to get the flux.
Perhaps we could move it to
Utils
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm okay with that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah utils is imported long before fields and boundary conditions too because it includes the kernel launching stuff
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@glwagner do you have any preferences as to where to put this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would also not object to removing it from the source code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you explain why that piece of code is necessary and what would be the impact of removing it? It's not 100% clear to me.