build based on 314c4ae

previews/PR1453/adjoints/index.html

@@ -100,4 +100,4 @@
 1 levels of nesting
 
 julia> grad(x -> x*grad(f, x), 1);
-2 levels of nesting</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../limitations/">« Limitations</a><a class="docs-footer-nextpage" href="../utils/">Utilities »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Friday 8 September 2023 03:04">Friday 8 September 2023</span>. Using Julia version 1.9.3.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+2 levels of nesting</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../limitations/">« Limitations</a><a class="docs-footer-nextpage" href="../utils/">Utilities »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Saturday 4 January 2025 21:20">Saturday 4 January 2025</span>. Using Julia version 1.11.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR1453/complex/index.html

@@ -27,4 +27,4 @@
 (8.0 + 12.0im, 0.0 + 0.0im)
 
 julia> wirtinger(x -> abs2(x), 1+2im)
-(1.0 - 2.0im, 1.0 + 2.0im)</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../utils/">« Utilities</a><a class="docs-footer-nextpage" href="../profiling/">Profiling »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Friday 8 September 2023 03:04">Friday 8 September 2023</span>. Using Julia version 1.9.3.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+(1.0 - 2.0im, 1.0 + 2.0im)</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../utils/">« Utilities</a><a class="docs-footer-nextpage" href="../profiling/">Profiling »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Saturday 4 January 2025 21:20">Saturday 4 January 2025</span>. Using Julia version 1.11.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR1453/internals/index.html

@@ -135,4 +135,4 @@
 julia> y, back = Zygote._pullback(bad, 1);
 
 julia> back(1) # ok, here's our issue. Lather, rinse, repeat.
-ERROR: bad</code></pre><p>Of course, our goal is that you never have to do this, but until Zygote is more mature it can be a useful way to narrow down test cases.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../profiling/">« Profiling</a><a class="docs-footer-nextpage" href="../glossary/">Glossary »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Friday 8 September 2023 03:04">Friday 8 September 2023</span>. Using Julia version 1.9.3.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+ERROR: bad</code></pre><p>Of course, our goal is that you never have to do this, but until Zygote is more mature it can be a useful way to narrow down test cases.</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../profiling/">« Profiling</a><a class="docs-footer-nextpage" href="../glossary/">Glossary »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Saturday 4 January 2025 21:20">Saturday 4 January 2025</span>. Using Julia version 1.11.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR1453/limitations/index.html

@@ -43,23 +43,24 @@
 julia> gradient(rand(3)) do y
          sum(g(y))
        end
-([2.0, 2.0, 2.0],)</code></pre><h2 id="Try-catch-statements-1"><a class="docs-heading-anchor" href="#Try-catch-statements-1">Try-catch statements</a><a class="docs-heading-anchor-permalink" href="#Try-catch-statements-1" title="Permalink"></a></h2><p>Any expressions involving <code>try</code>/<code>catch</code> statements is not supported.</p><pre><code class="language-julia">function tryme(x)
-  try
-    2 * x
-  catch e
-    throw(e)
-  end
-end
-
-julia&gt; gradient(rand(3)) do x
-         sum(tryme(x))
+([2.0, 2.0, 2.0],)</code></pre><h2 id="Try-catch-statements-1"><a class="docs-heading-anchor" href="#Try-catch-statements-1">Try-catch statements</a><a class="docs-heading-anchor-permalink" href="#Try-catch-statements-1" title="Permalink"></a></h2><p>Code containting try-catch blocks can be differentiated as long as no exception is actually thrown.</p><pre><code class="language-julia">julia&gt; function safe_sqrt(x)
+           try
+               sqrt(x)
+           catch
+               0.
+           end
        end
-ERROR: Compiling Tuple{typeof(tryme), Vector{Float64}}: try/catch is not supported.
-Refer to the Zygote documentation for fixes.
-https://fluxml.ai/Zygote.jl/latest/limitations
+safe_sqrt (generic function with 1 method)
 
-Stacktrace:
-  ...</code></pre><p>Here <code>tryme</code> uses a <code>try</code>/<code>catch</code> statement, and Zygote throws an error when trying to differentiate it as expected. <code>try</code>/<code>catch</code> expressions are used for error handling, but they are less common in Julia compared to some other languages.</p><h2 id="Foreign-call-expressions-1"><a class="docs-heading-anchor" href="#Foreign-call-expressions-1">Foreign call expressions</a><a class="docs-heading-anchor-permalink" href="#Foreign-call-expressions-1" title="Permalink"></a></h2><p>Foreign call expressions refer to expressions that call external libraries such as code written in C or Fortran. You may want to read more about these calls in the <a href="https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code/">Julia documentation</a>. Scientific computing libraries in Julia may call established C or Fortran libraries under the hood. Since the underlying code for a foreign call expression is not in Julia, it is not possible for Zygote to differentiate this expression.</p><p>Below, we define a function that calls a standard C function, <code>clock</code>. This function returns the Unix clock as an <code>Int32</code>.</p><pre><code class="language-julia">julia&gt; jclock(x) = ccall(:clock, Int32, ()) * 2
+julia&gt; gradient(safe_sqrt, 4.)
+(0.25,)
+
+julia&gt; val, pull = pullback(safe_sqrt, -1.)
+(0.0, Zygote.var&quot;#76#77&quot;{Zygote.Pullback{Tuple{typeof(safe_sqrt), Float64}, Any}}(∂(safe_sqrt)))
+
+julia&gt; pull(1.)
+ERROR: Can&#39;t differentiate function execution in catch block at #= REPL[2]:3 =#.
+Stacktrace:</code></pre><p>Here, the <code>safe_sqrt</code> function catches DomainError from the sqrt call when the input is out of domain and safely returns 0. Zygote is able to differentiate the function when no error is thrown by the sqrt call, but fails to differentiate when the control flow goes through the catch block.</p><h2 id="Foreign-call-expressions-1"><a class="docs-heading-anchor" href="#Foreign-call-expressions-1">Foreign call expressions</a><a class="docs-heading-anchor-permalink" href="#Foreign-call-expressions-1" title="Permalink"></a></h2><p>Foreign call expressions refer to expressions that call external libraries such as code written in C or Fortran. You may want to read more about these calls in the <a href="https://docs.julialang.org/en/v1/manual/calling-c-and-fortran-code/">Julia documentation</a>. Scientific computing libraries in Julia may call established C or Fortran libraries under the hood. Since the underlying code for a foreign call expression is not in Julia, it is not possible for Zygote to differentiate this expression.</p><p>Below, we define a function that calls a standard C function, <code>clock</code>. This function returns the Unix clock as an <code>Int32</code>.</p><pre><code class="language-julia">julia&gt; jclock(x) = ccall(:clock, Int32, ()) * 2
 jclock (generic function with 1 method)
 
 julia&gt; jclock(2)
@@ -87,4 +88,4 @@
     tot += x^n  # binds symbol `tot` to new value
   end
   return tot
-end</code></pre><p>However, sometimes such re-binding confuses Zygote, especially if the type of the value changes. Especially if the variable is &quot;boxed&quot;, as will happen if you re-bind from within a closure (such as the function created by a <code>do</code> block).</p><h2 id="Second-derivatives-1"><a class="docs-heading-anchor" href="#Second-derivatives-1">Second derivatives</a><a class="docs-heading-anchor-permalink" href="#Second-derivatives-1" title="Permalink"></a></h2><p>In principle Zygote supports taking derivatives of derivatives. There are, however, a few problems:</p><ul><li>Quite a few of its rules are not written in a way that is itself differentiable. For instance they may work by making an array then writing into it, which is mutation of the sort forbidden above. </li><li>The complexity of the code grows rapidly, as Zygote differentiates its own un-optimised output.</li><li>Reverse mode over reverse mode is seldom the best algorithm.</li></ul><p>The issue tracker has a label for <a href="https://github.com/FluxML/Zygote.jl/issues?q=is%3Aissue+is%3Aopen+label%3A%22second+order%22">second order</a>, which will outline where the bodies are buried.</p><p>Often using a different AD system over Zygote is a better solution. This is what <a href="../utils/#Zygote.hessian"><code>hessian</code></a> does, using ForwardDiff over Zygote, but other combinations are possible. (Note that rules defined here mean that Zygote over ForwardDiff is translated to ForwardDiff over ForwardDiff.)</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../">« Home</a><a class="docs-footer-nextpage" href="../adjoints/">Custom Adjoints »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Friday 8 September 2023 03:04">Friday 8 September 2023</span>. Using Julia version 1.9.3.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+end</code></pre><p>However, sometimes such re-binding confuses Zygote, especially if the type of the value changes. Especially if the variable is &quot;boxed&quot;, as will happen if you re-bind from within a closure (such as the function created by a <code>do</code> block).</p><h2 id="Second-derivatives-1"><a class="docs-heading-anchor" href="#Second-derivatives-1">Second derivatives</a><a class="docs-heading-anchor-permalink" href="#Second-derivatives-1" title="Permalink"></a></h2><p>In principle Zygote supports taking derivatives of derivatives. There are, however, a few problems:</p><ul><li>Quite a few of its rules are not written in a way that is itself differentiable. For instance they may work by making an array then writing into it, which is mutation of the sort forbidden above. </li><li>The complexity of the code grows rapidly, as Zygote differentiates its own un-optimised output.</li><li>Reverse mode over reverse mode is seldom the best algorithm.</li></ul><p>The issue tracker has a label for <a href="https://github.com/FluxML/Zygote.jl/issues?q=is%3Aissue+is%3Aopen+label%3A%22second+order%22">second order</a>, which will outline where the bodies are buried.</p><p>Often using a different AD system over Zygote is a better solution. This is what <a href="../utils/#Zygote.hessian"><code>hessian</code></a> does, using ForwardDiff over Zygote, but other combinations are possible. (Note that rules defined here mean that Zygote over ForwardDiff is translated to ForwardDiff over ForwardDiff.)</p></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../">« Home</a><a class="docs-footer-nextpage" href="../adjoints/">Custom Adjoints »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Saturday 4 January 2025 21:20">Saturday 4 January 2025</span>. Using Julia version 1.11.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

previews/PR1453/profiling/index.html

@@ -28,4 +28,4 @@
 │   %2 = (Base.mul_int)(Δ, 1)::Int64
 │   %3 = (Zygote.tuple)(nothing, %1, %2)::PartialTuple(Tuple{Nothing,Int64,Int64}, Any[Const(nothing, false), Int64, Int64])
 └──      return %3
-) =&gt; Tuple{Nothing,Int64,Int64}</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../complex/">« Complex Differentiation</a><a class="docs-footer-nextpage" href="../internals/">Internals »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Friday 8 September 2023 03:04">Friday 8 September 2023</span>. Using Julia version 1.9.3.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
+) =&gt; Tuple{Nothing,Int64,Int64}</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../complex/">« Complex Differentiation</a><a class="docs-footer-nextpage" href="../internals/">Internals »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Saturday 4 January 2025 21:20">Saturday 4 January 2025</span>. Using Julia version 1.11.2.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>