Avoid recursion heuristic?

[chriselrod]

Julia 1.7 + this PR:

julia> using ForwardDiff, StaticArrays
[ Info: Precompiling ForwardDiff [f6369f11-7733-5829-9624-2563aa707210]

julia> using ForwardDiff, StaticArrays

julia> const ARG = Ref{Any}()
Base.RefValue{Any}(#undef)

julia> foo(x) = (ARG[] = x; sum(x))
foo (generic function with 1 method)

julia> ForwardDiff.jacobian(x -> ForwardDiff.hessian(foo, x), @SVector(rand(8))); eltype(ARG[])
ForwardDiff.Dual{ForwardDiff.Tag{typeof(foo), ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 8}, 8}

julia> @descend_code_warntype convert(ForwardDiff.Dual{ForwardDiff.Tag{typeof(foo), ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, 2}, 2.3)
convert(::Type{ForwardDiff.Dual{T, V, N}}, x::Number) where {T, V, N} in ForwardDiff at /home/chriselrod/.julia/dev/ForwardDiff/src/dual.jl:385
Body::Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{typeof(foo), ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, 2}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Float64, Core.Const(Partials(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0))]), Core.Const(Partials(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)))]), Core.Const(Partials(Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)), Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0))))])
    1 ──       nothing                             │
385 2 ── %2  = %new(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, x, $(QuoteNode(Partials(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0))))::Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Float64, Core.Const(Partials(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0))])
    └───       goto #3                             ││┃││││   convert
    3 ──       goto #4                             │││┃││     Dual
    4 ──       goto #5                             ││││┃       convert
    5 ──       goto #6                             ││││
    6 ── %7  = Core.tuple(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0))::Core.Const((Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)))
    └─── %8  = %new(ForwardDiff.Partials{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, %7)::Core.Const(Partials(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)))
    7 ── %9  = %new(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, %2, %8)::Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Float64, Core.Const(Partials(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0))]), Core.Const(Partials(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)))])
    └───       goto #8                             │││││
    8 ──       goto #9                             ││││
    9 ──       goto #10                            │││
    10 ─       goto #11                            ││
    11 ─ %14 = Core.tuple(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0))::Core.Const((Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)))
    └─── %15 = %new(ForwardDiff.Partials{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, %14)::Core.Const(Partials(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)))
    12 ─ %16 = %new(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), %15)::Core.Const(Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)))
    └───       goto #13                            │││││┃│      Dual
    13 ─       goto #14                            ││││││
    14 ─       goto #15                            │││││
    15 ─ %20 = Core.tuple(%16, %16)::Core.Const((Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)), Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0))))
    └───       goto #16                            ││││
    16 ─ %22 = %new(ForwardDiff.Partials{2, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}}, %20)::Core.Const(Partials(Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)), Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0))))
    └───       goto #17                            ││
    17 ─       nothing                             │
    18 ─ %25 = %new(ForwardDiff.Dual{ForwardDiff.Tag{typeof(foo), ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, 2}, %9, %22)::Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{typeof(foo), ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, 2}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Float64, Core.Const(Partials(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0))]), Core.Const(Partials(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0), Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)))]), Core.Const(Partials(Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0)), Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}}(Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0),Dual{ForwardDiff.Tag{var"#1#2", Float64}}(0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0))))])
    └───       goto #19                            │││
    19 ─       goto #20                            ││
    20 ─       return %25                          │
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • ↩

Julia 1.7 + ForwardDiff's latest release:

julia> @descend_code_warntype convert(ForwardDiff.Dual{ForwardDiff.Tag{typeof(foo), ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, 2}, 2.3)
convert(::Type{ForwardDiff.Dual{T, V, N}}, x::Number) where {T, V, N} in ForwardDiff at /home/chriselrod/.julia/dev/ForwardDiff/src/dual.jl:385
Body::Any
385 1 ─ %1 = invoke ForwardDiff.convert($(Expr(:static_parameter, 2))::Type{ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}}, x::Float64)::Any
    │   %2 = invoke ForwardDiff.zero_tuple(Tuple{ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}}::Type{Tuple{ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}}})::Core.PartialStruct(Tuple{ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Partials{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, Any[Core.PartialStruct(NTuple{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}])])])]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Partials{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, Any[Core.PartialStruct(NTuple{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}])])])])])
    │   %3 = %new(ForwardDiff.Partials{2, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}}, %2)::Core.PartialStruct(ForwardDiff.Partials{2, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}}, Any[Core.PartialStruct(Tuple{ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Partials{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, Any[Core.PartialStruct(NTuple{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}])])])]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 4}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Partials{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, Any[Core.PartialStruct(NTuple{4, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, Any[Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}]), Core.PartialStruct(ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, Any[Core.Const(0.0), ForwardDiff.Partials{8, Float64}])])])])])])
    │   %4 = (ForwardDiff.Dual{ForwardDiff.Tag{typeof(foo), ForwardDiff.Dual{ForwardDiff.Tag{ForwardDiff.var"#112#113"{typeof(foo)}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}}, ForwardDiff.Dual{ForwardDiff.Tag{var"#1#2", Float64}, Float64, 8}, 8}}})(%1, %3)::Any
    └──      return %4                                          │
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %1  = < uncached > convert(::Type{…},::Float64)::Any
   %2  = < uncached > zero_tuple(::Type{…})::…

This causes problems for nested AD, especially when trying to limit chunk sizes of outer duals (why create large tuples where each element is already a large tuple}).

[codecov-commenter]

Codecov Report

Merging #574 (bba8b54) into master (9c2be97) will not change coverage.
The diff coverage is 66.66%.

@@           Coverage Diff           @@
##           master     #574   +/-   ##
=======================================
  Coverage   85.29%   85.29%           
=======================================
  Files           9        9           
  Lines         857      857           
=======================================
  Hits          731      731           
  Misses        126      126           

Impacted Files	Coverage Δ
src/dual.jl	74.24% <66.66%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c2be97...bba8b54. Read the comment docs.

[KristofferC]

Is there a good way to test this?

[chriselrod]

Is there a good way to test this?

I'm not sure.
@allocated, perhaps?
Note that @code_warntype lies, so you need Cthulhu (or benchmarks) to actually see this.

[chriselrod]

Master:

julia> using ForwardDiff

julia> @btime convert(ForwardDiff.Dual{Nothing,ForwardDiff.Dual{Nothing,ForwardDiff.Dual{Nothing,Float64,8},4},2}, 1.2);
  48.108 ns (1 allocation: 304 bytes)

julia> foo() = convert(ForwardDiff.Dual{Nothing,ForwardDiff.Dual{Nothing,ForwardDiff.Dual{Nothing,Float64,8},4},2}, 1.3)
foo (generic function with 1 method)

julia> @allocated foo()
304

PR:

julia> using ForwardDiff
[ Info: Precompiling ForwardDiff [f6369f11-7733-5829-9624-2563aa707210]

julia> @btime convert(ForwardDiff.Dual{Nothing,ForwardDiff.Dual{Nothing,ForwardDiff.Dual{Nothing,Float64,8},4},2}, 1.2);
  10.525 ns (0 allocations: 0 bytes)

julia> foo() = convert(ForwardDiff.Dual{Nothing,ForwardDiff.Dual{Nothing,ForwardDiff.Dual{Nothing,Float64,8},4},2}, 1.3)
foo (generic function with 1 method)

julia> @allocated foo()
0

The performance impact from the type instability can be pretty severe.

[chriselrod]

I've added the @allocated test.

[andreasnoack]

@KristofferC Do you have any further comments to this one or is it good to go?