extern(::Casted) seems ill-defined. (Remove Casted?)

[willtebbutt]

extern(cast(1)) returns 1. This is unfortunate behaviour as we loose all of the e.g. size information that (arguably) should be associated with the casted thing. For example the rrule for sum currently returns a casted 1 to represent an array of ones, which would have been the same size as the input. Consequently, without reference to the input you don't know what the appropriate size is for a particular Casted object.

Is there a particular reason that this route has been taken as opposed to, say, returning AbstractFills (from FillArrays.jl) which would be just as efficient in all cases that I can think of.

[jrevels]

Is there a particular reason that this route has been taken as opposed to, say, returning AbstractFills (from FillArrays.jl) which would be just as efficient in all cases that I can think of.

Mainly because I didn't know about FillArrays.jl, seems like a good idea to utilize that instead 🙂

[willtebbutt]

What do you reckon is the best way forwards? I guess we could either:

• Keep the Casted differential and just re-implement using FillArrays, or
• Get rid of the Casted differential and re-implement using FillArrays.

Any preference / have I missed an obvious option @jrevels ?

[jrevels]

Get rid of the Casted differential and re-implement using FillArrays.

This would be preferable, if possible, though I'm not sure if FillArrays covers everything that Casted covers.

Casted basically is meant to be Base.Broadcast.Broadcasted with a bit of extra behavior (the baked in * and + definitions since it's an AbstractDifferential). We could've defined that extra behavior directly on Base.Broadcast.Broadcasted, but of course that would be type piracy and can lead to some issues (even if the type piracy was hidden under a Cassette context).

Example where cast is more like broadcasted:

frule(::typeof(BLAS.asum), x) = (BLAS.asum(x), Rule(Δx -> sum(cast(sign, x) * Δx)))

rrule(::typeof(BLAS.asum), x) = (BLAS.asum(x), Rule(ΔΩ -> ΔΩ * cast(sign, x)))

Even if FillArrays doesn't support this particular use case, it'd be great to stop punning on Casted for cases where formal (i.e. tested against) support for FillArrays would suffice.

[willtebbutt]

Hmm yeah, this is interesting. Your above examples definitely aren't something that you could straightforwardly re-implement using FillArrays, so I agree that it might just be best to introduce FillArrays in rules when it makes sense and to keep the cast stuff when it doesn't.

[oxinabox]

From #18

Casted

It is kind of the generalization of One, and Zero.
(in that One() could also be written Casted(true) etc).
It lets us lazily delay computing a broadcast,
so that it can be fused later.
But I think in the short term we can simplify the code
by replacing say
Rule((Δx, Δy) -> sum(Δx * cast(y)) + sum(cast(x) * Δy))
with
Rule((Δx, Δy) -> sum(Δx .*y) + sum(x .* Δy)
(from here
which for that particular case would even be identical in performance I think.
Since it does not end up returning any kind of lazy computation.
And later we can try getting back the lazy computation and broadcast fusing by returning broadcasted.

So we can just get ride of it for now.