Improve presentation of CNN.

[ashinkarov]

This is purely a matter of presentation, but I think in this particular case this is kind of important.
In the ideal world with higher-order functions, I would want to write the CNN as follows:

out = inp |> conv k1 |> log 
          |> avgpool [2,2] |> conv k2 |> log 
          |> avgpool [2,2] |> conv fc |> reshape [10]

Unfortunately, there is no way to define |> in SaC. However, in this particular case, the same information flow that very closely mimics the picture can be achieved via custom binary operations.
The >--logistics-> function have to take a useless argument, but other than that it works fine.

After rewriting and comparing it with the picture, I have noticed that the last Logistics application is missing. Am I right?

[sbscholz]

Cool idea. 2 drawbacks: all functions are limited to exactly one extra parameter. More would not work (which is what would be needed for a more generic convolution) and if the function has only one (eg Logistics) then we would need postfix rather than preface, hence the ugly _.

The observation that postfix unary would solve the problem with logistics gave me an idea.

We could add some syntactic sugar that would solve both these issues....

we could look for expressions of the form expr |> f ( expr_1, ..., expo_n)
and replace it with f( expr_1, ..., expr_n, expr) or
f( expr, expr_1, ..., expr_n)

That way one could actually write out = in |> Conv ( k1) |> Logistics () |> AveragePool ([2,2], []) ...

or is this too crazy?

Would this be hard to put into the scanner/parser?

[sbscholz]

While all this is nice, it is of no use as soon as we do back-propagation. At that stage, we need all intermediates......