Neuronal Network Feed Forward: How to get multiple outputs?

[TaPSus]

Hey @jdermody,

I have a question about the Feed Forward networks. I want to give my net multiple inputs, and the expected output should be 3 different states. Well I worked with multiple output neurons and with just one output neuron which delivers values like (1/3 )*whichstate, but I am currently lost to achive stuff like this in Brightwire ^^ Here is a sample code, which I ripped and cusomized from the examples:

`using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading.Tasks;
using BrightWire;
using BrightWire.ExecutionGraph;

namespace MyNet
{
class Program
{
static void Main(string[] args)
{

        StringBuilder sb = new StringBuilder();

        sb.AppendLine("0,0,0,0,0,1");
        sb.AppendLine("1,1,1,0,0,1");
        sb.AppendLine("2,2,2,0,0,1");
        sb.AppendLine("0,0,1,0,1,0");
        sb.AppendLine("0,1,0,0,1,0");
        sb.AppendLine("1,0,0,0,1,0");
        sb.AppendLine("0,0,2,0,1,0");
        sb.AppendLine("0,2,0,0,1,0");
        sb.AppendLine("2,0,0,0,1,0");
        sb.AppendLine("2,2,1,0,1,0");
        sb.AppendLine("2,1,2,0,1,0");
        sb.AppendLine("1,2,2,0,1,0");
        sb.AppendLine("2,2,0,0,1,0");
        sb.AppendLine("2,0,2,0,1,0");
        sb.AppendLine("0,2,2,0,1,0");
        sb.AppendLine("1,1,0,0,1,0");
        sb.AppendLine("1,0,1,0,1,0");
        sb.AppendLine("0,1,1,0,1,0");
        sb.AppendLine("1,1,2,0,1,0");
        sb.AppendLine("1,2,1,0,1,0");
        sb.AppendLine("2,1,1,0,1,0");
        sb.AppendLine("0,1,2,1,0,0");
        sb.AppendLine("0,2,1,1,0,0");
        sb.AppendLine("1,0,2,1,0,0");
        sb.AppendLine("1,2,0,1,0,0");
        sb.AppendLine("2,0,1,1,0,0");
        sb.AppendLine("2,1,0,1,0,0");

        //data set 1
        BrightWire.IDataTable dataTable = BrightWire.BrightWireProvider.ParseCSV(sb.ToString());

        // the last column is the classification target ("Iris-setosa", "Iris-versicolor", or "Iris-virginica")
        var targetColumnIndex = dataTable.TargetColumnIndex = dataTable.ColumnCount - 3;

        // split the data table into training and test tables
        var split = dataTable.Split(trainingPercentage: 0.9);

        using (var lap = BrightWireProvider.CreateLinearAlgebra(false))
        {

            // create a neural network graph factory
            var graph = new GraphFactory(lap);

            // the default data table -> vector conversion uses one hot encoding of the classification labels, so create a corresponding cost function
            var errorMetric = graph.ErrorMetric.OneHotEncoding;

            // create the property set (use rmsprop gradient descent optimisation)
            graph.CurrentPropertySet
                .Use(graph.RmsProp())
            ;

            // create the training and test data sources
            var trainingData = graph.CreateDataSource(split.Training);
            var testData = trainingData.CloneWith(split.Test);
            

            // create a 4x8x3 neural network with relu and sigmoid activations
            const int HIDDEN_LAYER_SIZE = 200;
            var engine = graph.CreateTrainingEngine(trainingData, 0.09f, 1);
            graph.Connect(engine)
                .AddFeedForward(HIDDEN_LAYER_SIZE)
                .Add(graph.ReluActivation())
                .AddDropOut(0.28f)
                .AddFeedForward(engine.DataSource.OutputSize)
                .Add(graph.SigmoidActivation())
                .AddBackpropagation(errorMetric)

            ;
            Console.WriteLine(engine.DataSource.OutputSize.ToString());

            // train the network
            Console.WriteLine("Training a 4x8x3 neural network...");
            engine.Train(20000, testData, errorMetric, null, 1000);

            Console.WriteLine("RESULT: " + engine.Execute(new float[] { 0, 1, 0 }).Output.ToArray()[0]);
            Console.WriteLine("RESULT: " + engine.Execute(new float[] { 1, 0, 1 }).Output.ToArray()[0]);
            Console.WriteLine("RESULT: " + engine.Execute(new float[] { 0, 2, 0 }).Output.ToArray()[0]);
            Console.WriteLine("RESULT: " + engine.Execute(new float[] { 0, 1, 0 }).Output.ToArray()[0]);

        }

    }
}

}
`

The nn should learn if there are 3, 2 or 1 different numbers. How can I connect 3 different outputs and how do I have to set the train data for it?

Would be great if you have the time to respond ^^

[jdermody]

Hi, sorry it took so long to get back to you!

I think you want to create a table with an input vector column and a target output vector column. Similar to the MNIST example: https://github.com/jdermody/brightwire/blob/master/SampleCode/MNIST.cs

In this case the input vector contains the numbers as above and the output vector is of size 3. The simplest way to build this is to use the same one hot encoding that you are using. The vector would be:

[0,0,1] to output 1
[0,1,0] to output 2
etc

You could also experiment with different layouts for the output vector. For example, try a binary classifier for the error function and the following encoding:

[0,0,0] to output 1
[0,1,1] to output 2
etc