progress outputs

[jamesb93]

Hello FluCoMites,

It would be nice to have progress outputs from stdout perhaps issued with a verbose flag or something ala Max environment objects. I took a look at implementing myself - alas I lack the C++ skills to really get my head around it but perhaps putting it on your radar might make it find its way into a future release. With some guidance + clarity on how to implement it could be something I take on myself.

[weefuzzy]

Thanks @jamesb93. This is on the radar, but not as a matter of huge urgency to us. There would be two components to making this change:

1. Make the CLI wrapper use the threaded clients (which is where the progress count will come from)
2. Implementing the required stdout niceness. For this I was undecied about whether going to look for a nice external library to take care of this and argument handling etc., or to cobble something together and avoid the extra dependency.

Adding the threading isn't totally horrible, but it's not totally obvious either. First step would be to change the template argument in each of the .cpp files to <NRTThreaded[whatever]>, which then gives you the wrapper that runs the separate thread. Then you need something in main to to the polling. One can sort of see how this works in the wrappers for the other hosts, except that (of course) its buried amongst all the other things.

[jamesb93]

Great, that might be doable for myself! I appreciate the guidance. IMO a good stdout would just be the percentage without any other bells and whistles. Maybe a "\r" to clear the screen but then it can be rude to clear peoples consoles without asking, although lots of programs do this (I can think of progress bars in homebrew for example).

[jamesb93]

I had a go at this today, but I'll admit I'm way out of my depth tracking down errors to progress. It seems that the part of the wrapper which takes in the argc and argv is expecting that, so when the wrapper is changed to NRTThreaded it whinges because that doesn't have a run member function. I assume there is just another way of passing arguments to these processes or orchestrating run.

[weefuzzy]

Are you doing something like

#include <FluidCLIWrapper.hpp>
#include <clients/rt/LoudnessClient.hpp>

int main(int argc, const char* argv[])
{
  using namespace fluid::client;
  return CLIWrapper<NRTThreadedLoudnessClient>::run(argc, argv);
}

? That should work, at least to the extent of giving you error messages about something other than run (which will be very long and template-y)

[jamesb93]

This is what I get back:

In file included from /Users/james/dev/flucoma/flucoma-cli/src/fluid-ampgate/fluid-ampgate.cpp:12:
/Users/james/dev/flucoma/flucoma-cli/include/FluidCLIWrapper.hpp:393:42: error: too many arguments to function call, expected 0, have 1
    auto         result = client.process(context);
                          ~~~~~~~~~~~~~~ ^~~~~~~
/Users/james/dev/flucoma/flucoma-cli/src/fluid-ampgate/fluid-ampgate.cpp:18:48: note: in instantiation of member function
      'fluid::client::CLIWrapper<NRTThreadedAmpGateClient>::run' requested here
  return CLIWrapper<NRTThreadedAmpGateClient>::run(argc, argv);
                                               ^
/Users/james/dev/flucoma/flucoma-cli/build/_deps/flucoma-core-src/include/clients/rt/../common/FluidNRTClientWrapper.hpp:522:3: note: 'process' declared here
  Result process()
  ^
1 error generated.
make[2]: *** [src/fluid-ampgate/CMakeFiles/fluid-ampgate.dir/fluid-ampgate.cpp.o] Error 1
make[1]: *** [src/fluid-ampgate/CMakeFiles/fluid-ampgate.dir/all] Error 2
make: *** [all] Error 2

[weefuzzy]

That could be worse. Just don't pass the instance of FluidContext into process() on line 393 of include/FluidCLIWrapper.hpp

[jamesb93]

Okay I removed passing FluidContext. I assume that the next part is doing some error checking after the work has been complete, and writing when the return is good. I guess at this point you are dealing with pinging the result to see if its done? Are there any good instances of this I can look at in other wrappers?

[weefuzzy]

The Result is still useful. The corresponding bit of the PD wrapper might shed some light: https://github.com/flucoma/flucoma-pd/blob/e4d8349c4fa39f08bbee0521585fc0fc19ff443c/include/FluidPDWrapper.hpp#L255

Then for the pinging https://github.com/flucoma/flucoma-pd/blob/e4d8349c4fa39f08bbee0521585fc0fc19ff443c/include/FluidPDWrapper.hpp#L277

However, it'll be slightly different here because there's not a scheduler. So you'll want a loop after you call process that sleeps for a bit (possibly using std::this_thread::sleep_for), then checks the progress. When it's done, break out of the loop and the stuff from line 395 should be the same, I think.

[weefuzzy]

Maybe something like

Result result;
client.process();

while(true) 
{
    ProcessState state = client.checkProgress(res);
    if (state == ProcessState::kDone ||
        state == ProcessState::kDoneStillProcessing)
    {
      if (x->checkResult(result)) break; 
    }

    if (state != ProcessState::kDone)
    {
     std::cout << client.progress(); 
     std::this_thread::sleep_for(20ms); 
     continue; 
    }
}

    if (!result.ok())
    {
      // Output error

      std::cerr << result.message() << "\n";
    }
    else
    {
      // Write files

      bool allowCSV = true;
      params.template forEachParamType<BufferT, WriteFiles>(allowCSV);
    }

    return result.ok() ? 0 : -1;

[weefuzzy]

I don't remember offhand why we need allowCSV as a thing, but it might have been when Alex was still waiting for me to implement the CSV generation elsewhere

[weefuzzy]

I imagine you'll at the very least need to add #include <thread> to the file

[jamesb93]

Okay getting there. It doesn't know what x is but I'm looking at the pd wrapper now to decipher.

[jamesb93]

Okay yep, firmly out my depth here. checkResult seems to be a member function of the other wrappers which is why its passed that instance of the class in pd? So passing this->checkResult fails. Perhaps you can steer me again.

[weefuzzy]

Oops. You don't need it. Just break if the process state is kDone

…

On Fri, 7 Aug 2020, 22:03 James Bradbury, ***@***.***> wrote: Okay yep, firmly out my depth here. checkResult seems to be a member function of the other wrappers which is why its passed that instance of the class in pd? So passing this->checkResult fails. Perhaps you can steer me again. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGQHIYPRSPSJEYSWNLBFRKDR7RT27ANCNFSM4N4GY3UQ> .