Use the new Tensorflow collector/guide in train/parse. #16
Conversation
While at it, add progress bars to dpar-train.
There was a problem hiding this comment.
The changes seem fine in general. When going through the PR I noticed that there are quite some generic error messages if paths are not valid. While it surfaced here, it is probably worth a separate issue, given that this PR is concerned with the training / parse functionality.
Besides this it'd also be nice to have a working minimum example / a quickstart for both train / parse, but I see how this could be separate issue as well.
| let input_file = File::open(&matches.free[1]).or_exit(); | ||
| let reader = conllx::Reader::new(BufReader::new(FileProgress::new(input_file).or_exit())); | ||
| eprintln!("Vectorizing training data..."); | ||
| let (train_labels, train_inputs) = collect_data(&config, reader).or_exit(); |
There was a problem hiding this comment.
The program exits with the unspecific message Error: No such file or directory (os error 2) if there's a faulty path in the config.
Informing the user which file was not found will ease troubleshooting.
There was a problem hiding this comment.
Yeah, this is one of the last projects that I haven't converted to stdinout 0.4 yet, which allows you to provide error messages, exit status, etc.
For training, all the parts are not there yet ;).
There was a problem hiding this comment.
Also, another upgrade that I have postponed is going from error-chain to failure, which makes it easier to add context to errors.
While at it, add progress bars to dpar-train.