Please explain...

[nickperkinslondon]

What exactly do you mean by "duplicates"?
You never say.
Is it identical content?
Is it duplicate filenames?
Is it the full path name?
How does it work?
You should explain more in your description.

[hoho]

Basically, by «duplicates» I mean the files in the stream which would end in the same file after .pipe(gulp.dest(...)). I'll clarify the description, thanks.

[partizanos]

Please show a simple example use:
I tried the two following
file1
content:
123
123

gulp task:
gulp.src('*.xml')
.pipe(dedupe('error'))
would hit error? it doesn't.

Seconde case
gulp task:
gulp.src('*.xml')
.pipe(dedupe('error:true'))

has syntax error error

[hoho]

dedupe(options) optionally accepts options object.

please, try dedupe({error: true}).

[partizanos]

Thanks for your answer terminal after rtunning the task sais it has no errors
But my file has duplicate lines and it doesn't display something
do I do something wrong?
gulpfile.js
var gulp = require('gulp'),
gutil = require('gulp-util');
var dedupe = require('gulp-dedupe');
var print = require('gulp-print');

gulp.task('default', function() {
return gutil.log('Gulp is running!')
});

gulp.task('xml', function() {
    console.log("running xml task")

    // console.logs(gulp.src('*.xml'))
    gulp.src('*.xml')
        .pipe(dedupe({error: true}))
        // .pipe(print())
        // .pipe(print(function(filepath) {
        //     return "built: " + filepath;
        // }))
        // .pipe(readFile)
});

Terminal ouput:

> Starting 'xml'...
> running xml task
> inished 'xml' after 5.94 ms

in the directory myFile.xml
is read by the gulp task
contents:
123
123

does it work for you?

[hoho]

I think this is just not the plugin you're looking for. It's not looking for the duplicate lines within one file, it's looking for two (or more) files with the same name within the gulp stream (which is a set of files) and leaves just one of them.

[partizanos]

Ok, I was not sure by the description it might be a good addition. Thank you for the feedback.

[t1a2l]

Does thie plugin support directories? If i have file.js and then lib/*.js - and the directory has the file.js will it remove it?