createValueStream not containing all values

[foxfirecodes]

I've been enjoying using LevelUP quite a bit, in fact I'm writing my own wrapper around it to provide a simpler interface. I wanted to create methods for .keys(), .values(), and .entries(). I was converting the streams to Promises, but I've encountered an odd problem. The output from createValueStream() only contains 3 of my 11 entries.

Output from .values():

[ false, true, { value: false } ]

Output from .entries():

[
    { key: 'bool-test-false', value: false },
    { key: 'bool-test-true', value: true },
    { key: 'not-exist', value: { value: false } },
    { key: 'null-test', value: null },
    { key: 'number-test-neg', value: -1 },
    { key: 'number-test-one', value: 1 },
    { key: 'number-test-zero', value: 0 },
    { key: 'object-test', value: { bool: true, str: 'Hello!', emptyStr: '', subObject: [Object] } },
    { key: 'object-test-empty', value: {} },
    { key: 'string-test', value: 'Hello' },
    { key: 'string-test-empty', value: '' }
]

.keys() and .entries() both return proper values, and I can work around this by getting the entries and filtering to only the values, but that's not as efficient. Just a note, .createReadStream({ keys: false, values: true}) yields the same problem. Any ideas why this is happening?

[ralphtheninja]

@Rayzr522 Any chance you can provide some code we can try out to see what's going on?

[foxfirecodes]

db.unwrap().createValueStream()
    .on('data', value => {
        console.log(value);
    });

db.unwrap() just returns the actual LevelUP instance, so just replace that with your database instance.

Output:

false
true
{ value: false }

[juliangruber]

can you also give us the code that initializes the db and inserts the data? Ideally you could give us one script that we just save and run to reproduce the bug

[foxfirecodes]

At this point it would probably be better for you to see the code itself. Here's my test class, the database wrapper, and the simple method that converts streams to promises.

In case it matters, the database is set to json for the value type.

[juliangruber]

it's a lot of work to look through that. you'll have better chances having this fixed by providing us with a simple complete gist to reproduce your problem.

[foxfirecodes]

Yeah, sorry about that. Here you go.

[vweevers]

Might be because you're inserting a value of null at L20. Causing the underlying iterator stream to end early. The order of it makes sense too, as key-10 comes after key-1 lexicographically.

[foxfirecodes]

I commented that line out, but I still only got the 3 values. From before (false, true, and 'some value')

[juliangruber]

which node version do i need to run this? or do i need to transform using babel?

[juliangruber]

please also post how to run that file

[vweevers]

@Rayzr522 I tried the same, got all values. Be sure to run it on an empty database, e.g.

const db = level('./' + Date.now(), { valueEncoding: 'json' })

@juliangruber node 7 with --harmony-async-await

[foxfirecodes]

Ah, that worked now. Why do null values cause problems?

[juliangruber]

i only get the first value. however, we're not waiting for any callback to fire in this example. that's expected behavior.

[vweevers]

@juliangruber ah of course. Must have been luck when I ran it. So, ignoring promises and async/await for a moment:

const level = require('level');
const db = level('./' + Date.now(), { valueEncoding: 'json' });

db.batch([
  { key: 'key-1', value: 1 },
  { key: 'key-2', value: 2 },
  { key: 'key-3', value: null },
  { key: 'key-4', value: 4 },
  { key: 'key-5', value: 5 }
], function (err) {
  if (err) throw err

  db.createValueStream()
    .on('data', (value) => {
      console.log(value)
    });
});

This stops at the null value.

[foxfirecodes]

Why do null values cause problems?