Blacklist multiple Erlang releases

[wohali]

@iilyak ever seen this one?

test/javascript/tests/changes.js                               pass
test/javascript/tests/coffee.js                                pass
test/javascript/tests/compact.js                               pass
test/javascript/tests/config.js                                Error: 
FAIL isRunning
Stacktrace:
        waitForSuccess((function () {CouchDB.last_req = CouchDB.request("GET", "/");return typeof CouchDB.last_req == "object";}),"isRunning")@test/javascript/test_setup.js:64
        @test/javascript/cli_runner.js:46
Failed to execute script.
fail
=======================================================
JavaScript tests complete.
  Failed: 1.  Skipped or passed: 14.
Makefile:231: recipe for target 'javascript' failed
make: *** [javascript] Error 1

It's repeatable at will on my new laptop. In other words, make javascript always fails.

Your Environment

• Version used: master
• Operating System and version (desktop or mobile): debian stretch, up to date

[wohali]

[notice] 2019-01-14T07:56:41.928596Z node1@127.0.0.1 <0.15144.0> -------- rexi_buffer : started servers 
[notice] 2019-01-14T07:56:41.937638Z node1@127.0.0.1 <0.14937.0> -------- config: [features] scheduler set to true for reason nil
[notice] 2019-01-14T07:56:42.007170Z node1@127.0.0.1 <0.15182.0> 640d3f245f 127.0.0.1:15984 127.0.0.1 undefined GET /_node/node1@127.0.0.1/_system 200 ok 1
[notice] 2019-01-14T07:56:42.011893Z node1@127.0.0.1 <0.15182.0> be5192f0d9 127.0.0.1:15984 127.0.0.1 undefined GET /test_suite_db_eywat/bin_doc/foo.txt 200 ok 4
[notice] 2019-01-14T07:56:42.014992Z node1@127.0.0.1 <0.15182.0> d50ba1cf01 127.0.0.1:15984 127.0.0.1 undefined GET /test_suite_db_eywat/ 200 ok 3
[notice] 2019-01-14T07:56:42.015681Z node1@127.0.0.1 <0.15182.0> 18014b7636 127.0.0.1:15984 127.0.0.1 undefined GET /test_suite_db_eywat/ 200 ok 1
[notice] 2019-01-14T07:56:42.016378Z node1@127.0.0.1 <0.15182.0> 964c476f03 127.0.0.1:15984 127.0.0.1 undefined GET /test_suite_db_eywat/ 200 ok 1
[notice] 2019-01-14T07:56:42.017148Z node1@127.0.0.1 <0.15182.0> a6b12270da 127.0.0.1:15984 127.0.0.1 undefined GET /test_suite_db_eywat/ 200 ok 1
nifs/common/prim_file_nif.c:388:owner_death_callback(): Internal error: Failed to defer prim_file close.^M
[notice] 2019-01-14T07:56:42.020951Z node1@127.0.0.1 <0.15182.0> bb28d3dd56 127.0.0.1:15984 127.0.0.1 undefined DELETE /test_suite_db_eywat/?sync=true 200 ok 4
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed^M
[os_mon] memory supervisor port (memsup): Erlang has closed^M

Hmm.

ii  esl-erlang                            1:21.2.2-1                                   amd64        Erlang

Oops, let me try an older Erlang, that's not what's supposed to be installed.

/cc @dch

[wohali]

OK, that fixed it. We know 21.2 is bad. We should blacklist it in our config setup.

[wohali]

Duplicate of #1837.

• https://bugs.erlang.org/browse/ERL-821 (thanks @nickva )
  • 21.1.2 and forward fail, not yet fixed
• https://bugs.erlang.org/browse/ERL-827 (thanks @dch )
  • also fails in 21.2.2

We might also want to blacklist 20.3.8.10 and lower in the 20 series:

• https://bugs.erlang.org/browse/ERL-751 (thanks @davisp)
  • fixed in 20.3.8.11 and up

[wohali]

Well, poop. perhaps we can't.

https://github.com/rebar/rebar/blob/b6d309417c502ca243f810e5313bea36951ef038/src/rebar_require_vsn.erl#L92-L102

Eshell V9.3.3.7  (abort with ^G)
1> erlang:system_info(otp_release).
"20"

That's not enough info to go on. We'd have to write something.

[wohali]

Looks like rebar3 gives us blacklisted_otp_versions which is what we really want here.

erlang/rebar3@04fedc7

So this is really dependent on #1428 .

[AlexanderKaraberov]

@wohali Does this mean that it's not safe to update our production environment to OTP 21.2? (actually we already did this). We are using the latest CouchDB 2.3+our patch set. It's been several days since we migrated to OTP 21.2 and haven't observed any serious deterioration (as of yet). What would be your or other core maintainers advice? Shall we rollback to OTP 21.1 to be sure all the things work?

[nickva]

There is another one as well:

https://bugs.erlang.org/browse/ERL-830

This one might affect systems which use TLS connections (either as a server or via ibrowse as replication clients). This seems to be fixed in 21.2.3 (just released).

[nickva]

@AlexanderKaraberov

https://bugs.erlang.org/browse/ERL-821 : would affect tests so running tests would not work but would not affect production. This is fixed in the latest 21.2.3

https://bugs.erlang.org/browse/ERL-827 : not sure about this one, it is manifesting on FreeBSD, perhaps that's a necessary trigger for the bug and on other machines (Linux) might not happen. This is not fixed yet and and the cause it not known yet.

https://bugs.erlang.org/browse/ERL-830 : not sure how likely this one is to happen, it does seem serious. One the other hand, I'd imagine if random bytes of data are not sent it would break http protocol and json formatting so connections would be retried. This is also fixed in 21.2.3

Would you have the option of building a point release? Then I would try 21.2.3 perhaps. To be even more stable perhaps try 20.3.8.14, I've seen that run in production across hundreds of machine already for months without any issues.

[AlexanderKaraberov]

Hi @nickva ,
Thank you for a prompt and detailed feedback.

Would you have the option of building a point release? Then I would try 21.2.3 perhaps.

This option is good for us. And from what I see it addresses some of the pinpointed problems. OTP 21.x delivers a lot of performance improvements to I/O subsystem hence we would be happy to stay on this major release instead of rolling back to 20.x

[wohali]

We just heard from @dch that 21.2.3 seems to have fixed ERL-827. So it will be sufficient to blacklist 21.2, 21.2.1, and 21.2.2.