asan test status

[qiwzhang]

I have tried gcc 7.4 and and 5.3. Many asan "t" tests failed with memory leak detected in Nginx.

These are singlton objects, allocated only once during the whole process, such as api_manager object.

it is due to master process and worker process issue. If a memory is allocated in master, it should be freed in master. But we may only free it in worker process.

Some are simple fixes, other there are still many other places need to clean up.

[qiwzhang]

Jenkins presubmit test has asan. It works because its gcc is 4.9

[qiwzhang]

As it turns out that. "asan" in compiler <= 4.9 doesn't work. It will not report any memory leaks.

After using gcc 7.3.0. "asan" tests found some leaks. It as fixed by #596

[qiwzhang]

But "t" tests are using the whole nginx. Nginx has global init memory are not freed. e.g. in ngx_event_process_init, it allocated 3 buffers, connections, write_events, read_events, these 3 buffers are not freed.

But strangely, only some "t" tests complains about these leaks. Could you figure out why. Maybe a bug in "asan"

Like to turn on "asan" with latest gcc. But may have to

1. not to run "t" tests in "asan" or
2. figure out a way to suppress these Nginx leaks. But it seems that only clang asan support supression.

[qiwzhang]

BTW, nginx has to turn off master_process in order to run "asan"

master_process off;

[qiwzhang]

Tried with different gcc version

1. gcc 7.3.0 based on Debian 4.19.12-1rodete1.

asan works except "t" tests

2. gcc-7 based on Ubunto

asan doesn't work. error:

==5820==LeakSanitizer has encountered a fatal error.
==5820==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1
==5820==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc)

3. gcc-6.3 based on Debian9

//src/api_manager/auth/lib:auth_jwt_validator_test

this test failed with some SSL leaks which is false alarm

Direct leak of 24 byte(s) in 1 object(s) allocated from:
#0 0x7fe7c3b9fd28 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1d28)
#1 0x564188d40877 in OPENSSL_malloc (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x254877)
#2 0x564188d53499 in CRYPTO_set_thread_local (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x267499)
#3 0x564188cbd743 in err_get_state (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x1d1743)
#4 0x564188cbe8e7 in ERR_put_error (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x1d28e7)
#5 0x564188d31489 in RSA_padding_check_PKCS1_type_1 (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x245489)
#6 0x564188d3578e in RSA_verify_raw (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x24978e)
#7 0x564188d35c2b in RSA_verify (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x249c2b)
#8 0x564188cdc733 in pkey_rsa_verify (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x1f0733)
#9 0x564188cd7dfb in EVP_PKEY_verify (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x1ebdfb)
#10 0x564188cca259 in EVP_DigestVerifyFinal (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+0x1de259)
#11 0x564188c09699 in google::api_manager::auth::(anonymous namespace)::JwtValidatorImpl::VerifyPubkey(bool) (/root/.cache/bazel/_bazel_root/b570b5ccd0454dc9af9f65ab1833764d/execroot/main/bazel-out/k8-fastbuild/bin/src/api_manager/auth/lib/auth_jwt_validator_test+

[kleisauke]

Applying the no-pool patch to nginx core is recommended while running nginx with AddressSanitizer:
https://github.com/openresty/no-pool-nginx