Probable Native memory leak with Java11

[munendrasn]

Recently, We upgraded one of our service to Java11. We are running the service in Kubernetes. After this upgrade, we started facing restarts because of Kernel OOMKiller.
On debugging, We found Deflator was taking much of the off-heap memory.
All calls to our service have Accept-Encoding set to gzip. When we looked into GzipResponseStream, we observed GzipOutputStream(this uses Deflater which allocates native memory) is never closed

pippo/pippo-core/src/main/java/ro/pippo/core/gzip/GZipResponseStream.java

Line 45 in a6ab828

public void close() throws IOException {

Not closing GzipOutputStream has lead to native memory in other libraries too - https://issues.apache.org/jira/browse/CURATOR-354

@decebals Is there any reason why close() in GzipResponseStream doesn't call GzipOutputStream's close()?

[decebals]

Is there any reason why close() in GzipResponseStream doesn't call GzipOutputStream's close()?

I don't remember to have a specific need on this subject. I will take a look in history and I will come with news.
I'm sorry you're in trouble because of Pippo.

[munendrasn]

@decebals
Thanks for looking into it. Sharing a few other things we checked before raising the issue

• git-history for GzipResponseStream, it always used finish(). Didn't check the repo before the commit (to rule out it is moved from different pippo-package so history might be lost)

I'm sorry you're in trouble because of Pippo.

Bugs can arise but we love Pippo and use it in all our services written in Java.

[decebals]

My conclusion is that probable I wanted to reuse GZIPOutputStream. And from this reason I call only GZIPOutputStream.finsh method. I see in javadoc of finish method:

Finishes writing compressed data to the output stream without closing the underlying stream. Use this method when applying multiple filters in succession to the same output stream.

That class (GZipResponseStream) was introduced by #331.

@munendrasn Do you see any solution?

[munendrasn]

Fastest and straightforward approach would be to do close call. Other which I have seen in Apache-curator is to maintain a pool of deflators since that is real cost. apache/curator#282
Similar approach maybe in Jetty too
Were your planned reuse meant to maintain a pool of gzipoutputstream?

[munendrasn]

Currently, GzipOutputStream(unlike DeflatorOutputStream) doesn't take deflator as an arg in any constructor. Hence, I have raised PR #542 which closes the stream. This would memory leak and then we could build on top of this if reusability needs to be supported

[munendrasn]

@decebals
It is been ~18 months from the last release. Can we plan for a release?

[decebals]

@munendrasn

Can we plan for a release?

I asked the same question in this morning on Gitter:

Sure, I am available to release a new version if everything is ready.

[munendrasn]

I think we are good for a release. The release would contain some important fixes and improvements

• FileItem extraction
• Fix for Controller package
• Fix for memory leak
• Fix for content-length issue

Apart from the above, we have some dependency upgrades