LEAK: ByteBuf.release() was not called before it's garbage-collected

[asolntsev]

I often see this error when running UI tests (Selenide+Selenium+BrowserUpProxy+LittleProxy).

Seems like LittleProxy doesn't free up some resources, thus causing memory leak?

[] - [ERROR] 22-06-2022 23:46:33.393 - LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
Created at:
	io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:403)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
	io.netty.buffer.CompositeByteBuf.allocBuffer(CompositeByteBuf.java:1879)
	io.netty.buffer.CompositeByteBuf.copy(CompositeByteBuf.java:1524)
	io.netty.buffer.AbstractByteBuf.copy(AbstractByteBuf.java:1194)
	io.netty.handler.codec.http.HttpObjectAggregator$AggregatedFullHttpRequest.copy(HttpObjectAggregator.java:404)
	org.littleshoot.proxy.impl.ClientToProxyConnection.copy(ClientToProxyConnection.java:1073)
	org.littleshoot.proxy.impl.ClientToProxyConnection.doReadHTTPInitial(ClientToProxyConnection.java:243)
	org.littleshoot.proxy.impl.ClientToProxyConnection.readHTTPInitial(ClientToProxyConnection.java:222)
	org.littleshoot.proxy.impl.ClientToProxyConnection.readHTTPInitial(ClientToProxyConnection.java:86)
	org.littleshoot.proxy.impl.ProxyConnection.readHTTP(ProxyConnection.java:140)
	org.littleshoot.proxy.impl.ProxyConnection.read(ProxyConnection.java:120)
	org.littleshoot.proxy.impl.ProxyConnection.channelRead0(ProxyConnection.java:556)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
	org.littleshoot.proxy.impl.ProxyConnection$RequestReadMonitor.channelRead(ProxyConnection.java:683)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327)
	io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
	org.littleshoot.proxy.impl.ProxyConnection$BytesReadMonitor.channelRead(ProxyConnection.java:660)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
	io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	java.base/java.lang.Thread.run(Thread.java:833)

[sjoshid]

Hi @asolntsev
Im working with Selenium and BMP seems to be the only reliable way to add request headers.

Then I ran in to this issue. :(

It seems like #141 has addressed the issue. My simple plan is this: since BMP is dead (last release was in 2016) and is a Maven project, I can simply point latest of this version there. Run mvn package on BMP and voila.....issue fixed.

Thoughts?

[sjoshid]

Im facing the same issue even after pointing BMP to latest release of this ie 2.0.19 :(

[asolntsev]

Hi @sjoshid .
Sorry, I didn't understand what are you suggesting. You don't really need to rebuild BMP.
Instead, you can just have 2 dependencies in your pom.xml or build.gradle:

1. Latest BMP
2. Latest LittleProxy

NB! As a replacement for BrowserMobProxy, another project BrowserUpProxy was created. I moved to BrowserUpProxy in my projects. This is regularly updated and contains the latest LittleProxy. Probably you should also consider migration from BMP to BUP?

[sjoshid]

Here's my understanding.

• BMP is dependent on LP. BMP's pom has

<dependency>
    <groupId>net.lightbody.bmp</groupId>
    <artifactId>littleproxy</artifactId>
    <version>1.1.0-beta-bmp-17</version>
</dependency>

• I can certainly include BMP and LP as explicitly deps in my project but this has ramifications. What you are saying is have something like below im MY pom(?):

<dependency>
  <groupId>net.lightbody.bmp</groupId>
  <artifactId>browsermob-proxy</artifactId>
  <version>WHATEVER IS LATEST</version>
  <exclusions>
    <exclusion>
      <groupId>net.lightbody.bmp</groupId>
      <artifactId>littleproxy</artifactId>
    </exclusion>
  </exclusions>
</dependency>

<dependency>
  <groupId>net.lightbody.bmp</groupId>
  <artifactId>littleproxy</artifactId>
  <version>WHATEVER IS LATEST</version>
</dependency>

I don't think this is a good idea.

Instead, I can fix latest LP (having the leak)...use that in latest BMP and then use latest BMP in my project.

I hope I explained well enough.

another project BrowserUpProxy was created

That seems to be abandoned as well for mitmproxy. Let me migrate to mitmproxy. Hope I don't get any leaks with it.

Thanks for replying.