Optimize memory consumption during cache overload

[hrosa]

Load tests for cache eviction scenario showed memory consumption is excessive and can start triggering major GC collections.

#252 (comment)

[hrosa]

@hamsterksu curious to know if you are interested in tackling this one? :)

[hamsterksu]

hi @hrosa

i think if you run GC memory will be free. am i right?
so there are no memory leaks.

i have created small test - https://github.com/hamsterksu/trash/blob/master/test.java
and memory allocation behavior are similar for CachedRemoteStreamProvider and DirectRemoteStreamProvider

cace 100M
no cache

also i have found another issue with URI key.
when it calculates hashcode it executes - https://docs.oracle.com/javase/7/docs/api/java/net/URLStreamHandler.html#getHostAddress(java.net.URL)

profiler detects some possible deadlocks because it's synchronized - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/net/URLStreamHandler.java#428

i will provide more simple implementation of cache

[hamsterksu]

i think this implementation of cache is easier

    public InputStream getStream(URL uri) throws IOException {
        String key = uri.toString();
        Cache<String, byte[]> cache = getCache();

        byte[] stream = cache.get(key);
        if (stream == null) {
            stream = download(cache, uri);
        }

        return new ByteArrayInputStream(stream);
    }

    private synchronized byte[] download(Cache<String, byte[]> cache, final URL uri) throws IOException {
        String key = uri.toString();
        if (cache.containsKey(key)) {
            return cache.get(key);
        }
        byte[] bytes = IOUtils.toByteArray(uri.openStream());
        cache.put(key, bytes);
        return bytes;
    }

WDYT?

[hrosa]

Hi @hamsterksu

i think if you run GC memory will be free. am i right? so there are no memory leaks.

Correct, but I did not mention a leak. I mentioned that for cases where cache eviction happens frequently, the memory consumption becomes very high. Whatever is being kept in memory during eviction scenario is not being cleaned during minor GC.

Instead it is only cleaned during major GC which is not good for Media Server. Stop-the-world is the death of any real-time system so let's avoid it the best we can. This means making the cache more robust. Something is keeping data in memory during eviction scenario, let's find out what it is.

i think this implementation of cache is easier

Code looks clean but that synchronized worries me. Check if this implementation behaves well under load, like having 200 concurrent calls trying to access the same audio file.

[hamsterksu]

@hrosa i have attached 2 screenshots with and w/o cache.
what do you think about case w/o cache?

[hrosa]

Need more details about the test you ran. Also would advise to create tests longer than 2m.

[hamsterksu]

i'm trying to reproduce issue with memory in short period.

i just simulate case when we have small size of cache and don't have repeated requests.
so each new call will add new audio file to cache and display previous.
i use 50 threads to handle all requests (access to file through cache). there are ~ 15 * 100 000 requests

play method was copied from some mediaserver class.

[hamsterksu]

hi @hrosa

i think old gen space will grow up in any case if we use cache.
while it stores objects inside they will be moved to old gen space.
even if new objects displace old objects they will remain in old gen space.

it doesn't depend how often we replace/displace objects in the cache.
if we update cache very frequently old gen will grow with high speed.

so i think we need to use cache to store "static" message audio only
and don't use it for some kind of messages like greeting messages: "welcome user %id"

[hrosa]

@hamsterksu I'm thinking that we may cache the audio tracks in the filesystem rather than in-memory. Or maybe even off-heap.