Excessive memory usage

[K0bin]

I assume the fix for #269 has shipped now, in that case this is a new bug. Otherwise I'll just close it.

I can provide the actual archives later. I saved the command line launch arguments for now.

/home/robin/.local/share/Steam/ubuntu12_32/../ubuntu12_64/fossilize_replay /nvme2/SteamLibrary/steamapps/shadercache/2651280/fozpipelinesv6/steamapprun_pipeline_cache.9340404e5c3fd323/steamapp_pipeline_cache.foz /nvme2/SteamLibrary/steamapps/shadercache/2651280/fozpipelinesv6/steamapprun_pipeline_cache.9340404e5c3fd323/steam_pipeline_cache.foz --master-process --quiet-slave --shmem-fd 109 --control-fd 140 --spirv-val --num-threads 6 --on-disk-validation-whitelist /nvme2/SteamLibrary/steamapps/shadercache/2651280/fozpipelinesv6/steamapprun_pipeline_cache.9340404e5c3fd323/steam_pipeline_cache_whitelist --replayer-cache /nvme2/SteamLibrary/steamapps/shadercache/2651280/fozpipelinesv6/steamapprun_pipeline_cache.9340404e5c3fd323/replay_cache --device-index 0 --timeout-seconds 10 --implicit-whitelist 0

[kakra]

I don't think that fossilize_replay is using 3+ GB per process. Most of it should be shared memory. Could you check the PSS of the processes? htop should be able to display that. The PSS value excludes the shared memory usage from each process and then proportionally adds it back per process, which should result in a much more realistic memory usage pattern - not per process but as a total sum of all fossilize_replay processes.

[K0bin]

I'll try to remember that the next time it happens but I ran out of memory on a system with 32GB and fossilize was easily using 16 of those.

[kakra]

I'll try to remember that the next time it happens but I ran out of memory on a system with 32GB and fossilize was easily using 16 of those.

Yeah something was off because usually it should use around 1 - 1.5 GB, and around 1 GB of that is shared with its sibling processes. It is important to look not only at RES but also at SHR memory usage, and if your task viewer can show PSS, that's the value which combines those to.

The real memory usage of all fossilize siblings can be calculated by summing up (RES minus SHR) per each process, then adding the average of SHR of all sibling processes. This is more or less what the PSS value does (but rather per process instead of as a sum).

[K0bin]

This is either still a problem or the fix needs to roll out.

[HansKristian-Work]

Does that reproduce on master?

[KitFoxy]

Experiencing possibly the same issue on an up to date EndeavourOS install. Memory usage continues to climb with fossilize processes until out of memory at which point Steam and by extension these processes are killed. 32gb of Ram, was sat at 4gb used prior to attempting to launch something, at which point Fossilize used the remaining resources. The problem only presented after updating yesterday.