Skip to content

HBASE-27805 The chunk created by mslab may cause memory fragement and… #5193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bsglz merged 2 commits into from
Jul 27, 2023
Merged

HBASE-27805 The chunk created by mslab may cause memory fragement and… #5193

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
838c7d9
HBASE-27805 The chunk created by mslab may cause memory fragement and…
bsglz Apr 20, 2023
87e0692
fix review issues
bsglz Jul 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions src/main/asciidoc/_chapters/troubleshooting.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1566,3 +1566,25 @@ then check if you compiled with jdk8 and tried to run it on jdk7.
If so, this won't work.
Run on jdk8 or recompile with jdk7.
See link:https://issues.apache.org/jira/browse/HBASE-10607[HBASE-10607 JDK8 NoSuchMethodError involving ConcurrentHashMap.keySet if running on JRE 7].

=== Full gc caused by mslab when using G1

The default size of chunk used by mslab is 2MB, when using G1, if heapRegionSize equals 4MB, these chunks are allocated as humongous objects, exclusively allocating one region, then the remaining 2MB become memory fragment.

Lots of memory fragment may lead to full gc even if the percent of used heap not high enough.

The G1HeapRegionSize calculated by initial_heap_size and max_heap_size, here are some cases for better understand:

- xmx=10G -> region size 2M
- xms=10G, xmx=10G -> region size 4M
- xmx=20G -> region size 4M
- xms=20G, xmx=20G -> region size 8M
- xmx=30G -> region size 4M
- xmx=32G -> region size 8M

You can avoid this problem by reducing the chunk size a bit to 2047KB as below.

----
hbase.hregion.memstore.mslab.chunksize 2096128
----