Skip to content

Add libjemalloc-dev to ruby-base#142

Merged
dazuma merged 1 commit intoGoogleCloudPlatform:masterfrom
brianburnszd:patch-1
Feb 13, 2018
Merged

Add libjemalloc-dev to ruby-base#142
dazuma merged 1 commit intoGoogleCloudPlatform:masterfrom
brianburnszd:patch-1

Conversation

@brianburnszd
Copy link
Copy Markdown
Contributor

Using jemalloc can reduce Ruby memory usage by 30%.
It makes sense to include this in the base image.
Ref: https://medium.com/@adrienjarthon/ruby-jemalloc-results-for-updown-io-d3fb2d32f67f

@googlebot
Copy link
Copy Markdown

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.
  • If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot. The email used to register you as an authorized contributor must be the email used for the Git commit.
  • In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

@brianburnszd
Copy link
Copy Markdown
Contributor Author

I signed it!

@googlebot
Copy link
Copy Markdown

CLAs look good, thanks!

@mikhailov
Copy link
Copy Markdown

mikhailov commented Feb 12, 2018

The library jemalloc is not something new, many apps and even data stores use it as default memory allocator, for example Redis, Percona server where performance is crucial. Jemalloc has proven and tested in production.

To bring jemalloc to Ruby docker base image we either link is statically or dynamically and we benchmarked all three options: malloc, LD_PRELOAD=jemalloc and compile with jemalloc (what this PR is about). Based on benchmark results we can say that compiling jemalloc-dev brings substantial difference to performance characteristics, memory fragmentation and how much time GC spends.

Why is this PR required? Our current Ruby based images has jemalloc-dev included, but we are moving to GCP, so our docker build command fail due to missing library in the base image. This PR should help to unblock us. As a benefit every docker container using this Ruby images as a base docker image will see significantly improved performance and reduced memory footprint. It should bring infrastructure cost down as well.

P.R. It's proposed to be used with another Ruby project GoogleCloudPlatform/fluent-plugin-google-cloud where you can find the technical details, benchmarks and reason why this makes a difference, and one of Google engineers confirmed that jemalloc is already being used for that package:

The google-fluentd package uses jemalloc

@mikhailov
Copy link
Copy Markdown

cc @igorpeshansky

libgdbm-dev \
libgit2-dev \
libgmp-dev \
libjemalloc-dev \
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brianburnszd we probably also need libjemalloc1 package here in order to compile Ruby with jemalloc

Copy link
Copy Markdown
Contributor

@dazuma dazuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Would you mind alphabetizing the list? (Add the two new libs after libicu-dev). Otherwise this looks good to me; I'll merge it after you fix the order.

Full support for jemalloc will require rebuild of the precompiled Ruby binaries, and we'll need to provide a way for users to opt-in/out in case a compatibility issue comes up. I'll open an issue to look at those matters. But for now this base image change should be fine.

@brianburnszd
Copy link
Copy Markdown
Contributor Author

Thanks! List alphabetized.

Copy link
Copy Markdown
Contributor

@dazuma dazuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dazuma dazuma merged commit 0342bd5 into GoogleCloudPlatform:master Feb 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants