Limit number of the transformations threads

[striker2000]

When a search bot requests many images at the same time and these images are not in the cache, all transformation goroutines start in parallel causing the huge memory allocation, that leads to the termination of the process by OOM.

This fix is adding the limiter in the TransformingTransport that limits the number of running transformation threads to the number of logical CPUs. The metric http_requests_in_flight is adding as well to monitor the length of the requests queue.

Fixes #200.

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.34%. Comparing base (3c7d08f) to head (b2a9772).
Report is 140 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #235      +/-   ##
==========================================
+ Coverage   89.16%   89.34%   +0.17%     
==========================================
  Files           6        6              
  Lines         674      685      +11     
==========================================
+ Hits          601      612      +11     
  Misses         50       50              
  Partials       23       23              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[striker2000]

@googlebot I signed it!

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[willnorris]

Hey @striker2000, thanks for this patch! Sorry it's taken a couple of week to get back to you.

I think this is certainly a good approach; my only reservation is about where exactly inside of TransformingTransport.RoundTrip that we block. I haven't had a chance to profile this code to know exactly where the out of memory issue in #200 is occurring. If it's inside the cache, then where you have the limiter check is probably right. But if it's actually happening during the transform step, then we probably want to block right before calling Transform. That way, the should304 call can still return 304 responses without tying up the limiter.

Does that make sense?

[mdkent]

We're seeing fairly frequent OOM's (~10 a day) which are likely related to transformations. Is anyone using this patch successfully?

[Jorgevillada]

@mdkent thsi PR has a problem with inflight request.

I have updated with the current main branch and this PR. https://github.com/Jorgevillada/imageproxy

error copying response: readfrom tcp 127.0.0.1:8080->127.0.0.1:53150: write tcp 127.0.0.1:8080->127.0.0.1:53150: write: broken pipe

Some request never finish.

with main branch, the behavior with the ram is weird with invalid requests.