Feedback: some benchmark experiences with and without "ngx_http_neteye_security" framework used within SEnginx

[peterbowey]

@InfoHunter

As a result of looking through my current SEnginx 'debug' logs and the 'proxy' Upstream 'cold start' timings, I tried the following two SEnginx "core" code changes and a some client side http benchmarks (as before and after):

Code changes were:

1. Removed 'ngx_http_neteye_security' and the associated "neteye" code embedded in the following SEnginx files: A) src/http/ngx_http.c B) src/http/ngx_http_core_module.cand C) src/http/ngx_http_core_module.h.

2. Removed all "neteye" SEnginx modules except for the following requisites: A) ngx_http_upstream_fastest, B) ngx_http_upstream_persistence, and C) ngx_http_if_extend.

Test Results: The tested 'cold start' upstream proxy [3 streams as Unix sockets] load times went from ~3 seconds [cold] to 295ms on a cold start (with the above changes). The warm [valid] cache upstream load times did not change much [good], but the cache 'invalid' times were [also] improved as much as the cold start times [as above]. Notes: The three proxy upstream, as Unix sockets, pull static assets (only) for a local CDN - which are store (optimized) using nginx's "proxy_store" (on 1st pull).

This report is intended as 'basic user' feedback on the possible overheads of utilizing the current "ngx_http_neteye_security" framework API's - as placed into SEnginx core and the associated 'neteye' security modules.

I accept I could have some configuration issues to follow through on :)

I wish I could offer you a better stream of hard data and numbers, but it is a production server that I use SEnginx on, so my change window is always brief... :)

[InfoHunter]

@peterbowey

Hi Peter, how did you measure the perfomance at cilent side, what tools did you use ? I think I can solve this issue in my own environment (to improve the performance of SEnginx)

[peterbowey]

@InfoHunter

Hi Paul,

On the client side (event window testing), I simply used Firefox and timed the static http responses via the proxy up-streams (cold and warm caches). As I have custom named upstream static pools (as a local CDN), it was easy enough to metric this way.

I also metric response times with http://www.webpagetest.org/

[InfoHunter]

@peterbowey

Hi Peter, I've tested the performance of SEnginx with and without neteye security layer ( along with the security modules). I found that the results were almost the same. Since I didn't enable any security features during the test, the overhead introduced by those security components would be considered as "very little".

The configuration I've used during the test was really simple: just a standard proxy configuration fetching static contents from one backend server.

The tool I used is "ab"

Besides the above result, I have profiled SEnginx in the past to find the "hot spot" in SEnginx. According to the profiler, even a security module was enabled, all overheads added by neteye layer were very little, if no security module is enabled, the overheads could almost be ignored.

There is a result from the profiler (google performance tool)
Result that is tested with robot mitigation enabled:

[InfoHunter]

The samples might not be big enough, just for reference. : )

I also have some more old profiling result, if you need them, I can give them to you by email.

[peterbowey]

@InfoHunter

Hi Paul,

Many thanks for doing the above benchmark (fantastic work)!

That profiling makes a good study and I will read it through.....

I consider some of my previous benchmarks are impacted by my practice of using three Unix socket up-stream proxies for pure static assets (creating a local CDN pull domain). Security and cookies are likely 'non essential' here, as it is not a http(s) (or tcp) stream that can not be seen outside of the local system scope. By 'slowly reading new SEnginx 'debug based' logs, I am finding there are several options I can make obsolete (on the private socket pathways).

My benchmark discussion here, is based the over-head to pull a (local) static asset, using a private Unix socket into the CDN static domain name (as a local 'cache' directory), and thus store it via nginx "proxy_store" (on 1st pull). For benchmarks, I userm -f -r * and then time the event for nginx to 'fill' this directory via 'proxy_store'. With 10 domains I use, there can be a cold start 'rush' for up to 4,000 static assets to be stored, then mirrorred out as standard http traffic (@cdn).

Once, the asset is 'mirrored' at the CDN 'pull' domain, it becomes a standard optimized static domain. And it this point all interaction is via normal http(s), and nginx rules and security can apply.

My metric 'issue' is here - focused on the 'cold / empty' creation time of this private unix socket stream (cold cache time on a empty CDN 'proxy_store' directory). After this static pool is complete, I find there is very minimal impact of using the extra SEnginx 'security components'. So I fully agree with your own 'profiling' statistics.... (and thanks for sharing them!)

So it is my 'strange' local CDN pull method - I think :-)

However, I am having a good 'debug' run optimizing that method..

Thanks again for your great profiling Paul.

[peterbowey]

@InfoHunter

Paul if you have time to send your 'old profiling' result via my email - that would be a great 'study' in my own time.

Many thanks for your input and time. I see [and use] SEnginx as a fast growing and extremely powerful tool.

[InfoHunter]

@peterbowey

No problem, already sent.