Skip to content

Make HTTP/2 session and stream windows configurable #9085

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bneradt merged 4 commits into from
Oct 20, 2022
Merged

Make HTTP/2 session and stream windows configurable #9085

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e960ab7
Add HTTP/2 flow control configurability
bneradt Sep 10, 2022
b042ad2
Handle shrinking stream window sizes.
bneradt Oct 14, 2022
a063016
http2.flow_control.in.policy -> http2.flow_control.policy.in
bneradt Oct 19, 2022
39dcd61
http2.flow_control.policy.in -> http2.flow_control.policy_in
bneradt Oct 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion doc/admin-guide/files/records.config.en.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4152,7 +4152,40 @@ HTTP/2 Configuration
:reloadable:
:units: bytes

The initial window size for inbound connections.
The initial HTTP/2 stream window size for inbound connections that |TS| as a
receiver advertises to the peer. See IETF RFC 9113 section 5.2 for details
concerning HTTP/2 flow control. See
:ts:cv:`proxy.config.http2.flow_control.policy_in` for how HTTP/2 stream and
session windows are maintained over the lifetime of HTTP/2 sessions.

.. ts:cv:: CONFIG proxy.config.http2.flow_control.policy_in INT 0
:reloadable:

Specifies the mechanism |TS| uses to maintian flow control via the HTTP/2
stream and session windows for inbound connections. See IETF RFC 9113
section 5.2 for details concerning HTTP/2 flow control.

===== ===========================================================================================
Value Description
===== ===========================================================================================
``0`` Session and stream receive windows are initialized and maintained at the value as specified
in :ts:cv:`proxy.config.http2.initial_window_size_in` over the lifetime of HTTP/2
sessions.
``1`` Session receive windows are initialized to the value of the product of
:ts:cv:`proxy.config.http2.initial_window_size_in` and
:ts:cv:`proxy.config.http2.max_concurrent_streams_in` and are maintained as such over the
lifetime of HTTP/2 sessions. Stream windows are initialized to the value of
:ts:cv:`proxy.config.http2.initial_window_size_in` and are maintained as such over the
lifetime of each HTTP/2 stream.
``2`` Session receive windows are initialized to the value of the product of
:ts:cv:`proxy.config.http2.initial_window_size_in` and
:ts:cv:`proxy.config.http2.max_concurrent_streams_in` and are maintained as such over the
lifetime of HTTP/2 sessions. Stream windows are initialized to the value of
:ts:cv:`proxy.config.http2.initial_window_size_in` but are dynamically adjusted to the
session window size divided by the number of concurrent streams over the lifetime of HTTP/2
sessions. That is, stream window sizes dynamically adjust to fill the session window in
a way that shares the window equally among all concurrent streams.
===== ===========================================================================================

.. ts:cv:: CONFIG proxy.config.http2.max_frame_size INT 16384
:reloadable:
Expand Down
2 changes: 2 additions & 0 deletions mgmt/RecordsConfig.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1301,6 +1301,8 @@ static const RecordElement RecordsConfig[] =
,
{RECT_CONFIG, "proxy.config.http2.initial_window_size_in", RECD_INT, "65535", RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
,
{RECT_CONFIG, "proxy.config.http2.flow_control.policy_in", RECD_INT, "0", RECU_DYNAMIC, RR_NULL, RECC_STR, "[0-2]", RECA_NULL}
,
{RECT_CONFIG, "proxy.config.http2.max_frame_size", RECD_INT, "16384", RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
,
{RECT_CONFIG, "proxy.config.http2.header_table_size", RECD_INT, "4096", RECU_DYNAMIC, RR_NULL, RECC_STR, "^[0-9]+$", RECA_NULL}
Expand Down
72 changes: 41 additions & 31 deletions proxy/http2/HTTP2.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -567,35 +567,36 @@ http2_decode_header_blocks(HTTPHdr *hdr, const uint8_t *buf_start, const uint32_
}

// Initialize this subsystem with librecords configs (for now)
uint32_t Http2::max_concurrent_streams_in = 100;
uint32_t Http2::min_concurrent_streams_in = 10;
uint32_t Http2::max_active_streams_in = 0;
bool Http2::throttling = false;
uint32_t Http2::stream_priority_enabled = 0;
uint32_t Http2::initial_window_size = 65535;
uint32_t Http2::max_frame_size = 16384;
uint32_t Http2::header_table_size = 4096;
uint32_t Http2::max_header_list_size = 4294967295;
uint32_t Http2::accept_no_activity_timeout = 120;
uint32_t Http2::no_activity_timeout_in = 120;
uint32_t Http2::active_timeout_in = 0;
uint32_t Http2::push_diary_size = 256;
uint32_t Http2::zombie_timeout_in = 0;
float Http2::stream_error_rate_threshold = 0.1;
uint32_t Http2::stream_error_sampling_threshold = 10;
uint32_t Http2::max_settings_per_frame = 7;
uint32_t Http2::max_settings_per_minute = 14;
uint32_t Http2::max_settings_frames_per_minute = 14;
uint32_t Http2::max_ping_frames_per_minute = 60;
uint32_t Http2::max_priority_frames_per_minute = 120;
float Http2::min_avg_window_update = 2560.0;
uint32_t Http2::con_slow_log_threshold = 0;
uint32_t Http2::stream_slow_log_threshold = 0;
uint32_t Http2::header_table_size_limit = 65536;
uint32_t Http2::write_buffer_block_size = 262144;
float Http2::write_size_threshold = 0.5;
uint32_t Http2::write_time_threshold = 100;
uint32_t Http2::buffer_water_mark = 0;
uint32_t Http2::max_concurrent_streams_in = 100;
uint32_t Http2::min_concurrent_streams_in = 10;
uint32_t Http2::max_active_streams_in = 0;
bool Http2::throttling = false;
uint32_t Http2::stream_priority_enabled = 0;
uint32_t Http2::initial_window_size_in = 65535;
Http2FlowControlPolicy Http2::flow_control_policy_in = Http2FlowControlPolicy::STATIC_SESSION_AND_STATIC_STREAM;
uint32_t Http2::max_frame_size = 16384;
uint32_t Http2::header_table_size = 4096;
uint32_t Http2::max_header_list_size = 4294967295;
uint32_t Http2::accept_no_activity_timeout = 120;
uint32_t Http2::no_activity_timeout_in = 120;
uint32_t Http2::active_timeout_in = 0;
uint32_t Http2::push_diary_size = 256;
uint32_t Http2::zombie_timeout_in = 0;
float Http2::stream_error_rate_threshold = 0.1;
uint32_t Http2::stream_error_sampling_threshold = 10;
uint32_t Http2::max_settings_per_frame = 7;
uint32_t Http2::max_settings_per_minute = 14;
uint32_t Http2::max_settings_frames_per_minute = 14;
uint32_t Http2::max_ping_frames_per_minute = 60;
uint32_t Http2::max_priority_frames_per_minute = 120;
float Http2::min_avg_window_update = 2560.0;
uint32_t Http2::con_slow_log_threshold = 0;
uint32_t Http2::stream_slow_log_threshold = 0;
uint32_t Http2::header_table_size_limit = 65536;
uint32_t Http2::write_buffer_block_size = 262144;
float Http2::write_size_threshold = 0.5;
uint32_t Http2::write_time_threshold = 100;
uint32_t Http2::buffer_water_mark = 0;

void
Http2::init()
Expand All @@ -604,7 +605,16 @@ Http2::init()
REC_EstablishStaticConfigInt32U(min_concurrent_streams_in, "proxy.config.http2.min_concurrent_streams_in");
REC_EstablishStaticConfigInt32U(max_active_streams_in, "proxy.config.http2.max_active_streams_in");
REC_EstablishStaticConfigInt32U(stream_priority_enabled, "proxy.config.http2.stream_priority_enabled");
REC_EstablishStaticConfigInt32U(initial_window_size, "proxy.config.http2.initial_window_size_in");
REC_EstablishStaticConfigInt32U(initial_window_size_in, "proxy.config.http2.initial_window_size_in");

uint32_t flow_control_policy_in_int = 0;
REC_EstablishStaticConfigInt32U(flow_control_policy_in_int, "proxy.config.http2.flow_control.policy_in");
if (flow_control_policy_in_int > 2) {
Error("Invalid value for proxy.config.http2.flow_control.policy_in: %d", flow_control_policy_in_int);
flow_control_policy_in_int = 0;
}
flow_control_policy_in = static_cast<Http2FlowControlPolicy>(flow_control_policy_in_int);

REC_EstablishStaticConfigInt32U(max_frame_size, "proxy.config.http2.max_frame_size");
REC_EstablishStaticConfigInt32U(header_table_size, "proxy.config.http2.header_table_size");
REC_EstablishStaticConfigInt32U(max_header_list_size, "proxy.config.http2.max_header_list_size");
Expand Down Expand Up @@ -632,7 +642,7 @@ Http2::init()
// If any settings is broken, ATS should not start
ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, max_concurrent_streams_in}));
ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, min_concurrent_streams_in}));
ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_INITIAL_WINDOW_SIZE, initial_window_size}));
ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_INITIAL_WINDOW_SIZE, initial_window_size_in}));
ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_MAX_FRAME_SIZE, max_frame_size}));
ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_HEADER_TABLE_SIZE, header_table_size}));
ink_release_assert(http2_settings_parameter_is_valid({HTTP2_SETTINGS_MAX_HEADER_LIST_SIZE, max_header_list_size}));
Expand Down
21 changes: 16 additions & 5 deletions proxy/http2/HTTP2.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,15 @@

class HTTPHdr;

typedef unsigned Http2StreamId;
// [RFC 9113] 5.1.1 Stream identifiers.
using Http2StreamId = uint32_t;

constexpr Http2StreamId HTTP2_CONNECTION_CONTROL_STRTEAM = 0;
constexpr uint8_t HTTP2_FRAME_NO_FLAG = 0;
constexpr Http2StreamId HTTP2_CONNECTION_CONTROL_STREAM = 0;
constexpr uint8_t HTTP2_FRAME_NO_FLAG = 0;

// [RFC 7540] 6.9.2. Initial Flow Control Window Size
// the flow control window can be come negative so we need to track it with a signed type.
typedef int32_t Http2WindowSize;
using Http2WindowSize = int32_t;

extern const char *const HTTP2_CONNECTION_PREFACE;
const size_t HTTP2_CONNECTION_PREFACE_LEN = 24;
Expand Down Expand Up @@ -360,6 +361,15 @@ ParseResult http2_convert_header_from_2_to_1_1(HTTPHdr *);
ParseResult http2_convert_header_from_1_1_to_2(HTTPHdr *);
void http2_init();

/** Each of these values correspond to the flow control policy described in or
* records.config documentation for proxy.config.http2.flow_control.policy_in.
*/
enum class Http2FlowControlPolicy {
STATIC_SESSION_AND_STATIC_STREAM,
LARGE_SESSION_AND_STATIC_STREAM,
LARGE_SESSION_AND_DYNAMIC_STREAM,
};

// Not sure where else to put this, but figure this is as good of a start as
// anything else.
// Right now, only the static init() is available, which sets up some basic
Expand All @@ -373,7 +383,8 @@ class Http2
static uint32_t max_active_streams_in;
static bool throttling;
static uint32_t stream_priority_enabled;
static uint32_t initial_window_size;
static uint32_t initial_window_size_in;
static Http2FlowControlPolicy flow_control_policy_in;
static uint32_t max_frame_size;
static uint32_t header_table_size;
static uint32_t max_header_list_size;
Expand Down
Loading