Skip to content

Mem_Buf_limit is configurable via ConfigMap #574

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
daweim0 merged 1 commit into from
Jun 10, 2021
Merged

Mem_Buf_limit is configurable via ConfigMap #574

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions build/common/installer/scripts/td-agent-bit-conf-customizer.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
@default_service_interval = "1"
@default_buffer_chunk_size = "1"
@default_buffer_max_size = "1"
@default_mem_buf_limit = "10"

def is_number?(value)
true if Integer(value) rescue false
Expand All @@ -19,6 +20,7 @@ def substituteFluentBitPlaceHolders
interval = ENV["FBIT_SERVICE_FLUSH_INTERVAL"]
bufferChunkSize = ENV["FBIT_TAIL_BUFFER_CHUNK_SIZE"]
bufferMaxSize = ENV["FBIT_TAIL_BUFFER_MAX_SIZE"]
memBufLimit = ENV["FBIT_TAIL_MEM_BUF_LIMIT"]

serviceInterval = (!interval.nil? && is_number?(interval) && interval.to_i > 0 ) ? interval : @default_service_interval
serviceIntervalSetting = "Flush " + serviceInterval
Expand All @@ -32,8 +34,12 @@ def substituteFluentBitPlaceHolders
tailBufferMaxSize = tailBufferChunkSize
end

tailMemBufLimit = (!memBufLimit.nil? && is_number?(memBufLimit) && memBufLimit.to_i > 10) ? memBufLimit : @default_mem_buf_limit
tailMemBufLimitSetting = "Mem_Buf_Limit " + tailMemBufLimit + "m"

text = File.read(@td_agent_bit_conf_path)
new_contents = text.gsub("${SERVICE_FLUSH_INTERVAL}", serviceIntervalSetting)
new_contents = new_contents.gsub("${TAIL_MEM_BUF_LIMIT}", tailMemBufLimitSetting)
if !tailBufferChunkSize.nil?
new_contents = new_contents.gsub("${TAIL_BUFFER_CHUNK_SIZE}", "Buffer_Chunk_Size " + tailBufferChunkSize + "m")
else
Expand Down
2 changes: 1 addition & 1 deletion build/linux/installer/conf/td-agent-bit.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
DB /var/log/omsagent-fblogs.db
DB.Sync Off
Parser docker
Mem_Buf_Limit 10m
${TAIL_MEM_BUF_LIMIT}
${TAIL_BUFFER_CHUNK_SIZE}
${TAIL_BUFFER_MAX_SIZE}
Rotate_Wait 20
Expand Down
10 changes: 10 additions & 0 deletions build/linux/installer/scripts/tomlparser-agent-config.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@
@fbitFlushIntervalSecs = 0
@fbitTailBufferChunkSizeMBs = 0
@fbitTailBufferMaxSizeMBs = 0
@fbitTailMemBufLimitMBs = 0


def is_number?(value)
Expand Down Expand Up @@ -168,6 +169,12 @@ def populateSettingValuesFromConfigMap(parsedConfig)
@fbitTailBufferMaxSizeMBs = @fbitTailBufferChunkSizeMBs
puts "config::warn: since tail_buf_maxsize_megabytes not provided hence using tail_buf_maxsize_megabytes=#{@fbitTailBufferMaxSizeMBs} which is same as the value of tail_buf_chunksize_megabytes"
end

fbitTailMemBufLimitMBs = fbit_config[:tail_mem_buf_limit_megabytes]
if !fbitTailMemBufLimitMBs.nil? && is_number?(fbitTailMemBufLimitMBs) && fbitTailMemBufLimitMBs.to_i > 0
@fbitTailMemBufLimitMBs = fbitTailMemBufLimitMBs.to_i
puts "Using config map value: tail_mem_buf_limit_megabytes = #{@fbitTailMemBufLimitMBs}"
end
end
end
rescue => errorStr
Expand Down Expand Up @@ -212,6 +219,9 @@ def populateSettingValuesFromConfigMap(parsedConfig)
if @fbitTailBufferMaxSizeMBs > 0
file.write("export FBIT_TAIL_BUFFER_MAX_SIZE=#{@fbitTailBufferMaxSizeMBs}\n")
end
if @fbitTailMemBufLimitMBs > 0
file.write("export FBIT_TAIL_MEM_BUF_LIMIT=#{@fbitTailMemBufLimitMBs}\n")
end
# Close file after writing all environment variables
file.close
else
Expand Down