Skip to content

Dilipr/fluentd config updates #124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
r-dilip merged 2 commits into from
Aug 30, 2018
Merged

Dilipr/fluentd config updates #124

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
87a9cf8
Remove Log Processing from fluentd configuration
r-dilip Aug 16, 2018
308be41
Remove plugin references from base_container.data
r-dilip Aug 16, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 0 additions & 32 deletions installer/conf/container.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,18 +50,6 @@
]
</source>

# Container log
# Example line which matches the format:
# {"log"=>"Test 9th January\n", "stream"=>"stdout", "time"=>"2018-01-09T23:14:39.273429353Z", "ContainerID"=>"ee1ec26aa974af81b21fff24cef8ec78bf7ac1558b5de6f1eb1a5b28ecd6d559", "Image"=>"ubuntu", "Name"=>"determined_wilson", "SourceSystem"=>"Containers"}
# NOTE: The LogEntryTimeStamp is just being appended in the begining of the LogEntry field. This is the actual time the log was generated and the TimeGenerated field in Kusto is different
<source>
type containerlog_sudo_tail
pos_file /var/opt/microsoft/docker-cimprov/state/ContainerLogFile.pos.log
tag oms.container.log
format /\"log\"=>\"(?<LogEntry>.*)", \"stream\"=>\"(?<LogEntrySource>.*)", \"time\"=>\"(?<LogEntryTimeStamp>.*)", \"ContainerID\"=>\"(?<Id>.*)", \"Image\"=>\"(?<Image>.*)", \"Name\"=>\"(?<Name>.*)", \"SourceSystem\"=>\"(?<SourceSystem>.*)"}/
run_interval 60s
</source>

# Container host inventory
<source>
type omi
Expand Down Expand Up @@ -95,11 +83,6 @@
type filter_container
</filter>

# Seperate filter for container log
<filter oms.container.log>
type filter_container_log
</filter>

<match oms.api.ContainerNodeInventory**>
type out_oms_api
log_level debug
Expand Down Expand Up @@ -152,21 +135,6 @@
max_retry_wait 9m
</match>

<match oms.container.log>
type out_oms
log_level debug
num_threads 5
buffer_chunk_limit 20m
buffer_type file
buffer_path %STATE_DIR_WS%/out_oms_log*.buffer
buffer_queue_limit 20
buffer_queue_full_action drop_oldest_chunk
flush_interval 20s
retry_limit 10
retry_wait 15s
max_retry_wait 9m
</match>

<match oms.container.servicelog**>
type out_oms
log_level info
Expand Down
12 changes: 0 additions & 12 deletions installer/datafiles/base_container.data
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,11 @@ MAINTAINER: 'Microsoft Corporation'

/opt/microsoft/omsagent/plugin/filter_docker_log.rb; source/code/plugin/filter_docker_log.rb; 644; root; root
/opt/microsoft/omsagent/plugin/filter_container.rb; source/code/plugin/filter_container.rb; 644; root; root
/opt/microsoft/omsagent/plugin/filter_container_log.rb; source/code/plugin/filter_container_log.rb; 644; root; root

/opt/microsoft/omsagent/plugin/in_kube_podinventory.rb; source/code/plugin/in_kube_podinventory.rb; 644; root; root
/opt/microsoft/omsagent/plugin/in_kube_events.rb; source/code/plugin/in_kube_events.rb; 644; root; root
/opt/microsoft/omsagent/plugin/in_kube_logs.rb; source/code/plugin/in_kube_logs.rb; 644; root; root
/opt/microsoft/omsagent/plugin/KubernetesApiClient.rb; source/code/plugin/KubernetesApiClient.rb; 644; root; root
/opt/microsoft/omsagent/plugin/in_containerlog_sudo_tail.rb; source/code/plugin/in_containerlog_sudo_tail.rb; 644; root; root
/opt/microsoft/omsagent/plugin/containerlogtailfilereader.rb; source/code/plugin/containerlogtailfilereader.rb; 744; root; root

/etc/opt/microsoft/docker-cimprov/container.conf; installer/conf/container.conf; 644; root; root

Expand Down Expand Up @@ -88,15 +85,6 @@ WriteInstallInfo() {
}
WriteInstallInfo

#Setup sudo permission for containerlogtailfilereader
if [ -z $(cat /etc/sudoers.d/omsagent | grep /containerlogtailfilereader.rb) ]
then
chmod +w /etc/sudoers.d/omsagent
echo "#run containerlogtailfilereader.rb for docker-provider" >> /etc/sudoers.d/omsagent
echo "omsagent ALL=(ALL) NOPASSWD: /opt/microsoft/omsagent/ruby/bin/ruby /opt/microsoft/omsagent/plugin/containerlogtailfilereader.rb *" >> /etc/sudoers.d/omsagent
chmod 440 /etc/sudoers.d/omsagent
fi

# Get the state file in place with proper permissions
touch /var/opt/microsoft/docker-cimprov/state/LastEventQueryTime.txt
chmod 644 /var/opt/microsoft/docker-cimprov/state/LastEventQueryTime.txt
Expand Down
Loading