Update awslogs.md by mixja · Pull Request #3319 · docker/docs

mixja · 2017-05-18T08:58:29Z

Signed-off-by: Justin Menga justin.menga@gmail.com

Proposed changes

Add documentation for new AWS CloudWatch logs multiline pattern options

Unreleased project version

Docker Engine 17.06

Related issues

moby/moby#30891

mdlinville · 2017-05-23T17:09:28Z

@nathanleclaire @FrenchBen @friism PTAL

nrdlngr

Hi, and thanks for your contribution. Would you mind making the following suggested changes to your documentation?

nrdlngr · 2017-05-23T18:30:54Z


+### awslogs-multiline-pattern
+
+The `awslogs-multiline-pattern` specifies a regular expression pattern for identifying the start of a log message.  A log message is made of a line that matches the pattern and any following lines that don't match the pattern.  


The awslogs-multiline-pattern specifies a regular expression pattern for identifying the beginning of a container log entry. Any following lines that do not match the pattern are considered to be part of the current log entry. The next line that matches the pattern begins a new log entry.

nrdlngr · 2017-05-23T18:31:53Z

+> **Note**:
+> Multiline logging performs regular expression parsing and matching of all log messages, which may have a negative impact on logging performance.
+
+For example, to process the following log stream where new log messages start with the pattern `INFO`:


For example, to process the following log stream where new log entries begin with the pattern INFO:

nrdlngr · 2017-05-23T18:33:31Z

+             ...
+```
+
+Which will generate the following CloudWatch log events:


This configuration would generate the following CloudWatch log entries:

nrdlngr · 2017-05-23T18:44:42Z

+Which will generate the following CloudWatch log events:
+
+```
+# First event


Can you change "First event", "Second event", and "Third event" to use "entry" instead of event?

nrdlngr · 2017-05-23T18:48:02Z

+
+### awslogs-datetime-format
+
+The `awslogs-datetime-format` specifies an [strftime expression pattern](http://strftime.org) for identifying the start of a log message.  A log message is made of a line that matches the pattern and any following lines that don't match the pattern.  


The awslogs-datetime-format specifies a strftime expression pattern for for identifying the beginning of a container log entry. Any following lines that do not match the pattern are considered to be part of the current log entry. The next line that matches the pattern begins a new log entry.

nrdlngr · 2017-05-23T18:50:03Z

+> **Note**:
+> Multiline logging performs regular expression parsing and matching of all log messages, which may have a negative impact on logging performance.
+
+For example, to process the following log stream where new log messages start with the following timestamps:


For example, to process the following log stream where new log entries begin with the following timestamps:

nrdlngr · 2017-05-23T18:52:23Z

+[May 01, 2017 19:01:32] Another message was logged
+```
+
+An strftime expression of `[%b %d, %Y %H:%M:%S]` can be used:


You can use the strftime expression, [%b %d, %Y %H:%M:%S]:

nrdlngr · 2017-05-23T18:53:08Z

+             ...
+```
+
+Which will generate the following CloudWatch log events:


This configuration would generate the following CloudWatch log entries:

nrdlngr · 2017-05-23T18:53:33Z

+Which will generate the following CloudWatch log events:
+
+```
+# First event


Can you change "First event", "Second event", and "Third event" to use "entry" instead of event?

nrdlngr · 2017-05-23T18:54:04Z

+[May 01, 2017 19:01:32] Another message was logged
+```
+
+The following strftime codes are supported:


The following strftime codes are supported:

mixja · 2017-05-27T20:33:28Z

@nrdlngr - changes submitted as requested

mdlinville · 2017-05-30T18:07:44Z

@FrenchBen PTAL or nominate.

FrenchBen · 2017-05-31T22:58:57Z

I think this is more of a review by @samuelkarp - if he's happy with it, I don't see any blockers

samuelkarp · 2017-05-31T23:09:27Z

-Mac; for example: `sudo ifconfig lo0 alias 10.200.10.1/24`, and make sure that
-your service is listening on this address or `0.0.0.0` (ie not `127.0.0.1`).
-Then containers can connect to this address.
+The Mac has a changing IP address (or none if you have no network access). From


This looks like an unrelated change.

samuelkarp · 2017-05-31T23:10:26Z

 > Your AWS IAM policy must include the `logs:CreateLogGroup` permission before you attempt to use `awslogs-create-group`.

+### awslogs-multiline-pattern
+


Does this need {% include edge_only.md section="option" %} or did the feature make it into the 17.06.0 RC?

It still needs it even if so, because 17.06 is not released yet. The code is:

{% include edge_only.md section="option" %}

Or maybe "feature" instead of "option" if that fits better. The text is not arbitrary, but there are a few choices, see _inclues/edge_only.md for more info on how it works.

mdlinville · 2017-06-19T19:20:33Z

I interactive rebased and removed the unrelated commit.

mdlinville · 2017-06-22T22:18:59Z

This was already addressed in #3615

samuelkarp · 2017-06-22T22:21:52Z

@mstanleyjones It looks like #3615 lost a lot of the content from this one. Is that intentional?

mdlinville · 2017-06-22T22:32:03Z

Not necessarily, no. I'll review it and see what we need to keep from here. I wasn't focused on this PR when I did the work on that other one.

Signed-off-by: Justin Menga <justin.menga@gmail.com>

mdlinville · 2017-06-23T23:04:02Z

I'm taking this over to get the important info in here ported over.

samuelkarp · 2017-06-26T18:42:19Z

+INFO Another message was logged
+```
+
+You can use the strftime expression of `[%b %d, %Y %H:%M:%S]`:


Looks like this line is misplaced. Did you mean to have it say "You can use the regular expression of ^INFO:"?

samuelkarp · 2017-06-26T18:42:36Z

+For example, to process the following log stream where new log messages start with the pattern `INFO`:
+
+Consider the following log stream, where each log message should start with the
+patther `INFO`:


s/patther/pattern/

mixja mentioned this pull request May 18, 2017

Add awslogs multiline support moby/moby#30891

Merged

mdlinville added area/engine Issue affects Docker engine/daemon Tech Review: Needed labels May 23, 2017

nrdlngr suggested changes May 23, 2017

View reviewed changes

mixja force-pushed the awslogs_multiline_logging branch from 2f96537 to ff98880 Compare May 27, 2017 20:30

samuelkarp reviewed May 31, 2017

View reviewed changes

mdlinville force-pushed the awslogs_multiline_logging branch from 0ca55e2 to 09abadd Compare June 19, 2017 19:20

mdlinville closed this Jun 22, 2017

mdlinville reopened this Jun 22, 2017

mixja added 2 commits June 23, 2017 15:58

Add docs for awslogs-multiline-pattern and awslogs-datetime-format

882644a

Signed-off-by: Justin Menga <justin.menga@gmail.com>

Tech review changes

637cb38

Signed-off-by: Justin Menga <justin.menga@gmail.com>

mdlinville force-pushed the awslogs_multiline_logging branch from 09abadd to 637cb38 Compare June 23, 2017 23:01

mdlinville merged commit b1d32c9 into docker:master Jun 26, 2017

samuelkarp reviewed Jun 26, 2017

View reviewed changes


		### awslogs-multiline-pattern

		The `awslogs-multiline-pattern` specifies a regular expression pattern for identifying the start of a log message. A log message is made of a line that matches the pattern and any following lines that don't match the pattern.


		### awslogs-datetime-format

		The `awslogs-datetime-format` specifies an [strftime expression pattern](http://strftime.org) for identifying the start of a log message. A log message is made of a line that matches the pattern and any following lines that don't match the pattern.

		> Your AWS IAM policy must include the `logs:CreateLogGroup` permission before you attempt to use `awslogs-create-group`.

		### awslogs-multiline-pattern

Conversation

mixja commented May 18, 2017

Proposed changes

Unreleased project version

Related issues

Uh oh!

mdlinville commented May 23, 2017

Uh oh!

nrdlngr left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mixja commented May 27, 2017

Uh oh!

mdlinville commented May 30, 2017

Uh oh!

FrenchBen commented May 31, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

samuelkarp May 31, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdlinville May 31, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdlinville commented Jun 19, 2017

Uh oh!

mdlinville commented Jun 22, 2017

Uh oh!

samuelkarp commented Jun 22, 2017

Uh oh!

mdlinville commented Jun 22, 2017

Uh oh!

mdlinville commented Jun 23, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

samuelkarp May 31, 2017 •

edited

Loading

mdlinville May 31, 2017 •

edited

Loading