Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 20 additions & 4 deletions docs/admin-manual/log-management/be-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,22 +87,38 @@ The following configuration items are configured in the `be.conf` file.

## Enable DEBUG Log

BE's Debug log currently only supports modification through configuration files and restarting the BE node to take effect.
### Static Configuration

Set `sys_log_verbose_modules` and `sys_log_verbose_level` in `be.conf`:

```text
sys_log_verbose_modules=plan_fragment_executor,olap_scan_node
sys_log_verbose_level=3
```

`sys_log_verbose_modules` specifies the file names to be enabled, and wildcards (*) can be used. For example:
`sys_log_verbose_modules` Specifies the names of the files to be opened, which can be specified by the wildcard `*`. For example:

```text
sys_log_verbose_modules=*
```

indicates enabling all DEBUG logs.
will turn on all BE verbose log.

`sys_log_verbose_level` Indicates the level of DEBUG. The higher the number, the more detailed the DEBUG log. The value ranges from 1 to 10.

### Dynamic Modification

Since 2.1, the DEBUG log of BE supports dynamic modification via the following RESTful API:

```bash
curl -X POST "http://<be_host>:<webport>/api/glog/adjust?module=<module_name>&level=<level_number>"
```

The dynamic adjustment method also supports wildcards, e.g. using `module=*&level=10` will turn on all BE vlogs, but wildcards are not attached to individual module names. e.g. adjusting the vlog level of `moduleA` to `10`, then using `module=*&level=-1` will **NOT** turn off the vlog of `moduleA`'s vlog.

Note: Dynamically adjusted configurations are not persisted and will expire after a BE reboot.

`sys_log_verbose_level` indicates the level of DEBUG. The larger the number, the more detailed the DEBUG log. The value range is from 1 to 10.
In addition, GLOG will create the corresponding log module if the module does not exist (no real effect) and will not return an error, regardless of the method.

## Container Environment Log Configuration

Expand Down
61 changes: 61 additions & 0 deletions docs/admin-manual/open-api/be-http/be-vlog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
{
"title": "Modify BE VLOG",
"language": "en"
}
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->


## Request

`POST /api/glog/adjust?module=<module_name>&level=<level_number>`

## Description

This function is used to dynamically adjust the VLOG log on the BE side.

## Query parameters

* `module_name`
Module to set up VLOG, corresponding to BE without suffix filename

* `level_number`
VLOG level, from 1 to 10. And -1 for off

## Request body

None

## Response

```json
{
msg: "adjust vlog of xxx from -1 to 10 succeed",
code: 0
}
```

## Examples

```bash
curl -X POST "http://127.0.0.1:8040/api/glog/adjust?module=vrow_distribution&level=-1"
```
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,9 @@ under the License.
| `LOG_DIR` | `ENV(DORIS_HOME)/log` | | 所有日志的存放路径。默认为 BE 部署路径的 `log/` 目录下。注意这是一个环境变量,配置名需大写。 |
| `sys_log_level` | `INFO` | `INFO`, `WARNING`, `ERROR`, `FATAL` | `be.INFO` 的日志等级。默认为 INFO。不建议修改,INFO 等级包含许多关键日志信息。|
| `sys_log_roll_num` | 10 | | 控制 `be.INFO` 和 `be.WARNING` 最大文件数量。默认 10。当因为日志滚动或切分后,日志文件数量大于这个阈值后,老的日志文件将被删除 |
|`sys_log_verbose_modules`| | | 可以设置指定代码目录下的文件开启 DEBUG 级别日志。请参阅 "开启 DEBUG 日志" 章节 |
|`sys_log_verbose_level`| | | 请参阅 "开启 DEBUG 日志" 章节 |
|`sys_log_verbose_flags_v`| | | 请参阅 "开启 DEBUG 日志" 章节 |
| `sys_log_verbose_modules`| | | 可以设置指定代码目录下的文件开启 DEBUG 级别日志。请参阅 "开启 DEBUG 日志" 章节 |
| `sys_log_verbose_level`| | | 请参阅 "开启 DEBUG 日志" 章节 |
| `sys_log_verbose_flags_v`| | | 请参阅 "开启 DEBUG 日志" 章节 |
| `sys_log_roll_mode` | `SIZE-MB-1024` | `TIME-DAY`, `TIME-HOUR`, `SIZE-MB-nnn` | `be.INFO` 和 `be.WARNING` 日志的滚动策略。默认为 `SIZE-MB-1024`,即每个日志达到 1024MB 大小后,生成一个新的日志文件。也可以设置按天或按小时滚动 |
| `log_buffer_level` | 空 | 空 或 `-1` | BE 日志输出模式。默认情况下,BE 日志会异步下刷到磁盘日志文件中。如果设置为 -1,则日志内容会实时下刷。实时下刷会影响日志性能,但可以尽可能多的保留最新的日志。如在 BE 异常宕机情况下,可以看到最后的日志信息。|
| `disable_compaction_trace_log` | true | true, false | 默认为 true,即关闭 compaction 操作的 tracing 日志。如果为 false,则会打印和 Compaction 操作相关的 tracing 日志,用于排查问题。|
Expand All @@ -86,14 +86,16 @@ under the License.

## 开启 DEBUG 日志

BE 的 Debug 日志目前仅支持通过配置文件修改并重启 BE 节点以生效。
### 静态配置

在 `be.conf` 中设置 `sys_log_verbose_modules` 与 `sys_log_verbose_level`。

```text
sys_log_verbose_modules=plan_fragment_executor,olap_scan_node
sys_log_verbose_level=3
```

`sys_log_verbose_modules` 指定要开启的文件名,可以通过通配符 * 指定。比如:
`sys_log_verbose_modules` 指定要开启的文件名,可以通过通配符 `*` 指定。比如:

```text
sys_log_verbose_modules=*
Expand All @@ -103,6 +105,20 @@ sys_log_verbose_modules=*

`sys_log_verbose_level` 表示 DEBUG 的级别。数字越大,则 DEBUG 日志越详细。取值范围在 1-10。

### 动态调整

BE 的 DEBUG 日志从 2.1 开始支持动态修改,通过以下 RESTful API 即可:

```bash
curl -X POST "http://<be_host>:<webport>/api/glog/adjust?module=<module_name>&level=<level_number>"
```

动态调整方式同样支持通配符,例如使用 `module=*&level=10` 将打开所有 BE vlog。但通配符与单独的模块名互不隶属,例如将 `moduleA` 的 vlog 级别调整为 `10`,再使用 `module=*&level=-1`,并**不会**关闭 `moduleA` 的 vlog。

注意:动态调整的配置不会被持久化,BE 重启后将会失效。

另外无论通过何种方式,只要模块不存在,GLOG 将会创建对应日志模块(没有实际影响),并不会返回错误。

## 容器环境日志配置

在某些情况下,通过容器环境(如 k8s)部署 FE 进程。所有日志需要通过标准输出流而不是文件进行输出。
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
{
"title": "调整 BE VLOG",
"language": "zh-CN"
}
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->


## Request

`POST /api/glog/adjust?module=<module_name>&level=<level_number>`

## Description

该功能用于动态调整 BE 端 VLOG 日志。

## Query parameters

* `module_name`
要设置 VLOG 的模块,对应 BE 无后缀名的文件名

* `level_number`
VLOG 级别,从 1 到 10,另外 -1 为关闭

## Request body


## Response

```json
{
msg: "adjust vlog of xxx from -1 to 10 succeed",
code: 0
}
```

## Examples

```bash
curl -X POST "http://127.0.0.1:8040/api/glog/adjust?module=vrow_distribution&level=-1"
```
Original file line number Diff line number Diff line change
Expand Up @@ -86,14 +86,16 @@ under the License.

## 开启 DEBUG 日志

BE 的 Debug 日志目前仅支持通过配置文件修改并重启 BE 节点以生效。
### 静态配置

在 `be.conf` 中设置 `sys_log_verbose_modules` 与 `sys_log_verbose_level`。

```text
sys_log_verbose_modules=plan_fragment_executor,olap_scan_node
sys_log_verbose_level=3
```

`sys_log_verbose_modules` 指定要开启的文件名,可以通过通配符 * 指定。比如:
`sys_log_verbose_modules` 指定要开启的文件名,可以通过通配符 `*` 指定。比如:

```text
sys_log_verbose_modules=*
Expand All @@ -103,6 +105,20 @@ sys_log_verbose_modules=*

`sys_log_verbose_level` 表示 DEBUG 的级别。数字越大,则 DEBUG 日志越详细。取值范围在 1-10。

### 动态调整

通过以下 RESTful API 即可:

```bash
curl -X POST "http://<be_host>:<webport>/api/glog/adjust?module=<module_name>&level=<level_number>"
```

动态调整方式同样支持通配符,例如使用 `module=*&level=10` 将打开所有 BE vlog。但通配符与单独的模块名互不隶属,例如将 `moduleA` 的 vlog 级别调整为 `10`,再使用 `module=*&level=-1`,并**不会**关闭 `moduleA` 的 vlog。

注意:动态调整的配置不会被持久化,BE 重启后将会失效。

另外无论通过何种方式,只要模块不存在,GLOG 将会创建对应日志模块(没有实际影响),并不会返回错误。

## 容器环境日志配置

在某些情况下,通过容器环境(如 k8s)部署 FE 进程。所有日志需要通过标准输出流而不是文件进行输出。
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
{
"title": "调整 BE VLOG",
"language": "zh-CN"
}
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->


## Request

`POST /api/glog/adjust?module=<module_name>&level=<level_number>`

## Description

该功能用于动态调整 BE 端 VLOG 日志。

## Query parameters

* `module_name`
要设置 VLOG 的模块,对应 BE 无后缀名的文件名

* `level_number`
VLOG 级别,从 1 到 10,另外 -1 为关闭

## Request body


## Response

```json
{
msg: "adjust vlog of xxx from -1 to 10 succeed",
code: 0
}
```

## Examples

```bash
curl -X POST "http://127.0.0.1:8040/api/glog/adjust?module=vrow_distribution&level=-1"
```
Original file line number Diff line number Diff line change
Expand Up @@ -86,14 +86,16 @@ under the License.

## 开启 DEBUG 日志

BE 的 Debug 日志目前仅支持通过配置文件修改并重启 BE 节点以生效。
### 静态配置

在 `be.conf` 中设置 `sys_log_verbose_modules` 与 `sys_log_verbose_level`。

```text
sys_log_verbose_modules=plan_fragment_executor,olap_scan_node
sys_log_verbose_level=3
```

`sys_log_verbose_modules` 指定要开启的文件名,可以通过通配符 * 指定。比如:
`sys_log_verbose_modules` 指定要开启的文件名,可以通过通配符 `*` 指定。比如:

```text
sys_log_verbose_modules=*
Expand All @@ -103,6 +105,20 @@ sys_log_verbose_modules=*

`sys_log_verbose_level` 表示 DEBUG 的级别。数字越大,则 DEBUG 日志越详细。取值范围在 1-10。

### 动态调整

通过以下 RESTful API 即可:

```bash
curl -X POST "http://<be_host>:<webport>/api/glog/adjust?module=<module_name>&level=<level_number>"
```

动态调整方式同样支持通配符,例如使用 `module=*&level=10` 将打开所有 BE vlog。但通配符与单独的模块名互不隶属,例如将 `moduleA` 的 vlog 级别调整为 `10`,再使用 `module=*&level=-1`,并**不会**关闭 `moduleA` 的 vlog。

注意:动态调整的配置不会被持久化,BE 重启后将会失效。

另外无论通过何种方式,只要模块不存在,GLOG 将会创建对应日志模块(没有实际影响),并不会返回错误。

## 容器环境日志配置

在某些情况下,通过容器环境(如 k8s)部署 FE 进程。所有日志需要通过标准输出流而不是文件进行输出。
Expand Down
Loading