WASM filter has significant CPU overhead

[woehrl01]

Bug Report

Describe the bug

I'm using a WASM filter which does nothing than returning the logs. I'm using the dummy input with a rate 25 to test. The overhead is significant:

Wasm filter disabled:

Wasm filter enabled:

I believe this is because the WASM filter gets recreated for every line, instead of reusing it.

To Reproduce

• Rubular link if applicable:
• Example log message if applicable:

{"log":"YOUR LOG MESSAGE HERE","stream":"stdout","time":"2018-06-11T14:37:30.681701731Z"}

• Steps to reproduce the problem:

Expected behavior

Screenshots

Your Environment

• Version used: 2.0
• Configuration:
• Environment name and version (e.g. Kubernetes? What version?):
• Server type and version:
• Operating System and version:
• Filters and plugins: custom wasm filter

Additional context

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.

[Sharsie]

@cosmo0920 @edsiper Could we reopen this?
I've done some testing locally, so admitedly this does not represent a real-life scenario, but the performance difference is so huge that I feel like sharing it makes sense.

For the test I ran the following configurations

[SERVICE]
    Daemon       Off
    Log_Level    error
    HTTP_Server  Off

[INPUT]
    Name          dummy
    Tag           dummy.local
    Dummy         {"log":""}
    Samples       1000000
    Interval_nsec 1

[OUTPUT]
    Name  stdout
    Match *

with a lua filter returning the inputs as they are

[FILTER]
    Name  lua
    match dummy.*
    call  cb_print
    code  function cb_print(tag, timestamp, record) return tag, timestamp, record end

with a wasm filter built with tinygo(and flb_wamrc optimized)

[FILTER]
    Name          wasm
    match         dummy.*
    Wasm_Path     /etc/fluent-bit/noop-filter.wasm
    #Wasm_Path     /etc/fluent-bit/noop-filter.wasm.aot
    Function_Name go_filter

go code

package main

//export go_filter
func go_filter(tag *uint8, tag_len uint, time_sec uint, time_nsec uint, record *uint8, record_len uint) *uint8 {
	return record
}

func main() {}

All results are 1M samples with 1ns interval
No filter: 2.701253696s
Lua filter: 4.983801003s
Wasm filter: 711.233964105s
Wasm filter optimized with flb_wamrc: 124.094006389s

[cosmo0920]

Wasm filter will use larger resources because it needs to open each of cycles on execution. This reason is memory securities. wasm-micro-runtime requests to allocate and not to pollute the loaded memory region where is having wasm modules.
By this requirement, filter_wasm is eating up relatively high CPU.
Also, filter_wasm encodes JSON and msgpack each other. This is also one of the culprits of consuming CPU.