Skip to content

[Bug] Fe cannot be restarted, when the stream load record manager takes effect #5729

@EmmyMiao87

Description

@EmmyMiao87

Describe the bug

The persistence logic for FetchStreamLoadRecord is wrong which will cause the metadata to fail to read.

To Reproduce
Version 0.14.8
Steps to reproduce the behavior:

  1. execute stream load
  2. stop fe
  3. start fe
  4. fe cannot be restarted with error:
2021-04-27 20:05:02,233 ERROR (replayer|80) [BDBJournalCursor.next():117] fail to read journal entity key=349360906, will exit
java.io.EOFException: null
        at java.io.DataInputStream.readBoolean(DataInputStream.java:244) ~[?:1.8.0_131]
        at org.apache.doris.load.StreamLoadRecordMgr$FetchStreamLoadRecord.read(StreamLoadRecordMgr.java:162) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.journal.JournalEntity.readFields(JournalEntity.java:525) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.journal.bdbje.BDBJournalCursor.next(BDBJournalCursor.java:115) [palo-fe.jar:3.4.0]
        at org.apache.doris.catalog.Catalog.replayJournal(Catalog.java:2478) [palo-fe.jar:3.4.0]
        at org.apache.doris.catalog.Catalog$3.runOneCycle(Catalog.java:2267) [palo-fe.jar:3.4.0]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:116) [palo-fe.jar:3.4.0]

Expected behavior

Fe can be restarted normally.
The record of StreamLoadAudit could be replayed normally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/catalogIssues or PRs related to catalog managementkind/fixCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions