Skip to content

[Spark Load][Bug] Load job's state is incorrect after FE restart #4519

@xy720

Description

@xy720

Describe the bug
Load job's state will become incorrect after FE restart.

To Reproduce
Steps to reproduce the behavior:

Restart when the state is PENDDING

  1. Submit a spark load job
  2. The job state is PENDDING at the beginning.
  3. Restart FE
  4. Show load. And you can see the job state is CANCELLED with msg Label [xxx] has already been used.
  5. Open the web interface of the yarn cluster with browser, you can see that the spark job has been submitted successfully and is running well and normally.

Expected behavior
The job state should be PENDDING after FE restart.

Restart when the state is ETL

  1. Submit a spark load job
  2. Wait until the job state become ETL
  3. Restart FE
  4. Show load. And you can see the state of the job will always be stuck in ETL with progress 0%.
  5. Open the web interface of the yarn cluster with browser, you can see that the spark job has been submitted successfully and is running well and normally.

Expected behavior
The job state should be ETL after FE restart. And the state will not stuck in ETL.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions