Use derived parser to parse integration tests schedule file rather than maintaining manual parsing code.

[liurenjie1024]

Is your feature request related to a problem or challenge?

Currently in integration tests, we parse schedule definition using manually written code, see

iceberg-rust/crates/sqllogictest/src/schedule.rs

Line 109 in 4239a20

async fn parse_engines(

iceberg-rust/crates/sqllogictest/src/engine/datafusion.rs

Line 112 in 4239a20

async fn create_catalog(config: &TomlTable) -> Result<Arc<dyn Catalog>> {

There are many problems with this approach:

1. Difficult to read and maintain.
2. It mixes parsing and generation of catalog, engine runner.

We should separate the process of parsing and engine/catalog creation so that it would easier to test the parsing code.

Describe the solution you'd like

We should add some struct like following:

#[derive(Serialize, Deserialize)]
pub struct ScheduleDef {
   engines: HashMap<String, EngineDef>,
   steps: Vec<StepDef>
}

#[derive(Serialize, Deserialize)]
pub struct StepDef {
  engine_name: String,
  slt_file: String
}

#[derive(Serialize, Deserialize)]
pub enum EngineDef {
   Dafafusion { 
     ....
   }
}

#[derive(Serialize, Deserialize)]
pub struct DatafusionCatalogDef {
   ....  
}

And then we could parse them using generated code.

Willingness to contribute

None