Set DataFusion runtime configurations through SQL interface

[2010YOUY01]

Is your feature request related to a problem or challenge?

Now we're able to set session configurations through SQL interface:

• Doc about config: https://datafusion.apache.org/user-guide/configs.html
• Example to set it through executing an SQL query:

  set datafusion.execution.batch_size = 1024;
  SELECT * FROM tbl ORDER BY c1;

However, runtime configurations like memory limit can't be set like this, it has to be set through Rust APIs during setup like the following example:

datafusion/datafusion/core/tests/memory_limit/mod.rs

Lines 455 to 461 in 28451b5

	let builder = RuntimeEnvBuilder::new()
	.with_memory_pool(Arc::new(FairSpillPool::new(60 * 1024 * 1024)));
	let runtime = builder.build_arc().unwrap();
	let config = SessionConfig::new().with_sort_spill_reservation_bytes(40 * 1024 * 1024);
	let ctx = SessionContext::new_with_config_rt(config, runtime);

Although it's called RuntimeConfiguration, it's still some static parameters to be configured, instead of a long-running daemon. So I think it make sense to be able to change it from query to query, and also provide a easier to use API.
Also, they don't have a user manual like SessionConfig.

Describe the solution you'd like

1. Add documents for runtime configurations like memory limit similar to SessionConfig to the user-guide.
2. Make it configurable through SQL interface.

Describe alternatives you've considered

No response

Additional context

No response

[kumarlokesh]

take

[kumarlokesh]

@2010YOUY01 made an attempt to address this requirement here: #15594. Please have a look.