Support "dry run" mode

[AstroProfundis]

Feature Request

(This was an old and long-forgotten issue from the prototype version of tiup-cluster (tiops was its name) in our private repo, I copy it here in public so that someone might be interested to give some advises and / or work on it.)

Add support for --dry-run argument that only do basic checks and list effected nodes but not actually perform the operation.

• Support --dry-run global argument
• Perform basic checks with dry run mode
• List effected nodes and test SSH connections / credentials to them
• List rendered commands to be run on each host but not perform actual operations in dry run mode

When a user manages a large cluster, it could be confusing for them to estimate the amount of servers being effected or how many operations are being processed. By using dry run mode, users could get a list of servers that would be affected and major operations that would be processed, and get a chance to review their change plans.

This could also help users to find configuration errors before actually affecting the service.

The dry run mode should be a global parameter of tiup-cluster/tiup-dm/etc. commands, and users use it the same way as normal operations.

As of now (after v1.4.0 has been released), it might be possible to reuse some of the infrastructures from replay subcommand (the "checkpoint" design) to implement dry run mode.

[lucklove]

Maybe we can integrate this with checkpoint.

[9547]

Let me have a try.

[9547]

/assign

[lucklove]

@9547 It's a hard task, just ask any question here if you meet any problem.

[9547]

@AstroProfundis I have a few questions like to know, please give advice or comments

Perform basic checks with dry run mode

I'm wondering whether this is necessary, except exec, user can only use deploy, start/stop, scale in/out, not quite clear what we need to check for these commands.

List rendered commands to be run on each host but not perform actual operations in dry run mode

We can split this task into three:

1. For subcommands start/stop/enable/disable: maybe we can output the rendered commands with each results (enabled/disabled...);
2. For subcommands deploy/scale-in/scale-out: maybe we can output the cluster/dm's topology with each node's disk layout;
3. For subcommands clean/destroy: maybe we can output the rendered commands with each results (disk be removed, component be offline, etc)

[AstroProfundis]

For the "basic checks" I mean to let user know if the command itself is valid and correct, so all argument checks should be applied. And if there are anything we validated before changing things in the real operation, they should also be performed, examples are cluster existence check, conflict check for some commands and topology file validations.

I don't have any detailed plan on how to implement this, so I think your proposed tasks are good to start. I'd also suggest to first add a global argument, but not implement it for all sub command at once. We could do it one by one for each sub commands.

And also feel free to split more detailed (sub) tasks when needed, I'm worrying only 3 tasks would still be too large and hard to finish for each of them.