[Enhancement](sql-cache) Add partition update time for hms table and use it at sql-cache.

[dutyu]

Proposed changes

Relevant pr #23391 #21873

This pr mainly has these changes:

• rename ExternalTable.getLatestUpdateTime() to ExternalTable.getUpdateTime(), because there is already a method called getUpdateTime() existed at TableIf, and the meaning is the same, better to merge these two methods to avoid ambiguity.

• rename the field ExternalTable.lastestUpdateTime to schemaUpdateTime, and the default impl of ExternalTable.getUpdateTime() is just get the value of schemaUpdateTime, bacause schemaUpdateTime is the timestamp after scheme loading of external tables.

• add a field named partitionUpdateTime at HMSExternalTable, update partitionUpdateTime when processing hms partition events, override getUpdateTime() of HMSExternalTable, return the max value between schemaUpdateTime and partitionUpdateTime. The partitionUpdateTime will be refreshed when (1. add partitions 2. delete partitions 3. alter partitions) with hms event listener enabled.

Now FE does not record the update time of hms tbl's partitons, so the sql cache may be hit even the hive table's partitions have changed. This pr add a field to record the partition update time, and use it when enable sql-cache.
The cache will be missed if any partition has changed at hive side.

Use System.currentTimeMillis() but not the event time of hms event because we would better keep the same measurement with the schemaUpdateTime of external table. Add this value to ExternalObjectLog and let slave FEs replay it because it is better to keep the same value with all FEs, so the sql-cache can be hit by the querys through different FEs.

I have test with following steps:

1. Enable hms event listener, enable sql cache and query profile, submit a query with hms catalog two times (lh_test_p is a hive partition-table):

select count(0) from hive_safe_lycc.test.lh_test_p;

The second time will hit the sql cache:

The last update time of this hive table:

2. Add a partition as hive side by :

alter table lh_test_p add partition (pday='20230920');

Wait some time for processing hms events, execute select count(0) from hive_safe_lycc.test.lh_test_p; again and the cache will be missed:

And the last update time has changed too:

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

[dutyu]

run buildall

[dutyu]

run buildall

[doris-robot]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.87 seconds
stream load tsv: 610 seconds loaded 74807831229 Bytes, about 116 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162251255 Bytes

[dutyu]

run buildall

[doris-robot]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 51.36 seconds
stream load tsv: 615 seconds loaded 74807831229 Bytes, about 116 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162179919 Bytes

[dutyu]

run buildall

[doris-robot]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.31 seconds
stream load tsv: 603 seconds loaded 74807831229 Bytes, about 118 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162429709 Bytes

[dutyu]

run buildall

[doris-robot]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.41 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17161868593 Bytes

[morningman]

LGTM

[github-actions]

PR approved by at least one committer and no changes requested.

[github-actions]

PR approved by anyone and no changes requested.

[xinyiZzz]

LGTM

[xinyiZzz]

run buildall

[xinyiZzz]

run buildall

[dutyu]

run buildall

[dutyu]

run buildall

[doris-robot]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45 seconds
stream load tsv: 560 seconds loaded 74807831229 Bytes, about 127 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17162326076 Bytes

[doris-robot]

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.12 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162391773 Bytes

[xinyiZzz]

LGTM

[github-actions]

PR approved by at least one committer and no changes requested.

[morningman]

LGTM