Web console: Display compaction status#10438
Conversation
| setInternalValue({ | ||
| value, | ||
| stringified: stringifyJson(value), | ||
| }); |
There was a problem hiding this comment.
there is no logical change here... just wanted to have less nesting
| export interface MoreButtonProps { | ||
| children: React.ReactNode; | ||
| children: React.ReactNode | React.ReactNode[]; | ||
| altExtra?: React.ReactNode; |
There was a problem hiding this comment.
What do you think about adding another snapshot test that sets the new altExtra prop?
| export function formatPercent(n: number): string { | ||
| return (n * 100).toFixed(2) + '%'; | ||
| } |
There was a problem hiding this comment.
Missing unit test in general.spec.ts
| function progress(done: number, awaiting: number): number { | ||
| const d = done + awaiting; | ||
| if (!d) return 0; | ||
| return done / d; | ||
| } | ||
|
|
||
| function capitalizeFirst(str: string): string { | ||
| return str.slice(0, 1).toUpperCase() + str.slice(1).toLowerCase(); | ||
| } |
There was a problem hiding this comment.
May be worth adding unit tests for all these new utility methods
| <AsyncActionDialog | ||
| action={async () => { | ||
| const resp = await axios.post(`/druid/coordinator/v1/compaction/compact`, {}); | ||
| return resp.data; | ||
| }} | ||
| confirmButtonText="Force compaction run" | ||
| successText="Out of band compaction run has been initiated" | ||
| failText="Could not force compaction" | ||
| intent={Intent.DANGER} | ||
| onClose={() => { | ||
| this.setState({ showForceCompact: false }); | ||
| }} | ||
| > | ||
| <p>Are you sure you want to force a compaction run?</p> | ||
| <p>This functionality only exists for debugging and testing reasons.</p> | ||
| <p>If you are running it in production you are doing something wrong.</p> | ||
| </AsyncActionDialog> |
There was a problem hiding this comment.
Currently this is hard to test. The autocompaction E2E test could possibly be modified to use this instead of calling the trigger compaction API directly.
There was a problem hiding this comment.
I am 👍 to modify the e2e test
There was a problem hiding this comment.
I took a stab at modifying the autocompaction E2E test to use the new trigger compaction UI: #10469
| const compactionConfigsResp = await axios.get('/druid/coordinator/v1/config/compaction'); | ||
| const compactionConfigs = lookupBy( | ||
| compactionConfigsResp.data.compactionConfigs || [], | ||
| (c: CompactionConfig) => c.dataSource, | ||
| ); | ||
|
|
||
| const compactionStatusesResp = await axios.get('/druid/coordinator/v1/compaction/status'); | ||
| const compactionStatuses = lookupBy( | ||
| compactionStatusesResp.data.latestStatus || [], | ||
| (c: CompactionStatus) => c.dataSource, |
There was a problem hiding this comment.
This will be hard to unit test since it requires either a real or mock druid cluster. One option to make it more testable is to restructure the code to inject something like a compaction manager dependency via the DatasourcesView constructor. That way, the unit tests can have a compaction manager mock where the API responses are stubbed. #9956 used this approach to test SegmentTimeline with a mock for QueryManager.
There was a problem hiding this comment.
The testing refactor I suggested, could be fairly involved as it'd be good to add tests for the parts of the datasources view that are not being modified by this PR. Perhaps it'd be best to have a separate PR that has tests for all the parts.
|
I think this is good to go, and some of the tests that were addressed in review should be added in a subsequent PR |
|
thank you for the feedback! Will followup a PR with fancier e2e tests |
Restore the web console's ability to view a datasource's compaction configuration via the "action" menu. Refactoring done in apache#10438 introduced a regression that always caused the default compaction configuration to be shown via the "action" menu instead. Regression test is added in e2e-tests/auto-compaction.spec.ts.
Restore the web console's ability to view a datasource's compaction configuration via the "action" menu. Refactoring done in #10438 introduced a regression that always caused the default compaction configuration to be shown via the "action" menu instead. Regression test is added in e2e-tests/auto-compaction.spec.ts.
Restore the web console's ability to view a datasource's compaction configuration via the "action" menu. Refactoring done in apache#10438 introduced a regression that always caused the default compaction configuration to be shown via the "action" menu instead. Regression test is added in e2e-tests/auto-compaction.spec.ts.
) * Web console reindexing E2E test (#10453) Add an E2E test for the web console workflow of reindexing a Druid datasource to change the secondary partitioning type. The new test changes dynamic to single dim partitions since the autocompaction test already does dynamic to hashed partitions. Also, run the web console E2E tests in parallel to reduce CI time and change naming convention for test datasources to make it easier to map them to the corresponding test run. Main changes: 1) web-consolee2e-tests/reindexing.spec.ts - new E2E test 2) web-console/e2e-tests/component/load-data/data-connector/reindex.ts - new data loader connector for druid input source 3) web-console/e2e-tests/component/load-data/config/partition.ts - move partition spec definitions from compaction.ts - add new single dim partition spec definition * Fix UI datasources view edit action compaction (#10459) Restore the web console's ability to view a datasource's compaction configuration via the "action" menu. Refactoring done in #10438 introduced a regression that always caused the default compaction configuration to be shown via the "action" menu instead. Regression test is added in e2e-tests/auto-compaction.spec.ts.
* init compaction status * % compacted * final UI tweaks * extracted utils, added tests * add tests to general foramt functions
Restore the web console's ability to view a datasource's compaction configuration via the "action" menu. Refactoring done in apache#10438 introduced a regression that always caused the default compaction configuration to be shown via the "action" menu instead. Regression test is added in e2e-tests/auto-compaction.spec.ts.
3 participants
This PR adds a column to display the (new) compaction status API
This PR also:
skipOffsetFromLatestparameter...button allowing for a secret undocumented API option to run compaction immediately. This is very useful for debugging and the copy is made suitably scary to ensure users are dissuade from using this option.