Skip to content

Conversation

@Hastyshell
Copy link
Collaborator

Proposed changes

  1. Use column idx of ref block instead of new block to indicate the ref column.
  2. Rename some variables to clarify their meanings.
  3. Clarify some log msg.
  4. Add a minimal case to verify the change.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 5, 2024
@Hastyshell Hastyshell force-pushed the fix-column-nullable-check branch from 07e9ec9 to aa72d72 Compare August 5, 2024 11:27
@Hastyshell
Copy link
Collaborator Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@doris-robot
Copy link

TPC-H: Total hot run time: 41645 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aa72d72449f2a9cf28068a045481c60d50b9a284, data reload: false

------ Round 1 ----------------------------------
q1	18752	4214	4093	4093
q2	2046	206	204	204
q3	10469	1390	1374	1374
q4	10184	819	931	819
q5	7635	2938	2962	2938
q6	222	140	137	137
q7	1071	611	603	603
q8	9448	1734	1965	1734
q9	8598	6612	6603	6603
q10	8705	3825	3850	3825
q11	432	255	262	255
q12	427	233	237	233
q13	17752	2918	2976	2918
q14	275	252	251	251
q15	537	492	495	492
q16	496	406	397	397
q17	968	935	926	926
q18	8063	7264	7268	7264
q19	1378	1220	1216	1216
q20	582	319	349	319
q21	5273	4757	4757	4757
q22	358	288	287	287
Total cold run time: 113671 ms
Total hot run time: 41645 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4044	4136	4031	4031
q2	332	224	223	223
q3	2991	3019	3002	3002
q4	1890	1873	1903	1873
q5	5267	5236	5264	5236
q6	217	135	134	134
q7	2065	1680	1724	1680
q8	3182	3275	3252	3252
q9	8332	8284	8306	8284
q10	3776	3842	3836	3836
q11	554	474	466	466
q12	721	578	544	544
q13	16667	2944	2958	2944
q14	295	266	263	263
q15	518	484	491	484
q16	445	409	406	406
q17	1740	1700	1703	1700
q18	7880	7398	7216	7216
q19	1675	1674	1682	1674
q20	1981	1764	1754	1754
q21	5428	5213	5231	5213
q22	523	481	475	475
Total cold run time: 70523 ms
Total hot run time: 54690 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168525 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aa72d72449f2a9cf28068a045481c60d50b9a284, data reload: false

query1	915	372	373	372
query2	6480	1759	1732	1732
query3	6680	215	231	215
query4	20082	17454	17484	17454
query5	4302	505	520	505
query6	289	165	170	165
query7	4613	317	283	283
query8	254	198	197	197
query9	8539	2385	2372	2372
query10	440	281	266	266
query11	10649	10076	10118	10076
query12	144	89	88	88
query13	1636	400	377	377
query14	9942	6248	6128	6128
query15	205	166	166	166
query16	7335	471	417	417
query17	966	597	566	566
query18	1881	288	281	281
query19	206	150	150	150
query20	97	85	87	85
query21	214	98	101	98
query22	4010	4003	3947	3947
query23	33824	33077	33049	33049
query24	10386	3108	3046	3046
query25	713	405	411	405
query26	1769	159	153	153
query27	3009	272	282	272
query28	6997	1999	1987	1987
query29	1321	436	430	430
query30	294	153	153	153
query31	946	746	777	746
query32	102	57	62	57
query33	717	316	323	316
query34	916	475	487	475
query35	866	751	748	748
query36	991	875	858	858
query37	293	81	79	79
query38	2925	2762	2776	2762
query39	854	806	816	806
query40	291	116	115	115
query41	50	47	47	47
query42	125	103	103	103
query43	486	459	423	423
query44	1188	735	733	733
query45	209	181	181	181
query46	1090	804	779	779
query47	1828	1706	1722	1706
query48	361	288	291	288
query49	1203	443	439	439
query50	911	438	428	428
query51	6778	6671	6661	6661
query52	105	95	93	93
query53	258	181	182	181
query54	638	466	462	462
query55	83	78	86	78
query56	286	265	271	265
query57	1140	1045	1029	1029
query58	288	281	283	281
query59	2614	2500	2375	2375
query60	326	274	305	274
query61	116	117	117	117
query62	925	800	677	677
query63	219	184	194	184
query64	5880	1925	1894	1894
query65	3170	3118	3095	3095
query66	1427	332	331	331
query67	15540	14761	14785	14761
query68	7022	559	580	559
query69	734	391	336	336
query70	1130	1080	1086	1080
query71	530	281	283	281
query72	8011	2675	2516	2516
query73	962	324	330	324
query74	6057	5718	5657	5657
query75	4419	2708	2674	2674
query76	5613	1272	1293	1272
query77	751	314	305	305
query78	9538	8957	9005	8957
query79	3808	532	525	525
query80	956	513	513	513
query81	556	230	223	223
query82	767	131	130	130
query83	331	176	178	176
query84	277	84	79	79
query85	1450	321	304	304
query86	410	305	296	296
query87	3317	3100	3139	3100
query88	3782	2374	2422	2374
query89	449	287	299	287
query90	2073	190	189	189
query91	129	100	100	100
query92	62	51	53	51
query93	4671	614	608	608
query94	959	297	293	293
query95	384	266	260	260
query96	615	282	273	273
query97	3218	3022	3058	3022
query98	215	195	199	195
query99	1590	1284	1294	1284
Total cold run time: 279123 ms
Total hot run time: 168525 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aa72d72449f2a9cf28068a045481c60d50b9a284, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.06	0.07
query5	0.47	0.49	0.48
query6	1.13	0.72	0.71
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.57	0.50	0.51
query10	0.57	0.56	0.56
query11	0.15	0.12	0.12
query12	0.14	0.12	0.12
query13	0.63	0.61	0.60
query14	0.77	0.79	0.81
query15	0.90	0.87	0.86
query16	0.35	0.36	0.36
query17	0.99	0.99	0.99
query18	0.22	0.22	0.21
query19	1.87	1.75	1.79
query20	0.01	0.02	0.01
query21	15.40	0.75	0.66
query22	4.24	7.15	1.06
query23	17.85	1.44	1.42
query24	2.27	0.22	0.22
query25	0.18	0.08	0.08
query26	0.32	0.22	0.22
query27	0.45	0.23	0.23
query28	13.19	0.99	0.98
query29	12.54	3.30	3.27
query30	0.26	0.06	0.06
query31	2.86	0.41	0.40
query32	3.22	0.49	0.50
query33	2.96	2.95	3.00
query34	15.44	4.28	4.21
query35	4.31	4.30	4.27
query36	0.67	0.50	0.49
query37	0.19	0.16	0.16
query38	0.17	0.15	0.14
query39	0.04	0.03	0.04
query40	0.16	0.14	0.13
query41	0.10	0.04	0.05
query42	0.05	0.05	0.05
query43	0.05	0.03	0.04
Total cold run time: 107.78 s
Total hot run time: 29.89 s

@Hastyshell Hastyshell force-pushed the fix-column-nullable-check branch from aa72d72 to a2baaf3 Compare August 5, 2024 17:06
@Hastyshell
Copy link
Collaborator Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@doris-robot
Copy link

TPC-H: Total hot run time: 41864 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2baaf31740d2eeb0f39b671370e84cc05c4228f, data reload: false

------ Round 1 ----------------------------------
q1	19016	4240	4178	4178
q2	2922	209	226	209
q3	11459	1290	1383	1290
q4	11177	843	963	843
q5	8099	3027	2984	2984
q6	220	137	138	137
q7	1065	613	627	613
q8	9458	1806	1950	1806
q9	8515	6626	6651	6626
q10	8720	3863	3872	3863
q11	429	251	257	251
q12	418	233	234	233
q13	17757	2944	2967	2944
q14	275	242	244	242
q15	530	491	482	482
q16	490	387	389	387
q17	986	920	922	920
q18	8199	7318	7246	7246
q19	1377	1225	1209	1209
q20	580	327	332	327
q21	5358	4791	4812	4791
q22	350	286	283	283
Total cold run time: 117400 ms
Total hot run time: 41864 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4133	4084	4038	4038
q2	333	246	216	216
q3	3006	3009	3001	3001
q4	1874	1875	1868	1868
q5	5286	5274	5232	5232
q6	217	133	135	133
q7	2092	1707	1723	1707
q8	3207	3296	3222	3222
q9	8319	8275	8266	8266
q10	3773	3853	3884	3853
q11	563	448	459	448
q12	739	560	557	557
q13	14439	2995	2983	2983
q14	293	249	250	249
q15	527	484	481	481
q16	455	401	415	401
q17	1751	1720	1712	1712
q18	7671	7376	7241	7241
q19	1679	1672	1670	1670
q20	2005	1761	1761	1761
q21	5348	5174	5373	5174
q22	525	464	465	464
Total cold run time: 68235 ms
Total hot run time: 54677 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 168798 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2baaf31740d2eeb0f39b671370e84cc05c4228f, data reload: false

query1	923	398	365	365
query2	6498	1752	1731	1731
query3	6690	221	225	221
query4	19842	17384	17277	17277
query5	4287	526	521	521
query6	287	169	168	168
query7	4608	311	294	294
query8	257	205	198	198
query9	8517	2376	2353	2353
query10	440	274	269	269
query11	10643	10035	10046	10035
query12	136	90	88	88
query13	1653	380	379	379
query14	9064	6909	6934	6909
query15	237	165	168	165
query16	7074	485	466	466
query17	957	574	558	558
query18	1917	291	292	291
query19	206	148	146	146
query20	98	85	86	85
query21	217	102	101	101
query22	4119	4151	4005	4005
query23	33714	32903	33119	32903
query24	10406	3057	3014	3014
query25	712	402	414	402
query26	1858	162	155	155
query27	2951	281	278	278
query28	6990	1991	1984	1984
query29	1355	438	433	433
query30	293	153	151	151
query31	923	757	760	757
query32	109	56	59	56
query33	709	312	326	312
query34	909	487	483	483
query35	861	823	737	737
query36	987	867	884	867
query37	298	81	77	77
query38	2876	2806	2776	2776
query39	873	798	839	798
query40	285	118	113	113
query41	51	45	46	45
query42	122	102	105	102
query43	481	429	427	427
query44	1199	733	730	730
query45	203	178	177	177
query46	1074	813	771	771
query47	1815	1725	1751	1725
query48	362	295	292	292
query49	1185	439	430	430
query50	900	432	446	432
query51	6808	6976	6606	6606
query52	103	97	88	88
query53	251	178	176	176
query54	634	451	449	449
query55	77	75	74	74
query56	268	266	263	263
query57	1169	1034	1058	1034
query58	284	283	276	276
query59	2599	2583	2357	2357
query60	307	278	279	278
query61	102	98	95	95
query62	910	673	673	673
query63	214	179	180	179
query64	5927	1912	1877	1877
query65	3173	3106	3111	3106
query66	1441	376	352	352
query67	15344	14789	14812	14789
query68	4495	579	588	579
query69	658	327	311	311
query70	1114	1074	1045	1045
query71	477	273	283	273
query72	7998	2688	2493	2493
query73	784	328	330	328
query74	6002	5593	5657	5593
query75	3913	2743	2750	2743
query76	2983	1165	1291	1165
query77	686	322	313	313
query78	9399	8882	8893	8882
query79	1894	533	535	533
query80	966	517	502	502
query81	557	229	230	229
query82	812	130	130	130
query83	352	174	176	174
query84	271	85	81	81
query85	1318	303	337	303
query86	420	278	296	278
query87	3311	3119	3109	3109
query88	2879	2400	2450	2400
query89	388	287	290	287
query90	1970	193	192	192
query91	128	100	106	100
query92	70	50	55	50
query93	1857	622	613	613
query94	1067	304	292	292
query95	371	265	262	262
query96	613	287	280	280
query97	3247	3038	3045	3038
query98	224	209	202	202
query99	1592	1294	1298	1294
Total cold run time: 265598 ms
Total hot run time: 168798 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.59 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a2baaf31740d2eeb0f39b671370e84cc05c4228f, data reload: false

query1	0.04	0.05	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.49	0.48
query6	1.13	0.72	0.71
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.57	0.52	0.51
query10	0.57	0.56	0.55
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.61	0.61	0.60
query14	0.76	0.82	0.80
query15	0.93	0.87	0.86
query16	0.35	0.35	0.37
query17	0.99	0.98	1.00
query18	0.22	0.21	0.22
query19	1.81	1.75	1.77
query20	0.01	0.01	0.01
query21	15.40	0.74	0.66
query22	3.82	6.72	1.75
query23	17.91	1.48	1.31
query24	2.30	0.21	0.22
query25	0.19	0.08	0.08
query26	0.31	0.21	0.22
query27	0.46	0.23	0.23
query28	13.16	1.01	0.99
query29	12.59	3.29	3.25
query30	0.25	0.06	0.05
query31	2.88	0.43	0.42
query32	3.22	0.48	0.50
query33	2.98	3.00	3.00
query34	15.49	4.32	4.27
query35	4.30	4.29	4.33
query36	0.69	0.49	0.48
query37	0.19	0.16	0.17
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.16	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 107.56 s
Total hot run time: 30.59 s

@Hastyshell Hastyshell marked this pull request as ready for review August 5, 2024 23:44
@Hastyshell Hastyshell removed the doing label Aug 6, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 6, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Aug 6, 2024

PR approved by anyone and no changes requested.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit d935829 into apache:master Aug 7, 2024
Hastyshell added a commit to Hastyshell/doris that referenced this pull request Aug 8, 2024
…pache#38894)

1. Use column idx of ref block instead of new block to indicate the ref
column.
2. Rename some variables to clarify their meanings.
3. Clarify some log msg.
4. Add a minimal case to verify the change.
dataroaring pushed a commit that referenced this pull request Aug 8, 2024
…38894) (#39107)

## Proposed changes

1. Use column idx of ref block instead of new block to indicate the ref
column.
2. Rename some variables to clarify their meanings.
3. Clarify some log msg.
4. Add a minimal case to verify the change.
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
…38894)

## Proposed changes

1. Use column idx of ref block instead of new block to indicate the ref
column.
2. Rename some variables to clarify their meanings.
3. Clarify some log msg.
4. Add a minimal case to verify the change.
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
…pache#38894)

## Proposed changes

1. Use column idx of ref block instead of new block to indicate the ref
column.
2. Rename some variables to clarify their meanings.
3. Clarify some log msg.
4. Add a minimal case to verify the change.
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
…38894)

## Proposed changes

1. Use column idx of ref block instead of new block to indicate the ref
column.
2. Rename some variables to clarify their meanings.
3. Clarify some log msg.
4. Add a minimal case to verify the change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants