From b3f0474efaddab3236f0d58dcc4d77b214783c5f Mon Sep 17 00:00:00 2001
From: toutdesuite <guizhiluo2014@163.com>
Date: Mon, 20 Jul 2020 21:01:01 +0800
Subject: [PATCH] cherry pick #3332 to release-4.0

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
---
 media/statistics-1.png | Bin 0 -> 6361 bytes
 statistics.md          |  22 +++++++++++++++++++---
 2 files changed, 19 insertions(+), 3 deletions(-)
 create mode 100644 media/statistics-1.png

diff --git a/media/statistics-1.png b/media/statistics-1.png
new file mode 100644
index 0000000000000000000000000000000000000000..3bf886a5af3a9d4c1652fc12df80ee3132bed06c
GIT binary patch
literal 6361
zcmdT}cT`hZw~qxfBPyRV6p>;>MtUbiff>sn0ztYEiWozvNq__dDT)OV#gVEIKt%`;
zijV{(GyxImB_xzk#SjC83?-CM@&)I;wdQ;4y+7YtGk@%L*4cai_P*!-?%wC#bK|Z;
z%nt06-3I^w4wzrQXbS-Bh5!J&C_e}l0LD;iwQ$&r_|@uH0HExd*run5aDEJS+13gG
z2tN$~+_?_`@P(>7-v9ut7634F9RL8o005-XIEu|Vp<(w83$u#=LF}`Rdg1Ix^kwH@
zq4n^N>_TMs3G)Gf7;~#jz%M^20##+YccC%>04ZDZi@)068s?0KUb9EkeF$8gpC@()
z8x9ra6Q(Ihf4`qPkH!3yBV+$&Fp+%pp3iBzce!t3tqm+Lu2)7<MD38D9{o|w4Q}nL
z28hJb6F>g|`jET|c@572c7IKKXV|^E5nH3nstG=*tC)wITC!BSbjmUJ@z^fDzKHVS
z{g6{KxXW6g=hx9E8YV8rG{o(_C-I}`-@{-8QNFi_J#dxb6tuku)J#!_%9#rU84Gxz
zU>7dLK(hKw;b$G+7p_^Vp9{<{jf<b^QwKeF{a+XI&Le@^aLBFNs*WkATqnI574#C0
z=BF&T2R`Ow<~zEK7w4KfK4j^J?;TZl6UBR&*URBIjHH(8Tw>>%hB~`4ri>e}q;X=X
zr8yqUwMqI#V^$m8B~4FMP@he4PU&J|mQcm;eQPVZ**Wp7sMHL}Z;cOHJPhmJgs#}0
z4`~j%f-dzLgp;4uoSr6ZF4T1qP6`}r!{LJ1qHO#2;>Qt4WJ+{GhUAcLWy`28gL}2N
z{(c5DZF3*f9vwLTiba*foo1@SwDG{uK3%%;+z|V-qH>2=8ggvEfbx3LP_+y}g_5!*
z7H-XUaTfX!lA9JCoB>end7~jp@1p9)GmWMI&5#Qo8nxT6QO(cJk#7jx*(+v1!=dBR
z#NG0P{dL@~mJy;`Q(*~V!sEtjK0-Cwj=#s@)^gPYaoZ=pUP~6!OIg8fOj5+S0(&Lk
zAhO(T+e>6vkygh|5CSuWz+xfLNhDa-5I#AFY<LND$+SNb!0(u$Rn+VeL5JT&+m7ES
zaZU8yGQ$LqId1EJygFnKPi^=SV!CQB1v1tFH5A|`_9{30?d7c>XYM1nTQZVJYODH}
zv~=hZ2>f1B2^micT>Tc(V>I>$Bm98ua*W;3gwT}@)v*=51+FV_4d4^jf$hTONspbC
z-)MJe>3*7$R(jT9n(u~=NkT&M8aQHd_qHy4s-0bpV(a!RfEvsV@h`0sz`#}N5;2uF
zJ|k0#8YsltDGDtt{-OCYxW;HyAa07qL^&00I_V14<1-$nN9|U=V7c8EpK1mh%$4!$
z*!rrp$nm!cPt9#CR!6|p`i9&@ctrWVBnV`zp5RzZ8L%Pea*TKn5XHZh;z*4Jmgqim
z$gg$1-W`=cTikVcx%71X!FQ@~M2Q0sw6JUZQ-ng_UdesfY4YA~4j+)I5QtVeio+Be
z`_rGx-_<m#$Eek<yluL0uB2gS&Bm~9>KTZO+Y^;s9v-bCfx~n^@=s>2u{R3L;Roj3
zgSL?0DY474%e*HV+@Z3XeK#R62-BZu9zoW_EeC<6K_u_5Ce)&!-)g?el=`rlyikwt
zq@=}&x;%cE`P<JawA<HP-_-6fqlhma%a(Rgcf<xZ1dz)dF<#fU_9(T72ulK7Ur=Ma
zlo}-f<-DosR+)-*AB)npsO(Ta3u1pIX&TH_b1UG5XIp}j+r}!{`DK_RxQUPEQoVos
z+WBjJzeVxx7i<BQK(S{~Av1%4*A#GVdzB*zsM}MWrV4=vt+za{435x5c9>W$j^Ykq
zs)TiU;-K^5QVYCC=sv+1+lzSug3tcLPoJM2>`!*-5Bbi|Y%?y=3+X;IG{8Pn4(UvH
z*qq#}oMz-~Db=ndXiW(0dg^EBXbe;au`|&*iwk{XFH7s(b*76!LJG@WN*qmY*sKpM
zIXGs~FpaZzpp+jWd`IgH)Ajn_)XmS0&Aix`{T}qj?)ttzFSi9~BDg_FfF6G6c9n77
z{f8_x5p(DtMu(7!O=#Si^-C}+xxHXLH(vN4)#6sL1xsH&Sv?d|m>bt9H|x6+Qv9gI
zb~zwp+$kf&MDWsk=k|mvk7#T!(w`0goHAPP(Ym{?Kn`L$S$(Iv)+v8cneQAgJr^s1
zb7d_zww#9Z%d47A#$-k+GV$WLiLW~NwBMvuruIg+5UIk%C(ObczXvRgXf%>xTq!83
zJFX<QvUmfioQd0KE8MykytbQd()yX{)RN9ZBJt*-?74}zY1Fu3WXkpknrF^E*(S@h
zYYz=$rwVDW#dSD44>=EsOuG_oQ%8#Ag4riP4ROICZ-iwQ^N$f>yC-aB$p?*U55x1_
zs#h#_8rgEOg56oQ3X6BWCQk|j)wcal;Mzom1r#dlRi=O{Z!;NLUM5#p!KHROijng+
z;;C(cX7qK%mVRA5PGk{gXs4Yi;y3g|h7AU|0UK*=^T`sCN8PZl*_2>LLV<#CTPh(d
z+COuc@ydSPoQ5L3m|v?ZYz}@%sS}#&Xh$EaV)fBi7Mhn|J+qKnuwtV7+I>EX89s@Z
zRj@XV@Nlbvbvkc%get{<Z)!%lv1@#wnp1`g&S8ony0^mZ+O_gW)*<M=NSP(CWr~Cs
zc4Hzk<sFuj2eM!v7SfcY_K%|pD9U?@9XF~06%KsN_>-lmPfMJ#;@`jz*#Y5JKg?(D
z!Rzd7)ZD#@pCTCO(#W-3vlt)iB)C$%0-yKDEZZdvU5^1)VtgqU!j~-YPxAe!RoMJc
zRwx0PFMP@3zCds{*R(A*m>`-*en0J1+=^$9H-R!BASYfrtI)=7)lAh)K{{WY6^&1g
z8!!5$y=fDC$z(c9ecmZ7Z!<E7&|Iot(u2U)$d*-f@{af`8m7Y?tGFUYB_6i*j>Re|
z+b*+a4us_vQy)iWfhn6CH@>xEE}b`2G2N$D*)rau9A#IIhz_B;^IFH9!q1}>1#^vW
zl?8unq?<-{_&Rp+AAp$sgPv3Up}$9tX!REbjrp<Oo2Y6ZpKN5`k4;X_cfP{m#qq28
z&h360mKGy+>J2HyTakf%uu|7@uPgbJGgBqd3JE03b;#`5&>22LN!{ng;#BP1VpzP!
zPbI;&(NN^=ve_?d;fr_5ys+HPC=Jw*^A@b6!)355_o<R8r;o&{sM7I!nr(5ky~ur{
zO|PC-(lI>NpBov00UPl+f)6!0fuse<YuN$t=wX)esxq>^!~!R^mqIH0Dyw1TcX+m~
zh-RV@pqArOr%H{e!@}tXp=m1%z3i`;{D$`;0UHEPrbfPf)2n7<#^_cY0~Oeo;@n<c
zO)9e=*p}I|@?J-^FvsF(HDgc-B5z31@k9B2hZ{dR!eFikhSHuNYQ552`^8X=iAP2U
z=cy7CLxkWFu4{S)GgXynTPd8y9Po0=3$Vt*9fzfM(5;y_kH<)u|M`7(O$1FbAX|KJ
zXQ8ov_Zp_wLur}O>oWVZesm$~ATSHjTbN#vsT=hbeslQUjV{0r+BpdE_{VzK#a>wk
z&}4kRed9{x!0VR{u-~Um6pkGEmsZ}ly&tQhvP3WV#oMkT*7kBqz8!5@qn)edHzySb
zz(@Cs3Y?IIK2f6gobspX2CEC=GTGB|+21%z^9~ydHD9*!Rv5GIpYN*M%7j`7PJQN*
zxhZY3KGlkhtlo6EYfmel<J*Eaqwb4lj4Hy9a=4QlQIx~^yFr<9ydlNM2Ptl&3K9~`
zm3kF2bihp;8btES<oraL#EP?w;RmBb8j{0jJ5$4WO$Y<QH9wc^9|<f2MUI@fLUquu
z?RTYkDV~#b8P1g3l&HGf=<&g@`I!li`o~YpA}!n3Kfd#~ymwEQm<a9~*6KkIYIiXt
zh)OF%?nivD|Ki_4w@8OF{#6lW9;;B5kbzdLE4tm|h9xzhB~XCpdeX5syB<G1DeG1y
zF1{aExuAX_ks{w8?9|b9+l9Dzf_3RZwPv~Eng-4u;bFOaAQxgNTT8NYN;9$9hDv@Y
zIL5dZ8NhCqrzJZ0KI>(6QS7dFhKOU&HKs8lZ?-8e_sQE)H7y@3rH}T%2vP2KdH<^S
zKnqBB;D!rT3(~pIQUm3;THcu&@9;%#>H-$m@(B{wUGWtg#oEq6yzCRRuUJh^#J4x`
z=q|L0--Eb^V0URXRSkg~e>6$YJ43=_3CA-_uj%^wDP*YWx!5D+TcX)oW2*K;yn4sV
z>I8KZln#8**1v?^Sa{iZ=8~Dqc(_qh5M?qwXqC9BGF@x;R0roC)>>G?E8@GN{i~GA
zgGWxv<C6})M-gjp>RXx@1vRX^5Rp@79dy0^-YLj4aw=Y>$UU<3_(1z;tA&Gn^!f}s
zN3@&w&X}Gd`%zWD8g2x99puuXV#L%3Pn6qU$U=_zhG(suvTz@M;h1o;aXYD41@-|z
z^NQh&eKU5wovy7(adce}39fZeF@FE>HiN^&1?jSsZd(&gB_*R4#uePB{G%1-iJdc$
zXNQ<$Cz(!@xW2+@#BH5H(pjWEnd8WDUWY|yYRaC?Dl_(w<6AVo_N_22*VUZw&kCWl
zk`fn~Ax-`!2AQM0H>faR_EI<xb1;#vXu1t4R`rdm=9nWLru@>09Z67}h60OkS#E4|
ziTmi8|GBFh*36sgl~QJMIpK#KvC+4bX(Lf9zPB_xomVN21aSS@L$Ob{^9Uo@jVb)V
zSW%d1x|0Lp+{l8VCz!Ob>04^unFl7Q&5S`!dCP>_eJnll^D^)LfQiUX^E|e~C183e
z-Pq^$#x>nZZ`yu_SoFqdI#q)lo%6L@KX<_N@z|@}fk*VQ=AzfdeQmD$*DkC3`A;od
z*;zrR)w67rqz}rjNI#`-xox;6vZo6|cgvw_j^2kltn`$$`6X}~f3Hg?7QH;<F`&-B
z3d4=A{nATw8>erJpZ;_dSaVu(oUKw}RZ~dp`^0|gM6)i90Trt%NZyLD-MlV7`#P+#
zx#o4)!kN1ltPIbP`;2+vl8hY82f<r9sa^Bn&+9NN4ppggjV`O+Kho6Uxox`UD!G_G
zJp7^JbvI*#{sJC{v<X*eIcNNKO#q$`)m$}Ia$N|<S&46^*&c};bL5z)*cS#@aesE;
zR?3+!dKnDaM9>(KZN_JonPRNRu`N&qM+G}|l^iONi7eVwiAid;oq{IOWKhvHAx?s^
zDAcJ8HK+ZG-nC7iJ!U0f0w3eFB3nvLN_6Q4AyhqzYx?=+*W6sK*TIi2P-(&IGcK2%
zV)Oc|+hnFfuK7t*#{!tQrjl=;wi>mpe9>+2qToxmU@(~;MEV4g@?XiElX1OF+DaN}
z*ftnS^tKQEWKi{nOn!TgcRheg9+*<h*F0xu$;c?ORjdH}7zxsfU5Lm$XmNpw0D5bM
zqSJ-mE-=2VLzvBikxWiVpO!4#Kw+*}z4xv-Wqj3{3O-$^y{q*>8P@G=%UYAl9qMT5
zC#M;36e?)q7UMyd{G7OT1lQVRmTtRrtJS#@;nnq`qO|AmvNha&Xs)T@VW!lq%c}Ot
zNH0Gtu_C=mg3t5Helw0;|D7u^FDsAV-+mHb_b+<35{MU9Ik4u-?`WSPIo(e;#4O_F
zW2nj6q@nlQCV{fV!-b~Ck)KFzB83@SbSIiAYGXiVae;Nu7~z8q_}F6CB2xQOcm09b
zjMp#Of%1G-;gWo{wf_MAs-NS>o^F$9GIJeTXGiND-y3BF<40kc%((1WQg4K3r;OHQ
zytcX&;qGYgqG+@p_GUavZRSe*?`3!Qw1=_y*Bk)&ImzG?Cr>1>0?Gu#vs0W>th3D$
z%M9+_Md9%pdwO~<Jn)^>X4v%)J-Xady=W&UmDPi*I2VNU{`QzL^N}t`*3ePY!<d(Y
z$$I_^zjiwNLT_I5_P!O<_Jopun0{Y#(ANxQB3oy<^!dhgV=3$Y!+F)KXXgx~Pq%F^
zw3fqc)(XXjp=Zn0B3c`EW5eS)R@VH=N$nL@4~gt~lx!;Q?ku4l4SU$ez&PfrrK1>s
z98V#7MtuZx1UWYEwr1bKD_CuN&RR`x#W$vAR(OQL94;1F2sEe1r%s9eza6s*fXNa>
z$E$V9?}Z1>N|;O6FgHCha4;+gE*t<lTG}9WEs(mdw!M}iSWg$Mqo=9`0&8jAQ1Ju)
z2Ot1-)5k09{{~2eQg=?wAv+m@QC^r(&mcI!JirqHxAH{z!2FIGXc%c|t7~f=v(Yv<
zb_-<SVW4;Hztamf-WUuTtf?6q8mi&5gA$4mrGW}Uh#FY(gpp--BHMdo18#u97f~>5
z033+{gRcr*g8|xjLAB=p@cEt_489EaL3m>TdU`rbpw1<skMv*S{FjfD&rOUsK;K9U
zO!&4YbUE@D7b~9tI3^4Y2YkOOO*9f=TJ9+PQScvetn7te0BwC;eFH-s11&w3+*92`
z7m2^bvGWOs1N1dO`WENU$KFx25t`5c#q3~f4)F5vhYN3kCME!_`8}IIOgJJ!eBhxv
t9)=z};q|mYItGSXMta)1dU_t(+8$6XxBQV*Z(%fm`6bB3lJnPN{te_$2jTz#

literal 0
HcmV?d00001

diff --git a/statistics.md b/statistics.md
index 34b81cac1f78d..7d2581f8bfab4 100644
--- a/statistics.md
+++ b/statistics.md
@@ -6,10 +6,26 @@ aliases: ['/docs/stable/statistics/','/docs/v4.0/statistics/','/docs/stable/refe
 
 # Introduction to Statistics
 
-Based on the statistics, the TiDB optimizer chooses the most efficient query execution plan. The statistics collect table-level and column-level information.
+In TiDB, the statistical information you need to maintain includes the total number of rows in the table, the equal-depth histogram of columns, Count-Min Sketch, the number of `Null`s, the average length, the number of different values, etc. This document briefly introduces the histogram and Count-Min Sketch, and details the collection and maintenance of statistics.
 
-- The statistics of a table include the total number of rows and the number of updated rows.
-- The statistics of a column include the number of different values, the number of `NULL`, the histogram, the `TOPN` value that occurs most frequently in the column, and the Count-Min Sketch of the column.
+## Histogram
+
+A histogram is an approximate representation of the distribution of data. It divides the entire range of values into a series of buckets, and uses simple data to describe each bucket, such as the number of values ​​falling in the bucket. In TiDB, an equal-depth histogram is created for the specific columns of each table. The equal-depth histogram can be used to estimate the interval query.
+
+Here "equal-depth" means that the number of values ​​falling into each bucket is as equal as possible. For example, for a given set {1.6, 1.9, 1.9, 2.0, 2.4, 2.6, 2.7, 2.7, 2.8, 2.9, 3.4, 3.5}, you want to generate 4 buckets. The equal-depth histogram is as follows. It contains four buckets [1.6, 1.9], [2.0, 2.6], [2.7, 2.8], [2.9, 3.5]. The bucket depth is 3.
+
+![Equal-depth Histogram Example](/media/statistics-1.png)
+
+For details about the parameter that determines the upper limit to the number of histogram buckets, refer to [Manual Collection](#manual-collection). When the number of buckets is larger, the accuracy of the histogram is higher; however, higher accuracy is at the cost of the usage of memory resources. You can adjust this number appropriately according to the actual scenario.
+
+## Count-Min Sketch
+
+Count-Min Sketch is a hash structure. When an equivalence query contains `a = 1` or `IN` query (for example, `a in (1, 2, 3)`), TiDB uses this data structure for estimation.
+
+A hash collision might occur since Count-Min Sketch is a hash structure. In the `EXPLAIN` statement, if the estimate of the equivalent query deviates greatly from the actual value, it can be considered that a larger value and a smaller value have been hashed together. In this case, you can take one of the following ways to avoid the hash collision:
+
+- Modify the `WITH NUM TOPN` parameter. TiDB stores the high-frequency (top x) data separately, with the other data stored in Count-Min Sketch. Therefore, to prevent a larger value and a smaller value from being hashed together, you can increase the value of `WITH NUM TOPN`. In TiDB, its default value is 20. The maximum value is 1024. For more information about this parameter, see [Full Collection](#full-collection).
+- Modify two parameters `WITH NUM CMSKETCH DEPTH` and `WITH NUM CMSKETCH WIDTH`. Both affect the number of hash buckets and the collision probability. You can increase the values of the two parameters appropriately according to the actual scenario to reduce the probability of hash collision, but at the cost of higher memory usage of statistics. In TiDB, the default value of `WITH NUM CMSKETCH DEPTH` is 5, and the default value of `WITH NUM CMSKETCH WIDTH` is 2048. For more information about the two parameters, see [Full Collection](#full-collection).
 
 ## Collect statistics