From bb2570c63d4c09ff7b8ee9f1ee1d7daae6e63448 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 17 Jun 2020 17:58:06 +0800 Subject: [PATCH 1/9] Add Deployment Solutions for Multi-Data Centers in One City --- geo-redundancy-deployment.md | 72 ---------- ...-centers-in-one-city-deployment-sample.png | Bin 0 -> 72064 bytes multi-data-centers-in-one-city-deployment.md | 132 ++++++++++++++++++ 3 files changed, 132 insertions(+), 72 deletions(-) delete mode 100644 geo-redundancy-deployment.md create mode 100644 media/multi-data-centers-in-one-city-deployment-sample.png create mode 100644 multi-data-centers-in-one-city-deployment.md diff --git a/geo-redundancy-deployment.md b/geo-redundancy-deployment.md deleted file mode 100644 index 3d700e9bfcdf1..0000000000000 --- a/geo-redundancy-deployment.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: Cross-DC Deployment Solutions -category: how-to -aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/'] ---- - -# Cross-DC Deployment Solutions - -As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (hereinafter referred to as DC). This document is to introduce different deployment solutions in cross-DC environment. - -## 3-DC deployment solution - -TiDB, TiKV and PD are distributed among 3 DCs, which is the most common deployment solution with the highest availability. - -![3-DC Deployment Architecture](/media/deploy-3dc.png) - -### Advantages - -All the replicas are distributed among 3 DCs. Even if one DC is down, the other 2 DCs will initiate leader election and resume service within a reasonable amount of time (within 20s in most cases) and no data is lost. See the following diagram for more information: - -![Disaster Recovery for 3-DC Deployment](/media/deploy-3dc-dr.png) - -### Disadvantages - -The performance is greatly limited by the network latency. - -- For writes, all the data has to be replicated to at least 2 DCs. Because TiDB uses 2-phase commit for writes, the write latency is at least twice the latency of the network between two DCs. -- The read performance will also suffer if the leader is not in the same DC as the TiDB node with the read request. -- Each TiDB transaction needs to obtain TimeStamp Oracle (TSO) from the PD leader. So if TiDB and PD leader are not in the same DC, the performance of the transactions will also be impacted by the network latency because each transaction with write request will have to get TSO twice. - -### Optimizations - -If not all of the three DCs need to provide service to the applications, you can dispatch all the requests to one DC and configure the scheduling policy to migrate all the TiKV Region leader and PD leader to the same DC, as what we have done in the following test. In this way, neither obtaining TSO or reading TiKV Regions will be impacted by the network latency between DCs. If this DC is down, the PD leader and Region leader will be automatically elected in other surviving DCs, and you just need to switch the requests to the DC that are still online. - -![Read Performance Optimized 3-DC Deployment](/media/deploy-3dc-optimize.png) - -## 3-DC in 2 cities deployment solution - -This solution is similar to the previous 3-DC deployment solution and can be considered as an optimization based on the business scenario. The difference is that the distance between the 2 DCs within the same city is short and thus the latency is very low. In this case, we can dispatch the requests to the two DCs within the same city and configure the TiKV leader and PD leader to be in the 2 DCs in the same city. - -![2-DC in 2 Cities Deployment Architecture](/media/deploy-2city3dc.png) - -Compared with the 3-DC deployment, the 3-DC in 2 cities deployment has the following advantages: - -1. Better write performance. -2. Better usage of the resources because 2 DCs can provide services to the applications. -3. Even if one DC is down, the TiDB cluster will be still available and no data is lost. - -However, the disadvantage is that if the 2 DCs within the same city goes down, whose probability is higher than that of the outage of 2 DCs in 2 cities, the TiDB cluster will not be available and some of the data will be lost. - -## 2-DC + Binlog replication deployment solution - -The 2-DC + Binlog replication is similar to the MySQL Master-Slave solution. 2 complete sets of TiDB clusters (each complete set of the TiDB cluster includes TiDB, PD and TiKV) are deployed in 2 DCs, one acts as the Master and one as the Slave. Under normal circumstances, the Master DC handles all the requests and the data written to the Master DC is asynchronously written to the Slave DC via Binlog. - -![Data Replication in 2-DC in 2 Cities Deployment](/media/deploy-binlog.png) - -If the Master DC goes down, the requests can be switched to the slave cluster. Similar to MySQL, some data might be lost. But different from MySQL, this solution can ensure the high availability within the same DC: if some nodes within the DC are down, the online workloads won’t be impacted and no manual efforts are needed because the cluster will automatically re-elect leaders to provide services. - -![2-DC as a Mutual Backup Deployment](/media/deploy-backup.png) - -Some of our production users also adopt the 2-DC multi-active solution, which means: - -1. The application requests are separated and dispatched into 2 DCs. -2. Each DC has 1 cluster and each cluster has two databases: A Master database to serve part of the application requests and a Slave database to act as the backup of the other DC’s Master database. Data written into the Master database is replicated via Binlog to the Slave database in the other DC, forming a loop of backup. - -Please be noted that for the 2-DC + Binlog replication solution, data is asynchronously replicated via Binlog. If the network latency between 2 DCs is too high, the data in the Slave cluster will fall much behind of the Master cluster. If the Master cluster goes down, some data will be lost and it cannot be guaranteed the lost data is within 5 minutes. - -## Overall analysis for HA and DR - -For the 3-DC deployment solution and 3-DC in 2 cities solution, we can guarantee that the cluster will automatically recover, no human interference is needed and that the data is strongly consistent even if any one of the 3 DCs goes down. All the scheduling policies are to tune the performance, but availability is the top 1 priority instead of performance in case of an outage. - -For 2-DC + Binlog replication solution, we can guarantee that the cluster will automatically recover, no human interference is needed and that the data is strongly consistent even if any some of the nodes within the Master cluster go down. When the entire Master cluster goes down, manual efforts will be needed to switch to the Slave and some data will be lost. The amount of the lost data depends on the network latency and is decided by the network condition. diff --git a/media/multi-data-centers-in-one-city-deployment-sample.png b/media/multi-data-centers-in-one-city-deployment-sample.png new file mode 100644 index 0000000000000000000000000000000000000000..f5dd7eb8f873b02f8ccb74032e19b7f1427df282 GIT binary patch literal 72064 zcmeFZg;$kZ+b@g(C@Ct^ph$Ojh;)Orbffg57bT#SbT?AcjdTb|$D*V|q`SdIp2^;i zTlVvP@B0UwaTsF{_rm+0eO>bx*D^psP8=Nt9|Zvc0bNo;E3`pZrQyOe~D1kPq+p?+%*-F6+%EL4MjcIM+Bem+em2GAs{@ozy9ymtOV|6 zq==BRv(DxWvL3eRCGxKB!?uPwI+$K_)D*37=>q(o?M)gvIS9Ok-{Fcqj zalB4NDMhcp*B1C2>9LiJ>1JmQ7#_ude3ex8%0@@y=qrL`a_3{TLipD4#qa?t4nx$)h%&!B_Xft*#=)@znkwQ&1c})x=9VA6o=7Q_s;Fr}e?cw3!qlI4B&TNBbrR9_f=+a}QKe^1X1LJ@SOwT@?R*m7!XhD2R zrNwl$;_=R0aduBEBV#m;%FytWGl8WFSdmV{>xhVmL;Gcz@7$zgQ8@RQTKPdGX@Uj#rW^lP}=CJLpYE_58)}!4p-y=c7(er!!kHwIVHz?r2(5 z;t%7gPTj=?E8)uuPon5FUI|vJ!&bT?#qrt9o|cxDo|d*jXWw`tZ@^!EK#{ySJvypL zKB`yZHinr?M^Qcd^7cCznQOvj!`#tHWwMfok>_k~Qbh_ZnYT*I{pdS%vLD*`!s|9< zDOzp)4ey3awgYJD*DvSXS6P86mE|edgPAHa=}XY?P;=$>nfs#G>7)4!hE+g`CbbLuaY`rh1sO9jbQ9<&3NO86979x_UTsPa%11z*kqz-VCOHxNh>Q z!Coy-nL1Z7!}qTqq}25Ib+hG&s9#s)q*RWFdL}iV?GVkKtKy-Ln))Qdu4XIG8r=5- zX^X;swtd`qBP}HrNcXxzLSDDq_Gq~;(HLyk^|{QFj+KHy%2FKdroLM9U%8$YDCt5=kSBSUKw7i0|S6J7b6$iq>3CfKghlYm)d9nmpNK`!8fKjo z(zH3%|0eaK!SY12sqW$zBKUPN>dWPotsQh7K@ub8tL(Tpb1#qriIBYB`=We6jHi59 ze#D1iu^Iov8xi%eUyncZj=SspM>HhFdnv*uA7ySMe}0FC1T!>kJ&+j4K~oQYB=O$a z)2PLnINHXQ!Y3Jb4%4&nq)LsgszQ^bA!kIMBZg($pE z+775*Y$6j=afM0#;J%CDxW!9@U#DR68fGZo(Hx41F4ZYguwtv{6ZE9O(y)-X@lHyW zp62{Jj1Q3gH4&8HB1eapJusrD3x2RR}W!`~U3&eI7j3szXO^EJv%)*LEQI+`n0 z;w7Y|hvWUfY=wnvM!)D4t}Ux18g2F{vC5(~pL#Ykfr(Z!qfnx`5#od9TKh6BU~cBM zQ%N>$0m0}Dlo;9J*vFkObjNomc%@W(Yt7_%a{})T@q7hLV~)tHrA4jrs_!n&Xe%&z;+$~vgxaV5y1tNTm=nP?~??Xz*PQZ?5+DdqeqKzg@-{q{b z)v;N?lt0yF%D7*C;FDFc$WNy)J4$-;PVvL&Lyz%)(o1lvCWw53gjT#MwI(7SL zFMsBIQ& z-bKbvrA?5Y(7U-e?VK@ElU-8FvT6B3-KSG@(}F6xnfXHJhGuh&85i~@sG3+;)J7{? z4>~)xrrH~~N`k{uCr%_Yd0joc+F!UD=D1H8%~G;}k)R*fm}3+ld_Kkk~mkyzA=WyAyI$xgtKw{QiiqGLc_fe@dexWE@E%Qnj{)=ef4}(o=!C z%evP~abLDrV&^4rxEfnGcHNd8s#e7BE&3e0Q`qT*IJ8=umu3@)EH6Ait8A{-o7r5W zI1mV68G7xMNpQA@J%l|J>=2|!q&YcEG&;k>(7{*V6ur%oO`9nkW#>$k0@K2ncMs22 zpJF&D)n8s%?p;z?hAY+1f3$%r?UGr=AIx9ODrZ7|d+JvDXQ~PP*S+>!0M1 zAaR}rr;fzQkYN0Ppq??LuFx>5cM7S43vp&7lvP$jch16a!s?z5I1gZM` zp(~qbXx-fAsX`#>?rLwY-S+Y|L#>ouUNAY^Lp`R>M5Ue4n;Neo@?bqb9QR$~q2 zjIgrs%hoPld}w~Sny%TDzHWPE=_tDA))L_mVhfa5AJ$U}8xKun&89ssQg#c;MjPTli8q1_#c8$mh&Itp z##h2PKsHU;RzT8EElcU(C?mNuejp{X?1%uSL~2*tvQfAd^3{CHK?8ig-L&x`#QMRp zgWnvKcnt+U_#s8yLr)%AXrYJrAFTXXtJN1ha(kvO)ro~lg3I?091q^X?=3Lj`;FD9 z;+Rr26`(Iee?I(+BkXDWXrv0b_dfp3 zEf5Ejxb1qCnczjuo1>c6>`K4idn{r;RunDlhY>H{+$zoT&+2GqWo3cU2ZHWzk`K`E zM5biq&4!Q0bQ$;d_LdcQe-$+Y0IF{i_JOH%wENci>o?JxJ4|FeGn z^($cVvkk5{Cr{$>t62sy5vgYQQ}79$`xjctk^R3WRbsT(An;UZqgQr&HB%R ziOe2=#hq((@4KVQx%}QE5g|K(db^@&W5KK&H}sDeLIo3UoRp`WF}H3W z*X!M{fh_&mr0^3kR{+@sZ_M@ZyP9r%uHHU)7U0kulTkDYIjAxZ zH|~Oi)b&W7PoMbr=3=?@IPv#-z?X3TbhOAncf_>uPk?oMT;qBy9309Is5mhIXL8#u z2NCkvOT1E2B0pg8S~*zkdM+CHI6N*6m-VEIh|A^$0Pc*Ga!Fx1IaC+iSyItMJGQyI zGa-cBq}g)G5h&O+aWx_$hfNZ4axsP-A(9pr1&?WzB_K=PSv?4g#m)?Nc<=b8X&50Z z{g%s@l^Nop-GJI0>elOe-A2r`UmJQ=?`&KCtxZ~5Ix;hp>_UesR3YRs{p%4v>FW)! zx<2ws^8Dn*O_#MUB2gEB-7tQ9LLUL}zO)|(5t(kITkS1NaLUT;R^+s8%Y+dTqlwsF z0S+fyRQ(g>8eJBNOw2o69|de$)z1B;9%Fz^Pm7+K8PRH#WqN35hy`Z~`5@J%$H0%8 z5bM?e9kR!iQBFBM+AvLdMw)(wxW5WW*jgm6H-}6+mbQ+)$f}S4ya}50a+MM5>UFp_278Yn$$))l&yRk)mFVJAkKzk7n zVFoXSgHd+*q7xjtrr8#|!S`QPS!rMB;G79u&-}0nob-tz?YhI}r{S8j9%q&=I}In9 zfUxbipfawu7l8khEY@kTH+&89l+}Yf%Ju^fVj?2a3G!Z7T2N#1ABM46PAZ?slzj63 z0>T!)mYr$@@ecc$;-0ER$iV77tmEfFo zKbdh_)*vN4)eR!#euacVNae6T626k+Dwo>(;_~8fSiWaS`uBwlB_$o!fy|(_KpZD~5%}lxV3;{}z%IESbH3#0Kdr6wi z>onRh$}lqi0Va9}Z+4@D#B(Gyw4K=v1KJEWOeV!;8Sg@1)bs&3=5X}&_L(}TF^A2k z`!l#E7IYa&$D8G_Ze$X-Pj}GGSNDGXS_qW}0cG6s{(fz<`9zslfX}cIcKS8)rv-z?nBWsZf(C*5cs6OQw|dtZkU{PJCKd%vQ2lp zx;PnX^7Nc2HT1irOET)6ejak22Fv_F7WRkWmdAH@ky`Hqg+o(B<-L-tWkmm+K%j1;LpJ%3{AhBFT6L(#~2B zyC)_CQH!>ruIm=Nw$Mw=JEWu#`NG#4HJf*u?WVQoS>m>25MLdjxkKrqs51)39fFsNjyto3t~?pylOH}| zp&4U9c{9{lyB}zcYLps?)-1(nE_xvo4+I;tn)Z(bJ6xais0(lxFCxTtjo*n~{;^NM zrWH-CBzA(|I?nP=M;BKU4LgZ?a>%=CtZnb7SZE6cZ$$r7T+w`I_*p^|?4!5rhop@I0cg{E&VZdmr;W5~L#5AcA%tq5r$I(IwJUtI--Jvl ztAOJIV#EPvc()OC7FoDP&mWgm@Wm+xBpY*%JuYqDtFyR3j8!>FL9r!^=^k4`TEzPv zCR;TbN@sNZE+#WJE4E$NVBOU|n_*c_$&L^rhaEPw2eKj}+{kb)DAw~Pam>4mSJf2{ zoz&a;reA!`E$#IdW2ZZCZ3x;UR>@Ny?3rhOGX272Z72)MCws`(dY^Dw>m3Le%x#&V zIwH;1>>o4_7mq&TFplzn3=ieBnpR(F=1L2e-WeSDAiY0$&-a-M3#WRT+J4W4j`I2z^m}S8Fst z`Xa!;lc4j!mYNS5H=p;AElj@}`|yEqO3*W9rgk#?G8*uVrPl{bJ{S|Q zza=y7iTO@BH7H#OG)-udbiJTlh@M7%K^NK)dK9X|#_0E0$Dc^-)r;u-p|->L=c1vU zY?Gx{FYR3bkn-h~#)9LuQ+$mlP*%3dMPq2C20E(1qxarxRWvf{NQmQ&?q~KVR-~jT z7|J>}+yeN0HpoVY2x1!Vq*s|Sms&s2aJ^$eh_yE0c5R)qZeEd%XAu?n_2Z5^_c92H zVpq0L5g*!lNJ+b0s`(lvBPpPCtcSOCs%_G?HGJ!d7qm#P#9DFN8SU2bf@XWFd0C-J z`dm+b?DQm)Qo1+%m? zv;}-UeA5pK)oDZTdkhBnpo?en2Iph=I1TO>2El>nTUA!SyidQSUY(n|o4Q5O>BXQ=d;mb{CczE+5 zZP#;qUb<=P{CFjM=y=w^92u}$wKqaGDEbnHpzN!Ft5U3^jrlFKoHK__pYKkRY)kc> z95!wh@7?os(T_&pf~;*hY?}2>@b`ycKoK)=_O7KVTHzf3A4T!iXh^YbKY}h{f>#9x z&}83Qz_7O>SLDY-AI5t)!yTxeeaF=0TaL0 z=0pmu_deHf=-zFqScbw5N?bvYdSa)q7{wjG5Um@IVg!bLN>ObN5J*Y)1@rxwe1pGQ zp4)Pk;j_P8Ex%gD3q@kpk}9{DU~yl|RKhfv#zuFN5SB*>67rpWgOk_!sTY^mv0ytq zNUX;5VnY`K;FDc13u*ehuK|vWOJ;1s4G6(z$92S}lv?XeZ-yx@xT(|zGAx1TKzR9q ziU5!$v>Al}$WR=!p;2epGae*NV!A7Uq!!csh$8@{-XJ!pKI?TD)6OR7L_mB*BP}m@ zQDKGFs;ivW2wx_M2M$FeEe7=7502sLDC7m$Q;&RU4lJxCG%XfzeeVWA5cpdL>QE`A z20r3aO4tnD@uAUVLa=Z0xHt?58`E)>o#RI54;Ums_c>aK=%!3@eiC_cCQ_@yS z2qvqn85tW6Bit^I#%e8I#J`82aRv(c5Gl&FXzL9U2QUAT)juZI%a6C)whnBZ!vki0A7j%r0Lqx)6mw}>OflU3KK(l{YFvUB7k}$PKWKr@lOs}S`m7=xQD834QIlR zx@n77Bi~VR?s`mBS;q>V&tvYd=NGR-e!g7j{)9XUuJ2M^c+rzrLV*vn zYPxXb_m~7=`N;L*VAjW1yg%Ev0*ITFMQ!kIhieY>ppKnp=rxZUt`Y~g+-*Rx;Ustr zaaV^9jQ+}rq)SZSzFpXt&{lC}M0OO}#A552{CHg7X=Z@;|oJ0dOmL#;M zlUdgz$p?3>E~D{u4gj)Vx23TiItO?;7Vy5cy*8V&yKq7LQX9IRyT%Wolna^nxz!fbRvKnxF zO{;vllE!aM>mem0Fxex>?S2|#vwk9Mzvkv7i%ZZ$Mfk}^Sm<>P#MoV&@Q}&S5D!by z=tUbM_Txhdu}^+11U>%#-sl%H1J1tE%eK#P?rx`q66>2O2+Ol*wL!MNjIB$b6wWqP z*Jj$S^g|zbO!p!#=%-h)&Z45*;bwYHR=lY_!D}!2wshtktYjFE*pRS?KW&2+>-Dq2 zHl3IgFA?wrkLfIFoG~l57xtxL%eDvgp90~6Z8DGk)&N8c!k>$;w`a%DYf1qr#9@b- zEku9(=^+}g^X{ht;1qI3&F_n|X{BMIh1%kg^So2eSEXMNxnIO4w15Xqncu&z&hAoxKf#g#@?VcAJrF6;XFWb@Xxq7x;HiIV>kTNv?j0^EWQnatu2Qp!dZw=@0UE8=1a&UGzwK490U7_o8f zMSy+PUzzL!Y7D@+4|OjJ5ySaJ)T} zPq4;>h5(J#UTwcEqSG2oXY&XpXa!%S(VS`Hn&=%*m@fZEg9+y>s3Xs)ig7S?RgPz! z6UE!Dcdz-V-++S;Lnsy%p2*VTdE?2rQFmTE=jYGR>x9FW;Y#!OyWwL17M5`G=;QNO zvxqM*G58+otcLr3B~OkWQb#&IX*tvCK6#dox^VB^HEH_mhXV}3e5$fA{tuW8hQu}a{{ADfXV02}TBs8t`Fk$^uFC*v zKsJ%1Pv$@Cilz8~{mqcb&xHjb+FeURKbn1Rwg_k~Pyhq|D82tKQn;28L;3doZ>-bn zdgs!36bjG}P&K-4^h2*+^5WschrJm;=KmLcccR*M;g7rZz`rW@a3>h)A<)rUaJu6TahdLlX?`f*QM(?svEDT${Al@9tjXwAlL~ z2w3f=Icd zWNMvuh{wIIo|iRS0FmXiU~nK!xQ~!DeRmh+SqUKBEjFHQv)Zj19Bxi1)!MJ6R}o<; za(%u3b8R^LT3G5MtW~czzXBU=4A9VDa{;_pKE?CWWo;xkGJc`)6K9+)+KpPC>y?Ku z#okyXTyFL3o3{@LmV@(01=cM87wP=fxVFimr|BYJc~(4osRQ7;@Uu(&$QE=JIJ9yz@PvG zQTwp6PAig9K1TH8ldfE)>|R2YhnwYN+D^&J1?q#n&rm*7D}C-fTpKn%Tphely|xSO zUBA;+NFC^DF(Xu9hzR-GYSKa}yLJe`H32LX8Ze+lZoALD$-J>0l(#P?%wK0PpjGnv!lEpy_U)_kOLT@SeEbr!0caZHRC7GeQ5C$(;4 z{k1j+T6G2>lZ=8w3{X}hFwG!b=9kxGI2VZW@e2$baoP!+7EHp3w4e~|^_REC;1Rd8 ztttq(xFsSfWP1tI^$dI_vJPthLIwM~t(Y6qvA{7gU49kj4g_ilcz7dn$HPl(0!TOA z>|~#A2k4y?0@gE&Td|_MSBn)i^sXMw*KcILipU! z!<@sNxw+mhJjTXf>DN|m2f_C0kp1SkVv5Jv1FJzNxZTG=F8qfWhY*qQaZ&A;i&F%oCa{X4{wS3C~A3^_dy=794I79-1UMk*_W;$FD zU{9Jj;G|MkoKNZ<&p6GkG3!W!vEfZVjv-JR~Vc*Xaiwaf5B35o2U9t~&(1oj=Er$`v!j)})R$ikqxJV|?^!T$1q=a&q?^HrZX;`Fsn0D1;i;v>t;({|wOSlkFi?`iexd-0#j)^5rF zcAB-FW{hvV0PYgwzg<>b=lFlJ0IJXBLOX>}+y9W4CTN_trz1G6b^hb{+AYicNu5F? zzzw^uWV<$2&6x<;%))LYgFLn36oAnFH;Bl8I3WBdLwK8A*Kg9qfqQcFi@Dm3eku&w zP$VTJnEq7^@Gpp1rlO^l^q5NF>)!}2m_%^)pVX)LK);iDh2q+FFA-pENacKky;z79j*I^M%$PZOMJ=w8dUJSy~byEZe4g<(ARp1Z} zl^Tk>oUUbOd0d?S?+NG*GDZ}Lz#hX;eiP8>G<2hIC|`~7%|uyN`~oKME{6=?UE?ql z48$&AOfvuvG7>QAiC){eC~766)5A3u=%E=vjiUgu8twjQW-;n~z}lfbUPa>k<5oP6 zL*8RXo$20&5?}&IK=$uT%AcsPGZcu5ic6;OxsWwl)QgB;d89@PxSvV@_2Jro1bK!) zz^!_F>)@|o(jcdxSm&~%nCX4S_^oXzCW2gAYPQ~`FR1`zE!t!rhaubC|D?5Z;|B3e z{r{g30fAHFrji>$0TT!3DkAdfwRpCB3k`? zR?pFNjyJ27WKwD>q_CF|qS>b4AGdediTXHa6v~L;}^{vkY?+#iFkt>GuRYwZD zT_>ui`=O4@YO`ym^h&>poT>#=4_+fd>n1?MZ2v)wMt+wVH46sh){XUOp``LDT$1wg zvE}PcHog_uk0PZD?Sy>&^exY99v?EV1Clyg>tO!-gvO^q`gU1{zKeDEVp?9=d{$K( z`YwpJLz)!%9h4F~Fiw8jpCOOe*&Qa2Pvo$SC^c+~enc73N4S5x_Epu}@2?WrY9@QT zXsBY1Jb=@;(G5EcK{Ip5raaf}k|p;u@Gj z*i-uh36Os(e(oQE;} zV~(8QfSC3th3jxuhCBJyA0#$(jJML^)^F<{cYErDkK{z|4=kKqx8Z8NMG3pw+8L^K zAYi4;e|o}n23#8B{$lB+wz1jkg9f|!l4@@;_X;}aQ5PSH?HY$D{Qw&0ustX@=~QE* zBQ{}v($7D171Gps$h4!+vAX4C)G?c}pG`T@+AHn3=9GB2gOkGF8xa{v4oV~b0#bex zvAP~_K9eH9rn4ROp4Wb_$t#f10Au0{wjr^8H;u#NvymjuJbiz&V!nRPV8-V2BsI*S z8t0PdNG9Cd*uN^RgXv?~b3yH5h5VjfhL;4h=jg0LR&;lCBG&G7Mk%Z^^93>WgcFX1 z&%e#gu=yCYKmiH1n>47##VQ--DdKHyV46zql$LM_SqqLulBPY2IrpQ~t`C-&gxR`5#8en?XFr?@TDb4E;Y1`Py#1S^6xFWNuwrwU zcybx{qMf{qzN3#AUobp@;#-Hs?I#y?@iN*X`#fM?N94A&TKm(Eu{z3e!MLms znY_KHb6pu*8zj_(?70txs)aAV*)WspRT;bda4-2R%>8V{e_=W?Cj7#y&d4B9{C0uK zavc}Tbg&JB`2Grv5y0F3AT>}rz@!u|g6(_`8@{sl!V$xZ%tL133$bCG?sS(BICcxT zs=3k~Pi++){M_=_8KFa=qI@bSjP5 z5L@%_Ssv|rJ(`Y(DKkx@A2Vh2WMecD?SuHmx*x{My`+J5LYhR&v#7hNzicTYg?Bp* zi8Zk0YgH))oO7fVu9hq%OIamLc>l-V2ecveWL9JOc?f;2?T_2@^T@LMf7 zs)O9%1Y^^oQc=E_yKB_9;!WHULKwM~=+t>k&r6gF%YBidk@+IUQS2g8DLf*9CR;8` zVsef3bT2pR@kC;Y5=W8dGfMg7hwj{NpT~^epT>hPd@yKo6ot>!RZ3ph&8J31Q6-KR zN4dK?nYssKzPykZSOQr68WrYtM!@NiT4xlNbkv2gLXZf3oxECtS6@AYt0VsaS$%rV zKI`w*TlXY|C97Ok)WNA?7@>zq9(#$lBDL4vQmM-(F zSw}2%gsi(DA8&=*8VW>Q@j0b#N{^H0zR51H?++X{9WtJZIoj{14eg18A?Ke=6s#1h z^NH(Rtq8O1orP=YEcodO{nWiiT0HhS@qvWnf=MhUVCAi99(M2c`aEEr#!T7wibbkn8Nhr_PpjebPXWWld^Ad3x+(TBemM1{Px{E0l_ zxT}JrLnqxi97q0f0<%^FxmL!IrNh%Gm zPM9}rk8~MDZ*z42bWt*z@Q#P9{t)0!_7J;L&DqHnec6-XhCfVAoo&>&52xvqjd7sG zSr_BkxPZ`*QrIGf0is?aEJA7|GylZ2KeHzWeM;H4LN0`Y$WDu5rN!bGlh=`u&PJ?+ zG%vmDL&xq(m5)pBXjYSAN|BKcrmH=&3bbp~G5Z5t{FH@iJo;EBDvgC>mfW%Uutj3K zxUEq<^v=Wi>rT4%KhQ2o{8ZG2+rDO?Bz>qyPjpZ~7`V_8+7(D{XFN5E9+BWH{U#~4 z+~A~Ws|*Ui&F!q0>JikOSd|aa>+2SUJ}}c(>^@9L3b`lwdXd$v(YAX9LbEKV`&8Hc zSE1Wgn={_=y7>N*XGdq8d}E~k9f1g}9e4V~F>4hE9|f0pO~}VYQH+AP88H_M0wkx= zZY;l0U3R{N8lC>qUHn%1NMlwoyY;P2z+H8f?E3yb36&0vo+mP~+)5pJhJJU-3HmEn>3G16K|m;T6wu7|o3eTnc1D@VB+(1HvFki^+_u%C$i6+rm$XlA zRB^ngTpu(t!ZuN1l|`4ps}e#+Gb*_V|r7c@m5)O$2r+t>EInq@a}KWsdzQdbX)ZBwgt8|b)IBg z=I@yJc5u0{SdZR?+aCHsWTLU-V^4%1EWKBF9@0JK_OQ;0taFq4z7;)-_Y3 zi@e}6i)JZFDH#aZIMQjBtO23nn)1gwVt|6dw|n$ite52>WYmf<-oi_6m;KDVX~Myo zl&v0KMEvmqawt4q*WUVeFK-E>5NEBWP7`U*zQ?`U+X)Pblh&*&vhxq9vtJoipJCKd8H{&e1MeMt9*`Gd_j0bGx3OMn zQ@L2X(dP5!T%EL_80Hsx0<|^Y3XuBO13$RC2);i}?X;_1i4+!yHB57SA4w@End0V> zsaebbSs)tg3U)7wCU(DiDk}XryzL++->@MVI&l0x+10rH{RDJ#odBv!%fGWuQ8jIs zVXri-@>XSZ$+xTT{E>R|JuekmHPO$pSlC3g-#6Hzkz{$Lyk54A*gtyXYH6LT(IiTh z%AQEVf1!Y3@w!UpJ9D;VNupQmvB&C6lql7-s=Bp5u1y!m9FCh1Rl3DRm%Kp#E(277qO%ywAd_YH8p#V9oS z9eq^aFi?5)uzm+=_1@fK-C+*^n=S9@j1`BnNlXd~V>L4|2pY#bV z^|LE-2Io8WVw=)edHd&KY!`2KXK4 zvJES*bF4kBF*wL&i_-i_tNX05ZnlVE<`DXH7m97`vM-s1kDh6qJw>MLMY;|*qgE-Z z9kCi0LwQV>H?;EF-AQ(%&Kb|-sMGJ-xF|P~1=&j>epgyg#w)Tb+h}Loef`}@6rn&E z6>rscL?J=tMw1_A!tCvM9ltTX5*Pq$-J`)%4m-ECVtiJ@5LefgK94OLz1OUKukw<- zWkW(%QnBfb;k-5iLZgAPB^_oc{zyg0Q8h4ziZ(iLEV4yZI^_f(ZL-D_CZ|GGIE*;l z_? z8kk`9P8eBkHyV=VY>j1A|5qp{8jR8a= zJG9Ha^^7eRXMoAGntLoc=FFXaZPczM1Yg5b#Qy2C{!72Of6A_7DR%t}f#lk+?uSIl zvvK1+L3lixvxdF1*Z*+rK4HOkLh7)(poBuD1MS#S&~a16xLbdwu0X0U zu$JSkZlzg3%d1)~>g4!|m9uB0h@u1NcBW|o_!lRLzB}g5);XT!aSxpmPA}WO6bZ%UdQdHgUg1lu<>+G z>XBZ5_4E)_R_bkFfo6?R(f&Vez3DDs2Yjppf6k5f4Tfj}GDL;KIg zD7eK7?`R6zLkJ6Y2rKMRhk9%@LU>0A$Fp~oSslOWY1Ba!+xJ(O>mgl)@by+T;~!W2 zRMwXVxA3^njWy~FyfRapjWt(hr%r{xzGH15Yh`>Il#=mF|iU86Ov zm3?l*A>pZ?F8!G2sY8DTgtXFf_L-r;h0fwr~A@qfF|%TZy5j1sSRPo)^i7P4j)!FbJ!S zORv*&zs{n<4VB9#aT9qW+DA=S%2~NzPKUa&b{fckAMD}Z;7slnwM zLCBk6KYOyr)3O=neH#J&R^HvNh>j(r9|Ubnee`hDw7s9_D@+_3MdMnh6im2f&*9%B z0|G-{^>i3hqZ%3*l5tekB=vL5VPfHtOHj$%b#K()bK|IlEbr(omH)!q#Wj9eq@I~y zeGnt;65ga|>+UVq6dND9wz`TNy^NC7R3BLF_JJ%EMXM}6eaLiqrNMtdYC{E$_m6?; zEcscU%eb2xYn8slb4Mti-8Kd7v`HW8=7pltBiTVPCFRuuH{M(kT!)K?=>3>c$e0*{ z(=HA_)9vmJu*;IzBOR(%5e^6m2-fJ+>W;ejh50iURAzi&z;IC+z!|s{A6l;64Dpp zf%Inm{6weRqA191XO2`+5g?VsOp$mLmV=ez#qwwC?oX~D+jY)_waDm$#*pq*2+cqp z8vIudW_vd;BU)_PlCpl`A21DbiDRIO6o7qd0Dpe@2|gL zmBas1|LQ7v{{jt95L&h~imA{3&5zYcR?a16D(w)d>qFl#GJ2dW50p<7_a~ zmu?>SnSbB=_7uLhadH@cDMnu$&9ao23(6g7&NNb~n$Pq#E)sLEA~Cii=kHvSCXUfZ z(iT}ze_3abPM+deI;f=SD)PVjD8kgk!`qME`{Ql2;#V2o9yFLBeh8Yw5F>Vdy-2zuRww#YjD z-Lb~CReaM?@>th@9EA8;g-?-eQjg6{O3KdBjLl99Ra#F{KWdj53^wlwO%4k%%$EGw zEW%&1l5usOG41H-RCC4XBvCAQJkYbMDRK6eNce*dS2N5+PUu#Bjf0;2oMXjlZroiUK`??82!346%#xsd@v-a3wiJj2E_Jn$wdPX{OuJQ^c#!w4qwhF$LVKCRcdY>SFqE1vPaO0$zj~ zYI%eTCG2?ql&jHRf@AX8#!m0;7S?ilX-FTk^xZ;K{=pwI4 zTgc^0g}L*G&9pBDMwemdZ7f;WZWO%r;mRG-}_>82%RT&q_h zf6;XJ*ynkGxbaTEyGDxic%xcGD5dn^&ktE{OJ%RhWNc^NRT^)pYR{E0x*qN-a@PxD z(K^q%35sOCxHJzgTyj`lFh4(DXXL11hw*H;-*@PxRjo~z;?0e4)+=uGCSTwmxsBUS zPR~w$$g{z(oSDKyedcH_P*o6c$JMdBxq;W7vHYiTzLCr9N2YvnfkmJm0NW%kx6_M= zOgXI+UQ#v5`NknyxgovmF}14tK29g)QvNsThIR)s&on?SPFoa&s;m+>Cu-Da#&U8} zX;sT|9@A;PkZOyKIv%Tv`I!}%>oaZE0qb77N2yMf&$P#4Bc6EvAy7K%6w}mbABW~M zO*edSMI$x2tP!XG+*-X{1Y>)OoSlFzHt$QKM4Ca?k$I159(nMTedIaf#~d}0mlrq$iPGJw zbk7TU*oWcV?Mcluee3K+^n2Jn-Tj4iM_kd=)VZ~UR0Z9Br)p#2kmv=6S>1XWW&oRO zwZ7MTF^)06;#yL(Lf*I2!haM=8@t%2E3yBCij=upS&TosDH5)Odjc7&JbIO5Mx8sO zKtvgpo3npxux|S%I)+DLDFK~Ke)fF*gy&>7u3zm_7*A2Pv{ZCH1tJQLn&Qj1oNsAW zipXbu${`)L8}1w}F!A z1gKP20Df({q5I0JR%z{5TJ55|Z4}GCjt$6Cf1x$8LY~_fyPd*xk$9R{ITDimFAmdu zTw$*LDnB4Nj2RfTL{eRZ72zUC82%j6?RE7>qcKlkT`f+UQF0>qTh=ZG>~ zsdT^RHIC%y%8p@4#WFtIs{1k|Z0mM$>BMr{N^)4H+i+4?pjE93$z?%=`wp)xs*4Py zgn)lX;jF8NN{dg&K*giUq{|+DoPlH1pIJQOcJ$s$Ql#cBdZye$NCbpMgTj+lx$$ff zJD$zN-|gV?O0wnn2y$AnmK)^unm%;ROvU3zUL1*dBg3OcLaImNm@zwot}3I&akSWZ zoMFC&Rti0f0y}5U*z?zQ1#eUPjsNEL;A_3sz*#BN)iT}ssDSA0fRv>$c)Qjq?gE|@ zM-|W$srgDiYrV&j$&B*#;lc7pr2|8l=975K@P{8{MWwLWDpw#pEnB-B%D9U%Nd!H# zY?gh(&F@WjaZKGJB16}=H)mr`+5aCzwPz_{?^scu&CG>!IBtl806 zBOm?*WTh+nv%G0gK;$E=*TU)2K5m`Vp53`GW=RV7H*SB}H+5m)z%cAz9CeZu@Hnh) ztUy{&TJ9pqrWHIkHdN*Y@N9N~C+`9;c3~nF)^+d1U3Eojoh19b`(sIEQ9^H=A0N@n zHSNP>R{89#HKY;+0gp>pkRe|}mN1TFhKtpTn|VJeqBV>U4SqF=X>-jmz|N29AmS;6 z>-cGilFH}uzu3BSd4KaB*LLx%{Pf8xL*wCf`ZC+^CINwg5tU=?u_i?LAa+EL$`DbO zBE<^VrK@;(@)ugBq1Can=9PID)Y?U6>e>l9!xA+4Ctc@w+<0lHibk88+#2A^!YYcK zTG+JOk|9LgbmAB!4+8!VVP_o|<+k_jy;VXaHX>aLBGTO{-92#C7yb?GG>(p!;@&jQvVTuU!R^WA~OvCMmb91sR zpb)y<^f&5u+%_!ZLa0PH(5~gBs4*I_5%Y`0N`Sf9eWzc}B$V~9Sra)$r)cz zKSZD|Wm)ArO}ox9QdOV+g1*o9x^6^R11s*?8OE7`zT%*Hb=h>vwfCNC6~l>Yd9gJW zy&))(G3Pk?GhxCGw}FaBYzllXKXGl(FT~%u>X%S=cV$GR6>(_Chj{6i(65}7%_OvH zNQAf#0)b1-nW8ZmjiKUgEt8SihaD2aFwdwffhOo`%?=vdd zgTDV_wbzz^zIvJqz5I+F=aY(I)_AMtV>+(+r*G~Ry$l#lor6O#r=+wv`@;7LYw~yV zT)o4BH`NQdf*IA*Syyz9P+CVCoUN#F_jP}^$1v4f)lCK%r9sub%hwX8WTwq!W6EOJ zEcEGOBgu}t^SJ5Dtq&TtZlo=T<|gVwv6aC@4C-=%C>y4roi#mu4?7y2DoddY2PPvY z+q$i`#4Sp)mGu>jU`i2*9rFvN%WR!QdK0q_$gSh_&(gc;yTL$>?sxVh$-VWn2NfN)_R0Q@*?3bG> z{@yR)%o`Z4s5eQZetx#9`#K+`IjeVmXN4Trc~T zAlP5Q1X6LBRDnf?6cCeYr(E8$=nMbDLH$~o2{vT%!eKyBXF5dSaHU1sM8TFN;?I0l z3L(bp=dG1PzXQhn%6n}Y%4Qoe;GZ3Rz7;%v3REdSQe|tJtlRocKrx#vq^M>E;46^UTlG8P$4AhxTkct&lES?pNRImiyc(ct{KL zXly{I`nHlKg^9=A%#54TzjqXUn?wA%~-zqw)km zLZUlgWBg=#ce!->9B;hj!v#RNNzlQuGn&+ss!b!T;hAnr~#3X zuh8f$%&hglI}!aX_wh!1yH|iDENVzGmT5&kB$ZF@(hi4C^{!B(RD-g@6NC{tt|?y9 z;%-j0&7!8oX!2c)tgRPB*3v0mIq&$&^%6_5#kZ!|a1Py<3`)qXtqt^LrU9rnn1+r? z?v`k+c!a0Bcw)||xV*AR&Z9QwVk{EaVhRG(4F9BmX+qZp)C!Iiv+dapdQ4;Y)`L5j zvr{zbHP6Je$rl{J#r?9Il|-mBOa2{Jqeh3POD&%9zE05;LT-jS&A<){7^jLCdf@2i zQ5RaBmwal*8>stbcc*i3Q~YIed}QQzbIg1LE=G3S@~QI6-JsjGDKO-OP`TKpVY<h`-O}1F- zie#ibNNXgtmml$taZW|~bx>Bn{tnj=NL1PAd5SU_9noy`T#Hi+44fUEj30pzW}Q7v zNIIq~HUxVIblibh$aenB<*Qk%B%e{KSjgp1t#-z)_wTEm65LggDV-eYQ@h_eB%DaN z5QOU)OEp)z{BTp;T=@KAEkw!D57k|dNOUG*0?BW$$VXmV%w$Sm#SOiE225sE+^y=Z z&iWOji5sNI;Ds+0$@CV8)u({zNk0%74sUda+KuSsf7YsW1*INtfOZI9pQLM2r6m{K zJq=*HNkeqhW8RureDM{GT^M#@`zqiFn8%dh zw{hbm0E7@&5HRIARyUd{ziOA`#zx0mjSN1xJ*XkK$S)zoqPUlQ;t3(MzCKwS@|7&^ zVOoY*0AsY(?aQnM)+Cd%*wnlpe}au@Yvwwb!EG#1f#~k!9i1$S82+jUgVxsA&sxVM zAvQ#3)4bj@Ki2Bv45DXyl$QA1h^W}}pYF^}A>yVKe!tD7$L-hQVAw7FCO+p?uTpxM zACBSLxx$Ff%lOTxXfV>(Gl7OKY8a0;FjXF=lFXyXH~c9h+VZ50%VK+LDj5GfuxxU) z0`pLE+CBS(nXOPR(pz3M_ce4IKri<~DamI)=$Oc%-Q{b1KJ{1C!*< z*XQUpwAnIV(XRVwJjdI^1*tRy{Rlt>K8JLYjz$CbTbOE69H(A7vgsT?GViHaJPNd0 z?fwsXJ@GpDq!m#&Qis~OK87kXN)9nSSFZ)pyMwaRp885*f#H>1u|F*nf*Sh9Fg!{( z6BxGTevkzaP8BOl$^(nX@ZOV(n{Bqk;WJnFV!M3>x5?x-#gWDVE{JwpPi@9cfhw^` z`#tetAT68dN$J&fjaHqebmd%4A(Ni>By%1(1Yd9#lUkRqY-}O=^LUQxGVDk;U?F2k zEQmGQDv&a)0K-N%pSP&C1bW-Z=O^dF3&?UmPhtk+%1_k>f|7xk2=k@Zz>MXC zEx%V|ZalkfskkCnRn3|PRu%B@n+@K}vsl@)^b``EQn_Q!E6@3Bi9Q-Sc&Xog1^M{; ztq%32_ZP`VuWeu3?;Y`mGq+|&#=1~HN=O&&N&0%3)~{1VkDiho;iJ_YjW&6fq3sZ= z0-RdDfW`h@LAWOnNLk6g839&E0M`7F7P0(hr+TcY8}<&K35Va69VYa4t*YLgD2Vy! zQ;wYn-0)Utb+7us1bBI6GMgP=;)%d??#}0nPPU2f-%ghIc3)p4$leF^3iIu7RH)bO z`s5{U;9GS(XCyios_`cM-r%#ji`%AnPdp@d9~7U&+IB0=I0w>!0HYcAEWqmV0A7SK%qRU7v-VV)oOJ~t&8a95D=jk?IeJdn(| zLcmJS!925pQQ=2Qd0M`GF>-20 z9zIX&{cL>TpcSK~4VSv`Gf9ZJ<54*9f1iGHd`P=_E{ieR(w&ziYkZ|$>);w<+^W@) z=_Uya8qT{v?Vh#w?H2xhkvOfxcHgmj-4{#P{8Dyg)IWIqj~UNC4#jixY||Fh0{$k+qxoDY8)xt=J8VudcY z>oSm<7}{}{GkTKEX<@Vx7VD1CT;XP9m?ZO=P%(gdinZ!=r1SJDwVFP?U{Tk-kYcLY z8iJ4HzNipOXNS?{IGZ8IT(>V9TIls{@p2z&!(=KPwl5jw7Wdi zE415^W^FYz)BdE}h?ei8LGzdd^c62pk#5XiuM)K0^%URnKTLSFtCz*Pz(TyoiZ-Wh zn?YCW{d-q`HO=kQcIn&}2c?59jdlwG29piOilzD-Es1sG%EA|KHS~eJ$s5;8$;Ujh zLxYHg)X#OS9j8Po)4(i^LU$~7%h}SXlwcV#s86S&pYerdW1_?ow<%8BDPqdswlK*= zpf9@kiXIizeTj_ernE<-EO4wo1AqIf4ZwSJf`aYDyjKTV&aj>}CF$ouMyckYObIk8 zNbO%?gReh-&^%FF+`&qM4Y7TJgb~=sq-A3p77->Zg5u^(%C`4|Wm-u0mGzGpw#ge3W(VGv$Kq>nuKmx{i)}0VfD>l^5 z-Fso=Dle)MlaDQax7!>;EfLT%HHc@fD?20%E&4JqElNokl5F35YczXrr$=G_f}LIw zhhM_Nhdr!p*dY6}3(Pi1zjI&ZFaETjMhQV^Z53<|p}5VnwZEdj zEERO2oyZOB!i!B4Zy;Ycn+=a>yEseL`Ua1<6GC-+zFQt?uE!5&v8nuG0;fK=j1mx~ z#>GDns+mZg6?+JTwwS7m3;g*{axiBq*2-!nLa3TP&Yctp0Ot9R0;8G{#Dn8uRO{cx zT)@N6VbVA8ok481YE)>W1?ZK|xV?*afr4GWP?7b9{WPk#DT2bY#o$^gZw-|6jaIa3 z9gK&LFI|9-))tS6CK4y<(AO*{gxM~#J*h49FZy#nH&a;w8rc4B;3jSPA3#QcSaB$& ziPjv1oL~}Lf9mWq78Oz2Y#EbL`KkCH# zg+}2cQQhwa*&of9&d#{7@gCE8y*3s26%gt-`pby3={w+#9oKrTF7y<4lQdVlozR;x z56p$F`gkS#Au<;_p)a(pj_pOWdJ;&lD_JHS{nPm9<|!OA^%WJyhYwBr{S?|Ww%IJe zjZWBBJqVjil}P_#4bB1NC#wRp@I%7cnMZg8z7s}UK~oHN9jyj#_As4qFR+P6av=43 zuZdrEKg9&-Pef=b1lhjdIai-?XG%ZXv}QDLA48HHNVgl^S}XvWd%#VnzB_b0S2y$j z$p60r!c1xSv@+=eddiu?>c;-3_nWG9tL8vZJ@Ce&lDpiyMHsbEHkMtTAOm=t8E>yI z_}twFquF}ij%V?u0SbeaF#+nW?F!!_t?pexu86LWwU_31Gs>curk`i)OkiWH0Gml+Gcl&}+v5PD#%acZ@~y9H z6KeyOGk_rSyPyCQDKYEw(FX<^-8=0KpSSe^}D}Zz342zi>iEF{CRIowyn0)`vm_JI6s3%X=#R}F@I}N z5w`cG4+8kK1MU`3z$gGVE)T;>0I(Q2ofHewJH)pI@SehGXZGu~v!JC8j?h((yBXW* z{P#2Uw(QYWZbe$Q-Zj8aiH9t;)vW>W4tpX2j#U(7@JxcFouNtBu-u(*s`E zu_Jx)$j7?4QBCylU}FG?pbvCrr@aT@2GUC%05*!F7Gp)jdeX*MeK^SO-q|qbEKvit zZZf2wbm9BDA48GZorC@`=-_iwJnz6n3^4=&q3}fZ<&@D(3Lu5;SD;Dw>=pMa#g@2oh0SzZV#w;AUM9geheQrfn?N{Sb zk;Hk^ZRR#Wr~~w9SL`O!BVX($AWttZs$(gsMZ&X+@ec5sS^mg+Tw*%|kcQgtsQUMk zRmH$M#y+4zfTE}X8uKt0W8UHVtqky{4AZjFb0Ue}ujRUbz47`r_i!Oxewn^c|Z3`{Tt$ z-M`xqTF3&9)v$re>paZEbrsRjrtAlm<7$UH&{feo`u4s>5->Va&WMZ;3(4v5hb^;20L+Y{GUntLI;4fV-0AE z%zr4x0K$&{M6oNiYshvw@2`NI2{3rF|C`%>Aw?5_#`@e#>BFEnd z{u3WtgLwW6A1hTR|1UU4O6nbVz?H}U4<7?4hRyM;g?|YEfPVZRKU@xwp&EeP^j{3z z1CRCJqFRqRkPe_p_`k|zBL%C{ZB2B0{5uSN;NY!9p*PBzuYsg~LV;vNroaJU3kr{+ z7BiN4fX*yh08;AmcTqh20iDJ5DUcdT0*(`~wq2;r0br8p;qOJVB|-Oh52Pq?i6uZt z|3OofC9wSfbXDtjC;%Q%V5pNyk%Nzd;kkYXi2${lnW5+TKarWW#Kzw51U+_M0{=EI zLz1}2dNP+$b*HfHf17mxk?cFbr}>6HSv3n)YAZ;!pJxu%wenD8CaoQeYm4I*bD*` zZ~Uaf-u2j|oZrS=iO=^jsJz8XRdN^yqKWz3*&fummADNu)82dAq}YbWC3c_(L*j!! z)aoD%g_x0rcp~5Iue1TPG1V^izfn;=!Th;}W2K2F}w3%YAu-v|P{|Bq~elJP%X`t(yC*_g}A)8#7X;Tn+xZeZ~5WzA%X z&E!sqpxU+GvLH-BZvqUKYEeXtEId^U9Kc^dSkJ(-4rPYFymOWZL@wj2)j)5!({sRgG}>m~lZJ$NJb+1>)+z}r zzNnNeC&}J6-!>?A|CF&cl~Mk_WE`M z7L=%Bw4!+U_ZJDk|NYJQ4N-cHc;+~#;_gF$hS$3GRA@8))(_f@&;V)AVpbs_`(|+bb9wfTD4@Hv*i^#19fHQ5WsowA_g!&evnhkqv?>$Pr*3 zt5x(EX@~g&pSP30gv)0jT>pGLw%7Sl7f8IO!FoP~vTi5-X_%Cs$QyKb^EZXnw+GPR zoSOJwH#=2?aQnN(rFpWC@<*;Q+qk6!K3>;s<1<{>{QXm@7%2o3rS2vdoHU3+=IV+W zUQ^)5uys~muo^D@Gr$x)^Z2HIjmc>SFonDU?c$=y_FU`*n$>4~ZqynHngD!7ocKo- zqWJ*8X`1;6d`1LVI9H?3Jem-+ANALmk&`ZBUqQfTH{hf`PEw+tiqK~yw(=$c(5oCz zUDrYpkfao16~t3(wh&e5fB#pKN$TUR&_KV`~8a3`!%FHW%aaA*1v2hBX8G zf_85iI(x-mL*KtXaIQwl{RAFc8f^>{VHTfr>OU$C zAc@9i;b>{qPHtn0ML%qlGoK8QUs^!MH*6W0Q33V@)6b4jkII|c%y9A*^+wNuqtrZI zhOp?dpl;klj_Kx)YfN8E4&i4;nm9!<@^x|j4>#!H;8Ca;p2IuqJ$c0HVMV&7OSLHu zAX~=I2wzPAw4936lxyC>@n$m)Nv$+L6mS^>Zsp;Bnn3sUL8ldCBYz7w*J`v2(}{QL z_v^p!Ell|HbcoOqKEbrttKM)T!Z-X=@9#=AdNMfZ0^>7JuoT}UpYX-47hF{ zmZAVJ{P569iw#alslu0jn|<}61n4R(zCvj5i_H{(Ka-Yn(RBs@xaF+KE_ezN0P%g> zLjq9VeBMu|&??~IkgUHavcQiyPgtPXlMg_Gsb@CutlZM~RXrHa{!G$@yl*Ee&>o#S z$Fo*JPPlOBlBnVSXvn+37udJdTgU-^Cr*{-bud~CK%^8F(bnPMb;|hq3ikJ7w%uS7 zI+vj=+DUw;{ycsBmk|~?kOFx0EtuO}?L?^<>ln`-k3L;p5Q&o`iw+s0&&7b^-=7#E zXODjNk84?s_)~KAuVLO2i@I2SwG|;f2ooj$_BEDOvb?eXi1TaC1b9~L;fmO3@+njO zKLgw&zTiPLc6$T(pZ;eU=MTtbl2TKx{;7yPkQt>k5os8055P?1=7yV0n^nY?<7%_E z)kN-VByaU}yrzbx<=t03PRIEt0@>c|<{f)+09`v8WAbJ+$?_EbJJLK3+D-`cI-iX2 zzie&G=s+a|N_oH;KDbnZFO$pSlUtBFm9Ud`Ush>e2YuQZ5ZQ8TfM)7~iPPR{i*l*b zy?PJHZBGm(pGIn(;cEa`^Dvwre+i*5K5SdmQO!{p@$L#3vI1k=Gb8zGwJ!ssvem8+ zAUI5x)6LYgqTMC{V(q?;Y+_1ERvq4Lfu8js&$_bk@BCOs`L;d1XeDfSwFb>M`+Hfh zq^TwMR(fS86_o^EZ_o$Qx(DSHKm+WT_CEJVMp%4L65!nGoZN(drylkI=W^h|pte43 zpN8$_JdJtnvPA>*BTyvQx2tGe7RzH#?Wo17xA(>B`ONRVPScsq4p~v8TyqUDiJ@(y zXv~#4%C=yp#M+ruP@TBLYL`k$&GGJ`{_XGI$L5Xi=iXoXjOq&b?#Z=o9hc8IEH|pW zQM%Lap*SswA-`{_*6WoJ@v5!elMZbHY~ue`)sGQNV6;080o486pGa<>`{ZD^o5hBh zOZ$1Yv`Ed~WN=m5&QL|=Us7~-H{Un_Vf0-LypbyjG~lolWn#Ix&~G`nT;j$lu;ePx zCUI-t8sP1sFU$8UhH)yjMmPT%m z0Y7u)ybNVFx~uX(?Z;dyVkDdQ-^*H*)v3{A1o)1#KN&`Y4}-2W_&%5hwe_lgc!C?X z;A}`+)N?7vS1!b|VI?IVow3z?utX`RD5iRM&;`S{ykv8yhgJ&^%$;_kx`1;BhIbA{ z=X|_QMhHlu#!l6+dW9YhaMomxuWH(38~*Eums<1cei}YIl59++la8ahnV}ZAl}**e z2axG3VlmVu=lcw#r@%K51J`ViSl~2?fdB9O+%_E+KKluaZm~Cc!#x3ORI>4J5Enxi zeUW-)mWI)of(M#aS)R8>m=s>jhQhMj@bWVKp(Fc+x`aF}5sz^Q{ECBYgorJxOtqO( zAnEPSAMb;J$Cv%(+V+>t|$Mmoy*}@o3spkUt zKvS#njMvYI#*$sCE)fSTu5|Y%=?M*m^BM1Bf@LLTkTV=+;}bihf+L zw{3Y{F;p%kz=G5d)$1wdGAKihZOLg9CeT5*pIcg#AT&Mv&+-S z$cYm2?@39x(Au>X&?{@KENDzKfL=iyi$vv?JH_7Z8h=+4Z0l8DZrx*)j!sEgIxQUz zm51%f@RTiBdzIg#AOuF7d!iZBVDexpJ{$Hdp`c8ssn#vxVG}v{Ew}^-4?ZU(WT~*V z^lnoLA=0 z1f0w@oK8$hNL!+qI+IARYAk{$qTaf{SWLbMP?@TF4;br=Rcu-b)0X0KAzg`ir_*%9 zEsmSzZE2&~hTWw$sPg`jk3Y0Pwn1bwyYzBofnj~y~P zO(6$}xLob@c!=%gZ^m6`j$4xM$*I$T>o}H5+$=_f4LcXqScO^lELtf;R0XU#GQC+j zKP^AUq|e_6Q^cmb6SRAw{WucN^@2%Nddg4Mm zN#;kT|EOza@lD?sYnR1)T0jm}f>h{r=PDS8fGAsf^t1zBHC=ixb3%<&erPCh&;I-t zm|V47?u)%k$hX5C5RhiD`8hLQF^C}kN0RyW#qb1R09bhO0kG5oUl|VtfxpjG*~GL| zSUg^P_QHc{fjQCq%B@4Q(PN+va*So9u39lfuR<|{^IVsAcGNL|4=FN@^0um)XH^I1 zn4n1iJd+Y1z`&?&ANzXeL$c_6kxSWVgP;K8AAsUDQOIqUbTj=E)seZ_<;+PnWZtpB zy3`<6c%*x8vR|<4*3@t$jG$Y~rCV(JF`0z=&P&G}1p$V#nkG@b>Awbt6cb~!7KW4H zrykPCsh_uvfg6l^J$3~u5d~2T9dPxLA?tj48mggh-*&mgYcjo(o)0J>C%<&PmsXTV zOvAO%SijGkemDqNjbc}n+2jC!qL=}FKqqT&b1VK$!OOdAMRR_#^6SEvCBAdZyF84PRH8dei>XoqebE5{$n)4M@4L+k zi>0D-Pi~naiuysR!Q&qYZm!&vB|^@ix^7^I*$_?Kl&B8A=ZYiy0>G14i1QyjpWljvSn3j76R=zyDLe7 zPt6gNZ$X9C%{I&Lb{A}E&`yZILkHj!v~!Fm1v861kDuzrxvyfKumOq;0zq0A4z=Gg z*FGs!_PdLbMVwpLsut^?5N`a4gfSC#Nl{V(-M|b!JvRHCcYVE(Mb89~3` z$yqb*9XtAVug*(EB9DT1meJB^$N~2SC)f53a~5wM_;JMfNOcDRYDu&hr#Pa3`cBs5 zB>Kkt@G2zqPc;S!X)G8<+7%JAuGC36{7g3M!j(z49Y`RJ zyj6sZ9*Fnz4%V(VTgFnT2)`sKivb@7kuj9H_!L*x`=4FD1U=0t#MMsbI{7)}C8?KC zi}!*feAotL<{H9l!x<_aR8!}WsUx4&`#tCc zIga!Fi{kKQf-dBDG?ZJaMXIEFA_qCO^66EsRYa+dTYJ7?;kYnqvm^x{-YIUeWmYh~ zo3}2w5l6eOau!R#6ZO-?+fZv$G50yjE!)iG4N7=~hK018!i@1~b-&<^1QF)=Ut%dJ zDFG2PSH3Z<=RU&%ELP5ACj63o7C%VrW0!%mHV({1nxDm;_L7|jcikhheD7zW)+w(F zZ1<3n@)({0KZ@`}&P03NFSJE;?<4ZM=?F^wX{~2nSM@;ix_J?bZQltt=CC*QZvxL;D7J=>k`KCn6Ajps$% z9Jnb4{p|IH-a}aTZ4duD-|k3h96D0%#1_6^pWt|%&7|c>RJoy z^125{U=ff(2UW>x(%yN^p?lWNTYH+DyCZYaI|*755;%23CfPWdr^pwP8uw-~kGbFH z%f5>zKS;tjyv>^J{#EIYV6B`_oM& zqx?>!O#l@{&Uqc8P3-{EfFdq54oS0^+Hz$e!XCf#nGH}?N*Q=rU-gc+{;6;0=(yv^m5g8 zM7s=17bZOTW>&4FSIX%D7)EcT}H}KK8tDJnO<}+?UwsfS^lwg z@w?yXWT-Q5`dXB%x8|nLH%A8_Usz6OXJD?_j7cQW5>@HBIf%TLTjC^q&B&;*y(Sw^ zx4XWldtMziwSX1CR@~uQDIx4Q9}oH2?0$Wq(LGA&CLN0oX%iZKbjI8u`V9VvY%!sh zy;M2aAI%dYVNRlw{o~SnD1Nw=lHBr&ZNqYT%#|^~_!S6*hAA>pec{SAyNvLBeUd~x z6j;tQTdi&aK*QA+*GNHFHJjgHgXp>Y-sF4(Ia$xyg+{03&gJX1x4Z}2b+3cgA5Hhn zd1|z2(=zB6w@b+m?c|DHP265pepGWumk)ky5SV%i9GA=-t0_4aB0dw>T?MC~% z5-vlkuJ_|Kmv?3OkihQ{J!?eq=yU96A;x=53#{{$Le*}V!HQzvuPJr~l6FbS*z zj5xtBjvqh<0Robup6nHhytgb`raq@6TB_UySBK2}v)H+CqNMDC|_ z0wT(pp%8ps)k5gcfk1;sZ@mh=mfREukjlw6TREekj^YNBfdZ@NM;mzG72cgX)Zm)a zxYH6TczJZ%c2@G8&v3;6SUSnSH-BI9(KjzuSx8RHqnTO#dOfkYpER${dQwHN*DNgM zt+C90GLEM}@Fl;1IS#htzR)fqY27I!O)xQ{(lBFGqsliRV7l6WNdeNR(FJhoPqh&W zL6K@ZaK|jl?y+8y+uSk01;AaspmfmQ(#!aGcP9(Y^eAu>wG-*j?;5njTC5Qv%BOr60#NyuSxgQ}vS%f()92(V&n<3nxk{XS=8K=qkn=YGUIIy zF=AV(G}MwamG63gHe+KGSjOzk--&s27}eTMX|Q~zdRzb}s~W43OMKoaZz;Z`5$zLM zdD85Spa?)OuVBn0S_z<0PP?dz2L zi1QUSDksVt>rLuiHY>BoPaq~iW!%ngv7(;`+}a78Mt&Vb`Yce8lB?;! zv%kK0zvB0CJNu2lM)Ha3$@~RB+tI6n_z!{W15;HL2f~ZB=lkXKs7N4&{~moy55cG< znmuQY61%yVK8Z9J#sdz@!h7XP?G z5ja|R`HIT!(f!ZY&|+H{;;J8nqc3-8Ks6of_%^7XVlbyhucjwZEGEcXA1sg_e?0wM zi&Uf0La)NO#c9xMn4;jmOE1VT9oHkVFT~Nlg!g0Y^o)`)?Od}$aiv~a$Ou~szq$C27lY6#|K|?eBksg<*{9Svk_49g+mF&h$^nD z)Livkp@kIu?mVX5ggS>KsV2@X*RXC&zP<=#qA)-w9Abx>E%N-OBBat9v5YTypL6dc zh&Sy%Io{7bfje%%4{AMJd^;3Jj|j(JB^+#mqXCis&9^#bP z<0(LSW2bgv$aSeM(;2jgzhfExxN@dU6Sb62K(DgHB_Ze4?(qymSv`YM?sKCh<0u%F z3FUEgO3141RlM*w>~!r=%013Od=0oeae6YJ0mPne`AeAi9{D`EYuIk_r7G_|_i69G zYL7G-tqra5O}LvWo0%2;%<;GeE(I87XvfLg(l+ShMtMOaed(Gu%#vXkY0`XJ*QKc3 z4B+Jk0B6rT$qWicFjP~0wOwlx8EgH<+p>jFXwI$J%D7CAaE+Sk_l5&O{pYNPb|L88 ze(Q0zn+UXLFA_b;b$0twT~~vtknJg@6NIsJCb^%s1E>m3g9{;_W*haajeH?GPft)9fS zzDfh#^E-v?;i3e5vwqb%if$p6cJrpXrnp(}Z_REnMP5u5e0vP}6NmnIude_2DcJxn z)pFHnJmr;zvBX#0fJfBuXDT~-&=r%2pV9g}`!tlx>u0m7X9a) zQ%jCD!#veoWxy^wqy8J7UJA#BC6?!1@{N*tKjr7@-sPJzQJa9`&FWncY`!-_F-GCW zXeEl^8JxQL#`7<5qn(QIburq(eE`RqMAxU}@RN9V)KnatVL_q4@ z_xD7P|2$?u<W z7D$;-=~{h z{fwM`2pC31LKp_Qs|*{$0blM`7raKi`;jE^E5|~P3p>x45OS3txD)MWA7|_vdCpSt zRDq*|!2j0MpHC~YWmL%FkN%bLuAS1(1baW`ch+}J@o}Ak?FvkhL5V`Fs+;LTdD)I0 zO}2nba`d0sou*N*>GOiJ)|u?MQwpUODr0OSs&%{7K)KrT4z;?TQ_EXj+V^9@V}Al) z$oKp#8$cBmmweuN`-24q^A2oGfo(ewssjbe41ze)zmeya10(fmY6l85}8Lr&MfFcwUo^Vh)NyR3nN?F7<*K8a=Try2A z`po07k?X)DsbWDhh~_(;#?xU2M@z5JbM@0iI~w!A!LBKP`;Jv0*`^FsSp)r&x$UMD zs4ad?XtetQO@X5M{SAH$SCixRn>8pl4tYBTu6e21n1cP%Sb-SR+4s1~PSV6bdJASC z)?Po{CGxDZ>VgbVPwx0mU6t?&C|T$fQ@|fs?(^9Oq%27YDY-M3*a~EQJ~nV1FH?5% zjwpbL_B4P${s^iwq&F8Z#1b|eC0R=#Cu3jr?cQn7I30w~c^!SDdpYm!B=ayzP5E5s zMi+<5>|AFCzMpa-UNKXYN8_ba66TRU?lWv|uN>vRpE+Kq(4uF2t-L~gvC_Xdrym>I zm|Kl4K+B^PdmcfizM4)gH21vY>ncR+&?K!J;^%31zz8!V^{fx0!kBkIkS3vfIf`$5&?eZcS>P9` zH)+;$Ni}@onW;TPQdMn*LWeX!JWp${rZadLVJ1CxJ=uajoEs?h=d?J;mf17|Hof4c zt}4`;hwF2?4X*63{=~N}*|}aE64X?KW4i)1ejflwLXdVUH{sR&>S5Izmk>-XQ8gl9 z0U%LZy`lvEGx7up2nF6TZ=e|M$`!bs$b!*zwJg}=V|KGWUZkH&{cl5{a{Sg}uc-bzoz72e}^ zof<6z82ov^6@`l6+fyQ?xwP*kc~9ZQ?aTz%KCyNvkH^1nq-Rh$_)MvF1-n%6$?iPK zP)-t5@127)?R)PQ#~PUWZAg^{=X6pg@fFp)IZ9_dT7Bq3XV&4Zr6MSSNXKj1>)yD- zs-V((DyZid*|q}eOI>D^IrAc&j}eT5g8H}=%SB8??Hg+dZgA~c8aqOPr40Ir z3orik8$1nIqJT$t+s&Z!Eu1=_V%IV*WI*%_#)6Y8rq}AAMm`7Q0%t*wFhTT&T-g+L zaLH=NhQ;_59mMO;>GiwEPj~$s5^Q^8l4*yYml*F?arwUOr1T{mw|nk0P!&KaX}viL zR;0e(uuBe!xClG9Pu{8tSQ8WoS@OGE+gFL%$&6Q~vjEJbZJv~1CTkoHzUgX0Lxb@D}{I0ju1uLl{L zvae$$VNa_U<|~%uOa_4MF7=FFo6Nn%Zn_NtsWqldxm@8(%l&Xm!F}zwIv`m6+q146 zLCNnlbRI@lm%C^X3duDRbCX}+M14h5-)ieKrdHi$@zq0XG4};FWr31mm3~VlP%>nE zHlOivC0Hi%y|ltEdfee)Q0e6PkB={6U)l)BhL@y)gjfdO^!L4;U^1ify`I%1RgJCm zw5%FcQ*%}5=htE!>N{8xK#SUa3_`k7N(P`$=)xnRRXsYaIkNn$+Zv$; zxuN@r=G-O99na5DXIG-5v`Yf`wypW*m%i?+wr@DEBzi4(a24cK`G=>wyT_1Oya)M{ zzkf6;QV5&<*q(AP?9}6{gz3Zj&@q(X=0_2+W*qBlW$N?ws_6V6X31mOpbw%0f*#~0_;|2-ch%9DE^ysv^i!@_^g|pq?~4Y zglxKwdI@Q^Za>GD(UxbHD{(m6-F&`(?O=^&co+rOY81@-OfF~>mY{oPA~-EZ#}2hf zk6M(pgNBqT7?Mkaiu`!nY+ZADD8}vt+h|tf7~Ya(PZyF0cTtK+@i`i?z;1pgiXtkk zvLF&vC}jyxH10rCs*^v2<1cMeb`}rbtg#0hi|<6na3nPd8%N9F%qquGsU}r|?UCp> z*VaPPJHMY>E_7@tveC?z(HA4W^YHR}NSDJD+EzJA$F8bwcP|f}15{1&qZg-cP$*!3f1#&>Zyz zxzgHh4CVk?L{{qfUvCU?T81;K&U*dcT`9Qg2Fg@=j44&6)A|mB$W^RV+PeoEG#|`= zMoF4z?P$}@^sjzvC{5sr}_n^V& zH8UU_5dY7YP~U)RMi;^61J-deqoQYpIh}mg!o)R<-*+hI^Hnt02x=t!K}O!XlSS&a zHU+;RMfOGX>yXVNRcc8>}v97if zra);?3eD|aq0`%MYxjSSkx&vfpcpfH!9iqN@yl|q$G?P1Nhe4a<*Q|t$#d{B_ULM+ zedTAzEm830qE}Y=4Rzc5HnHs&J0P_N2~VRzevv~fsYfZ!z&7PtuoE7bheD6i$kXTzxK%&-=4F}1cR<+xb@?2Wcy6Zq9INT+L2l8(9r%#(i+qBeJ?Sb+# zW5q63EP3-ZPFcJANvjb#iezwSDU4vVT+vj3`|B@GrCBrA!b!o2cs3*TIR?RDLY~?b zAYitWJ2>xWe#3#y5Gsff{-|6Z5f!stEzZ_r;N&e1Do0 z2ZGmR?SSxmHwHZxo1;SljcjXU8Y{cyPkU0We3!xDq})ImlTiXfLcylcRNrS4BtPem zZCv`1fp~*i5^XCssEoJX-;w`=v9}J3a{K$Xx8fFq5CLgWkrsxKE+wVAq@=sM4Z6ES zTDrTG?vZYAr~zpPh8PCk#ooXBdG7a#_jv!>dmkQqT-VIHR({XV`GaZY>ZE5A)wT;*-XbpSjF2FF_r;B$L&Sg?*@qvKkN$p ztWM8Pw%?%nU+NmP*M*;=J}4UWInfi?yFMAxj;1{nau?1M&&wZ3f<- zD9%p_E(&~c@v`_OdVC$!m3QlAu~i2&L40AB?2I&gDwmh)EG>sGvC*_o;vTwiDzsRdo`JEJ99qzrkSj&JW^X^xThmF=G z5GR-_zT+*Mco=B)AU{q~D~ncXEc;j_D#)Lv^u%=h_eCEzV^-hup+9x*RPluJCk~i{ zq;=`#=FhQ<*~R@v5AwQpPhdn9{mc#ekT%o_7kD%v@5~4CD977E4Bq`Tc9T%qEFoQq zC`y}KEg-jTIu@mXW*`=c;b!Aw6cPhh;5o?kO8`3EG=OxWko~=xkUv44 zzFLAE)2zOOBRe*#eLhMQrCmFpHFcCRUsxqjZdrBkwV%#eS9EVECF-IKZa-IuPCgV7 z=}UsoXt#bEg6jL7LD;R3&i|2*_*|aE*e*3o0Rh2CdOElFxqV1jSR6dlYjk+P^7oH- zAi$fe=`_$6D0dFESI@<`NsF96mt>Y2;pGwToUh9^UoJuOb}kDA(X&ws(Sf_C>aCcO zPL)>ykqXYu;#xV0J@Aisg7E>K5HEC(ja}Or_r=Z%WsFtDj^(EDRD+%Vt(h=VW~xG8 z*yH*F4%rdB)-Y$?S#IkGaBm zpMAhti|y(DCc}CEqPBw)v##mXsefh{3OgmP_$$eM9fOD5H_@F{a%f=9m03;f1&9F_TH_e<#nlAs@e|_Z9AvWVPS~ zPyG2UD;o;U;I9Cu>VMuuIIq{XroD6aNS^>TXy|7sdX|jd_OIv%r~<)*8N}q9#G%$3 z1O1-rZ1nU6?>d?&j%XXb7|#X{;YoOial9=5S>$#}L*6R{D{4iqJWB|7+J2=v9dztc zYh~gqm^tLR_)R$>m55Pe6hgO)=joHqf+@1`ZfkbaaNNLE;?*mrW1?5QsOxpx?`S*} z-ydAYN{CTrLX;6zpW`gN3@wZh`hI4%yTg}d@5_QY% zJ+o6SmJi9p&1l%Ux>-Dua#p3!FqRjdW_wS6AbfM)%{O{cwuYZ!JH<{f%6+6-TMgH)3N)iB~Qk)scWz^pRrQ~o44xLnTW3bxQ z2J-im(AS>l*gzy=x@W={cl$|cpT!7xF%{PD?l)vgzWTw|aQkVGPb-#@VtT9X`wn7* z22|sOQ3?XM$B;;aO$n9hLd{Uvknm^3z1#nTvATzoq>1Z1@{C3y;278&^aZ|y8mepjF>?@3+IR(S;$ zZkIKR71kPwr6EdaE0*-ANRlSEbUPGgDJD`9s5c@DIAY;ofgbcC;=!2&HhBz|60=EEuCm&C|I!P8UD$2frhp%KF;swH^%n5p81AKORvktOMpYK7$Mg{_ zV`>lgdzt^TmyZG@a%}nQk8Pc(|6JrQRGWl>U+5pC<3U55Sn2bPxBl8MUqX*QO`?MR z+n()g{X=0?duB%K|K{mt5MA{HG`0!_)>G zWjaq8Ro8}M02%cD*oNW&y%i{*C!5{R9-*qAv^3SV3oQ9r{mJo*yl2Yz%dVc_Z)-wz z%~~^AFiU@@`DWz2)rcx92x7M0nJzPBWdFGGLuhq!d>Dl1Fsm4n4yJ%g@kP6~sNnCr z-~BgkTlppb&c;2}>Wp#fuCia+x^}2Tcu6B{Jb!Ag_X zJlS=yo>@CL0ThP9AnAht2Pt0}(EWT2BiXoTY$gu?24ykDulFYs&$6#M-}&O_yZY** z*|U6*DJOdiGXjo%VCi7|=PgB!J_-)g;btQ=h}!m8W`2Ce$Oqo&<5{Dq6ke^ilhq-1v))HV zD!BoG5>95b3&$e8Dz3Fy0Pd|n)7o|kdTpLkz!*P5?nlgN8Q;!~`p3{`d^(1i)sK1L z^MW~5#|{vuiDfB_xX2C}&m~`&QBxn4pH8t_2`$(w)TFGn|DTnTf@F;M|F@}hr+)b* zLr6d z2-WMWi#(JZw=^`(j@_yHk@Q!RO#LRqh`cPA)EbZ~NSj&IPkm!y43IQHWdr%66fl8e@E;134eL0wVVkb&1ICux&j^^~ z+?PH|TobO}=QxId7Kg0>?DYnqZxYu%_iNIB+RFWxyE=aFmng?~@3WB()IR@zV>?yb zJYB#q8vZW`hyu>q|L%;2j}o=Vyn6y1+^<1O-~Z4J;(0_FCI6TA8gCB#@h(s+bN@HM zwZ(Oh{U5-UW6?M1zmVH6R*Q`e{~C7xZ$4I&5}hVTsr|*K|46j`ZczIFYipHq{Qqt3 zt)TS3(Ywu?saAj%1Ea}nUf#L?uf`S?X0rdc#_r5iaQ{o=0td$b)YCr7v6v{35)t_V z{{-T3&v8vQS5(M0a zsjam8G?KsCX+dcF<~RN= zn^^%xCIwvze~T37zvV#de!kxN-!OG%y;h(!<$NDWE@%i^I~dPGE$E4i2BR=o{xj15 zYa0e;Nlj&CWiX@MZQh(IRT8E2x?(F*%>I&+b~~i}1TNVaYRNy3=0flXz+apV4ik?1 zsc~h>m@L6004RFv@ouSO;C<|`y7den-yQtavfr%oq1GHQu3%XBc;xkUj>2t`X2zvi ztK4FyXZS_Kp8PBWuF$x4fm{}2Pbc_@UklY<@E0DtF=n?jTh_H>g}~&E@Amv@C+ZDI zL%9E+k*LwCn+)Urps3$f*zJtNjDq@hcBF+mNsdti|2x}&~vtnvQqWyI~{^t!sc zo&V*5%6O+-C{Mw`{a=G@zka>l616V>(rsQvpdah|>uV@HPdse%_Csu0$=D3@u1>nd zwwHD&LE6aSX+xGwROXUz4O=+qC{O+G*I4l*jlZS@xgIPtpegG}heL0B$NK#mU4ESE zpTkB`LZ1P@4A2|V+XZWjYuRME?!s->0-&B*InMXajJFoDZj&rtbm=8MHF*ShV7I^R%g`aNHs_nJ zh5>}1pTvBv1+Y?rk+}xE>$0ZME4Ax`gI4PanHzW5!|lbowv&CITYchfkGlaqPI?AM zFyBX;E7^nQg+Zaf30x><>|;Ipo51{y-EH@(qB}?kqP#HD3< zXcNI(9<9~ndcb>%?bZtv913WXE6TrZb}Dw~SPXeT2dzrgoKgJxx~m4PKeeq{5qEZ} z-vq~O0uM)DWDn|5n z{rMlVn7hl``z0=Kp?IM7i$)d>Wu8)Dz*m|P<8%F&c=D$;m}^WKzkHNbO0c_4;K8P7 zCWqEWVx4ta`{u}WqNfpWJ;;#B7S(Q+X|_>(7^=sjJH6k-tP{3H86jI)ScE> z96<&%9}CufzpkhzAG7I2Q@!#?)xG*2m=bl7E#JQtk)9v~K%Rn`hns z(82rMMLhsnO-z;n>=jhNI@l*CmV`ARso`pE5ej?f&*i7t#dmAroKn15rFsc{zLa0c z(Hb87*_+wP=tb95B_ZuP$8A4a3axN72z0}=Keco#KDv{@N)@w3KrNWmf1;O;hahPne*cPla^e$A?YrOb zG%l_@=u6y+jYYy@Ny7}fY&tk^to<*1!={{c+ycY!=E2@|4Amrrb40rvArTCcJNmP` zFu^s84^VOz!vR}E6NHcZ2S~V`d#mU0&UZn@(*WS{OjfS%q37sg71GZ#7ID1Q^At{ooewqzL9#0+*S3%KV6%wRlVgCcjw#W@*4!W!{ceG9(F}gKL=dU+@1xY4kGTS7#7QWiRg_)iO|YYG7YJ}1{u-4O^Tvn z3!nEW7%Ge97oN9JGw;>DqL7LQ*#@t8d3mYjAi75--n^TlTGhK6XdZ^`$}4ueP;UXl zU~*}J^?5trKa&h|jUHv*mpzIp*F#W73OarkxPcmy%ac=67*xVANln^pci7Q-%s(og}n!ZFvAl_|V>NyuMe-Ru@;UryjZOXw8C=D{z;h|AK*GYh<(4+7%PzdmDaD!TtS~ z2^~JKB@jFruR7Ogks+jQ1oGLD`+oWfpelAeDo!CiT0s&gp&_W9p5N7aM3aTz5SqEf zZZO_(eEF?oNLc^6CQ?D7y`*CT=%vZ^*ChNGewHO!Yz=Qa^Lm~H?~_LKSriqT>`X_n zzeN|s6-LAXD0l14o3*5nV`OEL@y{Gytz~L{BXr=@G0eNf}jgHjNu9c;Dfc+{)nJ4V+-N0Ggn*}_8E8!0`q^H%G*=68Dw@WGHF>=XIqqCwL2 zhy7Ao_ggqvX026JQal#nLpT`_kt^?lSPu2@nQXo@!=fudzwxob! zV+`|#1z>c14XtxXPu@0bpARRQn@p9BfBa+GaY4ca_du2E4h(`9UNwVR1jnHV=->P@INJblUQd9~=6 zL`FX;bYU3meL5lu296+t*NU2D-tn@T*I%(nFa6l(H0eYO`cT%v8NkC8Ebqb57Ppi zcH=|+p;VV4@+1e2U>3~d7lqCVOEJmr?*aeyF3|6)*8@deP!+W=KU)rI`q|#XWRUh+ zmrWlme_s}kxW7yyYt(_q#fYw}B;2fFIarr3Uj7D2#vbhEvya)4eKF0->)_3a4CVo7 z9yp_#qp+H;h|Z>JrJXR9>rYOKjV&e{_4N{(DgM)RVaT8y=wPHia8=FOSaDn<#{83W0rJAKo4ej34T5+}E$kT+C zSKDh!g=YihzamKSx-G~+X(kUw+k?|&|jCrbU3)+s}7Z(?o*^b@FgEFJ@ zj?~KX-(HI_V(0JN;bXHmO}$LJ@TsO&p8UCRC0=7Zdc{y&?V~7P1aD>2y9> zlRQoC>+Qja_k5|Z(&|e!s^qyfH9?`>cSwVChc>997cq3|*}i`tYdJmLyOaEfsZWJW zND+r$thfwZIzhT_J(27nCX@RlA4$kK^?umg2ihbt(!W5N#=_D9tmvF8SiMN4gB`L#7(P1-b2XE5^rWVk_MW z_lrKN?PespQJ)0T9TVRk0uPTSH+*a^=YIvP3|7<32^D?5uR!oo5g9S~7GH4qrhEgK z_%RR5xV|jfZsCT@MvKYFkNk!3wSY92wH|fJ^y?V56`ve3x>A$Ux4M5Hgqd*LEqp`7 zCyww*x2Vs^3_LXf(WerSBmnc+erEzW&@vofO>niAv5UP!-&Mk#KC=IJ?e!c*SmN;X zwqc#L1`6j@|L>djJcZKi(eo0%_M6`|4bC5aRA(FQ|3X5y&;^usbM<=d#DY4@B-dZS z-24u?zrbA4fk^j0?%}^Q&Ojp+8^q0`@7M`5O{g`Clh$Wymv2IJj>Bxj_h$F+pWexi z678x$3oeG+*ZZJc|3_xn9)es}`ROyDkDM!!NFR~ck z<(hShiLcX2CS12;kN8IBv)uSkE(6>yBy15MJLKe~#ypT?MpDf|f0443E_2g_CBw`CHrXo^9sAa;8ekGAh6-M#Vig5ROnP(mI^eR?x4dROP}L;tt*FJ3&k zo1H%#%V$@LufH7=J?z-uZ|@&oLpi)QH;ro^ zGPrTKoCzrq@6S1nX4W4Jw<6NqHa{|*Q$MI94>~=6Vj=u=LzsdqJq|(8fIlL;_Gb>B zg;j#3S>~LX+z`$hH!auXrxxMw7x_92pQP}DKJ%a_DHEG}iHrFWnM|!d z`^ip=_pC!&TNj^}?o^d1TnhypR43Rg?uaRCpT1vB6XSCpB|Dh)`MksDBus)|6J=#s z=3dNWD8bL4J2$%Rr=ufDpdBZ&oBWH>byQiAD2Xh7&&xR%6XAOsD9{+Z0zD_C#Z5XgizXUV?@`mWIBU0|C^~CpIf;G^3@)6{AnrvIyrWxwLQ7wqwY(1xfByNi!}8Fs4ee>os_Qz{q?PU9L@<`=i>%V*6}&j<8bKB6 zqzU0$NPK2z=I@=fqW9N(RMB>A*?9ctu_5jqQQz_QSDoLfJWn$1$2QCYi1dBAYKN}N z1dYXJz1M6yi%UvlDBP#3ln`FdJA@ryKPd~y*D335)>A%$)_e#F4W$1G*K=p*<8N5; z@d+YR4ybefb9P2@I})z zs=W2Gv-CyxqBMsYKOOeBlA6K8`eBB!1e5m6!Nb5|soXC*UJ-Z(DMCYul%lYfy7w=` zMA+<9nb^4dbG~}|#g@#F4AjT$_5e@R#3MMAPIg-OhOSHn}Sb0{HU*3+R`Q9CVjHA96aRD*{ySE5LLH9J(1pV z(k$%s+Up>&<)DohC(Iiip^%ZmKa7A!G(vSneFx2Gnh#AZc((RRPS?)cMG=jD#=Q+0DvVo&D-!g}%s zr&;G*=3Lefq$^|)qiMHeN$Bxoh#-D}o|jU(h$9`% zG}SRUDJSTe6?Yo>m2tb2jkR{?!(U301RLQR`u*Ms*=*l(!DY%t$TeB+qco`g27z5; z!gz^B#jnZ9ct1;lKk}@oqbR?0NF6gH!w7TAR-F&BiO%TYi6wtPKa&=9u-+;rRTF=p z5C^8(qA(=p+z-iCdFzSN|R?RdPLw@92^G zjWUbKv1wSO=1Lu3|Hg3SN)~AB`e%^_`Gl}RjVWKct?ma}wHcZydxF(|HHa8{cllfR zW;9Vh_{%eFGEWDV#`SPwJq3(F^aS`WRm{j$^7~6Ok&*1bbOd+aE zV1kwPsi5GQDVN7Uf>}@yRU0?Ie9QRduJ@bt$|laBQnJ8uPIh%qV(UxWve^C{#W3$7 zMUp|!bjd*!T+f?N0^ys!!FexKrXZanXrZz!&nam)RamQjgp{TtaL2BRQOC1q4z`c7 zV-P3vK77}R*iY+Rf!=iR+>mrtOkDKCTXC+x=)BDqn)5*Ubk=L{Qs$Gvb_Yjv$9 zfs%Q2ep9i%A)eYkl<{W*7$Oq2>;1MSt7J5rWg33vJI(OG)wTt)5=EM8q@zL#7)}!P zZSyyGzT1X2ELwguP`nV*W?}QTYGFRZvZ|=kbI#PfzS_q$>##br=GTbXOVXDa2$aT6 z$>tEtSn^mI(fSG>{>4x-Tn}w;`PrNnK$uAOB!f1cI;Y>=%ZiUm@SVS*n@+Kcrdb)b zp~)A)@n6lb#G*0e^a!JP4nD7n?1h_EjNt}t^@GVy(-}=0_uV>^SwT^4l!Hv{P zxL~7o)JzOIJAG_E zLWxM=vKxJr&qB=suko1<&R&G`V|Bboi689;6n0`a$d8b3E*t!oE)j?G+El1pJ_emX z_|dFs=YEQ=GxiSd$}8`k63GBH4XqM_cm0Hw`sxzBXhb!;cDM*L1@nVSXvewgtohfY zDvpTW!PJrpGS5>TTAZGjIlK}vCi*!wf%l)r?B!0c2srzjH)e-LF?7Z zkgla; ziwGu8)QwlRrR^~_SS4>VU&TR&H2ZWcL_@BFbu>&; zFflStLo2=cqLEW7f1+L*TrDO;IB9VPN-?WtMcyfnbo-C`zlAh)D?6mSM^XA1QU~_R zI-44Ojwe)~b9A)T`pKFyR4%~3=sq+_%-4dF*T<3@!nPK&^;G&R(r(~qe$ZCi;kgQK zKtqkR&3dq04Z_)KS(wH%&f!alI{#UYiXjgd&cP2tjWnYb%e-mz*ZC{usUgx29Kx5= zXcBc8AAMVDJ{_KCrosGfEV`?GNv2_Pv$kgfCQ83^guX96pUW;ea&|+;V97< z>1n83#Ri3&J{9#$CH82aqCp^{mwN+?B&IBlB$sHk{ZK*c=K^o|@<);7x1n^a3~%(U zn3V-8`g5Y^6loSL2gHPOu_7;ALul5>Urhe}0B69T0k=k1<}SufHTKQLVVC^~c2a;$ z`IXn!lu=400*Nx+xG+8P5<9}8YBqR{vG2DPc^D-?`rsWZVMZ^st6t-SLBcO#nO%Kf zIWgiFsXF%C`i2xWQ|G!}j_6Ct!aDDjMESYBI$z6=8V|~^uHSs(cikoMMCufrASZ6> z!=TB1;XYM%j7%h>VsDPM;2G%`4{!s=a_fV0w4-d>YRdL<&I1*Wl-2hq8W}iqWbB+w zh~3(Wk>>8G-sPtQ^^#jR+stLGxrJ?}GT zm`>qpF>33#MjC71Gg)+jMtpwJV%K_VNsiPGe@}~-&1KnE<*?rd*L!V4w~9^~KF}i@i`-~Fv7e># z!GDV~D+eHYp3;{O%^)D!1$5`Tk8dDO(vS$mur(VPlU1MHk?>c@((|bi**fBRN4*m) z-FWbM!SK~sumPE*LHRkwQ$&K`^*RRZhT<&a`{6>V$I&!z;n5w&%t3oRws~V}+abi4N5q6t1YkN}OL8l7B zXPW0Tma4(8hnmyj^(|@ZAMrp>Ew%7 z^8KS%Cfa(IoCmXL zZy@C|P#%`PQ)?(`%fSst`Z+u+z>BJt(^|kXj7J;;F`ak7($pPmu!i06K*nW@n9}EJ znb>6ba%S0`ceJWmXQhCcIjV9{F>PVLnM)!Y{BI&#(LQvU^uk_AaeJdg1s*>|CF~k7jSRD9X03t(dhg`2}yGVfWJu*3;C-MtKTHE$6;n)61-v-e-c& zUfyijE*M>`;GI(3{SDcNwx*d~q1|As3}(HThEXB3Y*i?O%s|}^0+1NkI)StS1 z;7bteA9sq#LLd^Yt)}6VBps8=G64>0`%iFqw+bC>+ji_SpYFHcYdU0 zV<2qUL$?ExvL4>J!y|}Y6eM5X-PJuiN`%jFv|O&(AG*2X=$~Jzvr(XTM02Q|5PK7y zFIHBl)BTQ$s_+ZlBQ0MQ>4Wtz$l-TvpX7S(70qapZkeZf9-jr#{`ujW@XYG~?PmBw z)C38ZnL&M--rsmH>1Y_4Y44e^y9Gkogh{h!6$H?V#9Yp*^=)puQjETsb=S?>V5o%^^!n83hBTMavkul)asgynCDqrsf=7*#@eb3_?$crtw9$)`!$dcrkGD)vU*{=LT$W(KWuCTl4Rb}kDo zpR{f3X|eq#FI}L9p3e(wLEnZ-R5ylbU!?BlY{Q45i%e~Xv4Z!6`)&F7_4foOw04d$ z)6o`sSa&r`C>$aE+M7I)$)04axGQHc8TdlgBw%`7THqq_-xV5X_Km* zw9t$}B1y@$HErW&CG+a5{ld~rH{K(!-IIL!`YFTZ>f=e%c<-X+&g_<;9z>(wc}zNjq05g**>3=)={zASOifK zNu*$}{O2$B_xa%y;|!e1pT3w+Uye5)i&%uvBtBZOY0{W=34$17e-nbWW7<7F6L$7O zoW1{kE~-vO2`4Z(C&y;$4W3{nty^y7A*gK>jyxco7!9)BvuLb2mVC|=;=|} z`(VTE7A*Xo39qK}UeZ7}HitKaRN2!*W(iY2?vzNHI}meOt$QRah&C0;=U$1(%Z-YaoFveat( zp^7JqW_?~$;d~`0VM9`%QK>O&P$E^6FUP&eh=VqBo=P-w3vpt`zetYMEmLh?m>$r! zYa=ur20ji;0fOOY{8jY9f4{54_h2*a+4W<%ml%((sC~g=iaO%`y1c-tL50V)V7|a= zT<6lYW7qaH7h|hOZiS|-`S`=p6cj7%Y0d86k}qS?25kFD`Q^G^-G^q@ z20=E+kd>zWO@<@p_dI&}Yo_&yoO#pwZivIt7P|nmOrJNQhFB<4kV5BhO_N?mm}sVX z$YRiDkS1Yf@guaPr`ve!KVIDE6Rq&RQ!MFc#2 zh+$)=+K-K@hNXn;Ni#Pw8&_#tZ-3?7mW>g{cBGeWFvZL5mLFo*>jxV8ehfHOe%C#8 zYpkT3mN+2Bi^QXU^XZ7h{X*dg)fE&OiO^-|{v&v{@>I`bFQC$Jf>F!J ztjGN|>0EA2AO}~i#AaGv^C#{zmxE~ztJ#LfLlFMwz6G}Qlr0}ZP=rOC3D#AlA;`Bu z`FuU>ndVjSOurn}r1d~yD}AB(YK);t46Aakm)5M_h{MY?jvV_V$LVM9QPd)=&XMv+ z$;?LX*I+oP^zKN=f%YMBtDC>VW*e4o`uLp@51D#@bmb(GS!a*1ln zB7K74Kc8QJjXo;LNEUsC(myw*Z4<2tRF9QhGckUAq!H2oATqJB_=4MXaXZ zRW%nQ8-9)DVbrPT=`VD-B@D|%**=1FT>% zOTS8x$_|N^>MK`gJiQwnS_kLySZ>1hDz9(w^0x0;WTA~ZXy%#mW?~F^qYrAd z5{YY1M6DzM3tnX)kzkXLJ&Jmzsn}cwRp*yCL+HQ*x5aG}Xay2YExTLF|q0g2#?n-Y6 zpT2Fk`*Jr?dJ{zw@qP6unEs_@@PIJh zzMkLO2J2q)0IYZ$K9ZQ}bA7aVpvsvaTc4nKL|GDT{?-V?Ur_jDD?{mCgk2oW<5<5o zNjX$vs+Glmlp-l|jPc|in*ri0?h4*Dy5elrA>_eMlA7$B+APma*kOfV2NA`06BH6+ zW7lC^stVTfQ@l?{3CA9&+B0!s4@P!~Fm#sMFHK2`M@U8uyOgxtj_XQSc;F-E{nIV;EY?c=<%`xU*m%i7%R7N z2b)v(FNZ{%2x!Xvw-zg{PM@dd%mxuI(~Hkb>n=TEBMtgPyGMVB??gw+R9str4V3Wu z9g!kfbAKYb^qtzg{E62Lakjc1QE7Bpmplny??Qsdj#eR#ZfcbFNOxP$D`in>6Sq@* zC4I7T?ibdz*Z!5q(xN%R@7hG*eY9k@_=Rc)KyhBtGTLj<=E5~HPnMC|A#C3>)s;N9 z5acpVOxVnvv6qHB^a{%Q+OKo7#lNg~pHed8xnte|n@QjCP&q+8X^r;P^{*GyHPs)_ z$t|W5W+KNdix!JM3}(84Go}2ka@Rkx=<8vo`qM_iPTJi1(9^rCv!KV}aY20%YG9pw zwzJq|rRj8f+66ZC5nL*d`swFzyFB}BnpKDe%rm@k*2b$ykVExlefu$U`_BxTh9f8H zJ!m#pmK}F(wFEgCL!Aa<`uZ#d=Pkk zKZc-40`lY2bxDAp;AJ3TA{FudifVbCtnu5M4Z7F@5if_{8dwg~C>C7b^ACB5;?C+> zal!}f3=O8jhnT@DsoYT3l$)s;T>|)P!%|^*7#)}I_?ZSpqx$B1 zn^@itTGTsWdd(ZPAX}kM4Nx?HIb`n6LC#^5rEU0t80VRtc1dkps+4i|z;L1ZMdJNO zhR5R>EqxkjeWBA`dTD%^3}pm%#BBb!5Q|FWgyOsO-zY0noZV(o?g1e`QcABcP4`wT zgt)ra)S;xWAHcA}auN4tzSx`~JCK({M>=9r>_HooJ1nJ|NCcN|&cktsl7#ttbGNg2 zqWtW@@6zl+vPe1wUTOj4BnAxFHp7vDxbMEEqUSjjOe%cY=6U8MDk#y0Ysp(rbJU4l z)dY}#w7P-<9kZ6vhC#YT*&L{gA(fArweq@aO^3SD*zJR;dS^=;p@28{Eu_J@KTZl& z(9KPgZaXD%4*jcgPGUjQ&wjXe?on1P6?J zDhbAd$_+fPYH*(W{jx81&3TeiIX7xB#MTm!ZDso;Ln$XfUB_Um^7?S3YcNG+(3dB* zYkHA&)IhXC>GHaI=ma;-w$UZs8NUae%k!$@%}KfTjGh)vqrt4_;{NS=t5ouFo?_Z=6hb*+@`rP+WrX|I)YBI&0=?K z-roEi3rv=w^+}T8MadD@$b?feji^M|*Wro=SwG=gGEbXbcyE$YKf*L-Ze$3>+cCzz ze6&!KV5nfU!$DI-1E%f=bD}e(2^$MOx%4AX{-)K_t3L=e`0COpksanOn+xQvnt^UR zak?KZY^6wrsHf+6!l8d~0|(ho^4+~@ji_1%a17RVy_in*%PN`0%;I<@7!%HVx;cLZ zGg3A%7M-kXb1s^I@e(%acagx6Hcw~WeuU$YnUlGc7(!uxPINNN(Se0}QOy1-tZ5+F zQDKFS(Z4#gN!g>twfdaO$d2(Y zq9Qy^9UAR7Y8(lfUfSP|(P_mg^N}u`lrD71r@J9>6TSV%ed%q)%voDKFdP2`yNip; z8@-hfSV@}~Gg%-IiGg#z2I4=1zQG1*wzaymrdN8|ZZJ8sbrVue{DwK<-Nvc*Y^TEV zxn?c9`tFM*(@wZy)Y}h0w>=CKH=gt9`?{GjNIg-UApN(>&SzASbJo=RV52wH+ZK5Z z_W9ReMOTF641&s#55YCdJ0U5*py6@}|FXV45_OWwUh8#gV(_2EZ($^OaCH_Zg z;35|~cx`*}i~Q-AzfsC?RmOv`5Uc0SW$ufXdG)`eloS6MI3A>9rOmmZAc1J)rK>vB zjzFA!@C@wBxLiXksp=bckeI!W&BQLs#dOmVF*H~^(mPV;Ob^P$$l?Rnhv&H({L!pe zrVSJw0HwRTw~&9nwQ- z2uBe7>!#=Wq>2LeK`TQ3y6DLM{pM=mutByeVsieE08@QWoY=FFVSmQC+s{&kCsO)Z z1zeVmx1|*)i9N&A)FT*?<&?3xYs(>%z=tV2B~v!XZQwfr2(gum!d}szrIgR3i~}l* z0vWEY1XWulRn+iZ(!j+)${4w;NwKZvWBa6p{Uy~ARZry0x;*9F1=W#mQHUUFT7JWE zRCVOlsSLOeq|0Yio6=Ugq1&Mb=cE)Rye)_JAWe!MF-)-L#y zzncwL=y2&=8mlaEJ^=qcZL6W2^<;^MqefVzY`oB>cc)7j<||hxBgR^lOMsR*gXTVui-!go6x$AFdTzrSo__CVS}H7RmNHiA z*EtLY53gs5ZsKYY-T9hYsBHVJh^8qzZZ4}xy(j@&AqBHmj!76pr7m_7GHRfXyR|qi zpoI6^y7pOPmNa;3Lq?LAev!|o$#zy*^IwX8GOIU(R>e+zevPyHy3A#e%3J&GLb1o~ zh6rK>yL`7)+XfqD5Jh_XhNU586!x`fj=~A-O5Jmtl7| z6lq%6)gW~NIwiVOLRw_~tIs*wLXqa+QJGSfJoAu6Q=IaTK~da*0f+(t{iEweii^kv zkEx1_O37Ep`c9uz@RyrGsw<60l}r}c)tKrYmf~7wesB_%{6oAGom!wQWJl(a5nL#p z6M5SmBQ%pHsUMk!rKp~$5Axjp+RrX6d+Ldwav8VLhW$mWI#w^MCSyD|ooUn5A$?|B ztXItv-E-%qJdM{I&`uKi>1q_`6pL2OBxJ8T&{Y@B2NgWiA?C6FcwNK?Xgp4GuiYwG zUYz{kjA{@ttg^_au+vCg;OIZF)m_gHEt=PXk-*@YX zt4P7OUeCI@GUGY_{T0C(7L*$$#An?I3pD6JBZp>&adwgp+VwQ8>LQLgl^+YCPbTbF zQ-T6ipDkc-L#Q- zG{?*coP|1DD;<94o7^2K6)AC9=v_E$0CNQgpKFdupSte1{{${>-!-z(07P%xwj3aj zlt^%N$?AEs*nbAdXPMHCo3N~+5fep`a(*w;)l7Y0nA@{pFg8Adi_s^&j9V3Ce znHad8DKAg9<0>-vz4q7`N@vYfH&Hgp6)~2GY5s};;Vi^gdj#`L5XifLTfs*G!`TV0 z%BVmeiL21Mx&j3+SRhRWOOo6{tld=J7nQcwspZEDIoWG7y#3y_4_e71dBNj#K3hX@ zI#nOEqXHk|({|TZk~{x;2g4LT|M`-_=v}tJl0gu~u|k9-7*c}B9p9g$5B#$Hig7^4 zrnuNS+-M^lwj4;9xuFT4wf}g&fA%5hW4-nN)81KzMb&)`ACVZEp*uyoyK5*VL@+=a zDd`rZb7(2)Zi5gcq@902cZFE%$n4yvYOe_~;Fa8s?dqzEmwtK+&w4dS zRCH5xchoO#B4nRWo=P(*mCW9(;h(l)HpEazl^$vKWTK&-@ zk$FvWXUG)!h*uvBxlJY6ajoMNUtrc2C7k;-9&XHbadB5BS0x&{Y%5Pj%;&f`Ox=GM z2gY^0KV+J-ESurnrE}eUDl==@zL-Mhd*{c6&(3tjhf|hEe#6-u>7o?h`Oa=RY0nfD3^nfeYl-U++Av{PkfW?tjw&Nur& zHDL%gg^-0VT?FD-rVP6Gq`Xa}25CLyr&ramE&FfnrY%m#AETR zaopddekEP&U&0jrm8zYH4kqA7pVb^>(y!AZETBDN zM={d}n&2)UZM~Y@WBYxfnI$@FmR|=ZI6v#uy!Ge29Sk()ZSt!S1aoNf&1IsYv|raO zhBhrmb9cs43j^_pR2KT(Wf+$u{r5eSm4m4V`sV(9o1p#)zLu62P~}D6FlzDm8ktOg zHEB*%y$@RQBK}>TehN6<`tu0IhZ2z)zn)b+7(C)3D9ZHbJWXOB!y}%5A8{58%>M63 zw1ZW)9?l3@_QryE&DghgYNT71`Nl-W0;sh>z@FoAv#nIhj1IIJHb3s+4@j&^R9%b} z9)VovYr20taHlP)%0Pw=tdz}DZ{5a`{?=&Orb>gexf*!B8rTWcSWZ<1pBlZVK9#tW zk;Lbq%X^|i@|!OQ`OF7-8)6Yox{}`-mjH{H)Ai}mb`I*=J6%LXz-Q2zvMmB?{CkJw zYBnkTh$%3WNE*831=M?+RA=2S zV*J6eS=~h#v;2(VrEf zP&Vf1f^@KWFES-mA(cNLC4FPp7U@q zswTH)(^=E@IfGk2v-$DP0y6VNq2qZCkP03k6|E;5&uW*(+x$7Yohv$j_i z+)DW?GgbFZuv{OK*RaoL_)794xfd&|(D*e+=gA`R(6{!Ba{EEc z#8b_#vfQ)>nHBAu=m*DW7z+3G>`Z#HLwRdA63$K zpA5ZCW?#*F8Va>$88;N{bXXAR)ZEBvzxc7eFu+}9RNQp^{%*(Pl&$*N%2NIv)CUAR zHDQLVs_DEEo|NXQ{jX87Fcc!b6;CfM#?yK*UW%o8=4Jo9M)ll28V;``_|~+V3*>f} z7Zg@_0bw1YAY>ha6~8hMo9?s>%YfIZQmBPZgWPgDQ2Yd?823cuRKTtM7F`}~@K8UG zVii!x;QAV{wZPj4BE5Oke&OGC<(Jxs>t7oyQl6|d-v8H7g8Cxvf1mx_5xQT}TMVQ< zLGFa(x3ohmWJk=e{<_G`d~iPR{GjY_T@fd@avNCQug`0>|Gpq7^hNx0Z;CF7^Y;}L zIy8UXv0(~>tZx{osJ^7UVz9EZo!iu3V+eSx1^2-G#^{C!_Q3jr?) z1j$Gv?}MTI_aM6g-s;bT^9G3ehEtlP^Zyzxbym0U@9+FX5kU}b@b?QZhWcz&l>P5& zQa$G7kkDY*pNKMO&h-6n)dr>k-%srg^2z>IZLdAnlm6kv$bR2kKA4uMWBghZsH4#d zJfkn_r&|rC$3VO&+Q1Bpod5C7d!aPT2^W-_vpavWu?uQ`M1s-TV}W01^CX(k2LNT zfOn6Qh@WrhB}WU;H49PfxsZ#qRe9y@Ppw&cpmItVd!8RjF1V)4v{jml;%ELqc!}|XDqM53DwC-nRoP-UlXyf9T*zO7`dvnH4G6!O5+LOg(+Bj$m7f+ny5&W+W9Z*B<`b8X4@NmF@n_A0Km zas+;qt|I;`WfzV;upnWN#sE9+4SE>t*3Y#HPA>XYx+a4qqVBGKAs{-DAj z@Pp1ioyEiJV02gSWn@u>4>Px!MWU8ev4&ZU3(?!I-;O>#;?H$=ng>?hyOThCj`-wg zo3Vc8!7~B3<^F0g$9qqyf%U!X|NZ^at0ZFllJ7;oL_k&XU~RhAhR!!aU4=}GNexjxp7$BQvV91&=V3Mmi4d+GwJllF-7TTH+1Tm#+6Bd`SO z^JjecIT%59h1ie}C|fGsx-LR#m|&6n>Im|-!CamFBp*r%n}b3xyTW6E8uxn$sZ5~Cw`y)2tNykpOr@7wKXFe%OESY z`TA?6IS_Lqlm}vjXl24@kxEu;q+}iXk^odhI!!9jXONHo3Vh;5sDR=G;UXo}P{av3 z7KF?{KWu0ZcOQ{tkG>1n$N#vDc?o4zxptpgc|yK`sAkPMWd#Y0Go84*Z%b$fOzu-O*^3Y5PN+y=l#C2|<*)tAsJv{;Pe} z2ZPpv9;QYlmvrCgopo-PH3IPoK_Lg=lSOqQs5&*S5G2>LhB$(?@`8>0sezy18|T2& zS&40GflDX*2zkad=tl`cT^am+%~M97|pHLLtexbBa3dql@}#vIkf|7F)_r;pC7vb`H37Bo8`f^(d02@ zkc{AUB|!O|Eu|>;d_<*p^kY+$@pBvLL~E7PB^u%tb)$V6`{xA`v#L9HoX|M*InX#_ zx!7O7X}>t9|Av=5=~~Z_{}8wSZHoAVN4H)wlgvIm%o8<*UGHp4+Tz3I zOej8G;?K*ur(npM(zHJ?xLhHU=ypAII1h7jg2bl@f=oO8oc`){LOnj>b&_miwRnK( zzk6IRCbQ0FQ zBr&B6Dvd4P&~d=L2NTdrjDI5uhhG^_EXae!?k3@lwCCRl+&R1zk-&1i1^oA2t$fw= zR|GRwBTB|Tc+%S~?sTlCTBX`KH1V>BQ^#VdszIG08YG+?T#CJ`!-rx#t9<>S&8j2cb)=eMGZ_Loqx@2`kS|j zrMDp+Eq7tn#Jl4(X5UJIL0A=JiIKsF^dDRdR#U_fc*OK73H-BAAkS`ctX~t~8_2WacRKR)J(qo@ zXfAPGHA&XAUWpSSyL!g9nbG}&*y&n9xO4o=_};Xhh|wf^&R?+lrqqSrj`&EcVfUDK@`8+9farH;!}DsSY0xQJ9PcpJ8gzanUbaZb@DmH5C9J21&NVj*q01c;V!vmTG-ZC`M;$ijpl#PqmPMCJ4KA&gSNPhsX6lYO%j$S_ z(E(pS=J61PFg7p5W9=DIi0c}5++1*zBO6?WGulVfz3jQv~L`bYuqsZ z=!u{3gNgE;7ta3n@n?QRf_>e#j(g;tG$S^z7Ba^{a(Fn7=X$InhR*kn!ViLb(`NiI z_HQ5(SUXMnz3bvM@PdDaru9@KR%P_*)nkDvQZE(d0CNSS(gGdJD4dSny!4@rc6dMR zBXLI+g4yvB2KQ#JwkJtqzejx1Dm9kcr&9ClKa$=c)cVajJ;H=p(x&qkp2(l?CM@0K z)U&}_gpml{44bj(2#AH4X>+lplvnM@$4oMPCt!hC@qEYj>e;z2#YEjH9dLHM7Wz)C zL{7(1Z@}nF+bPQ7mR6}|J)G6B=72O52Ak1; z(0BvQtgd`}e<{cf-#7Q^g623{p;ED8-_Bha5k`tCfvY1~`DpZ^a-l_`8a$WuNyT0G z)p884u5DGFA7@jTr|*h%ET5OqwnDnY9Q1s~LgQ;y7(O#JQf4c1Szrp=#R)u*Rn_(} zu~sKQ1~Yikk+HCGy~kcwqfy?^N`v-H$q(OpH-4j%xM^s%1}^n@rt31SQMLjwgPDi$nu^eD4ut_5Kirw>A@AO89W#h- z0{%QtNY2e8;_pT#cg(KCWEQ2#zOqH!zjfcN<1w0tg<7w#*`(QT1>MwSVu+w zOQ)*bHA_A35Cc)V(t0#fMj>rF%Z6JMpSCfp1OJhmHKC&G<)^KlrB?>%No=a=EVor%#oAmTPLcdqsmsgN^Aapy^ zomX+2l(}t6TW}(+T|YkTP>(v95jKM>qR`)G2ccaN@}Ws$RY&m2wsG>;YSIT2-y~G# z8$VyR+&eI^$syn0vq{~>@5>K{nVdh*HQu<4A&lV^w*4d%Ps>^2t@5R=)@Jn_Q|HQsyNieNeuoYk%+ht z-mAG^sVo0kSkOQ+P%>|fXZ(pGK5N3L>$Nhe0H4go5N*mmZu-Z^2FJH~mV}ccIUDpC zt~d+wB#+Rk#*ZGV`$(oj>9Ux!R00X0ArQ98V3=|6jgB2>J0n|!xvG#zsJ|MZR`W@LhZQ%j9`jvOu@}Z+I6@&OXew(HmrCiasEdO` zmX&G_6FLH~GNxQ;Mn==Qa}b?4QZko^4&zM!5l!TSo8BSR1<5Zz-+=1A6(mdUB}rx6 zkVC(MiBpI*!nW|Ce7nNMJjs-Xqry8?l{}TM6HE3|`L0MbN0{xwji4KMZ-p>e^cL>R zW}C<%hSxFcUH$dj!n}n!uY}V~^2cD)j9=lZx9E8@N*nM9roS1^;IkQG6)|o>t1qyA zRQl2~a1+j)ETN_EGKJ*@k0Zyd$C$TicSp#I-+lo1NIkqh`bY%R zPQ$_o`axH_n#h9odvD90kc+>^88Nc!-uT*Z>Phf?!hB)`;mQM=CrTbH$qttCQ@B5H z%{n$r*>XpogxfO*9xLJ4eI4xxDGz!)GKGQ3vm6c&lX2W-=t2BfcX!oiDSL4aU7IWQN;pm` zgL8&?Ziq(m)gIhrC7esebX>Q%bhAxyX_Q|1C=<%6x?fgI=!H&p3J_K5JS9@HewmlD z=2+0Vcw~9=uJ5Js;@v5Fa%!;-XVs;pQDveyFU2anf+HCDC@ricVpuoWm8afZx&=LR zGGz&40Zo^gWrw0;y)*Y>FRi`Xe(?;3B@pJ}WR{H2*!d~uvzaS_$hWok8IQ4f)cuVA^JcD8CGCt;?(oW$(7w&rUb}*%5{l$HWf{G(@HR2k);1I`U z|Ggh+Z3iV1WGOv!Ep*R+RgTPT&b=%4NDplJ1 z=QK1oMIwkTVhErc*qijH#qs$CJtAcIY~0&C8ly;6;|Sg{+v_gR$%fVG!>g7AXzjL` z@P!wUbA|W&`Ry1u2Oi9jd*oIV)?B_I)XgHiDYdY>%khL z_yo91{Cp&QIBaKv(};9MF6V4nh3R#=f1~zB;l!?R`{T(d{l)Z~{^Jn#FMQYA&7m;< z&8Rn!nKF;!g{-Any>q=Mmh5jU%V~G)t3wa=1ZT)MBR`Is6t@Q3h8yobxxGOWp(S)b z{UoR!7hPaQY94zA{g}BA@y2YI1$wDimhXA)iIi{I7orQT+Ih!})D@8|Nd} zjeeb!rDa7rX_g);T&<4^P3?nx&E37&1f<<=ZIQJZpORFUB6c{uOzHl@YnL0-+RK+b^`aSa^9U358H-0NPW$CB6x z=~zSH5NR8HnNWg>0lRsT|CG7hT<@4}iq(_o9LIAI6_&BhgS3j2etFM%2d7(TPb<2* zyRv+nONFft>*13WXd9b*hayL>_PxP-LT?`(Te!z`UK+T2SRluAv6xk|T}{Xs1EK5Y z**irAR@0q#x2G6&LwzH+N5%5x)Dvwcsytq@lJ?dYji&VdI zKEm&pXTR{8tn(RBJL{oBhSuzhaSrBwN0A*Vz;%oTOx_9M&4h%` zbGtf78?PLeC((+eSJV63_j<&XH)K{brx?fLwk?yNU(eah&N>ON*F=72DJ)r9#2)5# zrJqowi?I|Zf%UyV;mwHCZhlUjb(-GPgQhL{)KOH?ND6FyXVId6CoH6VD($?HZ=oS8 zc(6H&#^SE`nMp`Mgn&78OXX%qr)hv&Tg*Q4tLcT5SkmEq;KfvVXYg`d7TImn<4#KJ zuThKAa44D5nUxa>$-Ht{w_Pu15&8>#vk{^p%MeTL@M~8>>*zEiG!U|4$uw0{=mbn$ zTn_NJk~PeoD}$J&ZnT{^olqM%+q!r=Qi~@D>aZ49uz2$6s7*Yws0&%qqJH*aN;H*O z8%iJ?VYQ^Mo5%}U&m>N%F{e?1b&mGu-AOUD4{ey)l^!QfMPi%d%&tzqmp+&v9Sgmy zMXhC0qIuQ1gjr-tHhDFP=&ij8O*qU85WQD^=4(xaFZAO)6e9Y@4M zwoJO8^srn&MpST&?~{nD&a6gv3-AiUPr__%Kjq(ko>089VDQea$N5@U*nGM+cA@6o zQj;$Y`OJD1nmzGo%`9WgO2zyE=zXk<#P`@T@tGrqqW#i7OJ^svPeGw*L-wj_LaNMf%M+DW9Ep_ zeVr17izbbnNU;P3K@tW5AB~wGqgCr1bx&~5Is<0#$k27GjW8Lw|GzhYqBp;`CYD;1f3Qcz2(2ej^|DnW6 zbIJ_O)x^M+J;yIHm!MR(hsu#oVJN%WikM%YB0D9~>Yuwm6srH$q&l_f1^*0m5Y|09 zt9Vy*3_}y56&e^|{)x5_3!V;(RfgoRhGF@$Ws%zvFh!7my&UmEP(O~DV2D)xzL4@J z3B(Nl{grm%3+%)FwkJ-Tpp+2!p2%lm>g>t3pYn~X; zPLG$PtXf2ZV_o=5Y7a3tL)Y9&T@+%K^o!;|G3QU${6a-EuLiG)@n>a60qPtt)1 z^%ypbAZ1Zgsvgwg(CQ&3=7M|?05TQ*dSazjzgC^M``H{ zvkfB91m0GOV4Z!7U1z5WGJg!WhT-nezN4~8C(y-$+=bN%96_Phn+fXCYq!ICDdLD} zRz->j%+XyV0t7Mer}9kBMPeUE$SaFx*hfW-*mU@5uB(v4?w;>N%EyWsaGt0Xiz>HS zmeVLFzP+=|`XztS@x~h$kv&|Wh!!uf+tLRS-V^HL8!Qs6VB}zg%kg@9z;_nzrV!AH z2ir#OsFDe<;2@Jug8KFBWY!MyV{JIDx|57aD$?hf;44b%s8NH+pJ|kS_j$T7Ijz}4 z`Mx(*>SlNR_@}Kx{NY!N*fTA!W?3@ON-4m@tmS07g&eA;YJ|nhm8p{Y1*1A`zKK>Q z`W4+M0;5xkE@tEYS8Gcyh3-d%rbj$ru@zKgcE2SUc`MU9_QzU9 zNwCUgY#TArm&BSuTlh90K*=y;u}myGN|(n>2;q>Zg|Un&T$@ij!BR|3<9U43S%!cj zKt4dt0kfIOEj;Xvsy>xhhl}Z(k`dg&d`0>z!AcVVTUC)r^JIEXKMb ziM_#a{tcf(ETp3kHCkdcq{3rFkdZ5~PjhY<)(^4K9Ejyze5XrE7HMHd>(Rd>8=t7Y z_(6rzn5;c*GcLa4Ac?vZT_L4}mDqH-wve*IeyzBJa5s@SGGndrrpGH9g7;`>)b}-D ziuze#OQ%j|Vpj8LqI|+QCdu3Y3^@KyFRe-0Ao1mh+C&!^l#uFWSGKl9)4``C7qRt% z9?l+(p7oxlp7iw|6*^1l{cB^%L+5Z3xU)z#R}@rY9Clbu553%%W_jIG!2_pVHjd$w z{xuQiIOt}GEiX+EESkw;tlApg8Z2!t&q9Z7@9$gFgo%9->hwxai;ch=m%Y9A{Djo&M-18O5&Nf5s@Mh=u`L4DC4)p zy-qY(WFPW1RNE0_^DTLXLxAC{hBNGp;UW?>)Ir@d27AoKcr9o&NJ5oJZVXK#nG+Yj zEu28`HA-fQUz=6?YF`Jkq>sD+LoIcUiRR9bic%d_980$tjkLtj!RR>EwK!fD-|?)e zyNFk#J4$P25bV~jecG+i+Bl9#l40+XmCr}r@y9RVd=(f3jGG}?L~-|N*v}|?&hY1t#qPM}h#~p$_&z`{AmYe{+_N75C-$>EL<}PC=?Mk#h(SY9Br2)c$b2sAW z$Fdc(V9o6-rN&`e!yXVAAnb67IR-IBTCdjHIcGj~_hcwwN~o{U=?%h6Hqim5L1NzP zx?{70UCv(3?GXDq)cvNZK%_m}7{#foei7lD$pX@vmH;O6GP-6z-{22u?UxV@egR5I zhiWaUxF}yrs2wk{eo#W1ANUgh_lm{}&BUZ3mtYv9cNMCYFuGiMu~$>E7~`(cDF~tG z+2!vT!-(kAzvP6=#cw$TD%0OpWKTL#Atz>4Y&0VQxNXXK5><^^KkGo^L*I6qx!(T%rdYbW62}9Xq z#k+7!NrY;V@~fShxw{Cw8XKs+>kg3S%QUSL-;6G=c2d}(F|r1iU#81wG-;b?gor_V z!3qfKdoeDPvMf2^nJ>r0?@qUY4s(l-4C+`ucUUr%APjsvIKllUAcb_o+%DrJ=Es*v zvLCS=U9PJ~k6yYkap-E_q|o!ajXwJ(pqg|8XW+p=K!AGy|K?>y0@IKR7Gm0J$qn@L zMWSlZmm!9i)s}h6g3+TNn$aHsw}D$tG1bQlsL8%THpx|9Du~@Ox+Ce_`u+AtMHjG}zyEgMweTk#0Sq3Bxe_+n=KY6U z0so1^*~-7h!R3 zLt3{@q)+6S9fHja?!P#z<(B;sJkW}(ALLRHzWwhA)6ZM1<*sTG`s-`=r>e~hDk?ZH zCLV%|dVMgkS@|370u04ZSPHO#e;y(6zoV`It@(4_4hDLA_56#vI*b=<{QU@N#$E>L zf|jcPB)gRUM8e?e|CPu^0f~Qq4KP&!e^N!XKytv}PqEyeh`ebDLaNi)jDC-Qvt;lI zhSmSVnoYJHzWSR71IaBc|2`sWkNs~l?qL00*Z+#-s>h6j?B!}ud*>$^7ccoG{qJk* z7O94VPki~8n*-Q?+uvswx*~s4ZjAm1x_^H8=A-Pc-*c>30n8tHR|Sxpx^l6MXRdvCaW5TcdK9?}eiB*DCKg08Wd5Fs>Z~sT<@ED@5b_s(f8KYT zI$)Xs)Y;AayKff@<{w}bhKmwRkR$Rpb7>TST={px8$=S)|2;QM9xjCc1hf|(|L*sd z2yDPF%`{#JW{8NrVDoM)ky9aXgKl7W36f0m)AJpJMCB$9T z$<6-#6@YP$sZg$5kBj<+^8t*qumvieI^uUN$gl!gBJKK}wE!R^3}yr!)dl6BkY9fVI|6}IAH0P)y_J3$jfD-?!R_vE|U;G~WV~H!jd#wE<+Mvl< z|6kMD2f$xH`Cb%v_SdUfj^?iXhu{QY@Lwqc#VSYuD3XE#8$SS4UGSgws1^WuSO2=k zeSjwZyyd!6XVo?%%1vN3%=ebxBH4d2uZc5@e@`0#NTU{(ktn-*LDmUo1=**EOL2`Y zpdwQkitR$lZva+=3)n4hcJ%>pI4*!g2tmG3Y~BKBs*J}7YQP#RuZA<-BX8pJh+hid>rj z3$=UPvjtwI$wNQ^FAB|qc6u~e1TD)CBCttKd*_ekQK?8hp zx>qg=bQ9(~13?h@pwaBpXO%R0x`oG^Qr_R_K+9v2G*XcAV|Kipth4Br#%J8Q@vMp= zt>ElalUG1icO+u~6$1z(oasS6zV?h+)e#W<4CV~D)dJAo8kNiNRzCbVt8AfJChObB z=12e{Nd`ph*@O<8Nk`o=@hL%)H_W+%wr>{KT6Q||7G5-biXJ@UwBSs7& zNkOR-LXfS}FCUJoZy33n7btf@rUdG2DT79P`#Q`hMs{H(%`|>cM!KG|>wXL(!lnVR zu)QgtoA&&IZMwXhdb?{8aqseZ`!!&qaTXs zoi4WNbG|u5D&xL+b!8pX-)k9rPbFlvcVA;??qR^2+Acl`)k0fw6mB_Z!c!HEzT7OOK3V?n1+LMEuMAboOf6`Z61}0FEc@ zDP1XjQNU2}@wA`qfhQfL8OX+%=lxP);e#QQW%b11>V-(Nu{n_g2$8khX9|!t8#=yv zpScxb-&;9e%h^Dii-VAp&APUNa}?pPh^$=@XD>lr86oVTgNS$RkHAh1u6x}wVbHy8CcQqLx++<3E)M=bOFpS;oS94i9sj!)`bEgr+hOi@ z?+vqh_7BG1AN?x2K~}|_-uW!xPGY?vKeUu=P$o+gNPf&w5-&>X)$1J}~%T1WfGIfudz;x9#P$ zLLyzKJAUoy>@nxi*y|qi!SEIe|IEmM} z`iNEJK0nxecv!)}b;KLvgFEn}$gRDph2frld(&MLSHzhU-kw9_(K2JIJniPwHQFcg zwBK^(ucmshtUN3d9CUw5ipp5?tH7ZCo}#B6HAvNg3izgxQZ1IGnb4%Yz5fsLQ9^Lu>0dVlN%;JeE;S{mr_Z=mtM&Ot+bl zHjs6dF-+ydfbs?xg{VP-6d`;NCsxEiuXkT3@Z?=%_M68Du()Y73UF!!D0ij`FMex_ zo78HydnVovfW2MnRc|9(6M$~6ABY-w+326;SfBA?7x9CvL0G@xE<~53Wb6JnFxfoC zT!YZrWNCAkfL5nXBPHWmKb1&Ar8+FRg`i(`EOR|>-S+3_W3O%odu2R)?(_4ETu8xfeltL8pv8@eJPOSwIqaGd1m_Kfsr;-opzp_ zEMEeR=!1In^nrs2S|(kI+kW`_Ot^lgV>*@ z2dP>}u=<@yRAT`8`=2z~v}-Q@>!(x4+aPJaF*`p*eKtlc{qyS!z<&2wZT>)Qh7aO! zeFObqyH`Mq$q{SHsm&bkPyb|W9s7XPv}|6)jp=^RCH7r=gKyz-m$A*ke|?<5|0+{Ku=wQ&l{5T0e_Gf08=nDsRp_0o5pLu6b5qAn)iy!WUm_ch4g z*wb-$D&uK8+Z)_R{VOqLfRC4)>F0zd`n~x5RaoN@glLxDlrFjf1HcyX@My_`8 zjh({kn{ECem1c}tJH@$uPN{Pa4P zY_rAXX1_;*@?C`vW$`?6&4t?#KtjKB@VfO)Ib~HN*Ut43vT4`2vi|ORO(l{DD zP;{G~;{KyC=U&8|L+!A2E%%<$1NNP44n%aE_ukDDx$jC%7s^YrNO8Jnb5(6i@zR-J znWa((X6dHxb8$mc1{x^0w@qY% z?ohHy`|G6n*fpgLG`gd#`7>)H?VUM}giEgNbIyEE*pG)-W@C;O zBIBb#OH zDoZd0LvCwJOM0fH&C^bL^-|l2q=hG6)ymwO0ZMU)J|^XKIFjD$ZkvAL93|g3yq-N! z6Y2C0#NSRjWE>Rmf~JukTb1Pn_!@l?mtTIYyy#ZAb|-;K(1NhJ38(5 z;k?qUK|BZWoz;h4YY+|n8NeUe)W+=9Zz2K?Yk?j;f7f#M#UeL+Z%d^;!^RuMZLVOf zc=(E?&e%_E*Hz@-yhomw@tSy)DH;)B-{q{?A-J+7_$|jKk_dZ>9$Bed(G2+$Pw9zr z|Lx+#|04(OSDF_pM)FtU+uzHQ|BO%kSxo)sO6`AMD?mB_{}231Cj5_#=0D4+{|AT2 bTIu)#LTlvy{tgQ@8aOmmv|*J>W+DFvp{t}y literal 0 HcmV?d00001 diff --git a/multi-data-centers-in-one-city-deployment.md b/multi-data-centers-in-one-city-deployment.md new file mode 100644 index 0000000000000..76cda123e22e4 --- /dev/null +++ b/multi-data-centers-in-one-city-deployment.md @@ -0,0 +1,132 @@ +--- +title: Deployment Solutions for Multi-Data Centers in One City +summary: Learn the deployment solutions for multi-data centers in one city. +category: how-to +aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/','/docs/dev/geo-redundancy-deployment/'] +--- + +# Deployment Solutions for Multi-Data Centers in One City + +As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (hereinafter referred to as DC). This document introduces different deployment solutions for multi-DCs in one city. + +## Raft protocol + +Raft is a distributed consensus algorithm. Using this algorithm, both PD and TiKV, among components of the TiDB cluster, achieve disaster recovery of data. Raft's disaster recovery capability is implemented through the following mechanism: + +- Raft members are essentially log replication and state machines. Among Raft members, data replication is implemented by replicating logs. Raft members change their own states in different conditions so as to elect a leader to provide external services. +- Raft is a voting system, which follows a majority protocol. In a Raft group, if a member gets the majority of votes, its membership changes to leader. This is to say when the majority of nodes are still in the Raft group, a leader can be elected to provide external services. + +To take advantage of Raft's reliability, the following conditions must be met in a real deployment scenario: + +- At least three servers are provided in case one center fails. +- At lease three racks are provided in case one center fails. +- At lease three data centers (DC) are provided in case one center fails. +- At lease three cities are planned for deployment in case data safety issue occurs in one city. + +From the conditions above, you can see that the native Raft protocol's support for even replicas is not so good. Considering the influence of cross-city network latency, three data centers in the same city might be most suitable for a high-availability and disaster-tolerant solution of Raft deployment. + +## Deployment solution for three data centers in one city + +TiDB clusters can be deployed in three data centers in the same city. In this solution, data replication across the three centers is implemented using the Raft protocol within the cluster. These three centers can provide read and write services to external at the same time. Data consistency is not affected even if one center fails. + +### Architecture + +TiDB, TiKV and PD are distributed among three data centers, which is the most common deployment solution with the highest availability. + +![3-DC Deployment Architecture](/media/deploy-3dc.png) + +**Advantages:** + +- All the replicas are distributed among three data centers, with high availability and disaster recovery capability. +- No data will be lost if one data center is down (RPO = 0). +- Even if one data center is down, the other two data centers will initiate leader election and automatically resume services within a reasonable amount of time (within 20 seconds in most cases) and no data is lost (RTO <= 20s). See the following diagram for more information: + +![Disaster Recovery for 3-DC Deployment](/media/deploy-3dc-dr.png) + +**Disadvantages** + +The performance is greatly limited by the network latency. + +- For writes, all the data has to be replicated to at least 2 DCs. Because TiDB uses 2-phase commit for writes, the write latency is at least twice the latency of the network between two DCs. +- The read performance will also suffer if the leader is not in the same DC as the TiDB node with the read request. +- Each TiDB transaction needs to obtain TimeStamp Oracle (TSO) from the PD leader. So if TiDB and PD leader are not in the same DC, the performance of the transactions will also be impacted by the network latency because each transaction with write request will have to get TSO twice. + +### Optimizations + +If not all of the three data centers need to provide service to the applications, you can dispatch all the requests to one DC and configure the scheduling policy to migrate all the TiKV Region leader and PD leader to the same DC. In this way, neither obtaining TSO or reading TiKV Regions will be impacted by the network latency between DCs. If this DC is down, the PD leader and Region leader will be automatically elected in other surviving DCs, and you just need to switch the requests to the DC that are still online. + +![Read Performance Optimized 3-DC Deployment](/media/deploy-3dc-optimize.png) + +**Advantages:** + +The cluster's read performance and the capability to get TSO can be improved. A configuration template of scheduling policy is as follows: + +```shell +-- Evicts all leaders of other DCs to one DC that provide services to the application. +config set label-property reject-leader LabelName labelValue + +-- Migrates PD leaders and sets priority. +member leader transfer pdName1 +member leader_priority pdName1 5 +member leader_priority pdName2 4 +member leader_priority pdName3 3 +``` + +**Disadvantages:** + +- Write scenarios are still affected by network latency across data centers. This is because Raft follows the majority protocol and all written data must be replicated to at least two data centers. +- The TiDB server is in one data center. +- Application traffic are processed by one data center and the performance is limited by the network bandwidth load of that data center. +- The capability to get TSO and the read performance are affected by whether the PD server and TiKV server are up in the data center that processes application traffic. If these servers are down, application is still affected by the cross-center network latency. + +### Deployment example + +#### Topology example + +This following example assumes that three data centers (IDC1, IDC2, and IDC3) are located in one city; each IDC has two sets of racks and each rack has three servers. The example ignores the mixed deployment or the scenario where one machine is deployed on multiple instances. The deployment of a TiDB cluster (three replicas) for three data centers in one city solution is as follows: + +![3-DC in one city](/media/multi-data-centers-in-one-city-deployment-sample.png) + +#### TiKV labels + +TiKV is a multi-Raft system where data is divided into Regions and each Region is 96 MB by default. Three replicas of each Region forms a Raft group. For a TiDB cluster of three replicas, because the number of Region replicas is independent of the TiKV instance numbers, three replicas of a Region are only scheduled to three TiKV instances. This means that even if the cluster is scaled out to have N TiKV instances, it is still a cluster of three replicas. + +Because a Raft group of three replicas tolerates failure of only one replica, even if the cluster is scaled out to have N TiKV instances, this cluster still tolerates failure of only one replica. Two failed TiKV instances might cause some Region to lose replicas and the data in this cluster is no longer complete. SQL requests that access data from these Regions will fail. The probability of two simultaneous failures among N TiKV instances is much higher than the probability of two simultaneous failures among 3 TiKV instances. This means that the more TiKV instances the multi-Raft system is scaled out to have, the less the availability of the system. + +Because of the limitation described above, `label` is used to describe the location information of TiKV. The label information is refreshed to the TiKV startup configuration file with deployment or rolling upgrade operations. The started TiKV reports its latest label information to PD. Based on the user-registered label name (the label metadata) and the TiKV topology, PD optimally schedules Region replicas and improves the system availability. + +#### TiKV labels planning example + +You need to design and plan TiKV labels according to your existing physical resources and the disaster recovery capability, which improves the availability and disaster recovery capability of the system. You also need to configure the relevant `tidb-ansible inventory.ini` file according to the planned topology: + +```ini +[tikv_servers] +TiKV-30 ansible_host=10.63.10.30 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z1,dc=d1,rack=r1,host=30" +TiKV-31 ansible_host=10.63.10.31 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z1,dc=d1,rack=r1,host=31" +TiKV-32 ansible_host=10.63.10.32 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z1,dc=d1,rack=r2,host=30" +TiKV-33 ansible_host=10.63.10.33 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z1,dc=d1,rack=r2,host=30" + +TiKV-34 ansible_host=10.63.10.34 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z2,dc=d1,rack=r1,host=34" +TiKV-35 ansible_host=10.63.10.35 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z2,dc=d1,rack=r1,host=35" +TiKV-36 ansible_host=10.63.10.36 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z2,dc=d1,rack=r2,host=36" +TiKV-37 ansible_host=10.63.10.36 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z2,dc=d1,rack=r2,host=37" + +TiKV-38 ansible_host=10.63.10.38 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z3,dc=d1,rack=r1,host=38" +TiKV-39 ansible_host=10.63.10.39 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z3,dc=d1,rack=r1,host=39" +TiKV-40 ansible_host=10.63.10.40 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z3,dc=d1,rack=r2,host=40" +TiKV-41 ansible_host=10.63.10.41 deploy_dir=/data/tidb_cluster/tikv tikv_port=20170 tikv_status_port=20180 labels="zone=z3,dc=d1,rack=r2,host=41" + +## Group variables +[pd_servers:vars] +location_labels = ["zone","dc","rack","host"] +``` + +In the example above, `zone` is the logical availability zone level and is used to control the isolation of replicas (currently three replicas are in the cluster). + +Considering that the data center might be scaled out, the three-layer label structure (`dc`, `rack`, `host`) is not directly used. Assume that `d2`, `d3`, and `d4` are to be scaled out, you only need to scale out the data centers in the corresponding availability zone and scale out the racks in the corresponding data center. + +If this three-layer label structure is directly used, after scaling out a data center, you might need to use a new label and data in TiKV as a whole needs to be rebalanced. + +### High availability and disaster recovery analysis + +The deployment solution for multiple data centers in one city guarantees when one data center fails, the cluster can automatically recover services without manual intervention. Data consistency is also guaranteed. Note that scheduling policies help optimize performance. But when failure occurs, these policies prioritize availability over performance. From a275324733bfe5a4467ac7381366d2161493ec09 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 17 Jun 2020 18:48:27 +0800 Subject: [PATCH 2/9] fix dead links --- TOC.md | 2 +- faq/tidb-faq.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/TOC.md b/TOC.md index c62831dd44623..7630118c1c2ec 100644 --- a/TOC.md +++ b/TOC.md @@ -104,7 +104,7 @@ + [SQL Plan Management](/sql-plan-management.md) + [Access Tables Using `IndexMerge`](/index-merge.md) + Tutorials - + [Geo-Redundant Deployment](/geo-redundancy-deployment.md) + + [Multi-DC in One City Deployment](/multi-data-centers-in-one-city-deployment.md) + Best Practices + [Java Application Development](/best-practices/java-app-best-practices.md) + [Use HAProxy](/best-practices/haproxy-best-practices.md) diff --git a/faq/tidb-faq.md b/faq/tidb-faq.md index 27a3302402d5b..edb40ed97e8e6 100644 --- a/faq/tidb-faq.md +++ b/faq/tidb-faq.md @@ -53,7 +53,7 @@ At the bottom layer, TiKV uses a model of replication log + State Machine to rep #### Does TiDB support distributed transactions? -Yes. TiDB distributes transactions across your cluster, whether it is a few nodes in a single location or many [nodes across multiple datacenters](/geo-redundancy-deployment.md). +Yes. TiDB distributes transactions across your cluster, whether it is a few nodes in a single location or many [nodes across multiple data centers](/multi-data-centers-in-one-city-deployment.md). Inspired by Google's Percolator, the transaction model in TiDB is mainly a two-phase commit protocol with some practical optimizations. This model relies on a timestamp allocator to assign the monotone increasing timestamp for each transaction, so conflicts can be detected. [PD](/architecture.md#placement-driver-server) works as the timestamp allocator in a TiDB cluster. From 5a1cab1fbb5f2526e15c744938287137f3d97e80 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Thu, 18 Jun 2020 14:04:28 +0800 Subject: [PATCH 3/9] Update multi-data-centers-in-one-city-deployment.md --- multi-data-centers-in-one-city-deployment.md | 56 ++++++++++---------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/multi-data-centers-in-one-city-deployment.md b/multi-data-centers-in-one-city-deployment.md index 76cda123e22e4..bd3f235803ae5 100644 --- a/multi-data-centers-in-one-city-deployment.md +++ b/multi-data-centers-in-one-city-deployment.md @@ -1,51 +1,51 @@ --- -title: Deployment Solutions for Multi-Data Centers in One City -summary: Learn the deployment solutions for multi-data centers in one city. +title: Deployment Solution for Multiple Data Centers in One City +summary: Learn the deployment solution for multi-data centers in one city. category: how-to aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/','/docs/dev/geo-redundancy-deployment/'] --- -# Deployment Solutions for Multi-Data Centers in One City +# Deployment Solution for Multiple Data Centers in One City -As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (hereinafter referred to as DC). This document introduces different deployment solutions for multi-DCs in one city. +As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (hereinafter referred to as DC). This document introduces the deployment solution for multi-DCs in one city. ## Raft protocol -Raft is a distributed consensus algorithm. Using this algorithm, both PD and TiKV, among components of the TiDB cluster, achieve disaster recovery of data. Raft's disaster recovery capability is implemented through the following mechanism: +Raft is a distributed consensus algorithm. Using this algorithm, both PD and TiKV, among components of the TiDB cluster, achieve disaster recovery of data. Raft's disaster recovery capability is implemented through the following mechanisms: - Raft members are essentially log replication and state machines. Among Raft members, data replication is implemented by replicating logs. Raft members change their own states in different conditions so as to elect a leader to provide external services. -- Raft is a voting system, which follows a majority protocol. In a Raft group, if a member gets the majority of votes, its membership changes to leader. This is to say when the majority of nodes are still in the Raft group, a leader can be elected to provide external services. +- Raft is a voting system, which follows a majority protocol. In a Raft group, if a member gets the majority of votes, its membership changes to leader. This is to say when the majority of nodes are still in the Raft group, a leader can be elected to provide services to the outside. To take advantage of Raft's reliability, the following conditions must be met in a real deployment scenario: -- At least three servers are provided in case one center fails. -- At lease three racks are provided in case one center fails. -- At lease three data centers (DC) are provided in case one center fails. +- At least three servers are provided in case one server fails. +- At lease three racks are provided in case one rack fails. +- At lease three DCs are provided in case one DC fails. - At lease three cities are planned for deployment in case data safety issue occurs in one city. -From the conditions above, you can see that the native Raft protocol's support for even replicas is not so good. Considering the influence of cross-city network latency, three data centers in the same city might be most suitable for a high-availability and disaster-tolerant solution of Raft deployment. +From the conditions above, you can see that the native Raft protocol's support for even number of replicas is not so good. Considering the influence of cross-city network latency, three DCs in the same city might be most suitable for a high-availability and disaster-tolerant solution of Raft deployment. -## Deployment solution for three data centers in one city +## Deployment solution for three DCs in one city -TiDB clusters can be deployed in three data centers in the same city. In this solution, data replication across the three centers is implemented using the Raft protocol within the cluster. These three centers can provide read and write services to external at the same time. Data consistency is not affected even if one center fails. +TiDB clusters can be deployed in three DCs in the same city. In this solution, data replication across the three DCs is implemented using the Raft protocol within the cluster. These three DCs can provide read and write services to the outside at the same time. Data consistency is not affected even if one DC fails. ### Architecture -TiDB, TiKV and PD are distributed among three data centers, which is the most common deployment solution with the highest availability. +TiDB, TiKV and PD are distributed among three DCs, which is the most common deployment solution with the highest availability. ![3-DC Deployment Architecture](/media/deploy-3dc.png) **Advantages:** -- All the replicas are distributed among three data centers, with high availability and disaster recovery capability. -- No data will be lost if one data center is down (RPO = 0). -- Even if one data center is down, the other two data centers will initiate leader election and automatically resume services within a reasonable amount of time (within 20 seconds in most cases) and no data is lost (RTO <= 20s). See the following diagram for more information: +- All replicas are distributed among three DCs, with high availability and disaster recovery capability. +- No data will be lost if one DC is down (RPO = 0). +- Even if one DC is down, the other two DCs will initiate leader election and automatically resume services within a reasonable amount of time (within 20 seconds in most cases) and no data is lost (RTO <= 20s). See the following diagram for more information: ![Disaster Recovery for 3-DC Deployment](/media/deploy-3dc-dr.png) **Disadvantages** -The performance is greatly limited by the network latency. +The performance is limited by the network latency. - For writes, all the data has to be replicated to at least 2 DCs. Because TiDB uses 2-phase commit for writes, the write latency is at least twice the latency of the network between two DCs. - The read performance will also suffer if the leader is not in the same DC as the TiDB node with the read request. @@ -53,7 +53,7 @@ The performance is greatly limited by the network latency. ### Optimizations -If not all of the three data centers need to provide service to the applications, you can dispatch all the requests to one DC and configure the scheduling policy to migrate all the TiKV Region leader and PD leader to the same DC. In this way, neither obtaining TSO or reading TiKV Regions will be impacted by the network latency between DCs. If this DC is down, the PD leader and Region leader will be automatically elected in other surviving DCs, and you just need to switch the requests to the DC that are still online. +If not all of the three DCs need to provide service to the applications, you can dispatch all the requests to one DC and configure the scheduling policy to migrate all the TiKV Region leader and PD leader to the same DC. In this way, neither obtaining TSO or reading TiKV Regions will be impacted by the network latency across DCs. If this DC is down, the PD leader and Region leader will be automatically elected in other surviving DCs, and you just need to switch the requests to the DC that are still online. ![Read Performance Optimized 3-DC Deployment](/media/deploy-3dc-optimize.png) @@ -74,16 +74,16 @@ member leader_priority pdName3 3 **Disadvantages:** -- Write scenarios are still affected by network latency across data centers. This is because Raft follows the majority protocol and all written data must be replicated to at least two data centers. -- The TiDB server is in one data center. -- Application traffic are processed by one data center and the performance is limited by the network bandwidth load of that data center. -- The capability to get TSO and the read performance are affected by whether the PD server and TiKV server are up in the data center that processes application traffic. If these servers are down, application is still affected by the cross-center network latency. +- Write scenarios are still affected by network latency across DCs. This is because Raft follows the majority protocol and all written data must be replicated to at least two DCs. +- The TiDB server is in one DC. +- Application traffic are processed by one DC and the performance is limited by the network bandwidth load of that DC. +- The capability to get TSO and the read performance are affected by whether the PD server and TiKV server are up in the DC that processes application traffic. If these servers are down, application is still affected by the cross-center network latency. ### Deployment example #### Topology example -This following example assumes that three data centers (IDC1, IDC2, and IDC3) are located in one city; each IDC has two sets of racks and each rack has three servers. The example ignores the mixed deployment or the scenario where one machine is deployed on multiple instances. The deployment of a TiDB cluster (three replicas) for three data centers in one city solution is as follows: +This following example assumes that three DCs (IDC1, IDC2, and IDC3) are located in one city; each IDC has two sets of racks and each rack has three servers. The example ignores the mixed deployment or the scenario where one machine is deployed on multiple instances. The deployment of a TiDB cluster (three replicas) for three DCs in one city is as follows: ![3-DC in one city](/media/multi-data-centers-in-one-city-deployment-sample.png) @@ -91,7 +91,7 @@ This following example assumes that three data centers (IDC1, IDC2, and IDC3) ar TiKV is a multi-Raft system where data is divided into Regions and each Region is 96 MB by default. Three replicas of each Region forms a Raft group. For a TiDB cluster of three replicas, because the number of Region replicas is independent of the TiKV instance numbers, three replicas of a Region are only scheduled to three TiKV instances. This means that even if the cluster is scaled out to have N TiKV instances, it is still a cluster of three replicas. -Because a Raft group of three replicas tolerates failure of only one replica, even if the cluster is scaled out to have N TiKV instances, this cluster still tolerates failure of only one replica. Two failed TiKV instances might cause some Region to lose replicas and the data in this cluster is no longer complete. SQL requests that access data from these Regions will fail. The probability of two simultaneous failures among N TiKV instances is much higher than the probability of two simultaneous failures among 3 TiKV instances. This means that the more TiKV instances the multi-Raft system is scaled out to have, the less the availability of the system. +Because a Raft group of three replicas tolerates failure of only one replica, even if the cluster is scaled out to have N TiKV instances, this cluster still tolerates failure of only one replica. Two failed TiKV instances might cause some Regions to lose replicas and the data in this cluster is no longer complete. SQL requests that access data from these Regions will fail. The probability of two simultaneous failures among N TiKV instances is much higher than the probability of two simultaneous failures among 3 TiKV instances. This means that the more TiKV instances the multi-Raft system is scaled out to have, the less the availability of the system. Because of the limitation described above, `label` is used to describe the location information of TiKV. The label information is refreshed to the TiKV startup configuration file with deployment or rolling upgrade operations. The started TiKV reports its latest label information to PD. Based on the user-registered label name (the label metadata) and the TiKV topology, PD optimally schedules Region replicas and improves the system availability. @@ -121,12 +121,12 @@ TiKV-41 ansible_host=10.63.10.41 deploy_dir=/data/tidb_cluster/tikv tikv_ location_labels = ["zone","dc","rack","host"] ``` -In the example above, `zone` is the logical availability zone level and is used to control the isolation of replicas (currently three replicas are in the cluster). +In the example above, `zone` is the logical availability zone level and used to control the isolation of replicas (currently three replicas are in the cluster). -Considering that the data center might be scaled out, the three-layer label structure (`dc`, `rack`, `host`) is not directly used. Assume that `d2`, `d3`, and `d4` are to be scaled out, you only need to scale out the data centers in the corresponding availability zone and scale out the racks in the corresponding data center. +Considering that the DC might be scaled out, the three-layer label structure (`dc`, `rack`, `host`) is not directly used. Assume that `d2`, `d3`, and `d4` are to be scaled out, you only need to scale out the DCs in the corresponding availability zone and scale out the racks in the corresponding DC. -If this three-layer label structure is directly used, after scaling out a data center, you might need to use a new label and data in TiKV as a whole needs to be rebalanced. +If this three-layer label structure is directly used, after scaling out a DC, you might need to use a new label and data in TiKV as a whole needs to be rebalanced. ### High availability and disaster recovery analysis -The deployment solution for multiple data centers in one city guarantees when one data center fails, the cluster can automatically recover services without manual intervention. Data consistency is also guaranteed. Note that scheduling policies help optimize performance. But when failure occurs, these policies prioritize availability over performance. +The deployment solution for multiple DCs in one city guarantees when one DC fails, the cluster can automatically recover services without manual intervention. Data consistency is also guaranteed. Note that scheduling policies help optimize performance. But when failure occurs, these policies prioritize availability over performance. From a78297a87ffdc5ced30487736d013a8591aaf853 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Thu, 18 Jun 2020 16:33:05 +0800 Subject: [PATCH 4/9] Update multi-data-centers-in-one-city-deployment.md --- multi-data-centers-in-one-city-deployment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/multi-data-centers-in-one-city-deployment.md b/multi-data-centers-in-one-city-deployment.md index bd3f235803ae5..903583b4de3d8 100644 --- a/multi-data-centers-in-one-city-deployment.md +++ b/multi-data-centers-in-one-city-deployment.md @@ -7,7 +7,7 @@ aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/','/docs/dev/g # Deployment Solution for Multiple Data Centers in One City -As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (hereinafter referred to as DC). This document introduces the deployment solution for multi-DCs in one city. +As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (DC). This document introduces the deployment solution for multi-DCs in one city. ## Raft protocol From d9f4d14c753560e8db1b8e4c4c431b6c0495fd3c Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 19 Jun 2020 11:17:18 +0800 Subject: [PATCH 5/9] Update multi-data-centers-in-one-city-deployment.md --- multi-data-centers-in-one-city-deployment.md | 32 ++++++++++---------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/multi-data-centers-in-one-city-deployment.md b/multi-data-centers-in-one-city-deployment.md index 903583b4de3d8..7fb67996a7d11 100644 --- a/multi-data-centers-in-one-city-deployment.md +++ b/multi-data-centers-in-one-city-deployment.md @@ -7,14 +7,14 @@ aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/','/docs/dev/g # Deployment Solution for Multiple Data Centers in One City -As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (DC). This document introduces the deployment solution for multi-DCs in one city. +As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (DC). This document introduces the deployment solution for multiple DCs in one city. ## Raft protocol -Raft is a distributed consensus algorithm. Using this algorithm, both PD and TiKV, among components of the TiDB cluster, achieve disaster recovery of data. Raft's disaster recovery capability is implemented through the following mechanisms: +Raft is a distributed consensus algorithm. Using this algorithm, both PD and TiKV, among components of the TiDB cluster, achieve disaster recovery of data, which is implemented through the following mechanisms: -- Raft members are essentially log replication and state machines. Among Raft members, data replication is implemented by replicating logs. Raft members change their own states in different conditions so as to elect a leader to provide external services. -- Raft is a voting system, which follows a majority protocol. In a Raft group, if a member gets the majority of votes, its membership changes to leader. This is to say when the majority of nodes are still in the Raft group, a leader can be elected to provide services to the outside. +- In essence, Raft members are log replication and state machines. Among Raft members, data replication is implemented by replicating logs. Raft members change their own states in different conditions so as to elect a leader to provide services to the outside. +- Raft is a voting system, which follows a majority protocol. In a Raft group, if a member gets the majority of votes, its membership changes to leader. This is to say when the majority of nodes remain in the Raft group, a leader can be elected to provide services to the outside. To take advantage of Raft's reliability, the following conditions must be met in a real deployment scenario: @@ -23,7 +23,7 @@ To take advantage of Raft's reliability, the following conditions must be met in - At lease three DCs are provided in case one DC fails. - At lease three cities are planned for deployment in case data safety issue occurs in one city. -From the conditions above, you can see that the native Raft protocol's support for even number of replicas is not so good. Considering the influence of cross-city network latency, three DCs in the same city might be most suitable for a high-availability and disaster-tolerant solution of Raft deployment. +From the conditions above, you can see that the native Raft protocol's support for even number of replicas is not so good. Considering the impact of cross-city network latency, three DCs in the same city might be most suitable for a highly available and disaster tolerant solution of Raft deployment. ## Deployment solution for three DCs in one city @@ -39,13 +39,13 @@ TiDB, TiKV and PD are distributed among three DCs, which is the most common depl - All replicas are distributed among three DCs, with high availability and disaster recovery capability. - No data will be lost if one DC is down (RPO = 0). -- Even if one DC is down, the other two DCs will initiate leader election and automatically resume services within a reasonable amount of time (within 20 seconds in most cases) and no data is lost (RTO <= 20s). See the following diagram for more information: +- Even if one DC is down, the other two DCs will initiate leader election and automatically resume services within a reasonable amount of time (within 20 seconds (s) in most cases) and no data is lost (RTO <= 20s). See the following diagram for more information: ![Disaster Recovery for 3-DC Deployment](/media/deploy-3dc-dr.png) **Disadvantages** -The performance is limited by the network latency. +The performance is affected by the network latency. - For writes, all the data has to be replicated to at least 2 DCs. Because TiDB uses 2-phase commit for writes, the write latency is at least twice the latency of the network between two DCs. - The read performance will also suffer if the leader is not in the same DC as the TiDB node with the read request. @@ -59,10 +59,10 @@ If not all of the three DCs need to provide service to the applications, you can **Advantages:** -The cluster's read performance and the capability to get TSO can be improved. A configuration template of scheduling policy is as follows: +The cluster's read performance and the capability to get TSO are improved. A configuration template of scheduling policy is as follows: ```shell --- Evicts all leaders of other DCs to one DC that provide services to the application. +-- Evicts all leaders of other DCs to the DC that provides services to the application. config set label-property reject-leader LabelName labelValue -- Migrates PD leaders and sets priority. @@ -76,20 +76,20 @@ member leader_priority pdName3 3 - Write scenarios are still affected by network latency across DCs. This is because Raft follows the majority protocol and all written data must be replicated to at least two DCs. - The TiDB server is in one DC. -- Application traffic are processed by one DC and the performance is limited by the network bandwidth load of that DC. +- All application traffic is processed by one DC and the performance is limited by the network bandwidth pressure of that DC. - The capability to get TSO and the read performance are affected by whether the PD server and TiKV server are up in the DC that processes application traffic. If these servers are down, application is still affected by the cross-center network latency. ### Deployment example #### Topology example -This following example assumes that three DCs (IDC1, IDC2, and IDC3) are located in one city; each IDC has two sets of racks and each rack has three servers. The example ignores the mixed deployment or the scenario where one machine is deployed on multiple instances. The deployment of a TiDB cluster (three replicas) for three DCs in one city is as follows: +The following example assumes that three DCs (IDC1, IDC2, and IDC3) are located in one city; each IDC has two sets of racks and each rack has three servers. The example ignores the hybrid deployment or the scenario where one machine is deployed on multiple instances. The deployment of a TiDB cluster (three replicas) for three DCs in one city is as follows: -![3-DC in one city](/media/multi-data-centers-in-one-city-deployment-sample.png) +![3-DC in One City](/media/multi-data-centers-in-one-city-deployment-sample.png) #### TiKV labels -TiKV is a multi-Raft system where data is divided into Regions and each Region is 96 MB by default. Three replicas of each Region forms a Raft group. For a TiDB cluster of three replicas, because the number of Region replicas is independent of the TiKV instance numbers, three replicas of a Region are only scheduled to three TiKV instances. This means that even if the cluster is scaled out to have N TiKV instances, it is still a cluster of three replicas. +TiKV is a multi-Raft system where data is divided into Regions and each Region is 96 MB by default. Three replicas of each Region form a Raft group. For a TiDB cluster of three replicas, because the number of Region replicas is independent of the TiKV instance numbers, three replicas of a Region are only scheduled to three TiKV instances. This means that even if the cluster is scaled out to have N TiKV instances, it is still a cluster of three replicas. Because a Raft group of three replicas tolerates failure of only one replica, even if the cluster is scaled out to have N TiKV instances, this cluster still tolerates failure of only one replica. Two failed TiKV instances might cause some Regions to lose replicas and the data in this cluster is no longer complete. SQL requests that access data from these Regions will fail. The probability of two simultaneous failures among N TiKV instances is much higher than the probability of two simultaneous failures among 3 TiKV instances. This means that the more TiKV instances the multi-Raft system is scaled out to have, the less the availability of the system. @@ -97,7 +97,7 @@ Because of the limitation described above, `label` is used to describe the locat #### TiKV labels planning example -You need to design and plan TiKV labels according to your existing physical resources and the disaster recovery capability, which improves the availability and disaster recovery capability of the system. You also need to configure the relevant `tidb-ansible inventory.ini` file according to the planned topology: +You need to design and plan TiKV labels according to your existing physical resources and the disaster recovery capability, which improves the availability and disaster recovery of the system. You also need to configure the relevant `tidb-ansible inventory.ini` file according to the planned topology: ```ini [tikv_servers] @@ -123,9 +123,9 @@ location_labels = ["zone","dc","rack","host"] In the example above, `zone` is the logical availability zone level and used to control the isolation of replicas (currently three replicas are in the cluster). -Considering that the DC might be scaled out, the three-layer label structure (`dc`, `rack`, `host`) is not directly used. Assume that `d2`, `d3`, and `d4` are to be scaled out, you only need to scale out the DCs in the corresponding availability zone and scale out the racks in the corresponding DC. +Considering that the DC might be scaled out in the future, the three-layer label structure (`dc`, `rack`, `host`) is not directly adopted. Assume that `d2`, `d3`, and `d4` are to be scaled out, you only need to scale out the DCs in the corresponding availability zone and scale out the racks in the corresponding DC. -If this three-layer label structure is directly used, after scaling out a DC, you might need to use a new label and data in TiKV as a whole needs to be rebalanced. +If this three-layer label structure is directly adopted, after scaling out a DC, you might need to use a new label and data in TiKV as a whole needs to be rebalanced. ### High availability and disaster recovery analysis From 3f78f2d963188cd047daa8454b72341904feec62 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Sun, 28 Jun 2020 19:51:23 +0800 Subject: [PATCH 6/9] address comments --- multi-data-centers-in-one-city-deployment.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/multi-data-centers-in-one-city-deployment.md b/multi-data-centers-in-one-city-deployment.md index 7fb67996a7d11..8d50604e9f735 100644 --- a/multi-data-centers-in-one-city-deployment.md +++ b/multi-data-centers-in-one-city-deployment.md @@ -1,8 +1,8 @@ --- -title: Deployment Solution for Multiple Data Centers in One City -summary: Learn the deployment solution for multi-data centers in one city. +title: Multiple Data Centers in One City Deployment +summary: Learn the deployment solution to multi-data centers in one city. category: how-to -aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/','/docs/dev/geo-redundancy-deployment/'] +aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/','/docs/dev/geo-redundancy-deployment/','/tidb/dev/geo-redundancy-deployment'] --- # Deployment Solution for Multiple Data Centers in One City @@ -39,7 +39,7 @@ TiDB, TiKV and PD are distributed among three DCs, which is the most common depl - All replicas are distributed among three DCs, with high availability and disaster recovery capability. - No data will be lost if one DC is down (RPO = 0). -- Even if one DC is down, the other two DCs will initiate leader election and automatically resume services within a reasonable amount of time (within 20 seconds (s) in most cases) and no data is lost (RTO <= 20s). See the following diagram for more information: +- Even if one DC is down, the other two DCs will initiate leader election and automatically resume services within a reasonable amount of time (within 20 seconds in most cases) and no data is lost (RTO <= 20s). See the following diagram for more information: ![Disaster Recovery for 3-DC Deployment](/media/deploy-3dc-dr.png) @@ -91,7 +91,7 @@ The following example assumes that three DCs (IDC1, IDC2, and IDC3) are located TiKV is a multi-Raft system where data is divided into Regions and each Region is 96 MB by default. Three replicas of each Region form a Raft group. For a TiDB cluster of three replicas, because the number of Region replicas is independent of the TiKV instance numbers, three replicas of a Region are only scheduled to three TiKV instances. This means that even if the cluster is scaled out to have N TiKV instances, it is still a cluster of three replicas. -Because a Raft group of three replicas tolerates failure of only one replica, even if the cluster is scaled out to have N TiKV instances, this cluster still tolerates failure of only one replica. Two failed TiKV instances might cause some Regions to lose replicas and the data in this cluster is no longer complete. SQL requests that access data from these Regions will fail. The probability of two simultaneous failures among N TiKV instances is much higher than the probability of two simultaneous failures among 3 TiKV instances. This means that the more TiKV instances the multi-Raft system is scaled out to have, the less the availability of the system. +Because a Raft group of three replicas tolerates only one replica failure, even if the cluster is scaled out to have N TiKV instances, this cluster still tolerates only one replica failure. Two failed TiKV instances might cause some Regions to lose replicas and the data in this cluster is no longer complete. SQL requests that access data from these Regions will fail. The probability of two simultaneous failures among N TiKV instances is much higher than the probability of two simultaneous failures among three TiKV instances. This means that the more TiKV instances the multi-Raft system is scaled out to have, the less the availability of the system. Because of the limitation described above, `label` is used to describe the location information of TiKV. The label information is refreshed to the TiKV startup configuration file with deployment or rolling upgrade operations. The started TiKV reports its latest label information to PD. Based on the user-registered label name (the label metadata) and the TiKV topology, PD optimally schedules Region replicas and improves the system availability. @@ -123,7 +123,7 @@ location_labels = ["zone","dc","rack","host"] In the example above, `zone` is the logical availability zone level and used to control the isolation of replicas (currently three replicas are in the cluster). -Considering that the DC might be scaled out in the future, the three-layer label structure (`dc`, `rack`, `host`) is not directly adopted. Assume that `d2`, `d3`, and `d4` are to be scaled out, you only need to scale out the DCs in the corresponding availability zone and scale out the racks in the corresponding DC. +Considering that the DC might be scaled out in the future, the three-layer label structure (`dc`, `rack`, `host`) is not directly adopted. Assuming that `d2`, `d3`, and `d4` are to be scaled out, you only need to scale out the DCs in the corresponding availability zone and scale out the racks in the corresponding DC. If this three-layer label structure is directly adopted, after scaling out a DC, you might need to use a new label and data in TiKV as a whole needs to be rebalanced. From 4b6d3a55933516d89f7f220d77e867ec3071d5fe Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Sun, 28 Jun 2020 19:53:26 +0800 Subject: [PATCH 7/9] Update multi-data-centers-in-one-city-deployment.md --- multi-data-centers-in-one-city-deployment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/multi-data-centers-in-one-city-deployment.md b/multi-data-centers-in-one-city-deployment.md index 8d50604e9f735..213b3e5301ca0 100644 --- a/multi-data-centers-in-one-city-deployment.md +++ b/multi-data-centers-in-one-city-deployment.md @@ -5,7 +5,7 @@ category: how-to aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/','/docs/dev/geo-redundancy-deployment/','/tidb/dev/geo-redundancy-deployment'] --- -# Deployment Solution for Multiple Data Centers in One City +# Multiple Data Centers in One City Deployment As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (DC). This document introduces the deployment solution for multiple DCs in one city. From e6775d32c1cbc066d6685dcb737f31b424f79f8d Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 30 Jun 2020 20:58:22 +0800 Subject: [PATCH 8/9] address comments from lilian Co-authored-by: Lilian Lee --- TOC.md | 2 +- multi-data-centers-in-one-city-deployment.md | 58 ++++++++++---------- 2 files changed, 31 insertions(+), 29 deletions(-) diff --git a/TOC.md b/TOC.md index 4af88696f5b3e..cc462b83244f6 100644 --- a/TOC.md +++ b/TOC.md @@ -107,7 +107,7 @@ + [SQL Plan Management](/sql-plan-management.md) + [Access Tables Using `IndexMerge`](/index-merge.md) + Tutorials - + [Multi-DC in One City Deployment](/multi-data-centers-in-one-city-deployment.md) + + [Multiple Data Centers in One City Deployment](/multi-data-centers-in-one-city-deployment.md) + Best Practices + [Use TiDB](/tidb-best-practices.md) + [Java Application Development](/best-practices/java-app-best-practices.md) diff --git a/multi-data-centers-in-one-city-deployment.md b/multi-data-centers-in-one-city-deployment.md index 213b3e5301ca0..f8f88f2c51182 100644 --- a/multi-data-centers-in-one-city-deployment.md +++ b/multi-data-centers-in-one-city-deployment.md @@ -1,37 +1,37 @@ --- title: Multiple Data Centers in One City Deployment summary: Learn the deployment solution to multi-data centers in one city. -category: how-to +category: tutorials aliases: ['/docs/dev/how-to/deploy/geographic-redundancy/overview/','/docs/dev/geo-redundancy-deployment/','/tidb/dev/geo-redundancy-deployment'] --- # Multiple Data Centers in One City Deployment -As a NewSQL database, TiDB excels in the best features of the traditional relational database and the scalability of the NoSQL database and is of course, highly available across data centers (DC). This document introduces the deployment solution for multiple DCs in one city. +As a NewSQL database, TiDB combines the best features of the traditional relational database and the scalability of the NoSQL database, and is highly available across data centers (DC). This document introduces the deployment of multiple DCs in one city. ## Raft protocol Raft is a distributed consensus algorithm. Using this algorithm, both PD and TiKV, among components of the TiDB cluster, achieve disaster recovery of data, which is implemented through the following mechanisms: -- In essence, Raft members are log replication and state machines. Among Raft members, data replication is implemented by replicating logs. Raft members change their own states in different conditions so as to elect a leader to provide services to the outside. -- Raft is a voting system, which follows a majority protocol. In a Raft group, if a member gets the majority of votes, its membership changes to leader. This is to say when the majority of nodes remain in the Raft group, a leader can be elected to provide services to the outside. +- The essential role of Raft members is to perform log replication and act as a state machine. Among Raft members, data replication is implemented by replicating logs. Raft members change their own states in different conditions to elect a leader to provide services. +- Raft is a voting system that follows the majority protocol. In a Raft group, if a member gets the majority of votes, its membership changes to leader. In other words, when the majority of nodes remain in the Raft group, a leader can be elected to provide services. To take advantage of Raft's reliability, the following conditions must be met in a real deployment scenario: -- At least three servers are provided in case one server fails. -- At lease three racks are provided in case one rack fails. -- At lease three DCs are provided in case one DC fails. -- At lease three cities are planned for deployment in case data safety issue occurs in one city. +- Use at least three servers in case one server fails. +- Use at least three racks in case one rack fails. +- Use at least three DCs in case one DC fails. +- Deploy TiDB in at least three cities in case data safety issue occurs in one city. -From the conditions above, you can see that the native Raft protocol's support for even number of replicas is not so good. Considering the impact of cross-city network latency, three DCs in the same city might be most suitable for a highly available and disaster tolerant solution of Raft deployment. +The native Raft protocol does not have a good support for an even number of replicas. Considering the impact of cross-city network latency, three DCs in the same city might be the most suitable solution to a highly available and disaster tolerant Raft deployment. -## Deployment solution for three DCs in one city +## Three DCs in one city deployment -TiDB clusters can be deployed in three DCs in the same city. In this solution, data replication across the three DCs is implemented using the Raft protocol within the cluster. These three DCs can provide read and write services to the outside at the same time. Data consistency is not affected even if one DC fails. +TiDB clusters can be deployed in three DCs in the same city. In this solution, data replication across the three DCs is implemented using the Raft protocol within the cluster. These three DCs can provide read and write services at the same time. Data consistency is not affected even if one DC fails. ### Architecture -TiDB, TiKV and PD are distributed among three DCs, which is the most common deployment solution with the highest availability. +TiDB, TiKV and PD are distributed among three DCs, which is the most common deployment with the highest availability. ![3-DC Deployment Architecture](/media/deploy-3dc.png) @@ -39,21 +39,21 @@ TiDB, TiKV and PD are distributed among three DCs, which is the most common depl - All replicas are distributed among three DCs, with high availability and disaster recovery capability. - No data will be lost if one DC is down (RPO = 0). -- Even if one DC is down, the other two DCs will initiate leader election and automatically resume services within a reasonable amount of time (within 20 seconds in most cases) and no data is lost (RTO <= 20s). See the following diagram for more information: +- Even if one DC is down, the other two DCs will automatically start leader election and automatically resume services within a reasonable amount of time (within 20 seconds in most cases, RTO <= 20s). See the following diagram for more information: ![Disaster Recovery for 3-DC Deployment](/media/deploy-3dc-dr.png) -**Disadvantages** +**Disadvantages:** -The performance is affected by the network latency. +The performance can be affected by the network latency. - For writes, all the data has to be replicated to at least 2 DCs. Because TiDB uses 2-phase commit for writes, the write latency is at least twice the latency of the network between two DCs. -- The read performance will also suffer if the leader is not in the same DC as the TiDB node with the read request. -- Each TiDB transaction needs to obtain TimeStamp Oracle (TSO) from the PD leader. So if TiDB and PD leader are not in the same DC, the performance of the transactions will also be impacted by the network latency because each transaction with write request will have to get TSO twice. +- The read performance will also be affected by the network latency if the leader is not in the same DC with the TiDB node that sends the read request. +- Each TiDB transaction needs to obtain TimeStamp Oracle (TSO) from the PD leader. So if the TiDB and PD leaders are not in the same DC, the performance of the transactions will also be affected by the network latency because each transaction with the write request has to obtain TSO twice. -### Optimizations +### Optimized architecture -If not all of the three DCs need to provide service to the applications, you can dispatch all the requests to one DC and configure the scheduling policy to migrate all the TiKV Region leader and PD leader to the same DC. In this way, neither obtaining TSO or reading TiKV Regions will be impacted by the network latency across DCs. If this DC is down, the PD leader and Region leader will be automatically elected in other surviving DCs, and you just need to switch the requests to the DC that are still online. +If not all of the three DCs need to provide services to the applications, you can dispatch all the requests to one DC and configure the scheduling policy to migrate all the TiKV Region leader and PD leader to the same DC. In this way, neither obtaining TSO nor reading TiKV Regions will be impacted by the network latency across DCs. If this DC is down, the PD leader and TiKV Region leader will be automatically elected in other surviving DCs, and you just need to switch the requests to the DCs that are still alive. ![Read Performance Optimized 3-DC Deployment](/media/deploy-3dc-optimize.png) @@ -75,29 +75,31 @@ member leader_priority pdName3 3 **Disadvantages:** - Write scenarios are still affected by network latency across DCs. This is because Raft follows the majority protocol and all written data must be replicated to at least two DCs. -- The TiDB server is in one DC. +- The TiDB server that provides services is only in one DC. - All application traffic is processed by one DC and the performance is limited by the network bandwidth pressure of that DC. -- The capability to get TSO and the read performance are affected by whether the PD server and TiKV server are up in the DC that processes application traffic. If these servers are down, application is still affected by the cross-center network latency. +- The capability to get TSO and the read performance are affected by whether the PD server and TiKV server are up in the DC that processes application traffic. If these servers are down, the application is still affected by the cross-center network latency. ### Deployment example +This section provides a topology example, and introduces TiKV labels and TiKV labels planning. + #### Topology example -The following example assumes that three DCs (IDC1, IDC2, and IDC3) are located in one city; each IDC has two sets of racks and each rack has three servers. The example ignores the hybrid deployment or the scenario where one machine is deployed on multiple instances. The deployment of a TiDB cluster (three replicas) for three DCs in one city is as follows: +The following example assumes that three DCs (IDC1, IDC2, and IDC3) are located in one city; each IDC has two sets of racks and each rack has three servers. The example ignores the hybrid deployment or the scenario where multiple instances are deployed on one machine. The deployment of a TiDB cluster (three replicas) on three DCs in one city is as follows: ![3-DC in One City](/media/multi-data-centers-in-one-city-deployment-sample.png) #### TiKV labels -TiKV is a multi-Raft system where data is divided into Regions and each Region is 96 MB by default. Three replicas of each Region form a Raft group. For a TiDB cluster of three replicas, because the number of Region replicas is independent of the TiKV instance numbers, three replicas of a Region are only scheduled to three TiKV instances. This means that even if the cluster is scaled out to have N TiKV instances, it is still a cluster of three replicas. +TiKV is a Multi-Raft system where data is divided into Regions and the size of each Region is 96 MB by default. Three replicas of each Region form a Raft group. For a TiDB cluster of three replicas, because the number of Region replicas is independent of the TiKV instance numbers, three replicas of a Region are only scheduled to three TiKV instances. This means that even if the cluster is scaled out to have N TiKV instances, it is still a cluster of three replicas. -Because a Raft group of three replicas tolerates only one replica failure, even if the cluster is scaled out to have N TiKV instances, this cluster still tolerates only one replica failure. Two failed TiKV instances might cause some Regions to lose replicas and the data in this cluster is no longer complete. SQL requests that access data from these Regions will fail. The probability of two simultaneous failures among N TiKV instances is much higher than the probability of two simultaneous failures among three TiKV instances. This means that the more TiKV instances the multi-Raft system is scaled out to have, the less the availability of the system. +Because a Raft group of three replicas tolerates only one replica failure, even if the cluster is scaled out to have N TiKV instances, this cluster still tolerates only one replica failure. Two failed TiKV instances might cause some Regions to lose replicas and the data in this cluster is no longer complete. SQL requests that access data from these Regions will fail. The probability of two simultaneous failures among N TiKV instances is much higher than the probability of two simultaneous failures among three TiKV instances. This means that the more TiKV instances the Multi-Raft system is scaled out to have, the less the availability of the system. Because of the limitation described above, `label` is used to describe the location information of TiKV. The label information is refreshed to the TiKV startup configuration file with deployment or rolling upgrade operations. The started TiKV reports its latest label information to PD. Based on the user-registered label name (the label metadata) and the TiKV topology, PD optimally schedules Region replicas and improves the system availability. #### TiKV labels planning example -You need to design and plan TiKV labels according to your existing physical resources and the disaster recovery capability, which improves the availability and disaster recovery of the system. You also need to configure the relevant `tidb-ansible inventory.ini` file according to the planned topology: +To improve the availability and disaster recovery of the system, you need to design and plan TiKV labels according to your existing physical resources and the disaster recovery capability. You also need to configure the relevant `tidb-ansible inventory.ini` file according to the planned topology: ```ini [tikv_servers] @@ -121,12 +123,12 @@ TiKV-41 ansible_host=10.63.10.41 deploy_dir=/data/tidb_cluster/tikv tikv_ location_labels = ["zone","dc","rack","host"] ``` -In the example above, `zone` is the logical availability zone level and used to control the isolation of replicas (currently three replicas are in the cluster). +In the example above, `zone` is the logical availability zone layer that controls the isolation of replicas (three replicas in the example cluster). Considering that the DC might be scaled out in the future, the three-layer label structure (`dc`, `rack`, `host`) is not directly adopted. Assuming that `d2`, `d3`, and `d4` are to be scaled out, you only need to scale out the DCs in the corresponding availability zone and scale out the racks in the corresponding DC. -If this three-layer label structure is directly adopted, after scaling out a DC, you might need to use a new label and data in TiKV as a whole needs to be rebalanced. +If this three-layer label structure is directly adopted, after scaling out a DC, you might need to apply new labels and the data in TiKV needs to be rebalanced. ### High availability and disaster recovery analysis -The deployment solution for multiple DCs in one city guarantees when one DC fails, the cluster can automatically recover services without manual intervention. Data consistency is also guaranteed. Note that scheduling policies help optimize performance. But when failure occurs, these policies prioritize availability over performance. +The multiple DCs in one city deployment can guarantee that if one DC fails, the cluster can automatically recover services without manual intervention. Data consistency is also guaranteed. Note that scheduling policies are used to optimize performance, but when failure occurs, these policies prioritize availability over performance. From 3a7d4c95ed3cb9cee4e7eae1f455772bb4ea355e Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 30 Jun 2020 20:59:34 +0800 Subject: [PATCH 9/9] Update multi-data-centers-in-one-city-deployment.md Co-authored-by: Lilian Lee --- multi-data-centers-in-one-city-deployment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/multi-data-centers-in-one-city-deployment.md b/multi-data-centers-in-one-city-deployment.md index f8f88f2c51182..bb69d9a9769df 100644 --- a/multi-data-centers-in-one-city-deployment.md +++ b/multi-data-centers-in-one-city-deployment.md @@ -29,7 +29,7 @@ The native Raft protocol does not have a good support for an even number of repl TiDB clusters can be deployed in three DCs in the same city. In this solution, data replication across the three DCs is implemented using the Raft protocol within the cluster. These three DCs can provide read and write services at the same time. Data consistency is not affected even if one DC fails. -### Architecture +### Simple architecture TiDB, TiKV and PD are distributed among three DCs, which is the most common deployment with the highest availability.