From dbe9927e40252edec8d9a9d63f93251c3e24c2cb Mon Sep 17 00:00:00 2001 From: MaximClouser Date: Thu, 27 Feb 2025 19:48:01 +0000 Subject: [PATCH 1/3] feat: add configurable scoring criteria for Qwen VLM verifier - Add score_type parameter to QwenVLMVerifier - Update documentation with new scoring options - Add support for Qwen 3B and 72B models - Update README with clearer verifier usage notes Fixes #2 --- README.md | 37 +++++++----- assets/qwen_verifier_node.jpeg | Bin 23011 -> 28927 bytes pyproject.toml | 2 +- src/inferencescale/nodes.py | 21 +++++-- src/inferencescale/qwen_verifier.py | 85 ++++++++++------------------ 5 files changed, 69 insertions(+), 76 deletions(-) diff --git a/README.md b/README.md index 391d236..8948391 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,10 @@ A ComfyUI extension implementing "Inference-time scaling for diffusion models be [View example workflows](workflows/) +## How It Works + +It uses random search and zero-order search to explore different noises and a verifier ensemble (ImageReward, CLIP, and a VLM grader using Qwen 2.5 7b/3b/72b) to generate the best image. By exploring the noise space, it finds a better starting seed and produces images of higher quality and better prompt alignment than simply increasing denoising steps, with the tradeoff being increased time and compute during inference. + ## Installation ### Prerequisites @@ -79,15 +83,19 @@ This is the main node implementing the random search and zero-order optimization - `vae`: (VAE) VAE model for decoding latents - `view_top_k`: (INT) Number of top images to show in grid - `search_algorithm`: Choice between "random" and "zero-order" -- `num_neighbors`: (INT) Number of neighbors per iteration in zero-order search (only used if search_algorithm is "zero-order") -- `lambda_threshold`: (FLOAT) Perturbation step size for zero-order search (only used if search_algorithm is "zero-order") + +> [!IMPORTANT] +> The following parameters are **only used for zero-order search** and have no effect when using random search: +> - `num_neighbors`: (INT) Number of neighbors per iteration in zero-order search +> - `lambda_threshold`: (FLOAT) Perturbation step size for zero-order search #### Optional Inputs: - `loaded_clip_score_verifier`: (CS_VERIFIER) CLIP model for scoring - `loaded_image_reward_verifier`: (IR_VERIFIER) ImageReward model - `loaded_qwen_verifier`: (QWN_VERIFIER) Qwen VLM model -**Note:** At least one verifier must be included! +> [!NOTE] +> The verifiers are optional - you can choose which ones to use by connecting them to the node. However, at least one verifier must be connected for the node to function! #### Outputs: - `Best Image`: The highest-scoring generated image @@ -103,6 +111,13 @@ Loads the Qwen VLM verifier model for image evaluation. #### Inputs: - `qwen_verifier_id`: Model identifier (default: "Qwen/Qwen2.5-VL-7B-Instruct") - `device`: Device to load model on ("cuda" or "cpu") +- `score_type`: Type of score to return from the evaluation (default: "overall_score"). Options: + - `overall_score`: Weighted average of all aspects + - `accuracy_to_prompt`: How well the image matches the text description + - `creativity_and_originality`: Uniqueness and creative interpretation + - `visual_quality_and_realism`: Overall visual quality, detail, and realism + - `consistency_and_cohesion`: Internal consistency and natural composition + - `emotional_or_thematic_resonance`: How well the image captures the intended mood/theme #### Outputs: - `qwen_verifier_instance`: Loaded Qwen verifier instance @@ -154,21 +169,13 @@ The model will be downloaded automatically on first use (you do not need to have ## Future Work +- [x] Enable configurable scoring criteria for Qwen VLM verifier + - Allow users to select specific aspects like visual quality, creativity, etc. + - Support individual aspect scoring - [ ] Add batch processing support for image generation (performance optimization) - [ ] Implement batched verification for multiple image-text pairs (speed optimization) -- [ ] Enable configurable scoring criteria for Qwen VLM verifier (currently only uses overall score) - - Allow users to select specific aspects like visual quality, creativity, etc. - - Support weighted combinations of multiple scoring criteria +- [ ] Add support for image-to-image and image+text conditioning to image models (currently only supports text-to-image models) -## Development - -To install development dependencies: - -```bash -cd inferencescale -pip install -e .[dev] -pre-commit install -``` ## License diff --git a/assets/qwen_verifier_node.jpeg b/assets/qwen_verifier_node.jpeg index f91b26a9b372b2f9a26d87d05a9e08565279d66f..7c735570a6de51d25098485a26a3a326cf98750b 100644 GIT binary patch literal 28927 zcmeFZ2UJsAw=lYC0wPibks1V)j)L?SL}?Aj(jhcuBz7nqokiXr+ zDEi}(gmh|m?pSK;YALJTSNcsLA@QT<&!7Ej?C9k7LRUqR{eiIwJLx(=2rvWWfCM1* z$jbHkZEfxQzi|HZ{u}?FUx(wr`2a=)f8jcknj}3+dX$fnKK)i#emVHhYX3QK{)x4# z6$$(uFD+cKb~4Zd?-#*~1<(U-0iwViKm`y3Zi2G; z&+Cf*u6qY?2A%_6zze_*umNlVH-H_u4dy%to&nY%tOhs%R)7#70Ky_5HW5JVmyE%- zVERjTf8{w-f_iKL$>RAd?;!^OlnH{Yx%`#Koezotw278C&mX;b^anoho8XYx_315< z-@gecssMl_36I}50svB8060V7@uyXI{8=7oa+3hie28xbXvhd?2~LOzH~>N#0wNj$ zd^-RE@sJSwPX9zqKuAPPa*mXY{5%C%p@JG9Bp@OpBqkyu`DJ_reqcF3OhZC@RrvO~ zOFEB8Ib7&O0^>5tI2B7<=yiusT(=&-2qHg!nc)f})3xi|JiL6OV&W2#Qqp(sDk-a| zs@>DmH!w5;*|U0L{nW#TVGi z{$LCI{)4msz!we37a=h*5i#j6z6c0C!HbB7nB=PPIojJgq>o%Kafk$x(J97dmbQ>{ z-qJ~jQz#e6lki1zY`H5ArT1?5fRBb5-^=3 zJNGM*ohSR9&i^A__?<5PN>qO)JctD3l!%y^m=ye@rXZ)F{A_CBu zh-d&Pfc13#8HpK9B)YoKu(D;Z5qjDgK!7Vl(Ve{F-6xy*p?3(`Wdc-j`)BZ6HEoX6 zfqgbgwJ&%8Hc0gNG=Br0sygR(#&_lQHdL4;#pi0vaqKLn4j$-zgj>&k)HN%|8%&iVZ#5ji z{juw0{N%hjNKdxDyp?ZP1)!zVHHHUfac}CVj-pkV66#iDnSivc$Q^&^c=970I`3|% z%`9gTa37?I_tFkkuJA9a))g~BDux4Cxo7f6X?Z7Kw2!{0f(U02`+xpfz(;x~@i^8b z0c@Q?4BaFh&iQyAa?`gf7y2uQ06s3JrcEp}wIU}y@>>oe5De!1reW;8Tv|MkqblNG z@g(zC)5smFB01c`iM~tb#8FtTn~dp-91p?z5$W*z8#6g~Wo}}=lXR0mSeJRfB1#~q!f@xO=r6gk z*sD6f{h2|kA<}+xM(dHm__pGF!KWj}{r72U71M3=ky4FUX0Uhhz^hmd_0JlHf)b}R zd7b$JFFIEIiX~l=0=2sPCaQ})CHyX`=_gLAFdf|vWY${MJbjD@+|axF?JumZJ>Zq> zA!RJdsCPbH5`dMfNpa2@J`^5cpL)rU@PU&GXpJJ42jMTdNq_B_i<-Xx0%5wCs z=BWwzV4S;B9AdYCaXl`=Gx;h>WhFyFOY_Ol_7$*V@ZhYa-I8A8kER)DsvbG#XX#99 zi<}FAm=Ocxm{Hv>#(A3vxoj#24&)5*-6Kt_NM<|oY~*o?b~M|+KjN>0v> zD_Rj}zDDR>;CZBXw!3Bz=at*280PzAbvid3AQY1b!B#?cQf|{ebjPmeiR}{}w#b3z z8U-(M&d3G{iq^Rw)eyo%*r5V3!UH}hI?LP*%3(Q;K9F^V%X~bItBYrLzBxyEDok(A zq)!Kl`#_4~YqRJkJWY+2=KrI`NV4qR+cxZwC!-ZK7&BhCR5HwVuzKRa>ARMzqsZI#mQek8Ot6n zP-1!?POwJ#;8!kLPxJyg! zppH7J!f=;R_3{)b+}Ck$!Nv*)BwZDC^MboNdlp6XF0;Hg&oRX_$>TSO>%*Rs&UUvB z?`UJT7a@_BtWZ0xiPa9&1Fd;VW(D-ENxN|C^8%hG56zP;9$E_L#m)JYNgW69U9OU) zLYnI>#?+j0!EPb@4oo_Oi(ua7{yUuxrtU#UdL!d^D}=(CqB!F3TG6D|@HwDEx)x(Q zG$cA3c`>0@C9)P*(K!s2wGBUle!TzaF5p*i?@E-*@p~2FtVA4b1u0f2JwgG)(}851 zhllEriIQJ^kQv{4v*R!)E~{)$>!fulNFU{5&>wLasrR#LQL7zFihX>Rh7&b~<#$#= zY4cITD&lYP04hWy>qf|qAzVPA3M#LISZez)Px~N5yRi&8z%DP{>WyZiNZ_>rmt8T;cVNF%>LqN$v_vxGt-2*-{oC~-5+ zG0v|H8lUVFves$qcU+9>_GYh6nUm~yZ&%)=!#3UQcASuMvSq#NUx~HC1cRd@yll6V$Jb!dG(9{4W7{JBjvIS5 zel2pLF3x2+?DFdBH`S)=TvDBCj~P8qujivTJCpH1)nWSS4GqdrEPs*J{UC(iNMi>9 zL+7H3gtGAx&l7>VyY@uHKW^%OV8Xg!KA&3P2Gf~*h4BEHXFCbv+XZL=X++jyw%=8rEu1xdN z`3AggBWyJJ(!-Of^4Yd34T2#@(VikNm@GjE>J5;)j1JlxM@$|;X}sbuOlAgHyzhy^ zYh$hq-&7%au){+a@30rUu)3Xide4^+m5;qDPl7H#@sEiWh*KOtM4UVKA%Q!7zjgP@ zW3|abx=V?`xM%C!WbfipOZQI)I1#4!LDA2;1RpQ@>6>#Zf)h*AkC(Zoc-d(;gW4~w z5>cd5q(X|TEI2WbP;-lTAhwa`!J2eah>A-#51hy5G$!pLyB$6(W$uGQwrE; zW?}u;JIWW-o^mR;wVH%R$P2Z1qg6 zD6IxZUB9WRpc`3*Mkt&WLTztQVwK1J`R#DRH3{wh97>_ODM(41h#k5ss<~Bh7i6ee zI9@T6k_TvLA+;vw4ujDu?W`0?kL*R}u!`LaS7YzTPx3e}lt))Ti6L8jaPP}P8>bU+ zt{Wije>o}CH4UW#=iYn%Pp1D(=QNy^{}1RKL9g;)Zj-tZ;hNPn%Fl=TZU`RFFqDjKn<@QN>eu1O^%=VE-_x8 z{9N(g>ngAmPU)|Q={@W`%a!#hf%&vIa-#c2EHQ>2wIfWEJc7Ejo+Ku>S!g(9=&xzL zE!vB3a(nG(k z$D5(2vKbSb4U=y}dd4;iZhFS8r>q9aGh+-6W|R&VKa0N=nol37kLghrsejD1L~Wq{ zWcw*8(sIhEU7iXPcPb5|K=N-^S;(MGHW@B+cIaa4wWy+%yPia$CA$(iZ*zPq9)?B| z$@E~VRL?%i%dc5L%Q30Bm|B>xicP*!PHuD15B_x&kA77W4MPG457BP6xWcm?AWAFB zvK-RU6l;{?hv=a6s(5hl%66-0_)1YxQ6jpoH-z{u$(NeP?@4`Dq7WILAJzp*`_Uq0)R}bpaGilc zv?4o!LNieY9YMcPRju$0s;j&66$A^mtQy~xc?n^ZJKa5%~*@@0jwaDYbOcZ7B^K-1V=7-#o?M3%mqO-a4T?L8n}CA&v?LJLl_(Im%H}|H7#^rT=Dq+cilHb|S))04TR=A-i49d05%p-xj=GW*5`otsM zdSiv5CYJwX=MBitq~H+eQ5?#34$C|Dk4LYqWn6r+R%P1e6*q5#_$z;%C*z{6ZIx*_ zNY-K#Mz5)~7`~e2UokYK!1C4BNS(hX9{LuIfPTxxb=Kg4wD1#z8px4a)(qd`cTM?* zbyMXUK9#rWB^aY$`Bus`fY8Cb4;~2Sw5@t#^<8Vc*5sF1{?T~kVx-#xh={yAC`qxu zNz$(AN0sT3S9p}qZ-K%8LG|q{^K(`!*Qx#k5f-U05t4j@os7k(e&0~?{)D&s~qg5EeVFt$g{&T&3kg}}aFE2_s z->|VIkAallS(tEXyY63PZvDHyWK9uQDR3`P9}iGVo%>YBOA*4V;=Pj>eqA|4r{=-K zm$TK3)OT(*PrsdWUQUBESwASP@l#nvt#g1zkis2{CUmx z_jdFRfz|!zlW$-L&~C&dvzYgI;3^L;8vD68gTmvogmmhthj?V8u_kc9mjC0ThPORd z$n~5BmOQMV7=GRF)(4zqAv9EPstw7S@O-$vmk~PKEqU^qti(5iD#_hkXji%3hk0y@ z;|19gy&T!0*n5k2XlYxBUPUmONXI4TE}eMsm&Z2fl3N)UkM5wJF*OBZm5{5ou%*G- zF#o~d@N|4NKjpulPQglYf6cdwLT}pPN_J-hwKx->)pMt55h=9=Pg?<;jg*x71sMP}e^y z!ajcAGMSU+AMNh$Vs-t)P|52gbdt8BRdfWAP5wfMO7`{K1uC)`E(?xIlt;99fyWnf z6=n@T*cTb)bAXi6)N8rhTbc^;#!0*n<1$m3r0C{96Ozf#BfRGE08vxe;`maNSD|uZ zp87|X)57CUaRZ8;E2N$}DJN-rZTArV92kgEJ$&0MXJ}_9$!?yyVLGXsi2-&>lllYN z8?j#!3Thl{jXdM;GPK~T_4$#kQd7;`eAX6Em-YJGnAB-w+iZZmrNPsOVG_5f0oI;1 z{^??QCJbk*u&Jr9Dw=NGq~W^*r}4zpyObHz{FJtA?3iK_8XE5SF#X0X@t8&Tg?!eM?+LlIjBLT@M~` zPi_35P1-7NmS&v%XySuYS8saJ{c)qxsiLW=?k^Gq->5@zB*Tq4FWU}JMCUFx(mL>0 z&sa9#vMHXkPJDOa^pP%hsEq75VI7G6$;z#dw(ifjx|ueqK`#2vg6l4cVuk_t5v=>6 z)aD2s0h-IO%82ChEa~fRRr61#T<)FH;MSK*Xy|@0t0ve~k`C3!-!#)UBM`Titd zZntq=twPJ@vS)0zf#(#MTlh?gJQ=qfk}w0F*GnAccWGuHRZ!~Pn{#Cc7}+GwGnS4H zlCr0fSJq;l&B~1I%o=asASqfdy&aa=wKIK_E4Ax~+x2(z5RxfOrG~|(UY!G_gi=}j z4$@j-$?BChInlM+lZy_YFoHhLKA-HBBdHkPQgimkW_a8boRENL*LfnvV zB4t|P&K;U{3rBR)&Z_oMJptTfB5hSQMUBcbbw>oA3b>>}PzF_DzL$_xG$u39!=#tX z-^S|*j4J&Z+iaxtI)x4V*8(SmJXOm z;~^})=fXCT2HJP_N>y1|MdGufgQHZT3(dD~)kaDAD@#Z1_w$PjKJIOkj_tTX4O8SU z-z#)sua&?~3}~vb+7d0V9hggEJXSVL!xPM6Hh3O?y(vB(9r}Gm^E;Ii z=k*4tImTh$^1|%Insgv?t@AdqRP+=W1sPF*kNF%=!e^>4|-ORV`EQEVB5{^S_Fr`zL(>ffAkefBCZT`Y6d%)Qkt7!LM3`T6n?1+0hse%+V1@q*SlL`7Fi! zC{?9(6d?E!tecQ}!6a;e?rit0?GC)qvNI=xg30Pqfl6G|OT?PPmtqwSJn;GrTp4G8 z&?btTl`!UXEUadK8AP^rySPQx2dN*RAf!M4P*}loe?9$Uv3LZKq>en%j5ln@{rimsxSM zIyJ#+!4ks31>M$ptILIB%5KnJ!o^EugT{ogkA`Z~kd>=@R0MPeYbBZbo5jCy zohCRawvPa+9{IWESxjw+Yb_5raP%@$Ej*EeoO!RyIxPBmc1nl{G9Qt@g-O0YBzgv{*D7sSCewrvc|I26G7xxx{eGq3>UyQovJ-U%KWoV-Icon2*?K>gU4nEpLHT`7vn~%N( z5$*{OqRlku2uAK8`wZt=5p2}AslU|J#h}CQtayEi5@PlsJc(jC8OH-yL8?peRtsOp z%Bj=4lV`DPdR`Vr$;*B-$1_!pTQ(J5V{`;-jm((T5{&0PN7AiB@4UCej7gKYX)_g; zs23)(aY0%ZHHQW1$fn3AXiMyM+IlKj<*;!VW7sjxZH{f8xDaLqRw>y%R|`8OfcH4u zS#MxbO^FWB`8O`3RC`j~^(q!N7I2t}{QEF-)1zxzv~KP+B!?ALH+~fBEOOtB^<@(s z9uS8gFF~afo5pV}>S_}mlyRlJxz}4I#log?u1>3r>Ua$gq`ok*!M<8Nfh;*xW}KgU z3fUjR1Jd%}p{vN$F73l>s-3W93Y7nN2I9R%dE}>Q9`mUuvIzt98wSfMx!f-c;M&9{UzKyc5?*B=F5ahKY8YL&GEz=( zQ-^e@fT}PHSj^e#k47slKDtKv#{{y3-uhBpPdAigZsEtUZq|OA`>q4_9$YvKw!rl4 zF;46n@L;%&Qd8i=&+RRT>SF%;;N1BEeg_-vQT}grt^d#s2?TETx`=w9gx0TF-(@T8 zIuE6FSgV5udXx0mxXky+T9jhaIwy97iby_R(eGkA48JZb8rl44s>pZW+VfiQcEf=J zLuHcG8@Ax=hx3qU5vMXODfw;CD_DaD^v?T_v&D6V6SJBp&yw0z=zA4>SNB^cd%C^> zy9#V_S-zLjEzk_F+};b`XTCxALk?ZDhX)pCI2^xz7V_fzu{)bFqL7n74 z`rF#z-Ocu|59{MEO<3t9#4MdhpyoQFZ`{Qw6<{7!m4pi}r3i}XFyB-aGJldRa*^Re zHN{+oN8{DA3|VPeoB1=Jw(;R!tGxQEln?hqxRdtQ6*cv%W@QxE4+ic5`!{H?*M^H^ z<%%8Bs1Z5iIjC=;O%bNNjw;Ei!SkM1;_f{6K^RFi-=MlvVu}&2e-hfB5?$_jIVouD zP~sg!N`XKs{ae3j8_tx3PH;bPV46+=d#U*aomYi)h_+^uG4*-dk#o7Urq?T{WS0!W zqph%;^4S$F(<*iJk$L$M3#BkUGhAlet+QtIo&sd8iCuzs$sK*B?ZbXql;?1R>mIv=NaXmq zC^_-RP)q_N|MU>@Mg$Z0kMh)8INBM@j(C+%ljhTAGhIu!=_~3ZmT6h*a$I}*m8pn| z06RPQ1yMK|rp^Sl6N!B9sAceFxWamr^S#!7JlR4t#j|Ob73$}OQp%*P4vh|u;uH{K z*sT`t=&YjgFhqxwTkz}YPi@zoSf_|(Ueu9yOWLCyp~xG-+c*KtZpXfc@QC}V##3fx z>QWD%L1LMNPpd2mM?Pa`oJh}C&#)@<4Vqqg)V66%jv{MeEQz(kwc9#l$pZ3^I)I03 za5A@^a!P&+w130(*Rxljo)X_QR`-NrQ_y zBD@SaZzf)6REctz`ASAw=}-n9&9SnrwluO1Ir|!a!rWVJ8#+sSu=Vg#myc8Q9`$n% zwb(bSn?G#A(UA5AQghT%^wzj(sY}tEvD(uru0BrFDPFT)zJScq7%0JTfb1Qekl{(( z%53Z6w&76sCf(FMuds{sm1)WJsPY?s=o;4Vin zHff(i<6Qf9ILx{>VWh|neo>wqCedah^tuQd^SK;m;PhBg84Lt7N}UHMb{3MHMq!k{Hh*A1`zk+?LY+C8@UiT#gl zMH1lg4=i1Tqn1(1X8TRZMcYG^SxmYmlNb^u3I?+IZJoHu&SwyKQ5r#H57c zb)&vdwkgP-L%Uhhs<*O&m8#3(D1$dRt96Tp1$cx7fck#&%bzUl;iD;yOPd(Qa1 z^9`HHn{&?uqr94I`_InyN1$yn{HPUJ{TP9HRve=_*PYOqyHZAa6L02UZd&S>Z0DaU zmIOB~FfuS1*U)%bPUexGJ{68Wy;b%d4>%w`7v2K!soUM-Pk#XnGn7X5lljVtEGp6XJW4BK*ekJw7`m#(vt z7gNfX31Z0uLkb){AAeMNGC%pi zvv_2Ci^WfJb3NkN!kGrPvSL)7-hPsx&d*?V*wbkoqeT#~xo*6eahO{~rz@%*0mYAF zU2Sph%mY#@IeW231FLsvR=|b!6K#@cLE(aqAn*`_)4an)+88qFaz?c$HU&Br=zX+G z{S?XI!)v9fyqBkp$IYw6&U(0;Qn@jMq#1bN(I#|%0eW_75B>5p z{TP*wJF9rkt&O-#?&egC4JX%6=u5VuKW#CU$j$^a^aD zXQ74Q(bOwEaJ2mq*ESAcZH1p6j;*Foo-|?cz%+br9g6t`t@sH)Av62q2XFx0lC#12 zf(V)%RSwz8@PPa!&;e3eB8x+SA!4hzMI7kj11(qQtmHF@{H*H?mw>e;8pH#C5()X? zJ%Gh%;Q{~2`u)EVg&F!=8$1w^L3QU(sz%{|a=SmU?PrOD??87C(y<1RzY*IRAV+md z1v+HX$??GMl+NGy`#kWA4{)hk4N^V~N9(6gNf)#*MPU!lu;9qPI7+aV{uf!Xe=%v{ zAr@_1W3K3Wkfw#>Oa2?V|Aq`NmVPEk_-F=3)0i*e0p!R34{>A!|B@OgfgGmJLS^Ml zDYg;N>)oMEn)R0y{?fVE>S1FE9)KP}j!v}Dxw!Fh*(RCoBQ^_{rv3ycKM5O15#Y`6 zH(h1_O-4p(aEA>%u#X2o-z_}n7jgeVk=mr0|Ea0{&p zmi^yRYv=p3icf4QQP>agl7y1{+0AVQh=K_Qfjo;YH#GEaETODFPb)bz&BuoVSOW@K zJ@&gAdE$BNRj%Rkxz6i~SI=Bk&{SpoDT{~&htZA21o%Hy@abxa{RPG10$%Pf)xLOO zO`l->Z++(P#_9F99)q?0t+}x-7G|g#16Euz%(+~uY$G=!TKPpAxO*H}uKj!80tdMNrPckz z*#EA#|G6dw8n{|P+&IGXF(uCO-AZqnzX*eeOC8IiR0rT0n{B@)(eP%2w9pYp|Dm~n zuZ`H9G>AW5!`USKWnSis9QcS0^gF^h4_0kY@=R9uE?VMS)gy%z&L8ol{f!y>5dhC; z&v93~qZcL`LEYXZe9e1ghn#T+JV1#^quB;RHvMN_gLr-aDNZpN`g3%U7zsU$T~%L( z?56jpk#z_u0dPD5>ux(J9cwmHpL3AMNL~BkP!tb{ff0i%NuR_dOQ*o2To*I>q;Kw$ z&t1X64xFd}dhZRxa#T{Ttgqbn!Ko+iWS%lqUACm?Phy`Pp1C`6YY%3xcm)jrxPve8 z9`6}!spB8`S zn+}$Y&P@x~n+PmV<>%bQ$v;qYIk7prw5DYskT2Q0FVq*Tsj$C1p@#FF?9P{L7ohX! zSL5=S@c>2*3#B1Ch3Y*`czi@XfGdTH$UNsE+K9#Q#)M|WoK-Q_Ss4FTN(rq>^ZP0! z^$v9RNGH0^XIwxd3U(a!^V=FphKhZKWUUq{ZPzB_3bC`t8=S%=vpw{GXpWz3uWo9=U@i4XS7& zFvW0pRwbLLuS>+6o6MND945v~y&>mt>t;tMIyVoViw$q2RLQ(u%q(ap#(Ur=-m7mA zBXZftA&GA%XYaCb(5+FX3U?I5pnSZZHK*5?3@ukvlh_LH%7_t5GI%tdCGK?~HiyaX zkY_@{JECb9wbhnT4M!n`g{x5oc4^;iA6#F)(o2;X_VY@i9Zu5r$eT3Us#Z<&^T(O# z##rtoxI4)b+$KRVeI_y3h+?xj(nQ9GP_n|UgFaV7bY-=uYQ5Y?3i2^_htRmNYxnO7$HUkm3sS|WT9~%X4e1-9 zA^Av0o3;eoq#+Zt2|F}7M}5pi&L;^t<1(J}?Pq@ceb_sTXWZr}r-s~}8ivWV^5h|b z_HX|EaphLu-$zw5!@5KXzWXYm%q%EIx8_Sn8T7x5ZgVu0SCd%@($xSWY*9B|@xXYo zXPdWUQU#JyMs-xEGu_!o_IX2#c`N%WbN3F3$E?`c0y;WB1c~}tF*aojh6#GgI4(Ub zBYmN)FRn~Q8ik{8QP1A=7^-Nm^&91dd^g`-HZb$N_J$zVRe|-y_hw@V{7Tp7X}@S$ zj`O#k^`tH2OkB7Jef5p(q1J8lS}bW5!)uK0qB#s}EKB&vULaT8epw>SVvREEzRP;b zXEXz{GXV)Fx0Cjdy%1-Q(7o+h)V-mv;rcX)=lNVG8FSATtVE#++*iz4>R4VjhrPEb zvkCOwrX9QM!uxk$cO^U5s)7((D>e?1 zo<)a%;BH z8{un}yU|FE%6n77+cdl@wBvNdI~BfSd=7!T)iMCAZkl^W%X7gj%@UA4>$_5H0yX7H zZEn{*1+IC%lRBm%W!zL?O2GqdMK&EcAzBBM=!S~C^o27S-TK<9vOR1ySJbvr$ko%H z7tc*05_%6PQTEK+>a7C{XNIP!%<>q#rU~M?f{)o_ZnLnZI+T}TMizRV zY0SuqoeK}Pwc2Vu-W*k;9c#*K%kMLCDnq4Tc2!G#QbCr4Zp*67_XvrdS+?J_tLU zmp@lr>K#W?;vf-9wCj{_NKh!@}AwC+B^H z>2tlHxhN3;oML9u>h}$E&?*7GmJ0(I6RK&Dz)FO1rFf z2kx!-3&GMA(9V&apBEB@M;5=9RcOh{){sst{gMy8_F7-(wo7c$SfH112l76 zX4}I15*qW35R$WjAKCJ5GOp#g)1!N?J)3==IJUDqc?eA9Fb%yMEo)xL9f@UFba=2T z@?b)(QYzz-*awKsV@8YX3h_538M~3y1GN6*-Z#SGPn&2lv{*A}tovr~6tujYD{D=n zDr7x#f4@a9ik&45lW~5Q?JQPS67*u9w@Z^ohh%2M$~wn~%paJTtc_UwFv%UbNpV_! zC9Ik~wT4J6((zOf25I#tv1Imrj-JL{A}_I*hs2Nfw{4e+b-W~~6?iR9-7;M@W9fZF z0h!!w_P;*t3n_%X?`jlA`5{6J*}slrp;^k5X&)W+Ib=N<&j#LRkaE%TuRx-jm}RBV z{PX>h3#QPD;wEQg1Kgx0W=S;D(f!J}%yXyr;ZItl7l|PEfP(=r1cSs8N9jLuRJw^6 za$w?a+h)_cEI5~VS}=bq9N+al`YS=)G+}eNw{0hfIYgO$^B zr(XIbam|;{lR(XWE_&R4jD$Tgx zMgfvPcP$jszupMv#JHoaIwzy|r_8apn$min)n}L-n9P4~{z+mG*gW8UkRA=ChuhTQ zfglEX>a!2KT&Pb!FcE@J9feLNb>y%aD>K_UT){CDAs)iV`hh(qnxjz1rVxfX8Z5y_ zX44#0ov}RxmDE1|tWPNOlXBd&YUxg2=iP@pS2kP}*jV#%okOgq-3Y9XP@!;c)8wZHe zQGuQ9Y@0al@&s0o=o}W7pT+sj=@tEvr0pdURgLUTW4;n#C)AH=(;uc5JB$$|Gc}Ci z@Uj$COzhXcWNL^9Dj3>kn32h?j>@YYX*}q>ck?qA8d^rTf?ChgwB%=X_Aywh`uPk~04e%%tBt zu~{(c;D<+fmqa=``CiJgg%J2cGtHD-SjU3Dq>QUBv_sariR4Cl;tA8L&o#Ybq}=&{oIkB-4>2@Hhr90IohFYePDY(7 zpm!s%-19RRg%AChwNOj`cM<2Ms%zraBzig1UQ7$YGN^(ZFRvMbUho=Pmg5$1TfzNclbAyXfoDXrZ}W&pA3@ht|w={?tFY66xi{`6WT)P9+fkGC5vJ zCe@oCkaj;&qza!aGio`L>6%<^k?FHB8kcp<*T^%rLyU6_BUCHv20k#ExA#>%_;kT0 z6Le&z`IEttbuemZ_10PF_<%GdgEr;hFS@9hMgYQ{@DEdy83#-1m3Ojltzy3gsXoY2HMp%$ z1i5h!0UQkU~A+I;Oojo2qm+RtVbvackiI=5| z`gXktiDOIIXq*Du)G3$mT}=LGO!hsDlA~kB`YiLHRn4fFX{t)wA%em1#IJPU$wcj` z-~p)>_4R{lxc1+3zjyeMpBaWzJsX{}Z$@uN!4y7n54#nhyd-T6E20(M-pkHZX> zqG-391r!cSVq_uc;WT&YDD{V5nJDq*^2DyZD}$_yA;Acf$cXvoIBG*b7O%vU~i_E89WZ%|R$T z-`u#fiy*%$iJp6q!GIVsVPr{jodyE|vgB5t!t~lJ?lh1X~;gGqNZL9zV=9Y>9kl)U{*3|0@20HpvV}r{)-U zLZAX0vyyj3S~}`cy(#kJ4fi4oDCLytZ0b4K)>v_I@g_>L@Y8G9n2KV}KO4e$$CpEVY|k^bSh^*-=+*(>AX^$4B}!Y1%t#`zEO45-y8e;U}P z0@~u8^XK)ljiJ!0+Gy*jm2vrZOAvSVQ^aDfe~^DqQ-w`XlMOUPp7-lmp;LvUr~W9d zgSw=G>Nf$h_sj39m8OLQ$R#B+sB-+p@IZ@ry-Yz-e;rCok4f-bNL+c}?cPb2C+U3f z25QiE#gpCOe+TovEv4j@`{^#FgQ8gDNUrd`tBKcRU;O0fpe>Xm6FY!*!Y4s*pGs#^ zN|*jM|B`aG0^gN-LT-L$~5d7mVouqj#2k zsfMf2{ZTN?;$U*0`OtFPG!4F7kU^oy)r^egmE)n!lO9^hF9J95gIJG8ryVCu$%Gj< zlJ#B&rj2pG+k*1}b$JA5`3$!a$4uLwLx+E`Y6{80I#$!g8`fmpWkK) z;w{gheJ&+Rkm=``feDQ6%&yaHbZ(MWYveUO9k{YKo9q3vFcx!15E^Px&RMCe0Oy;m zd8syJpcN*>>U4!r@F77ia3FGNFVBxkEENxss|0sxTwjazi5ox=2JnH=>I@Ee;Pxi; zbVUo5jiau!Jo)8mofClvgSS`dZO{Yq8a&V(hoghH>MX>b5Kn`)00s}82jY*~;x~li zH`uOK)xfeD=~!~ef(n)k zcLgle7{C$7qQF=bR4}N#LGJ8p|&TBw4?@m`ETj1V^`)zl{2NI63B< ze^ug&|0Ws|q;CHePTvmG8DWbrZD4)w6A@*Kk z`UYx=jf^{e4ueL@F(fyBJzj1)Yn*?k9X1#kZc8Rq|G^?zQgW-yTf zOvR>l(^gItoNwZ*Idj`YvI>>E?`^^jgGtT-OFsFov+RLf8>&l~2dGK&;aHvLxrEW(#gV4M*r% zytALfBC8V0Q#awB6dkejqJ}Ew-OBjFE-!&^Qt>^)=lSoJ|}-f zuu+ZDLK{dr>Wgp-7hd5=Lc!MzTm-%j@b}5QH70Fg z3oDg3ZSSgF_l{k0S*4(hXi5x$>0MYxgzWgr9kGK_B z)g-6sV&3$sNUYC!zBo4v!}|2SajbES&zj?d$Z!E{{Fiy&3*X}RTt*5?;9neOpvcQF z9gq#fuy8EfqpEW{EPXK6K&3aRYd;uWduGz|!0`A{S&BtDZtaR;cMaOT@jVQZEGH}^ zG4{!p))%S5_i~pE8)~a7D$7!g1@6dw%d*aj5ue;@`=Zw+hbhUgH~nPqPmgh>h&~5S zmT9aO%ypcfHqIf@P8`d#Ja3)I6t2c743zl5sdI*9B^FXuI=z|s)X~Zg%E8pBgHu8i zH}vG%G{viGGN{tU(n-vF7d7s*&Qg|+y3;(v#K=U#h4!D0arH4h;TGH+VXFw3Wk7Q{ z+-WPxosQOtAt?xdm0yCglAPZknQd%CFBg{;Y`_`7n55&9lHP*#J@pHuj-!=o=XYrW zL$0ckG|S28$&x!G&2Q+8x;ZmzPR2$&MIn-HZ0K4k6(yr;F1Pw?rS-y&TEOx%6V}J~14`GC5w@l_GW=;b=UM@~O z}p~-eKPFkuJ|&*kwKBUrkj^_%|ze+_*lk!;ds&Z+2w&Nq+*A2 z9S}nQ5o!4bbWPZ`F-eSl@CdLE49ThYCeBXfU*H2 z0>YrExA|=&3!4KNwI>GdBb5X%bddW9iVv;~1BhIAIAtS=uNEeL{G|Bhf}G;mOxpx$-4D z!XI7C=LB|}T$))vmG+asYGDq@-I#-ciB$^>wNpN7HPlgX`zNQB{Euhus!s1pmg=>8 z1JE(*MAbww^#QΠ#)kj@3n#NO^iJ&(O~sm*K54FjPSZ^*-5tut?%$zz6TB`Nhl< zPIPjhuQh7d1bz{-s#H7Zeph?j*%Ojl-fu?qZdpOF1O+1ex zGQm%P%9(e@=}1N1pTaqPES({ACOj_R9(*W4R4G37Hc{Vkf@T>)&hp@Imz)3Y5T){;^T%8U zFRysA4&na)5&s{@{|nMnE>m@NVgMcGINr@}Q zF)Mcne;G@=tl>^fS%;LquH?PdTkw~0lv}y|8`dF#Al@?yf_nb3=#9mIwx2`hfYpHR zqlL835dtCK{x~&G)icE!6cw91oSxI}-f=un=L<)HV8%6trm@i+f3n+u^(-?R@dn>z z$2ZZ$^4d#gc$Q1kL^hI>UXr4>sc+AQiP)#tD~_bZZ=%l9+#M|u+)=@ zYi2Er4R*48xSXYtU^X4f^uZgdRU8+$K9p(~I1_5P#6|Z$8z}ls=(|ORw3eQ4iFHm{-!PN`Dm z{y+?mm{nHt%`&I>^MIhQzWa(+8cDPE$kXuZJuLot4-s<-mDsOtL*X@Rx-fRX|D zoAYWUY1HK_Gi}XrZ~k?e=8}UK5#N)c)#BAzPGONKvsX{CQ9@P_`Kv9xb4_aX9bW>wMc(&6N2R1+s^A*lv4(48?7D zen0Az-Co@^913e-RdB~jTyToA zPnD%5C8u_^mlX9S{Dn=8gNt*sxWq}JKiDI5Sr)1xp0{JG_&kua0epmyq#85ga#=T( z&fkv52lTTw7q|p^%|^Ih>o2&*;v3qUlLkTU8K)DeQ}Xg;%!HlRB#up)ZDdfrTJd`* z>60F`_Tc-kHdgu~F)n^X`Z8m2Ohr1LF)FVo!sy(TUG=`JJuime@F7Qa30*E6AFCbd z?SVJvkkSvnnn%Z5VA8pgp-;$+fi)*)3hCZ>ET<}WA4OlgovNR1VdEs_93?3I)%Fu_ zNB<|-;Ru1@_Fh5uJ5sh{yUymNSc}F8T`Dud3wbuO<{k}Z+5c*AEx~<2Ci>AWjN$nK z+;{l%z|c`QHk3>Qzasyk{0~LB-Km7Z(w;q?nGxd;nFOWsLiq%Cb4^TK-}f+ng1(Dl zKU}7Xt3uKRT<*=6nt6L3F}YE0i`^2>_VC5$r@SN%rbW`kGW5m|FP3s!S#C9*p*(P zBv@{X$9^|4lE>nS>p=$o6N|~vC^DP@!v@sbuPS)=-NK-CbRa&Aj>!#XKwRqou2oVI zrpn(&wIH(A(0Pf1WS?SWiZ8F3w0VDG;){J>0-l%&4J<)!0p)9rD|#OGC8~usy3#v^ zuFd)8SxKPs5DTJYG2!2j5LnWNMMevA6U@3&O1MkKMeZkh3uTdASszJF`%rLapLN0@<_RK`yw_GUF{RCdZA*2OFeHfP?)F`$WpW-O$7a|cSs^k z*^PLHYI7s=({K)iqF=MRs^RAINvS16WU?l5x%D2PwlCm5<+=)<7K)SSsyZ4BS1ORd27i8x!Pd-@0|?G}FJCQj zmz!LR=A!hk8Cwv1?+SH@&wBOW3N0uuD(yMErgKK9jqQWL`8z}V^|L%w4^d3l6sZ!UEo;u21twfkTptSgkVr)Kj-?bN#Ho zGE!1(eeYfk;aI_3SDyGQRef5MUqP;ko${y1k8j>{9gW$s^Ubzs3$8c2GixpmJv=js z(8(#QDD@-fbc2`EI%E%hA8K?o97A;)`NW6NZwum~c%JX_zK73_`;@8DA&4}nJOw>s zus24sNqC%V%&e(`J&2zw+N-*&FTjTax6eQYed#ynK9-s4EPs|#qn!KnVvJbK>(7VI z3~a9)<(p*wp~4WxpUW5ZrR&<#R{Gfr&1H$ZxV(lis^JhUbagHtL7T07Zh0#Ovre4+i(sQ2(c94u^GpGf1b9UG$tlb7_t~g)yKXPxucV`HRRjYsUhEs{L#*nwqEe%Pd^zfN^W+q`#okL$^<{!b*W z#}4_YUfx7G2gWJ9U^|LL~G3P$nl)RNRiORd>N)-D^^I7yGL6u zPJ)0V*@2)0LWnjoJ34!s8&^BXu2)`E{GnpuQfJHJit5dCrgw`H4%fSLYWgXg8E2u4 zLG=Aj_%-x>G}Ie^eBP{-unu_;tMAg79a`La^WvkggU7E6**}z4Zj4a+asiH0^jKPy ztd5cO?C)@mYu$2tJ><#QubyThG!?b4aI*&Z<)zMTy810Q5_|djA)kwLKTLjhaT4F zW;=9zNMo)&FiYOK<$ea&iP}AhRom?N)3N@aRe1=xKfHC5m6%~$&0zIGTlteyQ2UP?j@m0-0pZ7MqpJoy8}6ZW>x1@pa3AO48$x zZMIKIEmvC!m=QrqZP~ul=A%ZRgW8?lS>Giddjiyq!Km(RM8{V&v9&&q;#-4iAk}>X zNf$lVAq^tT?XtLiwgH;NoX`(;tWMWzY}*g#AM|YT!CphXtxkRB75&9A=j^}w|JA|V`hWR0m#*?%0s6%h8KROpCEDQ-)trk$t1_*J1G`+#rnNAKd?|gLx{-jP#sh54{s~m?B^GeM{GD8YSGZeb8CH?^{`M zRLScrnBup(HuBWG}g20bC#prU_o;Ncx zk2{ij4#Y^rw(^Pn!2S19ta`r4)Dkcc405dukpP0t5Q)0>e|EI7w$YVoZ{7wtWl)jjA=Ppx>+RuFJ+oPS1u{}23U6D9DW!}@%v`*j|u&Pt>2D5ZMjGb5X+Xy+|r_ixdT z!;Xo)aS45JV;jqi$c&g9{%EDK*EOy+T&z?PboOe}?hN zswhsFR}lKtT`sY=3Ov`35+%%s zZ?Ecc+rA_T?Ip)CEh976E>dJniJ|L|h^v&q6FvM=?T6c`X7(pv>NxLkPPWo@u9hPC z+a!=&8TC=NE#o?{yRzCv@ALde*XkrdVngk7p(`Jn~vmH?pX{U62KdjuMVBRmRFq}&=xe11~@BTB%$+uLcb&%MdN?|HK|hdGJ) zklvG~YhV0#&aso8zK3y@9}O_#=SyfNjz?rVPC1d%2A%uvYIDV>E)Wl6uIYpteVZ1m z`LAAvTxg;f6B*<&Js!YIfZwGv4N~4$NsNFzc#gKaJ919AUhJZ`>^7#@isiU0l?R9-Uc zvwNR_cSSqshpP$@=ut1&>}+~+9*^u0_T)3vlmx1Dq{jLInq@h4t>+TsGx8@0D*#g2w(0E36`p9T-<(t4W7bb+#W+=jh_5rlN^D27io z0Gt<@8R}7j}iR(S#RxJpk{2#$5(1A2?$-zf=jD zMrhY5_~HUwfX=~`qX9gC7)=7B$>ysCUs$Nv7_H}J9vA|W_HG=qZiN8ap(3^kL{bYi` zw~Fe7HD3^3`Y0|3~63IMjh zSf5J8Fld3ev|GS`H-$-a02a8w#%eAh^?y^U{d+ln5%A~g{EdR&5QzM3p&JxQj}!+b z#&E+wG22|Tzw!c-ul`=MER8E*1%EP7@tZ`5i=hRqzYp<~fq#JcQydu#{N+-OdkG1e z4AvNunLN{{c1V6F9GK~WJmrWNCW$WdL>PwcPOm6)b50UqukYz|hr`*9a1E_Pe2Tt` z@8i_w(@JKO5=5SaTT2&tIW# zWU!V0^2*)CHlG*Ixh{sR7&oleEO81!99BTYc}UYzO7s1-^o2K0Wb6#>xFw`T%buA_ zIJWPPwz`n2sVTFgYZVc6FVb52q$9J+rF$YnOG~LJ`{O8hyrb{{Oy_!qXS#bG9xS+LH>hk@kcgH4EwO zc>Y5JRH86#n69vl36?o8HP6bLe%`orgM~tL4<=1p?a)iEg4~9w3kxslQ`FZ0e{3kmOvA zU`c;}BV{T37qC!A;X!~{${1AxV!SbOX4YrmF^@YYzS7{P)A7nH z7I-$=n%E7(<7L(PNx)ae$;5@8JGliLe~=7N)v~V|N(vOyd3pcE>NCiz%lBab&=XQy zdD$t*+mlYYv?`z%+poUQxC$1k)O?>0`ol~Rqz9{mTp7?fn9#}luv;=rJmm*GUeB*q zoiuTpxrUjA@p}IzM)&W9vdDjE>)-=(`6JGFm>-%$b;gzoN7sb&v3zwE}vR8*s3$jacReBU-V$)A20AzMn5Za7;Nza zibLhrDO1;{k?GBd7W|ai>onY;KDpvW@{W*?s>3wV#Kg~R`Af3WOO{?SQCd_kYE!sAPWMpwcL3wI19XcRgTb zPlw+WQoew3uLhJFptt-6kp?k~YP`Tfj~n+nelvV2?N?nLgRRa0FMWMyZQkgp@TE*+ zP~GR0U5VO`vWh~q&Lz(xRpV9D$f)wwHm1$VW&Ci@!xc!G8znrFSynbtJ#Th#&L=4- z{&I@la!jBj)8^z-e8F(h)50BZO+38d#%kMb-l2WaI_G?Ls;YPPrKzw9qCJA*nAl4= zPphvQ7Zi}8Q@Zj4a)+nXII^vuRQ4f0s~oQ`xdA&eg*UWWhrEFqBF*M@LPDR+vpTh^ zg44++MsCY*C|z+()=GYNX8{`YO*Ame=``Gok-;`^pWV14V{69b4LpFS^V?WPx7TH+ z>e!~G<@AK>fiL(@V2A ziDN?ks@wGDPeIc-H~_P}MXOnQr9o!xNm_@v(?lTpg{pDig7BvqRcb(m7$zZ2T%fZK zL$0>QwXQf(j5pp+kXPU-k_Yk^%ygz0@? zjnSTXUYvVZNE*Zx#DXDJU%t&5;Iy2lnXg9XRk#~4^U$0@ZaGcdj2RI;8dCK9bw~-= z-2hJTIejY0z_cDC;(c3 z#KzXgQ{~2u>!&>b_5GXwUmyEpzvTc%#ZP%%Gcr)>@7%8*%;#7j$`_gcE8Bl2DD3Qg zY{5r3@H%Jf>Fo=uKp3QN`1^XE(g7gNt8iC|D-;#2TJLc!+%jGma`%taLvMgtpC zelI4;h_qK^0;-j5%!Xgk@Jn~SBgrXPSlQS)1kVZyi=2~^zAPgvC$Dx@T|-k#`<_WP z&p#ylFLKd?auLB`P#Ec{To59E@Pg9Ah|gRkVNfw3wee!)my94|QcZhR*+wpK$q>za z*ZT_v3tVbm5OXTpZ^`~=f<^wnlI$}85&>iaN(zO+V5H!GI!ba%y8rD&m<9(cZo(u$1%-fMg3<#B02f?X zF@r3|-?S$H4G8%=;`JZcd$%{08F*r%kMmY_1V#MbSB(>SW9`(Q9MuBWkXXBwdE!K5cGuUV1n*mq+)5dIRuJO}Sv>fo z(ybSMSC?utQ^x+Rz@xcEA6<0wYVCaMAT1JKgOE7+aySfRnXjvzbRTiteX;`h?G@ZT z*6)|+Apj>XK)_EC+8OJfLqj4g7wr%MDAN_(k>x%fhE;z%9@^vWPi#5RT)9qMCj*t^ zJ}D+r8y;@)4HbFNObacl0NWQW5z)H3P5{Po$+nbG1mGiH`tF-SbEn=OftInif2a>2 zo$+YTG9Wrmfz}0uSs&;9%7vwDQJ8G*F?C8*-yynqTK-jnSE_vSE=2qic4!zWZ}Dff z*(9C{1Yob2$jl{lcIfcSsTx<~t)1q=e1Suy`J;GH{Qce5-tFz(e`#I+&>3sACU&Yg z=fhKlg-aF^01L3OM>_4+MA8y*2=Wk_QmjzNWbG@|HbX$UCB@kN`pSfBS%zO6Z`0d7 zb~F#ycJL?Seg>(tGx=PFEc37!UrX6|iD8{Th{3amaK6LPKCJ$Henkgc|=uCO(@rO-ZZsIiqSgDD8 z*L7BUOR!y~+WQvtBy%bxhVkS;sut;#(iMEWmGC?4x=`Ep_n{7s@*0w~eDfJAD;L&% zK|#r;@IgD{cm=<^1uP=h`n%X8AM=foMkxt6v06hB&gK$y*yQhf`gGXX4*xN_AJalP zK}NS;=sYj`GUZ!fdu~LMNOi6f%6#C2twO)-o{>K(P~u!Q8;YvY*=AikQhHU`5?4c# zp!%(>%qCo$t*JWIe%?+jzG8suPWZ7|{JXD42$3>`EdiKRB>UZ8Ef|taF_% zKBmt%GcN1$cM8QAKU0^opAf#NU>AvquB@NmoRwwznKY(Lv1=%o^_@{8nbMp~Nm7JjDED2kn)6wb2dx*^5i}Jw&^H|mma5C` z=4Z3FH|ecS#TeU#`qM2cWxbi050lQ{s#Gks{ngUSTe-c-i;=vdXHABcG&Q>VHUG=@ z=g*(Z(yE&uLYs`lKUKbe3?u5gzja1MOj)`*=ey8;%vczF9Z4>RLPx7{#Cm6`E2z%iZvBg_?ejdE2O;EVWZZZO%e;U5eP$k`_zRn_Q{nNR%|&{Qph&FssV*^ZV(B?fE~+Cvo^@oF_p2;(1(qm^!c z6TPD)Wq0wNNb~quqmzslCrbg;`RC}@zuM|}(dp}#H~sA?bYSoF z-NLJ`J=H!_cd?k=K@zBk#KECQ258rA1XT#F*8(EE;b|8)BZ{TwZNMe1QMc_T6@UJK zyP^pq=Q%<-D0`>brraK1zxVc4T({(9?rp5uq)ALjFuG^HFl}Krwwd>Q>&bg5)u%WqOnmTP=1x-Kz{D1g`aiz25?oG$_zP}R*t zhp~#p{A}25$cgc#oYuR|F}B1#{krnfv+tQ#nV^w}^C__*)aaDlu4&X2vof`u{89QgBj4w(QR`^u`(oSh)>IV*Et0(IlLWkMz(FTo z()xiC%ZG1GGV`Ck34nIw*4o<0_@JxvjC2wYN((mPZty{ajLK6PPI1CC)}(`8(MGSd zew1rcK_9E;v?1s>JoYSQI^TqgLD?iWAc5v=TJ3iOz{OL&_H(QI1MfaduA{Au{ z1I%w?gRjg+G?}2}wqq0aL=zBIf-@z8)6IEBC)oi{=FKG8D&hCey}5Z$a3^=kocFzHB;!l2;m-MiFgxI#EbWm;cIHfS z_V7rJRBYC`>Z%^oTbWK>Dfrmm#-S>h%r5x+v9-fSRr>eoZcjzYukX~V9dj&u7b!Q{fW#nhx+u-xKoEv;!LP{*vb|u!cIg@vZ*tSMTLDPAwjv-u zdLLPhMBrCrH@Jl+kVRhDlY#)aS;t=d9MyLDU5{8l8J}!owtTqL zj{vn+UNU|ajM6kt2equ8iVY)AWEtbEr!4-_9nv8JWAq?5{K%v3@yI@9lC*384mb#L zb~N+W_RyO-8B1a&*zcjT9eS=;Wvsb&-#TAo-gmY>Yw3l9vtYW$`A?s`KXvYC3&FFs z<<8c<8g)OT+-&tLS;t#{7gr4ZV!oe~O~GF+6xw9Rm@I?99%;w{>*U2f zv_k$VUH&n55e{SBdxJE$);7)T|g2e3(=Pj&Aj|D*t^2{lmGI)PiH~ia)EWIIT(uR13IXQU0yapB|_3j-1N- zC!?n0KMkQdu}Su)Y$~Ugu%O*d5-mCbK+)84PnGE}aisTn7Jr9Bv*;vnJSQ}H6dzEr z1Qt70>P^T`#VLh${K+$A}3G!3m6rmq}BeVr<8i-JZ|B6u1 z41X9q?=PT5@SbY)&TkVVhoM>oVDXyqsgQ|C5bBz^Rw)n%6<7W=w&6+qFaa1~#zg}` zyf`WAF6+ZnP^q07{Sux-237v+)S|!Jj$9vUA^?9@q`j}8*L-c|r!e#jV*WM$903^W zeoO!&xxqe>Ci3uDBjna3V#F6j>8aTNR#bDMj1*V-8)%5%K@a5(K2Izu z+zK4ZkT{?PUq{9ZpJmcfr0VO5jwt=hY%$%R=APY18;#KUekPdt$FGXB)?2&-pkUgI zy~qdnTR#baZOw0xP$>0)UF_7xCKh)QH3w z;7GxtZhbrt>=6~|O~-%Lo%fw@U$lNj>3R`X|5T#<9lM1k#bcAallarFvncXk8o8k& zg7CyEnluKmcH~eidgPm9_00Q1E-uhFsRa57hn_o`eyHUjc&m&TcCPPf;+zL%`nRs; z97q1SPSOu^_{X1SO|K+&EQ@oVj07$Ekn3-*dFQp0>m*I(e5k{(j+ryn=^cin zOHPiF1~m(QvV19D=~Fex;B>AWh!3JaY3Ti&`XZhn;D;7jjv zH~Z8Dv0*6>h;90fPTK3^L&fzfeO$L*Und2+MX|Saa>bfqkt(^oKM|AM5ZL=FY|={*3_a&KBJued_d5>+z#VZ&I$Ku_c?I z@yvpg7klxy>Xli;PYUoc$by9{W?a;*$&X7hQi!z?}DM!+2|- zFmrCPeoAXh*PhF<=~C!^!cv#b+mP{azHgo+I`rrZ10JU1a*(91j zz0NRUHW2LjBvo;fxmxzuS$= zv#TKp@h|UnmD^EY$|KpS_}$zjiHU2YGKsw6d#b7$zuuc4-NnM|8|wX%gnG`3Nu&ao6g`o+74vyHi1oIPxtBshM!;L+5M9Wr8cylA#o7u1fm>E%F51b}?TQ%8|>w2Vmkh7!Lp6mpADuiyxFC}bW_fYa zudqE`GI-H1imoMU1ujANi!NDC>d@rzMZ~wAx!6mbxbmq!W`s{=6T6+QrJqg zrwdCn4>rj$Sh(`5ovA~0cDvR>+SPbDL%UX3#$zE_K7jVQG%ayJ%Qf`O%LQtE^^Y~f zH8H;LTb__JOfJ2K%f=fL)f?v1<)^BU79cwCWSFN3Vl~`49URR270&r$?!n>(O&JC6 z`>%D|+Zobbgo)ghROwN(_6}hH`*`%!F%Rwpj1m{d>bXzeTCMukv8P^AjGdNh#Q{G^*o~TZaR3SRZOiDrK7 z`OhjL;+LL{-n=|QdDe4T@^xslslwOtOPP`K@kc&*3T6Tz6tzGA%AXQ|!U3}rA}nRW zDlc7&4J~B|X^YDKHtz)XnX_9b>kR(4@ zC+9|9$*?-~PwJ#pwO@J;-}$;}MJoi2TD_%sao(CSVB9RKBaC*Wyym0vqFnq)KTEbh zeTly)2T_kpWDhi)T#@1==j1`t6&v)>+`ZS*!S_=-bC@=38hal%o9JFrjqM!}VJ97W z{*FeHjpqkm1t)}!F2RXlAJ4bMP3q}(M}#teQTAHyEMn-?&rf@zw$U9ft5a%TDq7;^ zao6d*_;g;n9K=LZRZWygl8i5stXF`!4wXWRt!O~uHKwMvw5+etiG$>Uu}D(Z>$hII3bho)>cFM|6uPc{)q9*&hS%;EgayIG3leMsb(06;LsK(zV$4vnt^=Ny7VHxjQ~(? z;^naln%`oo3Il|?XLwt_vGng;O)wh0w_$kp1-acVJN}E3*$-Cu(gC5!mJtHraH2U! zniS%Su2eL4p9>W-A3pPSH=*(8Yu0i{erIN*Pn<6p8ri3oLb7rp%DnWN$A}QGm+b=! zPEDj`N|F1^8&(PAiM$S!%j|XTtqG2wxNrr5uw>U>(Su(LSMbpb)?C=URx_4TlqJy> z=)C1t`PhC#NrG-VJ(sJ;J@@cCvtmih^=Dp250WCJaq=e#N<6Ee7msv-mK&?kWp(8n zE|kcsKST-B-xSsQlt9myJ9wKJX8X$o#oZIWg>x>o2)S7O@jbd~ z;3b(Kt7fBCZQ?@?NfW{M%CI>)ayWysY`^g9aEkxQ)rlebHky!4A(ZNg>m3m185&Gj@iX<5o_-yPGjr4(ugp|Oj~QOoQ-#T9PY zD;&snW{hpbP>d}B7_H5*66A8aVMpYp%u5)-1%Zn=|2^i-gQb@Fmr^ux1mHoV$$|hr z;>a%`DLCd80XRDaz6wlCm4a*h7QI|ru`k-+&uhr>_SdenY8<~m(cCF}18jkwli(~8 z+IK^&lkO3M8lHXGxmv6omcNdw-%k{ZdKZkgpq+My1@mAG7*{u|Y*xICaMry+BQXUKE(pJ|< zaNv8w$jtTGakTF-O<{7w9&R<%!t0KD^8Q(_K3w@0WL&*BU%!KBifL%EOK+AQ8{+%T zggTFBCmlP$`JEm6JdkUor{Kqco9}ID&baH|7Uc8~HaLA4AF~U_sViK=-!V|;Eo_k4 z^hf)5A<067=2GH&y3N!NqOI1<6FjIa)!c2$YCd&4u-a$AfjS|mXtQd5`o%od?3B+u zZ&I@`wtu~ohimM-5^u;gG(M+r{eE6=%8it@nC zZ}^NNwnM;$6|eLTW8bkBbh%J0FJkbmvC)byY((R(hUBKh4Rf-KY@|-OK2Vfj802QO zBT9c_UGpTzaO42b@N6k&)~F>-;8^j<{_V~W6^>rG^&S1%)Ka#|(g z@dH?g5CRou9Yd{0{IIjyF|+=}iU_UBB6C-kxbj)Jpe> zME4BOyAPuwyz|A?>)ib9x71(EwD-6&O_cYt)q8`{)N|wu(mNlSAV=ny;04+ZdQ1iC zY#BzTYtp1s)-Tkt_flEz$U=B%HrbqI@n|E}TNl0Ci;IagqC>u*J3X&Gj1KMJ_K~u6 z-z?KVhq6htWX+p1jZAy@c%Ufl3{5(NofB8jS%aGlo?6SzOs~TcH)<-TGUk^{kJV>F zw4yY=Zs1tm=gnJaSMOh#niA~vv5XKHfQ!mLRJ-?N*e=q>JeAo$%DC- zm}c=+`SEB)xwq=`)X$tg^A=by@d+*{gpDO~>9gue}Y9m)I<>IH@Ir5TLN|CZOf| z*JiF(v9cv}*C&IiHDiOgG1)C+Z2|i`X|K~i2*F}4TP2o1GeHil!8Ze!6zDph8M~SL z9eaO8)A}J)E%s_%lwIm%TmInS@|tFbXCQN3Nc>#AKr}Gk{>VU?nCX9b-3^GvtE9{> zTQ7^=t5j4W0DR*FAQ1U1msSU>sE<2i%_2(xo;xTl;3FqP10LDe(kKMQAJ9g%>! ze1|DCivX;YhN-^&qRiY3Iv0L@YdNF&eHI^7w|EXjsrmoWe41iklw-7?@zWH<0bysu z`TqrgJJ@6)U0FOj)lvmUTjp-MX3P;TZtKG}=Xw8vEH23oTQGJ-!sjySv!qJ`UnHuzsjBC5;5pyt%d)bO@qnl4CO>2W0ZKE{LS`D0_P`Q z$A|_D+)G`n6@z#*w4AGA=meP3m0>Bvw2>|7>nSu?Qf=!5L~YJPx7{RG9il*~fbu7@ ztn3ON5)fd8ZD$T%$wyA!pGr&kiVK=-j{MksVuEL{Qkk>GGi+KD09|k)6h;8{b%Udi zd9e;3fw^C(P3x57Sg<23%aZ~65y>Wp96v>~hxM-F===AMIDR?-uj7UPx3BW5)@G)h z9-i}~ilXkq5qsqKpOX!C-Y}iwvWqrU?(JjiUAM^cwPL_rel<5szls+csePHHWWZm# zSV~3LJJHr5!Z#Nc)kYG2<)uEFwf#!4cGK9dyU9%*cS(Hv;KC*)1@E~nBlZu5xk(?W zRMXO`SLE0vlwdzhu_m2I6*&Un%w8Z^Q1A{2Eu3K&NLq%n3rLp0=t#YKOUgq`=G=|I z(bla=?sd=n)Wfan)$6(ej_GsHxzae^zappO0%)Li*^dDcWgBE9TjOg(gk0xE2_&v^_yB`_Gx{nx{w9ebb?|r%}qp4u5+~MF8 zt^&7>==f>Pl&i~Ygg!`;$|#?3k@;Mba(=1oZ2V>$4NBd(|J_^(+m>9hAL1c;c6CO0 zWK&93YTbjb!xm;R^-jH3N91aF&Qr;%RR2qK-ahF76C?}6*lxxetCvuMy}OKBxHkSa z`Un3{zHEN(JUj0k5xxo-AlEJ1*Ndikn1{{i=5UPNn|dy`Da{^|a#`%2HWxknnfGMp zZ}Ma^_;mT6KPu`BmALxFcf#GBHpcC9qqa{djK~IF!Dr9N)~(^GES_kWmMSV_gr<$d zDcBu->-rosV3_nW^x8GEoN3{B=8pN%m3n8rQ05@-NxwE%ov-TDBxe*nB`wVXZsN9`D6u+06BB zCzT4;$KRgqpWs{+gL`)N4n(D1P9+~2^bXOL3`pn>I@rI5m_J`rU%P|bILX;$fWNGB zJKW#wzED5teKE=#nLuI=ci++}KVRzDki%63cPAiO|jS8WjZy|E#sM*|lB^2j3 zpkJ**Bxdc9m|wv8$u_19*aR3TGBfFuTw4+wGzu2i&03;unlWORUw@q*P`L6o+IDBe zS)BdarQ#mqclxqyfS1fs&1@$z2e+zot&y?)gB$PUb<4|aT}d-|UimPOkty5pD|SoXj_Bos zWK)BlfdEA73(|*8mPOX;#Y0p|ob?U2w>Hx$Us@ENH59x-^@i*_KlKB~KCbZ`i7rc? zt+ip-W=@@TEvW=)uaa3OhXrkXcPQ0l z%~bWa`&n)OdV4WD_}(PAcaatj%|pwyc~As23|-e6toJ%!LZ3RT;!;vHDJMkVJ1Xo< zF-LMSt6mnCm^SF#LL0{8Ji|3Cox|_J5%f5&T#k-X!(MH8~Hf*N=tW&cVddhBKWD-K`7lKQJ zhd8mIH?5Q5dnDhnTldw3E7$lSDgu$peG70D7$oc@06WxNjt2==Vfx`Eksarx1wRe( z#_YWK?m)Tr?TwGZ>}VfvWf%(>Fic(Dx2DF~zQO_6P?9L#Xek?{>y@1zE^*Nrvq{?x z5tluKVP#daCg|eifmK-S7x010wq(3=e1%u zBwb&+1s&LeipN&&@20FX7UH}M5=Hzc)Nq4jU@5?`$*jpkoop7L-~&P-PP-ZLYn zd825_VE9gUH3VddPEihN%yQsgsL0?-FD3vH!~h-^@0c~8azuq(Y>Y=b8;(+njArYwl3Bc9d>{k?>rl&NvGq^WOnaro$mfa zqOC_II|~3;IQvUQy25Zp+d*{C2?=7y>YCy<6>=EVjNoE9MaT88HRfrK1!g*daET3M>nf}PIl#cNPl~P?qM_hgprJ5&)GH+|%?AQyw${k^783&V`ox3_Z(FlS%FA z(=|}atrn4&M)`iVy>$V0TmDnYCiJ9b!mVZ}E^`I8&H}ymIlE%FJE>%FN7>T-Ns`7u z-zS$}_|VM3``ULOo-DIa z!nGMM~m2>f&`& zTW!Ye>Euih{r_N!w?ixeCfA3X`%aRIf4H1oGewC=C)Os%FualVZcAX{dy{Sgy-!Y5 zKlKr6^{yeZn*iw5*VWgz#*3Ws4U|;oWD3{Q2JUcWD05f*%N&pXHjxGQ{?j+k*#=+V zE>|;_V{sGTWhxGydb0B@@K-pQsh5|J5BT|xMUIRFKNf`!Ei0;tvXjxY_CyJUlzzo1 zbNl}XmN%Qo(Z{zj&2>f2C0t`qWOkl4Pu>YUqd}oZPJA<6QfVdsqy6IX?c<2| zcAzJqY70@tpK11S{1IMtP-C7xgY)Xkz)3x?!Hec&ywrGpmxj+hu~IFRE>B<7yb;SV z?XXvTN-q%Jzs>N<9t=Hc0br;?3R1BWV2h99J(LjS@sC0L1RsUwyR@m8W{DB8)vZ8p!cm3kq zE)Bj6?g8RjtY@9H93yhY6kBoUmEm-)fLHThavA(3tJnH`4#So5te}Ql-)qOksn}U{LzwMGk&+tR`Ku1z;XV3Wxl#?nL6qUZD)|LeEHMjF2=T%$w4jR`P3&#ZAOpbsbqbmeEwkM zk{XQ-xr`p~)PIO%Flp=Pa@c!YhJ1hCkHqP1;SP_6t3Rj8-3#m*!HiMiwEq~${xOa` ze?L;qnZl;>TDSDjOssC;-ehWcR&C*p!j)bJqjv6NO6WwY>u882TBtLO9lJ#Ui1;2^ ztx39ExIOv9O}I+k_hyIOjn6gL?{UYVSW~szq#&RE?&p6G#jDngcwp6v2(4GY5X#vl zWLf*A$x`tIJr zG|Pu~eAuEP+d3D07wc+{A80t1{w`5uNmoaotxOuerXaZo3%Y~W^{v}au?fDCna>g( zpDxeEL$)uFrpzpwjgi4cAva%-t66M6?DqXKhzMvI3%?>Ww~c#f&4S!8O#!EeF=o`o zlQygG=m42hmnRsu0$rZ7dn}{town4Ko}L$1!RXf5e^?APn<0HoKDbb{pg+F=dZ1mC zE%=waYrrpY%B>;LduhHm+|E~^Sdor8jOj;!zg2Z=k5`e!8Q$H`aY}O$ zkyqNfEb5S~X3kH8q)VD(^;?ARiI0Aw`(%42wSo3=d|E2#MgYCVH?ht)z-iR-K3u^Jl?PKvx3>D|zGKg##Pv^_+ ztFyPZTKw6^*q52Ra^s(g7q+(aZ=N2b2ABrE{s#R^~4iXdj!tuEYC{a z<066*<#9!Y>s`#Smc@3<&SUP^F0A|7MjuczF zCwkq3{ha1G5Ob>mhG^$SSM(13w`cIR+#F?KmkBZv!u`G)GxqXLh}gIDEYs%;ql{|O z24UXj7t9lbYZ@70AZ>rphL1~~vsWS3(+MMHLuHJb|MHAMC*xA@cfN3CzB^P@i<$N#e$i*1 z^m}Tm3V0$P1EJC7*c5}&@(`Pn+u%D)m1!mV^P@{m{z&I~)9Y)Eh0hHUvvCt7qM3AL zZo8#()V8-?jLv^s9IsM5NI*T!Ruf=-+*EexlUex#YM0SVT3nIV@?P5Ga3=?Ug1{P_ zq?ICvz>R$e;Ev(;aHU$TPs!~BWQ@!au5IOsKHZ_=;GDzer@NANcUmvnhIevZB+W?N zsBTEgpJ5uosTWL7ShU2871VK*7fN_|bCGjM(~Hp6YG`{>-U;`4f$jKri^^4(*(93{ zdV7*wzZbwo^A++nL=(ePhmv4jd%tcnG84$^FT&8jGzS00Ny!F}Hi!t9El-%I#f1bk z;=9UB2!J}p%C2bKZmDMN*CuY3N>Q!bI-VZ%?WutBnJ3cV`btX(zPJEfvU=|48t{5l z@eEJ$zW$eKpkHcqWXEtVfA_P_ecd*TPPPHE*j=MG8kT!(Q*)hi3#CtdO!k@i$!F^ZIf zY0S=>%v&>?o6nd|B~0XgRW1AK0>7ClKx)9BND2wR!^TBNwg^?`t@RGLG;cXmirDC_ zn#z*sZ1(6RY2Xr0+Pl4^#i|In8RWsEc9Nzq&_2Q`V#5glTNoEkb1a0n&_#ax;hqQE z$CGa{DW&B}C0PP}w>Mr63U9*ciOFmMn}~ZPD!6of@3)qfe&q2ks5pIH3Dd-_$Vt&C&Rf;i+JkqoeqmX_=_8LqU2jN!=H)R~ibP==n_ z|F(s@WS7l7E;jJsrT)##M;|YwbKK72s$y%CV_W0&XizJd4f|x~-=+@)yw}=$$@$sG zIO(6LE9}ZVh^;GV|Wr0E`MQJ*EzP%JF;2FpUN^ShfN?6|W zOTSf?qRvW!H+<*nhpA=Qabqo}k-dk#3kVze{yjt%-BlGL{WV*}Xx}lur<25>V`fM& z>(j4u`@E|Xl(y_Yh<5b34mwikU9B!+Z=owXt%yhbsKbI4rW&(c+!Z=0^V*zCKKrcb zo-HAdR}=Gi;;AL8+Cdr=g`N7hew^K4_5MwN?NjieE~t^j71Cr44JYWX5o)FLbFea2 z0CmNBzWI$WSq<{xi|6UOEL$p!(eEFAznf)95|H@(&V6AwXe1ysjJtI#i|F2-RO?Yv z8ZQ(2s(2;W?B}S(?FlBGkH$OhmuFNwI2HIcIHLrfz=()awe4{Q|AdC-zmK-S(aOS5 z1ky<&E{74G(21wDrd4|S>f=~eevg+M-@D=JALRlbmt4tqw1F50_?w7ur7SEvVpAB; zjWrN@Ae|(13v0?`WmR3-*7bN9JdDd&T*FV;NjFxKSwCHzd=X3g$ZP4yFJ*g5X=y8bt#P_Vj;?)Tlv{2Yb?T4u&K!W?e| z_6f@<8PdCrZZTbczKFWrTZg0N2TASus5c6G)p*2;_Q&utlaQoi} zSl)lAKp46-hs1!#ojVDw(1Q>%W0LWh2QP z9|F$jE&6yS$f{Qe1-1Z@;;zK5o(Dzg9IJ#9m(1>fvqW_4&obWASfafkQZ!Z1CiWB{H;rYxzzp+EHCcptpnm#3_hn1k z4YKbnRjeIYCgZuuX5!S%{9J{PfgH%R1$}M9HT9XAEIFQAK@i3aTrp8*r{#2`dF#7} znUCVHbQyWgvwAu3IYPzsXlxuE?`~;3z|#Za+4;jz*Z;bVCjgeLN5iCrIPffSnkcI0 zBmwj)tWLH!X@L9Rr<7>WT%xy=raxw{ z^VKBwFEU0<(Bx-;K1?{|*3ld!J%}QuE9m8fW5;DNgEP=gHyWxcTS0BEO;t})RTZIH zK%V%80%Q>pcc)e$GHpOKJd`}i6v^*AJL!oW9?8iXA^`MOD(V1uTxma(cUP)t#Rptu z8JrZ6o=}S%{vZGt=GHYH_ODVPgp~w81bi;69v1}+b)sDQD?ZQxuy@_$y@Ot}*}1gV z_|YhQ%p=?u5*v@G27Bip1-MZaAIb<%|S4kQQ_@fZF4*4ril6pG?OEIpI!QF zp5sYY@wD@DK)LSQ3i>CcvrI49%70D(?B?u7$tqwpSm#)e_K3siS3P(HSNI=Q>`(|p zL5I;lDoWXuMf`NuA6x-9w^#7J4!?r@zeVH!|D~qwX1);lF+7*W&nFFwW51Ld^S`@Z z2+hihG@m^a@NM7?N}2Q%iSiY;VN$h!Ae<>Fn7VZ*_TlT)?8}XVxhFb&yw8~zrEZWG z&Nv_37YrMYvWpVoPyM;@%~Bf{ir^?PRbn4k>X03+_|g3QzG{C_>H{>~V%J_V(~q6B zf}42rZLN%tsqVqt-6C#sQg)u6;#`WFL2Tv%c=V|@;l%ZId2r|K4j6%~O;F~^KnKR2 z`T%4R;LtIJe|@^^=^hac`NGJ@j{o2@0Q~~PxWJQ@;?sjjTdc<|^DlO3G?4S+-+qB% zn-lpajgK8S2tbDWFml8F?~^i|l}cXUfKF-!U?4LL9L&G+T4kK?=37d{thd?AnXIVYA5cO!Gu>f@se%19RN zEBD=0?KdJ`s!r9Sp=e-6q%oh(@qe>w;n&1*gJa; zp&ExCsG+W;KfGr@W*4yZ(1w*N@^W||53~0DTWdhrrGxigO*$y`I`k+u=SFQyOUoB& zSL3=5Eu-$$7)GM&wc+rBd`)HVS6UWCUa6V`lR0s?TV(^(-Bl%4vpR0wS}y|gQ@b}K zuRRfRRxp9v%JM{!RG|>e*o1x#d{JNtnFgK3XZR}xucl1LX8B>#(WCO{ya0|h5#ObF zCtP6p?SVU;Qofuc+STYTs?< zfF7;JhObg(W=A7yhY1!sOgjRfI@8TGU@Y)-Z*4jFUM_7}8Mx2+fd}{#3e{t=s^j!rB}R0ix2!Aq4^35EI4;c zL>Mk3VA$B3!wkC+`3Xy1H^Eju7W(&*%0jTk#7uhq;V^!jHu_`P$x$|N65Ig| zviul_`r>tMmz_R_nIE6M1p<4WO<{>`W~z|8fS58oUG!!A$d*678fEWFIJ36UPvA44 z3~`{yP1ggs()DeuYEb8;AAU(*PPq%N7~TJ$cCP)ai93r=2s8+VRFP6}d91Jo(AHCB zVGYPYEg-lM0co}>21TVn5w#?Um<-}6@{T~&77qqQh&+U{qEtX4DgwHzk$_kNL_mQO zLkM6J^O)>+2x_%|!JgBflF6CO_jB+4+&kaLz4xQltkTCJd8O_rT;o(zhQF}Q662nH zwv+kz6k<~K4BzeWA+9L6V3-=~^tY03FcmE-r1bLft4n0w$Q6Hh0#rY3pR9R3Ipq^B zYV3$Q$e7xZAE3$kT&{yIfdUDquR=>+R2}>WDdXy?b_WcRtbX zBsWgtSg^{k(A}JS%C`&7gS@kyOU?d8|L^`^4G+>NHs)!YZxTx%7;Vu(OYC^yaCr&X z{v1Q{_B`@gL7OQM1K1yNP}D>D=+7Je)WiQ99|W{JHZQUxUM}g{81UmDdX?=5x$L=f zWiNPZKP!yIs;DN8QP?l|#1t!4{}&w*TRt~|x=Cq9DJZYjm$`2H4tM2+4g0MfntapA zX`bNym`ZJbiEoEL{v?;E^uJ);Jw=cIka!oYHiPm#j}|K;En-fvaPh^LM_iJ})Iuh? z+qga6cAf8c$?jx{hk;>u+7haD*FR>GiXGA{y5#VPzKZJRJUACHKEnNaCLpV}HDctq zFUc9_oC{a&Ge7B&LEUuQvC8${k$p~fOKORnf$~V}49u zeCMk!?S*r<9y;u3NeMA6@w#p>HCPyQp=cfPhZ^V;jkfw58_~&W5vA|NbW%&>$mn^G zbf18&p)3u0y)7;bCLPGE4CQRf3n@-rT?r`M0dHGGPzo^xe+Z$ux6= ztYjlRA7p4|{oDdlQZ^T%YPXgHAXj{pqL$PQpOY6Ur$C&wWcJfGgI*lj$r?9KXQFDw zG|w5kgl7J->j3=QvLySvy^g}8qpX>3HV0uwwp^-=h4V%89z(29 z>=O6X^)oqpJ`0v!tGqAo$SpXu%Af3cR2wfki;cGp>4>Di3IOZo@cp306ge{s%*5Iz zlO`Qs>Y&4EUe{(F}F{J#d-kWoY^U>2VTkT0D{S!ipji{a9q-QQ*XvZS?T5B0M(Zqg%HV0 zodk;XjC!kB3V4#x5p@;UY0I&l0DL|v2GPFx(vq$w2)Spv6-#&#)@SW5mASVmp6`dSG|){n=) zpI-ovo70HCjR==#k&oYQQue-O-B1i>^=dZYPp4xkTcpkApf;4TCasx*_5-5vgN_(m z{RBwIK;5|sE#QO?BU1fQ`$leOWG5!l&;rtTfji@GMkKle(vcA+@9ya{QoMF$NV!s| zi60g$l8%GWRPY)dxQB7HI(HUN#ZiE|ljf%aZF zP#FVcbATTfErO3|0X<@z^oad;{0E~TaIG2<70^z6d?B{#OsEqk~EUVAUB7Tmo(89Ac#x zq1#!@s}xp~F7R{ih!3%n${tLkfO zga;GL%E(QRPpb3g-JtT)i*O2J|2&9nvPp4PhHirvfrc!0?}WH|qW*{sZzZ<|NOd zNmFrHA4)!sGzf==RCX(fy#O0$RxraAU;i<{KYh-6|5VR24TwjO(DU zt$O)J*>Uv5%j9Cvn9~Q3b> z&4JSFij8KFsUo1KSW6Xa%{Bm2gwv|ENLeXC5P&t>47N| float: - # TODO Extend to handle any score key (instead of just overall score) as input - logger.info("Getting overall score") + def score(self, image, prompt: str, max_tokens: int = None, seed: int = 42) -> float: + logger.info(f"Getting {self.score_type} score") try: outputs = self.query_model(image, prompt, max_tokens, seed) - overall_score = outputs["overall_score"]["score"] + score = outputs[self.score_type]["score"] - if overall_score: - logger.debug(f"Overall score calculated: {overall_score}") - return float(overall_score) + if score is not None: + logger.debug(f"{self.score_type} score calculated: {score}") + return float(score) - logger.warning("Overall score not found in model output") + logger.warning(f"{self.score_type} score not found in model output") return 0.0 except Exception as e: - logger.error(f"Error getting overall score: {str(e)}", exc_info=True) + logger.error(f"Error getting {self.score_type} score: {str(e)}", exc_info=True) return 0.0 @@ -196,35 +193,15 @@ def get_overall_score(self, image, prompt: str, max_tokens: int = None, seed: in model_name = "Qwen/Qwen2.5-VL-7B-Instruct" - model = QwenVLMVerifier(model_name=model_name, device=device) - # model.load_model() + model = QwenVLMVerifier(model_name=model_name, device=device, score_type='visual_quality_and_realism') image_path = "596F6DF4-2856-436E-A981-649ABFB15F1B.jpeg" image = Image.open(image_path).convert("RGB") test_prompt = "A red bird and a fish." - response = model.query_model(image, test_prompt) - print("Model Response:", response) - - aspect_keys = [ - "accuracy_to_prompt", - "creativity_and_originality", - "visual_quality_and_realism", - "consistency_and_cohesion", - "emotional_or_thematic_resonance" - ] - - scores = [] - for key in aspect_keys: - if key in response and "score" in response[key]: - scores.append(response[key]["score"]) - - if scores: - average_score = sum(scores) / len(scores) - print("Average Score:", average_score) - else: - print("No scores found to average.") + visual_quality_score = model.score(image, test_prompt) + print(f"Visual quality score: {visual_quality_score}") # model.to_device('cpu') # model.to_device('cuda') @@ -232,5 +209,5 @@ def get_overall_score(self, image, prompt: str, max_tokens: int = None, seed: in response = model.query_model(image, test_prompt) print("Model Response:", response) - overall_score = model.get_overall_score(image, test_prompt) + overall_score = model.score(image, test_prompt) print(f"Overall score: {overall_score}") From 63d26c124c2d89332951e91b7416e0177e908a31 Mon Sep 17 00:00:00 2001 From: MaximClouser Date: Thu, 27 Feb 2025 20:30:53 +0000 Subject: [PATCH 2/3] expanding on algorithms and verifiers --- README.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 8948391..3769412 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,21 @@ A ComfyUI extension implementing "Inference-time scaling for diffusion models be ## How It Works -It uses random search and zero-order search to explore different noises and a verifier ensemble (ImageReward, CLIP, and a VLM grader using Qwen 2.5 7b/3b/72b) to generate the best image. By exploring the noise space, it finds a better starting seed and produces images of higher quality and better prompt alignment than simply increasing denoising steps, with the tradeoff being increased time and compute during inference. +This extension implements two different search algorithms to find the best possible image for your prompt: + +1. **Random Search**: The simplest approach - generates multiple images with different random noises and evaluates them to explore the noise space. + +2. **Zero-Order Search**: A more sophisticated approach that performs local optimization. It starts with a random noise, generates nearby variations by perturbing noise, and iteratively moves toward better results based on evaluation. + +To explore the noise space, the quality of generated images is evaluated using an ensemble of three verifiers: + +- **CLIP Score**: Measures how well the image matches the text prompt using OpenAI's CLIP model +- **ImageReward**: Evaluates image quality and prompt alignment using a specialized reward model +- **Qwen VLM**: Uses a large vision-language model to provide detailed scoring across multiple aspects (visual quality, creativity, prompt accuracy, etc.) + +By exploring the noise space and using these verifiers to guide the search, it can produce images of higher quality and better prompt alignment than simply increasing denoising steps, with the tradeoff being increased time and compute during inference. + +For more detailed information about the algorithms and methodology, please refer to the original paper from Google DeepMind: ["Inference-time scaling for diffusion models beyond scaling denoising steps"](https://arxiv.org/abs/2501.09732). ## Installation From 441dd50543f6da2968a7e80ba09d45655cf889b2 Mon Sep 17 00:00:00 2001 From: MaximClouser Date: Thu, 27 Feb 2025 20:36:58 +0000 Subject: [PATCH 3/3] version --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 3769412..070500f 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ # ComfyUI-InferenceTimeScaling +![Version](https://img.shields.io/badge/version-0.0.3-blue.svg) A ComfyUI extension implementing "Inference-time scaling for diffusion models beyond scaling denoising steps" ([Ma et al., 2025](https://arxiv.org/abs/2501.09732)). This extension provides inference-time optimization techniques to enhance diffusion-based image generation quality through random search and zero-order optimization algorithms, along with an ensemble verification system.