NetBSD Problem Report #58871

From www@netbsd.org  Tue Dec  3 19:50:03 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id BFA281A9238
	for <gnats-bugs@gnats.NetBSD.org>; Tue,  3 Dec 2024 19:50:03 +0000 (UTC)
Message-Id: <20241203195002.459691A9246@mollari.NetBSD.org>
Date: Tue,  3 Dec 2024 19:50:02 +0000 (UTC)
From: bsiegert@gmail.com
Reply-To: bsiegert@gmail.com
To: gnats-bugs@NetBSD.org
Subject: Stuck processes 
X-Send-Pr-Version: www-1.0

>Number:         58871
>Notify-List:    riastradh@NetBSD.org
>Category:       kern
>Synopsis:       Stuck processes
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          feedback
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Dec 03 19:55:00 +0000 2024
>Closed-Date:    
>Last-Modified:  Sat Jan 04 04:20:01 +0000 2025
>Originator:     Benny Siegert
>Release:        NetBSD 10_STABLE from 2024-11-17
>Organization:
The NetBSD Foundation
>Environment:
NetBSD rpi3.bentsukun.ch 10.0_STABLE NetBSD 10.0_STABLE (GENERIC) #0: Sun Nov 17 16:18:04 UTC 2024  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC evbarm
>Description:
I have two machines running evbarm-earmv7hf as Go CI builders. Both regularly get into a wedged state. I think this mostly happens during the golang.org/x/tools tests.

When this happens, you can still log into the machine with ssh and execute commands. "sudo reboot" however does nothing, otherwise the CI infra would get out of this state -- it tries rebooting when a test has problems.

Right now, the top output looks like this:

  PID USERNAME PRI NICE   SIZE   RES STATE       TIME   WCPU    CPU COMMAND
16334 swarming  95    0   601M  189M RUN/2       1:57 56.40% 56.40% completion.test
 1598 swarming  95    0   600M  178M RUN/2       0:30 44.74% 43.46% misc.test
16304 swarming  63    0   720M   86M RUN/2       2:31 35.25% 35.25% diagnostics.test
18154 swarming 127    0   593M   44M tstile/1    0:12 11.81% 11.47% modfile.test

These percentages have not changed in several minutes. None of these tasks can be killed, even with "kill -9".

This happens on two machines, a Raspberry Pi 3 and an OrangePi Plus 2E.
>How-To-Repeat:
git clone https://go.googlesource.com/tools go-tools
cd go-tools
go test ./... # may have to run multiple times
>Fix:
Any ideas on how to debug this further? If I attach gdb to one of these processes, it hangs.

>Release-Note:

>Audit-Trail:

From: Taylor R Campbell <riastradh@NetBSD.org>
To: Benny Siegert <bsiegert@gmail.com>
Cc: gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: kern/58871: Stuck processes
Date: Tue, 3 Dec 2024 20:03:51 +0000

 Can you start crash(8) and get output from `ps', `ps/w', and `show all
 tstiles'?

 Can you start crash(8) and stack traces from the processes not in RUN
 state, like the tstile one with `bt 0t18154'?

 If you can't start crash(8), can you enter ddb instead by sending
 break over serial, or by (perhaps on the next boot) setting sysctl -w
 hw.cnmagic=+++++ and then typing `+++++' on the console?

 Can you ktrace the processes in RUN state to see what syscalls they're
 running?

 Can you run dtrace to sample what's happening?

 dtrace -n 'profile:::profile-97 { @[stack()] = count() }'

State-Changed-From-To: open->feedback
State-Changed-By: maya@NetBSD.org
State-Changed-When: Tue, 10 Dec 2024 19:46:43 +0000
State-Changed-Why:
feedback requested


From: Benny Siegert <bsiegert@gmail.com>
To: Taylor R Campbell <riastradh@NetBSD.org>
Cc: gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: kern/58871: Stuck processes
Date: Tue, 10 Dec 2024 20:54:45 +0100

 This happened again, so I did some of the things you said at least:

 Am 03.12.24 um 21:03 schrieb Taylor R Campbell:
 > Can you start crash(8) and get output from `ps', `ps/w', and `show all
 > tstiles'?

 Crash version 10.0_STABLE, image version 10.0_STABLE.
 Kernel compiled without options LOCKDEBUG.
 Output from a running system is unreliable.
 crash> ps/w
 PID     LID          COMMAND     EMUL  PRI WAIT-MSG         WAIT-CHANNEL
 21931>21931            crash   netbsd   37                  0
 24382 24382               sh   netbsd   36 wait             9262b88c
 15343>15343           screen   netbsd   37                  0
 10543 10543           screen   netbsd   38 pause            92c32c80
 8677   8677               sh   netbsd   35 wait             957b854c
 3770   3770             find   netbsd   26 tstile           92b31f80
 28839 28839         postdrop   netbsd   43 netio            929e5150
 21098 21098         sendmail   netbsd   43 pipe_rd          92d0c814
 13750 13750              tee   netbsd   43 pipe_rd          91d21df4
 7775   7775               sh   netbsd   42 wait             95b6bacc
 24218 24218               sh   netbsd   43 wait             9170ea4c
 28302 28302             cron   netbsd   43 pipe_rd          92d0c8bc
 2374   2374               go   netbsd   40 tstile           95430f80
 8089 > 8089               go   netbsd   25                  0
 2534  17497    pipeline.test   netbsd   43 lwpwait          92a53a54
 2534  28139    pipeline.test   netbsd   43                  0
 24155 24155               sh   netbsd   43                  0
 24215 18770               go   netbsd   43 parked           95594c80
 24215  8412               go   netbsd   43 parked           92bdb3c0
 24215 24883               go   netbsd   43 parked           92c40a00
 24215  8759               go   netbsd   43 parked           9519e940
 24215 19943               go   netbsd   40 pipe_rd          9558aabc
 24215 23618               go   netbsd   43 wait             91ec930c
 24215  9006               go   netbsd   40 parked           92bdb680
 24215  5455               go   netbsd   41 wait             91ec930c
 24215 26737               go   netbsd   43 kqueue           95776eb8
 24215 13345               go   netbsd   43 parked           92d18d00
 24215 11347               go   netbsd   42 parked           9519d900
 24215  7163               go   netbsd   43 parked           92a678c0
 24215 24215               go   netbsd   38 parked           95531040
 4239  12336   result_adapter   netbsd   43 parked           955de1c0
 4239  26804   result_adapter   netbsd   43 kqueue           94980678
 4239   7242   result_adapter   netbsd   43 parked           92b5d900
 4239  10307   result_adapter   netbsd   42 parked           91f28780
 4239  10406   result_adapter   netbsd   43 parked           92b5d380
 4239  20061   result_adapter   netbsd   43 parked           91d153c0
 4239   4430   result_adapter   netbsd   43 parked           925377c0
 4239  19249   result_adapter   netbsd   43 wait             955d40cc
 4239   2513   result_adapter   netbsd   43 parked           951b36c0
 4239   4239   result_adapter   netbsd   42 parked           95538b80
 2274  27130              rdb   netbsd   43 parked           9557ec40
 2274  27888              rdb   netbsd   43                  0
 2274  18470              rdb   netbsd   43 parked           955949c0
 2274   2725              rdb   netbsd   43 parked           9557bc00
 2274  22316              rdb   netbsd   43 parked           95538340
 2274   9005              rdb   netbsd   43 wait             955d484c
 2274   8849              rdb   netbsd   43 parked           9519d0c0
 2274  22878              rdb   netbsd   43 parked           95461180
 2274  15929              rdb   netbsd   43                  0
 2274   2274              rdb   netbsd   43                  0
 19250 13439          bbagent   netbsd   43                  0
 19250  6648          bbagent   netbsd   43                  0
 19250 15191          bbagent   netbsd   43 lwpwait          95b6b0d4
 19250 10220          bbagent   netbsd   43                  0
 20545 20545       python3.11   netbsd   43 wait             957b804c
 3732   1996       python3.11   netbsd   43 lwpwait          9170e554
 3732   3732       python3.11   netbsd   43                  0
 3053   3053            getty   netbsd   39 ttyraw           915e0c28
 3187   3187            getty   netbsd   39 ttyraw           915e0a28
 3497   3497            getty   netbsd   39 ttyraw           915e0828
 2344   2344            login   netbsd   42 wait             9170e2cc
 662     662             cron   netbsd   43 nanoslp          929b1300
 658     658             estd   netbsd   43 nanoslp          92537a80

 648     648            inetd   netbsd   40 kqueue           9295dab8
 2892   2979    node_exporter   netbsd   41 parked           929b1b40
 2892    671    node_exporter   netbsd   43 parked           91f284c0
 2892    669    node_exporter   netbsd   43 parked           929b1880
 2892    668    node_exporter   netbsd   43 kqueue           92a09d38
 2892    664    node_exporter   netbsd   43 parked           929b15c0
 2892   2892    node_exporter   netbsd   43 parked           918bb300
 2853   2853             qmgr   netbsd   43 kqueue           9295d1b8
 2859   2859           master   netbsd   43                  0
 333   29950       python3.11   netbsd   43 parked           91808d00
 333     333       python3.11   netbsd   43 wait             91d1a2cc
 411     411             sshd   netbsd   43                  0
 394     394             ntpd   netbsd   43 pause            91f28200
 2358    328   bootstrapswarm   netbsd   43 parked           91f0f480
 2358    396   bootstrapswarm   netbsd   43 parked           91d15100
 2358    389   bootstrapswarm   netbsd   40 parked           91f0f740
 2358    385   bootstrapswarm   netbsd   42 parked           91f0f1c0
 2358    384   bootstrapswarm   netbsd   41 wait             91ec908c
 2358   1960   bootstrapswarm   netbsd   43                  0
 2358   2641   bootstrapswarm   netbsd   43 nanoslp          91ee4700

 2358   2358   bootstrapswarm   netbsd   43 parked           91ec16c0
 1998   1998         multilog   netbsd   43 pipe_rd          91d212cc
 2674   2674         multilog   netbsd   41 pipe_rd          91d21224
 1501   1501               sh   netbsd   40 wait             918ecacc
 2339   2339               sh   netbsd   41 wait             91d1aa4c
 2235   2235               sh   netbsd   41 wait             91d1a54c
 2100   2100        supervise   netbsd   43 poll             915d0280
 2128   2128        supervise   netbsd   43                  0
 2238   2238        supervise   netbsd   43 poll             915d0280

 1952   1952        supervise   netbsd   43 poll             913c5340
 2236   2236         multilog   netbsd   43 pipe_rd          918438ac
 2303   2303           svscan   netbsd   43 nanoslp          918bb5c0
 1884   1884          syslogd   netbsd   43 kqueue           91835f38
 2033   2033            mdnsd   netbsd   43 select           915c6cc0
 932     932           dhcpcd   netbsd   43 poll             915d0280
 990     990           dhcpcd   netbsd   36 poll             915c6f40
 879     879           dhcpcd   netbsd   43                  0
 991     991           dhcpcd   netbsd   43 poll             915c6cc0
 570     570          devpubd   netbsd   33 devmon           80aad06c
 1         1             init   netbsd   41 wait             9170e04c
 0       366           system   netbsd  123 physiod          9180fe04
 0       218           system   netbsd   43 bwfm0            9180f444

 0       217           system   netbsd   96 lnxcmplt         917c2a18
 0       216           system   netbsd  125 pooldrain        80abe900
 0       215           system   netbsd  124 syncer           91808200
 0       214           system   netbsd  126 pgdaemon         80abdcf0
 0       213           system   netbsd  123 data             917bdad4
 0       212           system   netbsd   96 semacv           80a86408
 0       211           system   netbsd   96 semacv           80a863f4
 0       210           system   netbsd   96 semacv           80a863e0
 0       207           system   netbsd   43 swwreboot        917dea44
 0       205           system   netbsd   96 sccomp           917c23dc
 0       203           system   netbsd   96 npfgcw           917bf144
 0       202           system   netbsd  222 rt_free          91709444
 0       201           system   netbsd   96 unpgc            80b274c8
 0       200           system   netbsd  222 key_timehandler  91709384

 0       199           system   netbsd  222 icmp6_wqinput    91707484
 0       198           system   netbsd  222 icmp6_wqinput    91707444
 0       197           system   netbsd  222 icmp6_wqinput    91707404
 0       196           system   netbsd  222 icmp6_wqinput    917073c4
 0       195           system   netbsd   96 usbevt           913cbb98
 0       194           system   netbsd  222 nd6_timer        916e5744
 0       193           system   netbsd  222 carp6_wqinput    913cad44
 0       192           system   netbsd  222 carp6_wqinput    913cad04
 0       170           system   netbsd  222 carp6_wqinput    913cacc4
 0       177           system   netbsd  222 carp6_wqinput    913cac84
 0       174           system   netbsd  222 carp_wqinput     913cabc4
 0       179           system   netbsd  222 carp_wqinput     913cab84
 0       176           system   netbsd  222 carp_wqinput     913cab44
 0        31           system   netbsd  222 carp_wqinput     913cab04

 0        63           system   netbsd  222 icmp_wqinput     913caa44
 0       126           system   netbsd  222 icmp_wqinput     913caa04
 0       125           system   netbsd  222 icmp_wqinput     913ca9c4
 0       124           system   netbsd  222 icmp_wqinput     913ca984
 0       123           system   netbsd  222 rt_timer         916e5684
 0       122           system   netbsd  125 vmem_rehash      915ebc84
 0       121           system   netbsd   43 vcmbox0          916e5444
 0       120           system   netbsd   96 usbtsk           80aacdac
 0       119           system   netbsd   96 usbtsk           80aacd8c
 0       118           system   netbsd   43 dwc2             916d58c4
 0       117           system   netbsd  221 mmctaskq         916e438c
 0       116           system   netbsd  221 mmctaskq         916e408c
 0       107           system   netbsd   43 xclocv           809e0c04
 0       105           system   netbsd  127 xcall            807c2448
 0       104           system   netbsd  223                  0
 0       103           system   netbsd  220                  0

 0       102           system   netbsd  221                  0
 0       101           system   netbsd  222                  0
 0    >  100           system   netbsd    0                  0
 0        99           system   netbsd  127                  0
 0        98           system   netbsd  223                  0
 0        97           system   netbsd  220                  0
 0        96           system   netbsd  221                  0
 0        30           system   netbsd  222                  0
 0        29           system   netbsd    0                  0
 0        28           system   netbsd  127 xcall            807c15c8
 0        27           system   netbsd  223                  0
 0        26           system   netbsd  220                  0
 0        25           system   netbsd  221                  0
 0        24           system   netbsd  222                  0
 0        23           system   netbsd    0                  0

 0        22           system   netbsd   43 lnxsyswq         913c5844
 0        21           system   netbsd   43 lnxubdwq         913c57c4
 0        20           system   netbsd   43 lnxpwrwq         913c5744
 0        19           system   netbsd   43 lnxlngwq         913c56c4
 0        18           system   netbsd   43 lnxhipwq         913c5644
 0        17           system   netbsd   43 lnxrcugc         809d7fc4
 0        16           system   netbsd   96 smtaskq          80aaf33c
 0        15           system   netbsd   43 pmfsuspend       913a2a44
 0        14           system   netbsd   43 pmfevent         913a2984
 0        13           system   netbsd   96 sopendfr         80b27484
 0        12           system   netbsd  222 ifwdog           913a28c4
 0        11           system   netbsd  222 iflnkst          913a2804
 0        10           system   netbsd   43 nfssilly         913a2744
 0         9           system   netbsd  125 vdrain           80b27f9c
 0         8           system   netbsd  125 mod_unld         80b1fd7c

 0         7           system   netbsd  127 xcall            807c0e88
 0         6           system   netbsd  223                  0
 0         5           system   netbsd  220                  0
 0         4           system   netbsd  221                  0
 0         3           system   netbsd  222                  0
 0         2           system   netbsd    0                  0
 0         0           system   netbsd  125 uvm              807fa9c0


 Is it normal that ps/w prints output continuously until you press Ctrl+C?

 > Can you start crash(8) and stack traces from the processes not in RUN
 > state, like the tstile one with `bt 0t18154'?

 I tried looking at the "find" process in tstile (3770) and the 
 "pipeline.test" process (2534) but got the following. Is this a bug in 
 crash(8)?

 According to htop, process 2534 was hogging 100% of one core. It looks 
 like it was actually spinning on the CPU?

 crash> bt/t 2534
 trace: pid 9524 not found
 crash> bt/t 3770
 trace: pid 14192 not found

 > Can you run dtrace to sample what's happening?
 > 
 > dtrace -n 'profile:::profile-97 { @[stack()] = count() }'

 Next time :) I had to enable dtrace in modules.conf.

 Sorry, this is probably not the most helpful answer. When this 
 invariably happens again, I will try the other debugging techniques.

 FWIW, typing "sync" made the whole machine hang, so it could also be 
 storage-related. The go processes are running off USB storage, and there 
 is also a swap partition on the USB storage. dmesg did not contain 
 anything relevant though.

 Would it be useful to run with a LOCKDEBUG kernel?

 -- 
 Benny

State-Changed-From-To: feedback->open
State-Changed-By: bsiegert@NetBSD.org
State-Changed-When: Tue, 10 Dec 2024 19:55:30 +0000
State-Changed-Why:
feedback provided


From: Taylor R Campbell <riastradh@NetBSD.org>
To: Benny Siegert <bsiegert@gmail.com>
Cc: gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: kern/58871: Stuck processes
Date: Tue, 10 Dec 2024 20:56:26 +0000

 > Date: Tue, 10 Dec 2024 20:54:45 +0100
 > From: Benny Siegert <bsiegert@gmail.com>
 > 
 > This happened again, so I did some of the things you said at least:
 > 
 > Am 03.12.24 um 21:03 schrieb Taylor R Campbell:
 > > Can you start crash(8) and get output from `ps', `ps/w', and `show all
 > > tstiles'?

 Would really like `ps' and `show all tstiles' too next time!

 > Is it normal that ps/w prints output continuously until you press Ctrl+C?

 Is it just taking a long time or it is going in a loop?  If there's a
 lot of processes/threads and the serial console is slow or the CPUs
 are busy running other threads it might just take a long time.

 > > Can you start crash(8) and stack traces from the processes not in RUN
 > > state, like the tstile one with `bt 0t18154'?

 Sorry, I meant `bt/t 0t18154'.

 > I tried looking at the "find" process in tstile (3770) and the 
 > "pipeline.test" process (2534) but got the following. Is this a bug in 
 > crash(8)?
 > 
 > According to htop, process 2534 was hogging 100% of one core. It looks 
 > like it was actually spinning on the CPU?
 > 
 > crash> bt/t 2534
 > trace: pid 9524 not found
 > crash> bt/t 3770
 > trace: pid 14192 not found

 You need to use `0t2534' for pid 2534 in decimal; otherwise the input
 is read in hexadecimal.  So these should have been:

 bt/t 0t2534
 bt/t 0t3770

 (Whether this is a bug in crash(8) is a matter of perspective...
 Certainly it is confusing that the input is interpreted as decimal and
 the output is presented in hexadecimal here!)

 > Sorry, this is probably not the most helpful answer. When this 
 > invariably happens again, I will try the other debugging techniques.

 OK, thanks!

 > Would it be useful to run with a LOCKDEBUG kernel?

 Sure, that could help, but it'll run muuuuuch slower and might paper
 over the symptoms by having less concurrency.

From: Benny Siegert <bsiegert@gmail.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: riastradh@NetBSD.org
Subject: Re: kern/58871: Stuck processes
Date: Sat, 14 Dec 2024 15:13:02 +0100

 This is a multi-part message in MIME format.
 --------------oZbu7N872voTY1zaznVMqxsk
 Content-Type: text/plain; charset=UTF-8; format=flowed
 Content-Transfer-Encoding: 7bit

 Here is another set of logs from a system in this state.

 What struck me is that like last time, there is a "find" process in 
 tstile state. I actually have a trace of where it is stuck, see below.

 I don't think the Go tests (which are the main thing running on these 
 boxes) run find at all, so maybe this is some cronjob?

 Am 03.12.24 um 21:05 schrieb Taylor R Campbell via gnats:
 >   Can you start crash(8) and get output from `ps', `ps/w', and `show all
 >   tstiles'?

 See below.

 >   Can you start crash(8) and stack traces from the processes not in RUN
 >   state, like the tstile one with `bt 0t18154'?

 Did that for the one process that was in tstile, the "find" one.

 >   Can you run dtrace to sample what's happening?
 >   
 >   dtrace -n 'profile:::profile-97 { @[stack()] = count() }'

 This made the system hang, for whatever reason.

 Hopefully, this is a tiny bit more useful.

 -- 
 Benny
 --------------oZbu7N872voTY1zaznVMqxsk
 Content-Type: text/plain; charset=UTF-8; name="bt-11160"
 Content-Disposition: attachment; filename="bt-11160"
 Content-Transfer-Encoding: base64

 Q3Jhc2ggdmVyc2lvbiAxMC4wX1NUQUJMRSwgaW1hZ2UgdmVyc2lvbiAxMC4wX1NUQUJMRS4K
 S2VybmVsIGNvbXBpbGVkIHdpdGhvdXQgb3B0aW9ucyBMT0NLREVCVUcuCk91dHB1dCBmcm9t
 IGEgcnVubmluZyBzeXN0ZW0gaXMgdW5yZWxpYWJsZS4KdHJhY2U6IHBpZCAxMTE2MCBsaWQg
 MTExNjAgYXQgMHhiY2NjZmIzYwoweGJjY2NmYjNjOiBtaV9zd2l0Y2grMHhjCjB4YmNjY2Zi
 NjQ6IHNsZWVwcV9ibG9jaysweGFjCjB4YmNjY2ZiZDQ6IHR1cm5zdGlsZV9ibG9jaysweDMy
 YwoweGJjY2NmYzQ0OiByd19lbnRlcisweDE0OAoweGJjY2NmYzVjOiBnZW5mc19sb2NrKzB4
 N2MKMHhiY2NjZmM5YzogVk9QX0xPQ0srMHg4NAoweGJjY2NmY2I0OiB2bl9sb2NrKzB4MTgK
 MHhiY2NjZmQ5YzogbmFtZWlfdHJ5ZW11bHJvb3QuY29uc3Rwcm9wLjArMHg2ZjAKMHhiY2Nj
 ZmRjYzogbmFtZWkrMHg1OAoweGJjY2NmZTM0OiBkb19zeXNfc3RhdGF0KzB4ZDAKMHhiY2Nj
 ZmVlYzogc3lzX19fbHN0YXQ1MCsweDJjCjB4YmNjY2ZmYWM6IHN5c2NhbGwrMHgxODgKLS0t
 IHRmIDB4YmNjY2ZmYjAgLS0tCg==
 --------------oZbu7N872voTY1zaznVMqxsk
 Content-Type: text/plain; charset=UTF-8; name="tstiles"
 Content-Disposition: attachment; filename="tstiles"
 Content-Transfer-Encoding: base64

 Q3Jhc2ggdmVyc2lvbiAxMC4wX1NUQUJMRSwgaW1hZ2UgdmVyc2lvbiAxMC4wX1NUQUJMRS4K
 S2VybmVsIGNvbXBpbGVkIHdpdGhvdXQgb3B0aW9ucyBMT0NLREVCVUcuCk91dHB1dCBmcm9t
 IGEgcnVubmluZyBzeXN0ZW0gaXMgdW5yZWxpYWJsZS4KICBQSUQgICBMSUQgICAgICAgICAg
 Q09NTUFORCAgICAgIFdBSVRJTkctRk9SICAgICBXQUlULUNIQU5ORUwKMTExNjAgMTExNjAg
 ICAgICAgICAgICAgZmluZCAgICAgICAgICAgICAgICAwICAgICAgICAgOTVlOWVhNDAK
 --------------oZbu7N872voTY1zaznVMqxsk
 Content-Type: text/plain; charset=UTF-8; name="ps-w"
 Content-Disposition: attachment; filename="ps-w"
 Content-Transfer-Encoding: base64

 Q3Jhc2ggdmVyc2lvbiAxMC4wX1NUQUJMRSwgaW1hZ2UgdmVyc2lvbiAxMC4wX1NUQUJMRS4K
 S2VybmVsIGNvbXBpbGVkIHdpdGhvdXQgb3B0aW9ucyBMT0NLREVCVUcuCk91dHB1dCBmcm9t
 IGEgcnVubmluZyBzeXN0ZW0gaXMgdW5yZWxpYWJsZS4KUElEICAgICBMSUQgICAgICAgICAg
 Q09NTUFORCAgICAgRU1VTCAgUFJJIFdBSVQtTVNHICAgICAgICAgV0FJVC1DSEFOTkVMCjQ0
 MyAgPiAgNDQzICAgICAgICAgICAgY3Jhc2ggICBuZXRic2QgICAyOCAgICAgICAgICAgICAg
 ICAgIDAKMTgxNTAgMTgxNTAgICAgICAgICAgICAgICBzaCAgIG5ldGJzZCAgIDI4IHdhaXQg
 ICAgICAgICAgICAgOTVjNDIyY2MKMTE4MDggMTE4MDggICAgICAgICAgIHBpY2t1cCAgIG5l
 dGJzZCAgIDQzIGtxdWV1ZSAgICAgICAgICAgOTQ1ZWFmNzgKMTExNjAgMTExNjAgICAgICAg
 ICAgICAgZmluZCAgIG5ldGJzZCAgIDI2IHRzdGlsZSAgICAgICAgICAgOTVlOWVhNDAKODQ2
 NiAgIDg0NjYgICAgICAgICBwb3N0ZHJvcCAgIG5ldGJzZCAgIDQzIG5ldGlvICAgICAgICAg
 ICAgOTI2NGNkZTgKMjIwNzcgMjIwNzcgICAgICAgICBzZW5kbWFpbCAgIG5ldGJzZCAgIDQz
 IHBpcGVfcmQgICAgICAgICAgOTUxZmI5NmMKNzI1NiAgIDcyNTYgICAgICAgICAgICAgIHRl
 ZSAgIG5ldGJzZCAgIDQzIHBpcGVfcmQgICAgICAgICAgOTUxZmI4MWMKMTgyNjMgMTgyNjMg
 ICAgICAgICAgICAgICBzaCAgIG5ldGJzZCAgIDQzIHdhaXQgICAgICAgICAgICAgOTE5NDFh
 Y2MKMjAyOTggMjAyOTggICAgICAgICAgICAgICBzaCAgIG5ldGJzZCAgIDQzIHdhaXQgICAg
 ICAgICAgICAgOTVjNDJhNGMKMTAzMTYgMTAzMTYgICAgICAgICAgICAgY3JvbiAgIG5ldGJz
 ZCAgIDQzIHBpcGVfcmQgICAgICAgICAgOTRmNDk4YWMKMjIzOCA+IDIyMzggICAgICAgICAg
 ICAgICBnbyAgIG5ldGJzZCAgIDI1ICAgICAgICAgICAgICAgICAgMAo4ODggICAgNDIwNSAg
 ICAgIHR5cGVzMi50ZXN0ICAgbmV0YnNkICAgNDMgICAgICAgICAgICAgICAgICAwCjg4OCAg
 ICAyMzEyICAgICAgdHlwZXMyLnRlc3QgICBuZXRic2QgICA0MyAgICAgICAgICAgICAgICAg
 IDAKODg4ICAgMTQ5MDUgICAgICB0eXBlczIudGVzdCAgIG5ldGJzZCAgIDQzIGx3cHdhaXQg
 ICAgICAgICAgOTI0M2ZiMTQKODg4ICAgICA4ODggICAgICB0eXBlczIudGVzdCAgIG5ldGJz
 ZCAgIDQzICAgICAgICAgICAgICAgICAgMAoxOTU5NyAyNzQ1MyAgICAgICAgICBjb21waWxl
 ICAgbmV0YnNkICAgNDMgbHdwd2FpdCAgICAgICAgICA5MWY2YmFkNAoxOTU5NyAyOTExMSAg
 ICAgICAgICBjb21waWxlICAgbmV0YnNkICAgNDMgICAgICAgICAgICAgICAgICAwCjE0MDg4
 ICA2NzIwICAgICAgICAgICAgICAgZ28gICBuZXRic2QgICA0MyAgICAgICAgICAgICAgICAg
 IDAKMTQwODggMjMyMTQgICAgICAgICAgICAgICBnbyAgIG5ldGJzZCAgIDQzICAgICAgICAg
 ICAgICAgICAgMAoxNDA4OCAgOTk0MSAgICAgICAgICAgICAgIGdvICAgbmV0YnNkICAgNDMg
 ICAgICAgICAgICAgICAgICAwCjE0MDg4ICA0MzgwICAgICAgICAgICAgICAgZ28gICBuZXRi
 c2QgICA0MyAgICAgICAgICAgICAgICAgIDAKMTkyMDUgICA1NjMgICAgICAgIHRlc3QudGVz
 dCAgIG5ldGJzZCAgIDQzICAgICAgICAgICAgICAgICAgMAoxOTIwNSAyNDEwMiAgICAgICAg
 dGVzdC50ZXN0ICAgbmV0YnNkICAgNDMgbHdwd2FpdCAgICAgICAgICA5NTBhODg1NAoyODQ5
 MyAxODkzMCAgICAgICAgICAgICAgIGdvICAgbmV0YnNkICAgNDMgICAgICAgICAgICAgICAg
 ICAwCjI4NDkzICAyNTc4ICAgICAgICAgICAgICAgZ28gICBuZXRic2QgICA0MSBwYXJrZWQg
 ICAgICAgICAgIDk1ZTk1NmMwCjI4NDkzIDI1MzYxICAgICAgICAgICAgICAgZ28gICBuZXRi
 c2QgICA0MiBwYXJrZWQgICAgICAgICAgIDk1MGE0M2MwCjI4NDkzIDEwODc1ICAgICAgICAg
 ICAgICAgZ28gICBuZXRic2QgICA0MSBwYXJrZWQgICAgICAgICAgIDk0OTU3NWMwCjI4NDkz
 ICA0NjUxICAgICAgICAgICAgICAgZ28gICBuZXRic2QgICA0MiBwYXJrZWQgICAgICAgICAg
 IDk1MzhiNGMwCjI4NDkzIDExNjM3ICAgICAgICAgICAgICAgZ28gICBuZXRic2QgICA0MyBw
 YXJrZWQgICAgICAgICAgIDk0OTU3YjQwCjI4NDkzICA2MTM3ICAgICAgICAgICAgICAgZ28g
 ICBuZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDk2NTEzNzAwCjI4NDkzIDE4ODI0ICAg
 ICAgICAgICAgICAgZ28gICBuZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDkxOTJlNjAw
 CjI4NDkzIDIzMDg5ICAgICAgICAgICAgICAgZ28gICBuZXRic2QgICA0MyBuYW5vc2xwICAg
 ICAgICAgIDkxZjZkYTAwCjI4NDkzIDI4NDkzICAgICAgICAgICAgICAgZ28gICBuZXRic2Qg
 ICA0MyAgICAgICAgICAgICAgICAgIDAKNzcwNSAgMjI4ODggICAgICAgICBzc2EudGVzdCAg
 IG5ldGJzZCAgIDQzIGx3cHdhaXQgICAgICAgICAgOTUwYThkNTQKNzcwNSAgMTU2MDggICAg
 ICAgICBzc2EudGVzdCAgIG5ldGJzZCAgIDQzICAgICAgICAgICAgICAgICAgMAo3NzA1ICAx
 ODc3MyAgICAgICAgIHNzYS50ZXN0ICAgbmV0YnNkICAgNDMgICAgICAgICAgICAgICAgICAw
 CjE3NzYxIDE3NzYxICAgICAgICAgICAgc2xlZXAgICBuZXRic2QgICA0MyAgICAgICAgICAg
 ICAgICAgIDAKMTQwOCAgIDE0MDggICAgICAgICAgICAgICBzaCAgIG5ldGJzZCAgIDQxIHdh
 aXQgICAgICAgICAgICAgOTcxMzZkOGMKOTc0NCAgMTU4OTYgICAgICAgICAgICAgICBnbyAg
 IG5ldGJzZCAgIDQzIHBhcmtlZCAgICAgICAgICAgOTY1MTNjODAKOTc0NCAgIDU5OTggICAg
 ICAgICAgICAgICBnbyAgIG5ldGJzZCAgIDM1IHdhaXQgICAgICAgICAgICAgOTcxNTEyY2MK
 OTc0NCAgIDE3NDggICAgICAgICAgICAgICBnbyAgIG5ldGJzZCAgIDQyIHBhcmtlZCAgICAg
 ICAgICAgOTUxNmM4ODAKOTc0NCAgMjI5NTYgICAgICAgICAgICAgICBnbyAgIG5ldGJzZCAg
 IDI3IHBhcmtlZCAgICAgICAgICAgOTUwYTQ2ODAKOTc0NCAgMTU3OTggICAgICAgICAgICAg
 ICBnbyAgIG5ldGJzZCAgIDQzIHBhcmtlZCAgICAgICAgICAgOTJjZDM3ODAKOTc0NCAgIDU4
 MjcgICAgICAgICAgICAgICBnbyAgIG5ldGJzZCAgIDQyIHBhcmtlZCAgICAgICAgICAgOTJj
 YzNhMDAKOTc0NCAgMjUxNTkgICAgICAgICAgICAgICBnbyAgIG5ldGJzZCAgIDQzIHBhcmtl
 ZCAgICAgICAgICAgOTUwYTQ5NDAKOTc0NCAgIDQ2MTEgICAgICAgICAgICAgICBnbyAgIG5l
 dGJzZCAgIDQzIHdhaXQgICAgICAgICAgICAgOTcxNTEyY2MKOTc0NCAgMTg5NTkgICAgICAg
 ICAgICAgICBnbyAgIG5ldGJzZCAgIDM5IHBhcmtlZCAgICAgICAgICAgOTUzOGI3ODAKOTc0
 NCAgMjYzMzkgICAgICAgICAgICAgICBnbyAgIG5ldGJzZCAgIDQzICAgICAgICAgICAgICAg
 ICAgMAo5NzQ0ICAyMTA4NyAgICAgICAgICAgICAgIGdvICAgbmV0YnNkICAgNDMgd2FpdCAg
 ICAgICAgICAgICA5NzE1MTJjYwo5NzQ0ICAyNjYxMCAgICAgICAgICAgICAgIGdvICAgbmV0
 YnNkICAgNDMgd2FpdCAgICAgICAgICAgICA5NzE1MTJjYwo5NzQ0ICAxNDI4NSAgICAgICAg
 ICAgICAgIGdvICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5NGUxODFjMAo5NzQ0
 ICAgOTc0NCAgICAgICAgICAgICAgIGdvICAgbmV0YnNkICAgNDEgcGFya2VkICAgICAgICAg
 ICA5MTgwZGE4MAoxMjExMyAgMjY2MiAgICAgICAgICAgICBkaXN0ICAgbmV0YnNkICAgNDMg
 cGFya2VkICAgICAgICAgICA5NTIzMDE4MAoxMjExMyAxNDE0OSAgICAgICAgICAgICBkaXN0
 ICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5MWZjYzUwMAoxMjExMyAxNjY1MCAg
 ICAgICAgICAgICBkaXN0ICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5NTFhZDk0
 MAoxMjExMyAyODE2OSAgICAgICAgICAgICBkaXN0ICAgbmV0YnNkICAgNDMgcGFya2VkICAg
 ICAgICAgICA5NTM4ZGE4MAoxMjExMyAgODc2NyAgICAgICAgICAgICBkaXN0ICAgbmV0YnNk
 ICAgNDMgcGFya2VkICAgICAgICAgICA5MmNkMzRjMAoxMjExMyAxMjExMyAgICAgICAgICAg
 ICBkaXN0ICAgbmV0YnNkICAgNDMgd2FpdCAgICAgICAgICAgICA5MWY2Yjg0Ywo3MDUxICAg
 NDE3MSAgICAgICAgICAgICAgIGdvICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5
 NWQ3NDY4MAo3MDUxICAgNjc5MyAgICAgICAgICAgICAgIGdvICAgbmV0YnNkICAgNDMgcGFy
 a2VkICAgICAgICAgICA5NWQ3NDk0MAo3MDUxICAxMTYwMSAgICAgICAgICAgICAgIGdvICAg
 bmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5NTM4YjIwMAo3MDUxICAgMjYwNCAgICAg
 ICAgICAgICAgIGdvICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5NTE2ODI0MAo3
 MDUxICAxMzAzOCAgICAgICAgICAgICAgIGdvICAgbmV0YnNkICAgNDMgd2FpdCAgICAgICAg
 ICAgICA5NzBlYzgwYwo3MDUxICAyOTAyMiAgICAgICAgICAgICAgIGdvICAgbmV0YnNkICAg
 NDMgICAgICAgICAgICAgICAgICAwCjcwNTEgICA3MDUxICAgICAgICAgICAgICAgZ28gICBu
 ZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDk1MGZiYTQwCjgwOTkgIDEwNTEwICAgcmVz
 dWx0X2FkYXB0ZXIgICBuZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDkyMjFiMDQwCjgw
 OTkgICA3NTkwICAgcmVzdWx0X2FkYXB0ZXIgICBuZXRic2QgICA0MyAgICAgICAgICAgICAg
 ICAgIDAKODA5OSAgMjI1MTUgICByZXN1bHRfYWRhcHRlciAgIG5ldGJzZCAgIDQzIHBhcmtl
 ZCAgICAgICAgICAgOTUxYWQ2ODAKODA5OSAgMTk0OTYgICByZXN1bHRfYWRhcHRlciAgIG5l
 dGJzZCAgIDM5IHBhcmtlZCAgICAgICAgICAgOTJjYzM3NDAKODA5OSAgIDc1NTUgICByZXN1
 bHRfYWRhcHRlciAgIG5ldGJzZCAgIDM5IHdhaXQgICAgICAgICAgICAgOTI0M2Y2MGMKODA5
 OSAgIDMxMzcgICByZXN1bHRfYWRhcHRlciAgIG5ldGJzZCAgIDQzIGtxdWV1ZSAgICAgICAg
 ICAgOTY5MWIzZjgKODA5OSAgMjkzMzMgICByZXN1bHRfYWRhcHRlciAgIG5ldGJzZCAgIDQw
 IHBhcmtlZCAgICAgICAgICAgOTUxNjg1MDAKODA5OSAgMjI5ODkgICByZXN1bHRfYWRhcHRl
 ciAgIG5ldGJzZCAgIDQzIHBhcmtlZCAgICAgICAgICAgOTVlOTU5ODAKODA5OSAgMTc4MjAg
 ICByZXN1bHRfYWRhcHRlciAgIG5ldGJzZCAgIDQzIG5hbm9zbHAgICAgICAgICAgOTUyMzBj
 ODAKODA5OSAgIDgwOTkgICByZXN1bHRfYWRhcHRlciAgIG5ldGJzZCAgIDM5IHBhcmtlZCAg
 ICAgICAgICAgOTUxYWQxMDAKMjE0NTcgIDE3NTMgICAgICAgICAgICAgIHJkYiAgIG5ldGJz
 ZCAgIDQzICAgICAgICAgICAgICAgICAgMAoyMTQ1NyAxNjMyNiAgICAgICAgICAgICAgcmRi
 ICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5MTZlMjA0MAoyMTQ1NyAgODU3NCAg
 ICAgICAgICAgICAgcmRiICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5MTZlMDdj
 MAoyMTQ1NyAyMzEwNyAgICAgICAgICAgICAgcmRiICAgbmV0YnNkICAgNDAgcGFya2VkICAg
 ICAgICAgICA5NTM4ZDI0MAoyMTQ1NyAgNzQxMyAgICAgICAgICAgICAgcmRiICAgbmV0YnNk
 ICAgNDAgcGFya2VkICAgICAgICAgICA5NjU2ZGM4MAoyMTQ1NyAgODkwMCAgICAgICAgICAg
 ICAgcmRiICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5NTM3MGEwMAoyMTQ1NyAy
 MjMzNSAgICAgICAgICAgICAgcmRiICAgbmV0YnNkICAgNDMgICAgICAgICAgICAgICAgICAw
 CjIxNDU3IDI1ODI1ICAgICAgICAgICAgICByZGIgICBuZXRic2QgICA0MCB3YWl0ICAgICAg
 ICAgICAgIDkxNzBkY2NjCjIxNDU3IDE0MjM3ICAgICAgICAgICAgICByZGIgICBuZXRic2Qg
 ICA0MyBuYW5vc2xwICAgICAgICAgIDk1MTY4ZDQwCjIxNDU3IDIxNDU3ICAgICAgICAgICAg
 ICByZGIgICBuZXRic2QgICA0MCBwYXJrZWQgICAgICAgICAgIDkxNmUyODgwCjE3MDYgICA3
 OTU2ICAgICAgZ29sYW5nYnVpbGQgICBuZXRic2QgICA0MyB3YWl0ICAgICAgICAgICAgIDky
 NDNmODhjCjE3MDYgIDE0NTk2ICAgICAgZ29sYW5nYnVpbGQgICBuZXRic2QgICA0MyBwYXJr
 ZWQgICAgICAgICAgIDk0OTUyNjAwCjE3MDYgIDI1MTA4ICAgICAgZ29sYW5nYnVpbGQgICBu
 ZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDk1MWFkM2MwCjE3MDYgIDE0Mzc1ICAgICAg
 Z29sYW5nYnVpbGQgICBuZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDk0OTUyOGMwCjE3
 MDYgICA3ODQwICAgICAgZ29sYW5nYnVpbGQgICBuZXRic2QgICA0MyBwYXJrZWQgICAgICAg
 ICAgIDkyYWUxOTQwCjE3MDYgIDI4MTAxICAgICAgZ29sYW5nYnVpbGQgICBuZXRic2QgICA0
 MyAgICAgICAgICAgICAgICAgIDAKMTcwNiAgIDE3MDYgICAgICBnb2xhbmdidWlsZCAgIG5l
 dGJzZCAgIDQzIGtxdWV1ZSAgICAgICAgICAgOTVkMjk1ZjgKMTM0NzMgMjc0ODMgICAgICAg
 ICAgYmJhZ2VudCAgIG5ldGJzZCAgIDQzIGx3cHdhaXQgICAgICAgICAgOTcwZWMzMTQKMTM0
 NzMgMjUyMjQgICAgICAgICAgYmJhZ2VudCAgIG5ldGJzZCAgIDQzICAgICAgICAgICAgICAg
 ICAgMAoxMzQ3MyAxMjI3OSAgICAgICAgICBiYmFnZW50ICAgbmV0YnNkICAgNDMgICAgICAg
 ICAgICAgICAgICAwCjI4MjQyICA4MzcwICAgICAgIHB5dGhvbjMuMTEgICBuZXRic2QgICA0
 MyAgICAgICAgICAgICAgICAgIDAKMjgyNDIgMjM2MjAgICAgICAgcHl0aG9uMy4xMSAgIG5l
 dGJzZCAgIDQzICAgICAgICAgICAgICAgICAgMAoyODI0MiAyODI0MiAgICAgICBweXRob24z
 LjExICAgbmV0YnNkICAgNDMgbHdwd2FpdCAgICAgICAgICA5MTcwZGE1NAozMzEzICAyNTM0
 MiAgICAgICBweXRob24zLjExICAgbmV0YnNkICAgNDMgcGFya2VkICAgICAgICAgICA5MmE2
 Y2JjMAozMzEzICAgMzMxMyAgICAgICBweXRob24zLjExICAgbmV0YnNkICAgNDMgd2FpdCAg
 ICAgICAgICAgICA5MWM2ZGQ4YwozMzE1ICAgMzMxNSAgICAgICAgICAgIGdldHR5ICAgbmV0
 YnNkICAgMzggdHR5cmF3ICAgICAgICAgICA5MTVlMGMyOAo3NTYgICAgIDc1NiAgICAgICAg
 ICAgIGdldHR5ICAgbmV0YnNkICAgMzggdHR5cmF3ICAgICAgICAgICA5MTVlMGEyOAo3NTcg
 ICAgIDc1NyAgICAgICAgICAgIGdldHR5ICAgbmV0YnNkICAgMzggdHR5cmF3ICAgICAgICAg
 ICA5MTVlMDgyOAoyNzIzICAgMjcyMyAgICAgICAgICAgIGxvZ2luICAgbmV0YnNkICAgNDIg
 d2FpdCAgICAgICAgICAgICA5MTcwZDJjYwo3NDMgICAgIDc0MyAgICAgICAgICAgICBjcm9u
 ICAgbmV0YnNkICAgNDMgbmFub3NscCAgICAgICAgICA5Mjg4MTM0MAoyOTA5ICAgMjkwOSAg
 ICAgICAgICAgICBlc3RkICAgbmV0YnNkICAgNDMgICAgICAgICAgICAgICAgICAwCjU5OCAg
 ICAgNTk4ICAgICAgICAgICAgaW5ldGQgICBuZXRic2QgICA0MSBrcXVldWUgICAgICAgICAg
 IDkxZjdhNGY4CjU4NyAgICAgNzAzICAgIG5vZGVfZXhwb3J0ZXIgICBuZXRic2QgICA0MSBw
 YXJrZWQgICAgICAgICAgIDkxOGMxNWMwCjU4NyAgICAyNjY4ICAgIG5vZGVfZXhwb3J0ZXIg
 ICBuZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDkyODgxYjgwCjU4NyAgICAyOTI1ICAg
 IG5vZGVfZXhwb3J0ZXIgICBuZXRic2QgICA0MyBrcXVldWUgICAgICAgICAgIDkyNWRjOGY4
 CjU4NyAgICAxMDA2ICAgIG5vZGVfZXhwb3J0ZXIgICBuZXRic2QgICA0MyBwYXJrZWQgICAg
 ICAgICAgIDkxYTJlOTAwCjU4NyAgICAgNzQ3ICAgIG5vZGVfZXhwb3J0ZXIgICBuZXRic2Qg
 ICA0MyAgICAgICAgICAgICAgICAgIDAKNTg3ICAgICA1ODcgICAgbm9kZV9leHBvcnRlciAg
 IG5ldGJzZCAgIDQzIHBhcmtlZCAgICAgICAgICAgOTI4ODE4YzAKNTg1ICAgICA1ODUgICAg
 ICAgICAgICAgcW1nciAgIG5ldGJzZCAgIDQzICAgICAgICAgICAgICAgICAgMAo1ODIgICAg
 IDU4MiAgICAgICAgICAgbWFzdGVyICAgbmV0YnNkICAgNDMga3F1ZXVlICAgICAgICAgICA5
 MWUxMWFiOAo2NjMgICAgIDY2OSAgIGJvb3RzdHJhcHN3YXJtICAgbmV0YnNkICAgNDIgd2Fp
 dCAgICAgICAgICAgICA5MWY5OGIwYwo2NjMgICAgIDY2OCAgIGJvb3RzdHJhcHN3YXJtICAg
 bmV0YnNkICAgNDEgcGFya2VkICAgICAgICAgICA5MWY2MzcwMAo2NjMgICAgIDY2NyAgIGJv
 b3RzdHJhcHN3YXJtICAgbmV0YnNkICAgNDMgICAgICAgICAgICAgICAgICAwCjY2MyAgICAg
 NjY2ICAgYm9vdHN0cmFwc3dhcm0gICBuZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDkx
 ZTE4NmMwCjY2MyAgICAgNjY1ICAgYm9vdHN0cmFwc3dhcm0gICBuZXRic2QgICA0MyBwYXJr
 ZWQgICAgICAgICAgIDkxZjhkNzgwCjY2MyAgICAgNjY0ICAgYm9vdHN0cmFwc3dhcm0gICBu
 ZXRic2QgICA0MyBuYW5vc2xwICAgICAgICAgIDkyMjFiMzAwCjY2MyAgICAgNjYzICAgYm9v
 dHN0cmFwc3dhcm0gICBuZXRic2QgICA0MyBwYXJrZWQgICAgICAgICAgIDkyMjFiYjQwCjY0
 MyAgICAgNjQzICAgICAgICAgICAgICAgc2ggICBuZXRic2QgICA0MSB3YWl0ICAgICAgICAg
 ICAgIDkxZjZiNWNjCjMzNSAgICAgMzM1ICAgICAgICAgICAgIHNzaGQgICBuZXRic2QgICA0
 MyAgICAgICAgICAgICAgICAgIDAKMTg1NCAgIDE4NTQgICAgICAgICAgICAgbnRwZCAgIG5l
 dGJzZCAgIDQzICAgICAgICAgICAgICAgICAgMAozOTkgICAgIDM5OSAgICAgICAgIG11bHRp
 bG9nICAgbmV0YnNkICAgNDMgcGlwZV9yZCAgICAgICAgICA5MWUyYTE3YwozOTggICAgIDM5
 OCAgICAgICAgIG11bHRpbG9nICAgbmV0YnNkICAgNDEgcGlwZV9yZCAgICAgICAgICA5MWUy
 YTAyYwoxOTk4ICAgMTk5OCAgICAgICAgICAgICAgIHNoICAgbmV0YnNkICAgNDEgd2FpdCAg
 ICAgICAgICAgICA5MTk0MTVjYwoyMzQzICAgMjM0MyAgICAgICAgICAgICAgIHNoICAgbmV0
 YnNkICAgNDEgd2FpdCAgICAgICAgICAgICA5MWM2ZDg4YwoyMzk2ICAgMjM5NiAgICAgICAg
 c3VwZXJ2aXNlICAgbmV0YnNkICAgNDMgcG9sbCAgICAgICAgICAgICA5MTVjNmY0MAoyMjMx
 ICAgMjIzMSAgICAgICAgc3VwZXJ2aXNlICAgbmV0YnNkICAgNDMgcG9sbCAgICAgICAgICAg
 ICA5MTVjNmY0MAoxOTc3ICAgMTk3NyAgICAgICAgc3VwZXJ2aXNlICAgbmV0YnNkICAgNDMg
 ICAgICAgICAgICAgICAgICAwCjE2NTAgICAxNjUwICAgICAgICBzdXBlcnZpc2UgICBuZXRi
 c2QgICA0MyBwb2xsICAgICAgICAgICAgIDkxM2M1MzQwCjIzNDUgICAyMzQ1ICAgICAgICAg
 bXVsdGlsb2cgICBuZXRic2QgICA0MyBwaXBlX3JkICAgICAgICAgIDkxODQzOTU0CjIyNTIg
 ICAyMjUyICAgICAgICAgICBzdnNjYW4gICBuZXRic2QgICA0MyAgICAgICAgICAgICAgICAg
 IDAKMjEwNyAgIDIxMDcgICAgICAgICAgc3lzbG9nZCAgIG5ldGJzZCAgIDQzICAgICAgICAg
 ICAgICAgICAgMAoyMTQyICAgMjE0MiAgICAgICAgICAgIG1kbnNkICAgbmV0YnNkICAgNDMg
 c2VsZWN0ICAgICAgICAgICA5MTNjNTM0MAo5NTIgICAgIDk1MiAgICAgICAgICAgZGhjcGNk
 ICAgbmV0YnNkICAgNDMgcG9sbCAgICAgICAgICAgICA5MTNjNTM0MAo5NTAgICAgIDk1MCAg
 ICAgICAgICAgZGhjcGNkICAgbmV0YnNkICAgNDMgICAgICAgICAgICAgICAgICAwCjk0OCAg
 ICAgOTQ4ICAgICAgICAgICBkaGNwY2QgICBuZXRic2QgICA0MyBwb2xsICAgICAgICAgICAg
 IDkxNWQwMjgwCjExMzAgICAxMTMwICAgICAgICAgICBkaGNwY2QgICBuZXRic2QgICA0MyAg
 ICAgICAgICAgICAgICAgIDAKODgwICAgICA4ODAgICAgICAgICAgZGV2cHViZCAgIG5ldGJz
 ZCAgIDM2IGRldm1vbiAgICAgICAgICAgODBhYWQwNmMKMSAgICAgICAgIDEgICAgICAgICAg
 ICAgaW5pdCAgIG5ldGJzZCAgIDQzIHdhaXQgICAgICAgICAgICAgOTE3MGQwNGMKMCAgICAg
 ICAzMDIgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMTI3IHBvb2xkaXNwICAgICAgICAg
 OTE4ODk2NTQKMCAgICAgICAzMDcgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDY0IHBv
 b2xkaXNwICAgICAgICAgOTE3ZGFhZDQKMCAgICAgICAzNjYgICAgICAgICAgIHN5c3RlbSAg
 IG5ldGJzZCAgMTIzIHBoeXNpb2QgICAgICAgICAgOTE4M2UwODQKMCAgICAgICAyMTggICAg
 ICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDQzIGJ3Zm0wICAgICAgICAgICAgOTE4MGY0NDQK
 MCAgICAgICAyMTcgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDk2IGxueGNtcGx0ICAg
 ICAgICAgOTE3YmVhMTgKMCAgICAgICAyMTYgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAg
 MTI1IHhjbG9jdiAgICAgICAgICAgODA5ZTBjMDQKMCAgICAgICAyMTUgICAgICAgICAgIHN5
 c3RlbSAgIG5ldGJzZCAgMTI0IHN5bmNlciAgICAgICAgICAgOTE3ZjRhNDAKMCAgICAgICAy
 MTQgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMTI2IHBnZGFlbW9uICAgICAgICAgODBh
 YmRjZjAKMCAgICAgICAyMTMgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMTIzIGRhdGEg
 ICAgICAgICAgICAgOTE3YmJhZDQKMCAgICAgICAyMTIgICAgICAgICAgIHN5c3RlbSAgIG5l
 dGJzZCAgIDk2IHNlbWFjdiAgICAgICAgICAgODBhODY0MDgKMCAgICAgICAyMTEgICAgICAg
 ICAgIHN5c3RlbSAgIG5ldGJzZCAgIDk2IHNlbWFjdiAgICAgICAgICAgODBhODYzZjQKMCAg
 ICAgICAyMTAgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDk2IHNlbWFjdiAgICAgICAg
 ICAgODBhODYzZTAKMCAgICAgICAyMDcgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDQz
 IHN3d3JlYm9vdCAgICAgICAgOTE3ZGVhNDQKMCAgICAgICAyMDUgICAgICAgICAgIHN5c3Rl
 bSAgIG5ldGJzZCAgIDk2IHNjY29tcCAgICAgICAgICAgOTE3YmU0ZGMKMCAgICAgICAyMDMg
 ICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDk2IG5wZmdjdyAgICAgICAgICAgOTE3YmQx
 ODQKMCAgICAgICAyMDIgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIHJ0X2ZyZWUg
 ICAgICAgICAgOTE3MDQzODQKMCAgICAgICAyMDEgICAgICAgICAgIHN5c3RlbSAgIG5ldGJz
 ZCAgIDk2IHVucGdjICAgICAgICAgICAgODBiMjc0YzgKMCAgICAgICAyMDAgICAgICAgICAg
 IHN5c3RlbSAgIG5ldGJzZCAgMjIyIGtleV90aW1laGFuZGxlciAgOTE3MDQyYzQKMCAgICAg
 ICAxOTkgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGljbXA2X3dxaW5wdXQgICAg
 OTE2ZmU0ODQKMCAgICAgICAxOTggICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGlj
 bXA2X3dxaW5wdXQgICAgOTE2ZmU0NDQKMCAgICAgICAxOTcgICAgICAgICAgIHN5c3RlbSAg
 IG5ldGJzZCAgMjIyIGljbXA2X3dxaW5wdXQgICAgOTE2ZmU0MDQKMCAgICAgICAxOTYgICAg
 ICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGljbXA2X3dxaW5wdXQgICAgOTE2ZmUzYzQK
 MCAgICAgICAxOTUgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDk2IHVzYmV2dCAgICAg
 ICAgICAgOTEzY2JiOTgKMCAgICAgICAxOTQgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAg
 MjIyIG5kNl90aW1lciAgICAgICAgOTE3MDQxNDQKMCAgICAgICAxOTMgICAgICAgICAgIHN5
 c3RlbSAgIG5ldGJzZCAgMjIyIGNhcnA2X3dxaW5wdXQgICAgOTE2ZmUzMDQKMCAgICAgICAx
 OTIgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGNhcnA2X3dxaW5wdXQgICAgOTE2
 ZmUyYzQKMCAgICAgICAxNzYgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGNhcnA2
 X3dxaW5wdXQgICAgOTE2ZmUyODQKMCAgICAgICAxNzUgICAgICAgICAgIHN5c3RlbSAgIG5l
 dGJzZCAgMjIyIGNhcnA2X3dxaW5wdXQgICAgOTE2ZmUyNDQKMCAgICAgICAxNzcgICAgICAg
 ICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGNhcnBfd3FpbnB1dCAgICAgOTE2ZmUxODQKMCAg
 ICAgICAxNzAgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGNhcnBfd3FpbnB1dCAg
 ICAgOTE2ZmUxNDQKMCAgICAgICAxNzMgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIy
 IGNhcnBfd3FpbnB1dCAgICAgOTE2ZmUxMDQKMCAgICAgICAgMzEgICAgICAgICAgIHN5c3Rl
 bSAgIG5ldGJzZCAgMjIyIGNhcnBfd3FpbnB1dCAgICAgOTE2ZmUwYzQKMCAgICAgICAgNjMg
 ICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGljbXBfd3FpbnB1dCAgICAgOTEzY2Fh
 NDQKMCAgICAgICAxMjYgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGljbXBfd3Fp
 bnB1dCAgICAgOTEzY2FhMDQKMCAgICAgICAxMjUgICAgICAgICAgIHN5c3RlbSAgIG5ldGJz
 ZCAgMjIyIGljbXBfd3FpbnB1dCAgICAgOTEzY2E5YzQKMCAgICAgICAxMjQgICAgICAgICAg
 IHN5c3RlbSAgIG5ldGJzZCAgMjIyIGljbXBfd3FpbnB1dCAgICAgOTEzY2E5ODQKMCAgICAg
 ICAxMjMgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIHJ0X3RpbWVyICAgICAgICAg
 OTE2ZTU1YzQKMCAgICAgICAxMjIgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMTI1ICAg
 ICAgICAgICAgICAgICAgMAowICAgICAgIDEyMSAgICAgICAgICAgc3lzdGVtICAgbmV0YnNk
 ICAgNDMgdmNtYm94MCAgICAgICAgICA5MTZlNTQ0NAowICAgICAgIDEyMCAgICAgICAgICAg
 c3lzdGVtICAgbmV0YnNkICAgOTYgdXNidHNrICAgICAgICAgICA4MGFhY2RhYwowICAgICAg
 IDExOSAgICAgICAgICAgc3lzdGVtICAgbmV0YnNkICAgOTYgdXNidHNrICAgICAgICAgICA4
 MGFhY2Q4YwowICAgICAgIDExOCAgICAgICAgICAgc3lzdGVtICAgbmV0YnNkICAgNDMgZHdj
 MiAgICAgICAgICAgICA5MTZkNThjNAowICAgICAgIDExNyAgICAgICAgICAgc3lzdGVtICAg
 bmV0YnNkICAyMjEgbW1jdGFza3EgICAgICAgICA5MTZlNDM4YwowICAgICAgIDExNiAgICAg
 ICAgICAgc3lzdGVtICAgbmV0YnNkICAyMjEgbW1jdGFza3EgICAgICAgICA5MTZlNDA4Ywow
 ICAgICAgIDEwNyAgICAgICAgICAgc3lzdGVtICAgbmV0YnNkICAgNDMgICAgICAgICAgICAg
 ICAgICAwCjAgICAgICAgMTA1ICAgICAgICAgICBzeXN0ZW0gICBuZXRic2QgIDEyNyB4Y2Fs
 bCAgICAgICAgICAgIDgwN2MyNDQ4CjAgICAgICAgMTA0ICAgICAgICAgICBzeXN0ZW0gICBu
 ZXRic2QgIDIyMyAgICAgICAgICAgICAgICAgIDAKMCAgICAgICAxMDMgICAgICAgICAgIHN5
 c3RlbSAgIG5ldGJzZCAgMjIwICAgICAgICAgICAgICAgICAgMAowICAgICAgIDEwMiAgICAg
 ICAgICAgc3lzdGVtICAgbmV0YnNkICAyMjEgICAgICAgICAgICAgICAgICAwCjAgICAgICAg
 MTAxICAgICAgICAgICBzeXN0ZW0gICBuZXRic2QgIDIyMiAgICAgICAgICAgICAgICAgIDAK
 MCAgICAgICAxMDAgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgICAwICAgICAgICAgICAg
 ICAgICAgMAowICAgICAgICA5OSAgICAgICAgICAgc3lzdGVtICAgbmV0YnNkICAxMjcgeGNh
 bGwgICAgICAgICAgICA4MDdjMWQwOAowICAgICAgICA5OCAgICAgICAgICAgc3lzdGVtICAg
 bmV0YnNkICAyMjMgICAgICAgICAgICAgICAgICAwCjAgICAgICAgIDk3ICAgICAgICAgICBz
 eXN0ZW0gICBuZXRic2QgIDIyMCAgICAgICAgICAgICAgICAgIDAKMCAgICAgICAgOTYgICAg
 ICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIxICAgICAgICAgICAgICAgICAgMAowICAgICAg
 ICAzMCAgICAgICAgICAgc3lzdGVtICAgbmV0YnNkICAyMjIgICAgICAgICAgICAgICAgICAw
 CjAgICAgPiAgIDI5ICAgICAgICAgICBzeXN0ZW0gICBuZXRic2QgICAgMCAgICAgICAgICAg
 ICAgICAgIDAKMCAgICAgICAgMjggICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMTI3ICAg
 ICAgICAgICAgICAgICAgMAowICAgICAgICAyNyAgICAgICAgICAgc3lzdGVtICAgbmV0YnNk
 ICAyMjMgICAgICAgICAgICAgICAgICAwCjAgICAgICAgIDI2ICAgICAgICAgICBzeXN0ZW0g
 ICBuZXRic2QgIDIyMCAgICAgICAgICAgICAgICAgIDAKMCAgICAgICAgMjUgICAgICAgICAg
 IHN5c3RlbSAgIG5ldGJzZCAgMjIxICAgICAgICAgICAgICAgICAgMAowICAgICAgICAyNCAg
 ICAgICAgICAgc3lzdGVtICAgbmV0YnNkICAyMjIgICAgICAgICAgICAgICAgICAwCjAgICAg
 ICAgIDIzICAgICAgICAgICBzeXN0ZW0gICBuZXRic2QgICAgMCAgICAgICAgICAgICAgICAg
 IDAKMCAgICAgICAgMjIgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDQzIGxueHN5c3dx
 ICAgICAgICAgOTEzYzU4NDQKMCAgICAgICAgMjEgICAgICAgICAgIHN5c3RlbSAgIG5ldGJz
 ZCAgIDQzIGxueHViZHdxICAgICAgICAgOTEzYzU3YzQKMCAgICAgICAgMjAgICAgICAgICAg
 IHN5c3RlbSAgIG5ldGJzZCAgIDQzIGxueHB3cndxICAgICAgICAgOTEzYzU3NDQKMCAgICAg
 ICAgMTkgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDQzIGxueGxuZ3dxICAgICAgICAg
 OTEzYzU2YzQKMCAgICAgICAgMTggICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDQzIGxu
 eGhpcHdxICAgICAgICAgOTEzYzU2NDQKMCAgICAgICAgMTcgICAgICAgICAgIHN5c3RlbSAg
 IG5ldGJzZCAgIDQzIGxueHJjdWdjICAgICAgICAgODA5ZDdmYzQKMCAgICAgICAgMTYgICAg
 ICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDk2IHNtdGFza3EgICAgICAgICAgODBhYWYzM2MK
 MCAgICAgICAgMTUgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgIDQzIHBtZnN1c3BlbmQg
 ICAgICAgOTEzYTJhNDQKMCAgICAgICAgMTQgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAg
 IDQzIHBtZmV2ZW50ICAgICAgICAgOTEzYTI5ODQKMCAgICAgICAgMTMgICAgICAgICAgIHN5
 c3RlbSAgIG5ldGJzZCAgIDk2IHNvcGVuZGZyICAgICAgICAgODBiMjc0ODQKMCAgICAgICAg
 MTIgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGlmd2RvZyAgICAgICAgICAgOTEz
 YTI4YzQKMCAgICAgICAgMTEgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyIGlmbG5r
 c3QgICAgICAgICAgOTEzYTI4MDQKMCAgICAgICAgMTAgICAgICAgICAgIHN5c3RlbSAgIG5l
 dGJzZCAgIDQzIG5mc3NpbGx5ICAgICAgICAgOTEzYTI3NDQKMCAgICAgICAgIDkgICAgICAg
 ICAgIHN5c3RlbSAgIG5ldGJzZCAgMTI1IHZkcmFpbiAgICAgICAgICAgODBiMjdmOWMKMCAg
 ICAgICAgIDggICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMTI1IG1vZF91bmxkICAgICAg
 ICAgODBiMWZkN2MKMCAgICAgICAgIDcgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMTI3
 IHhjYWxsICAgICAgICAgICAgODA3YzBlODgKMCAgICAgICAgIDYgICAgICAgICAgIHN5c3Rl
 bSAgIG5ldGJzZCAgMjIzICAgICAgICAgICAgICAgICAgMAowICAgICAgICAgNSAgICAgICAg
 ICAgc3lzdGVtICAgbmV0YnNkICAyMjAgICAgICAgICAgICAgICAgICAwCjAgICAgICAgICA0
 ICAgICAgICAgICBzeXN0ZW0gICBuZXRic2QgIDIyMSAgICAgICAgICAgICAgICAgIDAKMCAg
 ICAgICAgIDMgICAgICAgICAgIHN5c3RlbSAgIG5ldGJzZCAgMjIyICAgICAgICAgICAgICAg
 ICAgMAowICAgID4gICAgMiAgICAgICAgICAgc3lzdGVtICAgbmV0YnNkICAgIDAgICAgICAg
 ICAgICAgICAgICAwCjAgICAgICAgICAwICAgICAgICAgICBzeXN0ZW0gICBuZXRic2QgIDEy
 NSB1dm0gICAgICAgICAgICAgIDgwN2ZhOWMwCg==
 --------------oZbu7N872voTY1zaznVMqxsk
 Content-Type: text/plain; charset=UTF-8; name="ps"
 Content-Disposition: attachment; filename="ps"
 Content-Transfer-Encoding: base64

 Q3Jhc2ggdmVyc2lvbiAxMC4wX1NUQUJMRSwgaW1hZ2UgdmVyc2lvbiAxMC4wX1NUQUJMRS4K
 S2VybmVsIGNvbXBpbGVkIHdpdGhvdXQgb3B0aW9ucyBMT0NLREVCVUcuCk91dHB1dCBmcm9t
 IGEgcnVubmluZyBzeXN0ZW0gaXMgdW5yZWxpYWJsZS4KUElEICAgICBMSUQgUyBDUFUgICAg
 IEZMQUdTICAgICAgIFNUUlVDVCBMV1AgKiAgICAgICAgICAgICAgIE5BTUUgV0FJVAoyMTg5
 ND4yMTg5NCA3ICAgMCAgICAgICAgIDAgICAgICAgICAgIDkyYzk4NDQwICAgICAgICAgICAg
 ICBjcmFzaAoxODE1MCAxODE1MCAzICAgMCAgICAgICAxODAgICAgICAgICAgIDk1MTY4N2Mw
 ICAgICAgICAgICAgICAgICBzaCB3YWl0CjExODA4IDExODA4IDMgICAyICAgICAgIDE4MCAg
 ICAgICAgICAgOTUyMThjNDAgICAgICAgICAgICAgcGlja3VwIGtxdWV1ZQoxMTE2MCAxMTE2
 MCAzICAgMyAgICAgICAgIDAgICAgICAgICAgIDk1MTZjYjQwICAgICAgICAgICAgICAgZmlu
 ZCB0c3RpbGUKODQ2NiAgIDg0NjYgMyAgIDIgICAgICAgMTgwICAgICAgICAgICA5NTM3MDc0
 MCAgICAgICAgICAgcG9zdGRyb3AgbmV0aW8KMjIwNzcgMjIwNzcgMyAgIDMgICAgICAgMTgw
 ICAgICAgICAgICA5NjkwNWJjMCAgICAgICAgICAgc2VuZG1haWwgcGlwZV9yZAo3MjU2ICAg
 NzI1NiAzICAgMyAgICAgICAxODAgICAgICAgICAgIDk1MjMwOWMwICAgICAgICAgICAgICAg
 IHRlZSBwaXBlX3JkCjE4MjYzIDE4MjYzIDMgICAwICAgICAgIDE4MCAgICAgICAgICAgOTVk
 NzQxMDAgICAgICAgICAgICAgICAgIHNoIHdhaXQKMjAyOTggMjAyOTggMyAgIDAgICAgICAg
 MWMwICAgICAgICAgICA5MmRhYzMwMCAgICAgICAgICAgICAgICAgc2ggd2FpdAoxMDMxNiAx
 MDMxNiAzICAgMCAgICAgICAxODAgICAgICAgICAgIDk1MGIxNDAwICAgICAgICAgICAgICAg
 Y3JvbiBwaXBlX3JkCjIyMzggPiAyMjM4IDcgICAxICAgICAgICAgMCAgICAgICAgICAgOTVj
 NDM1YzAgICAgICAgICAgICAgICAgIGdvCjg4OCAgICA0MjA1IDIgICAxICAgMTEwMDAwMCAg
 ICAgICAgICAgOTIxZGIxNDAgICAgICAgIHR5cGVzMi50ZXN0Cjg4OCAgICAyMzEyIDIgICAx
 ICAgMTEwMDEwMCAgICAgICAgICAgOTVkNjBiYzAgICAgICAgIHR5cGVzMi50ZXN0Cjg4OCAg
 IDE0OTA1IDMgICAyICAgMTAwMDAwMCAgICAgICAgICAgOTIxZGJjNDAgICAgICAgIHR5cGVz
 Mi50ZXN0IGx3cHdhaXQKODg4ICAgICA4ODggMiAgIDEgICAxMTAwMTAwICAgICAgICAgICA5
 NDE0MDc0MCAgICAgICAgdHlwZXMyLnRlc3QKMTk1OTcgMjc0NTMgMyAgIDIgICAgICAgICAw
 ICAgICAgICAgICA5NTM3MGNjMCAgICAgICAgICAgIGNvbXBpbGUgbHdwd2FpdAoxOTU5NyAy
 OTExMSAyICAgMSAgICAxMDAxNDAgICAgICAgICAgIDkxODBkN2MwICAgICAgICAgICAgY29t
 cGlsZQoxNDA4OCAgNjcyMCAyICAgMSAgIDExMDAxMDAgICAgICAgICAgIDk2NTZkNzAwICAg
 ICAgICAgICAgICAgICBnbwoxNDA4OCAyMzIxNCAyICAgMSAgIDExMDAxMDAgICAgICAgICAg
 IDk1ZTk1YzQwICAgICAgICAgICAgICAgICBnbwoxNDA4OCAgOTk0MSAzICAgMiAgIDEwMDAw
 MDAgICAgICAgICAgIDk1YzQzODgwICAgICAgICAgICAgICAgICBnbyBsd3B3YWl0CjE0MDg4
 ICA0MzgwIDIgICAxICAgMTEwMDE0MCAgICAgICAgICAgOTQxNDBjYzAgICAgICAgICAgICAg
 ICAgIGdvCjE5MjA1ICAgNTYzIDIgICAxICAgMTEwMDEwMCAgICAgICAgICAgOTY1MTM5YzAg
 ICAgICAgICAgdGVzdC50ZXN0CjE5MjA1IDI0MTAyIDMgICAyICAgMTAwMDAwMCAgICAgICAg
 ICAgOTVkNjAwYzAgICAgICAgICAgdGVzdC50ZXN0IGx3cHdhaXQKMjg0OTMgMTg5MzAgMiAg
 IDEgICAxMDAwMTAwICAgICAgICAgICA5MWEyZTM4MCAgICAgICAgICAgICAgICAgZ28KMjg0
 OTMgIDI1NzggMyAgIDIgICAgICAgMTgwICAgICAgICAgICA5NWU5NTZjMCAgICAgICAgICAg
 ICAgICAgZ28gcGFya2VkCjI4NDkzIDI1MzYxIDMgICAxICAgICAgIDE4MCAgICAgICAgICAg
 OTUwYTQzYzAgICAgICAgICAgICAgICAgIGdvIHBhcmtlZAoyODQ5MyAxMDg3NSAzICAgMSAg
 ICAgICAxODAgICAgICAgICAgIDk0OTU3NWMwICAgICAgICAgICAgICAgICBnbyBwYXJrZWQK
 Mjg0OTMgIDQ2NTEgMyAgIDMgICAgICAgMTgwICAgICAgICAgICA5NTM4YjRjMCAgICAgICAg
 ICAgICAgICAgZ28gcGFya2VkCjI4NDkzIDExNjM3IDMgICAzICAgICAgIDFjMCAgICAgICAg
 ICAgOTQ5NTdiNDAgICAgICAgICAgICAgICAgIGdvIHBhcmtlZAoyODQ5MyAgNjEzNyAzICAg
 MiAgICAgICAxODAgICAgICAgICAgIDk2NTEzNzAwICAgICAgICAgICAgICAgICBnbyBwYXJr
 ZWQKMjg0OTMgMTg4MjQgMyAgIDAgICAgICAgMTgwICAgICAgICAgICA5MTkyZTYwMCAgICAg
 ICAgICAgICAgICAgZ28gcGFya2VkCjI4NDkzIDIzMDg5IDMgICAyICAgICAgIDE4MCAgICAg
 ICAgICAgOTFmNmRhMDAgICAgICAgICAgICAgICAgIGdvIG5hbm9zbHAKMjg0OTMgMjg0OTMg
 MiAgIDEgICAgICAgMTAwICAgICAgICAgICA5NDk1Nzg4MCAgICAgICAgICAgICAgICAgZ28K
 NzcwNSAgMjI4ODggMyAgIDIgICAxMDAwMDAwICAgICAgICAgICA5NjU2ZDE4MCAgICAgICAg
 ICAgc3NhLnRlc3QgbHdwd2FpdAo3NzA1ICAxNTYwOCAyICAgMSAgIDExMDAxMDAgICAgICAg
 ICAgIDkyYzk4OWMwICAgICAgICAgICBzc2EudGVzdAo3NzA1ICAxODc3MyAyICAgMSAgIDEx
 MDAxMDAgICAgICAgICAgIDk1MTY4YTgwICAgICAgICAgICBzc2EudGVzdAoxNzc2MSAxNzc2
 MSAyICAgMSAgICAgICAxNDAgICAgICAgICAgIDk0OTUyMDgwICAgICAgICAgICAgICBzbGVl
 cAoxNDA4ICAgMTQwOCAzICAgMCAgICAgICAxODAgICAgICAgICAgIDkzYWNjZDQwICAgICAg
 ICAgICAgICAgICBzaCB3YWl0Cjk3NDQgIDE1ODk2IDMgICAzICAgICAgIDE4MCAgICAgICAg
 ICAgOTY1MTNjODAgICAgICAgICAgICAgICAgIGdvIHBhcmtlZAo5NzQ0ICAgNTk5OCAzICAg
 MyAgICAgICAxODAgICAgICAgICAgIDk1MGIxMTQwICAgICAgICAgICAgICAgICBnbyB3YWl0
 Cjk3NDQgICAxNzQ4IDMgICAwICAgICAgIDE4MCAgICAgICAgICAgOTUxNmM4ODAgICAgICAg
 ICAgICAgICAgIGdvIHBhcmtlZAo5NzQ0ICAyMjk1NiAzICAgMyAgICAgICAxODAgICAgICAg
 ICAgIDk1MGE0NjgwICAgICAgICAgICAgICAgICBnbyBwYXJrZWQKOTc0NCAgMTU3OTggMyAg
 IDAgICAgICAgMTgwICAgICAgICAgICA5MmNkMzc4MCAgICAgICAgICAgICAgICAgZ28gcGFy
 a2VkCjk3NDQgICA1ODI3IDMgICAxICAgICAgIDE4MCAgICAgICAgICAgOTJjYzNhMDAgICAg
 ICAgICAgICAgICAgIGdvIHBhcmtlZAo5NzQ0ICAyNTE1OSAzICAgMiAgICAgICAxODAgICAg
 ICAgICAgIDk1MGE0OTQwICAgICAgICAgICAgICAgICBnbyBwYXJrZWQKOTc0NCAgIDQ2MTEg
 MyAgIDAgICAgICAgMTgwICAgICAgICAgICA5NWQ3NGMwMCAgICAgICAgICAgICAgICAgZ28g
 d2FpdAo5NzQ0ICAxODk1OSAzICAgMCAgICAgICAxODAgICAgICAgICAgIDk1MzhiNzgwICAg
 ICAgICAgICAgICAgICBnbyBwYXJrZWQKOTc0NCAgMjYzMzkgMiAgIDEgICAgICAgMTAwICAg
 ICAgICAgICA5NTIxODQwMCAgICAgICAgICAgICAgICAgZ28KOTc0NCAgMjEwODcgMyAgIDMg
 ICAgICAgMWMwICAgICAgICAgICA5NTE2YzA0MCAgICAgICAgICAgICAgICAgZ28gd2FpdAo5
 NzQ0ICAyNjYxMCAzICAgMyAgICAgICAxODAgICAgICAgICAgIDk1MzhkNTAwICAgICAgICAg
 ICAgICAgICBnbyB3YWl0Cjk3NDQgIDE0Mjg1IDMgICAyICAgICAgIDE4MCAgICAgICAgICAg
 OTRlMTgxYzAgICAgICAgICAgICAgICAgIGdvIHBhcmtlZAo5NzQ0ICAgOTc0NCAzICAgMiAg
 ICAgICAxODAgICAgICAgICAgIDkxODBkYTgwICAgICAgICAgICAgICAgICBnbyBwYXJrZWQK
 MTIxMTMgIDI2NjIgMyAgIDMgICAgICAgMTgwICAgICAgICAgICA5NTIzMDE4MCAgICAgICAg
 ICAgICAgIGRpc3QgcGFya2VkCjEyMTEzIDE0MTQ5IDMgICAyICAgICAgIDE4MCAgICAgICAg
 ICAgOTFmY2M1MDAgICAgICAgICAgICAgICBkaXN0IHBhcmtlZAoxMjExMyAxNjY1MCAzICAg
 MCAgICAgICAxODAgICAgICAgICAgIDk1MWFkOTQwICAgICAgICAgICAgICAgZGlzdCBwYXJr
 ZWQKMTIxMTMgMjgxNjkgMyAgIDEgICAgICAgMTgwICAgICAgICAgICA5NTM4ZGE4MCAgICAg
 ICAgICAgICAgIGRpc3QgcGFya2VkCjEyMTEzICA4NzY3IDMgICAzICAgICAgIDE4MCAgICAg
 ICAgICAgOTJjZDM0YzAgICAgICAgICAgICAgICBkaXN0IHBhcmtlZAoxMjExMyAxMjExMyAz
 ICAgMCAgICAgICAxODAgICAgICAgICAgIDk1MzcwMWMwICAgICAgICAgICAgICAgZGlzdCB3
 YWl0CjcwNTEgICA0MTcxIDMgICAyICAgICAgIDE4MCAgICAgICAgICAgOTVkNzQ2ODAgICAg
 ICAgICAgICAgICAgIGdvIHBhcmtlZAo3MDUxICAgNjc5MyAzICAgMyAgICAgICAxODAgICAg
 ICAgICAgIDk1ZDc0OTQwICAgICAgICAgICAgICAgICBnbyBwYXJrZWQKNzA1MSAgMTE2MDEg
 MyAgIDEgICAgICAgMTgwICAgICAgICAgICA5NTM4YjIwMCAgICAgICAgICAgICAgICAgZ28g
 cGFya2VkCjcwNTEgICAyNjA0IDMgICAxICAgICAgIDE4MCAgICAgICAgICAgOTUxNjgyNDAg
 ICAgICAgICAgICAgICAgIGdvIHBhcmtlZAo3MDUxICAxMzAzOCAzICAgMiAgICAgICAxODAg
 ICAgICAgICAgIDkxYTJlYmMwICAgICAgICAgICAgICAgICBnbyB3YWl0CjcwNTEgIDI5MDIy
 IDIgICAxICAgICAgIDE0MCAgICAgICAgICAgOTJjOThjODAgICAgICAgICAgICAgICAgIGdv
 CjcwNTEgICA3MDUxIDMgICAwICAgICAgIDE4MCAgICAgICAgICAgOTUwZmJhNDAgICAgICAg
 ICAgICAgICAgIGdvIHBhcmtlZAo4MDk5ICAxMDUxMCAzICAgMyAgICAgICAxODAgICAgICAg
 ICAgIDkyMjFiMDQwICAgICByZXN1bHRfYWRhcHRlciBwYXJrZWQKODA5OSAgIDc1OTAgMiAg
 IDEgICAgICAgMTAwICAgICAgICAgICA5NTIxODk4MCAgICAgcmVzdWx0X2FkYXB0ZXIKODA5
 OSAgMjI1MTUgMyAgIDIgICAgICAgMTgwICAgICAgICAgICA5NTFhZDY4MCAgICAgcmVzdWx0
 X2FkYXB0ZXIgcGFya2VkCjgwOTkgIDE5NDk2IDMgICAwICAgICAgIDE4MCAgICAgICAgICAg
 OTJjYzM3NDAgICAgIHJlc3VsdF9hZGFwdGVyIHBhcmtlZAo4MDk5ICAgNzU1NSAzICAgMSAg
 ICAgICAxODAgICAgICAgICAgIDk1ZTk1MTQwICAgICByZXN1bHRfYWRhcHRlciB3YWl0Cjgw
 OTkgICAzMTM3IDMgICAwICAgICAgIDE4MCAgICAgICAgICAgOTUwYTQxMDAgICAgIHJlc3Vs
 dF9hZGFwdGVyIGtxdWV1ZQo4MDk5ICAyOTMzMyAzICAgMyAgICAgICAxODAgICAgICAgICAg
 IDk1MTY4NTAwICAgICByZXN1bHRfYWRhcHRlciBwYXJrZWQKODA5OSAgMjI5ODkgMyAgIDIg
 ICAgICAgMTgwICAgICAgICAgICA5NWU5NTk4MCAgICAgcmVzdWx0X2FkYXB0ZXIgcGFya2Vk
 CjgwOTkgIDE3ODIwIDMgICAzICAgICAgIDE4MCAgICAgICAgICAgOTUyMzBjODAgICAgIHJl
 c3VsdF9hZGFwdGVyIG5hbm9zbHAKODA5OSAgIDgwOTkgMyAgIDMgICAgICAgMTgwICAgICAg
 ICAgICA5NTFhZDEwMCAgICAgcmVzdWx0X2FkYXB0ZXIgcGFya2VkCjIxNDU3ICAxNzUzIDIg
 ICAxICAgICAgIDEwMCAgICAgICAgICAgOTUxNmMzMDAgICAgICAgICAgICAgICAgcmRiCjIx
 NDU3IDE2MzI2IDMgICAzICAgICAgIDE4MCAgICAgICAgICAgOTE2ZTIwNDAgICAgICAgICAg
 ICAgICAgcmRiIHBhcmtlZAoyMTQ1NyAgODU3NCAzICAgMiAgICAgICAxODAgICAgICAgICAg
 IDkxNmUwN2MwICAgICAgICAgICAgICAgIHJkYiBwYXJrZWQKMjE0NTcgMjMxMDcgMyAgIDAg
 ICAgICAgMTgwICAgICAgICAgICA5NTM4ZDI0MCAgICAgICAgICAgICAgICByZGIgcGFya2Vk
 CjIxNDU3ICA3NDEzIDMgICAzICAgICAgIDE4MCAgICAgICAgICAgOTY1NmRjODAgICAgICAg
 ICAgICAgICAgcmRiIHBhcmtlZAoyMTQ1NyAgODkwMCAzICAgMiAgICAgICAxODAgICAgICAg
 ICAgIDk1MzcwYTAwICAgICAgICAgICAgICAgIHJkYiBwYXJrZWQKMjE0NTcgMjIzMzUgMiAg
 IDEgICAgICAgMTQwICAgICAgICAgICA5MmM2YTk4MCAgICAgICAgICAgICAgICByZGIKMjE0
 NTcgMjU4MjUgMyAgIDIgICAgICAgMTgwICAgICAgICAgICA5NTBiMWM0MCAgICAgICAgICAg
 ICAgICByZGIgd2FpdAoyMTQ1NyAxNDIzNyAzICAgMiAgICAgICAxODAgICAgICAgICAgIDk1
 MTY4ZDQwICAgICAgICAgICAgICAgIHJkYiBuYW5vc2xwCjIxNDU3IDIxNDU3IDMgICAxICAg
 ICAgIDE4MCAgICAgICAgICAgOTE2ZTI4ODAgICAgICAgICAgICAgICAgcmRiIHBhcmtlZAox
 NzA2ICAgNzk1NiAzICAgMSAgICAgICAxODAgICAgICAgICAgIDkxZjhkZDAwICAgICAgICBn
 b2xhbmdidWlsZCB3YWl0CjE3MDYgIDE0NTk2IDMgICAwICAgICAgIDE4MCAgICAgICAgICAg
 OTQ5NTI2MDAgICAgICAgIGdvbGFuZ2J1aWxkIHBhcmtlZAoxNzA2ICAyNTEwOCAzICAgMyAg
 ICAgICAxYzAgICAgICAgICAgIDk1MWFkM2MwICAgICAgICBnb2xhbmdidWlsZCBwYXJrZWQK
 MTcwNiAgMTQzNzUgMyAgIDAgICAgICAgMTgwICAgICAgICAgICA5NDk1MjhjMCAgICAgICAg
 Z29sYW5nYnVpbGQga3F1ZXVlCjE3MDYgICA3ODQwIDMgICAyICAgICAgIDFjMCAgICAgICAg
 ICAgOTJhZTE5NDAgICAgICAgIGdvbGFuZ2J1aWxkIHBhcmtlZAoxNzA2ICAyODEwMSAyICAg
 MSAgICAgICAxNDAgICAgICAgICAgIDk0OTU3MzAwICAgICAgICBnb2xhbmdidWlsZAoxNzA2
 ICAgMTcwNiAzICAgMiAgICAgICAxYzAgICAgICAgICAgIDkxYTJlMGMwICAgICAgICBnb2xh
 bmdidWlsZCBwYXJrZWQKMTM0NzMgMjc0ODMgMyAgIDIgICAxMDAwMDAwICAgICAgICAgICA5
 MmNjM2NjMCAgICAgICAgICAgIGJiYWdlbnQgbHdwd2FpdAoxMzQ3MyAyNTIyNCAyICAgMSAg
 IDExMDAxMDAgICAgICAgICAgIDk1MGM1YzgwICAgICAgICAgICAgYmJhZ2VudAoxMzQ3MyAx
 MjI3OSAyICAgMSAgIDExMDAxMDAgICAgICAgICAgIDkyY2QzMjAwICAgICAgICAgICAgYmJh
 Z2VudAoyODI0MiAgODM3MCAyICAgMSAgIDExMDAxNDAgICAgICAgICAgIDkyYzZhYzQwICAg
 ICAgICAgcHl0aG9uMy4xMQoyODI0MiAyMzYyMCAyICAgMSAgIDExMDAxNDAgICAgICAgICAg
 IDkyZGFjNWMwICAgICAgICAgcHl0aG9uMy4xMQoyODI0MiAyODI0MiAzICAgMiAgIDEwMDAw
 MDAgICAgICAgICAgIDkyZGFjYjQwICAgICAgICAgcHl0aG9uMy4xMSBsd3B3YWl0CjMzMTMg
 IDI1MzQyIDMgICAwICAgICAgIDE4MCAgICAgICAgICAgOTJhNmNiYzAgICAgICAgICBweXRo
 b24zLjExIHBhcmtlZAozMzEzICAgMzMxMyAzICAgMyAgICAgICAxYzAgICAgICAgICAgIDky
 YTZjOTAwICAgICAgICAgcHl0aG9uMy4xMSB3YWl0CjMzMTUgICAzMzE1IDMgICAyICAgICAg
 IDE4MCAgICAgICAgICAgOTFkOTAzYzAgICAgICAgICAgICAgIGdldHR5IHR0eXJhdwo3NTYg
 ICAgIDc1NiAzICAgMSAgICAgICAxODAgICAgICAgICAgIDkxZDkwNjgwICAgICAgICAgICAg
 ICBnZXR0eSB0dHlyYXcKNzU3ICAgICA3NTcgMyAgIDMgICAgICAgMTgwICAgICAgICAgICA5
 MWY4ZDRjMCAgICAgICAgICAgICAgZ2V0dHkgdHR5cmF3CjI3MjMgICAyNzIzIDMgICAwICAg
 ICAgIDFjMCAgICAgICAgICAgOTE4MGQ1MDAgICAgICAgICAgICAgIGxvZ2luIHdhaXQKNzQz
 ICAgICA3NDMgMyAgIDMgICAgICAgMTgwICAgICAgICAgICA5Mjg4MTM0MCAgICAgICAgICAg
 ICAgIGNyb24gbmFub3NscAoyOTA5ICAgMjkwOSAyICAgMSAgICAgICAxNDAgICAgICAgICAg
 IDkxZmNjN2MwICAgICAgICAgICAgICAgZXN0ZAo1OTggICAgIDU5OCAzICAgMyAgICAgICAx
 ODAgICAgICAgICAgIDkyODgxNjAwICAgICAgICAgICAgICBpbmV0ZCBrcXVldWUKNTg3ICAg
 ICA3MDMgMyAgIDMgICAgICAgMTgwICAgICAgICAgICA5MThjMTVjMCAgICAgIG5vZGVfZXhw
 b3J0ZXIgcGFya2VkCjU4NyAgICAyNjY4IDMgICAzICAgICAgIDE4MCAgICAgICAgICAgOTI4
 ODFiODAgICAgICBub2RlX2V4cG9ydGVyIHBhcmtlZAo1ODcgICAgMjkyNSAzICAgMyAgICAg
 ICAxYzAgICAgICAgICAgIDkyYTZjMzgwICAgICAgbm9kZV9leHBvcnRlciBrcXVldWUKNTg3
 ICAgIDEwMDYgMyAgIDEgICAgICAgMTgwICAgICAgICAgICA5MWEyZTkwMCAgICAgIG5vZGVf
 ZXhwb3J0ZXIgcGFya2VkCjU4NyAgICAgNzQ3IDIgICAxICAgICAgIDEwMCAgICAgICAgICAg
 OTIyMWI1YzAgICAgICBub2RlX2V4cG9ydGVyCjU4NyAgICAgNTg3IDMgICAwICAgICAgIDFj
 MCAgICAgICAgICAgOTI4ODE4YzAgICAgICBub2RlX2V4cG9ydGVyIHBhcmtlZAo1ODUgICAg
 IDU4NSAyICAgMSAgIDEwMDAxMDAgICAgICAgICAgIDkyYTZjMGMwICAgICAgICAgICAgICAg
 cW1ncgo1ODIgICAgIDU4MiAzICAgMCAgICAgICAxODAgICAgICAgICAgIDkxOGMxYjQwICAg
 ICAgICAgICAgIG1hc3RlciBuYW5vc2xwCjY2MyAgICAgNjY5IDMgICAxICAgICAgIDE4MCAg
 ICAgICAgICAgOTIyMWI4ODAgICAgIGJvb3RzdHJhcHN3YXJtIHdhaXQKNjYzICAgICA2Njgg
 MyAgIDEgICAgICAgMTgwICAgICAgICAgICA5MWY2MzcwMCAgICAgYm9vdHN0cmFwc3dhcm0g
 cGFya2VkCjY2MyAgICAgNjY3IDIgICAxICAgICAgIDEwMCAgICAgICAgICAgOTFkOTBjMDAg
 ICAgIGJvb3RzdHJhcHN3YXJtCjY2MyAgICAgNjY2IDMgICAxICAgICAgIDE4MCAgICAgICAg
 ICAgOTFlMTg2YzAgICAgIGJvb3RzdHJhcHN3YXJtIHBhcmtlZAo2NjMgICAgIDY2NSAzICAg
 MiAgICAgICAxODAgICAgICAgICAgIDkxZjhkNzgwICAgICBib290c3RyYXBzd2FybSBwYXJr
 ZWQKNjYzICAgICA2NjQgMyAgIDIgICAgICAgMTgwICAgICAgICAgICA5MjIxYjMwMCAgICAg
 Ym9vdHN0cmFwc3dhcm0gbmFub3NscAo2NjMgICAgIDY2MyAzICAgMCAgICAgICAxODAgICAg
 ICAgICAgIDkyMjFiYjQwICAgICBib290c3RyYXBzd2FybSBwYXJrZWQKNjQzICAgICA2NDMg
 MyAgIDEgICAgICAgMTgwICAgICAgICAgICA5MWY2ZDc0MCAgICAgICAgICAgICAgICAgc2gg
 d2FpdAozMzUgICAgIDMzNSAyICAgMSAgICAgICAxMDAgICAgICAgICAgIDkxZmNjZDQwICAg
 ICAgICAgICAgICAgc3NoZAoxODU0ICAgMTg1NCAyICAgMSAgIDEwMDAxMDAgICAgICAgICAg
 IDkxZmNjYTgwICAgICAgICAgICAgICAgbnRwZAozOTkgICAgIDM5OSAzICAgMiAgICAgICAx
 ODAgICAgICAgICAgIDkxZTE4MTQwICAgICAgICAgICBtdWx0aWxvZyBwaXBlX3JkCjM5OCAg
 ICAgMzk4IDMgICAxICAgICAgIDE4MCAgICAgICAgICAgOTFmOGQyMDAgICAgICAgICAgIG11
 bHRpbG9nIHBpcGVfcmQKMTk5OCAgIDE5OTggMyAgIDMgICAgICAgMTgwICAgICAgICAgICA5
 MTkyZThjMCAgICAgICAgICAgICAgICAgc2ggd2FpdAoyMzQzICAgMjM0MyAzICAgMCAgICAg
 ICAxODAgICAgICAgICAgIDkxZDkwMTAwICAgICAgICAgICAgICAgICBzaCB3YWl0CjIzOTYg
 ICAyMzk2IDMgICAyICAgICAgIDE4MCAgICAgICAgICAgOTFmNmQ0ODAgICAgICAgICAgc3Vw
 ZXJ2aXNlIHBvbGwKMjIzMSAgIDIyMzEgMyAgIDIgICAgICAgMTgwICAgICAgICAgICA5MWY2
 ZDFjMCAgICAgICAgICBzdXBlcnZpc2UgcG9sbAoxOTc3ICAgMTk3NyAyICAgMSAgICAgICAx
 NDAgICAgICAgICAgIDkxZjYzYzgwICAgICAgICAgIHN1cGVydmlzZQoxNjUwICAgMTY1MCAz
 ICAgMCAgICAgICAxODAgICAgICAgICAgIDkxZjYzOWMwICAgICAgICAgIHN1cGVydmlzZSBw
 b2xsCjIzNDUgICAyMzQ1IDMgICAyICAgICAgIDE4MCAgICAgICAgICAgOTFmNjM0NDAgICAg
 ICAgICAgIG11bHRpbG9nIHBpcGVfcmQKMjI1MiAgIDIyNTIgMiAgIDEgICAgICAgMTQwICAg
 ICAgICAgICA5MWY2MzE4MCAgICAgICAgICAgICBzdnNjYW4KMjEwNyAgIDIxMDcgMiAgIDEg
 ICAgICAgMTQwICAgICAgICAgICA5MWUxODk4MCAgICAgICAgICAgIHN5c2xvZ2QKMjE0MiAg
 IDIxNDIgMyAgIDAgICAgICAgMTgwICAgICAgICAgICA5MWUxODQwMCAgICAgICAgICAgICAg
 bWRuc2Qgc2VsZWN0Cjk1MiAgICAgOTUyIDMgICAwICAgICAgIDE4MCAgICAgICAgICAgOTE5
 MmUwODAgICAgICAgICAgICAgZGhjcGNkIHBvbGwKOTUwICAgICA5NTAgMiAgIDEgICAgICAg
 MTAwICAgICAgICAgICA5MThjMTg4MCAgICAgICAgICAgICBkaGNwY2QKOTQ4ICAgICA5NDgg
 MyAgIDMgICAgICAgMTgwICAgICAgICAgICA5MTgwZGQ0MCAgICAgICAgICAgICBkaGNwY2Qg
 cG9sbAoxMTMwICAgMTEzMCAyICAgMSAgICAgICAxNDAgICAgICAgICAgIDkxYTJlNjQwICAg
 ICAgICAgICAgIGRoY3BjZAo4ODAgICAgIDg4MCAzICAgMSAgICAgICAxODAgICAgICAgICAg
 IDkxOTJlMzQwICAgICAgICAgICAgZGV2cHViZCBkZXZtb24KMSAgICAgICAgIDEgMyAgIDAg
 ICAgICAgMTgwICAgICAgICAgICA5MTcwYzljMCAgICAgICAgICAgICAgIGluaXQgd2FpdAow
 ICAgICAgIDMwMiAzICAgMSAgICAgICAyMDAgICAgICAgICAgIDkxOGMxMzAwICAgICAgIHBv
 b2xkaXNwQDEyNyBwb29sZGlzcAowICAgICAgIDMwNyAzICAgMSAgICAgICAyMDAgICAgICAg
 ICAgIDkxOGMxMDQwICAgICAgICBwb29sZGlzcEA2NCBwb29sZGlzcAowICAgICAgIDM2NiAz
 ICAgMCAgICAgICAyMDAgICAgICAgICAgIDkxN2RjNzQwICAgICAgICAgICAgcGh5c2lvZCBw
 aHlzaW9kCjAgICAgICAgMjE4IDMgICAxICAgICAgIDIwMCAgICAgICAgICAgOTE3ZGM0ODAg
 ICAgICAgICAgICAgIGJ3Zm0wIGJ3Zm0wCjAgICAgICAgMjE3IDMgICAxICAgICAgIDM4MCAg
 ICAgICAgICAgOTE4MGQyNDAgICAgICAgICAgVkNISVFrYS0wIGxueGNtcGx0CjAgICAgICAg
 MjE2IDMgICAyICAgICAgIDIwMCAgICAgICAgICAgOTE3ZjRkMDAgICAgICAgICAgcG9vbGRy
 YWluIHhjbG9jdgowICAgICAgIDIxNSAzICAgMyAgICAgICAyMDAgICAgICAgICAgIDkxN2Y0
 YTQwICAgICAgICAgICAgaW9mbHVzaCBzeW5jZXIKMCAgICAgICAyMTQgMyAgIDAgICAgICAg
 MjAwICAgICAgICAgICA5MTdmNDc4MCAgICAgICAgICAgcGdkYWVtb24gcGdkYWVtb24KMCAg
 ICAgICAyMTMgMyAgIDMgICAgICAgMzgwICAgICAgICAgICA5MTdmNDRjMCAgICAgICAgICAg
 IHZjYXVkaW8gZGF0YQowICAgICAgIDIxMiAzICAgMyAgICAgICAzODAgICAgICAgICAgIDkx
 N2Y0MjAwICAgICAgICAgICBWQ0hJUXMtMCBzZW1hY3YKMCAgICAgICAyMTEgMyAgIDMgICAg
 ICAgMzgwICAgICAgICAgICA5MTdkY2NjMCAgICAgICAgICAgVkNISVFyLTAgc2VtYWN2CjAg
 ICAgICAgMjEwIDMgICAzICAgICAgIDM4MCAgICAgICAgICAgOTE3ZGNhMDAgICAgICAgICAg
 ICBWQ0hJUS0wIHNlbWFjdgowICAgICAgIDIwNyAzICAgMCAgICAgICAyMDAgICAgICAgICAg
 IDkxN2RjMWMwICAgICAgICAgIHN3d3JlYm9vdCBzd3dyZWJvb3QKMCAgICAgICAyMDUgMyAg
 IDAgICAgICAgMjAwICAgICAgICAgICA5MTVkMmQwMCAgICAgICAgICAgc2NzaWJ1czAgc2Nj
 b21wCjAgICAgICAgMjAzIDMgICAyICAgICAgIDIwMCAgICAgICAgICAgOTE3MGNjODAgICAg
 ICAgICAgICAgbnBmZ2MwIG5wZmdjdwowICAgICAgIDIwMiAzICAgMCAgICAgICAyMDAgICAg
 ICAgICAgIDkxNzBjNzAwICAgICAgICAgICAgcnRfZnJlZSBydF9mcmVlCjAgICAgICAgMjAx
 IDMgICAwICAgICAgIDIwMCAgICAgICAgICAgOTE3MGM0NDAgICAgICAgICAgICAgIHVucGdj
 IHVucGdjCjAgICAgICAgMjAwIDMgICAyICAgICAgIDIwMCAgICAgICAgICAgOTE3MGMxODAg
 ICAga2V5X3RpbWVoYW5kbGVyIGtleV90aW1laGFuZGxlCnIKMCAgICAgICAxOTkgMyAgIDMg
 ICAgICAgMjAwICAgICAgICAgICA5MTcwM2M0MCAgICBpY21wNl93cWlucHV0LzMgaWNtcDZf
 d3FpbnB1dAowICAgICAgIDE5OCAzICAgMiAgICAgICAyMDAgICAgICAgICAgIDkxNmUyMzAw
 ICAgIGljbXA2X3dxaW5wdXQvMiBpY21wNl93cWlucHV0CjAgICAgICAgMTk3IDMgICAxICAg
 ICAgIDIwMCAgICAgICAgICAgOTE2ZTBkNDAgICAgaWNtcDZfd3FpbnB1dC8xIGljbXA2X3dx
 aW5wdXQKMCAgICAgICAxOTYgMyAgIDAgICAgICAgMjAwICAgICAgICAgICA5MTZlMDUwMCAg
 ICBpY21wNl93cWlucHV0LzAgaWNtcDZfd3FpbnB1dAowICAgICAgIDE5NSAzICAgMiAgICAg
 ICAyMDAgICAgICAgICAgIDkxNmUyNWMwICAgICAgICAgICAgICAgdXNiMCB1c2JldnQKMCAg
 ICAgICAxOTQgMyAgIDIgICAgICAgMjAwICAgICAgICAgICA5MTcwMzk4MCAgICAgICAgICBu
 ZDZfdGltZXIgbmQ2X3RpbWVyCjAgICAgICAgMTkzIDMgICAzICAgICAgIDIwMCAgICAgICAg
 ICAgOTE3MDM2YzAgICAgY2FycDZfd3FpbnB1dC8zIGNhcnA2X3dxaW5wdXQKMCAgICAgICAx
 OTIgMyAgIDIgICAgICAgMjAwICAgICAgICAgICA5MTcwMzQwMCAgICBjYXJwNl93cWlucHV0
 LzIgY2FycDZfd3FpbnB1dAowICAgICAgIDE3NiAzICAgMSAgICAgICAyMDAgICAgICAgICAg
 IDkxNzAzMTQwICAgIGNhcnA2X3dxaW5wdXQvMSBjYXJwNl93cWlucHV0CjAgICAgICAgMTc1
 IDMgICAwICAgICAgIDIwMCAgICAgICAgICAgOTE2ZmZjMDAgICAgY2FycDZfd3FpbnB1dC8w
 IGNhcnA2X3dxaW5wdXQKMCAgICAgICAxNzcgMyAgIDMgICAgICAgMjAwICAgICAgICAgICA5
 MTZmZjk0MCAgICAgY2FycF93cWlucHV0LzMgY2FycF93cWlucHV0CjAgICAgICAgMTcwIDMg
 ICAyICAgICAgIDIwMCAgICAgICAgICAgOTE2ZmY2ODAgICAgIGNhcnBfd3FpbnB1dC8yIGNh
 cnBfd3FpbnB1dAowICAgICAgIDE3MyAzICAgMSAgICAgICAyMDAgICAgICAgICAgIDkxNmZm
 M2MwICAgICBjYXJwX3dxaW5wdXQvMSBjYXJwX3dxaW5wdXQKMCAgICAgICAgMzEgMyAgIDAg
 ICAgICAgMjAwICAgICAgICAgICA5MTZmZjEwMCAgICAgY2FycF93cWlucHV0LzAgY2FycF93
 cWlucHV0CjAgICAgICAgIDYzIDMgICAzICAgICAgIDIwMCAgICAgICAgICAgOTE2ZjdiYzAg
 ICAgIGljbXBfd3FpbnB1dC8zIGljbXBfd3FpbnB1dAowICAgICAgIDEyNiAzICAgMiAgICAg
 ICAyMDAgICAgICAgICAgIDkxNmY3OTAwICAgICBpY21wX3dxaW5wdXQvMiBpY21wX3dxaW5w
 dXQKMCAgICAgICAxMjUgMyAgIDEgICAgICAgMjAwICAgICAgICAgICA5MTZmNzY0MCAgICAg
 aWNtcF93cWlucHV0LzEgaWNtcF93cWlucHV0CjAgICAgICAgMTI0IDMgICAwICAgICAgIDIw
 MCAgICAgICAgICAgOTE2ZjczODAgICAgIGljbXBfd3FpbnB1dC8wIGljbXBfd3FpbnB1dAow
 ICAgICAgIDEyMyAzICAgMyAgICAgICAyMDAgICAgICAgICAgIDkxNmY3MGMwICAgICAgICAg
 ICBydF90aW1lciBydF90aW1lcgowICAgICAgIDEyMiAyICAgMSAgICAgICAyMDAgICAgICAg
 ICAgIDkxNmU2YjgwICAgICAgICB2bWVtX3JlaGFzaAowICAgICAgIDEyMSAzICAgMyAgICAg
 ICAyMDAgICAgICAgICAgIDkxNmUwYTgwICAgICAgICAgICAgdmNtYm94MCB2Y21ib3gwCjAg
 ICAgICAgMTIwIDMgICAzICAgICAgIDIwMCAgICAgICAgICAgOTE2ZTY4YzAgICAgICAgICB1
 c2J0YXNrLWRyIHVzYnRzawowICAgICAgIDExOSAzICAgMiAgICAgICAyMDAgICAgICAgICAg
 IDkxNmU2NjAwICAgICAgICAgdXNidGFzay1oYyB1c2J0c2sKMCAgICAgICAxMTggMyAgIDIg
 ICAgICAgMjAwICAgICAgICAgICA5MTZlNjM0MCAgICAgICAgICAgICAgIGR3YzIgZHdjMgow
 ICAgICAgIDExNyAzICAgMyAgICAgICAyMDAgICAgICAgICAgIDkxNmU2MDgwICAgICAgICAg
 ICAgIHNkbW1jMSBtbWN0YXNrcQowICAgICAgIDExNiAzICAgMyAgICAgICAyMDAgICAgICAg
 ICAgIDkxNmUyYjQwICAgICAgICAgICAgIHNkbW1jMCBtbWN0YXNrcQowICAgICAgIDEwNyAy
 ICAgMSAgICAgICAyNDAgICAgICAgICAgIDkxNmUwMjQwICAgICAgICAgIGVudGJ1dGxlcgow
 ICAgICAgIDEwNSAzICAgMyAgICAgICAyMDAgICAgICAgICAgIDkxNWQyYTQwICAgICAgICAg
 ICAgeGNhbGwvMyB4Y2FsbAowICAgICAgIDEwNCAxICAgMyAgICAgICAyMDAgICAgICAgICAg
 IDkxNWQyNzgwICAgICAgICAgIHNvZnRzZXIvMwowICAgICAgIDEwMyAxICAgMyAgICAgICAy
 MDAgICAgICAgICAgIDkxNWQyNGMwICAgICAgICAgIHNvZnRjbGsvMwowICAgICAgIDEwMiAx
 ICAgMyAgICAgICAyMDAgICAgICAgICAgIDkxNWQyMjAwICAgICAgICAgIHNvZnRiaW8vMwow
 ICAgICAgIDEwMSAxICAgMyAgICAgICAyMDAgICAgICAgICAgIDkxNWNkY2MwICAgICAgICAg
 IHNvZnRuZXQvMwowICAgID4gIDEwMCAxICAgMyAgICAgICAyMDEgICAgICAgICAgIDkxNWNk
 YTAwICAgICAgICAgICAgIGlkbGUvMwowICAgICAgICA5OSAzICAgMiAgICAgICAyMDAgICAg
 ICAgICAgIDkxNWNkNzQwICAgICAgICAgICAgeGNhbGwvMiB4Y2FsbAowICAgICAgICA5OCAx
 ICAgMiAgICAgICAyMDAgICAgICAgICAgIDkxNWNkNDgwICAgICAgICAgIHNvZnRzZXIvMgow
 ICAgICAgICA5NyAxICAgMiAgICAgICAyMDAgICAgICAgICAgIDkxNWNkMWMwICAgICAgICAg
 IHNvZnRjbGsvMgowICAgICAgICA5NiAxICAgMiAgICAgICAyMDAgICAgICAgICAgIDkxNWNh
 YzgwICAgICAgICAgIHNvZnRiaW8vMgowICAgICAgICAzMCAxICAgMiAgICAgICAyMDAgICAg
 ICAgICAgIDkxNWNhOWMwICAgICAgICAgIHNvZnRuZXQvMgowICAgID4gICAyOSAxICAgMiAg
 ICAgICAyMDEgICAgICAgICAgIDkxNWNhNzAwICAgICAgICAgICAgIGlkbGUvMgowICAgICAg
 ICAyOCAyICAgMSAgICAgICAyMDAgICAgICAgICAgIDkxNWNhNDQwICAgICAgICAgICAgeGNh
 bGwvMQowICAgICAgICAyNyAxICAgMSAgICAgICAyMDAgICAgICAgICAgIDkxNWNhMTgwICAg
 ICAgICAgIHNvZnRzZXIvMQowICAgICAgICAyNiAxICAgMSAgICAgICAyMDAgICAgICAgICAg
 IDkxNWI3YzQwICAgICAgICAgIHNvZnRjbGsvMQowICAgICAgICAyNSAxICAgMSAgICAgICAy
 MDAgICAgICAgICAgIDkxNWI3OTgwICAgICAgICAgIHNvZnRiaW8vMQowICAgICAgICAyNCAx
 ICAgMSAgICAgICAyMDAgICAgICAgICAgIDkxNWI3NmMwICAgICAgICAgIHNvZnRuZXQvMQow
 ICAgICAgICAyMyAxICAgMSAgICAgICAyMDEgICAgICAgICAgIDkxNWI3NDAwICAgICAgICAg
 ICAgIGlkbGUvMQowICAgICAgICAyMiAzICAgMCAgICAgICAyMDAgICAgICAgICAgIDkxNWI3
 MTQwICAgICAgICAgICBsbnhzeXN3cSBsbnhzeXN3cQowICAgICAgICAyMSAzICAgMCAgICAg
 ICAyMDAgICAgICAgICAgIDkxNWI2YzAwICAgICAgICAgICBsbnh1YmR3cSBsbnh1YmR3cQow
 ICAgICAgICAyMCAzICAgMSAgICAgICAyMDAgICAgICAgICAgIDkxNWI2OTQwICAgICAgICAg
 ICBsbnhwd3J3cSBsbnhwd3J3cQowICAgICAgICAxOSAzICAgMyAgICAgICAyMDAgICAgICAg
 ICAgIDkxNWI2NjgwICAgICAgICAgICBsbnhsbmd3cSBsbnhsbmd3cQowICAgICAgICAxOCAz
 ICAgMCAgICAgICAyMDAgICAgICAgICAgIDkxNWI2M2MwICAgICAgICAgICBsbnhoaXB3cSBs
 bnhoaXB3cQowICAgICAgICAxNyAzICAgMCAgICAgICAyMDAgICAgICAgICAgIDkxNWI2MTAw
 ICAgICAgICAgICBsbnhyY3VnYyBsbnhyY3VnYwowICAgICAgICAxNiAzICAgMiAgICAgICAy
 MDAgICAgICAgICAgIDkxNWIxYmMwICAgICAgICAgICAgIHN5c21vbiBzbXRhc2txCjAgICAg
 ICAgIDE1IDMgICAwICAgICAgIDIwMCAgICAgICAgICAgOTE1YjE5MDAgICAgICAgICBwbWZz
 dXNwZW5kIHBtZnN1c3BlbmQKMCAgICAgICAgMTQgMyAgIDAgICAgICAgMjAwICAgICAgICAg
 ICA5MTViMTY0MCAgICAgICAgICAgcG1mZXZlbnQgcG1mZXZlbnQKMCAgICAgICAgMTMgMyAg
 IDAgICAgICAgMjAwICAgICAgICAgICA5MTViMTM4MCAgICAgICAgIHNvcGVuZGZyZWUgc29w
 ZW5kZnIKMCAgICAgICAgMTIgMyAgIDMgICAgICAgMjAwICAgICAgICAgICA5MTViMTBjMCAg
 ICAgICAgICAgICBpZndkb2cgaWZ3ZG9nCjAgICAgICAgIDExIDMgICAyICAgICAgIDIwMCAg
 ICAgICAgICAgOTEzZDZiODAgICAgICAgICAgICBpZmxua3N0IGlmbG5rc3QKMCAgICAgICAg
 MTAgMyAgIDAgICAgICAgMjAwICAgICAgICAgICA5MTNkNjhjMCAgICAgICAgICAgbmZzc2ls
 bHkgbmZzc2lsbHkKMCAgICAgICAgIDkgMyAgIDMgICAgICAgMjQwICAgICAgICAgICA5MTNk
 NjYwMCAgICAgICAgICAgICB2ZHJhaW4gdmRyYWluCjAgICAgICAgICA4IDMgICAxICAgICAg
 IDI0MCAgICAgICAgICAgOTEzZDYzNDAgICAgICAgICAgbW9kdW5sb2FkIG1vZF91bmxkCjAg
 ICAgICAgICA3IDMgICAwICAgICAgIDIwMCAgICAgICAgICAgOTEzZDYwODAgICAgICAgICAg
 ICB4Y2FsbC8wIHhjYWxsCjAgICAgICAgICA2IDEgICAwICAgICAgIDIwMCAgICAgICAgICAg
 OTEzZDRiNDAgICAgICAgICAgc29mdHNlci8wCjAgICAgICAgICA1IDEgICAwICAgICAgIDIw
 MCAgICAgICAgICAgOTEzZDQ4ODAgICAgICAgICAgc29mdGNsay8wCjAgICAgICAgICA0IDEg
 ICAwICAgICAgIDIwMCAgICAgICAgICAgOTEzZDQ1YzAgICAgICAgICAgc29mdGJpby8wCjAg
 ICAgICAgICAzIDEgICAwICAgICAgIDIwMCAgICAgICAgICAgOTEzZDQzMDAgICAgICAgICAg
 c29mdG5ldC8wCjAgICAgICAgICAyIDEgICAwICAgICAgIDIwMSAgICAgICAgICAgOTEzZDQw
 NDAgICAgICAgICAgICAgaWRsZS8wCjAgICAgICAgICAwIDMgICAzICAgICAgIDIwMCAgICAg
 ICAgICAgODA3ZmE5YzAgICAgICAgICAgICBzd2FwcGVyIHV2bQo=

 --------------oZbu7N872voTY1zaznVMqxsk--

From: Taylor R Campbell <riastradh@NetBSD.org>
To: Benny Siegert <bsiegert@gmail.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/58871: Stuck processes
Date: Sat, 28 Dec 2024 15:13:44 +0000

 > Date: Sat, 14 Dec 2024 15:13:02 +0100
 > From: Benny Siegert <bsiegert@gmail.com>
 > 
 > Here is another set of logs from a system in this state.
 > 
 > What struck me is that like last time, there is a "find" process in 
 > tstile state. I actually have a trace of where it is stuck, see below.
 > 
 > I don't think the Go tests (which are the main thing running on these 
 > boxes) run find at all, so maybe this is some cronjob?

 Probably something in /etc/daily or /etc/security, yes.  You can
 disable these in /etc/daily.conf or /etc/security.conf; see the man
 pages for details.  But the symptoms suggests there's a deep problem
 here which we should really find the source of.

 > Am 03.12.24 um 21:05 schrieb Taylor R Campbell via gnats:
 > >   Can you start crash(8) and get output from `ps', `ps/w', and `show all
 > >   tstiles'?
 > 
 > See below.
 > 
 > >   Can you start crash(8) and stack traces from the processes not in RUN
 > >   state, like the tstile one with `bt 0t18154'?
 > 
 > Did that for the one process that was in tstile, the "find" one.

 Thanks, that's pretty weird: find is waiting for a lock but nobody is
 holding it!

   PID   LID          COMMAND      WAITING-FOR     WAIT-CHANNEL
 11160 11160             find                0         95e9ea40

 Just to be absolutely sure, can you send output of the following on
 your kernel?

 $ ident /netbsd | grep -F cpuswitch.S

 And can you share the disassembly of the function cpu_switchto in your
 kernel, with objdump or gdb?

 Also, if you have netbsd.gdb for the kernel, can you force a crash
 dump (enter ddb and run `sync'), so we can see the arguments and
 locals in the stack trace?  Curious to see what file find is stuck on.

 > >   Can you run dtrace to sample what's happening?
 > >   
 > >   dtrace -n 'profile:::profile-97 { @[stack()] = count() }'
 > 
 > This made the system hang, for whatever reason.

 Does it make the system hang only while in this state, or does it
 happen every time?  How does it hang -- is it completely unresponsive,
 can you enter ddb, can you interact with crash, or what?

 (See earlier messages on this ticket for ways to enter ddb if C-A-ESC
 isn't working.)

State-Changed-From-To: open->feedback
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Sat, 28 Dec 2024 15:15:36 +0000
State-Changed-Why:
more feedback requested


From: Benny Siegert <bsiegert@gmail.com>
To: Taylor R Campbell <riastradh@NetBSD.org>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/58871: Stuck processes
Date: Tue, 31 Dec 2024 21:50:35 +0100

 Am 28.12.24 um 16:13 schrieb Taylor R Campbell:
 >> Did that for the one process that was in tstile, the "find" one.
 > 
 > Thanks, that's pretty weird: find is waiting for a lock but nobody is
 > holding it!
 > 
 >    PID   LID          COMMAND      WAITING-FOR     WAIT-CHANNEL
 > 11160 11160             find                0         95e9ea40

 I have another hanging find process. Again, this is the only process in 
 tstile, same symptoms.

 Here's a backtrace of where it is stuck:

 db{0}> trace/t 0t15932
 trace: pid 15932 lid 15932 at 0xbced4b3c
 0xbced4b3c: netbsd:mi_switch+0xc
 0xbced4b64: netbsd:sleepq_block+0xac
 0xbced4bd4: netbsd:turnstile_block+0x32c
 0xbced4c44: netbsd:rw_enter+0x148
 0xbced4c5c: netbsd:genfs_lock+0x7c
 0xbced4c9c: netbsd:VOP_LOCK+0x84
 0xbced4cb4: netbsd:vn_lock+0x18
 0xbced4d9c: netbsd:namei_tryemulroot.constprop.0+0x6f0
 0xbced4dcc: netbsd:namei+0x58
 0xbced4e34: netbsd:do_sys_statat+0xd0
 0xbced4eec: netbsd:sys___lstat50+0x2c
 0xbced4fac: netbsd:syscall+0x188


 > Just to be absolutely sure, can you send output of the following on
 > your kernel?
 > 
 > $ ident /netbsd | grep -F cpuswitch.S

 This is the NetBSD 10.1 release, so:

       $NetBSD: cpuswitch.S,v 1.105.12.1 2023/07/31 13:36:32 martin Exp $

 > And can you share the disassembly of the function cpu_switchto in your
 > kernel, with objdump or gdb?


 (gdb) disassemble cpu_switchto
 Dump of assembler code for function cpu_switchto:
     0x8009ea7c <+0>:     mov     r12, sp
     0x8009ea80 <+4>:     push    {r4, r5, r6, r7, r12, lr}
     0x8009ea84 <+8>:     mov     r6, r1
     0x8009ea88 <+12>:    mov     r4, r0
     0x8009ea8c <+16>:    ldr     r5, [r6]
     0x8009ea90 <+20>:    ldr     r7, [r4, #32]
     0x8009ea94 <+24>:    strd    r8, [r7]
     0x8009ea98 <+28>:    strd    r10, [r7, #8]
     0x8009ea9c <+32>:    strd    r12, [r7, #16]
     0x8009eaa0 <+36>:    mrc     15, 0, r0, cr13, cr0, {2}
     0x8009eaa4 <+40>:    str     r0, [r7, #32]
     0x8009eaa8 <+44>:    cpsid   i
     0x8009eaac <+48>:    mcr     15, 0, r6, cr13, cr0, {4}
     0x8009eab0 <+52>:    dmb     sy
     0x8009eab4 <+56>:    str     r6, [r5, #896]  ; 0x380
     0x8009eab8 <+60>:    dmb     sy
     0x8009eabc <+64>:    ldr     r7, [r6, #32]
     0x8009eac0 <+68>:    ldr     sp, [r7, #20]
     0x8009eac4 <+72>:    cpsie   i
     0x8009eac8 <+76>:    ldr     r0, [r6, #80]   ; 0x50
     0x8009eacc <+80>:    tst     r0, #512        ; 0x200
     0x8009ead0 <+84>:    bne     0x8009eb18 <cpu_switchto+156>
     0x8009ead4 <+88>:    ldr     r0, [r7, #32]
     0x8009ead8 <+92>:    mcr     15, 0, r0, cr13, cr0, {2}
     0x8009eadc <+96>:    ldr     r0, [r6, #504]  ; 0x1f8
     0x8009eae0 <+100>:   mcr     15, 0, r0, cr13, cr0, {3}
     0x8009eae4 <+104>:   ldr     r0, [r5, #1068] ; 0x42c
     0x8009eae8 <+108>:   cmp     r0, #0
     0x8009eaec <+112>:   ldrne   r0, [r7, #48]   ; 0x30
     0x8009eaf0 <+116>:   vmsrne  fpexc, r0
     0x8009eaf4 <+120>:   ldr     r0, [r6, #308]  ; 0x134
     0x8009eaf8 <+124>:   ldr     r2, [r0, #132]  ; 0x84
     0x8009eafc <+128>:   cmp     r2, #0
     0x8009eb00 <+132>:   beq     0x8009eb18 <cpu_switchto+156>
     0x8009eb04 <+136>:   ldr     r8, [r6, #36]   ; 0x24
     0x8009eb08 <+140>:   ldr     r1, [r8, #76]   ; 0x4c
     0x8009eb0c <+144>:   bl      0x8043c368 <ras_lookup>
     0x8009eb10 <+148>:   cmn     r0, #1
     0x8009eb14 <+152>:   strne   r0, [r8, #76]   ; 0x4c
     0x8009eb18 <+156>:   ldrd    r8, [r7]
     0x8009eb1c <+160>:   ldrd    r10, [r7, #8]
     0x8009eb20 <+164>:   ldr     r12, [r7, #16]
     0x8009eb24 <+168>:   mov     r0, r4
     0x8009eb28 <+172>:   mov     r1, r6
     0x8009eb2c <+176>:   clrex
     0x8009eb30 <+180>:   pop     {r4, r5, r6, r7, r12, pc}
 End of assembler dump.

 > Also, if you have netbsd.gdb for the kernel, can you force a crash
 > dump (enter ddb and run `sync'), so we can see the arguments and
 > locals in the stack trace?  Curious to see what file find is stuck on.

 Is netbsd.gdb part of the release? I haven't found it on cdn.netbsd.org. 
 The "debug" set contains a file /usr/libdata/debug/netbsd-GENERIC.debug, 
 but that's all I have.

 -- 
 Benny

From: Benny Siegert <bsiegert@gmail.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, 
    riastradh@netbsd.org
Subject: Re: kern/58871: Stuck processes
Date: Fri, 3 Jan 2025 09:34:27 +0000 (UTC)

 On Tue, 31 Dec 2024, Benny Siegert via gnats wrote:

 > > Also, if you have netbsd.gdb for the kernel, can you force a crash
 > > dump (enter ddb and run `sync'), so we can see the arguments and
 > > locals in the stack trace?  Curious to see what file find is stuck on.

 The other issue is that "sync" in ddb hangs forever when the system is
 in this state.

 -- 
 Benny

From: "J. Hannken-Illjes" <hannken@mailbox.org>
To: NetBSD GNATS <gnats-bugs@netbsd.org>
Cc: 
Subject: Re: kern/58871: Stuck processes
Date: Fri, 3 Jan 2025 10:38:27 +0100

 Benny,  you could enter "reboot 104" to ddb.  No flush, no sync, just =
 crash dump.

 --
 J. Hannken-Illjes - hannken@mailbox.org=

From: Benny Siegert <bsiegert@gmail.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, 
    riastradh@netbsd.org
Subject: Re: kern/58871: Stuck processes
Date: Fri, 3 Jan 2025 20:35:18 +0000 (UTC)

 On Fri, 3 Jan 2025, J. Hannken-Illjes via gnats wrote:

 > Benny,  you could enter "reboot 104" to ddb.  No flush, no sync, just =
 > crash dump.

 Thanks! But:

 dump to dev 92,1 not possible

 -- 
 Benny

From: Taylor R Campbell <riastradh@NetBSD.org>
To: Benny Siegert <bsiegert@gmail.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/58871: Stuck processes
Date: Sat, 4 Jan 2025 04:18:11 +0000

 > Date: Tue, 31 Dec 2024 21:50:35 +0100
 > From: Benny Siegert <bsiegert@gmail.com>
 > 
 > Am 28.12.24 um 16:13 schrieb Taylor R Campbell:
 > > Just to be absolutely sure, can you send output of the following on
 > > your kernel?
 > > 
 > > $ ident /netbsd | grep -F cpuswitch.S
 > 
 > This is the NetBSD 10.1 release, so:
 > 
 >       $NetBSD: cpuswitch.S,v 1.105.12.1 2023/07/31 13:36:32 martin Exp $
 > 
 > > And can you share the disassembly of the function cpu_switchto in your
 > > kernel, with objdump or gdb?
 > 
 > (gdb) disassemble cpu_switchto
 > Dump of assembler code for function cpu_switchto:
 > [...]
 >     0x8009eaac <+48>:    mcr     15, 0, r6, cr13, cr0, {4}
 >     0x8009eab0 <+52>:    dmb     sy
 >     0x8009eab4 <+56>:    str     r6, [r5, #896]  ; 0x380
 >     0x8009eab8 <+60>:    dmb     sy

 So this has the barriers I added in pullup-10 #264 for PR kern/57240:
 Missing store-before-load barriers in cpu_switchto
 <https://gnats.NetBSD.org/57240>.  (Actually the store-before-load
 barrier (second dmb above) was already there, but it was missing a
 store-before-store barrier (first dmb above).  In any case, they're
 both there now -- and in softint_switch.)  Which rules out the
 hypothesis of a missing barrier for arm32 locks.

 Now it occurs to me that the lock in question is a reader/writer lock,
 not a mutex.  So probably what's going on here is that some thread
 holds (or leaked) a reader lock on the vnode, which doesn't show up in
 show all tstiles -- tracking a list of readers is costly (not even
 sure if LOCKDEBUG does that).  But maybe we can make a guess by
 finding what the file is through the syscall arguments to find.

 > > Also, if you have netbsd.gdb for the kernel, can you force a crash
 > > dump (enter ddb and run `sync'), so we can see the arguments and
 > > locals in the stack trace?  Curious to see what file find is stuck on.
 > 
 > Is netbsd.gdb part of the release? I haven't found it on cdn.netbsd.org. 
 > The "debug" set contains a file /usr/libdata/debug/netbsd-GENERIC.debug, 
 > but that's all I have.

 That's the debug data part of netbsd.gdb, which if combined with the
 netbsd kernel is good enough for a debugger.

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.