Can't upload WUs on work servers and 150.136.14.110

Moderators: Site Moderators, FAHC Science Team

Post Reply
Jandska
Posts: 18
Joined: Thu May 07, 2020 11:07 am

Can't upload WUs on work servers and 150.136.14.110

Post by Jandska »

Hi, I have problems with uploading my two WUs on two different work servers and then on one common collection server.

The messages are like this:

Code: Select all

10:15:36:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:16925 run:35 clone:7519 gen:23 core:0xa7 unit:0x0000001f81d528e50000000000231d5f
10:15:37:WU01:FS00:Uploading 6.12MiB to 129.213.40.229
10:15:37:WU01:FS00:Connecting to 129.213.40.229:8080
10:15:41:WU03:FS00:0xa7:Completed 225000 out of 250000 steps (90%)
10:16:07:WARNING:WU01:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
10:16:07:WU01:FS00:Trying to send results to collection server
10:16:07:WU01:FS00:Uploading 6.12MiB to 150.136.14.110
10:16:07:WU01:FS00:Connecting to 150.136.14.110:8080
10:16:09:WU02:FS01:0x22:Completed 810000 out of 1000000 steps (81%)
10:16:37:ERROR:WU01:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
Although apparently according to this: https://apps.foldingathome.org/wu#proje ... 519&gen=23 I already got my credits for this WU yesterday.

And now also this WU:

Code: Select all

12:50:32:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:13428 run:15318 clone:26 gen:1 core:0x22 unit:0x0000000112bc7d9a000000003bd6001a
12:50:32:WU02:FS01:Uploading 7.72MiB to 18.188.125.154
12:50:32:WU02:FS01:Connecting to 18.188.125.154:8080
12:51:03:WARNING:WU02:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:51:03:WU02:FS01:Trying to send results to collection server
12:51:03:WU02:FS01:Uploading 7.72MiB to 150.136.14.110
12:51:03:WU02:FS01:Connecting to 150.136.14.110:8080
12:51:34:ERROR:WU02:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
Top
HaloJones
Posts: 920
Joined: Thu Jul 24, 2008 10:16 am

Re: Can't upload WUs on work servers and 150.136.14.110

Post by HaloJones »

ditto
single 1070

Image
Top
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: Can't upload WUs on work servers and 150.136.14.110

Post by Neil-B »

Sorry to here you are having issues - this may be (I believe) one of a recurring scenario where a "short response" message in logged by the FAHClient on completed (successfully) upload of WU (that I have mentioned in a few other threads) ... It wouldn't surprise me if these are followed (at some point but usually next attempt) by a "server did not like ... dumped" message ... hopefully I'll get the time today to do a proper investigation - it was stymied last time I tried as the WUs I needed to check at the time were part of the batch that hadn't been logged into the WU status/Stats system - but they are now in so I can retrospectively sort it.

If anyone spots a pattern in their log of a "short response" to WU upload - which then appears to have actually uploaded on WU Status - and which is then subsequently dumped - could they post this as if it is happening as often as I think it might be there may be something that needs resolving - the issue being that the researchers may not notice this type of thing as the WU is from their perspective being returned ok - the only way it will show server side is in an uptick in duplicate return attempts and I am not sure if those will be being monitored.
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070
(Green/Bold = Active)
Top
Jandska
Posts: 18
Joined: Thu May 07, 2020 11:07 am

Re: Can't upload WUs on work servers and 150.136.14.110

Post by Jandska »

Yes...it seems according to "WU Status" that both WUs got me credits and appear as returned. But they're still trying to be send with the same message ("...short response...")
Top
Guvidus
Posts: 1
Joined: Sun Dec 06, 2020 7:06 pm

Re: Can't upload WUs on work servers and 150.136.14.110

Post by Guvidus »

Same problem. I now have 7 WU queued up, all getting the same error, but credit received. But for me it happens only when connected to my home internet connection which has really slow upload, about 2 mbps. With the 4G connection from my phone it works just fine, no error.
Top
Joe_H
Site Admin
Posts: 7875
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Can't upload WUs on work servers and 150.136.14.110

Post by Joe_H »

If the WUs are completely uploading, and checking with the app provided shows they have been received from your system and credited, that points to your home internet connection dropping the network ACK packet acknowledging receipt. That could be local to you systems at home, or done by at one of your ISP's routers or switches. The client will try resending the WU until it either expires or a send gets back an "already received" ACK packet, then it will remove the WU from being sent.

If you have confirmed that a WU has been received you can delete the work folder for it. Make careful note of the WU number in the queue. Then go to the work folder in the F@h data directory, the client will show this as the CWD: Pause folding. Then you can delete the folder in 'work' with the same number as the WU that you have confirmed has been sent and received. When you resume folding the missing work files will be detected and the queue for sending WUs cleaned up.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Top
Knish
Posts: 232
Joined: Tue Mar 17, 2020 5:20 am

Re: Can't upload WUs on work servers and 150.136.14.110

Post by Knish »

I'd like to re-emphasize that it's the Work Queue, ... NOT what's under the Folding Slots. So Work Queue is the lower box in the Advanced Control, and you should go by the number shown there.
Top
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Can't upload WUs on work servers and 150.136.14.110

Post by bruce »

Right. It's Absolutely NOT the slot number that you should be looking for, it's the queue position of the WU itself.
Top
matthewslt
Posts: 2
Joined: Fri Dec 11, 2020 11:31 am

Re: Can't upload WUs on work servers and 150.136.14.110

Post by matthewslt »

I'm having the exact same issue. Deleting the WUs isnt much of a solution as the client continuously tries to upload the WUs so effectively requires constant attention to keep the queue empty. Please could you look into this as it is constantly trying to upload and is using my already limited bandwidth.

Code: Select all


06:39:22:WU17:FS01:0x22:*********************** Log Started 2020-12-11T06:39:21Z ***********************
06:39:22:WU17:FS01:0x22:*************************** Core22 Folding@home Core ***************************
06:39:22:WU17:FS01:0x22:       Core: Core22
06:39:22:WU17:FS01:0x22:       Type: 0x22
06:39:22:WU17:FS01:0x22:    Version: 0.0.13
06:39:22:WU17:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
06:39:22:WU17:FS01:0x22:  Copyright: 2020 foldingathome.org
06:39:22:WU17:FS01:0x22:   Homepage: https://foldingathome.org/
06:39:22:WU17:FS01:0x22:       Date: Sep 19 2020
06:39:22:WU17:FS01:0x22:       Time: 02:35:58
06:39:22:WU17:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
06:39:22:WU17:FS01:0x22:     Branch: core22-0.0.13
06:39:22:WU17:FS01:0x22:   Compiler: Visual C++ 2015
06:39:22:WU17:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
06:39:22:WU17:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
06:39:22:WU17:FS01:0x22:   Platform: win32 10
06:39:22:WU17:FS01:0x22:       Bits: 64
06:39:22:WU17:FS01:0x22:       Mode: Release
06:39:22:WU17:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
06:39:22:WU17:FS01:0x22:             <peastman@stanford.edu>
06:39:22:WU17:FS01:0x22:       Args: -dir 17 -suffix 01 -version 706 -lifeline 3736 -checkpoint 15
06:39:22:WU17:FS01:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
06:39:22:WU17:FS01:0x22:             nvidia -gpu 0 -gpu-usage 100
06:39:22:WU17:FS01:0x22:************************************ libFAH ************************************
06:39:22:WU17:FS01:0x22:       Date: Sep 7 2020
06:39:22:WU17:FS01:0x22:       Time: 19:09:56
06:39:22:WU17:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
06:39:22:WU17:FS01:0x22:     Branch: HEAD
06:39:22:WU17:FS01:0x22:   Compiler: Visual C++ 2015
06:39:22:WU17:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
06:39:22:WU17:FS01:0x22:   Platform: win32 10
06:39:22:WU17:FS01:0x22:       Bits: 64
06:39:22:WU17:FS01:0x22:       Mode: Release
06:39:22:WU17:FS01:0x22:************************************ CBang *************************************
06:39:22:WU17:FS01:0x22:       Date: Sep 7 2020
06:39:22:WU17:FS01:0x22:       Time: 19:08:30
06:39:22:WU17:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
06:39:22:WU17:FS01:0x22:     Branch: HEAD
06:39:22:WU17:FS01:0x22:   Compiler: Visual C++ 2015
06:39:22:WU17:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
06:39:22:WU17:FS01:0x22:   Platform: win32 10
06:39:22:WU17:FS01:0x22:       Bits: 64
06:39:22:WU17:FS01:0x22:       Mode: Release
06:39:22:WU17:FS01:0x22:************************************ System ************************************
06:39:22:WU17:FS01:0x22:        CPU: Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
06:39:22:WU17:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 63 Stepping 2
06:39:22:WU17:FS01:0x22:       CPUs: 12
06:39:22:WU17:FS01:0x22:     Memory: 31.93GiB
06:39:22:WU17:FS01:0x22:Free Memory: 25.40GiB
06:39:22:WU17:FS01:0x22:    Threads: WINDOWS_THREADS
06:39:22:WU17:FS01:0x22: OS Version: 6.2
06:39:22:WU17:FS01:0x22:Has Battery: false
06:39:22:WU17:FS01:0x22: On Battery: false
06:39:22:WU17:FS01:0x22: UTC Offset: 0
06:39:22:WU17:FS01:0x22:        PID: 6728
06:39:22:WU17:FS01:0x22:        CWD: C:\ProgramData\FAHClient\work
06:39:22:WU17:FS01:0x22:************************************ OpenMM ************************************
06:39:22:WU17:FS01:0x22:   Revision: 189320d0
06:39:22:WU17:FS01:0x22:********************************************************************************
06:39:22:WU17:FS01:0x22:Project: 13438 (Run 3084, Clone 6, Gen 0)
06:39:22:WU17:FS01:0x22:Unit: 0x0000000012bc7d9a000000000c0c0006
06:39:22:WU17:FS01:0x22:Digital signatures verified
06:39:22:WU17:FS01:0x22:Folding@home GPU Core22 Folding@home Core
06:39:22:WU17:FS01:0x22:Version 0.0.13
06:39:22:WU17:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
06:39:22:WU17:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
06:39:22:WU17:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
06:39:22:WU17:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
06:39:22:WU17:FS01:0x22:There are 4 platforms available.
06:39:22:WU17:FS01:0x22:Platform 0: Reference
06:39:22:WU17:FS01:0x22:Platform 1: CPU
06:39:22:WU17:FS01:0x22:Platform 2: OpenCL
06:39:22:WU17:FS01:0x22:  opencl-device 0 specified
06:39:22:WU17:FS01:0x22:Platform 3: CUDA
06:39:22:WU17:FS01:0x22:  cuda-device 0 specified
06:39:35:WU17:FS01:0x22:Attempting to create CUDA context:
06:39:35:WU17:FS01:0x22:  Configuring platform CUDA
06:39:40:WU17:FS01:0x22:  Using CUDA and gpu 0
06:39:41:WU17:FS01:0x22:Completed 750000 out of 1000000 steps (75%)
06:40:08:WU19:FS00:0xa7:Completed 60000 out of 125000 steps (48%)
06:41:37:WU17:FS01:0x22:Completed 760000 out of 1000000 steps (76%)
06:42:25:WU19:FS00:0xa7:Completed 61250 out of 125000 steps (49%)
06:43:34:WU17:FS01:0x22:Completed 770000 out of 1000000 steps (77%)
06:44:42:WU19:FS00:0xa7:Completed 62500 out of 125000 steps (50%)
06:45:31:WU17:FS01:0x22:Completed 780000 out of 1000000 steps (78%)
06:46:58:WU19:FS00:0xa7:Completed 63750 out of 125000 steps (51%)
06:47:28:WU17:FS01:0x22:Completed 790000 out of 1000000 steps (79%)
06:49:14:WU19:FS00:0xa7:Completed 65000 out of 125000 steps (52%)
06:49:25:WU17:FS01:0x22:Completed 800000 out of 1000000 steps (80%)
06:49:25:WU17:FS01:0x22:Checkpoint completed at step 800000
06:50:01:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:17424 run:0 clone:85 gen:157 core:0x22 unit:0x000000bfcedfaa920000000000000055
06:50:01:WU04:FS01:Uploading 17.59MiB to 206.223.170.146
06:50:01:WU04:FS01:Connecting to 206.223.170.146:8080
06:50:31:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
06:51:22:WU17:FS01:0x22:Completed 810000 out of 1000000 steps (81%)
06:51:33:WU19:FS00:0xa7:Completed 66250 out of 125000 steps (53%)
06:51:45:WU12:FS01:Sending unit results: id:12 state:SEND error:NO_ERROR project:13438 run:2404 clone:31 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000964001f
06:51:45:WU12:FS01:Uploading 8.49MiB to 18.188.125.154
06:51:45:WU12:FS01:Connecting to 18.188.125.154:8080
06:52:15:WARNING:WU12:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
06:52:15:WU12:FS01:Trying to send results to collection server
06:52:15:WU12:FS01:Uploading 8.49MiB to 150.136.14.110
06:52:15:WU12:FS01:Connecting to 150.136.14.110:8080
06:52:28:WU07:FS01:Sending unit results: id:07 state:SEND error:NO_ERROR project:17421 run:0 clone:1370 gen:132 core:0x22 unit:0x0000009e80fccb09000000000000055a
06:52:28:WU07:FS01:Uploading 16.75MiB to 128.252.203.9
06:52:28:WU07:FS01:Connecting to 128.252.203.9:8080
06:52:45:ERROR:WU12:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
06:52:58:WARNING:WU07:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
06:52:58:WU07:FS01:Trying to send results to collection server
06:52:58:WU07:FS01:Uploading 16.75MiB to 128.252.203.1
06:52:58:WU07:FS01:Connecting to 128.252.203.1:8080
06:53:04:WU07:FS01:Upload 94.39%
06:53:19:WU17:FS01:0x22:Completed 820000 out of 1000000 steps (82%)
06:53:40:ERROR:WU07:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
06:53:49:WU19:FS00:0xa7:Completed 67500 out of 125000 steps (54%)
06:55:17:WU17:FS01:0x22:Completed 830000 out of 1000000 steps (83%)
06:56:06:WU19:FS00:0xa7:Completed 68750 out of 125000 steps (55%)
06:56:10:WU10:FS01:Sending unit results: id:10 state:SEND error:NO_ERROR project:13438 run:2154 clone:11 gen:1 core:0x22 unit:0x0000000112bc7d9a00000000086a000b
06:56:10:WU10:FS01:Uploading 8.50MiB to 18.188.125.154
06:56:10:WU10:FS01:Connecting to 18.188.125.154:8080
06:56:40:WARNING:WU10:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
06:56:40:WU10:FS01:Trying to send results to collection server
06:56:40:WU10:FS01:Uploading 8.50MiB to 150.136.14.110
06:56:40:WU10:FS01:Connecting to 150.136.14.110:8080
06:57:01:WU18:FS01:Sending unit results: id:18 state:SEND error:NO_ERROR project:13438 run:2768 clone:18 gen:1 core:0x22 unit:0x0000000112bc7d9a000000000ad00012
06:57:01:WU18:FS01:Uploading 8.49MiB to 18.188.125.154
06:57:01:WU18:FS01:Connecting to 18.188.125.154:8080
06:57:10:ERROR:WU10:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
06:57:14:WU17:FS01:0x22:Completed 840000 out of 1000000 steps (84%)
06:57:31:WARNING:WU18:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
06:57:31:WU18:FS01:Trying to send results to collection server
06:57:31:WU18:FS01:Uploading 8.49MiB to 150.136.14.110
06:57:31:WU18:FS01:Connecting to 150.136.14.110:8080
06:58:02:ERROR:WU18:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
06:58:22:WU19:FS00:0xa7:Completed 70000 out of 125000 steps (56%)
06:59:11:WU17:FS01:0x22:Completed 850000 out of 1000000 steps (85%)
06:59:12:WU17:FS01:0x22:Checkpoint completed at step 850000
07:00:38:WU19:FS00:0xa7:Completed 71250 out of 125000 steps (57%)
07:01:09:WU17:FS01:0x22:Completed 860000 out of 1000000 steps (86%)
07:01:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17423 run:0 clone:495 gen:117 core:0xa7 unit:0x0000008280fccb0900000000000001ef
07:01:17:WU00:FS00:Uploading 10.26MiB to 128.252.203.9
07:01:17:WU00:FS00:Connecting to 128.252.203.9:8080
07:01:38:WU06:FS00:Sending unit results: id:06 state:SEND error:NO_ERROR project:13822 run:391 clone:5 gen:58 core:0xa7 unit:0x0000004580fccb090000000001870005
07:01:38:WU06:FS00:Uploading 11.26MiB to 128.252.203.9
07:01:38:WU06:FS00:Connecting to 128.252.203.9:8080
07:01:47:WARNING:WU00:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:01:47:WU00:FS00:Trying to send results to collection server
07:01:47:WU00:FS00:Uploading 10.26MiB to 128.252.203.1
07:01:47:WU00:FS00:Connecting to 128.252.203.1:8080
07:02:09:WARNING:WU06:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:02:09:WU06:FS00:Trying to send results to collection server
07:02:09:WU06:FS00:Uploading 11.26MiB to 128.252.203.1
07:02:09:WU06:FS00:Connecting to 128.252.203.1:8080
07:02:18:ERROR:WU00:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
07:02:40:ERROR:WU06:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
07:02:54:WU19:FS00:0xa7:Completed 72500 out of 125000 steps (58%)
07:03:05:WU17:FS01:0x22:Completed 870000 out of 1000000 steps (87%)
07:05:02:WU17:FS01:0x22:Completed 880000 out of 1000000 steps (88%)
07:05:10:WU19:FS00:0xa7:Completed 73750 out of 125000 steps (59%)
07:05:22:WU03:FS01:Sending unit results: id:03 state:SEND error:NO_ERROR project:14908 run:339 clone:4 gen:227 core:0x22 unit:0x0000011681d59d690000000001530004
07:05:22:WU03:FS01:Uploading 19.79MiB to 129.213.157.105
07:05:22:WU03:FS01:Connecting to 129.213.157.105:8080
07:05:52:WARNING:WU03:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:05:52:WU03:FS01:Trying to send results to collection server
07:05:52:WU03:FS01:Uploading 19.79MiB to 150.136.14.110
07:05:52:WU03:FS01:Connecting to 150.136.14.110:8080
07:06:15:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:17420 run:0 clone:1399 gen:122 core:0x22 unit:0x0000009580fccb090000000000000577
07:06:15:WU02:FS01:Uploading 16.71MiB to 128.252.203.9
07:06:15:WU02:FS01:Connecting to 128.252.203.9:8080
07:06:21:WU02:FS01:Upload 41.88%
07:06:23:ERROR:WU03:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:06:27:WU02:FS01:Upload 50.11%
07:06:33:WU02:FS01:Upload 59.09%
07:06:39:WU02:FS01:Upload 68.44%
07:06:45:WU02:FS01:Upload 76.29%
07:06:51:WU02:FS01:Upload 86.01%
07:06:57:WU02:FS01:Upload 93.86%
07:06:59:WU17:FS01:0x22:Completed 890000 out of 1000000 steps (89%)
07:07:29:WU19:FS00:0xa7:Completed 75000 out of 125000 steps (60%)
07:07:30:WARNING:WU02:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:07:31:WU02:FS01:Trying to send results to collection server
07:07:31:WU02:FS01:Uploading 16.71MiB to 128.252.203.1
07:07:31:WU02:FS01:Connecting to 128.252.203.1:8080
07:07:37:WU02:FS01:Upload 53.10%
07:07:43:WU02:FS01:Upload 62.08%
07:07:49:WU02:FS01:Upload 71.80%
07:07:55:WU02:FS01:Upload 80.03%
07:08:01:WU02:FS01:Upload 89.75%
07:08:07:WU02:FS01:Upload 97.60%
07:08:38:ERROR:WU02:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:08:56:WU17:FS01:0x22:Completed 900000 out of 1000000 steps (90%)
07:08:57:WU17:FS01:0x22:Checkpoint completed at step 900000
07:09:05:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:17421 run:0 clone:581 gen:152 core:0x22 unit:0x000000b780fccb090000000000000245
07:09:05:WU01:FS01:Uploading 16.75MiB to 128.252.203.9
07:09:05:WU01:FS01:Connecting to 128.252.203.9:8080
07:09:11:WU01:FS01:Upload 90.31%
07:09:17:WU01:FS01:Upload 98.90%
07:09:46:WU19:FS00:0xa7:Completed 76250 out of 125000 steps (61%)
07:09:49:WARNING:WU01:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:09:49:WU01:FS01:Trying to send results to collection server
07:09:49:WU01:FS01:Uploading 16.75MiB to 128.252.203.1
07:09:49:WU01:FS01:Connecting to 128.252.203.1:8080
07:09:55:WU01:FS01:Upload 48.52%
07:10:02:WU01:FS01:Upload 57.85%
07:10:08:WU01:FS01:Upload 69.04%
07:10:15:WU01:FS01:Upload 77.63%
07:10:21:WU01:FS01:Upload 86.96%
07:10:27:WU05:FS01:Sending unit results: id:05 state:SEND error:NO_ERROR project:17420 run:0 clone:789 gen:157 core:0x22 unit:0x000000bc80fccb090000000000000315
07:10:27:WU01:FS01:Upload 95.54%
07:10:27:WU05:FS01:Uploading 16.73MiB to 128.252.203.9
07:10:27:WU05:FS01:Connecting to 128.252.203.9:8080
07:10:33:WU05:FS01:Upload 1.12%
07:10:39:WU05:FS01:Upload 13.08%
07:10:45:WU05:FS01:Upload 21.67%
07:10:51:WU05:FS01:Upload 31.76%
07:10:54:WU17:FS01:0x22:Completed 910000 out of 1000000 steps (91%)
07:10:57:WU05:FS01:Upload 39.98%
07:11:01:ERROR:WU01:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:11:03:WU05:FS01:Upload 48.57%
07:11:09:WU05:FS01:Upload 59.04%
07:11:15:WU05:FS01:Upload 68.00%
07:11:21:WU05:FS01:Upload 77.34%
07:11:27:WU05:FS01:Upload 84.82%
07:11:33:WU05:FS01:Upload 95.28%
07:12:03:WU19:FS00:0xa7:Completed 77500 out of 125000 steps (62%)
07:12:06:WARNING:WU05:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:12:06:WU05:FS01:Trying to send results to collection server
07:12:06:WU05:FS01:Uploading 16.73MiB to 128.252.203.1
07:12:06:WU05:FS01:Connecting to 128.252.203.1:8080
07:12:12:WU05:FS01:Upload 54.55%
07:12:18:WU05:FS01:Upload 63.89%
07:12:24:WU05:FS01:Upload 72.49%
07:12:30:WU05:FS01:Upload 81.45%
07:12:36:WU05:FS01:Upload 91.92%
07:12:42:WU05:FS01:Upload 100.00%
07:12:45:WU09:FS01:Sending unit results: id:09 state:SEND error:NO_ERROR project:13438 run:2069 clone:43 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000815002b
07:12:45:WU09:FS01:Uploading 8.50MiB to 18.188.125.154
07:12:45:WU09:FS01:Connecting to 18.188.125.154:8080
07:12:51:WU17:FS01:0x22:Completed 920000 out of 1000000 steps (92%)
07:12:51:WU09:FS01:Upload 19.86%
07:12:57:WU09:FS01:Upload 43.40%
07:13:03:WU09:FS01:Upload 61.06%
07:13:09:WU09:FS01:Upload 78.72%
07:13:12:ERROR:WU05:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:13:15:WU09:FS01:Upload 97.11%
07:13:45:WARNING:WU09:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:13:45:WU09:FS01:Trying to send results to collection server
07:13:45:WU09:FS01:Uploading 8.50MiB to 150.136.14.110
07:13:45:WU09:FS01:Connecting to 150.136.14.110:8080
07:13:51:WU09:FS01:Upload 92.69%
07:14:19:WU19:FS00:0xa7:Completed 78750 out of 125000 steps (63%)
07:14:23:ERROR:WU09:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:14:48:WU17:FS01:0x22:Completed 930000 out of 1000000 steps (93%)
07:16:04:WU08:FS01:Sending unit results: id:08 state:SEND error:NO_ERROR project:14905 run:195 clone:0 gen:294 core:0x22 unit:0x0000017d81d59d690000000000c30000
07:16:04:WU08:FS01:Uploading 11.29MiB to 129.213.157.105
07:16:04:WU08:FS01:Connecting to 129.213.157.105:8080
07:16:34:WU19:FS00:0xa7:Completed 80000 out of 125000 steps (64%)
07:16:35:WARNING:WU08:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:16:35:WU08:FS01:Trying to send results to collection server
07:16:35:WU08:FS01:Uploading 11.29MiB to 129.213.40.229
07:16:35:WU08:FS01:Connecting to 129.213.40.229:8080
07:16:45:WU17:FS01:0x22:Completed 940000 out of 1000000 steps (94%)
07:17:05:ERROR:WU08:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:17:58:WU15:FS01:Sending unit results: id:15 state:SEND error:NO_ERROR project:17424 run:0 clone:1061 gen:106 core:0x22 unit:0x0000007ccedfaa920000000000000425
07:17:58:WU15:FS01:Uploading 17.59MiB to 206.223.170.146
07:17:58:WU15:FS01:Connecting to 206.223.170.146:8080
07:18:29:WARNING:WU15:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:18:42:WU17:FS01:0x22:Completed 950000 out of 1000000 steps (95%)
07:18:43:WU17:FS01:0x22:Checkpoint completed at step 950000
07:18:51:WU19:FS00:0xa7:Completed 81250 out of 125000 steps (65%)
07:19:40:WU13:FS01:Sending unit results: id:13 state:SEND error:NO_ERROR project:14905 run:324 clone:5 gen:205 core:0x22 unit:0x0000011581d59d690000000001440005
07:19:40:WU13:FS01:Uploading 11.29MiB to 129.213.157.105
07:19:40:WU13:FS01:Connecting to 129.213.157.105:8080
07:20:10:WARNING:WU13:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:20:10:WU13:FS01:Trying to send results to collection server
07:20:10:WU13:FS01:Uploading 11.29MiB to 150.136.14.110
07:20:10:WU13:FS01:Connecting to 150.136.14.110:8080
07:20:40:WU17:FS01:0x22:Completed 960000 out of 1000000 steps (96%)
07:20:40:ERROR:WU13:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:21:10:WU19:FS00:0xa7:Completed 82500 out of 125000 steps (66%)
07:22:37:WU17:FS01:0x22:Completed 970000 out of 1000000 steps (97%)
07:22:41:WU11:FS00:Sending unit results: id:11 state:SEND error:NO_ERROR project:16812 run:8 clone:690 gen:81 core:0xa8 unit:0x0000005eb2aec48a00000000000802b2
07:22:41:WU11:FS00:Uploading 8.34MiB to 178.174.196.138
07:22:41:WU11:FS00:Connecting to 178.174.196.138:8080
07:23:11:WARNING:WU11:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:23:11:WU11:FS00:Trying to send results to collection server
07:23:12:WU11:FS00:Uploading 8.34MiB to 129.213.157.105
07:23:12:WU11:FS00:Connecting to 129.213.157.105:8080
07:23:27:WU19:FS00:0xa7:Completed 83750 out of 125000 steps (67%)
07:23:42:ERROR:WU11:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
07:24:34:WU17:FS01:0x22:Completed 980000 out of 1000000 steps (98%)
07:25:44:WU19:FS00:0xa7:Completed 85000 out of 125000 steps (68%)
07:26:31:WU17:FS01:0x22:Completed 990000 out of 1000000 steps (99%)
07:26:31:WU16:FS01:Connecting to assign1.foldingathome.org:80
07:26:31:WU16:FS01:Assigned to work server 18.188.125.154
07:26:31:WU16:FS01:Requesting new work unit for slot 01: gpu:3:0 GP104 [GeForce GTX 1080] 8873 from 18.188.125.154
07:26:31:WU16:FS01:Connecting to 18.188.125.154:8080
07:26:34:WU16:FS01:Downloading 6.71MiB
07:26:38:WU16:FS01:Download complete
07:26:38:WU16:FS01:Received Unit: id:16 state:DOWNLOAD error:NO_ERROR project:13438 run:3202 clone:47 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c82002f
07:28:00:WU19:FS00:0xa7:Completed 86250 out of 125000 steps (69%)
07:28:27:WU17:FS01:0x22:Completed 1000000 out of 1000000 steps (100%)
07:28:27:WU17:FS01:0x22:Average performance: 73.9726 ns/day
07:28:28:WU17:FS01:0x22:Checkpoint completed at step 1000000
07:28:38:WU17:FS01:0x22:Saving result file ..\logfile_01.txt
07:28:38:WU17:FS01:0x22:Saving result file checkpointIntegrator.xml.bz2
07:28:38:WU17:FS01:0x22:Saving result file checkpointState.xml.bz2
07:28:39:WU17:FS01:0x22:Saving result file globals.csv
07:28:39:WU17:FS01:0x22:Saving result file positions.xtc
07:28:39:WU17:FS01:0x22:Saving result file science.log
07:28:39:WU17:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
07:28:39:WU17:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
07:28:39:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
07:28:39:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
07:28:39:WU17:FS01:Connecting to 18.188.125.154:8080
07:28:39:WU16:FS01:Starting
07:28:39:WU16:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 16 -suffix 01 -version 706 -lifeline 16580 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
07:28:39:WU16:FS01:Started FahCore on PID 8200
07:28:39:WU16:FS01:Core PID:15312
07:28:39:WU16:FS01:FahCore 0x22 started
07:28:40:WU16:FS01:0x22:*********************** Log Started 2020-12-11T07:28:39Z ***********************
07:28:40:WU16:FS01:0x22:*************************** Core22 Folding@home Core ***************************
07:28:40:WU16:FS01:0x22:       Core: Core22
07:28:40:WU16:FS01:0x22:       Type: 0x22
07:28:40:WU16:FS01:0x22:    Version: 0.0.13
07:28:40:WU16:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
07:28:40:WU16:FS01:0x22:  Copyright: 2020 foldingathome.org
07:28:40:WU16:FS01:0x22:   Homepage: https://foldingathome.org/
07:28:40:WU16:FS01:0x22:       Date: Sep 19 2020
07:28:40:WU16:FS01:0x22:       Time: 02:35:58
07:28:40:WU16:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
07:28:40:WU16:FS01:0x22:     Branch: core22-0.0.13
07:28:40:WU16:FS01:0x22:   Compiler: Visual C++ 2015
07:28:40:WU16:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
07:28:40:WU16:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
07:28:40:WU16:FS01:0x22:   Platform: win32 10
07:28:40:WU16:FS01:0x22:       Bits: 64
07:28:40:WU16:FS01:0x22:       Mode: Release
07:28:40:WU16:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
07:28:40:WU16:FS01:0x22:             <peastman@stanford.edu>
07:28:40:WU16:FS01:0x22:       Args: -dir 16 -suffix 01 -version 706 -lifeline 8200 -checkpoint 15
07:28:40:WU16:FS01:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
07:28:40:WU16:FS01:0x22:             nvidia -gpu 0 -gpu-usage 100
07:28:40:WU16:FS01:0x22:************************************ libFAH ************************************
07:28:40:WU16:FS01:0x22:       Date: Sep 7 2020
07:28:40:WU16:FS01:0x22:       Time: 19:09:56
07:28:40:WU16:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
07:28:40:WU16:FS01:0x22:     Branch: HEAD
07:28:40:WU16:FS01:0x22:   Compiler: Visual C++ 2015
07:28:40:WU16:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
07:28:40:WU16:FS01:0x22:   Platform: win32 10
07:28:40:WU16:FS01:0x22:       Bits: 64
07:28:40:WU16:FS01:0x22:       Mode: Release
07:28:40:WU16:FS01:0x22:************************************ CBang *************************************
07:28:40:WU16:FS01:0x22:       Date: Sep 7 2020
07:28:40:WU16:FS01:0x22:       Time: 19:08:30
07:28:40:WU16:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
07:28:40:WU16:FS01:0x22:     Branch: HEAD
07:28:40:WU16:FS01:0x22:   Compiler: Visual C++ 2015
07:28:40:WU16:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
07:28:40:WU16:FS01:0x22:   Platform: win32 10
07:28:40:WU16:FS01:0x22:       Bits: 64
07:28:40:WU16:FS01:0x22:       Mode: Release
07:28:40:WU16:FS01:0x22:************************************ System ************************************
07:28:40:WU16:FS01:0x22:        CPU: Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
07:28:40:WU16:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 63 Stepping 2
07:28:40:WU16:FS01:0x22:       CPUs: 12
07:28:40:WU16:FS01:0x22:     Memory: 31.93GiB
07:28:40:WU16:FS01:0x22:Free Memory: 25.50GiB
07:28:40:WU16:FS01:0x22:    Threads: WINDOWS_THREADS
07:28:40:WU16:FS01:0x22: OS Version: 6.2
07:28:40:WU16:FS01:0x22:Has Battery: false
07:28:40:WU16:FS01:0x22: On Battery: false
07:28:40:WU16:FS01:0x22: UTC Offset: 0
07:28:40:WU16:FS01:0x22:        PID: 15312
07:28:40:WU16:FS01:0x22:        CWD: C:\ProgramData\FAHClient\work
07:28:40:WU16:FS01:0x22:************************************ OpenMM ************************************
07:28:40:WU16:FS01:0x22:   Revision: 189320d0
07:28:40:WU16:FS01:0x22:********************************************************************************
07:28:40:WU16:FS01:0x22:Project: 13438 (Run 3202, Clone 47, Gen 0)
07:28:40:WU16:FS01:0x22:Unit: 0x0000000012bc7d9a000000000c82002f
07:28:40:WU16:FS01:0x22:Reading tar file core.xml
07:28:40:WU16:FS01:0x22:Reading tar file integrator.xml.bz2
07:28:40:WU16:FS01:0x22:Reading tar file state.xml.bz2
07:28:40:WU16:FS01:0x22:Reading tar file system.xml.bz2
07:28:40:WU16:FS01:0x22:Digital signatures verified
07:28:40:WU16:FS01:0x22:Folding@home GPU Core22 Folding@home Core
07:28:40:WU16:FS01:0x22:Version 0.0.13
07:28:40:WU16:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
07:28:40:WU16:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
07:28:40:WU16:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
07:28:40:WU16:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
07:28:40:WU16:FS01:0x22:There are 4 platforms available.
07:28:40:WU16:FS01:0x22:Platform 0: Reference
07:28:40:WU16:FS01:0x22:Platform 1: CPU
07:28:40:WU16:FS01:0x22:Platform 2: OpenCL
07:28:40:WU16:FS01:0x22:  opencl-device 0 specified
07:28:40:WU16:FS01:0x22:Platform 3: CUDA
07:28:40:WU16:FS01:0x22:  cuda-device 0 specified
07:28:53:WU16:FS01:0x22:Attempting to create CUDA context:
07:28:53:WU16:FS01:0x22:  Configuring platform CUDA
07:28:59:WU16:FS01:0x22:  Using CUDA and gpu 0
07:28:59:WU16:FS01:0x22:Completed 0 out of 1000000 steps (0%)
07:29:00:WU16:FS01:0x22:Checkpoint completed at step 0
07:29:09:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:29:09:WU17:FS01:Trying to send results to collection server
07:29:09:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
07:29:09:WU17:FS01:Connecting to 150.136.14.110:8080
07:29:40:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:29:40:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
07:29:40:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
07:29:40:WU17:FS01:Connecting to 18.188.125.154:8080
07:30:10:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:30:10:WU17:FS01:Trying to send results to collection server
07:30:10:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
07:30:10:WU17:FS01:Connecting to 150.136.14.110:8080
07:30:17:WU19:FS00:0xa7:Completed 87500 out of 125000 steps (70%)
07:30:41:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:30:41:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
07:30:41:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
07:30:41:WU17:FS01:Connecting to 18.188.125.154:8080
07:30:53:WU16:FS01:0x22:Completed 10000 out of 1000000 steps (1%)
07:31:11:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:31:11:WU17:FS01:Trying to send results to collection server
07:31:11:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
07:31:11:WU17:FS01:Connecting to 150.136.14.110:8080
07:31:38:WU14:FS01:Sending unit results: id:14 state:SEND error:NO_ERROR project:13438 run:2630 clone:24 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000a460018
07:31:38:WU14:FS01:Uploading 8.50MiB to 18.188.125.154
07:31:38:WU14:FS01:Connecting to 18.188.125.154:8080
07:31:42:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:32:09:WARNING:WU14:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:32:09:WU14:FS01:Trying to send results to collection server
07:32:09:WU14:FS01:Uploading 8.50MiB to 150.136.14.110
07:32:09:WU14:FS01:Connecting to 150.136.14.110:8080
07:32:18:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
07:32:18:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
07:32:18:WU17:FS01:Connecting to 18.188.125.154:8080
07:32:35:WU19:FS00:0xa7:Completed 88750 out of 125000 steps (71%)
07:32:39:ERROR:WU14:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:32:46:WU16:FS01:0x22:Completed 20000 out of 1000000 steps (2%)
07:32:49:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:32:49:WU17:FS01:Trying to send results to collection server
07:32:49:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
07:32:49:WU17:FS01:Connecting to 150.136.14.110:8080
07:33:20:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:34:40:WU16:FS01:0x22:Completed 30000 out of 1000000 steps (3%)
07:34:52:WU19:FS00:0xa7:Completed 90000 out of 125000 steps (72%)
07:34:55:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
07:34:55:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
07:34:55:WU17:FS01:Connecting to 18.188.125.154:8080
07:35:26:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:35:26:WU17:FS01:Trying to send results to collection server
07:35:26:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
07:35:26:WU17:FS01:Connecting to 150.136.14.110:8080
07:35:56:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:36:34:WU16:FS01:0x22:Completed 40000 out of 1000000 steps (4%)
07:37:11:WU19:FS00:0xa7:Completed 91250 out of 125000 steps (73%)
07:38:27:WU16:FS01:0x22:Completed 50000 out of 1000000 steps (5%)
07:38:28:WU16:FS01:0x22:Checkpoint completed at step 50000
07:39:10:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
07:39:10:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
07:39:10:WU17:FS01:Connecting to 18.188.125.154:8080
07:39:28:WU19:FS00:0xa7:Completed 92500 out of 125000 steps (74%)
07:39:40:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:39:40:WU17:FS01:Trying to send results to collection server
07:39:40:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
07:39:40:WU17:FS01:Connecting to 150.136.14.110:8080
07:40:10:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:40:22:WU16:FS01:0x22:Completed 60000 out of 1000000 steps (6%)
07:41:46:WU19:FS00:0xa7:Completed 93750 out of 125000 steps (75%)
07:42:15:WU16:FS01:0x22:Completed 70000 out of 1000000 steps (7%)
07:44:04:WU19:FS00:0xa7:Completed 95000 out of 125000 steps (76%)
07:44:09:WU16:FS01:0x22:Completed 80000 out of 1000000 steps (8%)
******************************* Date: 2020-12-11 *******************************
07:46:01:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
07:46:01:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
07:46:01:WU17:FS01:Connecting to 18.188.125.154:8080
07:46:03:WU16:FS01:0x22:Completed 90000 out of 1000000 steps (9%)
07:46:22:WU19:FS00:0xa7:Completed 96250 out of 125000 steps (77%)
07:46:31:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:46:31:WU17:FS01:Trying to send results to collection server
07:46:31:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
07:46:31:WU17:FS01:Connecting to 150.136.14.110:8080
07:47:02:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:47:56:WU16:FS01:0x22:Completed 100000 out of 1000000 steps (10%)
07:47:57:WU16:FS01:0x22:Checkpoint completed at step 100000
07:48:39:WU19:FS00:0xa7:Completed 97500 out of 125000 steps (78%)
07:49:51:WU16:FS01:0x22:Completed 110000 out of 1000000 steps (11%)
07:50:01:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:17424 run:0 clone:85 gen:157 core:0x22 unit:0x000000bfcedfaa920000000000000055
07:50:01:WU04:FS01:Uploading 17.59MiB to 206.223.170.146
07:50:01:WU04:FS01:Connecting to 206.223.170.146:8080
07:50:32:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:50:59:WU19:FS00:0xa7:Completed 98750 out of 125000 steps (79%)
07:51:45:WU12:FS01:Sending unit results: id:12 state:SEND error:NO_ERROR project:13438 run:2404 clone:31 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000964001f
07:51:45:WU12:FS01:Uploading 8.49MiB to 18.188.125.154
07:51:45:WU12:FS01:Connecting to 18.188.125.154:8080
07:51:45:WU16:FS01:0x22:Completed 120000 out of 1000000 steps (12%)
07:52:15:WARNING:WU12:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:52:15:WU12:FS01:Trying to send results to collection server
07:52:15:WU12:FS01:Uploading 8.49MiB to 150.136.14.110
07:52:15:WU12:FS01:Connecting to 150.136.14.110:8080
07:52:28:WU07:FS01:Sending unit results: id:07 state:SEND error:NO_ERROR project:17421 run:0 clone:1370 gen:132 core:0x22 unit:0x0000009e80fccb09000000000000055a
07:52:28:WU07:FS01:Uploading 16.75MiB to 128.252.203.9
07:52:28:WU07:FS01:Connecting to 128.252.203.9:8080
07:52:46:ERROR:WU12:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:52:59:WARNING:WU07:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:52:59:WU07:FS01:Trying to send results to collection server
07:52:59:WU07:FS01:Uploading 16.75MiB to 128.252.203.1
07:52:59:WU07:FS01:Connecting to 128.252.203.1:8080
07:53:16:WU19:FS00:0xa7:Completed 100000 out of 125000 steps (80%)
07:53:30:ERROR:WU07:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:53:39:WU16:FS01:0x22:Completed 130000 out of 1000000 steps (13%)
07:55:33:WU16:FS01:0x22:Completed 140000 out of 1000000 steps (14%)
07:55:33:WU19:FS00:0xa7:Completed 101250 out of 125000 steps (81%)
07:56:10:WU10:FS01:Sending unit results: id:10 state:SEND error:NO_ERROR project:13438 run:2154 clone:11 gen:1 core:0x22 unit:0x0000000112bc7d9a00000000086a000b
07:56:10:WU10:FS01:Uploading 8.50MiB to 18.188.125.154
07:56:10:WU10:FS01:Connecting to 18.188.125.154:8080
07:56:40:WARNING:WU10:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:56:40:WU10:FS01:Trying to send results to collection server
07:56:40:WU10:FS01:Uploading 8.50MiB to 150.136.14.110
07:56:40:WU10:FS01:Connecting to 150.136.14.110:8080
07:57:01:WU18:FS01:Sending unit results: id:18 state:SEND error:NO_ERROR project:13438 run:2768 clone:18 gen:1 core:0x22 unit:0x0000000112bc7d9a000000000ad00012
07:57:01:WU18:FS01:Uploading 8.49MiB to 18.188.125.154
07:57:01:WU18:FS01:Connecting to 18.188.125.154:8080
07:57:07:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
07:57:07:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
07:57:07:WU17:FS01:Connecting to 18.188.125.154:8080
07:57:10:ERROR:WU10:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:57:27:WU16:FS01:0x22:Completed 150000 out of 1000000 steps (15%)
07:57:28:WU16:FS01:0x22:Checkpoint completed at step 150000
07:57:32:WARNING:WU18:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:57:32:WU18:FS01:Trying to send results to collection server
07:57:32:WU18:FS01:Uploading 8.49MiB to 150.136.14.110
07:57:32:WU18:FS01:Connecting to 150.136.14.110:8080
07:57:37:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
07:57:37:WU17:FS01:Trying to send results to collection server
07:57:37:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
07:57:37:WU17:FS01:Connecting to 150.136.14.110:8080
07:57:49:WU19:FS00:0xa7:Completed 102500 out of 125000 steps (82%)
07:58:02:ERROR:WU18:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:58:08:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
07:59:22:WU16:FS01:0x22:Completed 160000 out of 1000000 steps (16%)
08:00:05:WU19:FS00:0xa7:Completed 103750 out of 125000 steps (83%)
08:01:16:WU16:FS01:0x22:Completed 170000 out of 1000000 steps (17%)
08:01:17:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17423 run:0 clone:495 gen:117 core:0xa7 unit:0x0000008280fccb0900000000000001ef
08:01:17:WU00:FS00:Uploading 10.26MiB to 128.252.203.9
08:01:17:WU00:FS00:Connecting to 128.252.203.9:8080
08:01:38:WU06:FS00:Sending unit results: id:06 state:SEND error:NO_ERROR project:13822 run:391 clone:5 gen:58 core:0xa7 unit:0x0000004580fccb090000000001870005
08:01:38:WU06:FS00:Uploading 11.26MiB to 128.252.203.9
08:01:38:WU06:FS00:Connecting to 128.252.203.9:8080
08:01:48:WARNING:WU00:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:01:48:WU00:FS00:Trying to send results to collection server
08:01:48:WU00:FS00:Uploading 10.26MiB to 128.252.203.1
08:01:48:WU00:FS00:Connecting to 128.252.203.1:8080
08:02:09:WARNING:WU06:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:02:09:WU06:FS00:Trying to send results to collection server
08:02:09:WU06:FS00:Uploading 11.26MiB to 128.252.203.1
08:02:09:WU06:FS00:Connecting to 128.252.203.1:8080
08:02:18:ERROR:WU00:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
08:02:22:WU19:FS00:0xa7:Completed 105000 out of 125000 steps (84%)
08:02:40:ERROR:WU06:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
08:03:10:WU16:FS01:0x22:Completed 180000 out of 1000000 steps (18%)
08:04:39:WU19:FS00:0xa7:Completed 106250 out of 125000 steps (85%)
08:05:04:WU16:FS01:0x22:Completed 190000 out of 1000000 steps (19%)
08:05:22:WU03:FS01:Sending unit results: id:03 state:SEND error:NO_ERROR project:14908 run:339 clone:4 gen:227 core:0x22 unit:0x0000011681d59d690000000001530004
08:05:22:WU03:FS01:Uploading 19.79MiB to 129.213.157.105
08:05:22:WU03:FS01:Connecting to 129.213.157.105:8080
08:05:53:WARNING:WU03:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:05:53:WU03:FS01:Trying to send results to collection server
08:05:53:WU03:FS01:Uploading 19.79MiB to 150.136.14.110
08:05:53:WU03:FS01:Connecting to 150.136.14.110:8080
08:06:15:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:17420 run:0 clone:1399 gen:122 core:0x22 unit:0x0000009580fccb090000000000000577
08:06:15:WU02:FS01:Uploading 16.71MiB to 128.252.203.9
08:06:15:WU02:FS01:Connecting to 128.252.203.9:8080
08:06:21:WU02:FS01:Upload 42.63%
08:06:23:ERROR:WU03:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:06:27:WU02:FS01:Upload 50.48%
08:06:33:WU02:FS01:Upload 59.83%
08:06:39:WU02:FS01:Upload 69.18%
08:06:45:WU02:FS01:Upload 77.04%
08:06:51:WU02:FS01:Upload 87.13%
08:06:57:WU02:FS01:Upload 94.24%
08:06:58:WU19:FS00:0xa7:Completed 107500 out of 125000 steps (86%)
08:06:58:WU16:FS01:0x22:Completed 200000 out of 1000000 steps (20%)
08:06:59:WU16:FS01:0x22:Checkpoint completed at step 200000
08:07:30:WARNING:WU02:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:07:30:WU02:FS01:Trying to send results to collection server
08:07:30:WU02:FS01:Uploading 16.71MiB to 128.252.203.1
08:07:30:WU02:FS01:Connecting to 128.252.203.1:8080
08:07:36:WU02:FS01:Upload 52.73%
08:07:42:WU02:FS01:Upload 60.21%
08:07:48:WU02:FS01:Upload 71.05%
08:07:54:WU02:FS01:Upload 77.78%
08:08:00:WU02:FS01:Upload 87.88%
08:08:06:WU02:FS01:Upload 97.60%
08:08:38:ERROR:WU02:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:08:53:WU16:FS01:0x22:Completed 210000 out of 1000000 steps (21%)
08:09:05:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:17421 run:0 clone:581 gen:152 core:0x22 unit:0x000000b780fccb090000000000000245
08:09:05:WU01:FS01:Uploading 16.75MiB to 128.252.203.9
08:09:05:WU01:FS01:Connecting to 128.252.203.9:8080
08:09:11:WU01:FS01:Upload 93.67%
08:09:14:WU19:FS00:0xa7:Completed 108750 out of 125000 steps (87%)
08:09:45:WARNING:WU01:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:09:45:WU01:FS01:Trying to send results to collection server
08:09:45:WU01:FS01:Uploading 16.75MiB to 128.252.203.1
08:09:45:WU01:FS01:Connecting to 128.252.203.1:8080
08:09:51:WU01:FS01:Upload 51.87%
08:09:58:WU01:FS01:Upload 63.44%
08:10:04:WU01:FS01:Upload 72.77%
08:10:10:WU01:FS01:Upload 82.10%
08:10:16:WU01:FS01:Upload 91.06%
08:10:27:WU05:FS01:Sending unit results: id:05 state:SEND error:NO_ERROR project:17420 run:0 clone:789 gen:157 core:0x22 unit:0x000000bc80fccb090000000000000315
08:10:27:WU05:FS01:Uploading 16.73MiB to 128.252.203.9
08:10:27:WU05:FS01:Connecting to 128.252.203.9:8080
08:10:33:WU05:FS01:Upload 15.69%
08:10:39:WU05:FS01:Upload 26.16%
08:10:45:WU05:FS01:Upload 34.75%
08:10:48:WU16:FS01:0x22:Completed 220000 out of 1000000 steps (22%)
08:10:51:WU05:FS01:Upload 45.21%
08:10:52:ERROR:WU01:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:10:57:WU05:FS01:Upload 53.43%
08:11:03:WU05:FS01:Upload 62.77%
08:11:09:WU05:FS01:Upload 72.86%
08:11:15:WU05:FS01:Upload 79.96%
08:11:21:WU05:FS01:Upload 89.68%
08:11:28:WU05:FS01:Upload 97.90%
08:11:31:WU19:FS00:0xa7:Completed 110000 out of 125000 steps (88%)
08:11:59:WARNING:WU05:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:11:59:WU05:FS01:Trying to send results to collection server
08:11:59:WU05:FS01:Uploading 16.73MiB to 128.252.203.1
08:11:59:WU05:FS01:Connecting to 128.252.203.1:8080
08:12:05:WU05:FS01:Upload 56.79%
08:12:11:WU05:FS01:Upload 63.15%
08:12:17:WU05:FS01:Upload 72.86%
08:12:23:WU05:FS01:Upload 81.83%
08:12:29:WU05:FS01:Upload 91.17%
08:12:42:WU16:FS01:0x22:Completed 230000 out of 1000000 steps (23%)
08:12:46:WU09:FS01:Sending unit results: id:09 state:SEND error:NO_ERROR project:13438 run:2069 clone:43 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000815002b
08:12:46:WU09:FS01:Uploading 8.50MiB to 18.188.125.154
08:12:46:WU09:FS01:Connecting to 18.188.125.154:8080
08:12:53:WU09:FS01:Upload 43.40%
08:12:59:WU09:FS01:Upload 66.21%
08:13:04:ERROR:WU05:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:13:05:WU09:FS01:Upload 86.07%
08:13:40:WARNING:WU09:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:13:40:WU09:FS01:Trying to send results to collection server
08:13:40:WU09:FS01:Uploading 8.50MiB to 150.136.14.110
08:13:40:WU09:FS01:Connecting to 150.136.14.110:8080
08:13:48:WU19:FS00:0xa7:Completed 111250 out of 125000 steps (89%)
08:14:15:ERROR:WU09:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:14:36:WU16:FS01:0x22:Completed 240000 out of 1000000 steps (24%)
08:15:03:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
08:15:03:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
08:15:03:WU17:FS01:Connecting to 18.188.125.154:8080
08:15:34:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:15:34:WU17:FS01:Trying to send results to collection server
08:15:34:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
08:15:34:WU17:FS01:Connecting to 150.136.14.110:8080
08:16:04:WU08:FS01:Sending unit results: id:08 state:SEND error:NO_ERROR project:14905 run:195 clone:0 gen:294 core:0x22 unit:0x0000017d81d59d690000000000c30000
08:16:04:WU08:FS01:Uploading 11.29MiB to 129.213.157.105
08:16:04:WU08:FS01:Connecting to 129.213.157.105:8080
08:16:05:WU19:FS00:0xa7:Completed 112500 out of 125000 steps (90%)
08:16:05:ERROR:WU17:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:16:30:WU16:FS01:0x22:Completed 250000 out of 1000000 steps (25%)
08:16:31:WU16:FS01:0x22:Checkpoint completed at step 250000
08:16:35:WARNING:WU08:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:16:35:WU08:FS01:Trying to send results to collection server
08:16:35:WU08:FS01:Uploading 11.29MiB to 129.213.40.229
08:16:35:WU08:FS01:Connecting to 129.213.40.229:8080
08:17:06:ERROR:WU08:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:17:58:WU15:FS01:Sending unit results: id:15 state:SEND error:NO_ERROR project:17424 run:0 clone:1061 gen:106 core:0x22 unit:0x0000007ccedfaa920000000000000425
08:17:58:WU15:FS01:Uploading 17.59MiB to 206.223.170.146
08:17:58:WU15:FS01:Connecting to 206.223.170.146:8080
08:18:22:WU19:FS00:0xa7:Completed 113750 out of 125000 steps (91%)
08:18:28:WU16:FS01:0x22:Completed 260000 out of 1000000 steps (26%)
08:18:29:WARNING:WU15:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:19:40:WU13:FS01:Sending unit results: id:13 state:SEND error:NO_ERROR project:14905 run:324 clone:5 gen:205 core:0x22 unit:0x0000011581d59d690000000001440005
08:19:40:WU13:FS01:Uploading 11.29MiB to 129.213.157.105
08:19:40:WU13:FS01:Connecting to 129.213.157.105:8080
08:20:10:WARNING:WU13:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:20:10:WU13:FS01:Trying to send results to collection server
08:20:10:WU13:FS01:Uploading 11.29MiB to 150.136.14.110
08:20:10:WU13:FS01:Connecting to 150.136.14.110:8080
08:20:24:WU16:FS01:0x22:Completed 270000 out of 1000000 steps (27%)
08:20:40:WU19:FS00:0xa7:Completed 115000 out of 125000 steps (92%)
08:20:41:ERROR:WU13:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:22:20:WU16:FS01:0x22:Completed 280000 out of 1000000 steps (28%)
08:22:41:WU11:FS00:Sending unit results: id:11 state:SEND error:NO_ERROR project:16812 run:8 clone:690 gen:81 core:0xa8 unit:0x0000005eb2aec48a00000000000802b2
08:22:41:WU11:FS00:Uploading 8.34MiB to 178.174.196.138
08:22:41:WU11:FS00:Connecting to 178.174.196.138:8080
08:22:56:WU19:FS00:0xa7:Completed 116250 out of 125000 steps (93%)
08:23:12:WARNING:WU11:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:23:12:WU11:FS00:Trying to send results to collection server
08:23:12:WU11:FS00:Uploading 8.34MiB to 129.213.157.105
08:23:12:WU11:FS00:Connecting to 129.213.157.105:8080
08:23:42:ERROR:WU11:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
08:24:17:WU16:FS01:0x22:Completed 290000 out of 1000000 steps (29%)
08:25:13:WU19:FS00:0xa7:Completed 117500 out of 125000 steps (94%)
08:26:13:WU16:FS01:0x22:Completed 300000 out of 1000000 steps (30%)
08:26:14:WU16:FS01:0x22:Checkpoint completed at step 300000
08:27:29:WU19:FS00:0xa7:Completed 118750 out of 125000 steps (95%)
08:28:11:WU16:FS01:0x22:Completed 310000 out of 1000000 steps (31%)
08:29:46:WU19:FS00:0xa7:Completed 120000 out of 125000 steps (96%)
08:30:07:WU16:FS01:0x22:Completed 320000 out of 1000000 steps (32%)
08:31:38:WU14:FS01:Sending unit results: id:14 state:SEND error:NO_ERROR project:13438 run:2630 clone:24 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000a460018
08:31:38:WU14:FS01:Uploading 8.50MiB to 18.188.125.154
08:31:38:WU14:FS01:Connecting to 18.188.125.154:8080
08:32:03:WU19:FS00:0xa7:Completed 121250 out of 125000 steps (97%)
08:32:04:WU16:FS01:0x22:Completed 330000 out of 1000000 steps (33%)
08:32:08:WARNING:WU14:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:32:08:WU14:FS01:Trying to send results to collection server
08:32:08:WU14:FS01:Uploading 8.50MiB to 150.136.14.110
08:32:08:WU14:FS01:Connecting to 150.136.14.110:8080
08:32:39:ERROR:WU14:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
08:33:30:FS01:Shutting core down
08:33:31:WU16:FS01:0x22:WARNING:Console control signal 1 on PID 15312
08:33:31:WU16:FS01:0x22:Exiting, please wait. . .
08:33:31:WU16:FS01:0x22:Folding@home Core Shutdown: INTERRUPTED
08:33:31:WU16:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
08:34:19:WU19:FS00:0xa7:Completed 122500 out of 125000 steps (98%)
08:36:27:WU19:FS00:0xa7:Completed 123750 out of 125000 steps (99%)
08:36:28:WU20:FS00:Connecting to assign1.foldingathome.org:80
08:36:28:WU20:FS00:Assigned to work server 128.252.203.1
08:36:28:WU20:FS00:Requesting new work unit for slot 00: cpu:6 from 128.252.203.1
08:36:28:WU20:FS00:Connecting to 128.252.203.1:8080
08:36:30:WU20:FS00:Downloading 1.86MiB
08:36:30:WU20:FS00:Download complete
08:36:30:WU20:FS00:Received Unit: id:20 state:DOWNLOAD error:NO_ERROR project:16461 run:69 clone:2 gen:22 core:0xa7 unit:0x0000001680fccb010000000000450002
08:38:29:WU19:FS00:0xa7:Completed 125000 out of 125000 steps (100%)
08:38:35:WU19:FS00:0xa7:Saving result file ..\logfile_01.txt
08:38:35:WU19:FS00:0xa7:Saving result file frame138.trr
08:38:35:WU19:FS00:0xa7:Saving result file frame138.xtc
08:38:35:WU19:FS00:0xa7:Saving result file md.log
08:38:35:WU19:FS00:0xa7:Saving result file science.log
08:38:35:WU19:FS00:0xa7:Folding@home Core Shutdown: FINISHED_UNIT
08:38:36:WU19:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
08:38:36:WU19:FS00:Sending unit results: id:19 state:SEND error:NO_ERROR project:13821 run:510 clone:5 gen:138 core:0xa7 unit:0x0000009780fccb090000000001fe0005
08:38:36:WU19:FS00:Uploading 11.30MiB to 128.252.203.9
08:38:36:WU19:FS00:Connecting to 128.252.203.9:8080
08:38:36:WU20:FS00:Starting
08:38:36:WU20:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit-avx-256/a7-0.0.19/Core_a7.fah/FahCore_a7.exe -dir 20 -suffix 01 -version 706 -lifeline 16580 -checkpoint 15 -np 6
08:38:36:WU20:FS00:Started FahCore on PID 20236
08:38:36:WU20:FS00:Core PID:15668
08:38:36:WU20:FS00:FahCore 0xa7 started
08:38:37:WU20:FS00:0xa7:*********************** Log Started 2020-12-11T08:38:36Z ***********************
08:38:37:WU20:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
08:38:37:WU20:FS00:0xa7:       Type: 0xa7
08:38:37:WU20:FS00:0xa7:       Core: Gromacs
08:38:37:WU20:FS00:0xa7:       Args: -dir 20 -suffix 01 -version 706 -lifeline 20236 -checkpoint 15 -np
08:38:37:WU20:FS00:0xa7:             6
08:38:37:WU20:FS00:0xa7:************************************ CBang *************************************
08:38:37:WU20:FS00:0xa7:       Date: Nov 27 2019
08:38:37:WU20:FS00:0xa7:       Time: 03:40:09
08:38:37:WU20:FS00:0xa7:   Revision: d25803215b59272441049dfa05a0a9bf7a6e3c48
08:38:37:WU20:FS00:0xa7:     Branch: master
08:38:37:WU20:FS00:0xa7:   Compiler: Visual C++ 2008
08:38:37:WU20:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
08:38:37:WU20:FS00:0xa7:   Platform: win32 10
08:38:37:WU20:FS00:0xa7:       Bits: 64
08:38:37:WU20:FS00:0xa7:       Mode: Release
08:38:37:WU20:FS00:0xa7:************************************ System ************************************
08:38:37:WU20:FS00:0xa7:        CPU: Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
08:38:37:WU20:FS00:0xa7:     CPU ID: GenuineIntel Family 6 Model 63 Stepping 2
08:38:37:WU20:FS00:0xa7:       CPUs: 12
08:38:37:WU20:FS00:0xa7:     Memory: 31.93GiB
08:38:37:WU20:FS00:0xa7:Free Memory: 24.50GiB
08:38:37:WU20:FS00:0xa7:    Threads: WINDOWS_THREADS
08:38:37:WU20:FS00:0xa7: OS Version: 6.2
08:38:37:WU20:FS00:0xa7:Has Battery: false
08:38:37:WU20:FS00:0xa7: On Battery: false
08:38:37:WU20:FS00:0xa7: UTC Offset: 0
08:38:37:WU20:FS00:0xa7:        PID: 15668
08:38:37:WU20:FS00:0xa7:        CWD: C:\ProgramData\FAHClient\work
08:38:37:WU20:FS00:0xa7:******************************** Build - libFAH ********************************
08:38:37:WU20:FS00:0xa7:    Version: 0.0.19
08:38:37:WU20:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
08:38:37:WU20:FS00:0xa7:  Copyright: 2019 foldingathome.org
08:38:37:WU20:FS00:0xa7:   Homepage: https://foldingathome.org/
08:38:37:WU20:FS00:0xa7:       Date: Nov 25 2019
08:38:37:WU20:FS00:0xa7:       Time: 17:12:41
08:38:37:WU20:FS00:0xa7:   Revision: d5b5c747532224f986b7cd02c968ed9a20c16d6e
08:38:37:WU20:FS00:0xa7:     Branch: master
08:38:37:WU20:FS00:0xa7:   Compiler: Visual C++ 2008
08:38:37:WU20:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
08:38:37:WU20:FS00:0xa7:   Platform: win32 10
08:38:37:WU20:FS00:0xa7:       Bits: 64
08:38:37:WU20:FS00:0xa7:       Mode: Release
08:38:37:WU20:FS00:0xa7:************************************ Build *************************************
08:38:37:WU20:FS00:0xa7:       SIMD: avx_256
08:38:37:WU20:FS00:0xa7:********************************************************************************
08:38:37:WU20:FS00:0xa7:Project: 16461 (Run 69, Clone 2, Gen 22)
08:38:37:WU20:FS00:0xa7:Unit: 0x0000001680fccb010000000000450002
08:38:37:WU20:FS00:0xa7:Reading tar file core.xml
08:38:37:WU20:FS00:0xa7:Reading tar file frame22.tpr
08:38:37:WU20:FS00:0xa7:Digital signatures verified
08:38:37:WU20:FS00:0xa7:Calling: mdrun -s frame22.tpr -o frame22.trr -x frame22.xtc -cpt 15 -nt 6
08:38:37:WU20:FS00:0xa7:Steps: first=5500000 total=250000
08:38:38:WU20:FS00:0xa7:Completed 1 out of 250000 steps (0%)
08:39:06:WARNING:WU19:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:39:06:WU19:FS00:Trying to send results to collection server
08:39:06:WU19:FS00:Uploading 11.30MiB to 128.252.203.1
08:39:06:WU19:FS00:Connecting to 128.252.203.1:8080
08:39:29:WU20:FS00:0xa7:Completed 2500 out of 250000 steps (1%)
08:39:37:ERROR:WU19:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
08:39:37:WU19:FS00:Sending unit results: id:19 state:SEND error:NO_ERROR project:13821 run:510 clone:5 gen:138 core:0xa7 unit:0x0000009780fccb090000000001fe0005
08:39:37:WU19:FS00:Uploading 11.30MiB to 128.252.203.9
08:39:37:WU19:FS00:Connecting to 128.252.203.9:8080
08:40:07:WARNING:WU19:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:40:07:WU19:FS00:Trying to send results to collection server
08:40:07:WU19:FS00:Uploading 11.30MiB to 128.252.203.1
08:40:07:WU19:FS00:Connecting to 128.252.203.1:8080
08:40:21:WU20:FS00:0xa7:Completed 5000 out of 250000 steps (2%)
08:40:38:ERROR:WU19:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
08:40:38:WU19:FS00:Sending unit results: id:19 state:SEND error:NO_ERROR project:13821 run:510 clone:5 gen:138 core:0xa7 unit:0x0000009780fccb090000000001fe0005
08:40:38:WU19:FS00:Uploading 11.30MiB to 128.252.203.9
08:40:38:WU19:FS00:Connecting to 128.252.203.9:8080
08:41:09:WARNING:WU19:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:41:09:WU19:FS00:Trying to send results to collection server
08:41:09:WU19:FS00:Uploading 11.30MiB to 128.252.203.1
08:41:09:WU19:FS00:Connecting to 128.252.203.1:8080
08:41:13:WU20:FS00:0xa7:Completed 7500 out of 250000 steps (3%)
08:41:15:WU19:FS00:Upload 96.76%
08:41:45:ERROR:WU19:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
08:42:06:WU20:FS00:0xa7:Completed 10000 out of 250000 steps (4%)
08:42:15:WU19:FS00:Sending unit results: id:19 state:SEND error:NO_ERROR project:13821 run:510 clone:5 gen:138 core:0xa7 unit:0x0000009780fccb090000000001fe0005
08:42:15:WU19:FS00:Uploading 11.30MiB to 128.252.203.9
08:42:15:WU19:FS00:Connecting to 128.252.203.9:8080
08:42:46:WARNING:WU19:FS00:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:42:46:WU19:FS00:Trying to send results to collection server
08:42:46:WU19:FS00:Uploading 11.30MiB to 128.252.203.1
08:42:46:WU19:FS00:Connecting to 128.252.203.1:8080
08:42:52:WU19:FS00:Upload 90.13%
08:42:58:WU19:FS00:Upload 96.76%
08:42:59:WU20:FS00:0xa7:Completed 12500 out of 250000 steps (5%)
08:43:07:WU19:FS00:Upload 100.00%
08:43:37:ERROR:WU19:FS00:Exception: 10002: Received short response, expected 512 bytes, got 0
08:43:52:WU20:FS00:0xa7:Completed 15000 out of 250000 steps (6%)
08:44:06:WU17:FS01:Sending unit results: id:17 state:SEND error:NO_ERROR project:13438 run:3084 clone:6 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000c0c0006
08:44:06:WU17:FS01:Uploading 8.50MiB to 18.188.125.154
08:44:06:WU17:FS01:Connecting to 18.188.125.154:8080
08:44:36:WARNING:WU17:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
08:44:36:WU17:FS01:Trying to send results to collection server
08:44:36:WU17:FS01:Uploading 8.50MiB to 150.136.14.110
08:44:36:WU17:FS01:Connecting to 150.136.14.110:8080
08:44:45:WU20:FS00:0xa7:Completed 17500 out of 250000 steps (7%)
08:44:53:WU19:FS00:Sending unit results: id:19 state:SEND error:NO_ERROR project:13821 run:510 clone:5 gen:138 core:0xa7 unit:0x0000009780fccb090000000001fe0005
08:44:53:WU19:FS00:Uploading 11.30MiB to 128.252.203.9
08:44:53:WU19:FS00:Connecting to 128.252.203.9:8080
Last edited by Joe_H on Fri Dec 11, 2020 3:12 pm, edited 1 time in total.
Reason: merged two 1st posts
Top
Joe_H
Site Admin
Posts: 7875
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Can't upload WUs on work servers and 150.136.14.110

Post by Joe_H »

matthewslt wrote:I'm having the exact same issue. Deleting the WUs isnt much of a solution as the client continuously tries to upload the WUs so effectively requires constant attention to keep the queue empty. Please could you look into this as it is constantly trying to upload and is using my already limited bandwidth.
Welcome to the folding support forum.

Looking partway through your log I see WUs being uploaded to several different servers, and spot checking a few shows they had uploaded and been credited. This is pointing towards a problem with your internet connection, not the client. Somewhere the acknowledgement packets are getting dropped or filtered before they get to your PC and the client software.

One thing to try if you are using a wifi connection is connecting directly with an ethernet cable to your router or moving the system closer. If that does not help, posting the results of a traceroute to a few of the servers involved may help us give you some ideas on where to check next. Also, if you have an option to try a different ISP, that might show whether the problem is with your current ISP's setup or equipment.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Top
matthewslt
Posts: 2
Joined: Fri Dec 11, 2020 11:31 am

Re: Can't upload WUs on work servers and 150.136.14.110

Post by matthewslt »

Hi Joe,
Thank you for taking a look.
Im already on an ethernet cable which runs into my router. Im running a draytek 2860 router. I will try a wifi via my 4G phone to see if that helps shortly. See tracert below.
I know what mean when you say this is a connection causing the problem, however I think the client could deal with this in a less problematic fashion i.e instead of constantly retrying to upload it could do with a timeout or reduce the frequency when it tries. Currently it just fills my entire upload bandwidth the whole time.

Code: Select all

C:\Users\laurence>tracert 128.252.203.9

Tracing route to islay.seas.wustl.edu [128.252.203.9]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  10.10.8.1
  2     6 ms     7 ms     6 ms  172.16.14.190
  3     7 ms     8 ms     8 ms  133.hiper04.sheff.dial.plus.net.uk [195.166.143.133]
  4     8 ms     8 ms     7 ms  132.hiper04.sheff.dial.plus.net.uk [195.166.143.132]
  5     8 ms     9 ms     8 ms  195.99.125.144
  6     8 ms    10 ms    17 ms  peer7-et-3-1-4.telehouse.ukcore.bt.net [109.159.252.168]
  7    10 ms     8 ms    10 ms  166-49-214-194.gia.bt.net [166.49.214.194]
  8     *        *        *     Request timed out.
  9     9 ms     8 ms     8 ms  be2870.ccr41.lon13.atlas.cogentco.com [154.54.58.173]
 10    77 ms    98 ms    77 ms  be2099.ccr31.bos01.atlas.cogentco.com [154.54.82.34]
 11    77 ms    77 ms    77 ms  be3599.ccr21.alb02.atlas.cogentco.com [66.28.4.237]
 12    88 ms   111 ms   110 ms  be2878.ccr21.cle04.atlas.cogentco.com [154.54.26.129]
 13   106 ms    96 ms    96 ms  be2717.ccr41.ord01.atlas.cogentco.com [154.54.6.221]
 14   102 ms   102 ms   102 ms  be2349.rcr21.stl03.atlas.cogentco.com [154.54.80.182]
 15   100 ms   101 ms   100 ms  xe-0-0-2-900w-mmr-wu-rt-0.net.wustl.edu [38.104.162.78]
 16   103 ms   103 ms   103 ms  xe-0-0-6-eps-l29-wu-rt-0.net.wustl.edu [128.252.100.125]
 17   113 ms   113 ms   113 ms  po10-epsl29-mll02-wu-vml-10.net.wustl.edu [128.252.161.21]
 18   102 ms   122 ms   102 ms  vl308-epsl29-mll02-wu-vml-11.net.wustl.edu [128.252.161.146]
 19   102 ms   102 ms   102 ms  po5-engineering-core.net.wustl.edu [128.252.161.133]
 20   102 ms   102 ms   103 ms  islay.seas.wustl.edu [128.252.203.9]

Trace complete.

C:\Users\laurence>tracert 150.136.14.110

Tracing route to oracle1.foldingathome.org [150.136.14.110]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  10.10.8.1
  2     5 ms     4 ms     5 ms  172.16.14.190
  3     *        8 ms    11 ms  133.hiper04.sheff.dial.plus.net.uk [195.166.143.133]
  4     8 ms     8 ms     8 ms  132.hiper04.sheff.dial.plus.net.uk [195.166.143.132]
  5     8 ms    10 ms     9 ms  195.99.125.144
  6     8 ms     8 ms     8 ms  peer7-et-3-1-4.telehouse.ukcore.bt.net [109.159.252.168]
  7     8 ms     8 ms     8 ms  166-49-214-194.gia.bt.net [166.49.214.194]
  8    10 ms     8 ms    11 ms  212.119.4.136
  9     *        9 ms    13 ms  ae-0.r21.londen12.uk.bb.gin.ntt.net [129.250.3.214]
 10    83 ms     *       85 ms  ae-13.r25.asbnva02.us.bb.gin.ntt.net [129.250.2.111]
 11    83 ms    82 ms    82 ms  ae-1.r05.asbnva02.us.bb.gin.ntt.net [129.250.2.20]
 12    81 ms    81 ms    84 ms  140.91.196.234
 13     *        *        *     Request timed out.
 14     *        *        *     Request timed out.
 15     *        *        *     Request timed out.
 16     *        *        *     Request timed out.
 17     *        *        *     Request timed out.
 18     *        *        *     Request timed out.
 19     *        *        *     Request timed out.
 20     *        *        *     Request timed out.
 21     *        *        *     Request timed out.
 22     *        *        *     Request timed out.
 23     *        *        *     Request timed out.
 24     *        *        *     Request timed out.
 25     *        *        *     Request timed out.
 26     *        *        *     Request timed out.
 27     *        *        *     Request timed out.
 28     *        *        *     Request timed out.
 29     *        *        *     Request timed out.
 30     *        *        *     Request timed out.

Trace complete.

C:\Users\laurence>tracert 18.188.125.154

Tracing route to aws3.foldingathome.org [18.188.125.154]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  10.10.8.1
  2     5 ms     5 ms     5 ms  172.16.14.190
  3     *        *        *     Request timed out.
  4     7 ms     8 ms     8 ms  128.hiper04.sheff.dial.plus.net.uk [195.166.143.128]
  5     8 ms     9 ms     7 ms  core1-BE1.colindale.ukcore.bt.net [195.99.125.132]
  6    11 ms     9 ms     8 ms  peer7-et-0-1-1.telehouse.ukcore.bt.net [109.159.252.148]
  7    10 ms     8 ms     8 ms  166-49-214-194.gia.bt.net [166.49.214.194]
  8    74 ms    74 ms    74 ms  ixp1-ae-7.us-nyc.gia.bt.net [166.49.195.61]
  9    81 ms    74 ms    74 ms  166-49-204-133.gia.bt.net [166.49.204.133]
 10     *        *        *     Request timed out.
 11     *        *        *     Request timed out.
 12     *        *        *     Request timed out.
 13   118 ms    91 ms    91 ms  52.95.1.120
 14   102 ms   101 ms   123 ms  52.95.1.131
 15   102 ms   102 ms   102 ms  52.95.1.130
 16     *        *        *     Request timed out.
 17     *        *        *     Request timed out.
 18     *        *        *     Request timed out.
 19     *        *        *     Request timed out.
 20     *        *        *     Request timed out.
 21     *        *        *     Request timed out.
 22     *        *        *     Request timed out.
 23     *        *        *     Request timed out.
 24   102 ms   102 ms   102 ms  aws3.foldingathome.org [18.188.125.154]

Trace complete.

C:\Users\laurence>
Top
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: Can't upload WUs on work servers and 150.136.14.110

Post by Neil-B »

Worth using the WU status app to check the PRCG for the WUs waiting to upload ... look and see if they have actually been uploaded and your client simply doesnt know this ... there have been reports of this behaviour
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070
(Green/Bold = Active)
Top
zexmaxwell
Posts: 32
Joined: Fri Mar 06, 2020 5:20 pm

Re: Can't upload WUs on work servers and 150.136.14.110

Post by zexmaxwell »

I have a huge back log of WU lining up for this server.

here is the log for one of them. I have a total of 7 waiting to be uploaded.
looks like 18.188.125.154 as well.

Router settings were not changed. this started to happen when I updated from 7.6.13(?) 7.6.21

Code: Select all

*********************** Log Started 2020-12-11T00:42:49Z ***********************
*********************** Log Started 2020-12-11T00:42:49Z ***********************
*********************** Log Started 2020-12-11T00:42:49Z ***********************
******************************* Date: 2020-12-11 *******************************
09:27:38:WU04:FS01:Connecting to assign1.foldingathome.org:80
09:27:38:WU04:FS01:Assigned to work server 18.188.125.154
09:27:38:WU04:FS01:Requesting new work unit for slot 01: gpu:1:0 GP102 [GeForce GTX 1080 Ti] 11380 from 18.188.125.154
09:27:38:WU04:FS01:Connecting to 18.188.125.154:8080
09:27:43:WU04:FS01:Downloading 6.70MiB
09:27:43:WU04:FS01:Download complete
09:27:43:WU04:FS01:Received Unit: id:04 state:DOWNLOAD error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
09:29:02:WU04:FS01:Starting
09:29:02:WU04:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\lsoli\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 04 -suffix 01 -version 706 -lifeline 5152 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
09:29:02:WU04:FS01:Started FahCore on PID 17980
09:29:02:WU04:FS01:Core PID:14380
09:29:02:WU04:FS01:FahCore 0x22 started
09:29:03:WU04:FS01:0x22:*********************** Log Started 2020-12-11T09:29:02Z ***********************
09:29:03:WU04:FS01:0x22:*************************** Core22 Folding@home Core ***************************
09:29:03:WU04:FS01:0x22:       Core: Core22
09:29:03:WU04:FS01:0x22:       Type: 0x22
09:29:03:WU04:FS01:0x22:    Version: 0.0.13
09:29:03:WU04:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:29:03:WU04:FS01:0x22:  Copyright: 2020 foldingathome.org
09:29:03:WU04:FS01:0x22:   Homepage: https://foldingathome.org/
09:29:03:WU04:FS01:0x22:       Date: Sep 19 2020
09:29:03:WU04:FS01:0x22:       Time: 02:35:58
09:29:03:WU04:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
09:29:03:WU04:FS01:0x22:     Branch: core22-0.0.13
09:29:03:WU04:FS01:0x22:   Compiler: Visual C++ 2015
09:29:03:WU04:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:29:03:WU04:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
09:29:03:WU04:FS01:0x22:   Platform: win32 10
09:29:03:WU04:FS01:0x22:       Bits: 64
09:29:03:WU04:FS01:0x22:       Mode: Release
09:29:03:WU04:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
09:29:03:WU04:FS01:0x22:             <peastman@stanford.edu>
09:29:03:WU04:FS01:0x22:       Args: -dir 04 -suffix 01 -version 706 -lifeline 17980 -checkpoint 15
09:29:03:WU04:FS01:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
09:29:03:WU04:FS01:0x22:             nvidia -gpu 0 -gpu-usage 100
09:29:03:WU04:FS01:0x22:************************************ libFAH ************************************
09:29:03:WU04:FS01:0x22:       Date: Sep 7 2020
09:29:03:WU04:FS01:0x22:       Time: 19:09:56
09:29:03:WU04:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
09:29:03:WU04:FS01:0x22:     Branch: HEAD
09:29:03:WU04:FS01:0x22:   Compiler: Visual C++ 2015
09:29:03:WU04:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:29:03:WU04:FS01:0x22:   Platform: win32 10
09:29:03:WU04:FS01:0x22:       Bits: 64
09:29:03:WU04:FS01:0x22:       Mode: Release
09:29:03:WU04:FS01:0x22:************************************ CBang *************************************
09:29:03:WU04:FS01:0x22:       Date: Sep 7 2020
09:29:03:WU04:FS01:0x22:       Time: 19:08:30
09:29:03:WU04:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
09:29:03:WU04:FS01:0x22:     Branch: HEAD
09:29:03:WU04:FS01:0x22:   Compiler: Visual C++ 2015
09:29:03:WU04:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:29:03:WU04:FS01:0x22:   Platform: win32 10
09:29:03:WU04:FS01:0x22:       Bits: 64
09:29:03:WU04:FS01:0x22:       Mode: Release
09:29:03:WU04:FS01:0x22:************************************ System ************************************
09:29:03:WU04:FS01:0x22:        CPU: Intel(R) Core(TM) i5-9600KF CPU @ 3.70GHz
09:29:03:WU04:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 158 Stepping 13
09:29:03:WU04:FS01:0x22:       CPUs: 6
09:29:03:WU04:FS01:0x22:     Memory: 31.94GiB
09:29:03:WU04:FS01:0x22:Free Memory: 17.39GiB
09:29:03:WU04:FS01:0x22:    Threads: WINDOWS_THREADS
09:29:03:WU04:FS01:0x22: OS Version: 6.2
09:29:03:WU04:FS01:0x22:Has Battery: false
09:29:03:WU04:FS01:0x22: On Battery: false
09:29:03:WU04:FS01:0x22: UTC Offset: -6
09:29:03:WU04:FS01:0x22:        PID: 14380
09:29:03:WU04:FS01:0x22:        CWD: C:\Users\lsoli\AppData\Roaming\FAHClient\work
09:29:03:WU04:FS01:0x22:************************************ OpenMM ************************************
09:29:03:WU04:FS01:0x22:   Revision: 189320d0
09:29:03:WU04:FS01:0x22:********************************************************************************
09:29:03:WU04:FS01:0x22:Project: 13438 (Run 3275, Clone 9, Gen 0)
09:29:03:WU04:FS01:0x22:Unit: 0x0000000012bc7d9a000000000ccb0009
09:29:03:WU04:FS01:0x22:Reading tar file core.xml
09:29:03:WU04:FS01:0x22:Reading tar file integrator.xml.bz2
09:29:03:WU04:FS01:0x22:Reading tar file state.xml.bz2
09:29:03:WU04:FS01:0x22:Reading tar file system.xml.bz2
09:29:03:WU04:FS01:0x22:Digital signatures verified
09:29:03:WU04:FS01:0x22:Folding@home GPU Core22 Folding@home Core
09:29:03:WU04:FS01:0x22:Version 0.0.13
09:29:03:WU04:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
09:29:03:WU04:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
09:29:03:WU04:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
09:29:03:WU04:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
09:29:03:WU04:FS01:0x22:There are 4 platforms available.
09:29:03:WU04:FS01:0x22:Platform 0: Reference
09:29:03:WU04:FS01:0x22:Platform 1: CPU
09:29:03:WU04:FS01:0x22:Platform 2: OpenCL
09:29:03:WU04:FS01:0x22:  opencl-device 0 specified
09:29:03:WU04:FS01:0x22:Platform 3: CUDA
09:29:03:WU04:FS01:0x22:  cuda-device 0 specified
09:29:29:WU04:FS01:0x22:Attempting to create CUDA context:
09:29:29:WU04:FS01:0x22:  Configuring platform CUDA
09:29:36:WU04:FS01:0x22:  Using CUDA and gpu 0
09:29:36:WU04:FS01:0x22:Completed 0 out of 1000000 steps (0%)
09:29:37:WU04:FS01:0x22:Checkpoint completed at step 0
09:31:05:WU04:FS01:0x22:Completed 10000 out of 1000000 steps (1%)
09:32:33:WU04:FS01:0x22:Completed 20000 out of 1000000 steps (2%)
09:34:01:WU04:FS01:0x22:Completed 30000 out of 1000000 steps (3%)
09:35:29:WU04:FS01:0x22:Completed 40000 out of 1000000 steps (4%)
09:36:57:WU04:FS01:0x22:Completed 50000 out of 1000000 steps (5%)
09:36:58:WU04:FS01:0x22:Checkpoint completed at step 50000
09:38:28:WU04:FS01:0x22:Completed 60000 out of 1000000 steps (6%)
09:39:57:WU04:FS01:0x22:Completed 70000 out of 1000000 steps (7%)
09:41:26:WU04:FS01:0x22:Completed 80000 out of 1000000 steps (8%)
09:42:54:WU04:FS01:0x22:Completed 90000 out of 1000000 steps (9%)
09:44:23:WU04:FS01:0x22:Completed 100000 out of 1000000 steps (10%)
09:44:23:WU04:FS01:0x22:Checkpoint completed at step 100000
09:45:52:WU04:FS01:0x22:Completed 110000 out of 1000000 steps (11%)
09:47:21:WU04:FS01:0x22:Completed 120000 out of 1000000 steps (12%)
09:48:50:WU04:FS01:0x22:Completed 130000 out of 1000000 steps (13%)
09:50:23:WU04:FS01:0x22:Completed 140000 out of 1000000 steps (14%)
09:51:57:WU04:FS01:0x22:Completed 150000 out of 1000000 steps (15%)
09:51:58:WU04:FS01:0x22:Checkpoint completed at step 150000
09:53:27:WU04:FS01:0x22:Completed 160000 out of 1000000 steps (16%)
09:54:57:WU04:FS01:0x22:Completed 170000 out of 1000000 steps (17%)
09:56:28:WU04:FS01:0x22:Completed 180000 out of 1000000 steps (18%)
09:57:57:WU04:FS01:0x22:Completed 190000 out of 1000000 steps (19%)
09:59:26:WU04:FS01:0x22:Completed 200000 out of 1000000 steps (20%)
09:59:27:WU04:FS01:0x22:Checkpoint completed at step 200000
10:00:56:WU04:FS01:0x22:Completed 210000 out of 1000000 steps (21%)
10:02:25:WU04:FS01:0x22:Completed 220000 out of 1000000 steps (22%)
10:03:54:WU04:FS01:0x22:Completed 230000 out of 1000000 steps (23%)
10:05:23:WU04:FS01:0x22:Completed 240000 out of 1000000 steps (24%)
10:06:51:WU04:FS01:0x22:Completed 250000 out of 1000000 steps (25%)
10:06:52:WU04:FS01:0x22:Checkpoint completed at step 250000
10:08:21:WU04:FS01:0x22:Completed 260000 out of 1000000 steps (26%)
10:09:53:WU04:FS01:0x22:Completed 270000 out of 1000000 steps (27%)
10:11:26:WU04:FS01:0x22:Completed 280000 out of 1000000 steps (28%)
10:12:57:WU04:FS01:0x22:Completed 290000 out of 1000000 steps (29%)
10:14:29:WU04:FS01:0x22:Completed 300000 out of 1000000 steps (30%)
10:14:30:WU04:FS01:0x22:Checkpoint completed at step 300000
10:16:01:WU04:FS01:0x22:Completed 310000 out of 1000000 steps (31%)
10:17:33:WU04:FS01:0x22:Completed 320000 out of 1000000 steps (32%)
10:19:05:WU04:FS01:0x22:Completed 330000 out of 1000000 steps (33%)
10:20:36:WU04:FS01:0x22:Completed 340000 out of 1000000 steps (34%)
10:22:08:WU04:FS01:0x22:Completed 350000 out of 1000000 steps (35%)
10:22:09:WU04:FS01:0x22:Checkpoint completed at step 350000
10:23:41:WU04:FS01:0x22:Completed 360000 out of 1000000 steps (36%)
10:25:12:WU04:FS01:0x22:Completed 370000 out of 1000000 steps (37%)
10:26:42:WU04:FS01:0x22:Completed 380000 out of 1000000 steps (38%)
10:28:13:WU04:FS01:0x22:Completed 390000 out of 1000000 steps (39%)
10:29:44:WU04:FS01:0x22:Completed 400000 out of 1000000 steps (40%)
10:29:45:WU04:FS01:0x22:Checkpoint completed at step 400000
10:31:17:WU04:FS01:0x22:Completed 410000 out of 1000000 steps (41%)
10:32:48:WU04:FS01:0x22:Completed 420000 out of 1000000 steps (42%)
10:34:19:WU04:FS01:0x22:Completed 430000 out of 1000000 steps (43%)
10:35:49:WU04:FS01:0x22:Completed 440000 out of 1000000 steps (44%)
10:37:19:WU04:FS01:0x22:Completed 450000 out of 1000000 steps (45%)
10:37:20:WU04:FS01:0x22:Checkpoint completed at step 450000
10:38:51:WU04:FS01:0x22:Completed 460000 out of 1000000 steps (46%)
10:40:21:WU04:FS01:0x22:Completed 470000 out of 1000000 steps (47%)
10:41:50:WU04:FS01:0x22:Completed 480000 out of 1000000 steps (48%)
10:43:19:WU04:FS01:0x22:Completed 490000 out of 1000000 steps (49%)
10:44:48:WU04:FS01:0x22:Completed 500000 out of 1000000 steps (50%)
10:44:49:WU04:FS01:0x22:Checkpoint completed at step 500000
10:46:16:WU04:FS01:0x22:Completed 510000 out of 1000000 steps (51%)
10:47:48:WU04:FS01:0x22:Completed 520000 out of 1000000 steps (52%)
10:49:16:WU04:FS01:0x22:Completed 530000 out of 1000000 steps (53%)
10:50:43:WU04:FS01:0x22:Completed 540000 out of 1000000 steps (54%)
10:52:11:WU04:FS01:0x22:Completed 550000 out of 1000000 steps (55%)
10:52:12:WU04:FS01:0x22:Checkpoint completed at step 550000
10:53:40:WU04:FS01:0x22:Completed 560000 out of 1000000 steps (56%)
10:55:08:WU04:FS01:0x22:Completed 570000 out of 1000000 steps (57%)
10:56:35:WU04:FS01:0x22:Completed 580000 out of 1000000 steps (58%)
10:58:03:WU04:FS01:0x22:Completed 590000 out of 1000000 steps (59%)
10:59:32:WU04:FS01:0x22:Completed 600000 out of 1000000 steps (60%)
10:59:33:WU04:FS01:0x22:Checkpoint completed at step 600000
11:01:00:WU04:FS01:0x22:Completed 610000 out of 1000000 steps (61%)
11:02:29:WU04:FS01:0x22:Completed 620000 out of 1000000 steps (62%)
11:03:57:WU04:FS01:0x22:Completed 630000 out of 1000000 steps (63%)
11:05:24:WU04:FS01:0x22:Completed 640000 out of 1000000 steps (64%)
11:06:53:WU04:FS01:0x22:Completed 650000 out of 1000000 steps (65%)
11:06:54:WU04:FS01:0x22:Checkpoint completed at step 650000
11:08:22:WU04:FS01:0x22:Completed 660000 out of 1000000 steps (66%)
11:09:52:WU04:FS01:0x22:Completed 670000 out of 1000000 steps (67%)
11:11:20:WU04:FS01:0x22:Completed 680000 out of 1000000 steps (68%)
11:12:49:WU04:FS01:0x22:Completed 690000 out of 1000000 steps (69%)
11:14:18:WU04:FS01:0x22:Completed 700000 out of 1000000 steps (70%)
11:14:19:WU04:FS01:0x22:Checkpoint completed at step 700000
11:15:48:WU04:FS01:0x22:Completed 710000 out of 1000000 steps (71%)
11:17:16:WU04:FS01:0x22:Completed 720000 out of 1000000 steps (72%)
11:18:45:WU04:FS01:0x22:Completed 730000 out of 1000000 steps (73%)
11:20:07:WU04:FS01:0x22:Completed 740000 out of 1000000 steps (74%)
11:21:25:WU04:FS01:0x22:Completed 750000 out of 1000000 steps (75%)
11:21:26:WU04:FS01:0x22:Checkpoint completed at step 750000
11:22:45:WU04:FS01:0x22:Completed 760000 out of 1000000 steps (76%)
11:24:05:WU04:FS01:0x22:Completed 770000 out of 1000000 steps (77%)
11:25:24:WU04:FS01:0x22:Completed 780000 out of 1000000 steps (78%)
11:26:44:WU04:FS01:0x22:Completed 790000 out of 1000000 steps (79%)
11:28:03:WU04:FS01:0x22:Completed 800000 out of 1000000 steps (80%)
11:28:03:WU04:FS01:0x22:Checkpoint completed at step 800000
11:29:23:WU04:FS01:0x22:Completed 810000 out of 1000000 steps (81%)
11:30:42:WU04:FS01:0x22:Completed 820000 out of 1000000 steps (82%)
11:32:05:WU04:FS01:0x22:Completed 830000 out of 1000000 steps (83%)
11:33:27:WU04:FS01:0x22:Completed 840000 out of 1000000 steps (84%)
******************************* Date: 2020-12-11 *******************************
09:27:38:WU04:FS01:Connecting to assign1.foldingathome.org:80
09:27:38:WU04:FS01:Assigned to work server 18.188.125.154
09:27:38:WU04:FS01:Requesting new work unit for slot 01: gpu:1:0 GP102 [GeForce GTX 1080 Ti] 11380 from 18.188.125.154
09:27:38:WU04:FS01:Connecting to 18.188.125.154:8080
09:27:43:WU04:FS01:Downloading 6.70MiB
09:27:43:WU04:FS01:Download complete
09:27:43:WU04:FS01:Received Unit: id:04 state:DOWNLOAD error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
09:29:02:WU04:FS01:Starting
09:29:02:WU04:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\lsoli\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 04 -suffix 01 -version 706 -lifeline 5152 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
09:29:02:WU04:FS01:Started FahCore on PID 17980
09:29:02:WU04:FS01:Core PID:14380
09:29:02:WU04:FS01:FahCore 0x22 started
09:29:03:WU04:FS01:0x22:*********************** Log Started 2020-12-11T09:29:02Z ***********************
09:29:03:WU04:FS01:0x22:*************************** Core22 Folding@home Core ***************************
09:29:03:WU04:FS01:0x22:       Core: Core22
09:29:03:WU04:FS01:0x22:       Type: 0x22
09:29:03:WU04:FS01:0x22:    Version: 0.0.13
09:29:03:WU04:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:29:03:WU04:FS01:0x22:  Copyright: 2020 foldingathome.org
09:29:03:WU04:FS01:0x22:   Homepage: https://foldingathome.org/
09:29:03:WU04:FS01:0x22:       Date: Sep 19 2020
09:29:03:WU04:FS01:0x22:       Time: 02:35:58
09:29:03:WU04:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
09:29:03:WU04:FS01:0x22:     Branch: core22-0.0.13
09:29:03:WU04:FS01:0x22:   Compiler: Visual C++ 2015
09:29:03:WU04:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:29:03:WU04:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
09:29:03:WU04:FS01:0x22:   Platform: win32 10
09:29:03:WU04:FS01:0x22:       Bits: 64
09:29:03:WU04:FS01:0x22:       Mode: Release
09:29:03:WU04:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
09:29:03:WU04:FS01:0x22:             <peastman@stanford.edu>
09:29:03:WU04:FS01:0x22:       Args: -dir 04 -suffix 01 -version 706 -lifeline 17980 -checkpoint 15
09:29:03:WU04:FS01:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
09:29:03:WU04:FS01:0x22:             nvidia -gpu 0 -gpu-usage 100
09:29:03:WU04:FS01:0x22:************************************ libFAH ************************************
09:29:03:WU04:FS01:0x22:       Date: Sep 7 2020
09:29:03:WU04:FS01:0x22:       Time: 19:09:56
09:29:03:WU04:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
09:29:03:WU04:FS01:0x22:     Branch: HEAD
09:29:03:WU04:FS01:0x22:   Compiler: Visual C++ 2015
09:29:03:WU04:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:29:03:WU04:FS01:0x22:   Platform: win32 10
09:29:03:WU04:FS01:0x22:       Bits: 64
09:29:03:WU04:FS01:0x22:       Mode: Release
09:29:03:WU04:FS01:0x22:************************************ CBang *************************************
09:29:03:WU04:FS01:0x22:       Date: Sep 7 2020
09:29:03:WU04:FS01:0x22:       Time: 19:08:30
09:29:03:WU04:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
09:29:03:WU04:FS01:0x22:     Branch: HEAD
09:29:03:WU04:FS01:0x22:   Compiler: Visual C++ 2015
09:29:03:WU04:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:29:03:WU04:FS01:0x22:   Platform: win32 10
09:29:03:WU04:FS01:0x22:       Bits: 64
09:29:03:WU04:FS01:0x22:       Mode: Release
09:29:03:WU04:FS01:0x22:************************************ System ************************************
09:29:03:WU04:FS01:0x22:        CPU: Intel(R) Core(TM) i5-9600KF CPU @ 3.70GHz
09:29:03:WU04:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 158 Stepping 13
09:29:03:WU04:FS01:0x22:       CPUs: 6
09:29:03:WU04:FS01:0x22:     Memory: 31.94GiB
09:29:03:WU04:FS01:0x22:Free Memory: 17.39GiB
09:29:03:WU04:FS01:0x22:    Threads: WINDOWS_THREADS
09:29:03:WU04:FS01:0x22: OS Version: 6.2
09:29:03:WU04:FS01:0x22:Has Battery: false
09:29:03:WU04:FS01:0x22: On Battery: false
09:29:03:WU04:FS01:0x22: UTC Offset: -6
09:29:03:WU04:FS01:0x22:        PID: 14380
09:29:03:WU04:FS01:0x22:        CWD: C:\Users\lsoli\AppData\Roaming\FAHClient\work
09:29:03:WU04:FS01:0x22:************************************ OpenMM ************************************
09:29:03:WU04:FS01:0x22:   Revision: 189320d0
09:29:03:WU04:FS01:0x22:********************************************************************************
09:29:03:WU04:FS01:0x22:Project: 13438 (Run 3275, Clone 9, Gen 0)
09:29:03:WU04:FS01:0x22:Unit: 0x0000000012bc7d9a000000000ccb0009
09:29:03:WU04:FS01:0x22:Reading tar file core.xml
09:29:03:WU04:FS01:0x22:Reading tar file integrator.xml.bz2
09:29:03:WU04:FS01:0x22:Reading tar file state.xml.bz2
09:29:03:WU04:FS01:0x22:Reading tar file system.xml.bz2
09:29:03:WU04:FS01:0x22:Digital signatures verified
09:29:03:WU04:FS01:0x22:Folding@home GPU Core22 Folding@home Core
09:29:03:WU04:FS01:0x22:Version 0.0.13
09:29:03:WU04:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
09:29:03:WU04:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
09:29:03:WU04:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
09:29:03:WU04:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
09:29:03:WU04:FS01:0x22:There are 4 platforms available.
09:29:03:WU04:FS01:0x22:Platform 0: Reference
09:29:03:WU04:FS01:0x22:Platform 1: CPU
09:29:03:WU04:FS01:0x22:Platform 2: OpenCL
09:29:03:WU04:FS01:0x22:  opencl-device 0 specified
09:29:03:WU04:FS01:0x22:Platform 3: CUDA
09:29:03:WU04:FS01:0x22:  cuda-device 0 specified
09:29:29:WU04:FS01:0x22:Attempting to create CUDA context:
09:29:29:WU04:FS01:0x22:  Configuring platform CUDA
09:29:36:WU04:FS01:0x22:  Using CUDA and gpu 0
09:29:36:WU04:FS01:0x22:Completed 0 out of 1000000 steps (0%)
09:29:37:WU04:FS01:0x22:Checkpoint completed at step 0
09:31:05:WU04:FS01:0x22:Completed 10000 out of 1000000 steps (1%)
09:32:33:WU04:FS01:0x22:Completed 20000 out of 1000000 steps (2%)
09:34:01:WU04:FS01:0x22:Completed 30000 out of 1000000 steps (3%)
09:35:29:WU04:FS01:0x22:Completed 40000 out of 1000000 steps (4%)
09:36:57:WU04:FS01:0x22:Completed 50000 out of 1000000 steps (5%)
09:36:58:WU04:FS01:0x22:Checkpoint completed at step 50000
09:38:28:WU04:FS01:0x22:Completed 60000 out of 1000000 steps (6%)
09:39:57:WU04:FS01:0x22:Completed 70000 out of 1000000 steps (7%)
09:41:26:WU04:FS01:0x22:Completed 80000 out of 1000000 steps (8%)
09:42:54:WU04:FS01:0x22:Completed 90000 out of 1000000 steps (9%)
09:44:23:WU04:FS01:0x22:Completed 100000 out of 1000000 steps (10%)
09:44:23:WU04:FS01:0x22:Checkpoint completed at step 100000
09:45:52:WU04:FS01:0x22:Completed 110000 out of 1000000 steps (11%)
09:47:21:WU04:FS01:0x22:Completed 120000 out of 1000000 steps (12%)
09:48:50:WU04:FS01:0x22:Completed 130000 out of 1000000 steps (13%)
09:50:23:WU04:FS01:0x22:Completed 140000 out of 1000000 steps (14%)
09:51:57:WU04:FS01:0x22:Completed 150000 out of 1000000 steps (15%)
09:51:58:WU04:FS01:0x22:Checkpoint completed at step 150000
09:53:27:WU04:FS01:0x22:Completed 160000 out of 1000000 steps (16%)
09:54:57:WU04:FS01:0x22:Completed 170000 out of 1000000 steps (17%)
09:56:28:WU04:FS01:0x22:Completed 180000 out of 1000000 steps (18%)
09:57:57:WU04:FS01:0x22:Completed 190000 out of 1000000 steps (19%)
09:59:26:WU04:FS01:0x22:Completed 200000 out of 1000000 steps (20%)
09:59:27:WU04:FS01:0x22:Checkpoint completed at step 200000
10:00:56:WU04:FS01:0x22:Completed 210000 out of 1000000 steps (21%)
10:02:25:WU04:FS01:0x22:Completed 220000 out of 1000000 steps (22%)
10:03:54:WU04:FS01:0x22:Completed 230000 out of 1000000 steps (23%)
10:05:23:WU04:FS01:0x22:Completed 240000 out of 1000000 steps (24%)
10:06:51:WU04:FS01:0x22:Completed 250000 out of 1000000 steps (25%)
10:06:52:WU04:FS01:0x22:Checkpoint completed at step 250000
10:08:21:WU04:FS01:0x22:Completed 260000 out of 1000000 steps (26%)
10:09:53:WU04:FS01:0x22:Completed 270000 out of 1000000 steps (27%)
10:11:26:WU04:FS01:0x22:Completed 280000 out of 1000000 steps (28%)
10:12:57:WU04:FS01:0x22:Completed 290000 out of 1000000 steps (29%)
10:14:29:WU04:FS01:0x22:Completed 300000 out of 1000000 steps (30%)
10:14:30:WU04:FS01:0x22:Checkpoint completed at step 300000
10:16:01:WU04:FS01:0x22:Completed 310000 out of 1000000 steps (31%)
10:17:33:WU04:FS01:0x22:Completed 320000 out of 1000000 steps (32%)
10:19:05:WU04:FS01:0x22:Completed 330000 out of 1000000 steps (33%)
10:20:36:WU04:FS01:0x22:Completed 340000 out of 1000000 steps (34%)
10:22:08:WU04:FS01:0x22:Completed 350000 out of 1000000 steps (35%)
10:22:09:WU04:FS01:0x22:Checkpoint completed at step 350000
10:23:41:WU04:FS01:0x22:Completed 360000 out of 1000000 steps (36%)
10:25:12:WU04:FS01:0x22:Completed 370000 out of 1000000 steps (37%)
10:26:42:WU04:FS01:0x22:Completed 380000 out of 1000000 steps (38%)
10:28:13:WU04:FS01:0x22:Completed 390000 out of 1000000 steps (39%)
10:29:44:WU04:FS01:0x22:Completed 400000 out of 1000000 steps (40%)
10:29:45:WU04:FS01:0x22:Checkpoint completed at step 400000
10:31:17:WU04:FS01:0x22:Completed 410000 out of 1000000 steps (41%)
10:32:48:WU04:FS01:0x22:Completed 420000 out of 1000000 steps (42%)
10:34:19:WU04:FS01:0x22:Completed 430000 out of 1000000 steps (43%)
10:35:49:WU04:FS01:0x22:Completed 440000 out of 1000000 steps (44%)
10:37:19:WU04:FS01:0x22:Completed 450000 out of 1000000 steps (45%)
10:37:20:WU04:FS01:0x22:Checkpoint completed at step 450000
10:38:51:WU04:FS01:0x22:Completed 460000 out of 1000000 steps (46%)
10:40:21:WU04:FS01:0x22:Completed 470000 out of 1000000 steps (47%)
10:41:50:WU04:FS01:0x22:Completed 480000 out of 1000000 steps (48%)
10:43:19:WU04:FS01:0x22:Completed 490000 out of 1000000 steps (49%)
10:44:48:WU04:FS01:0x22:Completed 500000 out of 1000000 steps (50%)
10:44:49:WU04:FS01:0x22:Checkpoint completed at step 500000
10:46:16:WU04:FS01:0x22:Completed 510000 out of 1000000 steps (51%)
10:47:48:WU04:FS01:0x22:Completed 520000 out of 1000000 steps (52%)
10:49:16:WU04:FS01:0x22:Completed 530000 out of 1000000 steps (53%)
10:50:43:WU04:FS01:0x22:Completed 540000 out of 1000000 steps (54%)
10:52:11:WU04:FS01:0x22:Completed 550000 out of 1000000 steps (55%)
10:52:12:WU04:FS01:0x22:Checkpoint completed at step 550000
10:53:40:WU04:FS01:0x22:Completed 560000 out of 1000000 steps (56%)
10:55:08:WU04:FS01:0x22:Completed 570000 out of 1000000 steps (57%)
10:56:35:WU04:FS01:0x22:Completed 580000 out of 1000000 steps (58%)
10:58:03:WU04:FS01:0x22:Completed 590000 out of 1000000 steps (59%)
10:59:32:WU04:FS01:0x22:Completed 600000 out of 1000000 steps (60%)
10:59:33:WU04:FS01:0x22:Checkpoint completed at step 600000
11:01:00:WU04:FS01:0x22:Completed 610000 out of 1000000 steps (61%)
11:02:29:WU04:FS01:0x22:Completed 620000 out of 1000000 steps (62%)
11:03:57:WU04:FS01:0x22:Completed 630000 out of 1000000 steps (63%)
11:05:24:WU04:FS01:0x22:Completed 640000 out of 1000000 steps (64%)
11:06:53:WU04:FS01:0x22:Completed 650000 out of 1000000 steps (65%)
11:06:54:WU04:FS01:0x22:Checkpoint completed at step 650000
11:08:22:WU04:FS01:0x22:Completed 660000 out of 1000000 steps (66%)
11:09:52:WU04:FS01:0x22:Completed 670000 out of 1000000 steps (67%)
11:11:20:WU04:FS01:0x22:Completed 680000 out of 1000000 steps (68%)
11:12:49:WU04:FS01:0x22:Completed 690000 out of 1000000 steps (69%)
11:14:18:WU04:FS01:0x22:Completed 700000 out of 1000000 steps (70%)
11:14:19:WU04:FS01:0x22:Checkpoint completed at step 700000
11:15:48:WU04:FS01:0x22:Completed 710000 out of 1000000 steps (71%)
11:17:16:WU04:FS01:0x22:Completed 720000 out of 1000000 steps (72%)
11:18:45:WU04:FS01:0x22:Completed 730000 out of 1000000 steps (73%)
11:20:07:WU04:FS01:0x22:Completed 740000 out of 1000000 steps (74%)
11:21:25:WU04:FS01:0x22:Completed 750000 out of 1000000 steps (75%)
11:21:26:WU04:FS01:0x22:Checkpoint completed at step 750000
11:22:45:WU04:FS01:0x22:Completed 760000 out of 1000000 steps (76%)
11:24:05:WU04:FS01:0x22:Completed 770000 out of 1000000 steps (77%)
11:25:24:WU04:FS01:0x22:Completed 780000 out of 1000000 steps (78%)
11:26:44:WU04:FS01:0x22:Completed 790000 out of 1000000 steps (79%)
11:28:03:WU04:FS01:0x22:Completed 800000 out of 1000000 steps (80%)
11:28:03:WU04:FS01:0x22:Checkpoint completed at step 800000
11:29:23:WU04:FS01:0x22:Completed 810000 out of 1000000 steps (81%)
11:30:42:WU04:FS01:0x22:Completed 820000 out of 1000000 steps (82%)
11:32:05:WU04:FS01:0x22:Completed 830000 out of 1000000 steps (83%)
11:33:27:WU04:FS01:0x22:Completed 840000 out of 1000000 steps (84%)
11:34:47:WU04:FS01:0x22:Completed 850000 out of 1000000 steps (85%)
11:34:47:WU04:FS01:0x22:Checkpoint completed at step 850000
11:36:06:WU04:FS01:0x22:Completed 860000 out of 1000000 steps (86%)
11:37:25:WU04:FS01:0x22:Completed 870000 out of 1000000 steps (87%)
11:38:45:WU04:FS01:0x22:Completed 880000 out of 1000000 steps (88%)
11:40:04:WU04:FS01:0x22:Completed 890000 out of 1000000 steps (89%)
11:41:23:WU04:FS01:0x22:Completed 900000 out of 1000000 steps (90%)
11:41:24:WU04:FS01:0x22:Checkpoint completed at step 900000
11:42:43:WU04:FS01:0x22:Completed 910000 out of 1000000 steps (91%)
11:44:02:WU04:FS01:0x22:Completed 920000 out of 1000000 steps (92%)
11:45:21:WU04:FS01:0x22:Completed 930000 out of 1000000 steps (93%)
11:46:40:WU04:FS01:0x22:Completed 940000 out of 1000000 steps (94%)
11:47:59:WU04:FS01:0x22:Completed 950000 out of 1000000 steps (95%)
11:48:00:WU04:FS01:0x22:Checkpoint completed at step 950000
11:49:18:WU04:FS01:0x22:Completed 960000 out of 1000000 steps (96%)
11:50:38:WU04:FS01:0x22:Completed 970000 out of 1000000 steps (97%)
11:51:56:WU04:FS01:0x22:Completed 980000 out of 1000000 steps (98%)
11:53:15:WU04:FS01:0x22:Completed 990000 out of 1000000 steps (99%)
11:54:33:WU04:FS01:0x22:Completed 1000000 out of 1000000 steps (100%)
11:54:33:WU04:FS01:0x22:Average performance: 109.645 ns/day
11:54:34:WU04:FS01:0x22:Checkpoint completed at step 1000000
11:54:41:WU04:FS01:0x22:Saving result file ..\logfile_01.txt
11:54:41:WU04:FS01:0x22:Saving result file checkpointIntegrator.xml.bz2
11:54:41:WU04:FS01:0x22:Saving result file checkpointState.xml.bz2
11:54:41:WU04:FS01:0x22:Saving result file globals.csv
11:54:41:WU04:FS01:0x22:Saving result file positions.xtc
11:54:41:WU04:FS01:0x22:Saving result file science.log
11:54:41:WU04:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
11:54:41:WU04:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
11:54:41:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
11:54:41:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
11:54:41:WU04:FS01:Connecting to 18.188.125.154:8080
11:55:12:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
11:55:12:WU04:FS01:Trying to send results to collection server
11:55:12:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
11:55:12:WU04:FS01:Connecting to 150.136.14.110:8080
11:55:42:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
11:55:42:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
11:55:42:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
11:55:42:WU04:FS01:Connecting to 18.188.125.154:8080
11:56:12:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
11:56:12:WU04:FS01:Trying to send results to collection server
11:56:12:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
11:56:12:WU04:FS01:Connecting to 150.136.14.110:8080
11:56:43:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
11:56:43:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
11:56:43:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
11:56:43:WU04:FS01:Connecting to 18.188.125.154:8080
11:57:13:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
11:57:13:WU04:FS01:Trying to send results to collection server
11:57:13:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
11:57:13:WU04:FS01:Connecting to 150.136.14.110:8080
11:57:44:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
11:58:20:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
11:58:20:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
11:58:20:WU04:FS01:Connecting to 18.188.125.154:8080
11:58:50:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
11:58:50:WU04:FS01:Trying to send results to collection server
11:58:50:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
11:58:50:WU04:FS01:Connecting to 150.136.14.110:8080
11:59:21:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:00:57:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:00:57:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:00:57:WU04:FS01:Connecting to 18.188.125.154:8080
12:01:28:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:01:28:WU04:FS01:Trying to send results to collection server
12:01:28:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:01:28:WU04:FS01:Connecting to 150.136.14.110:8080
12:01:58:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:05:12:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:05:12:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:05:12:WU04:FS01:Connecting to 18.188.125.154:8080
12:05:42:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:05:42:WU04:FS01:Trying to send results to collection server
12:05:42:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:05:42:WU04:FS01:Connecting to 150.136.14.110:8080
12:06:13:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:12:03:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:12:03:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:12:03:WU04:FS01:Connecting to 18.188.125.154:8080
12:12:33:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:12:33:WU04:FS01:Trying to send results to collection server
12:12:33:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:12:33:WU04:FS01:Connecting to 150.136.14.110:8080
12:13:03:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:23:09:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:23:09:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:23:09:WU04:FS01:Connecting to 18.188.125.154:8080
12:23:39:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:23:39:WU04:FS01:Trying to send results to collection server
12:23:39:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:23:39:WU04:FS01:Connecting to 150.136.14.110:8080
12:24:09:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:41:05:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:41:05:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:41:05:WU04:FS01:Connecting to 18.188.125.154:8080
12:41:35:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:41:35:WU04:FS01:Trying to send results to collection server
12:41:35:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:41:35:WU04:FS01:Connecting to 150.136.14.110:8080
12:42:06:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
******************************* Date: 2020-12-11 *******************************
13:10:08:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
13:10:08:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
13:10:08:WU04:FS01:Connecting to 18.188.125.154:8080
13:10:38:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
13:10:38:WU04:FS01:Trying to send results to collection server
13:10:38:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
13:10:38:WU04:FS01:Connecting to 150.136.14.110:8080
13:11:08:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
13:57:06:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
13:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
13:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
13:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
13:57:37:WU04:FS01:Trying to send results to collection server
13:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
13:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
13:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
14:57:06:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
14:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
14:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
14:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
14:57:37:WU04:FS01:Trying to send results to collection server
14:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
14:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
14:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
15:57:07:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
15:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
15:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
15:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
15:57:37:WU04:FS01:Trying to send results to collection server
15:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
15:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
15:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
16:57:07:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
16:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
16:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
16:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
16:57:37:WU04:FS01:Trying to send results to collection server
16:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
16:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
16:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
11:58:20:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
11:58:20:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
11:58:20:WU04:FS01:Connecting to 18.188.125.154:8080
11:58:50:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
11:58:50:WU04:FS01:Trying to send results to collection server
11:58:50:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
11:58:50:WU04:FS01:Connecting to 150.136.14.110:8080
11:59:21:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:00:57:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:00:57:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:00:57:WU04:FS01:Connecting to 18.188.125.154:8080
12:01:28:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:01:28:WU04:FS01:Trying to send results to collection server
12:01:28:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:01:28:WU04:FS01:Connecting to 150.136.14.110:8080
12:01:58:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:05:12:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:05:12:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:05:12:WU04:FS01:Connecting to 18.188.125.154:8080
12:05:42:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:05:42:WU04:FS01:Trying to send results to collection server
12:05:42:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:05:42:WU04:FS01:Connecting to 150.136.14.110:8080
12:06:13:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:12:03:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:12:03:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:12:03:WU04:FS01:Connecting to 18.188.125.154:8080
12:12:33:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:12:33:WU04:FS01:Trying to send results to collection server
12:12:33:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:12:33:WU04:FS01:Connecting to 150.136.14.110:8080
12:13:03:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:23:09:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:23:09:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:23:09:WU04:FS01:Connecting to 18.188.125.154:8080
12:23:39:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:23:39:WU04:FS01:Trying to send results to collection server
12:23:39:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:23:39:WU04:FS01:Connecting to 150.136.14.110:8080
12:24:09:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
12:41:05:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
12:41:05:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
12:41:05:WU04:FS01:Connecting to 18.188.125.154:8080
12:41:35:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
12:41:35:WU04:FS01:Trying to send results to collection server
12:41:35:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
12:41:35:WU04:FS01:Connecting to 150.136.14.110:8080
12:42:06:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
******************************* Date: 2020-12-11 *******************************
13:10:08:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
13:10:08:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
13:10:08:WU04:FS01:Connecting to 18.188.125.154:8080
13:10:38:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
13:10:38:WU04:FS01:Trying to send results to collection server
13:10:38:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
13:10:38:WU04:FS01:Connecting to 150.136.14.110:8080
13:11:08:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
13:57:06:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
13:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
13:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
13:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
13:57:37:WU04:FS01:Trying to send results to collection server
13:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
13:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
13:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
14:57:06:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
14:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
14:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
14:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
14:57:37:WU04:FS01:Trying to send results to collection server
14:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
14:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
14:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
15:57:07:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
15:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
15:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
15:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
15:57:37:WU04:FS01:Trying to send results to collection server
15:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
15:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
15:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
16:57:07:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
16:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
16:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
16:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
16:57:37:WU04:FS01:Trying to send results to collection server
16:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
16:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
16:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
17:57:07:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
17:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
17:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
17:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
17:57:37:WU04:FS01:Trying to send results to collection server
17:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
17:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
17:58:07:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
******************************* Date: 2020-12-11 *******************************
18:57:07:WU04:FS01:Sending unit results: id:04 state:SEND error:NO_ERROR project:13438 run:3275 clone:9 gen:0 core:0x22 unit:0x0000000012bc7d9a000000000ccb0009
18:57:07:WU04:FS01:Uploading 8.50MiB to 18.188.125.154
18:57:07:WU04:FS01:Connecting to 18.188.125.154:8080
18:57:37:WARNING:WU04:FS01:Exception: Failed to send results to work server: 10002: Received short response, expected 512 bytes, got 0
18:57:37:WU04:FS01:Trying to send results to collection server
18:57:37:WU04:FS01:Uploading 8.50MiB to 150.136.14.110
18:57:37:WU04:FS01:Connecting to 150.136.14.110:8080
18:58:08:ERROR:WU04:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
My room is always Hot.
Top
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Can't upload WUs on work servers and 150.136.14.110

Post by bruce »

If you're getting these responses AND there is evidence that the WU has already been credited, that points to a problem with your router.
* When a WU is successfully uploaded, the server sends a ACK message confirming that it was received.
* When the client receives that ACK, the local copy of the WU is "cleaned up" (deleted). When the ACK is NOT received, the client keeps the local copy so it can retry the upload.
* Some router firmware "optimizes" the data stream, and discards the ACK as an unnecessary message. Thus WUs which have been successfully uploaded are re-processed forever (until they expire). The router has a setting allowing you to disable this stream optimization.

* When the server sees your client uploading a WU which has already been credited, it simply assumes that you're trying to get credit for the same WU twice and it rightfully discards the duplicated uploads. You need to change the settings of your router owhich is "too smart" and it unnecessarily discarding messages that are, in fact, needed.
Top
Joe_H
Site Admin
Posts: 7875
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Can't upload WUs on work servers and 150.136.14.110

Post by Joe_H »

matthewslt wrote:I know what mean when you say this is a connection causing the problem, however I think the client could deal with this in a less problematic fashion i.e instead of constantly retrying to upload it could do with a timeout or reduce the frequency when it tries. Currently it just fills my entire upload bandwidth the whole time.
Currently the client does reduce the frequency of upload retries, it backs off by an increasing amount of time until the retries are an hour apart. Complicating things is that multiple uploads are not getting their acknowledgement packets back and multiple WUs are overlapping in trying to upload. The maximum time between retries used to be allowed to grow even longer, but multiple complaints from people doing folding earlier this year led to capping it at 1 hour.

Another issue is that each upload is handled separately, so they do not get sent one at a time. An enhancement request has been put in for the client in the past, no idea when or if that will be added.

Ultimately the client depends on a reliable network connection. If packets like ACK packets are regularly dropped there will be problems. The server will not wait for an acknowledgment that an ACK of the receipt of a WU was not received. That is inherent in the HTTP protocols. In a browser you would just refresh the page if it didn't load fully, the client attempts to resend the WU. A different way of handling this would require writing new code for both the servers and the client software. I personally don't know how best to change this, perhaps others will enter enhancement requests on the F@h GitHub site for issues that the developer can consider along with the volunteers helping on creating the next open source version of the client.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Top
Post Reply

Return to “Issues with a specific server”