Unable to upload to server 66.170.111.50

Moderators: Site Moderators, FAHC Science Team

comixgoddess
Posts: 88
Joined: Wed Apr 08, 2020 9:57 pm
Location: Pacific Northwest

Unable to upload to server 66.170.111.50

Post by comixgoddess »

Here is the section of my log showing the initial upload attempt --

04:25:48:WU00:FS00:0xa7:Completed 125000 out of 125000 steps (100%)
04:25:59:WU00:FS00:0xa7:Saving result file ../logfile_01.txt
04:25:59:WU00:FS00:0xa7:Saving result file frame0.trr
04:25:59:WU00:FS00:0xa7:Saving result file frame0.xtc
04:25:59:WU00:FS00:0xa7:Saving result file md.log
04:25:59:WU00:FS00:0xa7:Saving result file science.log
04:25:59:WU00:FS00:0xa7:Folding@home Core Shutdown: FINISHED_UNIT
04:25:59:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
04:25:59:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17411 run:0 clone:220 gen:0 core:0xa7 unit:0x000000dc000000000000440300000000
04:25:59:WU00:FS00:Uploading 9.94MiB to 66.170.111.50
04:25:59:WU00:FS00:Connecting to 66.170.111.50:8080
04:26:05:WU00:FS00:Upload 37.74%
04:26:11:WU00:FS00:Upload 80.52%
04:26:14:WU00:FS00:Upload complete
04:26:14:WU00:FS00:Server responded PLEASE_WAIT (464)
04:26:14:WARNING:WU00:FS00:Failed to send results, will try again later

So far, there have been 9 attempts to upload, all with the same result. According to the server stats page, the server is set to "Accept" and has 959.91GiB of space but no collection server.
Image
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: Unable to upload to server 66.170.111.50

Post by Neil-B »

Would recommend you check the wu status app and see if upload actually completed
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
comixgoddess
Posts: 88
Joined: Wed Apr 08, 2020 9:57 pm
Location: Pacific Northwest

Re: Unable to upload to server 66.170.111.50

Post by comixgoddess »

Neil-B wrote:Would recommend you check the wu status app and see if upload actually completed
Thanks for the suggestion, Neil-B. I hadn't thought of that. Unfortunately, the WU status page is showing one completion that was returned on 2020-09-22 15:12:13 to user Telefonica_SA. Now up to 24 attempts.
Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Unable to upload to server 66.170.111.50

Post by bruce »

The team that manages that server has been investigating what is (was?) wrong with that server. I don't have a progress report, but they have been aware that something needs (needed?) fixing.
comixgoddess
Posts: 88
Joined: Wed Apr 08, 2020 9:57 pm
Location: Pacific Northwest

Re: Unable to upload to server 66.170.111.50

Post by comixgoddess »

Unfortunately, this WU (project:17411 run:0 clone:220 gen:0) expired. My log now shows --

01:57:54:WARNING:WU00:FS00:Past final deadline 2020-12-20T01:57:53Z, dumping
01:57:54:WU00:FS00:Cleaning up

The time frame for this one seemed quite short --
Assigned 2020-12-18 T01:57:53Z
Timeout 2020-12-19 T01:57:53Z
Expiration 2020-12-20 T01:57:53Z
Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Unable to upload to server 66.170.111.50

Post by bruce »

Sorry to hear that.... but it does happen.
superpan
Posts: 11
Joined: Mon Dec 07, 2020 7:31 am

Re: Unable to upload to server 66.170.111.50

Post by superpan »

Same here... after 27 hourly attempts similar to...

00:57:08:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17407 run:0 clone:798 gen:0 core:0xa7 unit:0x0000031e00000000000043ff00000000
00:57:08:WU00:FS00:Uploading 9.94MiB to 66.170.111.50
00:57:08:WU00:FS00:Connecting to 66.170.111.50:8080
00:57:14:WU00:FS00:Upload 19.49%
00:57:20:WU00:FS00:Upload 40.24%
00:57:26:WU00:FS00:Upload 60.37%
00:57:32:WU00:FS00:Upload 80.49%
00:57:39:WU00:FS00:Upload complete
00:57:39:WU00:FS00:Server responded PLEASE_WAIT (464)

finally...

WARNING:WU00:FS00:Past final deadline 2020-12-20T02:07:36Z, dumping

Seems to be working OK now.
--
Pete

Image
gunnarre
Posts: 567
Joined: Sun May 24, 2020 7:23 pm
Location: Norway

Re: Unable to upload to server 66.170.111.50

Post by gunnarre »

This server is still not accepting work units. According to the server stats page, it has run out of disk space. https://apps.foldingathome.org/serverstats

Code: Select all

22:01:13:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
22:01:13:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:01:13:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:01:13:WU01:FS00:Connecting to 66.170.111.50:8080
22:01:19:WU01:FS00:Upload 8.19%
22:01:25:WU01:FS00:Upload 16.37%
22:01:31:WU01:FS00:Upload 23.93%
22:01:37:WU01:FS00:Upload 30.85%
22:01:43:WU01:FS00:Upload 39.04%
22:01:49:WU01:FS00:Upload 49.11%
22:01:55:WU01:FS00:Upload 56.67%
22:02:01:WU01:FS00:Upload 64.23%
22:02:07:WU01:FS00:Upload 71.15%
22:02:13:WU01:FS00:Upload 77.45%
22:02:19:WU01:FS00:Upload 90.04%
22:02:24:WU01:FS00:Upload complete
22:02:24:WU01:FS00:Server responded PLEASE_WAIT (464)
22:02:24:WARNING:WU01:FS00:Failed to send results, will try again later
22:02:24:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:02:24:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:02:24:WU01:FS00:Connecting to 66.170.111.50:8080
22:02:30:WU01:FS00:Upload 9.45%
22:02:36:WU01:FS00:Upload 19.52%
22:02:42:WU01:FS00:Upload 32.11%
22:02:48:WU01:FS00:Upload 41.56%
22:02:54:WU01:FS00:Upload 45.97%
22:03:00:WU01:FS00:Upload 51.00%
22:03:06:WU01:FS00:Upload 57.93%
22:03:12:WU01:FS00:Upload 67.38%
22:03:18:WU01:FS00:Upload 77.45%
22:03:24:WU01:FS00:Upload 83.75%
22:03:30:WU01:FS00:Upload 90.67%
22:03:36:WU01:FS00:Upload 98.23%
22:03:37:WU01:FS00:Upload complete
22:03:38:WU01:FS00:Server responded PLEASE_WAIT (464)
22:03:38:WARNING:WU01:FS00:Failed to send results, will try again later
22:03:38:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:03:38:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:03:38:WU01:FS00:Connecting to 66.170.111.50:8080
22:03:44:WU01:FS00:Upload 9.45%
22:03:50:WU01:FS00:Upload 15.11%
22:03:56:WU01:FS00:Upload 21.41%
22:04:02:WU01:FS00:Upload 27.71%
22:04:08:WU01:FS00:Upload 34.63%
22:04:14:WU01:FS00:Upload 44.71%
22:04:20:WU01:FS00:Upload 53.52%
22:04:26:WU01:FS00:Upload 61.71%
22:04:32:WU01:FS00:Upload 69.89%
22:04:38:WU01:FS00:Upload 78.08%
22:04:44:WU01:FS00:Upload 85.01%
22:04:50:WU01:FS00:Upload 90.04%
22:04:56:WU01:FS00:Upload 98.86%
22:04:58:WU01:FS00:Upload complete
22:04:59:WU01:FS00:Server responded PLEASE_WAIT (464)
22:04:59:WARNING:WU01:FS00:Failed to send results, will try again later
22:05:15:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:05:15:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:05:15:WU01:FS00:Connecting to 66.170.111.50:8080
22:05:21:WU01:FS00:Upload 7.56%
22:05:27:WU01:FS00:Upload 13.22%
22:05:33:WU01:FS00:Upload 18.89%
22:05:39:WU01:FS00:Upload 25.19%
22:05:45:WU01:FS00:Upload 32.74%
22:05:51:WU01:FS00:Upload 40.30%
22:05:57:WU01:FS00:Upload 49.74%
22:06:03:WU01:FS00:Upload 59.19%
22:06:09:WU01:FS00:Upload 68.63%
22:06:15:WU01:FS00:Upload 78.08%
22:06:21:WU01:FS00:Upload 88.15%
22:06:27:WU01:FS00:Upload 94.45%
22:06:32:WU01:FS00:Upload complete
22:06:32:WU01:FS00:Server responded PLEASE_WAIT (464)
22:06:32:WARNING:WU01:FS00:Failed to send results, will try again later
22:07:52:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:07:52:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:07:52:WU01:FS00:Connecting to 66.170.111.50:8080
22:07:58:WU01:FS00:Upload 6.93%
22:08:04:WU01:FS00:Upload 12.59%
22:08:10:WU01:FS00:Upload 19.52%
22:08:16:WU01:FS00:Upload 26.45%
22:08:22:WU01:FS00:Upload 37.78%
22:08:28:WU01:FS00:Upload 48.48%
22:08:34:WU01:FS00:Upload 56.67%
22:08:40:WU01:FS00:Upload 67.38%
22:08:46:WU01:FS00:Upload 77.45%
22:08:52:WU01:FS00:Upload 86.27%
22:08:58:WU01:FS00:Upload 93.82%
22:09:03:WU01:FS00:Upload complete
22:09:03:WU01:FS00:Server responded PLEASE_WAIT (464)
22:09:03:WARNING:WU01:FS00:Failed to send results, will try again later
22:12:06:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:12:06:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:12:06:WU01:FS00:Connecting to 66.170.111.50:8080
22:12:12:WU01:FS00:Upload 11.96%
22:12:18:WU01:FS00:Upload 18.26%
22:12:24:WU01:FS00:Upload 24.56%
22:12:30:WU01:FS00:Upload 31.48%
22:12:36:WU01:FS00:Upload 37.78%
22:12:42:WU01:FS00:Upload 42.82%
22:12:48:WU01:FS00:Upload 53.52%
22:12:54:WU01:FS00:Upload 63.60%
22:13:00:WU01:FS00:Upload 72.41%
22:13:06:WU01:FS00:Upload 79.34%
22:13:12:WU01:FS00:Upload 90.04%
22:13:18:WU01:FS00:Upload 98.23%
22:13:20:WU01:FS00:Upload complete
22:13:20:WU01:FS00:Server responded PLEASE_WAIT (464)
22:13:20:WARNING:WU01:FS00:Failed to send results, will try again later
22:18:58:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:18:58:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:18:58:WU01:FS00:Connecting to 66.170.111.50:8080
22:19:04:WU01:FS00:Upload 11.33%
22:19:10:WU01:FS00:Upload 20.15%
22:19:16:WU01:FS00:Upload 27.71%
22:19:22:WU01:FS00:Upload 33.37%
22:19:28:WU01:FS00:Upload 40.30%
22:19:34:WU01:FS00:Upload 49.11%
22:19:40:WU01:FS00:Upload 58.56%
22:19:46:WU01:FS00:Upload 72.41%
22:19:52:WU01:FS00:Upload 80.60%
22:19:58:WU01:FS00:Upload 92.56%
22:20:01:WU01:FS00:Upload complete
22:20:01:WU01:FS00:Server responded PLEASE_WAIT (464)
22:20:01:WARNING:WU01:FS00:Failed to send results, will try again later
22:30:03:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:30:03:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:30:03:WU01:FS00:Connecting to 66.170.111.50:8080
22:30:09:WU01:FS00:Upload 4.41%
22:30:15:WU01:FS00:Upload 16.37%
22:30:21:WU01:FS00:Upload 23.30%
22:30:27:WU01:FS00:Upload 28.96%
22:30:33:WU01:FS00:Upload 37.15%
22:30:39:WU01:FS00:Upload 48.48%
22:30:45:WU01:FS00:Upload 57.93%
22:30:51:WU01:FS00:Upload 62.34%
22:30:57:WU01:FS00:Upload 68.00%
22:31:03:WU01:FS00:Upload 76.19%
22:31:09:WU01:FS00:Upload 84.38%
22:31:15:WU01:FS00:Upload 93.19%
22:31:21:WU01:FS00:Upload complete
22:31:21:WU01:FS00:Server responded PLEASE_WAIT (464)
22:31:21:WARNING:WU01:FS00:Failed to send results, will try again later
22:48:00:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
22:48:00:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
22:48:00:WU01:FS00:Connecting to 66.170.111.50:8080
22:48:06:WU01:FS00:Upload 13.85%
22:48:12:WU01:FS00:Upload 23.30%
22:48:18:WU01:FS00:Upload 34.00%
22:48:24:WU01:FS00:Upload 47.23%
22:48:30:WU01:FS00:Upload 60.45%
22:48:36:WU01:FS00:Upload 72.41%
22:48:42:WU01:FS00:Upload 82.49%
22:48:48:WU01:FS00:Upload 98.86%
22:48:49:WU01:FS00:Upload complete
22:48:49:WU01:FS00:Server responded PLEASE_WAIT (464)
22:48:49:WARNING:WU01:FS00:Failed to send results, will try again later
23:17:02:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:17408 run:0 clone:124 gen:214 core:0xa7 unit:0x0000007c000000d60000440000000000
23:17:02:WU01:FS00:Uploading 9.93MiB to 66.170.111.50
23:17:02:WU01:FS00:Connecting to 66.170.111.50:8080
23:17:08:WU01:FS00:Upload 14.48%
23:17:14:WU01:FS00:Upload 26.45%
23:17:20:WU01:FS00:Upload 35.26%
23:17:26:WU01:FS00:Upload 50.37%
23:17:32:WU01:FS00:Upload 62.97%
23:17:38:WU01:FS00:Upload 77.45%
23:17:44:WU01:FS00:Upload 95.08%
23:17:46:WU01:FS00:Upload complete
23:17:46:WU01:FS00:Server responded PLEASE_WAIT (464)
23:17:46:WARNING:WU01:FS00:Failed to send results, will try again later
Image
Online: GTX 1660 Super, GTX 1080, GTX 1050 Ti 4G OC, RX580 + occasional CPU folding in the cold.
Offline: Radeon HD 7770, GTX 960, GTX 950
Joe_H
Site Admin
Posts: 7856
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Unable to upload to server 66.170.111.50

Post by Joe_H »

They are aware of the disk space having run out, and are working on it.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Foxbat
Posts: 95
Joined: Wed Dec 05, 2007 10:23 pm
Hardware configuration: Apple Mac Pro 1,1 2x2.66 GHz Dual-Core Xeon w/10 GB RAM | EVGA GTX 960, Zotac GTX 750 Ti | Ubuntu 14.04 LTS
Dell Precision T7400 2x3.0 GHz Quad-Core Xeon w/16 GB RAM | Zotac GTX 970 | Ubuntu 14.04 LTS
Apple iMac Retina 5K 4.00 GHz Core i7 w/8 GB RAM | OS X 10.11.3 (El Capitan)
Location: Michiana, USA

Re: Unable to upload to server 66.170.111.50

Post by Foxbat »

That's good to hear, my stuck WU {17415 (0, 14, 558)} for this server is on attempt 17 so far. The expiration is tomorrow at 13:24 UTC.

I'm curious, the Work Server is 66.170.111.50, but my FAH Client reports the Collection Server is 0.0.0.0, not an actual IPv4 address like my other WUs currently being Folded. Is this a "use any server" shortcut?
Image
Callaghan
Posts: 6
Joined: Tue Feb 02, 2021 8:04 am
Location: Somerset, UK

Re: Unable to upload to server 66.170.111.50

Post by Callaghan »

As above i too cannot upload to server 66.170.111.50 and have an expiration is tomorrow at 14:11:11Z. 17414 (0, 434, 480)
Joe_H
Site Admin
Posts: 7856
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Unable to upload to server 66.170.111.50

Post by Joe_H »

Foxbat wrote:I'm curious, the Work Server is 66.170.111.50, but my FAH Client reports the Collection Server is 0.0.0.0, not an actual IPv4 address like my other WUs currently being Folded. Is this a "use any server" shortcut?
A CS reported as 0.0.0.0 means none was assigned at the time the WU was downloaded to your system. All WUs need to go back to their WS, a CS is not always set for use as a backup when the WS is unavailable.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
new08
Posts: 188
Joined: Fri Jan 04, 2008 11:02 pm
Hardware configuration: Hewlett-Packard 1494 Win10 Build 1836
GeForce [MSI] GTX 950
Runs F@H Ver7.6.21
[As of Jan 2021]
Location: England

Re: Unable to upload to server 66.170.111.50

Post by new08 »

This is still sending out project work -as I wait for my last 17411 CPU unit to expire in two hours.
It's not the small points -it's the wasted effort and power usage.
I don't usually defer to F@H CPU work, but understood it was valued..
I never see this on BOINC, tbh.
Image
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Unable to upload to server 66.170.111.50

Post by bruce »

When a WU passes the timeout, a duplicate is issued. I don't think there's a limit to the number of duplicates that might be reissued. Eventually one of them will be completed and the project will move on. (Only the first one is needed for science.)

Most or all of the other copies that are in-flight will eventually be completed and should receive credit. The duplicated logs and results will eventually be discarded. When that cleanup processing happens probably matters.

The server doesn't have a message saying "Somebody else completed that WU before you did so we're discarding your result" That message SHOULD NEVER be issued, because you did nothing wrong but the server should still silently discard the duplicate results after awarding you credit.

FAH was designed to avoid issuing duplicate WUs except when they're really needed. Shortening the timeout interval does increase the frequency of WU duplication. In any case, no more duplicates should be sent after the first result is accepted.

This sounds like an unforeseen result of shortened timeouts.
susanreads
Posts: 23
Joined: Sat Apr 04, 2020 7:57 pm

Re: Unable to upload to server 66.170.111.50

Post by susanreads »

I had this a few days ago, with the unit eventually being dumped. I've finished two different units since without a problem, then this morning I was assigned another unit for the same project, from the same work server. Can I expect the same thing to happen? I have to leave it running overnight in order to finish before the deadline, and I resent doing that if it's just going to be thrown away.
Post Reply