GPUs stuck on Update_Core

If you think it might be a driver problem, see viewforum.php?f=79

Moderators: Site Moderators, FAHC Science Team

Arkharion
Posts: 3
Joined: Tue Apr 07, 2020 2:26 pm

GPUs stuck on Update_Core

Post by Arkharion »

Hi there,

I hope, I reached the correct Sub-Forum for my issue.
I also searched the forums for a solution but couldn't find something
that realy fits my issue. Or maybe I am too stupid or just blind ;-)

A few weeks ago I noticed, that my GPUs are not folding anymore. They are stuck in "Update_Core".
In the logs I can see, that they are downloading. But they always stop around 95%-99% and never reach 100%.
If I restart the FAH client, they start downloading again from the beginning.
I also tried to let them "sit" and waited for a few days - but the state does not change.
I also can see, that there are WUs assigned to all GPUs.
Folding on CPU works fine. All three GPUs are correctly recognized, both as CUDA and OpenGL. Logs do not show any errors.
I run the FAH Client on two different Hosts, issue is the same. Users are local Admins, FAH Client is allowed in Firewall.

What I tried to fix it:
- Updated FAH Client
- Updated GPU Drivers
- Deleted WorkUnit directories from GPUs
- Removed GPU Folding Flots and added them again
- Restart of FAH Client
- Restart of Host

Please advice on how to proceed to get my GPUs folding again.

Logs attached.

Host1:

Code: Select all

*********************** Log Started 2020-10-08T09:53:37Z ***********************
09:53:37:Trying to access database...
09:53:37:Successfully acquired database lock
09:53:37:Read GPUs.txt
09:53:37:Enabled folding slot 00: READY cpu:6
09:53:37:Enabled folding slot 01: READY gpu:0:GP104 [GeForce GTX 1080] 8873
09:53:37:Enabled folding slot 02: READY gpu:1:GP104 [GeForce GTX 1080] 8873
09:53:37:****************************** FAHClient ******************************
09:53:37:        Version: 7.6.13
09:53:37:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:53:37:      Copyright: 2020 foldingathome.org
09:53:37:       Homepage: https://foldingathome.org/
09:53:37:           Date: Apr 27 2020
09:53:37:           Time: 21:21:01
09:53:37:       Revision: 5a652817f46116b6e135503af97f18e094414e3b
09:53:37:         Branch: master
09:53:37:       Compiler: Visual C++ 2008
09:53:37:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:53:37:       Platform: win32 10
09:53:37:           Bits: 32
09:53:37:           Mode: Release
09:53:37:           Args: --open-web-control
09:53:37:         Config: C:\Users\Fabian\AppData\Roaming\FAHClient\config.xml
09:53:37:******************************** CBang ********************************
09:53:37:           Date: Apr 24 2020
09:53:37:           Time: 17:07:55
09:53:37:       Revision: ea081a3b3b0f4a37c4d0440b4f1bc184197c7797
09:53:37:         Branch: master
09:53:37:       Compiler: Visual C++ 2008
09:53:37:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:53:37:       Platform: win32 10
09:53:37:           Bits: 32
09:53:37:           Mode: Release
09:53:37:******************************* System ********************************
09:53:37:            CPU: Intel(R) Core(TM) i7-9800X CPU @ 3.80GHz
09:53:37:         CPU ID: GenuineIntel Family 6 Model 85 Stepping 4
09:53:37:           CPUs: 16
09:53:37:         Memory: 31.69GiB
09:53:37:    Free Memory: 21.20GiB
09:53:37:        Threads: WINDOWS_THREADS
09:53:37:     OS Version: 6.2
09:53:37:    Has Battery: false
09:53:37:     On Battery: false
09:53:37:     UTC Offset: 2
09:53:37:            PID: 9712
09:53:37:            CWD: C:\Users\Fabian\AppData\Roaming\FAHClient
09:53:37:  Win32 Service: false
09:53:37:             OS: Windows 10 Enterprise
09:53:37:        OS Arch: AMD64
09:53:37:           GPUs: 2
09:53:37:          GPU 0: Bus:23 Slot:0 Func:0 NVIDIA:8 GP104 [GeForce GTX 1080] 8873
09:53:37:          GPU 1: Bus:101 Slot:0 Func:0 NVIDIA:8 GP104 [GeForce GTX 1080] 8873
09:53:37:  CUDA Device 0: Platform:0 Device:0 Bus:23 Slot:0 Compute:6.1 Driver:11.1
09:53:37:  CUDA Device 1: Platform:0 Device:1 Bus:101 Slot:0 Compute:6.1 Driver:11.1
09:53:37:OpenCL Device 0: Platform:0 Device:0 Bus:23 Slot:0 Compute:1.2 Driver:456.55
09:53:37:OpenCL Device 1: Platform:0 Device:1 Bus:101 Slot:0 Compute:1.2 Driver:456.55
09:53:37:******************************* libFAH ********************************
09:53:37:           Date: Apr 15 2020
09:53:37:           Time: 14:53:14
09:53:37:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
09:53:37:         Branch: master
09:53:37:       Compiler: Visual C++ 2008
09:53:37:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:53:37:       Platform: win32 10
09:53:37:           Bits: 32
09:53:37:           Mode: Release
09:53:37:***********************************************************************
09:53:37:<config>
09:53:37:  <!-- Folding Slot Configuration -->
09:53:37:  <cause v='COVID_19'/>
09:53:37:
09:53:37:  <!-- Network -->
09:53:37:  <proxy v=':8080'/>
09:53:37:
09:53:37:  <!-- Remote Command Server -->
09:53:37:  <password v='*****'/>
09:53:37:
09:53:37:  <!-- Slot Control -->
09:53:37:  <power v='full'/>
09:53:37:
09:53:37:  <!-- User Information -->
09:53:37:  <passkey v='*****'/>
09:53:37:  <team v='225605'/>
09:53:37:  <user v='Arkharion'/>
09:53:37:
09:53:37:  <!-- Folding Slots -->
09:53:37:  <slot id='0' type='CPU'>
09:53:37:    <cpus v='6'/>
09:53:37:  </slot>
09:53:37:  <slot id='1' type='GPU'/>
09:53:37:  <slot id='2' type='GPU'/>
09:53:37:</config>
09:53:37:WU00:FS01:Downloading core from http://cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah
09:53:37:WU00:FS01:Connecting to cores.foldingathome.org:80
09:53:37:WU03:FS00:Starting
09:53:37:WU03:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\Fabian\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/win/64bit-avx-256/a7-0.0.19/Core_a7.fah/FahCore_a7.exe -dir 03 -suffix 01 -version 706 -lifeline 9712 -checkpoint 15 -np 6
09:53:37:WU03:FS00:Started FahCore on PID 16176
09:53:37:WU03:FS00:Core PID:8848
09:53:37:WU03:FS00:FahCore 0xa7 started
09:53:38:WU03:FS00:0xa7:*********************** Log Started 2020-10-08T09:53:37Z ***********************
09:53:38:WU03:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
09:53:38:WU03:FS00:0xa7:       Type: 0xa7
09:53:38:WU03:FS00:0xa7:       Core: Gromacs
09:53:38:WU03:FS00:0xa7:       Args: -dir 03 -suffix 01 -version 706 -lifeline 16176 -checkpoint 15 -np
09:53:38:WU03:FS00:0xa7:             6
09:53:38:WU03:FS00:0xa7:************************************ CBang *************************************
09:53:38:WU03:FS00:0xa7:       Date: Nov 27 2019
09:53:38:WU03:FS00:0xa7:       Time: 03:40:09
09:53:38:WU03:FS00:0xa7:   Revision: d25803215b59272441049dfa05a0a9bf7a6e3c48
09:53:38:WU03:FS00:0xa7:     Branch: master
09:53:38:WU03:FS00:0xa7:   Compiler: Visual C++ 2008
09:53:38:WU03:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:53:38:WU03:FS00:0xa7:   Platform: win32 10
09:53:38:WU03:FS00:0xa7:       Bits: 64
09:53:38:WU03:FS00:0xa7:       Mode: Release
09:53:38:WU03:FS00:0xa7:************************************ System ************************************
09:53:38:WU03:FS00:0xa7:        CPU: Intel(R) Core(TM) i7-9800X CPU @ 3.80GHz
09:53:38:WU03:FS00:0xa7:     CPU ID: GenuineIntel Family 6 Model 85 Stepping 4
09:53:38:WU03:FS00:0xa7:       CPUs: 16
09:53:38:WU03:FS00:0xa7:     Memory: 31.69GiB
09:53:38:WU03:FS00:0xa7:Free Memory: 21.18GiB
09:53:38:WU03:FS00:0xa7:    Threads: WINDOWS_THREADS
09:53:38:WU03:FS00:0xa7: OS Version: 6.2
09:53:38:WU03:FS00:0xa7:Has Battery: false
09:53:38:WU03:FS00:0xa7: On Battery: false
09:53:38:WU03:FS00:0xa7: UTC Offset: 2
09:53:38:WU03:FS00:0xa7:        PID: 8848
09:53:38:WU03:FS00:0xa7:        CWD: C:\Users\Fabian\AppData\Roaming\FAHClient\work
09:53:38:WU03:FS00:0xa7:******************************** Build - libFAH ********************************
09:53:38:WU03:FS00:0xa7:    Version: 0.0.19
09:53:38:WU03:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:53:38:WU03:FS00:0xa7:  Copyright: 2019 foldingathome.org
09:53:38:WU03:FS00:0xa7:   Homepage: https://foldingathome.org/
09:53:38:WU03:FS00:0xa7:       Date: Nov 25 2019
09:53:38:WU03:FS00:0xa7:       Time: 17:12:41
09:53:38:WU03:FS00:0xa7:   Revision: d5b5c747532224f986b7cd02c968ed9a20c16d6e
09:53:38:WU03:FS00:0xa7:     Branch: master
09:53:38:WU03:FS00:0xa7:   Compiler: Visual C++ 2008
09:53:38:WU03:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:53:38:WU03:FS00:0xa7:   Platform: win32 10
09:53:38:WU03:FS00:0xa7:       Bits: 64
09:53:38:WU03:FS00:0xa7:       Mode: Release
09:53:38:WU03:FS00:0xa7:************************************ Build *************************************
09:53:38:WU03:FS00:0xa7:       SIMD: avx_256
09:53:38:WU03:FS00:0xa7:********************************************************************************
09:53:38:WU03:FS00:0xa7:Project: 16804 (Run 73, Clone 307, Gen 39)
09:53:38:WU03:FS00:0xa7:Unit: 0x0000003182ed0b915e9deef229bb5220
09:53:38:WU03:FS00:0xa7:Digital signatures verified
09:53:38:WU03:FS00:0xa7:Calling: mdrun -s frame39.tpr -o frame39.trr -cpi state.cpt -cpt 15 -nt 6
09:53:38:WU03:FS00:0xa7:Steps: first=19500000 total=500000
09:53:38:WU03:FS00:0xa7:Completed 279207 out of 500000 steps (55%)
09:53:39:WU00:FS01:FahCore 22: Downloading 77.67MiB
09:53:39:8:127.0.0.1:New Web session
09:53:45:WU00:FS01:FahCore 22: 0.72%
09:53:51:WU00:FS01:FahCore 22: 1.13%
09:53:57:WU00:FS01:FahCore 22: 1.85%
09:54:04:WU00:FS01:FahCore 22: 2.49%
09:54:06:WU03:FS00:0xa7:Completed 280000 out of 500000 steps (56%)
09:54:11:WU00:FS01:FahCore 22: 2.90%
09:54:17:WU00:FS01:FahCore 22: 3.38%
09:54:23:WU00:FS01:FahCore 22: 3.94%
09:54:29:WU00:FS01:FahCore 22: 4.27%
09:54:35:WU00:FS01:FahCore 22: 4.75%
09:54:37:WU01:FS02:Downloading core from http://cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah
09:54:37:WU01:FS02:Connecting to cores.foldingathome.org:80
09:54:38:WU01:FS02:FahCore 22: Downloading 77.67MiB
09:54:41:WU00:FS01:FahCore 22: 5.23%
09:54:44:WU01:FS02:FahCore 22: 0.72%
09:54:49:WU00:FS01:FahCore 22: 5.79%
09:54:50:WU01:FS02:FahCore 22: 1.21%
09:54:55:WU00:FS01:FahCore 22: 6.12%
09:54:56:WU01:FS02:FahCore 22: 1.77%
09:55:01:WU00:FS01:FahCore 22: 6.60%
09:55:02:WU01:FS02:FahCore 22: 2.17%
09:55:08:WU00:FS01:FahCore 22: 7.24%
09:55:08:WU01:FS02:FahCore 22: 2.58%
09:55:14:WU01:FS02:FahCore 22: 3.14%
09:55:15:WU00:FS01:FahCore 22: 7.73%
09:55:20:WU01:FS02:FahCore 22: 3.70%
09:55:21:WU00:FS01:FahCore 22: 8.21%
09:55:26:WU01:FS02:FahCore 22: 4.35%
09:55:27:WU00:FS01:FahCore 22: 8.85%
09:55:32:WU01:FS02:FahCore 22: 4.91%
09:55:33:WU00:FS01:FahCore 22: 9.25%
09:55:38:WU01:FS02:FahCore 22: 5.47%
09:55:40:WU00:FS01:FahCore 22: 9.74%
09:55:44:WU01:FS02:FahCore 22: 6.20%
09:55:46:WU00:FS01:FahCore 22: 10.14%
09:55:50:WU01:FS02:FahCore 22: 6.92%
09:55:52:WU00:FS01:FahCore 22: 10.62%
09:55:56:WU01:FS02:FahCore 22: 7.48%
09:55:59:WU00:FS01:FahCore 22: 10.94%
09:56:02:WU01:FS02:FahCore 22: 8.05%
09:56:06:WU00:FS01:FahCore 22: 11.51%
09:56:08:WU01:FS02:FahCore 22: 8.69%
09:56:12:WU00:FS01:FahCore 22: 11.91%
09:56:14:WU01:FS02:FahCore 22: 9.25%
09:56:18:WU00:FS01:FahCore 22: 12.55%
09:56:20:WU01:FS02:FahCore 22: 9.90%
09:56:24:WU00:FS01:FahCore 22: 12.96%
09:56:26:WU01:FS02:FahCore 22: 10.46%
09:56:30:WU00:FS01:FahCore 22: 13.44%
09:56:32:WU01:FS02:FahCore 22: 10.78%
09:56:36:WU00:FS01:FahCore 22: 14.08%
09:56:38:WU01:FS02:FahCore 22: 11.35%
09:56:42:WU00:FS01:FahCore 22: 14.81%
09:56:44:WU01:FS02:FahCore 22: 12.15%
09:56:48:WU00:FS01:FahCore 22: 15.21%
09:56:50:WU01:FS02:FahCore 22: 12.88%
09:56:54:WU00:FS01:FahCore 22: 15.85%
09:56:56:WU01:FS02:FahCore 22: 13.44%
09:57:00:WU03:FS00:0xa7:Completed 285000 out of 500000 steps (57%)
09:57:00:WU00:FS01:FahCore 22: 16.50%
09:57:02:WU01:FS02:FahCore 22: 13.92%
09:57:06:WU00:FS01:FahCore 22: 16.98%
09:57:08:WU01:FS02:FahCore 22: 14.32%
09:57:12:WU00:FS01:FahCore 22: 17.46%
09:57:14:WU01:FS02:FahCore 22: 14.89%
09:57:18:WU00:FS01:FahCore 22: 17.95%
09:57:20:WU01:FS02:FahCore 22: 15.21%
09:57:24:WU00:FS01:FahCore 22: 18.67%
09:57:26:WU01:FS02:FahCore 22: 15.53%
09:57:30:WU00:FS01:FahCore 22: 19.23%
09:57:33:WU01:FS02:FahCore 22: 15.85%
09:57:36:WU00:FS01:FahCore 22: 19.80%
09:57:39:WU01:FS02:FahCore 22: 16.26%
09:57:42:WU00:FS01:FahCore 22: 20.36%
09:57:45:WU01:FS02:FahCore 22: 16.74%
09:57:48:WU00:FS01:FahCore 22: 20.84%
09:57:51:WU01:FS02:FahCore 22: 17.30%
09:57:54:WU00:FS01:FahCore 22: 21.33%
09:57:57:WU01:FS02:FahCore 22: 18.03%
09:58:00:WU00:FS01:FahCore 22: 21.89%
09:58:03:WU01:FS02:FahCore 22: 18.43%
09:58:06:WU00:FS01:FahCore 22: 22.61%
09:58:09:WU01:FS02:FahCore 22: 19.15%
09:58:12:WU00:FS01:FahCore 22: 23.02%
09:58:15:WU01:FS02:FahCore 22: 19.64%
09:58:18:WU00:FS01:FahCore 22: 23.34%
09:58:21:WU01:FS02:FahCore 22: 20.20%
09:58:24:WU00:FS01:FahCore 22: 23.98%
09:58:27:WU01:FS02:FahCore 22: 20.84%
09:58:30:WU00:FS01:FahCore 22: 24.46%
09:58:33:WU01:FS02:FahCore 22: 21.49%
09:58:37:WU00:FS01:FahCore 22: 24.87%
09:58:40:WU01:FS02:FahCore 22: 21.97%
09:58:43:WU00:FS01:FahCore 22: 25.35%
09:58:46:WU01:FS02:FahCore 22: 22.85%
09:58:49:WU00:FS01:FahCore 22: 25.83%
09:58:52:WU01:FS02:FahCore 22: 23.98%
09:58:55:WU00:FS01:FahCore 22: 26.31%
09:58:58:WU01:FS02:FahCore 22: 24.71%
09:59:01:WU00:FS01:FahCore 22: 26.96%
09:59:04:WU01:FS02:FahCore 22: 25.51%
09:59:07:WU00:FS01:FahCore 22: 27.52%
09:59:10:WU01:FS02:FahCore 22: 25.99%
09:59:13:WU00:FS01:FahCore 22: 28.17%
09:59:16:WU01:FS02:FahCore 22: 26.39%
09:59:20:WU00:FS01:FahCore 22: 28.73%
09:59:22:WU01:FS02:FahCore 22: 26.96%
09:59:26:WU00:FS01:FahCore 22: 29.37%
09:59:28:WU01:FS02:FahCore 22: 27.60%
09:59:32:WU00:FS01:FahCore 22: 30.50%
09:59:35:WU01:FS02:FahCore 22: 28.25%
09:59:38:WU00:FS01:FahCore 22: 31.06%
09:59:41:WU01:FS02:FahCore 22: 28.89%
09:59:44:WU00:FS01:FahCore 22: 31.63%
09:59:47:WU01:FS02:FahCore 22: 29.37%
09:59:50:WU00:FS01:FahCore 22: 32.11%
09:59:51:WU03:FS00:0xa7:Completed 290000 out of 500000 steps (58%)
09:59:53:WU01:FS02:FahCore 22: 30.10%
09:59:56:WU00:FS01:FahCore 22: 32.75%
09:59:59:WU01:FS02:FahCore 22: 30.58%
10:00:02:WU00:FS01:FahCore 22: 33.15%
10:00:05:WU01:FS02:FahCore 22: 31.22%
10:00:09:WU00:FS01:FahCore 22: 33.56%
10:00:11:WU01:FS02:FahCore 22: 31.87%
10:00:15:WU00:FS01:FahCore 22: 33.88%
10:00:17:WU01:FS02:FahCore 22: 32.27%
10:00:21:WU00:FS01:FahCore 22: 34.52%
10:00:23:WU01:FS02:FahCore 22: 32.83%
10:00:27:WU00:FS01:FahCore 22: 35.17%
10:00:29:WU01:FS02:FahCore 22: 33.48%
10:00:33:WU00:FS01:FahCore 22: 35.65%
10:00:35:WU01:FS02:FahCore 22: 34.12%
10:00:39:WU00:FS01:FahCore 22: 36.13%
10:00:41:WU01:FS02:FahCore 22: 34.60%
10:00:45:WU00:FS01:FahCore 22: 36.70%
10:00:47:WU01:FS02:FahCore 22: 35.25%
10:00:51:WU00:FS01:FahCore 22: 37.34%
10:00:53:WU01:FS02:FahCore 22: 36.05%
10:00:57:WU00:FS01:FahCore 22: 37.66%
10:00:59:WU01:FS02:FahCore 22: 36.45%
10:01:03:WU00:FS01:FahCore 22: 38.14%
10:01:05:WU01:FS02:FahCore 22: 36.94%
10:01:09:WU00:FS01:FahCore 22: 38.79%
10:01:12:WU01:FS02:FahCore 22: 37.50%
10:01:15:WU00:FS01:FahCore 22: 39.27%
10:01:18:WU01:FS02:FahCore 22: 37.98%
10:01:21:WU00:FS01:FahCore 22: 40.16%
10:01:24:WU01:FS02:FahCore 22: 38.63%
10:01:27:WU00:FS01:FahCore 22: 40.80%
10:01:30:WU01:FS02:FahCore 22: 39.27%
10:01:33:WU00:FS01:FahCore 22: 41.77%
10:01:36:WU01:FS02:FahCore 22: 39.75%
10:01:39:WU00:FS01:FahCore 22: 42.57%
10:01:42:WU01:FS02:FahCore 22: 40.24%
10:01:45:WU00:FS01:FahCore 22: 43.37%
10:01:48:WU01:FS02:FahCore 22: 40.72%
10:01:52:WU00:FS01:FahCore 22: 44.02%
10:01:54:WU01:FS02:FahCore 22: 41.36%
10:01:58:WU00:FS01:FahCore 22: 44.74%
10:02:00:WU01:FS02:FahCore 22: 41.85%
10:02:04:WU00:FS01:FahCore 22: 45.39%
10:02:06:WU01:FS02:FahCore 22: 42.25%
10:02:10:WU00:FS01:FahCore 22: 45.87%
10:02:12:WU01:FS02:FahCore 22: 42.89%
10:02:16:WU00:FS01:FahCore 22: 46.35%
10:02:18:WU01:FS02:FahCore 22: 43.70%
10:02:22:WU00:FS01:FahCore 22: 47.08%
10:02:24:WU01:FS02:FahCore 22: 44.42%
10:02:28:WU00:FS01:FahCore 22: 47.40%
10:02:30:WU01:FS02:FahCore 22: 45.23%
10:02:34:WU00:FS01:FahCore 22: 47.88%
10:02:37:WU01:FS02:FahCore 22: 45.71%
10:02:40:WU00:FS01:FahCore 22: 48.44%
10:02:42:WU03:FS00:0xa7:Completed 295000 out of 500000 steps (59%)
10:02:43:WU01:FS02:FahCore 22: 46.27%
10:02:46:WU00:FS01:FahCore 22: 49.25%
10:02:49:WU01:FS02:FahCore 22: 47.56%
10:02:52:WU00:FS01:FahCore 22: 49.73%
10:02:55:WU01:FS02:FahCore 22: 48.12%
10:02:59:WU00:FS01:FahCore 22: 50.30%
10:03:02:WU01:FS02:FahCore 22: 48.44%
10:03:05:WU00:FS01:FahCore 22: 50.70%
10:03:08:WU01:FS02:FahCore 22: 48.85%
10:03:11:WU00:FS01:FahCore 22: 51.10%
10:03:14:WU01:FS02:FahCore 22: 49.33%
10:03:17:WU00:FS01:FahCore 22: 51.50%
10:03:21:WU01:FS02:FahCore 22: 49.89%
10:03:23:WU00:FS01:FahCore 22: 51.99%
10:03:27:WU01:FS02:FahCore 22: 50.38%
10:03:29:WU00:FS01:FahCore 22: 52.47%
10:03:34:WU01:FS02:FahCore 22: 50.86%
10:03:36:WU00:FS01:FahCore 22: 52.95%
10:03:40:WU01:FS02:FahCore 22: 51.18%
10:03:42:WU00:FS01:FahCore 22: 53.27%
10:03:46:WU01:FS02:FahCore 22: 51.66%
10:03:48:WU00:FS01:FahCore 22: 53.68%
10:03:52:WU01:FS02:FahCore 22: 52.15%
10:03:54:WU00:FS01:FahCore 22: 54.16%
10:03:58:WU01:FS02:FahCore 22: 52.79%
10:04:00:WU00:FS01:FahCore 22: 54.80%
10:04:04:WU01:FS02:FahCore 22: 53.68%
10:04:06:WU00:FS01:FahCore 22: 55.37%
10:04:10:WU01:FS02:FahCore 22: 54.40%
10:04:12:WU00:FS01:FahCore 22: 55.93%
10:04:16:WU01:FS02:FahCore 22: 54.88%
10:04:18:WU00:FS01:FahCore 22: 56.57%
10:04:22:WU01:FS02:FahCore 22: 55.28%
10:04:24:WU00:FS01:FahCore 22: 57.30%
10:04:28:WU01:FS02:FahCore 22: 56.01%
10:04:30:WU00:FS01:FahCore 22: 57.86%
10:04:34:WU01:FS02:FahCore 22: 56.65%
10:04:36:WU00:FS01:FahCore 22: 58.42%
10:04:40:WU01:FS02:FahCore 22: 57.14%
10:04:42:WU00:FS01:FahCore 22: 58.91%
10:04:46:WU01:FS02:FahCore 22: 57.62%
10:04:48:WU00:FS01:FahCore 22: 59.31%
10:04:52:WU01:FS02:FahCore 22: 58.10%
10:04:54:WU00:FS01:FahCore 22: 59.79%
10:04:58:WU01:FS02:FahCore 22: 58.58%
10:05:01:WU00:FS01:FahCore 22: 60.19%
10:05:04:WU01:FS02:FahCore 22: 59.15%
10:05:07:WU00:FS01:FahCore 22: 60.68%
10:05:10:WU01:FS02:FahCore 22: 59.95%
10:05:13:WU00:FS01:FahCore 22: 61.24%
10:05:16:WU01:FS02:FahCore 22: 60.43%
10:05:19:WU00:FS01:FahCore 22: 61.88%
10:05:22:WU01:FS02:FahCore 22: 61.08%
10:05:25:WU00:FS01:FahCore 22: 62.45%
10:05:28:WU01:FS02:FahCore 22: 61.40%
10:05:31:WU00:FS01:FahCore 22: 63.09%
10:05:33:WU03:FS00:0xa7:Completed 300000 out of 500000 steps (60%)
10:05:34:WU01:FS02:FahCore 22: 61.96%
10:05:37:WU00:FS01:FahCore 22: 63.57%
10:05:40:WU01:FS02:FahCore 22: 62.45%
10:05:43:WU00:FS01:FahCore 22: 64.14%
10:05:46:WU01:FS02:FahCore 22: 62.93%
10:05:49:WU00:FS01:FahCore 22: 64.86%
10:05:52:WU01:FS02:FahCore 22: 63.49%
10:05:55:WU00:FS01:FahCore 22: 65.59%
10:05:58:WU01:FS02:FahCore 22: 63.90%
10:06:01:WU00:FS01:FahCore 22: 66.23%
10:06:04:WU01:FS02:FahCore 22: 64.54%
10:06:07:WU00:FS01:FahCore 22: 66.63%
10:06:10:WU01:FS02:FahCore 22: 65.18%
10:06:13:WU00:FS01:FahCore 22: 67.03%
10:06:16:WU01:FS02:FahCore 22: 65.67%
10:06:19:WU00:FS01:FahCore 22: 67.52%
10:06:22:WU01:FS02:FahCore 22: 66.23%
10:06:25:WU00:FS01:FahCore 22: 67.92%
10:06:28:WU01:FS02:FahCore 22: 66.71%
10:06:31:WU00:FS01:FahCore 22: 68.48%
10:06:34:WU01:FS02:FahCore 22: 67.19%
10:06:37:WU00:FS01:FahCore 22: 69.13%
10:06:40:WU01:FS02:FahCore 22: 67.84%
10:06:43:WU00:FS01:FahCore 22: 69.85%
10:06:46:WU01:FS02:FahCore 22: 68.48%
10:06:49:WU00:FS01:FahCore 22: 70.57%
10:06:52:WU01:FS02:FahCore 22: 69.13%
10:06:55:WU00:FS01:FahCore 22: 71.14%
10:06:58:WU01:FS02:FahCore 22: 69.53%
10:07:01:WU00:FS01:FahCore 22: 71.70%
10:07:04:WU01:FS02:FahCore 22: 70.17%
10:07:07:WU00:FS01:FahCore 22: 72.26%
10:07:10:WU01:FS02:FahCore 22: 70.74%
10:07:13:WU00:FS01:FahCore 22: 72.83%
10:07:16:WU01:FS02:FahCore 22: 71.22%
10:07:19:WU00:FS01:FahCore 22: 73.47%
10:07:22:WU01:FS02:FahCore 22: 71.70%
10:07:25:WU00:FS01:FahCore 22: 73.95%
10:07:28:WU01:FS02:FahCore 22: 72.43%
10:07:31:WU00:FS01:FahCore 22: 74.44%
10:07:34:WU01:FS02:FahCore 22: 72.99%
10:07:37:WU00:FS01:FahCore 22: 75.00%
10:07:40:WU01:FS02:FahCore 22: 73.47%
10:07:43:WU00:FS01:FahCore 22: 75.56%
10:07:46:WU01:FS02:FahCore 22: 73.95%
10:07:49:WU00:FS01:FahCore 22: 76.13%
10:07:52:WU01:FS02:FahCore 22: 74.28%
10:07:55:WU00:FS01:FahCore 22: 76.53%
10:07:59:WU01:FS02:FahCore 22: 74.76%
10:08:01:WU00:FS01:FahCore 22: 77.01%
10:08:05:WU01:FS02:FahCore 22: 75.16%
10:08:07:WU00:FS01:FahCore 22: 77.58%
10:08:11:WU01:FS02:FahCore 22: 75.72%
10:08:13:WU00:FS01:FahCore 22: 78.22%
10:08:17:WU01:FS02:FahCore 22: 76.29%
10:08:19:WU00:FS01:FahCore 22: 78.54%
10:08:23:WU01:FS02:FahCore 22: 76.85%
10:08:25:WU03:FS00:0xa7:Completed 305000 out of 500000 steps (61%)
10:08:25:WU00:FS01:FahCore 22: 79.02%
10:08:29:WU01:FS02:FahCore 22: 77.49%
10:08:31:WU00:FS01:FahCore 22: 79.75%
10:08:35:WU01:FS02:FahCore 22: 78.06%
10:08:37:WU00:FS01:FahCore 22: 80.31%
10:08:41:WU01:FS02:FahCore 22: 78.54%
10:08:43:WU00:FS01:FahCore 22: 80.71%
10:08:47:WU01:FS02:FahCore 22: 78.94%
10:08:49:WU00:FS01:FahCore 22: 81.36%
10:08:53:WU01:FS02:FahCore 22: 79.35%
10:08:55:WU00:FS01:FahCore 22: 82.08%
10:08:59:WU01:FS02:FahCore 22: 79.75%
10:09:02:WU00:FS01:FahCore 22: 82.56%
10:09:06:WU01:FS02:FahCore 22: 80.31%
10:09:08:WU00:FS01:FahCore 22: 83.05%
10:09:12:WU01:FS02:FahCore 22: 80.87%
10:09:14:WU00:FS01:FahCore 22: 83.53%
10:09:18:WU01:FS02:FahCore 22: 81.44%
10:09:20:WU00:FS01:FahCore 22: 84.09%
10:09:24:WU01:FS02:FahCore 22: 82.08%
10:09:26:WU00:FS01:FahCore 22: 84.66%
10:09:30:WU01:FS02:FahCore 22: 82.56%
10:09:32:WU00:FS01:FahCore 22: 85.22%
10:09:36:WU01:FS02:FahCore 22: 83.05%
10:09:38:WU00:FS01:FahCore 22: 85.70%
10:09:42:WU01:FS02:FahCore 22: 83.61%
10:09:44:WU00:FS01:FahCore 22: 86.03%
10:09:48:WU01:FS02:FahCore 22: 84.01%
10:09:50:WU00:FS01:FahCore 22: 86.67%
10:09:55:WU01:FS02:FahCore 22: 84.42%
10:09:56:WU00:FS01:FahCore 22: 87.15%
10:10:01:WU01:FS02:FahCore 22: 84.98%
10:10:02:WU00:FS01:FahCore 22: 87.63%
10:10:07:WU01:FS02:FahCore 22: 85.54%
10:10:08:WU00:FS01:FahCore 22: 88.36%
10:10:13:WU01:FS02:FahCore 22: 86.03%
10:10:14:WU00:FS01:FahCore 22: 89.00%
10:10:19:WU01:FS02:FahCore 22: 86.51%
10:10:20:WU00:FS01:FahCore 22: 89.49%
10:10:25:WU01:FS02:FahCore 22: 86.99%
10:10:26:WU00:FS01:FahCore 22: 89.97%
10:10:31:WU01:FS02:FahCore 22: 88.04%
10:10:32:WU00:FS01:FahCore 22: 90.53%
10:10:37:WU01:FS02:FahCore 22: 88.44%
10:10:38:WU00:FS01:FahCore 22: 91.34%
10:10:43:WU01:FS02:FahCore 22: 89.00%
10:10:44:WU00:FS01:FahCore 22: 92.14%
10:10:49:WU01:FS02:FahCore 22: 89.49%
10:10:51:WU00:FS01:FahCore 22: 92.78%
10:10:55:WU01:FS02:FahCore 22: 90.13%
10:10:57:WU00:FS01:FahCore 22: 93.43%
10:11:02:WU01:FS02:FahCore 22: 90.53%
10:11:04:WU00:FS01:FahCore 22: 93.91%
10:11:08:WU01:FS02:FahCore 22: 90.93%
10:11:10:WU00:FS01:FahCore 22: 94.47%
10:11:14:WU01:FS02:FahCore 22: 91.58%
10:11:16:WU00:FS01:FahCore 22: 95.12%
10:11:19:WU03:FS00:0xa7:Completed 310000 out of 500000 steps (62%)
10:11:20:WU01:FS02:FahCore 22: 92.46%
10:11:22:WU00:FS01:FahCore 22: 95.76%
10:11:26:WU01:FS02:FahCore 22: 93.03%
10:11:28:WU00:FS01:FahCore 22: 96.49%
10:11:32:WU01:FS02:FahCore 22: 93.67%
10:11:34:WU00:FS01:FahCore 22: 97.29%
10:11:38:WU01:FS02:FahCore 22: 94.15%
10:11:40:WU00:FS01:FahCore 22: 97.85%
10:11:45:WU01:FS02:FahCore 22: 94.64%
10:11:46:WU00:FS01:FahCore 22: 98.34%
10:11:51:WU01:FS02:FahCore 22: 95.12%
10:11:52:WU00:FS01:FahCore 22: 98.98%
10:11:57:WU01:FS02:FahCore 22: 95.60%
10:11:58:WU00:FS01:FahCore 22: 99.62%
10:12:03:WU01:FS02:FahCore 22: 96.16%
10:12:09:WU01:FS02:FahCore 22: 96.89%
10:12:15:WU01:FS02:FahCore 22: 97.45%
10:12:21:WU01:FS02:FahCore 22: 98.10%
10:12:27:WU01:FS02:FahCore 22: 98.58%
10:12:33:WU01:FS02:FahCore 22: 99.14%
10:12:39:WU01:FS02:FahCore 22: 99.62%
10:14:08:WU03:FS00:0xa7:Completed 315000 out of 500000 steps (63%)
10:16:56:WU03:FS00:0xa7:Completed 320000 out of 500000 steps (64%)
10:19:46:WU03:FS00:0xa7:Completed 325000 out of 500000 steps (65%)
Host2 (just restarted - Core currently downloading. Not expecting a different result):

Code: Select all

*********************** Log Started 2020-10-08T10:18:22Z ***********************
10:18:22:Trying to access database...
10:18:22:Successfully acquired database lock
10:18:22:Read GPUs.txt
10:18:22:Enabled folding slot 00: PAUSED cpu:4 (by user)
10:18:23:Enabled folding slot 01: PAUSED gpu:0:GP106 [GeForce GTX 1060 6GB] 4372 (by user)
10:18:23:****************************** FAHClient ******************************
10:18:23:        Version: 7.6.13
10:18:23:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
10:18:23:      Copyright: 2020 foldingathome.org
10:18:23:       Homepage: https://foldingathome.org/
10:18:23:           Date: Apr 27 2020
10:18:23:           Time: 21:21:01
10:18:23:       Revision: 5a652817f46116b6e135503af97f18e094414e3b
10:18:23:         Branch: master
10:18:23:       Compiler: Visual C++ 2008
10:18:23:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
10:18:23:       Platform: win32 10
10:18:23:           Bits: 32
10:18:23:           Mode: Release
10:18:23:           Args: --open-web-control
10:18:23:         Config: C:\Users\Fabian\AppData\Roaming\FAHClient\config.xml
10:18:23:******************************** CBang ********************************
10:18:23:           Date: Apr 24 2020
10:18:23:           Time: 17:07:55
10:18:23:       Revision: ea081a3b3b0f4a37c4d0440b4f1bc184197c7797
10:18:23:         Branch: master
10:18:23:       Compiler: Visual C++ 2008
10:18:23:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
10:18:23:       Platform: win32 10
10:18:23:           Bits: 32
10:18:23:           Mode: Release
10:18:23:******************************* System ********************************
10:18:23:            CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
10:18:23:         CPU ID: GenuineIntel Family 6 Model 94 Stepping 3
10:18:23:           CPUs: 8
10:18:23:         Memory: 15.94GiB
10:18:23:    Free Memory: 11.96GiB
10:18:23:        Threads: WINDOWS_THREADS
10:18:23:     OS Version: 6.2
10:18:23:    Has Battery: false
10:18:23:     On Battery: false
10:18:23:     UTC Offset: 2
10:18:23:            PID: 3084
10:18:23:            CWD: C:\Users\Fabian\AppData\Roaming\FAHClient
10:18:23:  Win32 Service: false
10:18:23:             OS: Windows 10 Enterprise
10:18:23:        OS Arch: AMD64
10:18:23:           GPUs: 1
10:18:23:          GPU 0: Bus:2 Slot:0 Func:0 NVIDIA:7 GP106 [GeForce GTX 1060 6GB] 4372
10:18:23:  CUDA Device 0: Platform:0 Device:0 Bus:2 Slot:0 Compute:6.1 Driver:11.1
10:18:23:OpenCL Device 0: Platform:0 Device:0 Bus:2 Slot:0 Compute:1.2 Driver:456.71
10:18:23:******************************* libFAH ********************************
10:18:23:           Date: Apr 15 2020
10:18:23:           Time: 14:53:14
10:18:23:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
10:18:23:         Branch: master
10:18:23:       Compiler: Visual C++ 2008
10:18:23:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
10:18:23:       Platform: win32 10
10:18:23:           Bits: 32
10:18:23:           Mode: Release
10:18:23:***********************************************************************
10:18:23:<config>
10:18:23:  <!-- Folding Slot Configuration -->
10:18:23:  <cause v='COVID_19'/>
10:18:23:
10:18:23:  <!-- Network -->
10:18:23:  <proxy v=':8080'/>
10:18:23:
10:18:23:  <!-- Remote Command Server -->
10:18:23:  <password v='*****'/>
10:18:23:
10:18:23:  <!-- Slot Control -->
10:18:23:  <power v='FULL'/>
10:18:23:
10:18:23:  <!-- User Information -->
10:18:23:  <passkey v='*****'/>
10:18:23:  <team v='225605'/>
10:18:23:  <user v='Arkharion'/>
10:18:23:
10:18:23:  <!-- Folding Slots -->
10:18:23:  <slot id='0' type='CPU'>
10:18:23:    <cpus v='4'/>
10:18:23:    <paused v='true'/>
10:18:23:  </slot>
10:18:23:  <slot id='1' type='GPU'>
10:18:23:    <paused v='true'/>
10:18:23:  </slot>
10:18:23:</config>
10:18:24:4:127.0.0.1:New Web session
10:18:36:FS00:Unpaused
10:18:36:FS01:Unpaused
10:18:36:WU01:FS01:Downloading core from http://cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah
10:18:36:WU01:FS01:Connecting to cores.foldingathome.org:80
10:18:36:WU00:FS00:Starting
10:18:36:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\Fabian\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/win/64bit-avx-256/a7-0.0.19/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 706 -lifeline 3084 -checkpoint 15 -np 4
10:18:36:WU00:FS00:Started FahCore on PID 4256
10:18:36:WU00:FS00:Core PID:9188
10:18:36:WU00:FS00:FahCore 0xa7 started
10:18:36:WU00:FS00:0xa7:*********************** Log Started 2020-10-08T10:18:36Z ***********************
10:18:36:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
10:18:36:WU00:FS00:0xa7:       Type: 0xa7
10:18:36:WU00:FS00:0xa7:       Core: Gromacs
10:18:36:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 706 -lifeline 4256 -checkpoint 15 -np 4
10:18:36:WU00:FS00:0xa7:************************************ CBang *************************************
10:18:36:WU00:FS00:0xa7:       Date: Nov 27 2019
10:18:36:WU00:FS00:0xa7:       Time: 03:40:09
10:18:36:WU00:FS00:0xa7:   Revision: d25803215b59272441049dfa05a0a9bf7a6e3c48
10:18:36:WU00:FS00:0xa7:     Branch: master
10:18:36:WU01:FS01:FahCore 22: Downloading 77.67MiB
10:18:36:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
10:18:36:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
10:18:36:WU00:FS00:0xa7:   Platform: win32 10
10:18:36:WU00:FS00:0xa7:       Bits: 64
10:18:36:WU00:FS00:0xa7:       Mode: Release
10:18:36:WU00:FS00:0xa7:************************************ System ************************************
10:18:36:WU00:FS00:0xa7:        CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
10:18:36:WU00:FS00:0xa7:     CPU ID: GenuineIntel Family 6 Model 94 Stepping 3
10:18:36:WU00:FS00:0xa7:       CPUs: 8
10:18:36:WU00:FS00:0xa7:     Memory: 15.94GiB
10:18:36:WU00:FS00:0xa7:Free Memory: 11.91GiB
10:18:36:WU00:FS00:0xa7:    Threads: WINDOWS_THREADS
10:18:36:WU00:FS00:0xa7: OS Version: 6.2
10:18:36:WU00:FS00:0xa7:Has Battery: false
10:18:36:WU00:FS00:0xa7: On Battery: false
10:18:36:WU00:FS00:0xa7: UTC Offset: 2
10:18:36:WU00:FS00:0xa7:        PID: 9188
10:18:36:WU00:FS00:0xa7:        CWD: C:\Users\Fabian\AppData\Roaming\FAHClient\work
10:18:36:WU00:FS00:0xa7:******************************** Build - libFAH ********************************
10:18:36:WU00:FS00:0xa7:    Version: 0.0.19
10:18:36:WU00:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
10:18:36:WU00:FS00:0xa7:  Copyright: 2019 foldingathome.org
10:18:36:WU00:FS00:0xa7:   Homepage: https://foldingathome.org/
10:18:36:WU00:FS00:0xa7:       Date: Nov 25 2019
10:18:36:WU00:FS00:0xa7:       Time: 17:12:41
10:18:36:WU00:FS00:0xa7:   Revision: d5b5c747532224f986b7cd02c968ed9a20c16d6e
10:18:36:WU00:FS00:0xa7:     Branch: master
10:18:36:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
10:18:36:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
10:18:36:WU00:FS00:0xa7:   Platform: win32 10
10:18:36:WU00:FS00:0xa7:       Bits: 64
10:18:36:WU00:FS00:0xa7:       Mode: Release
10:18:36:WU00:FS00:0xa7:************************************ Build *************************************
10:18:36:WU00:FS00:0xa7:       SIMD: avx_256
10:18:36:WU00:FS00:0xa7:********************************************************************************
10:18:36:WU00:FS00:0xa7:Project: 16507 (Run 0, Clone 1230, Gen 168)
10:18:36:WU00:FS00:0xa7:Unit: 0x000000ce8f59f36f5eae4fbd8a2ea9d7
10:18:36:WU00:FS00:0xa7:Digital signatures verified
10:18:36:WU00:FS00:0xa7:Calling: mdrun -s frame168.tpr -o frame168.trr -x frame168.xtc -cpi state.cpt -cpt 15 -nt 4
10:18:37:WU00:FS00:0xa7:Steps: first=42000000 total=250000
10:18:38:WU00:FS00:0xa7:Completed 9232 out of 250000 steps (3%)
10:18:42:WU01:FS01:FahCore 22: 0.48%
10:18:48:WU01:FS01:FahCore 22: 1.13%
10:18:54:WU01:FS01:FahCore 22: 1.69%
10:19:00:WU01:FS01:FahCore 22: 2.25%
10:19:06:WU01:FS01:FahCore 22: 2.74%
10:19:12:WU01:FS01:FahCore 22: 3.06%
10:19:18:WU01:FS01:FahCore 22: 3.46%
10:19:23:Removing old file 'configs/config-20200407-113949.xml'
10:19:23:Saving configuration to config.xml
10:19:23:<config>
10:19:23:  <!-- Folding Slot Configuration -->
10:19:23:  <cause v='COVID_19'/>
10:19:23:
10:19:23:  <!-- Network -->
10:19:23:  <proxy v=':8080'/>
10:19:23:
10:19:23:  <!-- Remote Command Server -->
10:19:23:  <password v='*****'/>
10:19:23:
10:19:23:  <!-- Slot Control -->
10:19:23:  <power v='FULL'/>
10:19:23:
10:19:23:  <!-- User Information -->
10:19:23:  <passkey v='*****'/>
10:19:23:  <team v='225605'/>
10:19:23:  <user v='Arkharion'/>
10:19:23:
10:19:23:  <!-- Folding Slots -->
10:19:23:  <slot id='0' type='CPU'>
10:19:23:    <cpus v='4'/>
10:19:23:  </slot>
10:19:23:  <slot id='1' type='GPU'/>
10:19:23:</config>
10:19:24:WU01:FS01:FahCore 22: 4.02%
10:19:30:WU01:FS01:FahCore 22: 4.59%
10:19:36:WU01:FS01:FahCore 22: 5.07%
10:19:43:WU01:FS01:FahCore 22: 5.63%
10:19:49:WU01:FS01:FahCore 22: 6.04%
10:19:55:WU01:FS01:FahCore 22: 6.52%
10:20:01:WU01:FS01:FahCore 22: 7.32%
10:20:07:WU01:FS01:FahCore 22: 8.21%
10:20:13:WU01:FS01:FahCore 22: 8.77%
10:20:19:WU01:FS01:FahCore 22: 9.33%
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPUs stuck on Update_Core

Post by bruce »

There were some old bugs downloading FAHCores on systems with more than one active GPU. Those problems were supposedly fixed in V7.6.13 but if there's a corner case still lurking somewhere, you might try pausing all but one GPU and see if it can download the new core.
Arkharion
Posts: 3
Joined: Tue Apr 07, 2020 2:26 pm

Re: GPUs stuck on Update_Core

Post by Arkharion »

Hi Bruce,
thanks for the reply.
I noticed this in the release-notes, too and already tried as you suggested, but it did not change anything. Also the host with only 1GPU was suffering from the same issue.
I also tried to manually delete the core files to get a fresh installation.

Anyhow: I figured it out myself.
Due to me working form home, I noticed some issues with VPN connectivity. My current providers router has known bug which causes packet-loss in some situations.
The core downloads for cpu are quite small in comparison to the gpu-cores. It looks like the download was taking longer then my connection was stable. The downloaded GPU cores always seem to be corrupted.
Therefor the FAH Client was not able to process the download. Unfortunatley the client does not give any kind of log output in this case and I it was just a guess.

What I did to check my asumption: Switch one host to mobile network via usb-tethering, stop FAH Client, delete all core-directories.
Started the client and watched the core files being downloaded, unpacked and started right away. Afterwards i copied the freshly downloaded core to the other host and everything is up&running now.

So there is no real bug in the your client and the root-cause lies within my providers router.
But it would be really great, if the FAH Client writes some log, if there are any errors after downloading the core-files - this would make it easier for other people who might have simliar issues.
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPUs stuck on Update_Core

Post by bruce »

I'm not sure what kind of a log could have been provided that would have been useful. I'm glad you figured it out, but at what point before you started successful debugging steps would FAH client have had enough information to have told you something useful?

FAH depends on you having a reliable network connection.
Arkharion
Posts: 3
Joined: Tue Apr 07, 2020 2:26 pm

Re: GPUs stuck on Update_Core

Post by Arkharion »

Something like "tried to unzip/decompress..." or "uncompression failed" or "could not process downloaded file".
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPUs stuck on Update_Core

Post by bruce »

That error is probably telling you that the downloaded code is not a coherent compressed file. If you're getting internet errors during download, the FAHCore is not going to work from the corrupted package.
[SG]Felix
Posts: 2
Joined: Wed Mar 03, 2021 9:33 am

Re: GPUs stuck on Update_Core

Post by [SG]Felix »

I have the same problem.

is there any known way to speed up the download of the core? Because when looking on my connection monitor, there is about 1,1mBit/s of traffic coming in. available bandwith is about 950mBit/s.
If there is no possibility to speed the download up, or setting auto check in place and resending missing packet, i won't be able to compute for FAH anymore.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: GPUs stuck on Update_Core

Post by Neil-B »

Just an off the wall suggestion but if you are happy to you could try disabling av/firewall as it downloads in case it is an inspection or corruption issue linked to to these ... not guaranteeing even related or will help but might be worth a try if you are happy to trust the fsh servers
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: GPUs stuck on Update_Core

Post by Neil-B »

... or use an alternative download such a a mobile hotspot to bypass router issues?
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
bollix47
Posts: 2941
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: GPUs stuck on Update_Core

Post by bollix47 »

Another possible solution could involve upgrading to the latest client 7.6.21.

https://download.foldingathome.org/rele ... 21_x86.exe
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: GPUs stuck on Update_Core

Post by Neil-B »

bollix47 wrote:Another possible solution could involve upgrading to the latest client 7.6.21.

https://download.foldingathome.org/rele ... 21_x86.exe
Good spot ... Might help ... but stuck on core updates have been seen on latest client ... most appear to have been linked to av/firewall/router issues from what I can see.
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPUs stuck on Update_Core

Post by bruce »

So the bottom line is FAH cannot be used on an unreliable ISP connection???

How does Development reproduce that one?
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: GPUs stuck on Update_Core

Post by Neil-B »

Tbh I don't think anyone is asking them to? ... more a case of doing what we can to help folders with this issue track down a solution which I hope is what we are trying to do :) ... of course a backup manual procedure for downloading cores in a manner they can't be tampered with would be a nice bunny out of the hat from the devs ;)
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: GPUs stuck on Update_Core

Post by Neil-B »

Wouldn't surprise me to find the rash of short responses and the core downloading issues are in some way down to the same issues ... but I am simply speculating and am in no way technically qualified to suggest this is actually the case
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPUs stuck on Update_Core

Post by bruce »

Some download sites use some kind of enhanced tcp/ip protocol which validates an interrupted (partial) download and effectively resumes from the point of the error. I don't know what it's called or how it works, but that would certainly help download a big file like Core_22 over a corrupt connection. Dropped packets can be re-transmitted out-of-order or something like that.
Post Reply