System Hangs... unsure why? Old and New client

Moderators: Site Moderators, FAHC Science Team

EcipCH
Posts: 16
Joined: Thu Mar 05, 2020 11:51 am

System Hangs... unsure why? Old and New client

Post by EcipCH »

So.. sometimes when I'm folding (CPU and GPU), I could be opening Firefox, or Thunderbird, or maybe LibreOffice, anything really, and the system will hang completely, requiring a hard reset.

I'm not sure if its because of the Folding@Home software, but it ONLY happens when Folding@Home is running GPU tasks, making me thing that the GPU and/or drivers are involved somehow.
After the hard reset, my custom fan curve will also be reset to factory defaults in Radeon Software (latest).

I've tried to find a clue in the Windows logs, but its blank (other than system experienced unexpected shut down blah blah).

Anyone else having this issue with AMD?

AMD Ryzen 7 3800X & RX 5700 XT.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: System Hangs... unsure why? Old and New client

Post by Neil-B »

Is your RX 5700 XT at stock speeds or overclocked?
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
Joe_H
Site Admin
Posts: 7857
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: System Hangs... unsure why? Old and New client

Post by Joe_H »

Which version of the drivers are you using? There were reports of some recent versions having problems if F@h was running and some other software would also be using the GPU. That bug is supposed to be fixed in the most recent release(s).
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
NickDawg41
Posts: 4
Joined: Tue May 12, 2020 9:01 pm

Re: System Hangs... unsure why? Old and New client

Post by NickDawg41 »

I'm having a similar issue with 5500xt.

Sometimes it will hang while opening a browser but will finally be one usable within about 20-30 seconds.

DRIVER VERSIONS USED:
19.12.2 - Issue persists
20.1.3 - Issue persists
20.4.1 - issue persists

Also can't complete a work unit for project 13404. Farthest I get it 5-6%

Tried lowering wattage, maxing out fan, lowering clock to AMD stock... Nothing as of yet.
ajm
Posts: 754
Joined: Sat Mar 21, 2020 5:22 am
Location: Lucerne, Switzerland

Re: System Hangs... unsure why? Old and New client

Post by ajm »

Maybe disable the hardware acceleration of the browser(s)? It can create issues when folding on GPU.
New Bios?
EcipCH
Posts: 16
Joined: Thu Mar 05, 2020 11:51 am

Re: System Hangs... unsure why? Old and New client

Post by EcipCH »

Neil-B wrote:Is your RX 5700 XT at stock speeds or overclocked?
No. Stock.
In fact, when I'm folding, I set "Power limit" slider in Radeon Software to the minimum, or -50. This way the card can fold all day happily at 64°.
EcipCH
Posts: 16
Joined: Thu Mar 05, 2020 11:51 am

Re: System Hangs... unsure why? Old and New client

Post by EcipCH »

Joe_H wrote:Which version of the drivers are you using? There were reports of some recent versions having problems if F@h was running and some other software would also be using the GPU. That bug is supposed to be fixed in the most recent release(s).
20.2.2
ajm wrote:Maybe disable the hardware acceleration of the browser(s)? It can create issues when folding on GPU.
New Bios?
It doesnt just happen with a browser exclusively.. I could also open LibreOffice.
Two weeks ago, the computer was on for 7 days straight, and it never happened once. It's odd.
bruce
Posts: 20910
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: System Hangs... unsure why? Old and New client

Post by bruce »

I'm having what seems like system hangs but that's really not what is happening. Some projects that are being assigned to my GPUs are extremely large. It turns out that they are putting major demands on my machine's RAM. If I look at the Windows task manager, it reports a "working set" of memory required for each active task. If I add up all the numbers for anything that's active, the total greatly exceeds my actual RAM size. This produces whats called "trashing" where the OS has to move large amounts of memory to/from Virtual Memory which is on Disk. In my case, this produces large delays in my ability to do anything useful and the computer appears to be hung. If I simply take my hands off the Keyboard/Mouse and do nothing for a while, the system eventually recovers but not before I have decided that it's really hung.

you may not be experiencing the same problem, but I have to ask: how long did you wait before you decided the system was hung? What device contains your primary paging area and how big is it? (also called Virtual Memory under your System Performance settings.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: System Hangs... unsure why? Old and New client

Post by Neil-B »

I wonder if it is worth me firing up a GPU slot on my server and seeing if I can snag one of these - GPU is just within usable specs, but I shouldn't have RAM issues so might be able to spot/see how much RAM it actually grabs for peak use … Any Thoughts? … Might this provide any useful insight?
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
ajm
Posts: 754
Joined: Sat Mar 21, 2020 5:22 am
Location: Lucerne, Switzerland

Re: System Hangs... unsure why? Old and New client

Post by ajm »

One of my machines is now folding only on GPU. Ressource Monitor:

Image

EDIT: During transition between two WUs, it needs double the RAM:

Image

EDIT2: Depending on the WU, so it seems, it also can suddenly need over 30% more RAM:

Image

EDIT3: Now at 1,445,408. In a worst case scenario, one GPU slot can thus need, at times, up to 3 GB RAM.

EDIT4: Now at ~2.5M (953,140 + 1,553,412) during a new transition:

Image
Last edited by ajm on Thu May 14, 2020 12:50 pm, edited 4 times in total.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: System Hangs... unsure why? Old and New client

Post by Neil-B »

Have fired up a GPU slot and am playing with performance Monitor to capture memory usage stats (might also do some Processing stats whilst I am at it) … I'll keep an eye out for "spikes" in usage.
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
EcipCH
Posts: 16
Joined: Thu Mar 05, 2020 11:51 am

Re: System Hangs... unsure why? Old and New client

Post by EcipCH »

Alright guys, so the machine froze earlier today exactly as it does when running F@H, except that it wasnt running F@H when it did it. In fact, it wasnt doing anything at all. Pretty sure I clicked to open Firefox and it bit dirt.

So clearly, it's not the F@H client.

Back to square one.

I updated the AMD Chipset drivers and Realtek audio drivers just now.. See how it goes.
EcipCH
Posts: 16
Joined: Thu Mar 05, 2020 11:51 am

Re: System Hangs... unsure why? Old and New client

Post by EcipCH »

Well, this just in...

AMD has released new BIOS Microcode for AM4 Motherboards

AGESA 1.0.0.5
AGESA 1.0.0.5 improves POST (time) with select Micron Technology DDR4-3200 memory chips. An intermittent virtual memory error with certain Realtek onboard Ethernet PHY chips has been fixed. The microcode also improves PCI-Express bus stability and interoperability, in general. A PCIe lane configuration issue with Ryzen 3 Pro 2100GE has been fixed. Besides these, all other performance- and stability-improvements part of older 1.0.0.4 a/ab/abb/abba microcodes are incorporated into 1.0.0.5. Keep an eye on the BIOS updates section of your socket AM4 motherboard's product page on its company website.

So... go have a look at your manufacturers website for a BIOS firmware update.
EcipCH
Posts: 16
Joined: Thu Mar 05, 2020 11:51 am

Re: System Hangs... unsure why? Old and New client

Post by EcipCH »

It may be a bit too soon to really tell, but I think updating to latest BIOS solved my issue.
Last edited by EcipCH on Tue May 19, 2020 7:42 am, edited 1 time in total.
Neil-B
Posts: 2027
Joined: Sun Mar 22, 2020 5:52 pm
Hardware configuration: 1: 2x Xeon E5-2697v3@2.60GHz, 512GB DDR4 LRDIMM, SSD Raid, Win10 Ent 20H2, Quadro K420 1GB, FAH 7.6.21
2: Xeon E3-1505Mv5@2.80GHz, 32GB DDR4, NVME, Win10 Pro 20H2, Quadro M1000M 2GB, FAH 7.6.21 (actually have two of these)
3: i7-960@3.20GHz, 12GB DDR3, SSD, Win10 Pro 20H2, GTX 750Ti 2GB, GTX 1080Ti 11GB, FAH 7.6.21
Location: UK

Re: System Hangs... unsure why? Old and New client

Post by Neil-B »

Fingers crossed for you !!
2x Xeon E5-2697v3, 512GB DDR4 LRDIMM, SSD Raid, W10-Ent, Quadro K420
Xeon E3-1505Mv5, 32GB DDR4, NVME, W10-Pro, Quadro M1000M
i7-960, 12GB DDR3, SSD, W10-Pro, GTX1080Ti
i9-10850K, 64GB DDR4, NVME, W11-Pro, RTX3070

(Green/Bold = Active)
Post Reply