Project 2671 Hangs at Random
Posted: Wed Aug 26, 2009 9:24 pm
I'm experiencing a random hang of my Linux SMP folding computer. I'm running Fedora 11 basic install with current updates. The computer seems to hang only when processing Project 2671 WUs. I've not had any problems with 2669 WUs but 2671 has been hanging up for several weeks now with increasing frequency. I deleted the folding directory, downloaded and reinstalled fresh yesterday to no avail. Without Project 2671, this computer runs perfectly for as long as I allow it to go. With Project 2671, I can expect it to hang several times a day. No errors are indicated and the screen, mouse, and keyboard are frozen. I can restart the computer and the WU continues processing (without SSE so it knows it terminated incorrectly) until it finishes or hangs. The log of a successful 2669 run and subsequent 2671 run with 3 hangups follows. Any advice you experts could share would be greatly appreciated. Thanks in advance!
Code: Select all
--- Opening Log file [August 25 16:32:09]
# SMP Client ##################################################################
###############################################################################
Folding@Home Client Version 6.02
http://folding.stanford.edu
###############################################################################
###############################################################################
Launch directory: /home/folding/folding
Executable: ./fah6
Arguments: -smp -verbosity 9
[16:32:09] - Ask before connecting: No
[16:32:09] - User name: HendricksSA (Team 0)
[16:32:09] - User ID: 381E417376BD9808
[16:32:09] - Machine ID: 1
[16:32:09]
[16:32:09] Work directory not found. Creating...
[16:32:09] Could not open work queue, generating new queue...
[16:32:09] - Preparing to get new work unit...
[16:32:09] - Autosending finished units...
[16:32:09] Trying to send all finished work units
[16:32:09] + No unsent completed units remaining.
[16:32:09] - Autosend completed
[16:32:09] + Attempting to get work packet
[16:32:09] - Will indicate memory of 7200 MB
[16:32:09] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 7, Stepping: 7
[16:32:09] - Connecting to assignment server
[16:32:09] Connecting to http://assign.stanford.edu:8080/
[16:32:09] Posted data.
[16:32:09] Initial: 40AB; - Successful: assigned to (171.64.65.56).
[16:32:09] + News From Folding@Home: Welcome to Folding@Home
[16:32:10] Loaded queue successfully.
[16:32:10] Connecting to http://171.64.65.56:8080/
[16:32:16] Posted data.
[16:32:16] Initial: 0000; - Receiving payload (expected size: 4830990)
[16:32:24] - Downloaded at ~589 kB/s
[16:32:24] - Averaged speed for that direction ~589 kB/s
[16:32:24] + Received work.
[16:32:24] + Closed connections
[16:32:24]
[16:32:24] + Processing work unit
[16:32:24] Core required: FahCore_a2.exe
[16:32:24] Core not found.
[16:32:24] - Core is not present or corrupted.
[16:32:24] - Attempting to download new core...
[16:32:24] + Downloading new core: FahCore_a2.exe
[16:32:24] Downloading core (/~pande/Linux/x86/Core_a2.fah from www.stanford.edu)
[16:32:24] Initial: AFDE; + 10240 bytes downloaded
[16:32:25] Initial: 2DF4; + 20480 bytes downloaded
[16:32:25] Initial: BA52; + 30720 bytes downloaded
[16:32:25] Initial: B719; + 40960 bytes downloaded
[16:32:25] Initial: 88F5; + 51200 bytes downloaded
[16:32:25] Initial: D186; + 61440 bytes downloaded
[16:32:25] Initial: C494; + 71680 bytes downloaded
[16:32:25] Initial: 2436; + 81920 bytes downloaded
[16:32:25] Initial: 6AB5; + 92160 bytes downloaded
[16:32:25] Initial: 15A1; + 102400 bytes downloaded
[16:32:25] Initial: 95C3; + 112640 bytes downloaded
[16:32:25] Initial: 467E; + 122880 bytes downloaded
[16:32:25] Initial: 52FD; + 133120 bytes downloaded
[16:32:25] Initial: F7AD; + 143360 bytes downloaded
[16:32:25] Initial: FF03; + 153600 bytes downloaded
[16:32:25] Initial: DB32; + 163840 bytes downloaded
[16:32:25] Initial: 8D05; + 174080 bytes downloaded
[16:32:25] Initial: 6120; + 184320 bytes downloaded
[16:32:25] Initial: 18D5; + 194560 bytes downloaded
[16:32:25] Initial: B434; + 204800 bytes downloaded
[16:32:25] Initial: 2BC7; + 215040 bytes downloaded
[16:32:25] Initial: 5768; + 225280 bytes downloaded
[16:32:25] Initial: EBB7; + 235520 bytes downloaded
[16:32:25] Initial: EE47; + 245760 bytes downloaded
[16:32:25] Initial: 8CA6; + 256000 bytes downloaded
[16:32:25] Initial: DA87; + 266240 bytes downloaded
[16:32:25] Initial: B36F; + 276480 bytes downloaded
[16:32:25] Initial: 750C; + 286720 bytes downloaded
[16:32:25] Initial: 4387; + 296960 bytes downloaded
[16:32:25] Initial: 3DF7; + 307200 bytes downloaded
[16:32:25] Initial: 6C1C; + 317440 bytes downloaded
[16:32:25] Initial: E07D; + 327680 bytes downloaded
[16:32:25] Initial: CCFB; + 337920 bytes downloaded
[16:32:26] Initial: F9E9; + 348160 bytes downloaded
[16:32:26] Initial: DB75; + 358400 bytes downloaded
[16:32:26] Initial: F5A2; + 368640 bytes downloaded
[16:32:26] Initial: E643; + 378880 bytes downloaded
[16:32:26] Initial: ADA5; + 389120 bytes downloaded
[16:32:26] Initial: 796A; + 399360 bytes downloaded
[16:32:26] Initial: 2929; + 409600 bytes downloaded
[16:32:26] Initial: 4725; + 419840 bytes downloaded
[16:32:26] Initial: 8807; + 430080 bytes downloaded
[16:32:26] Initial: 1736; + 440320 bytes downloaded
[16:32:26] Initial: 52A1; + 450560 bytes downloaded
[16:32:26] Initial: E44D; + 460800 bytes downloaded
[16:32:26] Initial: B51A; + 471040 bytes downloaded
[16:32:26] Initial: 3D9B; + 481280 bytes downloaded
[16:32:26] Initial: 43AD; + 491520 bytes downloaded
[16:32:26] Initial: B6D6; + 501760 bytes downloaded
[16:32:26] Initial: 6070; + 512000 bytes downloaded
[16:32:26] Initial: 8E38; + 522240 bytes downloaded
[16:32:26] Initial: 7294; + 532480 bytes downloaded
[16:32:26] Initial: A895; + 542720 bytes downloaded
[16:32:26] Initial: C9E7; + 552960 bytes downloaded
[16:32:26] Initial: D6A1; + 563200 bytes downloaded
[16:32:26] Initial: 250D; + 573440 bytes downloaded
[16:32:26] Initial: 9AB0; + 583680 bytes downloaded
[16:32:26] Initial: 88A2; + 593920 bytes downloaded
[16:32:26] Initial: 6298; + 604160 bytes downloaded
[16:32:26] Initial: 5A02; + 614400 bytes downloaded
[16:32:26] Initial: 9DEC; + 624640 bytes downloaded
[16:32:26] Initial: E7EE; + 634880 bytes downloaded
[16:32:26] Initial: C229; + 645120 bytes downloaded
[16:32:26] Initial: A221; + 655360 bytes downloaded
[16:32:26] Initial: 33CA; + 665600 bytes downloaded
[16:32:26] Initial: 8F5A; + 675840 bytes downloaded
[16:32:26] Initial: 3C59; + 686080 bytes downloaded
[16:32:26] Initial: F0D6; + 696320 bytes downloaded
[16:32:26] Initial: 62D7; + 706560 bytes downloaded
[16:32:26] Initial: 960E; + 716800 bytes downloaded
[16:32:26] Initial: DF4D; + 727040 bytes downloaded
[16:32:26] Initial: 4ACF; + 737280 bytes downloaded
[16:32:26] Initial: 1422; + 747520 bytes downloaded
[16:32:26] Initial: 5F27; + 757760 bytes downloaded
[16:32:26] Initial: 00A2; + 768000 bytes downloaded
[16:32:26] Initial: E542; + 778240 bytes downloaded
[16:32:26] Initial: B745; + 788480 bytes downloaded
[16:32:26] Initial: 2A30; + 798720 bytes downloaded
[16:32:26] Initial: EE1E; + 808960 bytes downloaded
[16:32:26] Initial: 827D; + 819200 bytes downloaded
[16:32:26] Initial: ABF2; + 829440 bytes downloaded
[16:32:26] Initial: 2A95; + 839680 bytes downloaded
[16:32:26] Initial: A1C8; + 849920 bytes downloaded
[16:32:26] Initial: 2FB9; + 860160 bytes downloaded
[16:32:26] Initial: 56B3; + 870400 bytes downloaded
[16:32:26] Initial: 2589; + 880640 bytes downloaded
[16:32:26] Initial: 20D0; + 890880 bytes downloaded
[16:32:26] Initial: 1B7B; + 901120 bytes downloaded
[16:32:26] Initial: 1D1D; + 911360 bytes downloaded
[16:32:26] Initial: 5CF5; + 921600 bytes downloaded
[16:32:26] Initial: AF0D; + 931840 bytes downloaded
[16:32:26] Initial: F5C8; + 942080 bytes downloaded
[16:32:26] Initial: 4A27; + 952320 bytes downloaded
[16:32:26] Initial: A4EA; + 962560 bytes downloaded
[16:32:26] Initial: 4145; + 972800 bytes downloaded
[16:32:26] Initial: 06C7; + 983040 bytes downloaded
[16:32:26] Initial: 5656; + 993280 bytes downloaded
[16:32:26] Initial: 7BDC; + 1003520 bytes downloaded
[16:32:26] Initial: A956; + 1013760 bytes downloaded
[16:32:26] Initial: 3205; + 1024000 bytes downloaded
[16:32:26] Initial: A44B; + 1034240 bytes downloaded
[16:32:26] Initial: 4620; + 1044480 bytes downloaded
[16:32:26] Initial: 36F7; + 1054720 bytes downloaded
[16:32:26] Initial: EAF9; + 1064960 bytes downloaded
[16:32:26] Initial: 6522; + 1075200 bytes downloaded
[16:32:26] Initial: F08A; + 1085440 bytes downloaded
[16:32:27] Initial: E582; + 1095680 bytes downloaded
[16:32:27] Initial: E54C; + 1105920 bytes downloaded
[16:32:27] Initial: D7D3; + 1116160 bytes downloaded
[16:32:27] Initial: ACB0; + 1126400 bytes downloaded
[16:32:27] Initial: AD60; + 1136640 bytes downloaded
[16:32:27] Initial: 13AF; + 1146880 bytes downloaded
[16:32:27] Initial: 6301; + 1157120 bytes downloaded
[16:32:27] Initial: 274A; + 1167360 bytes downloaded
[16:32:27] Initial: 2927; + 1177600 bytes downloaded
[16:32:27] Initial: 4953; + 1187840 bytes downloaded
[16:32:27] Initial: 40D4; + 1198080 bytes downloaded
[16:32:27] Initial: 4AEC; + 1208320 bytes downloaded
[16:32:27] Initial: 8298; + 1218560 bytes downloaded
[16:32:27] Initial: E975; + 1228800 bytes downloaded
[16:32:27] Initial: 20E4; + 1239040 bytes downloaded
[16:32:27] Initial: 8B86; + 1249280 bytes downloaded
[16:32:27] Initial: 2092; + 1259520 bytes downloaded
[16:32:27] Initial: 529D; + 1269760 bytes downloaded
[16:32:27] Initial: 57F2; + 1280000 bytes downloaded
[16:32:27] Initial: C63A; + 1290240 bytes downloaded
[16:32:27] Initial: F76A; + 1300480 bytes downloaded
[16:32:27] Initial: D3B4; + 1310720 bytes downloaded
[16:32:27] Initial: A881; + 1320960 bytes downloaded
[16:32:27] Initial: D81D; + 1331200 bytes downloaded
[16:32:27] Initial: 4BF0; + 1341440 bytes downloaded
[16:32:27] Initial: 4D0F; + 1351680 bytes downloaded
[16:32:27] Initial: 8C80; + 1361920 bytes downloaded
[16:32:27] Initial: 0CBA; + 1372160 bytes downloaded
[16:32:27] Initial: F4C9; + 1382400 bytes downloaded
[16:32:27] Initial: 0ECB; + 1392640 bytes downloaded
[16:32:27] Initial: 796A; + 1402880 bytes downloaded
[16:32:27] Initial: 118A; + 1413120 bytes downloaded
[16:32:27] Initial: 0FCE; + 1423360 bytes downloaded
[16:32:27] Initial: 4045; + 1433600 bytes downloaded
[16:32:27] Initial: BB62; + 1443840 bytes downloaded
[16:32:27] Initial: CFD1; + 1454080 bytes downloaded
[16:32:27] Initial: BBA1; + 1464320 bytes downloaded
[16:32:27] Initial: 2594; + 1474560 bytes downloaded
[16:32:27] Initial: 88C6; + 1484800 bytes downloaded
[16:32:27] Initial: 1304; + 1495040 bytes downloaded
[16:32:27] Initial: F039; + 1505280 bytes downloaded
[16:32:27] Initial: C7CE; + 1515520 bytes downloaded
[16:32:27] Initial: D1A2; + 1525760 bytes downloaded
[16:32:27] Initial: F375; + 1536000 bytes downloaded
[16:32:27] Initial: F5D0; + 1546240 bytes downloaded
[16:32:27] Initial: 5419; + 1556480 bytes downloaded
[16:32:27] Initial: 11D2; + 1566720 bytes downloaded
[16:32:27] Initial: F77C; + 1576960 bytes downloaded
[16:32:27] Initial: 271F; + 1587200 bytes downloaded
[16:32:27] Initial: C5F5; + 1597440 bytes downloaded
[16:32:27] Initial: EC87; + 1607680 bytes downloaded
[16:32:27] Initial: C6C1; + 1617920 bytes downloaded
[16:32:27] Initial: BBD2; + 1628160 bytes downloaded
[16:32:27] Initial: 68FB; + 1638400 bytes downloaded
[16:32:27] Initial: 5BD4; + 1648640 bytes downloaded
[16:32:27] Initial: 7276; + 1658880 bytes downloaded
[16:32:27] Initial: 51D1; + 1669120 bytes downloaded
[16:32:27] Initial: 14A0; + 1679360 bytes downloaded
[16:32:27] Initial: A8D5; + 1689600 bytes downloaded
[16:32:27] Initial: B89B; + 1699840 bytes downloaded
[16:32:27] Initial: 4E79; + 1710080 bytes downloaded
[16:32:28] Initial: 6BB6; + 1720320 bytes downloaded
[16:32:28] Initial: 49B2; + 1730560 bytes downloaded
[16:32:28] Initial: 6B0C; + 1740800 bytes downloaded
[16:32:28] Initial: D8D6; + 1751040 bytes downloaded
[16:32:28] Initial: 8467; + 1761280 bytes downloaded
[16:32:28] Initial: AE61; + 1771520 bytes downloaded
[16:32:28] Initial: 097E; + 1781760 bytes downloaded
[16:32:28] Initial: 01C3; + 1785668 bytes downloaded
[16:32:28] Verifying core Core_a2.fah...
[16:32:28] Signature is VALID
[16:32:28]
[16:32:28] Trying to unzip core FahCore_a2.exe
[16:32:28] Decompressed FahCore_a2.exe (4382312 bytes) successfully
[16:32:28] + Core successfully engaged
[16:32:35]
[16:32:35] + Processing work unit
[16:32:35] Core required: FahCore_a2.exe
[16:32:35] Core found.
[16:32:35] Working on Unit 01 [August 25 16:32:35]
[16:32:35] + Working ...
[16:32:35] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 01 -checkpoint 15 -verbose -lifeline 4492 -version 602'
[16:32:35]
[16:32:35] *------------------------------*
[16:32:35] Folding@Home Gromacs SMP Core
[16:32:35] Version 2.08 (Mon May 18 14:47:42 PDT 2009)
[16:32:35]
[16:32:35] Preparing to commence simulation
[16:32:35] - Ensuring status. Please wait.
[16:32:35] Working with standard loops on this execution.
[16:32:35] - Files status OK
[16:32:36] - Expanded Called DecompressByteArray: compressed_data_size=Called DecompressByteArray: compressed_data_size=4830478 data_size=- Digital signature verified
[16:32:36]
[16:32:36] Project: 2669 (Run - Digital signature veriAssembly optimizations on if available.
[16:32:36] Entering Entering M.D.
[16:32:45] one 47, Gen 183)
[16:32:45]
[16:32:45] Entering M.D.
[16:32:54] (0%)
[16:39:45] Completed 2500 out of 250000 steps (1%)
[16:46:27] Completed 5000 out of 250000 steps (2%)
[16:53:03] Completed 7500 out of 250000 steps (3%)
[16:59:40] Completed 10000 out of 250000 steps (4%)
[17:06:22] Completed 12500 out of 250000 steps (5%)
[17:13:05] Completed 15000 out of 250000 steps (6%)
[17:19:45] Completed 17500 out of 250000 steps (7%)
[17:26:24] Completed 20000 out of 250000 steps (8%)
[17:33:02] Completed 22500 out of 250000 steps (9%)
[17:39:45] Completed 25000 out of 250000 steps (10%)
[17:46:22] Completed 27500 out of 250000 steps (11%)
[17:53:04] Completed 30000 out of 250000 steps (12%)
[17:59:44] Completed 32500 out of 250000 steps (13%)
[18:06:22] Completed 35000 out of 250000 steps (14%)
[18:13:05] Completed 37500 out of 250000 steps (15%)
[18:19:45] Completed 40000 out of 250000 steps (16%)
[18:26:21] Completed 42500 out of 250000 steps (17%)
[18:32:58] Completed 45000 out of 250000 steps (18%)
[18:39:36] Completed 47500 out of 250000 steps (19%)
[18:46:13] Completed 50000 out of 250000 steps (20%)
[18:52:53] Completed 52500 out of 250000 steps (21%)
[18:59:35] Completed 55000 out of 250000 steps (22%)
[19:06:11] Completed 57500 out of 250000 steps (23%)
[19:12:46] Completed 60000 out of 250000 steps (24%)
[19:19:27] Completed 62500 out of 250000 steps (25%)
[19:26:09] Completed 65000 out of 250000 steps (26%)
[19:32:53] Completed 67500 out of 250000 steps (27%)
[19:39:33] Completed 70000 out of 250000 steps (28%)
[19:46:11] Completed 72500 out of 250000 steps (29%)
[19:52:48] Completed 75000 out of 250000 steps (30%)
[19:59:31] Completed 77500 out of 250000 steps (31%)
[20:06:11] Completed 80000 out of 250000 steps (32%)
[20:12:50] Completed 82500 out of 250000 steps (33%)
[20:19:35] Completed 85000 out of 250000 steps (34%)
[20:26:14] Completed 87500 out of 250000 steps (35%)
[20:32:52] Completed 90000 out of 250000 steps (36%)
[20:39:31] Completed 92500 out of 250000 steps (37%)
[20:46:09] Completed 95000 out of 250000 steps (38%)
[20:52:49] Completed 97500 out of 250000 steps (39%)
[20:59:32] Completed 100000 out of 250000 steps (40%)
[21:06:13] Completed 102500 out of 250000 steps (41%)
[21:12:49] Completed 105000 out of 250000 steps (42%)
[21:19:29] Completed 107500 out of 250000 steps (43%)
[21:26:16] Completed 110000 out of 250000 steps (44%)
[21:32:58] Completed 112500 out of 250000 steps (45%)
[21:39:36] Completed 115000 out of 250000 steps (46%)
[21:46:16] Completed 117500 out of 250000 steps (47%)
[21:52:55] Completed 120000 out of 250000 steps (48%)
[21:59:37] Completed 122500 out of 250000 steps (49%)
[22:06:16] Completed 125000 out of 250000 steps (50%)
[22:12:51] Completed 127500 out of 250000 steps (51%)
[22:19:34] Completed 130000 out of 250000 steps (52%)
[22:26:12] Completed 132500 out of 250000 steps (53%)
[22:32:09] - Autosending finished units...
[22:32:09] Trying to send all finished work units
[22:32:09] + No unsent completed units remaining.
[22:32:09] - Autosend completed
[22:32:51] Completed 135000 out of 250000 steps (54%)
[22:39:35] Completed 137500 out of 250000 steps (55%)
[22:46:15] Completed 140000 out of 250000 steps (56%)
[22:52:57] Completed 142500 out of 250000 steps (57%)
[22:59:36] Completed 145000 out of 250000 steps (58%)
[23:06:15] Completed 147500 out of 250000 steps (59%)
[23:13:07] Completed 150000 out of 250000 steps (60%)
[23:20:00] Completed 152500 out of 250000 steps (61%)
[23:26:44] Completed 155000 out of 250000 steps (62%)
[23:33:42] Completed 157500 out of 250000 steps (63%)
[23:40:40] Completed 160000 out of 250000 steps (64%)
[23:47:26] Completed 162500 out of 250000 steps (65%)
[23:54:07] Completed 165000 out of 250000 steps (66%)
[00:00:52] Completed 167500 out of 250000 steps (67%)
[00:07:44] Completed 170000 out of 250000 steps (68%)
[00:14:38] Completed 172500 out of 250000 steps (69%)
[00:21:26] Completed 175000 out of 250000 steps (70%)
[00:28:08] Completed 177500 out of 250000 steps (71%)
[00:35:02] Completed 180000 out of 250000 steps (72%)
[00:41:46] Completed 182500 out of 250000 steps (73%)
[00:48:28] Completed 185000 out of 250000 steps (74%)
[00:55:14] Completed 187500 out of 250000 steps (75%)
[01:02:03] Completed 190000 out of 250000 steps (76%)
[01:08:50] Completed 192500 out of 250000 steps (77%)
[01:15:46] Completed 195000 out of 250000 steps (78%)
[01:22:31] Completed 197500 out of 250000 steps (79%)
[01:29:18] Completed 200000 out of 250000 steps (80%)
[01:36:01] Completed 202500 out of 250000 steps (81%)
[01:43:02] Completed 205000 out of 250000 steps (82%)
[01:50:03] Completed 207500 out of 250000 steps (83%)
[01:56:49] Completed 210000 out of 250000 steps (84%)
[02:03:33] Completed 212500 out of 250000 steps (85%)
[02:10:20] Completed 215000 out of 250000 steps (86%)
[02:17:04] Completed 217500 out of 250000 steps (87%)
[02:23:51] Completed 220000 out of 250000 steps (88%)
[02:30:35] Completed 222500 out of 250000 steps (89%)
[02:37:13] Completed 225000 out of 250000 steps (90%)
[02:43:55] Completed 227500 out of 250000 steps (91%)
[02:50:35] Completed 230000 out of 250000 steps (92%)
[02:57:18] Completed 232500 out of 250000 steps (93%)
[03:03:57] Completed 235000 out of 250000 steps (94%)
[03:10:40] Completed 237500 out of 250000 steps (95%)
[03:17:26] Completed 240000 out of 250000 steps (96%)
[03:24:08] Completed 242500 out of 250000 steps (97%)
[03:30:50] Completed 245000 out of 250000 steps (98%)
[03:37:31] Completed 247500 out of 250000 steps (99%)
[03:44:12] Completed 250000 out of 250000 steps (100%)
[03:44:13] DynamicWrapper: Finished Work Unit: sleep=10000
[03:44:23]
[03:44:23] Finished Work Unit:
[03:44:23] - Reading up to 21123792 from "work/wudata_01.trr": Read 21123792
[03:44:23] trr file hash check passed.
[03:44:23] - Reading up to 4510056 from "work/wudata_01.xtc": Read 4510056
[03:44:23] xtc file hash check passed.
[03:44:23] edr file hash check passed.
[03:44:23] logfile size: 181235
[03:44:23] Leaving Run
[03:44:26] - Writing 25959835 bytes of core data to disk...
[03:44:27] ... Done.
[03:46:14] - Shutting down core
[03:46:14]
[03:46:14] Folding@home Core Shutdown: FINISHED_UNIT
[03:47:47] CoreStatus = 64 (100)
[03:47:47] Unit 1 finished with 84 percent of time to deadline remaining.
[03:47:47] Updated performance fraction: 0.843661
[03:47:47] Sending work to server
[03:47:47] + Attempting to send results
[03:47:47] - Reading file work/wuresults_01.dat from core
[03:47:47] (Read 25959835 bytes from disk)
[03:47:47] Connecting to http://171.64.65.56:8080/
[03:49:36] Posted data.
[03:49:36] Initial: 0000; - Uploaded at ~216 kB/s
[03:49:44] - Averaged speed for that direction ~216 kB/s
[03:49:44] + Results successfully sent
[03:49:44] Thank you for your contribution to Folding@Home.
[03:49:44] + Starting local stats count at 1
[03:50:36] - Warning: Could not delete all work unit files (1): Core file absent
[03:50:36] Trying to send all finished work units
[03:50:36] + No unsent completed units remaining.
[03:50:36] - Preparing to get new work unit...
[03:50:36] + Attempting to get work packet
[03:50:36] - Will indicate memory of 7200 MB
[03:50:36] - Connecting to assignment server
[03:50:36] Connecting to http://assign.stanford.edu:8080/
[03:50:36] Posted data.
[03:50:36] Initial: 43AB; - Successful: assigned to (171.67.108.24).
[03:50:36] + News From Folding@Home: Welcome to Folding@Home
[03:50:36] Loaded queue successfully.
[03:50:36] Connecting to http://171.67.108.24:8080/
[03:50:42] Posted data.
[03:50:42] Initial: 0000; - Receiving payload (expected size: 4835024)
[03:50:49] - Downloaded at ~674 kB/s
[03:50:49] - Averaged speed for that direction ~632 kB/s
[03:50:49] + Received work.
[03:50:49] Trying to send all finished work units
[03:50:49] + No unsent completed units remaining.
[03:50:49] + Closed connections
[03:50:49]
[03:50:49] + Processing work unit
[03:50:49] Core required: FahCore_a2.exe
[03:50:49] Core found.
[03:50:49] Working on Unit 02 [August 26 03:50:49]
[03:50:49] + Working ...
[03:50:49] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 02 -checkpoint 15 -verbose -lifeline 4492 -version 602'
[03:50:49]
[03:50:49] *------------------------------*
[03:50:49] Folding@Home Gromacs SMP Core
[03:50:49] Version 2.08 (Mon May 18 14:47:42 PDT 2009)
[03:50:49]
[03:50:49] Preparing to commence simulation
[03:50:49] - Ensuring status. Please wait.
[03:50:50] Called DecompressByteArray: compressed_data_size=4834512 data_size=24039989, decompressed_data_size=24039989 diff=0
[03:50:50] - Digital signature verified
[03:50:50]
[03:50:50] Project: 2671 (Run 48, Clone 39, Gen 89)
[03:50:50]
[03:50:50] Assembly optimizations on if available.
[03:50:50] Entering M.D.
[03:51:00] Run 48, Clone 39, Gen 89)
[03:51:00]
[03:51:00] Entering M.D.
[03:57:40] pleted 2500 out of 250000 steps (1%)
[04:04:09] Completed 5000 out of 250000 steps (2%)
[04:10:44] Completed 7500 out of 250000 steps (3%)
[04:17:18] Completed 10000 out of 250000 steps (4%)
[04:23:51] Completed 12500 out of 250000 steps (5%)
[04:30:22] Completed 15000 out of 250000 steps (6%)
[04:32:09] - Autosending finished units...
[04:32:09] Trying to send all finished work units
[04:32:09] + No unsent completed units remaining.
[04:32:09] - Autosend completed
[04:36:54] Completed 17500 out of 250000 steps (7%)
[04:43:30] Completed 20000 out of 250000 steps (8%)
[04:50:02] Completed 22500 out of 250000 steps (9%)
[04:56:30] Completed 25000 out of 250000 steps (10%)
[05:02:58] Completed 27500 out of 250000 steps (11%)
[05:09:30] Completed 30000 out of 250000 steps (12%)
[05:16:01] Completed 32500 out of 250000 steps (13%)
[05:22:32] Completed 35000 out of 250000 steps (14%)
[05:29:03] Completed 37500 out of 250000 steps (15%)
[05:35:32] Completed 40000 out of 250000 steps (16%)
[05:42:04] Completed 42500 out of 250000 steps (17%)
[05:48:32] Completed 45000 out of 250000 steps (18%)
[05:55:03] Completed 47500 out of 250000 steps (19%)
[06:01:36] Completed 50000 out of 250000 steps (20%)
[06:08:10] Completed 52500 out of 250000 steps (21%)
[06:14:49] Completed 55000 out of 250000 steps (22%)
[06:21:20] Completed 57500 out of 250000 steps (23%)
*** HERE IS WHERE IT HUNG WITH NO ERRORS INDICATED ***
--- Opening Log file [August 26 06:36:58]
# SMP Client ##################################################################
###############################################################################
Folding@Home Client Version 6.02
http://folding.stanford.edu
###############################################################################
###############################################################################
Launch directory: /home/folding/folding
Executable: ./fah6
Arguments: -smp -verbosity 9
[06:36:58] - Ask before connecting: No
[06:36:58] - User name: HendricksSA (Team 0)
[06:36:58] - User ID: 381E417376BD9808
[06:36:58] - Machine ID: 1
[06:36:58]
[06:36:58] Loaded queue successfully.
[06:36:58]
[06:36:58] - Autosending finished units...
[06:36:58] + Processing work unit
[06:36:58] Trying to send all finished work units
[06:36:58] Core required: FahCore_a2.exe
[06:36:58] + No unsent completed units remaining.
[06:36:58] - Autosend completed
[06:36:58] Core found.
[06:36:58] Working on Unit 02 [August 26 06:36:58]
[06:36:58] + Working ...
[06:36:58] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 02 -checkpoint 15 -verbose -lifeline 2949 -version 602'
[06:36:58]
[06:36:58] *------------------------------*
[06:36:58] Folding@Home Gromacs SMP Core
[06:36:58] Version 2.08 (Mon May 18 14:47:42 PDT 2009)
[06:36:58]
[06:36:58] Preparing to commence simulation
[06:36:58] - Ensuring status. Please wait.
[06:37:08] - Looking at optimizations...
[06:37:08] - Working with standard loops on this execution.
[06:37:08] - Files status OK
[06:37:09] - Expanded 4834512 -> 24039989 (decompressed 497.2 percent)
[06:37:09] Called DecompressByteArray: compressed_data_size=4834512 data_size=24039989, decompressed_data_size=24039989 diff=0
[06:37:09] - Digital signature verified
[06:37:09]
[06:37:09] Project: 2671 (Run 48, Clone 39, Gen 89)
[06:37:09]
[06:37:09] Entering M.D.
[06:37:15] Using Gromacs checkpoints
[06:37:18] Resuming from checkpoint
[06:37:18] Verified work/wudata_02.log
[06:37:18] Verified work/wudata_02.trr
[06:37:19] Verified work/wudata_02.xtc
[06:37:19] Verified work/wudata_02.edr
[06:37:19] Completed 57510 out of 250000 steps (23%)
[06:43:53] Completed 60000 out of 250000 steps (24%)
[06:50:24] Completed 62500 out of 250000 steps (25%)
[06:56:55] Completed 65000 out of 250000 steps (26%)
[07:03:25] Completed 67500 out of 250000 steps (27%)
[07:09:56] Completed 70000 out of 250000 steps (28%)
[07:16:28] Completed 72500 out of 250000 steps (29%)
[07:22:58] Completed 75000 out of 250000 steps (30%)
[07:29:27] Completed 77500 out of 250000 steps (31%)
*** HERE IS WHERE IT HUNG WITH NO ERRORS INDICATED ***
--- Opening Log file [August 26 11:31:29]
# SMP Client ##################################################################
###############################################################################
Folding@Home Client Version 6.02
http://folding.stanford.edu
###############################################################################
###############################################################################
Launch directory: /home/folding/folding
Executable: ./fah6
Arguments: -smp -verbosity 9 -pause
[11:31:29] - Ask before connecting: No
[11:31:29] - User name: HendricksSA (Team 0)
[11:31:29] - User ID: 381E417376BD9808
[11:31:29] - Machine ID: 1
[11:31:29]
[11:31:29] Loaded queue successfully.
[11:31:29]
[11:31:29] - Autosending finished units...
[11:31:29] + Processing work unit
[11:31:29] Trying to send all finished work units
[11:31:29] Core required: FahCore_a2.exe
[11:31:29] + No unsent completed units remaining.
[11:31:29] Core found.
[11:31:29] - Autosend completed
[11:31:29] Working on Unit 02 [August 26 11:31:29]
[11:31:29] + Working ...
[11:31:29] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 02 -checkpoint 15 -verbose -lifeline 2949 -version 602'
[11:31:29]
[11:31:29] *------------------------------*
[11:31:29] Folding@Home Gromacs SMP Core
[11:31:29] Version 2.08 (Mon May 18 14:47:42 PDT 2009)
[11:31:29]
[11:31:29] Preparing to commence simulation
[11:31:29] - Ensuring status. Please wait.
[11:31:39] - Looking at optimizations...
[11:31:39] - Working with standard loops on this execution.
[11:31:39] - Files status OK
[11:31:40] - Expanded 4834512 -> 24039989 (decompressed 497.2 percent)
[11:31:40] Called DecompressByteArray: compressed_data_size=4834512 data_size=24039989, decompressed_data_size=24039989 diff=0
[11:31:40] - Digital signature verified
[11:31:40]
[11:31:40] Project: 2671 (Run 48, Clone 39, Gen 89)
[11:31:40]
[11:31:40] Entering M.D.
[11:31:46] Using Gromacs checkpoints
[11:31:49] Resuming from checkpoint
[11:31:50] Verified work/wudata_02.log
[11:31:50] Verified work/wudata_02.trr
[11:31:50] Verified work/wudata_02.xtc
[11:31:50] Verified work/wudata_02.edr
[11:31:50] Completed 77510 out of 250000 steps (31%)
[11:38:24] Completed 80000 out of 250000 steps (32%)
[11:45:04] Completed 82500 out of 250000 steps (33%)
[11:51:37] Completed 85000 out of 250000 steps (34%)
[11:58:12] Completed 87500 out of 250000 steps (35%)
[12:04:49] Completed 90000 out of 250000 steps (36%)
[12:11:22] Completed 92500 out of 250000 steps (37%)
[12:18:18] Completed 95000 out of 250000 steps (38%)
[12:24:53] Completed 97500 out of 250000 steps (39%)
[12:31:26] Completed 100000 out of 250000 steps (40%)
[12:38:02] Completed 102500 out of 250000 steps (41%)
[12:44:34] Completed 105000 out of 250000 steps (42%)
[12:51:07] Completed 107500 out of 250000 steps (43%)
[12:57:38] Completed 110000 out of 250000 steps (44%)
[13:04:12] Completed 112500 out of 250000 steps (45%)
[13:10:46] Completed 115000 out of 250000 steps (46%)
[13:17:16] Completed 117500 out of 250000 steps (47%)
[13:23:47] Completed 120000 out of 250000 steps (48%)
[13:30:17] Completed 122500 out of 250000 steps (49%)
[13:36:50] Completed 125000 out of 250000 steps (50%)
[13:43:22] Completed 127500 out of 250000 steps (51%)
[13:49:53] Completed 130000 out of 250000 steps (52%)
[13:56:24] Completed 132500 out of 250000 steps (53%)
[14:02:59] Completed 135000 out of 250000 steps (54%)
[14:09:33] Completed 137500 out of 250000 steps (55%)
[14:16:05] Completed 140000 out of 250000 steps (56%)
[14:22:38] Completed 142500 out of 250000 steps (57%)
[14:29:11] Completed 145000 out of 250000 steps (58%)
[14:35:44] Completed 147500 out of 250000 steps (59%)
[14:42:18] Completed 150000 out of 250000 steps (60%)
[14:48:55] Completed 152500 out of 250000 steps (61%)
[14:55:26] Completed 155000 out of 250000 steps (62%)
[15:02:02] Completed 157500 out of 250000 steps (63%)
[15:08:36] Completed 160000 out of 250000 steps (64%)
[15:15:10] Completed 162500 out of 250000 steps (65%)
[15:21:45] Completed 165000 out of 250000 steps (66%)
[15:28:19] Completed 167500 out of 250000 steps (67%)
[15:34:51] Completed 170000 out of 250000 steps (68%)
[15:41:23] Completed 172500 out of 250000 steps (69%)
[15:47:58] Completed 175000 out of 250000 steps (70%)
[15:54:32] Completed 177500 out of 250000 steps (71%)
[16:01:06] Completed 180000 out of 250000 steps (72%)
[16:07:39] Completed 182500 out of 250000 steps (73%)
[16:14:14] Completed 185000 out of 250000 steps (74%)
[16:20:51] Completed 187500 out of 250000 steps (75%)
[16:27:31] Completed 190000 out of 250000 steps (76%)
[16:34:07] Completed 192500 out of 250000 steps (77%)
[16:40:43] Completed 195000 out of 250000 steps (78%)
[16:47:20] Completed 197500 out of 250000 steps (79%)
[16:53:56] Completed 200000 out of 250000 steps (80%)
[17:00:29] Completed 202500 out of 250000 steps (81%)
[17:07:01] Completed 205000 out of 250000 steps (82%)
[17:13:34] Completed 207500 out of 250000 steps (83%)
[17:20:07] Completed 210000 out of 250000 steps (84%)
*** HERE IS WHERE IT HUNG WITH NO ERRORS INDICATED ***
--- Opening Log file [August 26 17:51:29]
# SMP Client ##################################################################
###############################################################################
Folding@Home Client Version 6.02
http://folding.stanford.edu
###############################################################################
###############################################################################
Launch directory: /home/folding/folding
Executable: ./fah6
Arguments: -smp -verbosity 9 -pause
[17:51:29] - Ask before connecting: No
[17:51:29] - User name: HendricksSA (Team 0)
[17:51:29] - User ID: 381E417376BD9808
[17:51:29] - Machine ID: 1
[17:51:29]
[17:51:29] Loaded queue successfully.
[17:51:29]
[17:51:29] - Autosending finished units...
[17:51:29] + Processing work unit
[17:51:29] Trying to send all finished work units
[17:51:29] Core required: FahCore_a2.exe
[17:51:29] + No unsent completed units remaining.
[17:51:29] Core found.
[17:51:29] - Autosend completed
[17:51:29] Working on Unit 02 [August 26 17:51:29]
[17:51:29] + Working ...
[17:51:29] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 02 -checkpoint 15 -verbose -lifeline 2980 -version 602'
[17:51:29]
[17:51:29] *------------------------------*
[17:51:29] Folding@Home Gromacs SMP Core
[17:51:29] Version 2.08 (Mon May 18 14:47:42 PDT 2009)
[17:51:29]
[17:51:29] Preparing to commence simulation
[17:51:29] - Ensuring status. Please wait.
[17:51:39] - Looking at optimizations...
[17:51:39] - Working with standard loops on this execution.
[17:51:39] - Files status OK
[17:51:40] - Expanded 4834512 -> 24039989 (decompressed 497.2 percent)
[17:51:40] Called DecompressByteArray: compressed_data_size=4834512 data_size=24039989, decompressed_data_size=24039989 diff=0
[17:51:40] - Digital signature verified
[17:51:40]
[17:51:40] Project: 2671 (Run 48, Clone 39, Gen 89)
[17:51:40]
[17:51:40] Entering M.D.
[17:51:46] Using Gromacs checkpoints
[17:51:49] Resuming from checkpoint
[17:51:49] Verified work/wudata_02.log
[17:51:50] Verified work/wudata_02.trr
[17:51:50] Verified work/wudata_02.xtc
[17:51:50] Verified work/wudata_02.edr
[17:51:51] Completed 210010 out of 250000 steps (84%)
[17:58:31] Completed 212500 out of 250000 steps (85%)
[18:05:04] Completed 215000 out of 250000 steps (86%)
[18:11:40] Completed 217500 out of 250000 steps (87%)
[18:18:10] Completed 220000 out of 250000 steps (88%)
[18:24:41] Completed 222500 out of 250000 steps (89%)
[18:31:11] Completed 225000 out of 250000 steps (90%)
[18:37:44] Completed 227500 out of 250000 steps (91%)
[18:44:16] Completed 230000 out of 250000 steps (92%)
[18:50:48] Completed 232500 out of 250000 steps (93%)
[18:57:23] Completed 235000 out of 250000 steps (94%)
[19:03:55] Completed 237500 out of 250000 steps (95%)
[19:10:27] Completed 240000 out of 250000 steps (96%)
[19:16:58] Completed 242500 out of 250000 steps (97%)
[19:23:30] Completed 245000 out of 250000 steps (98%)
[19:30:04] Completed 247500 out of 250000 steps (99%)
[19:36:32] Completed 250000 out of 250000 steps (100%)
[19:36:33] DynamicWrapper: Finished Work Unit: sleep=10000
[19:36:43]
[19:36:43] Finished Work Unit:
[19:36:43] - Reading up to 21189024 from "work/wudata_02.trr": Read 21189024
[19:36:43] trr file hash check passed.
[19:36:43] - Reading up to 27683928 from "work/wudata_02.xtc": Read 27683928
[19:36:43] xtc file hash check passed.
[19:36:43] edr file hash check passed.
[19:36:43] logfile size: 183651
[19:36:43] Leaving Run
[19:36:45] - Writing 49201355 bytes of core data to disk...
[19:36:46] ... Done.
[19:39:31] - Shutting down core
[19:39:31]
[19:39:31] Folding@home Core Shutdown: FINISHED_UNIT
[19:40:05] CoreStatus = 64 (100)
[19:40:05] Unit 2 finished with 78 percent of time to deadline remaining.
[19:40:05] Updated performance fraction: 0.811962
[19:40:05] Sending work to server
[19:40:05] + Attempting to send results
[19:40:05] - Reading file work/wuresults_02.dat from core
[19:40:05] (Read 49201355 bytes from disk)
[19:40:05] Connecting to http://171.67.108.24:8080/
[19:43:34] Posted data.
[19:43:34] Initial: 0000; - Uploaded at ~228 kB/s
[19:43:35] - Averaged speed for that direction ~222 kB/s
[19:43:35] + Results successfully sent
[19:43:35] Thank you for your contribution to Folding@Home.
[19:43:35] + Number of Units Completed: 2
[19:45:05] - Warning: Could not delete all work unit files (2): Core file absent
[19:45:05] Trying to send all finished work units
[19:45:05] + No unsent completed units remaining.
[19:45:05] + Closed connections
[19:45:05] + Paused after finishing unit
[19:45:05] Press Enter to continue, Ctrl-C to exit...