Headless Arch Linux [Mining to F@H]

FAH provides a V7 client installer for Debian / Mint / Ubuntu / RedHat / CentOS / Fedora. Installation on other distros may or may not be easy but if you can offer help to others, they would appreciate it.

Moderators: Site Moderators, FAHC Science Team

Post Reply
Asgaroth
Posts: 29
Joined: Sun Dec 09, 2018 12:06 am

Headless Arch Linux [Mining to F@H]

Post by Asgaroth »

Hi All,

My conscience finally got the better of me and I've decided to switch from mining to F@H :)

My Hardware:
  • Motherboard: Asus B250 Mining Expert - specs here
  • CPU: Intel Core i7 6700K - specs here
  • Memory: Corsair Vengeance LPX 16GB Kit - specs here
  • USB Risers x1 -> x16 - specs here
  • GPU: nVidia: Zotac GeForce 1070 Ti AMP Extreme (6 of these) - product page here
  • GPU: AMD: PowerColor Red Devil RX580 8GB Golden Sample (7 of these) - product page here
I am currently testing this out on Windows 10 but would prefer to switch to Linux. I am used to Manjaro Linux so am thinking of switching to Arch Linux for this box, it will be dedicated to F@H.

I understand that:
  • I will not be able to drive all the GPU's with the 4 core 8 thread CPU that I have here, I'm currently driving 6 1070 ti's with the CPU at 85% utilization on Windows 10, GPU's are each at 80% utilization with temps of 45 - 50 degrees celcius
  • I wont get max PPD due to the reduction in PCIe lanes to the GPU's. I've read that the 1070 Ti should be getting in the range of 800K PPD, when running on Linux, assuming a 20% reduction, I'm guessing these should be pushing around 680K PPD each
What I would like to know from those that have gone before me is:
  1. Anyone have any pointers on an initial headless setup for a F@H system
  2. Any BIOS tweaks recommended? (I'll try to see if I can get these risers operating at Gen-3 speeds, currently testing at Gen - 2)
  3. With a headless setup, can you still use nvidia-settings to set power-limit, core/memory over-clocks?
  4. Do the Arch users use the AUR package for F@H or do you decompress the deb directly from Standford?
Any tips/tricks/suggestions/criticisms/pointers/gotcha's are greatly appreciated.

EDIT: I'll be documenting my setup and configuration below in the hopes it helps someone else looking to go down this route.
Last edited by Asgaroth on Sat Dec 15, 2018 12:21 pm, edited 1 time in total.
There are two major products that came out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence.
-- Jeremy S. Anderson
Asgaroth
Posts: 29
Joined: Sun Dec 09, 2018 12:06 am

Re: Headless Arch Linux [Mining to F@H]

Post by Asgaroth »

Arch Linux Installation

NOTE: Work In Progress

Details:
  • Disto Used: Arch Linux 2018.12.01
  • Boot Method: UEFI
Assumptions:
  1. You've created a USB boot flash drive of the ISO image and booted your system to the Arch prompt.
  2. Single hard disk drive, in my case I have a 500GB disk referenced as sda.
  3. IP address is given out by DHCP and is a static lease (does not change)
References:
  • Arch Wiki Installation Guide (here)
Steps:
  1. Setup your keyboard mapping.

    Code: Select all

    loadkeys uk
  2. Enable NTP

    Code: Select all

    timedatectl set-ntp true
  3. Partition the drive.
    NOTE: Use the GPT partition scheme for UEFI booting, my drive was already configured with the GPT partition scheme so I didn't need to change it.
    We will be creating the following partition layout.
    1. EFI System Partition
    2. Linux Swap Partition
    3. Linux Filesystem Partition

    Code: Select all

    gdisk /dev/sda
      n
      1
      enter
      +550M
      ef00
      
      n
      2
      enter
      +64G
      8200
      
      n
      3
      enter
      enter
      8300
      
      w
      y
    
  4. Create the file systems:

    Code: Select all

    mkfs.fat -F32 /dev/sda1
    mkswap /dev/sda2
    swapon /dev/sda2
    mkfs.ext4 /dev/sda3
    
  5. Mount the "root" file system:

    Code: Select all

    mount /dev/sda3 /mnt
  6. Setup your local mirror list:
    • Go to the Pacman Mirror List Generator, make your selections, and generate the mirror list.
    • Copy the generated mirror list and paste it into the mirror list on the Arch boot stick

      Code: Select all

      vi /etc/pacman.d/mirrorlist
  7. Enable the multilib repository:
    • Edit pacman.conf

      Code: Select all

      vi /etc/pacman.conf 
    • Uncomment the following two lines

      Code: Select all

      [multilib]
      Include = /etc/pacman.d/mirrorlist
      
  8. Install the base system

    Code: Select all

    pacstrap /mnt base base-devel linux linux-firmware
  9. Generate the file system table

    Code: Select all

    genfstab -U -p /mnt >> /mnt/etc/fstab
  10. Change root to the mounted filesystem we just boot-strapped

    Code: Select all

    arch-chroot /mnt
  11. Setup your locale (Note: I'm in Ireland so modify for where you are)

    Code: Select all

    sed -i -e 's|^#\(en_IE.UTF-8 UTF-8.*\)|\1|' /etc/locale.gen
    locale-gen
    echo "LANG=en_IE.UTF-8" > /etc/locale.conf
    export LANG=en_IE.UTF-8
    
  12. Setup your timezone (I'm in Ireland so modify for where you are)

    Code: Select all

    ln -sf /usr/share/zoneinfo/Europe/Dublin /etc/localtime
    hwclock --systohc --utc
    
  13. Setup the keymap for your installed system

    Code: Select all

    echo "KEYMAP=uk" > /etc/vconsole.conf
    
  14. Setup your system's hostname

    Code: Select all

    echo "thanatos" > /etc/hostname
    
  15. Setup your hosts file.
    • NOTE:Change the address for your host below to match your own static lease address

    Code: Select all

    cat << __EOFL __ > /etc/hosts
    127.0.0.1	localhost
    ::1		   localhost
    172.16.17.4 thanatos.localdomain	thanatos
    __EOFL__
    
  16. Get your network interface name

    Code: Select all

    # ip link show
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    2: enp0s31f6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
        link/ether b0:6e:bf:5f:a0:7c brd ff:ff:ff:ff:ff:ff
    
  17. Setup a systemd unit file to enable DHCP on this interface.
    • NOTE: This wont activate on the initial boot, we will enable the systemd-network unit file as a post-install task.

    Code: Select all

    cat << __EOFL__ > /etc/systemd/network/20-wired.network
    [Match]
    Name=enp0s31f6
    
    [Network]
    LinkLocalAddressing=no
    IPv6AcceptRA=no
    DHCP=ipv4
    
    [DHCP]
    UseDomains=yes
    __EOFL__
    
  18. Enable the multilib repository in your locally installed system (See step 7 above)
  19. Perform a system update.

    Code: Select all

    pacman -Syu
  20. Set the root user's password

    Code: Select all

    passwd root
  21. Create a "normal" user account for yourself (optional)

    Code: Select all

    useradd -mg users -G wheel,storage,power -s /bin/bash username
    passwd username
    
  22. Install the sudo application

    Code: Select all

    pacman -S sudo
  23. Enable users in the wheel group to have sudo access

    Code: Select all

    sed -i -e 's|^# \(%wheel ALL=(ALL) ALL\)|\1|' /etc/sudoers
    
  24. Install the GRUB bootloader and UEFI related packages

    Code: Select all

    pacman -S grub efibootmgr dosfstools os-prober mtools
    
  25. Install the GRUB bootloader with UEFI support

    Code: Select all

    mkdir /boot/EFI
    mount /dev/sda1 /boot/EFI
    grub-install --target=x86_64-efi --bootloader-id=grub_uefi --recheck
    
  26. Install the Intel micro-code update package
    • We do this before generating the GRUB boot configuration file as GRUB will detect this is installed and enable it on the kernel boot options.
    • For AMD CPU's install amd-ucode

    Code: Select all

    pacman -S intel-ucode
    
  27. Generate the grub boot configuration

    Code: Select all

    grub-mkconfig -o /boot/grub/grub.cfg
  28. Exit out of the Arch chrooted environment

    Code: Select all

    exit
  29. Unmount file systems

    Code: Select all

    umount -R /mnt
  30. Reboot the system
    • NOTE: You may need to remove your USB boot stick if your BIOS is configured to boot off of USB first.

    Code: Select all

    reboot
Last edited by Asgaroth on Tue Oct 06, 2020 9:26 am, edited 8 times in total.
There are two major products that came out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence.
-- Jeremy S. Anderson
Asgaroth
Posts: 29
Joined: Sun Dec 09, 2018 12:06 am

Re: Headless Arch Linux [Mining to F@H]

Post by Asgaroth »

Post Install Tasks

NOTE: Work In Progress

After your first reboot login as root at the system console and perform the following steps
  1. We need to enable the systemd network/resolver unit files so that systemd will manage our network configuration.

    Code: Select all

    systemctl enable systemd-networkd
    systemctl enable systemd-resolved
    
    NOTE:
    • Ensure /etc/resolv.conf is a symlink to /run/systemd/resolve/resolv.conf
  2. Reboot the host to enable the network.

    Code: Select all

    reboot
  3. Enable NTP

    Code: Select all

    timedatectl set-ntp true
  4. Install OpenSSH and start/enable it.
    • After the startup of ssh, I assume, from here on out, you are logged in as a "normal" user

    Code: Select all

    pacman -S openssh
    systemctl start sshd
    systemctl enable sshd
    
  5. Install nVidia related drivers and packages

    Code: Select all

    sudo pacman -S libglvnd nvidia cuda
  6. Blacklist the OpenSource nouveau driver

    Code: Select all

    sudo cat << __EOFL__ > /etc/modprobe.d/blacklist.conf
    blacklist nouveau
    blacklist rivafb
    blacklist nvidiafb
    blacklist rivatv
    blacklist nv
    blacklist uvcvideo
    __EOFL__
    
  7. Install the GIT application

    Code: Select all

    sudo pacman -S git
  8. Install the YAY package (This enables us to install packages from the AUR)

    Code: Select all

    cd ~
    mkdir git_repos
    cd git_repos
    git clone https://aur.archlinux.org/yay.git
    cd yay
    makepkg -si
    
  9. Install the OpenCL libraries for AMD GPU's

    Code: Select all

    yay -S opencl-amd
  10. Reboot to disable the blacklisted drivers and activate the nvidia drivers

    Code: Select all

    sudo reboot
General System Maintenance
  • Updating the system (while enforcing package list re-sync with mirrors)

    Code: Select all

    # pacman -Syyuu
  • Updating packages from AUR

    Code: Select all

    yay -Su
  • Updating keyrings

    Code: Select all

    pacman -Sy archlinux-keyring
Last edited by Asgaroth on Fri Mar 13, 2020 11:23 am, edited 12 times in total.
There are two major products that came out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence.
-- Jeremy S. Anderson
Asgaroth
Posts: 29
Joined: Sun Dec 09, 2018 12:06 am

Re: Headless Arch Linux [Mining to F@H]

Post by Asgaroth »

F@H Client Installation and Initial Configuration

NOTE: Work In Progress

References:
  1. Install foldingathome package

    Code: Select all

    yay -S foldingathome
  2. Make sure FAHClient can see your configuration

    Code: Select all

    cd /opt/fah
    sudo /opt/fah/FAHClient --info
    
    11:49:01:INFO(1):Read GPUs.txt
    ***************************** Folding@home Client ******************************
             Website: https://foldingathome.org/
           Copyright: (c) 2009-2018 foldingathome.org
              Author: Joseph Coffland <joseph@cauldrondevelopment.com>
                Args: --info
    ************************************ Build *************************************
             Version: 7.5.1
                Date: May 11 2018
                Time: 19:59:04
          Repository: Git
            Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
              Branch: master
            Compiler: GNU 6.3.0 20170516
             Options: -std=gnu++98 -O3 -funroll-loops
            Platform: linux2 4.14.0-3-amd64
                Bits: 64
                Mode: Release
    ************************************ System ************************************
                 CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
              CPU ID: GenuineIntel Family 6 Model 94 Stepping 3
                CPUs: 8
              Memory: 15.55GiB
         Free Memory: 10.93GiB
             Threads: POSIX_THREADS
          OS Version: 4.19
         Has Battery: false
          On Battery: false
          UTC Offset: 0
                 PID: 21978
                 CWD: /opt/fah
                  OS: Linux 4.19.8-arch1-1-ARCH x86_64
             OS Arch: AMD64
                GPUs: 13
               GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
               GPU 1: Bus:4 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                      470/480/570/580]
               GPU 2: Bus:6 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                      470/480/570/580]
               GPU 3: Bus:8 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                      470/480/570/580]
               GPU 4: Bus:9 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                      470/480/570/580]
               GPU 5: Bus:10 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                      470/480/570/580]
               GPU 6: Bus:11 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
               GPU 7: Bus:12 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                      470/480/570/580]
               GPU 8: Bus:13 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
               GPU 9: Bus:14 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
              GPU 10: Bus:15 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
              GPU 11: Bus:16 Slot:0 Func:0 AMD:5 Ellesmere XT [Radeon RX
                      470/480/570/580]
              GPU 12: Bus:17 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1070 Ti] 8186
       CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:6.1 Driver:10.0
       CUDA Device 1: Platform:0 Device:1 Bus:11 Slot:0 Compute:6.1 Driver:10.0
       CUDA Device 2: Platform:0 Device:2 Bus:13 Slot:0 Compute:6.1 Driver:10.0
       CUDA Device 3: Platform:0 Device:3 Bus:14 Slot:0 Compute:6.1 Driver:10.0
       CUDA Device 4: Platform:0 Device:4 Bus:15 Slot:0 Compute:6.1 Driver:10.0
       CUDA Device 5: Platform:0 Device:5 Bus:17 Slot:0 Compute:6.1 Driver:10.0
     OpenCL Device 0: Platform:0 Device:0 Bus:4 Slot:0 Compute:1.2 Driver:2671.3
     OpenCL Device 1: Platform:0 Device:1 Bus:6 Slot:0 Compute:1.2 Driver:2671.3
     OpenCL Device 2: Platform:0 Device:2 Bus:8 Slot:0 Compute:1.2 Driver:2671.3
     OpenCL Device 3: Platform:0 Device:3 Bus:9 Slot:0 Compute:1.2 Driver:2671.3
     OpenCL Device 4: Platform:0 Device:4 Bus:10 Slot:0 Compute:1.2 Driver:2671.3
     OpenCL Device 5: Platform:0 Device:5 Bus:12 Slot:0 Compute:1.2 Driver:2671.3
     OpenCL Device 6: Platform:0 Device:6 Bus:16 Slot:0 Compute:1.2 Driver:2671.3
     OpenCL Device 7: Platform:1 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:415.22
     OpenCL Device 8: Platform:1 Device:1 Bus:11 Slot:0 Compute:1.2 Driver:415.22
     OpenCL Device 9: Platform:1 Device:2 Bus:13 Slot:0 Compute:1.2 Driver:415.22
    OpenCL Device 10: Platform:1 Device:3 Bus:14 Slot:0 Compute:1.2 Driver:415.22
    OpenCL Device 11: Platform:1 Device:4 Bus:15 Slot:0 Compute:1.2 Driver:415.22
    OpenCL Device 12: Platform:1 Device:5 Bus:17 Slot:0 Compute:1.2 Driver:415.22
    ********************************************************************************
    
  3. Generate an empty initial configuration

    Code: Select all

    cd /opt/fah
    sudo ./FAHClient --configure
    
  4. Make a backup of the config created.

    Code: Select all

    sudo cp /opt/fah/config.xml /opt/fah/config.xml.bak
  5. Edit the configuration and allow remote access
    • Manually editing the configuration file is not recommended, however, it did work for me to get started.
    • Substitute your passcode/team/username in the example config.
    • The address "172.16.17.5" in the example below is the address of the machine where you will be connecting from (Using the F@H client)
    • I've disabled remote authentication here, it's not recommended, so enable it if you require it

    Code: Select all

    sudo vi /opt/fah/config.xml

    Code: Select all

    <config>
      <!-- Folding Slot Configuration -->
      <smp v='false'/>
    
      <!-- HTTP Server -->
      <allow v='127.0.0.1 172.16.17.5'/>
    
      <!-- Remote Command Server -->
      <command-allow-no-pass v='127.0.0.1 172.16.17.5'/>
    
      <!-- Slot Control -->
      <power v='full'/>
    
      <!-- User Information -->
      <passkey v='YOUR_PASSKEY'/>
      <team v='YOUR_TEAM'/>
      <user v='YOUR_USERNAME'/>
    
      <!-- Web Server -->
      <web-allow v='127.0.0.1 172.16.17.5'/>
    </config>
    
  6. Start the client

    Code: Select all

    sudo systemctl start foldingathome
  7. Enable the client to start at boot

    Code: Select all

    sudo systemctl enable foldingathome
  8. Now you can head over to your "management" host, start the F@H client and connect to your headless F@H rig.
Last edited by Asgaroth on Sun Dec 16, 2018 11:51 am, edited 2 times in total.
There are two major products that came out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence.
-- Jeremy S. Anderson
Asgaroth
Posts: 29
Joined: Sun Dec 09, 2018 12:06 am

Re: Headless Arch Linux [Mining to F@H]

Post by Asgaroth »

Well, one thing I've noticed since switching from Windows 10 to Arch Linux is that my estimated PPD went from 1.2M to 4.3M! That is a massive jump with no hardware changes at all.

Image

I know this is just an estimation on the client and is bound to fluctuate from day to day, but its been running for 15 hours now and it is estimating 4.3M PPD, that roughly equates to around 710K PPD per card, not bad going, I'll keep an eye on it and see how it goes.

Now to do more reading around to see if there are any client tweaks I can do.

I'll update these posts if I need to tweak some other settings (eg: overclocking, I suspect I'll need a virtual X server for that, WIP)
Last edited by Asgaroth on Thu Dec 20, 2018 6:13 pm, edited 1 time in total.
There are two major products that came out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence.
-- Jeremy S. Anderson
foldy
Posts: 2061
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Headless Arch Linux [Mining to F@H]

Post by foldy »

Windows bottlenecks on x1 risers for FAH so this is why Linux gives you the PPD booster.

Be sure to disable the FAH CPU slot because all CPU is needed to feed the GPUs.

Thank you for sharing your Arch Linux experience
ProDigit
Posts: 242
Joined: Sun Dec 09, 2018 10:23 pm

Re: Headless Arch Linux [Mining to F@H]

Post by ProDigit »

I've noticed, that my Core i7 4500u processor got very slow results running the client at -1 or 3 or 4 CPU cores under Windows.
I only have GalliumOS as alternate OS, and haven't been able to run the client on it. Only from Google Chrome browser.
But in Windows, I found the best results with the client were 2 CPU cores on my system, even if the CPU has 4 threads.

The difference between -1 (automatic), 3, and 2 cores was between 1400-1700 credits, and 4500 credits.
The Core i7 4500u is a 2 core 4 thread system.

Load at 2 and 3 cores is identical (86% CPU).
I'm not sure if 4 cores brought it up to 100%, but either way, performance is lowered when more than 2 cores are selected.
In my case, 2 cores also allow the fan to run at mid-RPM range, rather than full speed.
Post Reply