Creating the Master NFS Node

Install with a typical SL6 Non-PAE i686 kernel. This will make setup of the other machines a lot easier!

Ensure in TDAQ puppet script, $myprepare is set to True before installing the machine as otherwise, a load of extra stuff gets installed which isn't needed by the SBC image

Done in Puppet

* Setup tftp:

yum install xinetd tftp-server 
emacs /etc/xinetd.d/tftp -> Ensure tftp is on and set to /tftpboot dir 
service xinetd start 
chkconfig xinetd on

* Grab previous settings from old install:

rsync -av root@epat5:/tftpboot /.

* Setup the nfs export:

[root@eplvm010 ~]# more /etc/exports 
# List of permanent exports - additional puppet ones are added after 
/nfs/root eplynx?.ph.bham.ac.uk(rw,sync,no_root_squash) 
/nfs/snapshot eplynx?.ph.bham.ac.uk(rw,sync,no_root_squash)

* Copy an up-to-date PXELinux:

yum install syslinux syslinux-tftpboot 
cp /var/lib/tftpboot/pxelinux.0 /tftpboot/linux-install/.

* Make sure the ethcfg settings are correct:

[root@eplvm010 ~]# more /tftpboot/linux-install/pxelinux.cfg/diskless-eth0.cfg 

default linux

label linux 
  kernel image/vmlinuz 
  append initrd=image/initrd.img bigphysarea=2048 ramdisk_size=20480 root=nfs:147.188.47.139:/nfs/root ETHERNET=eth0

* Make sure the correct vmlinuz kernel is present

[root@epdt109 ~]# cp /boot/vmlinuz-2.6.32-504.el6.nonpae.i686 /tftpboot/linux-install/image/vmlinuz

* Packages installed by puppet:

subversion glibc-headers glibc-devel libxml2-devel kernel-devel gcc procmail openldap-devel xorg-x11-proto-devel libXt-devel openmotif-devel libXpm-devel libXaw-devel pam-devel uuid-devel libuuid-devel nfs-utils compat-gcc-34-g77

Setting up diskless boot

* Create copy of live system

rsync -va --exclude='/proc/*' --exclude='/sys/*' --exclude='/afs' --exclude='/nfs' --exclude='/opt/atlas' --exclude='/home' / /nfs/root/

* Setup READONLY for the clients:

[root@eplvm010 ~]# more /nfs/root/etc/sysconfig/readonly-root
 # Set to 'yes' to mount the system filesystems read-only. 
READONLY=yes 
# Set to 'yes' to mount various temporary state as either tmpfs
# or on the block device labelled RW_LABEL. Implied by READONLY 
TEMPORARY_STATE=no 
# Place to put a tmpfs for temporary scratch writable space 
RW_MOUNT=/var/lib/stateless/writable 
# Label on local filesystem which can be used for temporary scratch space 
RW_LABEL=stateless-rw 
# Options to use for temporary mount 
RW_OPTIONS= 
# Label for partition with persistent data 
STATE_LABEL=stateless-state 
# Where to mount to the persistent data 
STATE_MOUNT=/var/lib/stateless/state 
# Options to use for peristent mount 
STATE_OPTIONS=

* Fix client fstab

emacs /nfs/root/etc/fstab

none /tmp tmpfs defaults 0 0 
tmpfs /dev/shm tmpfs defaults 0 0 
sysfs /sys sysfs defaults 0 0 
proc /proc proc defaults 0 0 
epdt109.ph.bham.ac.uk:/home /home nfs nfsvers=3,bg,hard,intr,nolock,tcp 0 0 
epdt109.ph.bham.ac.uk:/opt/atlas /opt/atlas nfs nfsvers=3,bg,hard,intr,nolock,tcp 0 0

* Turn off some things

chroot /nfs/root 
chkconfig nfs off 
chkconfig puppet off 
yum -y remove 
yum-autoupdate
exit

* Fix the hostname:

emacs /nfs/root/etc/sysconfig/network

NETWORKING=yes

* Fix the network scripts

emacs /nfs/root//etc/sysconfig/network-scripts/ifcfg-eth*

DEVICE="eth0" BOOTPROTO="dhcp" HWADDR= ONBOOT="yes"

* Make sure vmlinuz and initrd.img are correct

cp /boot/vmlinuz-2.6.32-431.5.1.el6.nonpae.i686 /tftpboot/linux-install/image/vmlinuz 
dracut initrd.img 2.6.32-431.5.1.el6.nonpae.i686 
cp initrd.img /tftpboot/linux-install/image/.

Installing TDAQ/L1Calo software stack

* Checkout the l1calo scripts:

su - bhamt1 
kinit mslater ; aklog 
svn co $SVNROOT/LVL1/l1calo/scripts/trunk LVL1/l1calo/scripts 
ln -s LVL1/l1calo l1calo 
exit

* Now grab all the l1calo stuff

su - bhamt1 
cd ${HOME}/LVL1/l1calo/ 
export L1CALO_SYSTEM= 
getallpkg 
exit

* Build L1Calo?

su - bhamt1 
emacs /opt/atlas/tdaq/tdaq-05-05-00/TDAQExternal/cmt/requirements -> i686 "/usr/lib/jvm/java-1.7.0-oracle" 
cd ${HOME}/l1calo/L1CaloRelease/cmt 
cmt broadcast cmt config 
cmt broadcast -buildpkg 
ln -s /home/bhamt1/LVL1/l1calo/installed/database/v15/l1calo /home/bhamt1/LVL1/l1calo/installed/database/l1calo 
cmt broadcast -buildpkg >& ~/out.txt 
exit

* Build the drivers

su - bhamt1 
cd /opt/atlas/tdaq/tdaq-05-05-00/ROSRCDdrivers/cmt 
emacs /opt/atlas/tdaq/tdaq-05-05-00/ROSRCDdrivers/cmt/requirements -> uncomment: 
macro base_binaries "test_drivers" 
    x86_64-slc6 "test_drivers driver_64" 
    x86_64-slc5 "test_drivers driver_64" 
    i686-slc6 "test_drivers driver_32" 
    i686 "test_drivers driver_32" 
emacs /opt/atlas/tdaq/tdaq-05-05-00/ROSRCDdrivers/src/Makefile_32 -> Change $(MAKE) line to: 
KCPPFLAGS='$(CPPFLAGS)' $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) 
modules -> remove refs to solar.o robinnp.o robintty.o

service ntpd stop && ntpdate 0.fedora.pool.ntp.org && service ntpd start 
cmt bro cmt config 
cmt bro make 
mkdir /opt/atlas/vme_all_in_one 

Now do this on this SBC to build the drivers:

cd /opt/atlas/tdaq/tdaq-05-05-00/ROSRCDdrivers/cmt && cmt config && make && cp -r ../i686-slc6-gcc48-opt/* /opt/atlas/vme_all_in_one/. 
cd /opt/atlas/tdaq/tdaq-05-05-00/io_rcc/cmt && cmt config && make && cp -r ../i686-slc6-gcc48-opt/* /opt/atlas/vme_all_in_one/. 
cd /opt/atlas/tdaq/tdaq-05-05-00/cmem_rcc/cmt && cmt config && make && cp -r ../i686-slc6-gcc48-opt/* /opt/atlas/vme_all_in_one/. 
cd /opt/atlas/tdaq/tdaq-05-05-00/vme_rcc/cmt && cmt config && make && cp -r ../i686-slc6-gcc48-opt/* /opt/atlas/vme_all_in_one/.

???? edit ${L1UTILROOT}/tdaq/${myTdaqVersion}/ROSRCDdrivers/src/drivers_tdaq

Useful Links:

http://serverfault.com/questions/237924/pxe-boot-fails-with-message-no-default-or-ui-configuration-directive-found https://www.redhat.com/archives/rhelv6-list/2010-November/msg00034.html https://coda.jlab.org/wiki/index.php/CentOS6_Linux_Diskless_Setup https://docs.oracle.com/cd/E20815_01/html/E20821/gjqeh.html https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/diskless-nfs-config.html

-- MarkSlater - 29 May 2015