Creating the Master NFS Node
Install with a typical SL6 Non-PAE i686 kernel. This will make setup of the other machines
a lot easier!
Ensure in TDAQ puppet script, $myprepare is set to True before installing the machine as otherwise, a load of extra stuff gets installed which isn't needed by the SBC image
Done in Puppet
* Setup tftp:
yum install xinetd tftp-server
emacs /etc/xinetd.d/tftp -> Ensure tftp is on and set to /tftpboot dir
service xinetd start
chkconfig xinetd on
* Grab previous settings from old install:
rsync -av root@epat5:/tftpboot /.
* Setup the nfs export:
[root@eplvm010 ~]# more /etc/exports
# List of permanent exports - additional puppet ones are added after
/nfs/root eplynx?.ph.bham.ac.uk(rw,sync,no_root_squash)
/nfs/snapshot eplynx?.ph.bham.ac.uk(rw,sync,no_root_squash)
* Copy an up-to-date PXELinux:
yum install syslinux syslinux-tftpboot
cp /var/lib/tftpboot/pxelinux.0 /tftpboot/linux-install/.
* Make sure the ethcfg settings are correct:
[root@eplvm010 ~]# more /tftpboot/linux-install/pxelinux.cfg/diskless-eth0.cfg
default linux
label linux
kernel image/vmlinuz
append initrd=image/initrd.img bigphysarea=2048 ramdisk_size=20480 root=nfs:147.188.47.139:/nfs/root ETHERNET=eth0
* Make sure the correct vmlinuz kernel is present
[root@epdt109 ~]# cp /boot/vmlinuz-2.6.32-504.el6.nonpae.i686 /tftpboot/linux-install/image/vmlinuz
* Packages installed by puppet:
subversion glibc-headers glibc-devel libxml2-devel kernel-devel gcc procmail openldap-devel xorg-x11-proto-devel libXt-devel openmotif-devel libXpm-devel libXaw-devel pam-devel uuid-devel libuuid-devel nfs-utils compat-gcc-34-g77
Setting up diskless boot
* Create copy of live system
rsync -va --exclude='/proc/*' --exclude='/sys/*' --exclude='/afs' --exclude='/nfs' --exclude='/opt/atlas' --exclude='/home' / /nfs/root/
* Setup READONLY for the clients:
[root@eplvm010 ~]# more /nfs/root/etc/sysconfig/readonly-root
# Set to 'yes' to mount the system filesystems read-only.
READONLY=yes
# Set to 'yes' to mount various temporary state as either tmpfs
# or on the block device labelled RW_LABEL. Implied by READONLY
TEMPORARY_STATE=no
# Place to put a tmpfs for temporary scratch writable space
RW_MOUNT=/var/lib/stateless/writable
# Label on local filesystem which can be used for temporary scratch space
RW_LABEL=stateless-rw
# Options to use for temporary mount
RW_OPTIONS=
# Label for partition with persistent data
STATE_LABEL=stateless-state
# Where to mount to the persistent data
STATE_MOUNT=/var/lib/stateless/state
# Options to use for peristent mount
STATE_OPTIONS=
* Fix client fstab
emacs /nfs/root/etc/fstab
none /tmp tmpfs defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
epdt109.ph.bham.ac.uk:/home /home nfs nfsvers=3,bg,hard,intr,nolock,tcp 0 0
epdt109.ph.bham.ac.uk:/opt/atlas /opt/atlas nfs nfsvers=3,bg,hard,intr,nolock,tcp 0 0
* Turn off some things
chroot /nfs/root
chkconfig nfs off
chkconfig puppet off
yum -y remove
yum-autoupdate
exit
* Fix the hostname:
emacs /nfs/root/etc/sysconfig/network
NETWORKING=yes
* Fix the network scripts
emacs /nfs/root//etc/sysconfig/network-scripts/ifcfg-eth*
DEVICE="eth0" BOOTPROTO="dhcp" HWADDR= ONBOOT="yes"
* Make sure vmlinuz and initrd.img are correct
cp /boot/vmlinuz-2.6.32-431.5.1.el6.nonpae.i686 /tftpboot/linux-install/image/vmlinuz
dracut initrd.img 2.6.32-431.5.1.el6.nonpae.i686
cp initrd.img /tftpboot/linux-install/image/.
Installing TDAQ/L1Calo software stack
* Checkout the l1calo scripts:
su - bhamt1
kinit mslater ; aklog
svn co $SVNROOT/LVL1/l1calo/scripts/trunk LVL1/l1calo/scripts
ln -s LVL1/l1calo l1calo
exit
* Now grab all the l1calo stuff
su - bhamt1
cd ${HOME}/LVL1/l1calo/
export L1CALO_SYSTEM=
getallpkg
exit
* Build
L1Calo?
su - bhamt1
emacs /opt/atlas/tdaq/tdaq-05-05-00/TDAQExternal/cmt/requirements -> i686 "/usr/lib/jvm/java-1.7.0-oracle"
cd ${HOME}/l1calo/L1CaloRelease/cmt
cmt broadcast cmt config
cmt broadcast -buildpkg
ln -s /home/bhamt1/LVL1/l1calo/installed/database/v15/l1calo /home/bhamt1/LVL1/l1calo/installed/database/l1calo
cmt broadcast -buildpkg >& ~/out.txt
exit
* Build the drivers
su - bhamt1
cd /opt/atlas/tdaq/tdaq-05-05-00/ROSRCDdrivers/cmt
emacs /opt/atlas/tdaq/tdaq-05-05-00/ROSRCDdrivers/cmt/requirements -> uncomment:
macro base_binaries "test_drivers"
x86_64-slc6 "test_drivers driver_64"
x86_64-slc5 "test_drivers driver_64"
i686-slc6 "test_drivers driver_32"
i686 "test_drivers driver_32"
emacs /opt/atlas/tdaq/tdaq-05-05-00/ROSRCDdrivers/src/Makefile_32 -> Change $(MAKE) line to:
KCPPFLAGS='$(CPPFLAGS)' $(MAKE) -C $(KDIR) SUBDIRS=$(PWD)
modules -> remove refs to solar.o robinnp.o robintty.o
service ntpd stop && ntpdate 0.fedora.pool.ntp.org && service ntpd start
cmt bro cmt config
cmt bro make
mkdir /opt/atlas/vme_all_in_one
Now do this on this SBC to build the drivers:
cd /opt/atlas/tdaq/tdaq-05-05-00/ROSRCDdrivers/cmt && cmt config && make && cp -r ../i686-slc6-gcc48-opt/* /opt/atlas/vme_all_in_one/.
cd /opt/atlas/tdaq/tdaq-05-05-00/io_rcc/cmt && cmt config && make && cp -r ../i686-slc6-gcc48-opt/* /opt/atlas/vme_all_in_one/.
cd /opt/atlas/tdaq/tdaq-05-05-00/cmem_rcc/cmt && cmt config && make && cp -r ../i686-slc6-gcc48-opt/* /opt/atlas/vme_all_in_one/.
cd /opt/atlas/tdaq/tdaq-05-05-00/vme_rcc/cmt && cmt config && make && cp -r ../i686-slc6-gcc48-opt/* /opt/atlas/vme_all_in_one/.
???? edit ${L1UTILROOT}/tdaq/${myTdaqVersion}/ROSRCDdrivers/src/drivers_tdaq
Useful Links:
http://serverfault.com/questions/237924/pxe-boot-fails-with-message-no-default-or-ui-configuration-directive-found https://www.redhat.com/archives/rhelv6-list/2010-November/msg00034.html https://coda.jlab.org/wiki/index.php/CentOS6_Linux_Diskless_Setup https://docs.oracle.com/cd/E20815_01/html/E20821/gjqeh.html https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/diskless-nfs-config.html
--
MarkSlater - 29 May 2015