OS

Setting up with SLC 5.3, and hoping that all the ancillary software will behave. This is a testbed. I am not planning to make any kernel mods to this one -- I want to test VMware's claim on Fusion 2.0.5 that idle CPU consumption will be down. I also want to see if SLC5 will do OK with the ATLAS grid and offline sw.

I am partitioning into 5 disks -- main, Data (100GB) (keep a small stub for distribution), Atlas (40 GB, mostly for distros), Grid (10GB, for VDT and DQ2), ROOT (5 GB, for ROOT installations)

I will be disabling the tinfoil hat stuff, and enabling auto-update -- I hope to use a stock kernel for this machine.

Software

Install ROOT, VDT Client, VDT xrootd, python 2.5, batch system (PBS?) and basics like vi/emacs and such. Keep it minimal, otherwise -- basic graphic, but no bells or whistles.

Scripts

Config scripts for:

• User creation with skel and sudo
• Athena download and setup
• Athena instantiation and run
• NFS setup
• xrootd setup?
• Sudo enables
• cron jobs on root for CRL handling, etc.
• Enable SSH, AFS, etc.

Configuration

/etc/skel needs to be set to allow script directories

Old style

Choose the VM type:

This will be a standard SLC4 machine, not an x86_64.

The name can be set to anything desired - in this case, we’ll call it Tier3.

Do not be shy about setting up large disks. It is not easy to expand them after initial setup, and it IS easy to shrink them back down if they get too big. So be generous. Initial disk (for /) can be 60 GB.

Tell it where to find the disk image you are using for installation. It is worthwhile to download the DVD ISO from

http://linuxsoft.cern.ch/cern/slc4X/iso/SLC46_i386_dvd.iso

and select the ISO as the CD-ROM drive. You can also burn a DVD - this is recommended for speed reasons.

Unselect the checkbox to “Start virtual machine and install operating system now”. There is a lot more configuration to do before you actually start this up.

When you click Finish, the VM will initiate and open the Settings window.Select the Memory option and boost the memory available to the VM to 512 MB or 768 MB. This is easy to change in the future.

Select the Processors option and enable two.

Select the + button and go to Add Hard Disk...

Add the following four disks -

Home - 70 GB Installations - 75 GB AtlasSoftware - 80 GB Data - 300 GB

The different sizes allow easier identification when partitioning in Linux setup. The large number of disks allows modularity and flexibility - there is no real space penalty for creating an empty disk.

Having AtlasSoftware as 80 GB allows for up to 7 full installations of Athena at the same time. This is not the recommended setup, but the latitude is there if needed.

Likewise, the Data disk is capable of expanding to 300 GB, to accommodate large datasets. The recommended usage, however, would be to store the datasets via Shared Folders on another physical drive for better access speed.

Select the + button and go to Add Shared Folder...

Create a folder somewhere in your machine (I chose /DataShare on my laptop, but a similar folder on a separate physical volume, like /Volumes/BigDisk/DataShare, would be preferable)

Check the Enabled box, and select the created folder for sharing. This mountpoint will be in /mnt/hgfs/DataShare (or whatever you name it).Now, we boot. Press Enter for graphical setup. The VM will start spooling up, and will eventually present a installation screen.

Click Next three times (selecting any languages or keyboard layout options you might need) and on the Installation Type screen, select Custom. Click Next.

On the Disk Partitioning Setup screen, select Manually partition with Disk Druid and click Next. You will have to click Yes several times in modal dialog boxes to agree to erase all data on the several VM disks we created in previous steps.When partitioning the primary partition, double-click /dev/sda and create swap space with the first 256 cylinders to allow a swap space of 2 GB. Once created, take the rest of the free space and create the / mountpoint with the remaining space. Set up the remaining disks as the mountpoints /home, /Installations, /Atlas and /data. For help identifying the disks that correspond to those mountpoints, refer to the sizes defined earlier in configuring the VM. Select the default Boot Loader Configuration, but change the hostname to set manually as tier3 (or whatever you choose to make it). The next four screens can be configured at will. In the default VM, the root password will be T3ChangeMe. Please be quick to change the root password upon installing the VM.In the Package Group Selection screen, there will be a number of choices made. The guiding principles will be: ATLAS functionality - coding, network and physics tools. Configuration tools - easy GUI access to system configuration for intermediate users Elimination of “toys” and nonessential, bulky software

For this reason, only Gnome will be retained among the Desktop Environments, and all the Office, Game and Graphical Internet software will be discarded. Any of these can be reinstalled as needed later.

Activated: OpenAFS Client CERN Add-Ons (requiring X11) Additional perl modules Editors Engineering and Scientific Server Configuration Tools (minus Samba) Network Servers (VNC) Development Tools X Software Development Legacy Software Development Administration Tools System Tools Kerberos 4 Clients and Libraries

Removed: Graphical Internet Office/Productivity Sound and Video Authoring and Publishing Graphics Games Web Server Mail Server Windows File Server DNS Name Server FTP Server PostgreSQL Server MySQL Database News Server Legacy Network Server GNOME Software Development KDE Software Development Printing Support

A total of 2506 MB.Fire it up! The installation begins after the click of the Next button. This is where there is no going back - all the Linux configuration starts from scratch here if you have messed something up.

Since you’re in a VM on a disk image, this process can be paused by pausing the VM or sleeping the host computer, or both. Installing from an ISO will take around 10 minutes.

The next step (once the SLC4 installation is completed and the VM has rebooted itself and run a yum update) is to configure the release. Amusingly, the yum update will take substantially longer than the initial installation.

Enable NTP as you click through the next two screens, and accept the display preferences as offered. The user account has been set to tier3 (Tier 3 Basic account, password T3ChangeMe )

Enable the CERN customizations, but turn off automatic system updates (let it check) because we are going to modify the kernel.

You setup is finished, and you can log in either as root or tier3.Log in and open a terminal window.

/sbin/ifconfig

Extract the IP address 192.168.xxx.xxx for use below

su (enter password) vim /etc/hosts 192.168.xxx.xxx tier3 tier3.your.domain

You can also change this to a hostname of your choice, especially in multi-VM installations.

exit cd Scripts . KeyGeneration.sh

This will set up ssh keys between this VM and the lxplus account of your choice for CVS convenience.

cd Scripts . CVS_Setup.sh

This will initiate good CVS behavior with the ssh keys in place.

Kernel Rebuild mkdir -p ~/Kernels cd ~/Kernels wget http://linuxsoft.cern.ch/cern/slc4X/updates/SRPMS/kernel-2.6.9-67.0.15.EL.cern.src.rpm su rpm -i kernel-2.6.9-67.0.15.EL.cern.src.rpm cd /usr/src/redhat/SOURCES for i in *i686*config; do sed s/"# CONFIG_SCSI_BUSLOGIC is not set"/"CONFIG_SCSI_BUSLOGIC=m"/g $i > $i.tmp; mv $i.tmp $i; done tar jxvf linux-2.6.9.tar.bz2 mkdir -p linux-2.6.9-vmware/include/asm-i386 cp -pr linux-2.6.9/include/asm-i386/param.h linux-2.6.9-vmware/include/asm-i386/param.h vim linux-2.6.9-vmware/include/asm-i386/param.h

Change # define HZ 1000 -> # define HZ 100. Not lower than 100 - otherwise other timings are set incorrectly. diff -urN linux-2.6.9/include/asm-i386/param.h linux-2.6.9-vmware/include/asm-i386/param.h > linux-2.6.9-x86-vmware.patch

vim /usr/src/redhat/SPECS/kernel-2.6.9-67.0.15.EL.cern.spec

Change %define release 67.0.7.EL.cern to %define release 67.0.7.EL.cernvm

Add the following after the line Patch4: ... Patch5: linux-2.6.9-x86-vmware.patch

And after %patch4 -p1 %patch5 -p1

rpmbuild -ba --target=i686 /usr/src/redhat/SPECS/kernel-2.6.9-67.0.15.EL.cern.spec (builds all - source and binaries. The -bb option would allow a binary-only build. Runs for a couple of hours. This could probably be reduced by eliminating the xenU and hugemem builds.)

(can be modified by doing

%define buildup 1 %define buildsmp 1 %define buildsource 0 %define buildhugemem 0 %define buildlargesmp 0 %define builddoc 0 %define buildxen 0 %define kabi 1

Installing the RPM:

rpm -ivh /usr/src/redhat/RPMS/kernel-smp-devel-2.6.9-67.0.15.EL.cernvm.i686.rpm rpm -ivh /usr/src/redhat/RPMS/kernel-smp-2.6.9-67.0.15.EL.cernvm.i686.rpm

Yay! Works and reboots. Consumption around 10%

Now trying the same thing with HZ=50 and USER_HZ=50 - to avoid problems with USER_HZ being larger than HZ.

OpenAFS build Checkpoint!!

Mount the SLC DVD (or ISO) su rpm -e afs_tools-1.06-8.SL4.cern.noarch cern-wrappers-1-6.slc4.noarch for i in `rpm -qa|grep openafs`; do rpm -e $i; done wget http://linuxsoft.cern.ch/cern/slc4X/updates/SRPMS/openafs-1.4.4-5.cern.src.rpm wget http://linuxsoft.cern.ch/cern/slc4X/SRPMS/SRPMS/openafs-1.4.4-7.cern.src.rpm rpm -ivh openafs-1.4.4-5.cern.src.rpm rpmbuild -ba --target=i686 /usr/src/redhat/SPECS/openafs.spec

rpm -ivh openafs-1.4.4-7.cern.src.rpm rpm -ivh /usr/src/redhat/RPMS/i686/kernel-devel-2.6.9-67.0.15.EL.cernvm.i686.rpm rpm -ivh /usr/src/redhat/RPMS/i686/kernel-debuginfo-2.6.9-67.0.15.EL.cernvm.i686.rpm rpmbuild -ba --target=i686 /usr/src/redhat/SPECS/openafs.spec rpm -ivh /media/cdrecorder/SL/RPMS/openafs-1.4.4-5.cern.i386.rpm rpm -ivh /media/cdrecorder/SL/RPMS/openafs-[k,c,d]* /usr/src/redhat/RPMS/i686/kernel-module-openafs-2.6.9-67.0.15.EL.cernvmsmp-1.4.4-5.cern.i686.rpm rpm -ivh /media/cdrecorder/SL/RPMS/cern-wrappers-1-5.noarch.rpm /media/cdrecorder/SL/RPMS/afs_tools-1.06-8.SL4.cern.noarch.rpm /etc/init.d/afs start

Other Software:

Install VMware Tools: Virtual Machine->Install VMware Tools... cd /media/cdrecorder rpm -ivh VMwareTools-7.6.2-72241.i386.rpm vmware-config-tools.pl (one failure. Removing and reinstalling the VMware Tools rpm did the trick) For screen size, I choose [2] - 800x600 As a regular user, now... Now we get Python 2.5 and build/install it. Build ROOT from scratch to get the python and other libs needed. For the ROOT configuration, I looked to the appropriate ROOT dist page: http://root.cern.ch/root/myconfig/Version519.config.html - repeated for all versions and took the exact flags they used. ./configure (with flags) --prefix=/Installation/root/5.19 make -j2 make cintdlls su make install exit

Ran into a couple of AFS timeouts. Just restarted the make process - not an issue.

Networking configuration 10 MB/s - copying from old VM, to Mac, via NAT. Back (with the pcnet32 driver) - 12.5-15.5 MB/s Testing the relative merits of the vmxnet driver seems wise.

/etc/init.d/network stop /sbin/rmmod pcnet32 /sbin/rmmod vmxnet /sbin/modprobe vmxnet /etc/init.d/network start

Good! 14-17 MB/s. Worthwhile. Speeds seem to vary with disk activity on the host as well.

DQ2 and gLite

Disk Recreation Shrinking disks seems to be futile. An empty 300 GB disk is still over 1 GB. I am going to try to recreate and replace disks on a VM after emptying them. Seems that checkpoints tend to inflate even unused and empty disks. Avoid them.

OK. Created a new disk, needed to do the following: When the reboot finished, some filesystem labels (like LABEL=/data) in the fstab no longer existed, because I had deleted them. No sweat, just did the following:

fdisk /dev/sde n,p,1 - New primary partition #1 t, 83 - Set partition type to extended w - Write /sbin/mkfs -t ext3 /dev/sde1 /sbin/e2label /dev/sde1 /data mkdir /holder

Add or fix this line in the /etc/fstab file Data.vmdk now down to 61 MB.

Moving things from /Installations to /holder, then recreating /Installations. I’ll remove /holder before shipping.

Reducing Installations to 30 GB, Data to 85 GB. Reduces total space. Easy enough to reconfigure on the other end, I suppose.

Be careful about the mkfs calls, though - I accidentally blew out a disk without meaning to. Need to rebuild ROOT now.

Done.

Changing /Atlas to 50 GB.

Trying a few things to shrink the image - duplicating the main disk and then renaming things into place

-- AldenStradling - 26 Jun 2009

• Tier3VM_Log.pdf: Old T3 VM Setup Log