Installing Debian Lenny (5.0.1) with OpenVZ on IBM System x3650

This is a complete step by step log of the installation of Debian Lenny GNU/Linux using OpenVZ virtualization technology on a factory sealed IBM xSeries 3650 2U rack-mountable server.

Hardware

The server is a standard 7979 model with 16G of additional RAM and 4x500G SAS drives in RAID10 configuration. An IBM RSAII SlimLine adapter is also installed.

Goals

  • Assembling the HW
  • Installing the base OS
  • Enabling all drivers and base SW features
  • Installing OpenVZ
  • Stress testing the system

Installing Debian Lenny

Also see my gallery of screenshots taken during the installation procedure!

    • extract bnx2-06-4.0.5.fw from the .deb (eg.: with alien) and put it on the root dir of a pen-drive
    • why? - the closed source firmware for the latest broadcom NICs are excluded from the Debian (lenny) installer
    • why? - debian thinks this is the way of encouraging broadcom to release its firmware code to the public
    • the result? - annoying debian users. narrowing debian's market penetration. including the firmware in non-free repositories anyway. :-/
  1. Do not forget to plug the pen-drive into the x3650!
  2. See and follow the screenshots of Debian Lenny 5.0.1 Installation procedure
  3. Installing general packages needed:
    apt-get install sudo mc gcc make libncurses5-dev locales bzip2 chrony kernel-package patch ipmitool acpi ethtool
  4. The installation procedure messed up the locales so I had to do:
    dpkg-reconfigure locales
    • Selected locales:
        en_US.ISO-8859-1... done
        en_US.ISO-8859-15... done
        en_US.UTF-8... done
        hu_HU.ISO-8859-2... done
        hu_HU.UTF-8... done
        en_US.UTF-8... done
    • set LC_TIME → hu_HU.utf8
    • set LANG → en_US.UTF-8

Installing and configuring IPMI base management software

  1. Configure kernel modules to be loaded at next reboot
    echo -e "\nipmi_msghandler\nipmi_watchdog\nipmi_si\nipmi_devintf\ncoretemp" >> /etc/modules
  2. Enable the IPMI event daemon at next reboot
    echo "ENABLED=true" > /etc/default/ipmievd

Do not forget to set up email notifications. The IPMI event daemon is useless unless you get notified of the events!

Setting up the Linux firewall (iptables)

  1. Init the nat ip table
    iptables -L -t nat
  2. Save an empty ip table as base template
    iptables-save > /etc/network/iptables_eth0.stop
  3. create /etc/network/iptables_eth0.start as:
    *filter
    :INPUT DROP [0:0]
    :FORWARD ACCEPT [0:0]
    :OUTPUT ACCEPT [0:0]
    # allow local and locally originated traffic
    [0:0] -A INPUT -i lo -j ACCEPT
    [0:0] -A INPUT -s 10.20.20.0/24 -j ACCEPT
    [0:0] -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
    # allow public services
    [0:0] -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
    [0:0] -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
    COMMIT
    *nat
    :PREROUTING ACCEPT [0:0]
    :POSTROUTING ACCEPT [0:0]
    :OUTPUT ACCEPT [0:0]
    # SNAT VEs
    -A POSTROUTING -o eth0 -s 10.20.20.0/24 -j MASQUERADE
    COMMIT
    *mangle
    :PREROUTING ACCEPT [0:0]
    :INPUT ACCEPT [0:0]
    :FORWARD ACCEPT [0:0]
    :OUTPUT ACCEPT [0:0]
    :POSTROUTING ACCEPT [0:0]
    COMMIT
  4. create /etc/network/if-up.d/iptables as
    #! /bin/sh
    # load iptables script if exists
    set -e
    conffile=/etc/network/iptables_$IFACE.$MODE
    if [ -f $conffile ]; then
      iptables-restore < $conffile
    fi
  5. Then make it executable
    chmod +x /etc/network/if-up.d/iptables
  6. Enable IP forwarding for the primary interface
    sysctl -w net.ipv4.conf.eth0.forwarding=1
  7. reboot as a test or activate configuration with
    iptables-restore < /etc/network/iptables_eth0.start

Setting up Munin to monitor the system locally

  1. Install Munin and thttpd
    apt-get install munin munin-node thttpd
  2. Fetch and configure the IPMI sensor plugin
    wget http://muninexchange.projects.linpro.no/download.php?phid=579 -O /usr/share/munin/plugins/ipmitool_sensor_
    chmod +x /usr/share/munin/plugins/ipmitool_sensor_
    ln -s /usr/share/munin/plugins/ipmitool_sensor_ /etc/munin/plugins/ipmitool_sensor_fan
    ln -s /usr/share/munin/plugins/ipmitool_sensor_ /etc/munin/plugins/ipmitool_sensor_temp
    ln -s /usr/share/munin/plugins/ipmitool_sensor_ /etc/munin/plugins/ipmitool_sensor_volt
    echo -e "\n\n[ipmitool_sensor*]\nuser root\ntimeout 20" >> /etc/munin/plugin-conf.d/munin-node
  3. Restart the Munin node
    /etc/init.d/munin-node restart

Install IBM ServeRAID Management commandline interface

WARNING: DIRTY HACK follows

Unfortunately, raid management software provided by IBM is rather old, even if my CD is dated 2008-12-31, the Linux software is made somewhere back in 2003-2007. All insides and outs of GCC have changed since then. The command line utilities arcconf and hrconf both require libstdc++.so.5, which is unavailable for Debian Lenny.

To make things worse libstdc++.so.5 depends on gcc-3.3, but the oldest version of gcc in Lenny is 3.41). I had no option but to do a quick google search for libstdc++.so.5 deb packages and obtain one vintage specimen from an early ubuntu warty archive (thank you guys for keeping that online! ;-) ).

Following this all I had to do was to copy the file into /usr/lib32 (this would be /usr/lib on x86 systems) and upload the shipped utilities to /root/serveraid. The initial warnings can be safely ignored. You can use x64 files and the missing library from my archive if you dare.

Setting up OpenVZ

  1. Getting OpenVZ
    cd /usr/src
    wget http://download.openvz.org/kernel/branches/2.6.27/2.6.27-briullov.1/patches/patch-briullov.1-combined.gz
    wget http://download.openvz.org/kernel/branches/2.6.27/2.6.27-briullov.1/configs/kernel-2.6.27-x86_64.config.ovz
    • Why not using a 'stable' OpenVZ release? Because:
      • the latest 'stable' (023stab048.6) is for linux-2.6.9
      • linux-2.6.9 is missing several critical drivers (eg.: the broadcom network driver)
      • linux-2.6.9 is missing several critical security fixes thus can be apostrophized as „insecure” and „obsolete”
      • ovz 023stab048.6 itself is different in many aspects (eg.: less separation in the VE level, see: http://git.openvz.org/?p=linux-2.6.27-openvz;a=log;h=v2.6.27-briullov)
    • Why not using the official Debian Lenny kernel (as advised on http://wiki.openvz.org/Installation_on_Debian)? Because:
      • Did anyone ever seriously try and test this damn excuse for a production quality kernel?
  2. Getting Linux
    wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.27.tar.bz2
    tar xjf linux-2.6.27.tar.bz2
  3. Patching Linux with OpenVZ
    cd /usr/src/linux-2.6.27
    zcat ../patch-briullov.1-combined.gz | patch -p1
    cp ../kernel-2.6.27-x86_64.config.ovz .config
  4. Compiling, packaging and installing Linux
    make-kpkg --append-to-version -winnetou-ovz-briullov.1 --revision 1 --initrd buildpackage
    dpkg -i ../linux-{image,headers}-2.6.27.21-winnetou-ovz-briullov.1_1_amd64.deb
  5. Cleaning up
    reboot
    apt-get remove linux-image-2.6-amd64 linux-image-2.6.26-2-amd64
  6. Configuring the OpenVZ Lenny software repository
    echo -e "\ndeb http://download.openvz.org/debian-systs lenny openvz" >> /etc/apt/sources.list''
    wget -q http://download.openvz.org/debian-systs/dso_archiv_signing_key.asc -O- | apt-key add - && apt-get update
  7. Installing OpenVZ management software and a minimal Lenny OS template
    apt-get install vzctl vzquota vzctl-ostmpl-debian-5.0-amd64-minimal
  8. Create several split resource configuration templates based on the resources available in the system, with 1G of swap space. Easily used afterwards with vzctl create.
    for x in 2 4 8 16 32; do vzsplit -f $x -n $x -s 1048576; done

Testing OpenVZ

  1. Create and configure test VE
    vzctl create 100 --ostemplate debian-5.0-amd64-minimal --config 4 --ipadd 10.20.20.100 --hostname test
    vzctl set 100 --save --nameserver `cat /etc/resolv.conf | grep nameserver | cut -f2 -d' '`
  2. Start and enter test VE
    vzctl start 100
    vzctl enter 100
  3. Update the VE
    apt-get update
    apt-get upgrade
  4. Do some stress testing
    apt-get install stress
    stress --cpu 8 --io 4 --vm 2 --vm-bytes 128M --hdd 8
  5. Do not forget to exit from VE 100 before continuing!
1)
do you remember the old flames between GCC 3.3 and 3.4?

Linkbacks

Use the following URL for manually sending trackbacks: http://rigo.info/lib/plugins/linkback/exe/trackback.php/en:lenny_on_x3650
en/lenny_on_x3650.txt · Utolsó módosítás: 2009-04-17 00:00 (külső szerkesztés)
CC Attribution-Noncommercial-Share Alike 4.0 International
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0