Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide

In: Open Source|Operating Systems

26 Jun 2007

I had to spend some time setting up Xen on one of the new Dell servers we bought and while there was some documentation around, I had to constantly refer to the different sources since there wasn’t a complete page which had everything I needed for my particular situation – I have a host machine with Ubuntu Feisty Fawn installed and wanted to install a couple of Xen guest domains on it. One other thing I found lacking in the documentation is the explanation of commands or answers to the question “so why the hell am I doing this?”. I’m not considering LVM at the moment and am happy with loopback disk file images too.

So I think it’s a good idea to write this down somewhere for the next time I have to do the same bloody thing over again on any new servers.

Some caveats before we start:

  • Like I said, I’m not gonna talk about LVM for disk images since I know next to nothing about configuring it at the moment. The host machine I have isn’t setup with LVM and I haven’t got the time to look into LVM either.
  • I’m quite the networking newbie, so my setup uses the default Xen bridge installed via apt-get. Check out Xen networking on the Xen wiki for more complex network setups.
  • I’m not gonna talk about putting a Linux distro other than that of the host machine into the Xen virtual machines. In other words, I have Feisty Fawn for both dom0 and all my domUs.
  • I probably did some completely stupid things. Stuff like xen-tools exist to simplify Xen administration, but I haven’t had a chance to use them. Let me know if you spot an error or have a better way of doing something please!

Some terms that deserve explaining (most guides out there just use “DomU” and “Dom0″ without any ceremony, and assume you know what they mean):

  • Domain-0 or Dom0: This refers to the host machine OS. You know, the OS of the actual physical server that you have.
  • DomU: This refers to a Xen guest domain. A DomU is a single Xen virtual machine. The “U” stands for “unprivileged” I believe.

OK let’s go!

Installing the Xen Server

First off, we need to install the Xen server on the host machine (or Dom0). Thankfully, there is a Xen server package in Feisty Fawn’s apt-get repository.

  • Make sure ‘universe’ is enabled in /etc/apt/sources.list and run: apt-get install ubuntu-xen-server
  • Reboot the machine. After boot, running sudo xm list should show you Domain-0 (which is the host machine that’s running Xen).
  • Edit /etc/xen/xend-config.sxp and uncomment this line:
    (network-script network-bridge)
  • Comment out this line:
    (network-script network-dummy)
  • Restart xend:
    sudo xend restart
  • When you run ‘ifconfig’, you should see the ‘xenbr0′ interface. This is the Xen network bridge.

Creating a base image

Now let’s create a base Xen image. We’ll use this as a template for any future images (domUs).

  • Use ‘dd’ to create the images (I place them in /xen-images/):
    dd if=/dev/zero of=/xen-images/feisty_base.img bs=1024k count=4000
    dd if=/dev/zero of=/xen-images/feisty_base_swap.img bs=1024k count=1000

    These commands create image files of 4GB and 1GB for your virtual OS and its swap respectively. ‘count’ is the number of blocks (block size = 1024k).

  • Create the filesystem (we’re using ext3 in this case) within the virtual OS image file.
    mkfs.ext3 /xen-images/feisty_base.img

    Answer ‘yes’ to proceed anyway when it complains about the file not being a block device.

  • Do the same for the swap file:
    mkswap /xen-images/feisty_base_swap.img
  • Now, restrict the permissions on the images so no one else but root can screw around with them.
    chmod 640 /xen-images/feisty_base*
  • Create a mount point for the image. We need this to mount the image to install and configure Ubuntu Feisty Fawn the way we like it. We can then replicate this base image for any future virtual machines.
    mkdir /xen-images/mnt
  • Mount the base image via the loop device:
    mount -o loop /xen-images/feisty_base.img /xen-images/mnt

    Now we have full access to the base image’s filesystem. What we need to do now is to put a base install of Feisty Fawn on it.

Installing and configuring the base image

Next, we have to put Ubuntu onto the image and configure it to our liking.

  • Install debootstrap, a handy tool that bootstraps a basic Debian (Ubuntu in this case) system.
    apt-get install debootstrap
  • Bootstrap the base image with ‘feisty’:
    debootstrap feisty /xen-images/mnt

    After this is done you should see the basic Ubuntu file hierarchy:

    ls /xen-images/mnt
  • Copy your sources.list into the base image:
    cp /etc/apt/sources.list /xen-images/mnt/etc/apt/
  • Copy your kernel modules as well (replace 2.6.19-4-server with whatever kernel version you have):
    cp -a /lib/modules/2.6.19-4-server/ /xen-images/mnt/lib/modules/
  • Install the libc6-xen package, which is a Xen-compatible glibc that allows applications to use Thread-Local Storage (TLS).
    apt-get install libc6-xen

    Taken from the Xen FAQ:

    Some modern distributions ship with a ‘TLS’ version of glibc that is not fully compatible with Xen. To use Xen reliably and with maximum performance you must disable the incompatible glibc.

  • Configure networking by editing /xen-images/mnt/etc/network/interfaces:
    auto lo
    iface lo inet loopback
    
    # Uncomment this and fill up with your networks settings.
    #auto eth0
    #iface eth0 inet static
    #  address 192.168.0.201
    #  netmask 255.255.255.0
    #  broadcast 192.168.255.255
    # gateway 192.168.0.1
    #  dns-nameservers 192.168.0.1

    Notice how the network settings are commented out – this is because we are leaving the base image as a template from which other images would be made from.

  • Edit the hosts file (/xen-images/mnt/etc/hosts):
    127.0.0.1       localhost localhost.localdomain
    127.0.0.1       guest
    
    # The following lines are desirable for IPv6 capable hosts
    ::1     ip6-localhost ip6-loopback
    fe00::0 ip6-localnet
    ff00::0 ip6-mcastprefix
    ff02::1 ip6-allnodes
    ff02::2 ip6-allrouters
    ff02::3 ip6-allhosts
  • Give the base image a generic hostname by editing /xen-images/mnt/etc/hostname. In this case I’m just naming it ‘guest’ – actual virtual machines would have sensible hostnames.
  • Edit the filesystem table (/xen-images/mnt/etc/fstab).
    proc            /proc           proc    defaults        0       0
    /dev/hda1       /               ext3    defaults,errors=remount-ro 0       1
    /dev/hda2       none            swap    sw              0       0

    Don’t worry, the /dev/hda* would be configured to point to the images you created, not to your actual physical harddisks.

  • You’re done with the initial configuration and have enough to bootup the image. Let’s unmount the image:
    umount /xen-images/mnt
  • Now, we need to create a Xen config for the base image so we can boot into it. Create a file named /etc/xen/vm_feisty_base.config.example.sxp (you can name it whatever you like of course):
    name = "feisty_base"
    kernel = "/boot/vmlinuz-2.6.19-4-server"
    ramdisk = "/boot/initrd.img-2.6.19-4-server"
    root = "/dev/hda1 ro"
    memory = 512
    disk = ['file:/xen-images/feisty_base.img,hda1,w','file:/xen-images/feisty_base_swap.img,hda2,w']
    
    # Network.
    hostname = "feisty_base"
    vif = ['bridge=xenbr0']
    dhcp = "off"
    ip = "192.168.0.201"
    netmask = "255.255.255.0"
    gateway = "192.168.0.1"

An explanation of the configuration options:

  • name – a descriptive name of your image.
  • kernel – path to the kernel.
  • ramdisk – path to the initrd (initial RAM disk) image.
  • root – point this to your host machine’s harddisk where your image files are.
  • memory – the amount of memory assigned to the virtual machine in megabytes
  • disk – specify a list of block devices that will be export to the virtual machine. In this example, we’re mapping the /xen-images/feisty_base.img image file to /dev/hda1 in the virtual image (same for the swap image). This should match whatever config you did in /xen-images/mnt/etc/fstab.
  • hostname – hostname of your virtual machine.
  • vif – virtual network interface that your virtual machine will use. In this case, we’re using the default ‘xenbr0′ bridge that Xen installs. For alternative configurations, check out the networking page on the Xen wiki.
  • ip – specify your virtual machine’s IP here.
  • netmask and gateway – netmask and gateway that your virtual machine should use.
  • Now, start the base virtual machine:
    xm create /etc/xen/vm_feisty_base.config.example.sxp -c

    You should see a console (the ‘-c’ option connects you to the console immediately after booting up the virtual machine. Login as root (empty password) – you’ll want to change the password (with ‘passwd’).

  • Enable shadowed passwords.
    shadowconfig on
  • Create a backup of /etc/network/interfaces and edit /etc/network/interfaces to put in correct values so you can access the Internet from the virtual machine:
    cp /etc/network/interfaces /etc/network/interfaces.bak
    vim /etc/network/interfaces
  • Start networking:
    ifup -a
  • Install the Ubuntu standard system after running an update:
    apt-get update
    apt-get install ubuntu-standard
  • I also like to upgrade any packages that have been updated, if any:
    apt-get upgrade
  • Install the OpenSSH server so that you can access the virtual machine via ssh:
    apt-get install ssh
  • At this point, do any configuration or installation of packages that you want this base image to have. Remember that we’re going to replicate this base image for future Xen virtual machines so don’t go overboard and install stuff you’re not gonna use.
  • Stop networking and put back the original /etc/network/interfaces:
    ifdown -a
    mv /etc/network/interfaces.bak /etc/network/interfaces
  • Exit the guest console with Ctrl-]. This will bring you back to dom0′s shell.
  • Shutdown the base Xen domU (-w means “wait for the domU to shutdown completely”):
    xm shutdown feisty_base -w

You’re done setting up the base image! Now you have a nicely configured base image.

Creating your first virtual server

The fruits of thy labor are almost at hand – with the base image, we can now make a copy of it for our first Xen virtual machine.

  • Make copies of the base image and the swap image (let’s call our virtual server “yuki”):
    cp /xen-images/feisty_base.img /xen-images/yuki.img
    cp /xen-images/feisty_base_swap.img /xen-images/yuki_swap.img

    You may want to resize the base image depending on your disk space needs. (And, of course, you can create your own swap image like we did earlier instead of copying the base image’s swap file.)

  • Make a copy of the base Xen configuration file that we created for our base image earlier (you can use whatever name you like):
    cp /etc/xen/vm_feisty_base.config.example.sxp /etc/xen/vm_yuki.config.sxp
  • Edit the /etc/xen/vm_yuki.config.sxp file, changing the values to suit your virtual server (you probably want to change at least the ‘name’, ‘disk’, ‘ip, and ‘hostname’ values).
    name = "yuki"
    disk = ['file:/xen-images/yuki.img,hda1,w','file:/xen-images/yuki_swap.img,hda2,w']
    hostname = "yuki"
    ip = "192.168.0.202"
  • Now start up your cloned virtual server and log into it like we did earlier with the base image (if you haven’t changed the password, you can login as root with an empty password).
    xm create -c /etc/xen/vm_yuki.config.sxp
  • The first thing you should do is to secure your root password (if you haven’t done so in the base image earlier).
    passwd
  • Now, you’ll want to change these files:
    • /etc/network/interfaces
    • /etc/hostname
    • /etc/hosts
  • With the proper network settings, you should be able to bring up networking to connect to the internet (assuming that your dom0 is connected to the internet).
    ifup -a
    ping google.com
  • At this point, you are free to customize your virtual machine as you see fit – install nginx for use as a static file server, Rails, MySQL server, whatever.
  • But first, let’s make sure this Xen virtual server starts up automatically on boot. Exit the virtual machine console with Ctrl-]
    ln -s /etc/xen/vm_yuki.config.sxp /etc/xen/auto/

    Any Xen domain configuration placed in /etc/xen/auto/ will be started up automatically on boot (and shutdown when the host machine is shutdown too).

That’s it! You have a functional Xen guest domain that starts up on boot and that can connect to the Internet. Oh before I forget, some useful commands when dealing with the Xen images:

  • xm create /path/to/config_file – Starts up the guest domain with the given Xen domU configuration file. (xm create -c to login to the console immediately after booting the image.)
  • xm console yuki – Open the console for the ‘yuki’ Xen domain.
  • xm shutdown yuki – Shutdown the ‘yuki’ domU.
  • xm list – Lists all the Xen domains you have running.

Resizing your virtual machine disk

Let’s say you wanna add 1GB of disk space to your virtual machine (named ‘yuki’).

Note that LVM is usually recommended for Xen setups, so you may wanna take a look at that instead.

  • Create a temporary 1GB file:
    dd if=/dev/zero of=/tmp/temp_expand bs=1024k count=1000
  • Bring down the virtual machine (I’ve found that a shutdown is needed so that the virtual machine recognizes the increased disk space):
    xm shutdown yuki
  • Append the file to the disk image of the virtual machine (/xen-images/yuki.img in this example). Be sure to use ‘>>’ (which means append) – using a single ‘>’ would overwrite the disk image with your temporary file, which you most definitely don’t want!
    cat /tmp/temp_expand >> /xen-images/yuki.img
  • Resize the file system:
    resize2fs -f /xen-images/yuki.img
  • Startup the virtual machine again:
    xm create -c /etc/xen/vm_yuki.config.sxp
  • Remember to login to your virtual machine to verify that the disk space has increased (df -h)

21 Responses to Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide

Avatar

Free as in Time » Blog Archive » Mmmm... Xen

June 28th, 2007 at 10pm

[...] I have yet to give this a try, but it looks very helpful: Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide. [...]

Avatar

Geekularity » links for 2007-06-29

June 30th, 2007 at 7am

[...] Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide – redemption in a blog (tags: ubuntu xen) [...]

Avatar

links for 2007-06-30 « My Weblog

June 30th, 2007 at 12pm

[...] Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide – redemption in a blog (tags: tutorial linux virtualization) [...]

Avatar

Alternageek » Show Notes for AlternaGeek Episode 7

July 24th, 2007 at 9pm

[...] XenExpress on Feisty: Complete Newbies Guide to Installing Xen on Ubuntu Feisty Ubuntu Forums – Installing [...]

Avatar

vainov

July 27th, 2007 at 8pm

I am following this path exactly as shown above. When I get to
xm create /etc/xen/vm_feisty_base.config.example.sxp -c

I get the following error:
Error: (22, 'Invalid argument')
I’m running on a Dell Dimension 8400 w a P4/3.0GHz computer w HT enabled.
Any ideas?
BTW:
There is a typo in the code sample
debootstrap feisty /xen-image/mnt

Should be:
debootstrap feisty /xen-images/mnt

Avatar

vainov

July 27th, 2007 at 9pm

Comment to my previous posting:
Kernel images v 2.6.20-15 don’t seem to work. If I use images v 2.6.19-4 then everything works fine.
Additional typo appears in listing for /etc/xen/vm_feisty_base.config.example.sxp

The swap file is named feisty_base_swap.img in the initial procedure, but in the sample the ‘disk =’ paramter requests a swap file named base_swap.img.

Avatar

Chu Yeow

July 28th, 2007 at 1pm

vainov: Thanks for pointing out the typos! I’ve fixed them. And I also faced a similar problem getting the 2.6.20-15 kernel images to work so sorry can’t help you there.

Avatar

Neil

August 21st, 2007 at 8am

My CPU will not support Xen, on an IBM T42 laptop, you need a PAE enabled CPU,

cat /proc/cpuinfo | grep pae will tell you if you can run Xen.

http://kbase.redhat.com/faq/FAQ_108_10486.shtm

Avatar

zucaritas

August 24th, 2007 at 10pm

I can’t get the network running, and when i do

I get an error saying
Package libc6-xen is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package libc6-xen has no installation candidate

I’m trying to install xen on a amd64 based laptop. Any help?
Thanks. The guide is great other than this particular prob.

Avatar

zucaritas

August 24th, 2007 at 10pm

sorry, it was when i do
apt-get install libc6-xen

Avatar

Two wins! « Awed Manor

September 9th, 2007 at 8am

[...] be able to import VMWare virtual machine images – but then I saw the “quick and easy” instructions.  No, thanks.  I know that some of it is involved in setting up the template so everything else [...]

Avatar

pradeep

September 9th, 2007 at 8pm

After the following steps, my network stops :
# Edit /etc/xen/xend-config.sxp and uncomment this line:

(network-script network-bridge)

# Comment out this line:

(network-script network-dummy)

# Restart xend:

sudo xend restart

I have a static IP assigned to eth0. Do I need any additional steps?

Thanks !!

– noob

Avatar

Mmmm… Xen

November 9th, 2007 at 2am

[...] I have yet to give this a try, but it looks very helpful: Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide. [...]

Avatar

Trung

November 29th, 2007 at 10am

The created VPS will sometime produce this error

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = “en_US.UTF-8″
are supported and installed on your system.
perl: warning: Falling back to the standard locale (“C”).

To fix this:

sudo locale-gen en_US.UTF-8
sudo dpkg-reconfigure locales

(from https://help.ubuntu.com/community/Xen)

Avatar

Kalmi

December 1st, 2007 at 1am

multiverse needs to be enabled too…
I needed it for the xen-linux package…

Avatar

A Fresh Cup » Blog Archive » Double Shot #66

December 29th, 2007 at 9pm

[...] Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide – I hope not to have to tackle this any time soon, but if I do, this looks like a good unified guide. [...]

Avatar

???0000?????? » Blog Archive » ??xen?????????

March 10th, 2008 at 4pm

[...] Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide [...]

Avatar

Sailface » Blog Archive » links for 2008-07-18

July 18th, 2008 at 8pm

[...] Installing Xen on Ubuntu Feisty Fawn – the complete newbie’s guide | redemption in a blog (tags: xen ubuntu virtualization) [...]

Avatar

Assertions and Deviations :: Virtualization :: Blank or black screen when booting Ubuntu on Xen

May 29th, 2009 at 12pm

[...] with a generic Intel 960 series motherboard and a lack-tastic X3100 video card). There are various tutorials and howtos on doing this, so it went [...]

Avatar

Setting up virtualization on Ubuntu with KVM - redemption in a blog

February 1st, 2010 at 10pm

[...] being a happy Xen user for several years now, I’ve recently had to switch to an alternative virtualization solution. [...]

Avatar

Some useful Links and References - Xen Tutorial

October 4th, 2010 at 9pm