After being a happy Xen user for several years now, I’ve recently had to switch to an alternative virtualization solution. My colleague Arun (@iamclovin) actually struggled for a week with Xen VMs that locked up on Hardy; we’ve had much success with Hardy and Xen before, so we attributed it to a hardware problem since these were our first blade servers.

Out of ideas, we tried Karmic (Ubuntu 9.10) only to discover that Xen support via the apt package system is gone. I went down the path of compiling a paravirt_ops Dom0 kernel (this article was very useful) but ended up deciding the process took far too long despite being successful.

With KVM gaining official support from Ubuntu as the virtualization solution, I ended up ditching Xen and switching to KVM for these new servers on Karmic. The rest of the entry is a step-by-step guide on setting up KVM VMs on a Ubuntu server; I’m putting this down because like all wikis, the Ubuntu KVM wiki has grown a little too organically to be useful.

Preparing a host server for KVM

1. Update and upgrade apt packages (use your own discretion on whether this is necessary):

   aptitude update && aptitude dist-upgrade

2. Check whether CPU supports hardware virtualization:

   egrep '(vmx|svm)' --color=always /proc/cpuinfo

   You should see lines with either “vmx” or “svm” highlighted.

3. Install these packages:

   aptitude install kvm libvirt-bin ubuntu-vm-builder bridge-utils

   If you see a FATAL: Error inserting kvm_intel message during installation, it means that virtualization is not enabled in your machine’s BIOS. You’ll need to reboot your machine, enter the BIOS setup and enable virtualization (you’ll have to hunt for the option).
   
   After enabling virtualization in the BIOS and rebooting, run:

   modprobe kvm-intel

   There should be no error shown (in fact, no console response).

4. Optionally, install virt-top, a top-like tool for your VMs:

   aptitude install virt-top

5. Verify that you can connect to the hypervisor:

   virsh -c qemu:///system list

   You should see something like this:

   Connecting to uri: qemu:///system
    Id Name                 State
    ----------------------------------

6. Setup a network bridge on the server for VMs. Edit /etc/network/interfaces so it looks like this (use your own IPs):

   auto lo
   iface lo inet loopback
   
   auto eth0
   iface eth0 inet manual
   
   auto br0
   iface br0 inet static
    address 192.168.1.222
    netmask 255.255.255.0
    network 192.168.1.0
    broadcast 192.168.1.255
    gateway 192.168.1.167
    bridge_ports eth0
    bridge_stp off
    bridge_fd 9
    bridge_hello 2
    bridge_maxage 12
    bridge_maxwait 0

7. Make sure that you have a direct console to the server because you’re going to restart networking:

   /etc/init.d/networking restart

8. Verify that your changes took place with ifconfig. You should see 2 entries like these:

   br0       Link encap:Ethernet  HWaddr 00:11:22:33:44:55
             inet addr:192.168.1.215  Bcast:192.168.1.255  Mask:255.255.255.0
             inet6 addr: fe80::223:aeff:fefe:1f14/64 Scope:Link
             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
             RX packets:1099 errors:0 dropped:0 overruns:0 frame:0
             TX packets:50 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:0
             RX bytes:74665 (74.6 KB)  TX bytes:6223 (6.2 KB)
   
   eth0      Link encap:Ethernet  HWaddr 00:66:77:88:99:00
             inet6 addr: fe80::223:aeff:fefe:1f14/64 Scope:Link
             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
             RX packets:4939 errors:0 dropped:0 overruns:0 frame:0
             TX packets:39 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:1000
             RX bytes:532798 (532.7 KB)  TX bytes:5585 (5.5 KB)
             Interrupt:36 Memory:da000000-da012800

Setting up a VM

1. Setup a VM with vmbuilder:

   vmbuilder kvm ubuntu \
       -v \
       --suite=karmic \
       --libvirt=qemu:///system \
       --arch=amd64 \
       --cpus=2 \
       --mem=2048 \
       --swapsize=4096 \
       --rootsize=20480 \
       --flavour=server \
       --hostname=billiejean \
       --ip=192.168.1.240 \
       --mask=255.255.255.0 \
       --net=192.168.1.0 \
       --bcast=192.168.1.255 \
       --gw=192.168.1.167 \
       --dns='202.157.163.157 202.157.131.118' \
       --bridge=br0 \
       --mirror=http://archive.ubuntu.com/ubuntu \
       --components='main,universe' \
       --addpkg=openssh-server \
       --user=administrator \
       --pass=icanhaspasswd \
       --dest=/root/vm-billiejean \
       --tmpfs=-

   The options you need to care about are:

   1. suite: Version of Ubuntu to install (e.g. karmic, hardy).
   2. cpus: Number of CPUs to assign to VM.
   3. mem: Amount of RAM in MB to assign to VM.
   4. swapsize: Size of swap in MB of VM.
   5. rootsize: Size of root filesystem in MB of VM.
   6. flavour: The “flavour” of kernel to use in the VM. Either “virtual” or “server”.
   7. hostname: Hostname of VM.
   8. ip: IP address of VM.
   9. mask: Netmask of VM.
   10. net: Network of VM.
   11. bcast: Broadcast address of VM.
   12. gw: Gateway of VM.
   13. dns: DNS server(s) for VM.
   14. addpkg: APT packages to install in the VM. openssh-server is needed so that we can login to the VM to setup the virsh console.
   15. user and pass: User account that’s setup for you to access the VM.
   16. dest: Destination directory on server where VM disk image will reside.
2. If your VM is created successfully, there’ll be a config file for the VM in /etc/libvirt/qemu/ (e.g. /etc/libvirt/qemu/billiejean.xml), and a disk image in the directory specified in the --dest option (e.g. /root/vm-billiejean/disk0.qcow2).
3. You can verify that it works by starting the VM and SSHing into it (virsh console will not work yet).

   virsh start billiejean

Converting Disk Images to LVM Logical Volumes

Now, we have the VM setup but it’s running off a disk image. For better performance, running the VM off a LVM logical volume will optimize disk IO.

vmbuilder is supposed to support the --raw option to write the VM to a block device (such as a LVM logical volume), but I’ve had no success with it (as does Mark Imbriaco, sysadmin of 37signals: http://twitter.com/markimbriaco/status/7437688341 and http://twitter.com/markimbriaco/status/7437699338). We’re going to convert the disk images using qemu-img and write the bits into a LVM logical volume instead.

1. Stop the VM if it’s running:

   virsh shutdown billiejean

2. Convert the VM’s qcow2 (QEMU image format) disk image to a raw disk image:

   qemu-img convert disk0.qcow2 -O raw disk0.raw

3. Create a logical volume to house the VM, making sure it’s big enough for the VM’s rootsize and swapsize options:

   lvcreate -L24GB -n <logical_volume_name> <volume_group_name>

4. Copy raw image into the logical volume:

   dd if=disk0.raw of=/dev/<volume_group_name>/<logical_volume_name> bs=1M

   This will take awhile (the bigger your image, the longer it takes).

5. Edit the VM’s config so that it uses your new logical volume:

   virsh edit billiejean

   Change <disk> to point to the logical volume:

   <disk type='block' device='disk'>
     <source dev='/dev/<volume_group_name>/<logical_volume_name>'/>
     <target dev='hda' bus='ide'/>
   </disk>

6. Startup the VM. You might wanna rename the original disk0.qcow2 image first just to make sure your VM isn’t still using it.
7. Once you’re sure your VM is running off your LVM logical volume, you can delete or backup the original qcow2 disk image.

Getting a console to your VM from the Host Server

Now, we have to setup the VM so that virsh console works. This is a console to the VM from the host server that works even when the networking in the VM is not.

1. Edit the VM’s settings:

   virsh edit billiejean

   In the <devices> block, add:

   <serial type='pty'>
     <target port='0'/>
   </serial>

2. Startup your VM:

   virsh start billiejean

3. SSH into VM and create a file /etc/init/ttyS0.conf:

   start on stopped rc RUNLEVEL=[2345]
   stop on runlevel [!2345]
   
   respawn
   exec /sbin/getty -8 38400 ttyS0 vt102

   Start the tty with:

   start ttyS0

4. Still in the VM, install acpid so that the VM will respond to shutdown commands from the server:

   aptitude install acpid

5. Reboot the VM.
6. Verify the console works by opening a console to the VM from the server:

   virsh console billiejean

   You may have to hit “Enter” before you see any console output.

Miscellaneous VM Setup

That’s it, your VM is ready! You’ll probably want to do these:

• Set a root password and possibly delete the user you setup using vmbuilder.
• Set the timezone with dpkg-reconfigure tzdata.

God.meddle do |god|
  god.watch do |w|
    ...
  end
end

Interesting code to write. And feels safe too to have God watching over your mongrels. Oh and you can use it to clean up stale mongrel PID files without patching mongrel_rails. Right now I use monit a lot and it’s really rather unDRY to have repeated configs for each mongrel on the server (is there a better way of doing this in monit?). God would be able to DRY it up.

/var/railsapps/APP_NAME/shared/log/production.log {
  daily
  rotate 28
  missingok
  compress
  sharedscripts
  postrotate
    for i in `ls /var/railsapps/APP_NAME/shared/log/*.pid`; do
      kill -USR2 `cat $i`
    done
  endscript
}

Thanks to Rimuhosting‘s fantabulous support, I managed to regain access to the VPSs. Oh yeah, big lesson learnt here: when configuring NIS clients, be sure to leave at least one user account with sudo privileges or set the root account’s password (Ubuntu doesn’t come with a root account enabled by default – run sudo passwd to set it and enable the root account as a side effect). Big big lesson learnt. Don’t let this bite you in the ass if you’re a sysadmin-ish person.

Anyway, /var/log/syslog was complaining this:

ypbind[1026]: Unable to register (YPBINDPROG, YPBINDVERS, udp)

Which turned out to be a portmap problem. So yeah portmap wasn’t running! For a moment there I thought we had an easy solution. Sadly, /etc/init.d/portmap start was throwing this inexplicable error (i.e. Google wasn’t of much help):

root@zoidberg:/sbin# /etc/init.d/portmap  restart
 * Stopping portmap daemon...
   ...done.
 * Starting portmap daemon...
dhclient.4.x 2.*.5, December 30th, 2000
Compiled on Dec 12 2003 09:55:54
Features: DBG, SEE, ALS, SEF
init: Couldn't open the file ./libuuid.so ->
   ...fail!

It sure isn’t caturday. After hours spent scrambling to find out why portmap can’t find libuuid.so, reinstalling the NIS .deb, comparing configuration files between working NIS clients and these busted ones, it finally dawned on me that I have yet to re-install portmap (not that it was logically sound that I’d have to reinstall it, but at this point I was willing to try anything). Annoyingly enough, it worked!

apt-get remove portmap nis
apt-get install nis

I still don’t know WTF happened there. If anyone has any clue please please point me to it.

So I think it’s a good idea to write this down somewhere for the next time I have to do the same bloody thing over again on any new servers.

Some caveats before we start:

• Like I said, I’m not gonna talk about LVM for disk images since I know next to nothing about configuring it at the moment. The host machine I have isn’t setup with LVM and I haven’t got the time to look into LVM either.
• I’m quite the networking newbie, so my setup uses the default Xen bridge installed via apt-get. Check out Xen networking on the Xen wiki for more complex network setups.
• I’m not gonna talk about putting a Linux distro other than that of the host machine into the Xen virtual machines. In other words, I have Feisty Fawn for both dom0 and all my domUs.
• I probably did some completely stupid things. Stuff like xen-tools exist to simplify Xen administration, but I haven’t had a chance to use them. Let me know if you spot an error or have a better way of doing something please!

Some terms that deserve explaining (most guides out there just use “DomU” and “Dom0″ without any ceremony, and assume you know what they mean):

• Domain-0 or Dom0: This refers to the host machine OS. You know, the OS of the actual physical server that you have.
• DomU: This refers to a Xen guest domain. A DomU is a single Xen virtual machine. The “U” stands for “unprivileged” I believe.

OK let’s go!

Installing the Xen Server

First off, we need to install the Xen server on the host machine (or Dom0). Thankfully, there is a Xen server package in Feisty Fawn’s apt-get repository.

• Make sure ‘universe’ is enabled in /etc/apt/sources.list and run: apt-get install ubuntu-xen-server
• Reboot the machine. After boot, running sudo xm list should show you Domain-0 (which is the host machine that’s running Xen).
• Edit /etc/xen/xend-config.sxp and uncomment this line:

  (network-script network-bridge)

• Comment out this line:

  (network-script network-dummy)

• Restart xend:

  sudo xend restart

• When you run ‘ifconfig’, you should see the ‘xenbr0′ interface. This is the Xen network bridge.

Creating a base image

Now let’s create a base Xen image. We’ll use this as a template for any future images (domUs).

• Use ‘dd’ to create the images (I place them in /xen-images/):

  dd if=/dev/zero of=/xen-images/feisty_base.img bs=1024k count=4000
  dd if=/dev/zero of=/xen-images/feisty_base_swap.img bs=1024k count=1000

  These commands create image files of 4GB and 1GB for your virtual OS and its swap respectively. ‘count’ is the number of blocks (block size = 1024k).

• Create the filesystem (we’re using ext3 in this case) within the virtual OS image file.

  mkfs.ext3 /xen-images/feisty_base.img

  Answer ‘yes’ to proceed anyway when it complains about the file not being a block device.

• Do the same for the swap file:

  mkswap /xen-images/feisty_base_swap.img

• Now, restrict the permissions on the images so no one else but root can screw around with them.

  chmod 640 /xen-images/feisty_base*

• Create a mount point for the image. We need this to mount the image to install and configure Ubuntu Feisty Fawn the way we like it. We can then replicate this base image for any future virtual machines.

  mkdir /xen-images/mnt

• Mount the base image via the loop device:

  mount -o loop /xen-images/feisty_base.img /xen-images/mnt

  Now we have full access to the base image’s filesystem. What we need to do now is to put a base install of Feisty Fawn on it.

Installing and configuring the base image

Next, we have to put Ubuntu onto the image and configure it to our liking.

• Install debootstrap, a handy tool that bootstraps a basic Debian (Ubuntu in this case) system.

  apt-get install debootstrap

• Bootstrap the base image with ‘feisty’:

  debootstrap feisty /xen-images/mnt

  After this is done you should see the basic Ubuntu file hierarchy:

  ls /xen-images/mnt

• Copy your sources.list into the base image:

  cp /etc/apt/sources.list /xen-images/mnt/etc/apt/

• Copy your kernel modules as well (replace 2.6.19-4-server with whatever kernel version you have):

  cp -a /lib/modules/2.6.19-4-server/ /xen-images/mnt/lib/modules/

• Install the libc6-xen package, which is a Xen-compatible glibc that allows applications to use Thread-Local Storage (TLS).

  apt-get install libc6-xen

  Taken from the Xen FAQ:

  Some modern distributions ship with a ‘TLS’ version of glibc that is not fully compatible with Xen. To use Xen reliably and with maximum performance you must disable the incompatible glibc.

• Configure networking by editing /xen-images/mnt/etc/network/interfaces:

  auto lo
  iface lo inet loopback
  
  # Uncomment this and fill up with your networks settings.
  #auto eth0
  #iface eth0 inet static
  #  address 192.168.0.201
  #  netmask 255.255.255.0
  #  broadcast 192.168.255.255
  # gateway 192.168.0.1
  #  dns-nameservers 192.168.0.1

  Notice how the network settings are commented out – this is because we are leaving the base image as a template from which other images would be made from.

• Edit the hosts file (/xen-images/mnt/etc/hosts):

  127.0.0.1       localhost localhost.localdomain
  127.0.0.1       guest
  
  # The following lines are desirable for IPv6 capable hosts
  ::1     ip6-localhost ip6-loopback
  fe00::0 ip6-localnet
  ff00::0 ip6-mcastprefix
  ff02::1 ip6-allnodes
  ff02::2 ip6-allrouters
  ff02::3 ip6-allhosts

• Give the base image a generic hostname by editing /xen-images/mnt/etc/hostname. In this case I’m just naming it ‘guest’ – actual virtual machines would have sensible hostnames.
• Edit the filesystem table (/xen-images/mnt/etc/fstab).

  proc            /proc           proc    defaults        0       0
  /dev/hda1       /               ext3    defaults,errors=remount-ro 0       1
  /dev/hda2       none            swap    sw              0       0

  Don’t worry, the /dev/hda* would be configured to point to the images you created, not to your actual physical harddisks.

• You’re done with the initial configuration and have enough to bootup the image. Let’s unmount the image:

  umount /xen-images/mnt

• Now, we need to create a Xen config for the base image so we can boot into it. Create a file named /etc/xen/vm_feisty_base.config.example.sxp (you can name it whatever you like of course):

  name = "feisty_base"
  kernel = "/boot/vmlinuz-2.6.19-4-server"
  ramdisk = "/boot/initrd.img-2.6.19-4-server"
  root = "/dev/hda1 ro"
  memory = 512
  disk = ['file:/xen-images/feisty_base.img,hda1,w','file:/xen-images/feisty_base_swap.img,hda2,w']
  
  # Network.
  hostname = "feisty_base"
  vif = ['bridge=xenbr0']
  dhcp = "off"
  ip = "192.168.0.201"
  netmask = "255.255.255.0"
  gateway = "192.168.0.1"

An explanation of the configuration options:

• name – a descriptive name of your image.
• kernel – path to the kernel.
• ramdisk – path to the initrd (initial RAM disk) image.
• root – point this to your host machine’s harddisk where your image files are.
• memory – the amount of memory assigned to the virtual machine in megabytes
• disk – specify a list of block devices that will be export to the virtual machine. In this example, we’re mapping the /xen-images/feisty_base.img image file to /dev/hda1 in the virtual image (same for the swap image). This should match whatever config you did in /xen-images/mnt/etc/fstab.
• hostname – hostname of your virtual machine.
• vif – virtual network interface that your virtual machine will use. In this case, we’re using the default ‘xenbr0′ bridge that Xen installs. For alternative configurations, check out the networking page on the Xen wiki.
• ip – specify your virtual machine’s IP here.
• netmask and gateway – netmask and gateway that your virtual machine should use.

• Now, start the base virtual machine:

  xm create /etc/xen/vm_feisty_base.config.example.sxp -c

  You should see a console (the ‘-c’ option connects you to the console immediately after booting up the virtual machine. Login as root (empty password) – you’ll want to change the password (with ‘passwd’).

• Enable shadowed passwords.

  shadowconfig on

• Create a backup of /etc/network/interfaces and edit /etc/network/interfaces to put in correct values so you can access the Internet from the virtual machine:

  cp /etc/network/interfaces /etc/network/interfaces.bak
  vim /etc/network/interfaces

• Start networking:

  ifup -a

• Install the Ubuntu standard system after running an update:

  apt-get update
  apt-get install ubuntu-standard

• I also like to upgrade any packages that have been updated, if any:

  apt-get upgrade

• Install the OpenSSH server so that you can access the virtual machine via ssh:

  apt-get install ssh

• At this point, do any configuration or installation of packages that you want this base image to have. Remember that we’re going to replicate this base image for future Xen virtual machines so don’t go overboard and install stuff you’re not gonna use.
• Stop networking and put back the original /etc/network/interfaces:

  ifdown -a
  mv /etc/network/interfaces.bak /etc/network/interfaces

• Exit the guest console with Ctrl-]. This will bring you back to dom0′s shell.
• Shutdown the base Xen domU (-w means “wait for the domU to shutdown completely”):

  xm shutdown feisty_base -w

You’re done setting up the base image! Now you have a nicely configured base image.

Creating your first virtual server

The fruits of thy labor are almost at hand – with the base image, we can now make a copy of it for our first Xen virtual machine.

• Make copies of the base image and the swap image (let’s call our virtual server “yuki”):

  cp /xen-images/feisty_base.img /xen-images/yuki.img
  cp /xen-images/feisty_base_swap.img /xen-images/yuki_swap.img

  You may want to resize the base image depending on your disk space needs. (And, of course, you can create your own swap image like we did earlier instead of copying the base image’s swap file.)

• Make a copy of the base Xen configuration file that we created for our base image earlier (you can use whatever name you like):

  cp /etc/xen/vm_feisty_base.config.example.sxp /etc/xen/vm_yuki.config.sxp

• Edit the /etc/xen/vm_yuki.config.sxp file, changing the values to suit your virtual server (you probably want to change at least the ‘name’, ‘disk’, ‘ip, and ‘hostname’ values).

  name = "yuki"
  disk = ['file:/xen-images/yuki.img,hda1,w','file:/xen-images/yuki_swap.img,hda2,w']
  hostname = "yuki"
  ip = "192.168.0.202"

• Now start up your cloned virtual server and log into it like we did earlier with the base image (if you haven’t changed the password, you can login as root with an empty password).

  xm create -c /etc/xen/vm_yuki.config.sxp

• The first thing you should do is to secure your root password (if you haven’t done so in the base image earlier).

  passwd

• Now, you’ll want to change these files:
  • /etc/network/interfaces
  • /etc/hostname
  • /etc/hosts
• With the proper network settings, you should be able to bring up networking to connect to the internet (assuming that your dom0 is connected to the internet).

  ifup -a
  ping google.com

• At this point, you are free to customize your virtual machine as you see fit – install nginx for use as a static file server, Rails, MySQL server, whatever.
• But first, let’s make sure this Xen virtual server starts up automatically on boot. Exit the virtual machine console with Ctrl-]

  ln -s /etc/xen/vm_yuki.config.sxp /etc/xen/auto/

  Any Xen domain configuration placed in /etc/xen/auto/ will be started up automatically on boot (and shutdown when the host machine is shutdown too).

That’s it! You have a functional Xen guest domain that starts up on boot and that can connect to the Internet. Oh before I forget, some useful commands when dealing with the Xen images:

• xm create /path/to/config_file – Starts up the guest domain with the given Xen domU configuration file. (xm create -c to login to the console immediately after booting the image.)
• xm console yuki – Open the console for the ‘yuki’ Xen domain.
• xm shutdown yuki – Shutdown the ‘yuki’ domU.
• xm list – Lists all the Xen domains you have running.

Resizing your virtual machine disk

Let’s say you wanna add 1GB of disk space to your virtual machine (named ‘yuki’).

Note that LVM is usually recommended for Xen setups, so you may wanna take a look at that instead.

• Create a temporary 1GB file:

  dd if=/dev/zero of=/tmp/temp_expand bs=1024k count=1000

• Bring down the virtual machine (I’ve found that a shutdown is needed so that the virtual machine recognizes the increased disk space):

  xm shutdown yuki

• Append the file to the disk image of the virtual machine (/xen-images/yuki.img in this example). Be sure to use ‘>>’ (which means append) – using a single ‘>’ would overwrite the disk image with your temporary file, which you most definitely don’t want!

  cat /tmp/temp_expand >> /xen-images/yuki.img

• Resize the file system:

  resize2fs -f /xen-images/yuki.img

• Startup the virtual machine again:

  xm create -c /etc/xen/vm_yuki.config.sxp

• Remember to login to your virtual machine to verify that the disk space has increased (df -h)

Some of the features are pretty cool, most of them are mundane. For me, I am really looking forward to:

• the new Finder – Finder sucks so bad as a file system interface. I use Path Finder, which incidentally had an upgrade recently to 4.7. The new Finder looks pretty (Cover Flow for files, sexy!) but what I’m really hoping for is a Finder with which you can actually be productive.
• Ruby and Rails baked right into Mac OS X – while installing Rails and upgrading Ruby is a breeze on Mac OS X as it is right now, having these installed by default is pretty sweet. Even Capistrano will be included. Now, how one upgrades Ruby is another thing though…
• Time Machine – Even though I already own a licensed copy of SuperDuper!, I’m still eager to use Time Machine. I mean, who isn’t hooked on the time travel metaphor yet? It’s like System Restore done right (with the advantage of hindsight, of course).

What are you looking forward in Leopard?

Having just upgraded to Ubuntu Feisty Fawn, I set to installing nginx and tried to get WordPress (which needs PHP, of course) running on it. Thanks to the new nginx package in the Feisty universe repository, getting nginx up and PHP (CGI version) and running is really straightforward. There’re lots of tutorials to help you out as well.

So that’s that. The part that bugged me though, was how the tutorials always provided a script to start your PHP FastCGI processes, but never provided a way to ensure your FastCGI processes started up on a reboot (i.e. an init script). Once again, Google doesn’t disappoint, turning up this php-cgi init script. Tweak the init script a little, put it in /etc/init.d/php-fastcgi, sudo chmod +x /etc/init.d/php-fastcgi, run sudo update-rc.d php-fastcgi defaults and place the configuration for the script in /etc/default/php-fastcgi, and you’re done. You now have a PHP FastCGI init script that spawns and kills your PHP FastCGI processes.

Note: Lighttpd does come with a spawn-fcgi binary that does the same, but I’m having trouble finding a suitable init script for spawn-fcgi (only the Gentoo emerge seems to have the init script). Still, this script works and I didn’t really want to install lightty on my starving VPS.

The init script: /etc/init.d/php-fastcgi

#! /bin/sh
### BEGIN INIT INFO
# Provides:          php-fastcgi
# Required-Start:    $all
# Required-Stop:     $all
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start and stop php-cgi in external FASTCGI mode
# Description:       Start and stop php-cgi in external FASTCGI mode
### END INIT INFO

# Author: Kurt Zankl <[EMAIL PROTECTED]>

# Do NOT "set -e"

PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="php-cgi in external FASTCGI mode"
NAME=php-fastcgi
DAEMON=/usr/bin/php-cgi
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
PHP_CONFIG_FILE=/etc/php5/cgi/php.ini

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions

# If the daemon is not enabled, give the user a warning and then exit,
# unless we are stopping the daemon
if [ "$START" != "yes" -a "$1" != "stop" ]; then
        log_warning_msg "To enable $NAME, edit /etc/default/$NAME and set START=yes"
        exit 0
fi

# Process configuration
export PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS
DAEMON_ARGS="-q -b $FCGI_HOST:$FCGI_PORT -c $PHP_CONFIG_FILE"

do_start()
{
        # Return
        #   0 if daemon has been started
        #   1 if daemon was already running
        #   2 if daemon could not be started
        start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
                || return 1
        start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON \
                --background --make-pidfile --chuid $EXEC_AS_USER --startas $DAEMON -- \
                $DAEMON_ARGS \
                || return 2
}

do_stop()
{
        # Return
        #   0 if daemon has been stopped
        #   1 if daemon was already stopped
        #   2 if daemon could not be stopped
        #   other if a failure occurred
        start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE > /dev/null # --name $DAEMON
        RETVAL="$?"
        [ "$RETVAL" = 2 ] && return 2
        # Wait for children to finish too if this is a daemon that forks
        # and if the daemon is only ever run from this initscript.
        # If the above conditions are not satisfied then add some other code
        # that waits for the process to drop all resources that could be
        # needed by services started subsequently.  A last resort is to
        # sleep for some time.
        start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
        [ "$?" = 2 ] && return 2
        # Many daemons don't delete their pidfiles when they exit.
        rm -f $PIDFILE
        return "$RETVAL"
}
case "$1" in
  start)
        [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
        do_start
        case "$?" in
                0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
                2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
        esac
        ;;
  stop)
        [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
        do_stop
        case "$?" in
                0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
                2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
        esac
        ;;
  restart|force-reload)
        log_daemon_msg "Restarting $DESC" "$NAME"
        do_stop
        case "$?" in
          0|1)
                do_start
                case "$?" in
                        0) log_end_msg 0 ;;
                        1) log_end_msg 1 ;; # Old process is still running
                        *) log_end_msg 1 ;; # Failed to start
                esac
                ;;
          *)
                # Failed to stop
                log_end_msg 1
                ;;
        esac
        ;;
  *)
        echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
        exit 3
        ;;
esac

Config file for init script (the init script looks for this): /etc/default/php-fastcgi

START=yes

# Which user runs PHP? (default: www-data)

EXEC_AS_USER=www-data

# Host and TCP port for FASTCGI-Listener (default: localhost:9000)

FCGI_HOST=localhost
FCGI_PORT=9000

# Environment variables, which are processed by PHP

PHP_FCGI_CHILDREN=4
PHP_FCGI_MAX_REQUESTS=1000

nginx config file: /etc/nginx/nginx.conf

location ~ \.php$ {
  fastcgi_pass   127.0.0.1:9000;
  fastcgi_index  index.php;
  fastcgi_param  SCRIPT_FILENAME  /var/www/blog.codefront.net$fastcgi_script_name;
  include        /etc/nginx/fastcgi.conf;
}

My fastcgi.conf file (I’m not sure I need everything here…)

fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;

fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;

fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx;

fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;

# PHP only, required if PHP was built with --enable-force-cgi-redirect
fastcgi_param  REDIRECT_STATUS    200;

If you’ve found other neat ways to use the free Ubuntu stickers (or even the Apple stickers that comes with iPods), do post it!

I’ve been using wget for its auto-resume support, but have switched to using Axel since one of my colleagues at Bezurk introduced me to it. If you’re a wget or curl fan, Axel is almost a drop-in replacement (although it doesn’t handle multiple redirects or broken connections too well).

Install it:

# On a Mac, with Darwin Ports.
sudo port install axel

# On Ubuntu.
sudo apt-get install axel

Windows users would require cygwin to get Axel to work for them (what, a Windows user and you don’t have cygwin installed already?).

Now go download some files. If you need a good place just to test the speeds, go to YUI Theater and download some videos (watch them too, most of them are pretty good, like Douglas Crockford’s and the Firebug videos). Run it on the command line by typing:

axel -n 10 http://example.com/some_file.mov

The -n 10 option tells Axel to use a maximum of 10 simultaneous connections when downloading the file. Another useful option is -a, which outputs a wget-like report of download progress in a few lines rather than filling up your screen with download progress messages.

Check out the speeds I managed to get:

250MB in 3 minutes, with an average download speed of 1339KB/s. That’s pretty damn fast. Comparatively, I could only get speeds of around 40KB/s using Firefox. It’s hard not to love this raw speed and I think you might too.

Oh, and to go off-topic here, it’s nice to know we are steadily chipping away at the Windows user base in Bezurk. The colleague who introduced me to Axel recently switched to Ubuntu (he’s been waiting for Ubuntu 7.04). It was painful to set it up correctly (the KDE part of Kubuntu, not Ubuntu itself), but I think he’s much happier working on Linux for some inexplicable reason.

Phew, and that was all the work you needed to do to migrate your Subversion repository to another server. We’ve barely touched that other server that we wanted to use for mirroring the repository!

“Bootstrapping” your mirror SVN server for svnsync

This mirror SVN server also needs Subversion 1.4.x installed, so go ahead and do the (almost) same things we’ve done in part 1 to get Subversion installed. You should be able to use the .deb package generated by checkinstall on your main Subversion server to install Subversion 1.4.x on the mirror SVN server. Just scp it to the mirror SVN server and install it (dpkg -i subversion-1.4.3.deb).

With Subversion installed, create a new Subversion repository (using svnadmin create, remember?), but don’t load the repository dump like we did on the main SVN server in part 1 (of course, since we are going to be mirroring the repository!).

Setting up svnsync to mirror your repository

First, create a SVN user for svnsync to use – let’s call this the ‘svnsync user’. The easiest (and best) way to do this is edit the svnserve.conf and passwd files:

conf/svnserve.conf

# Uncomment this line.
password-db = passwd

conf/passwd

svnsync = secret

This gives read and write access to the ‘svnsync user’. The svnsync program will authenticate with our repositories as this user via the svn:// protocol (i.e. via svnserve).

Next, we need to create a pre-revprop-change hook for the destination repository. The svnsync documentation has a detailed explanation. Create a hooks/pre-revprop-change file under your destination repository’s directory.

#!/bin/sh
USER="$3"

if [ "$USER" = "svnsync" ]; then exit 0; fi

echo "Only the svnsync user can change revprops" >&2
exit 1

Make it executable, and then initialize the sync:

chmod +x hooks/pre-revprop-change
svnsync init file:///var/svn/repositories/destination_repos svn://source.host/source_repos

Don’t worry, this only sets up the sync – there’s no actual data copying yet. Syncing your repository data may take a long time if you have a big source repository, so I suggest using nohup to run the code overnight (or something), or at least saving the output in a log. Either way, the command to start the sync is:

svnsync sync --username svnsync file:///var/svn/repositories/testsync/

You should start seeing svnsync committing in changes from your source repository. Instant gratification (well, almost)! Should your svnsync process get aborted or killed, you can remove the hanging lock by running:

svn propdel svn:sync-lock --revprop -r 0

Setting up ‘on-the-fly’ syncing

So now you have your source and destination repositories synced, but what happens when you start committing changes to your source repository? Nothing! That’s because svnsync is merely a passive syncing tool (meaning you have to run it to sync, instead of it knowing when to sync automatically).

There are two ways you can setup ‘real-time’ syncing:

1. Use cron (or a similar scheduler) on the destination repository server. Add something like this to your crontab:

   * * * * * /usr/local/bin/svnsync --non-interactive sync svn://source.host/source_repos

   This basically runs svnsync on your destination repository server every minute to pull down any changes to your source repository.

2. Add a post-commit hook to the source repository. I found this svnsync entry by Paul Querna that has a sample post-commit hook. If I recall correctly I tried it but it didn’t work for me, so I settled on using cron to sync up my repositories.

Things that I skipped

There’re some things that I skipped over while writing this, mainly to do with SVN authentication.

• If you’re accessing your repository via the svn+ssh:// protocol, you’ve to manage the (group) permissions of the repository files in the filesystem appropriately (basically the repository should be group writable by your users). chmod and chown are your friends, as is NIS (or something similar) to manage your users. I use these steps to create a new SVN repository that gets access via the svn+ssh:// protocol:

  
  sudo mkdir /var/svn/repositories/funky_project
  mkdir /tmp/funky_project
  mkdir /tmp/funky_project/trunk
  mkdir /tmp/funky_project/branches
  mkdir /tmp/funky_project/tags
  sudo svnadmin create /var/svn/repositories/funky_project
  sudo svn import /tmp/YourProjectNameHere file:////var/svn/repositories/funky_project -m "Initial import."
  rm -rf /tmp/funky_project
  sudo chown -R www-data:www-data /var/svn/repositories/funky_project
  sudo chmod -R g+w /var/svn/repositories/funky_project
      

  As you can see, my SVN users are part of the www-data group, and the repository directory is made group-writable.

• The svn:// protocol has authentication configuration files in the conf/ directory of your repository. The SVN book has a section explaining how to configure authentication for svnserve.
• Apache httpd can be used to expose your SVN repositories via the WebDAV protocol. This allows for the very commonly seen http:// repository URLs (especially for Open Source projects). Configuration is a little more involved and you would probably have to install Apache from source as well. The SVN book has the details.

Wrapping up

I hope someone found this entry useful – I know I could have used one when I was setting up Subversion and svnsync.

Initial setup

Ever since Subversion 1.4 was released, I’d been eying the new svnsync tool because we had a single repository that was not, erm, really backed up (we had daily server backups and occasional manual repository dumps but that was it). svnsync promised to make repository mirroring simple, and after doing some repository migration and upgrading, I can assure you it really does make things easier than any other (more manual) repository backup solutions I had seen before. This is a walkthrough of how you can upgrade your pre-1.4 SVN repositories to 1.4.x, and setup svnsync to mirror your repositories. It’s going to be very biased to Ubuntu but I’m sure you can translate any Ubuntu specific steps to your favorite distros.

Here’s our initial setup:

• A pre-1.4 (it was version 1.2.3) SVN repository that needed to be upgraded and migrated to another server.
• 2 cleanly installed Ubuntu 6.06 LTS VPSs, one of which is the intended target for the repository migration. The other would mirror the new 1.4.x repository (using svnsync).

An un-installable Subversion 1.4.x?

I wish I could have simply ran sudo apt-get install subversion and have Ubuntu pull down the latest 1.4.x .debs. Unfortunately, the version of Subversion in the Ubuntu apt-get repository is still 1.3.1 (which doesn’t have svnsync). If anyone knows a reliable way to install Subversion 1.4.x via apt-get, let me know! I looked around for a good edge sources.list but came back empty-handed.

I balk at installing stuff from source because I never did figure out how to easily clean out the stuff that gets installed. All thanks to this reluctance, I went digging around and found checkinstall. This thing is awesome – I wonder why I didn’t manage to find it earlier.

What checkinstall basically allows you to do is, instead of running the usual make install after the usual configure and make steps, it creates a Debian package (it also does RPMs and Slackware packages) for you that is easily un-installable with dpkg, and then proceeds to install the files just as it would have for any other deb.

On Ubuntu it’s really easy to install checkinstall, just:

sudo apt-get install checkinstall

Now, you no longer should type 'make install' - always use the 'checkinstall' command instead:

./configure
make
sudo checkinstall # instead of "make install"

checkinstall will ask you a bunch of stuff but you can just go with the defaults for most of them - I did name my packages 'XXX from source', like 'Subversion 1.4.3 from source' so it's easier to check which packages are checkinstall-generated with a simple grep to dpkg -l.

checkinstall generates a .deb (Debian) package before it actually installs your software (Subversion, in this case). It should tell you right at the end of its installation process about where to find this .deb and how to uninstall your newly installed (from source!) package (something like dpkg -r subversion-1.4.3). Don't delete this .deb yet as we will be using it to install Subversion 1.4.x on our mirror SVN server.

Installing an un-installable Subversion 1.4.x on Ubuntu

Now, blessed with our new checkinstall-granted powers, we can install Subversion from source without any qualms. Before we start, be sure to purge any existing Subversion packages you may have installed (do note that if you're using any packages that depend on the official Ubuntu Subversion packages, you may run into library version problems).

dpkg -l | grep svn
dpkg -l | grep subversion

sudo dpkg --purge subversion
sudo dpkg --purge libsvn0

Now, it's time to get the source. Get it from the official Subversion website. Look for the source code download - the file should be something like this: http://subversion.tigris.org/downloads/subversion-1.4.3.tar.gz. Remember to get SVN dependencies (something like this: http://subversion.tigris.org/downloads/subversion-deps-1.4.3.tar.gz) as well as these are needed for access to 'http://' scheme SVN repositories. If you want to use Subversion to connect to a server via a http:// or https:// URL, you will require these dependencies (more specifically, the Neon library).

Use something efficient like wget, curl or Axel (love Axel) to get the sources on the server where you want to install Subversion. Unpack them to the same directory. configure. make. checkinstall.

tar zxf subversion-1.4.3.tar.gz
tar zxf subversion-deps-1.4.3.tar.gz
cd subversion-1.4.3
./configure  # Be sure to read the INSTALL file for any options you may want to set (such as SSL)
make
sudo checkinstall

If you get a warning "configure: WARNING: we have configured without BDB filesystem support" during your configure step, you'll get by just fine. Unless you specifically want your Subversion repositories in Berkeley DB format, we can ignore the warning (Subversion will use FSFS filesystem for your repositories) - see FSFS notes and Choosing a Data Store if you want to make an educated decision.

Anyway, now with a brand new Subversion 1.4.x installed, we are finally ready for the real work - migrating your Subversion repository!

Dumping and importing a repository

Dumping a Subversion repository is dead easy:

svnadmin dump /path/to/repository > repository_name.dump

Depending on how big your repository is, you could end up with a pretty large dump file. gzip it, then scp it over to your new server, then gunzip it. Use svnadmin to load the repository dump.

cd /var/svn  # I like to keep my svn repositories under /var/svn
mkdir repository_name
svnadmin create repository_name
svnadmin load repository_name < /path/to/repository_name.dump

If you have a good pipe between the source and destination servers, you can do this in a one-liner:

svnadmin dump /path/to/repository | ssh -C [IP/domain of destination server] svnadmin load /path/to/new_repository

Of course, all this dumping would require a temporary suspension of any repository write actions otherwise you're just going to have an inconsistent dump - just send out an email to your fellow developers and disable svn access.

Setting up access to your new repository

Now, you have a Subversion repository that is only accessible via the local filesystem (file:// 'protocol'), which isn't very useful. We'll need to setup remote access. Your Subversion repository can be accessed in a variety of ways, including:

• svnserve standalone daemon (svn://)
• svnserve with inetd (svn://)
• svnserve over a SSH tunnel (svn+ssh://)
• over the HTTP protocol (http:// and https://)

The svnserve documentation details how to deal with the first 3, and setting up http:// and https:// access to your protocol is really a subject that deserves its own tutorial. Try the SVN book or Google.

Personally I prefer svn+ssh:// access for internal projects since it allows me to unify authentication for my Subversion repositories with UNIX user accounts. Be wary of an angry cadre Windows developers though, since they need to take quite a good number of steps to setup public key authentication and integrate it with their svn clients on Windows machines. Integration with TortoiseSVN is quite a pain, though my Windows-using colleague at work found these useful: Putty and TortoiseSVN, Using Cygwin, Keychain, SVN+SSH and TortoiseSVN in Windows.

svn:// access

I also expose my repositories via svn:// (as we'll see later, this is useful for allowing access to a svnsync user without messing around with any UNIX user accounts) and use the xinetd daemon (apt-get install xinetd on Ubuntu to install) to launch svnserve process. If you're taking this path, create a file (I name it 'svn') in /etc/xinet.d to tell xinetd about svnserve.

In /etc/xinet.d/svn:

service svn
{
        port                    = 3690
        socket_type             = stream
        protocol                = tcp
        wait                    = no
        user                    = www-data
        server                  = /usr/local/bin/svnserve
        server_args             = -i -r /var/svn
}

Notice that I needed to use the full path to svnserve (do a which svnserve to get the full path, making sure this is the 1.4.x version that you just installed). The server_args parameter also bears some explanation. The -i option tells svnserve to use inetd (xinetd is a variant of inetd, sorta). The -r /var/svn option tells svnserve to only expose repositories below that path. This basically translates your repository at /var/svn/my_cool_project to be accessible via svn://your.hostname/my_cool_project.

svn+ssh:// access

Accessing your repository this way basically logs in to the host server of your repository over SSH, invokes the svnserve process, and accesses your repository in a very file://-like manner. What this means is that your repository path is taken from the root of your filesystem. An example: a repository located in /var/svn/my_cool_project would be available at svn+ssh://your.hostname/var/svn/my_cool_project. For this reason I often symlink /svn to /var/svn (to get repository URLs like svn+ssh://your.hostname/svn/my_cool_project instead).

Relocating working copies

Now, all your working copies are still pointing to the old Subversion server - no need to fret, a simple svn switch fixes things:

svn switch --relocate [from] [to]

Replace '[from]' and '[to]' with the source and destination Subversion repository URLs.

Remember to stop access to your old server so no one is making commits to the wrong place.

Setting up svnsync

I'd intended to write this entire piece in one blog post, but I'm running out of steam at this point. In Part 2, we'll actually setup svnsync for some repository mirroring goodness!

This sign at Raffles Place MRT used to show train arrival times:

It’s been there for about 3 days, was still there when I checked this morning.

• Getting more debug info when connecting with your ssh client: Add a ‘-v’ option to your ssh command (e.g. ssh chuyeow@remotehost -v -v -v). Add more ‘-v’ for more detailed debug (you can do up to ‘-v -v -v’ I think).
• Debugging on the remote host by running sshd in debug mode: Run ‘/usr/sbin/sshd -d -p 2222′ on the remote host and connect to it. ’2222′ here is the port number of the sshd process you started on the remote host.
• tail the authentication log: Run ‘tail -f /var/log/auth.log’ on the remotehost. You can watch the log as you try to connect via SSH with your key.
• Make sure your ssh key agent is running: Do a ‘ps aux|grep ssh-agent’. Make sure your key agent is running. If you’re not using ssh-agent (I like keychain from Gentoo, or SSHKeyChain for Mac OS X), do whatever you have to do to ensure that your keychain is running.
• Make sure your private key is added to the ssh key agent: Do a ‘ssh-add -l’ to check that ssh-agent has your key. Likewise, if you are using something else, check your keychain application has your private key.
• Check the permissions on your home directory, .ssh directory, and the authorized_keys file: If your ssh server is running with ‘StrictModes on’, it will refuse to use your public keys in the ~/.ssh/authorized_keys file. Your home directory should be writable only by you, ~/.ssh should be 700, and authorized_keys should be 600.

Tailing the authentication log was the clincher for me this time – my problem was the group permissions on the home folder were incorrectly set (the error message I got from auth.log was: ‘Authentication refused: bad ownership or modes for directory /home/chuyeow’). Just had to fix it so it was no longer group-writable. Of course, this can also be fixed by turning setting ‘StrictModes off’ in your sshd config (/etc/ssh/sshd_config), but it’s not really recommended. Plus, you may not always have the rights to edit that file anyway.

First point of reference: Print from Windows XP to a shared Mac printer tip on Mac OS X Hints. The tip suggests you select a PostScript driver in Windows after finding it on the network (which requires you to do several things first, but we will come to that later). This worked, but it was sub-optimal because you couldn’t use the printer driver software to do stuff like 2-up printing (i.e. print 2 pages per side per sheet). This post will show you how to share a USB printer from Mac OS X to Windows PCs with full driver capability.

1. On the Mac (the one that the USB printer that you want to share is connected to), go to the Sharing preferences pane and ensure Printer Sharing and Windows Sharing are both turned on.
2. Fire up your browser and go to http://127.0.0.1:631 – this is the web interface to CUPS. When asked to enter a password, login with your Mac OS X user account (it has to be an administrator account).
3. Go to the Printers tab and add a new printer (yes, in addition to any existing printer configuration that already exist for the same printer). Choose a name that’s short and descriptive (no spaces). For the purposes of this guide, let’s call it ‘uberprinter’. Best to keep it under 12 characters since Windows is finicky.
4. When asked to select a device, select USB printer.
5. You’ll be asked for a Device URI. To find out, open up a terminal and type lpinfo -v. You should see your USB printer coming up. Mine came up as ‘direct usb://HP/Deskjet%201280?serial=CN516851RPUN’. Copy and paste this (without the ‘direct’ part – i.e., I’d have copied ‘usb://HP/Deskjet%201280?serial=CN516851RPUN’) into the ‘Device URI’ field.
6. Select a ‘Make’ of ‘Raw‘. Keep going until the printer is added.
7. You’re done configuring from the Mac. But before you go, determine your Mac’s IP address (do a ‘ifconfig’ in a shell or fire up System Profiler and check out the ‘Network’ item) – note it down somewhere. Now it’s time to hook up your Windows machine to use the shared printer.
8. OK now go to your Windows machine and add a new printer (Control Panel -> Printers and Faxes). Select ‘A network printer… blah blah’. Don’t browse for the printer, you will enter its IP address directly in the ‘URL’ field. Enter ‘http://your.macs.ip.address:631/printers/uberprinter’ (replacing ‘your.macs.ip.address’ with your Mac’s IP address and ‘uberprinter’ with the short name you gave your printer). If you can’t remember your printer’s name, just scurry back to the Mac and browse to http://127.0.0.1:631/printers/. You should be able to see the printer you added listed there – its name is linked there.
9. Now, all that’s left is to install the correct Windows printer driver on the Windows machine. If you’re lucky Windows already has your driver, if not do whatever you need to get the proper driver – after all, the purpose of jumping through all these hoops is to get full printer driver functionality off the shared Mac printer.

That’s it. That’ll teach you for not getting a print server or one of those new printers with network functionality.

Problems?

Some readers wrote in with their own difficulties and have kindly allowed me to share their solutions. First off, Patrick McKrell who had a solution for cases where you still are just not able to print from the Windows machine (it involves killing a daemon so it’s pretty sweet).

Thank you for making available your instructions for sharing a USB printer connected to a Mac using OS X with a Windows PC. All of your points worked flawlessly on the Mac. My Mac is a B&W G3 running OS X 10.2.8. The printer is an HP PSC 1510.

…

Back to Windows Add Printer. Is my driver there? No. I browsed the Windows file system to Program Files/HP/. No luck. Couldn’t find or add my driver. Next I unplugged the printer’s USB cable from the Mac and into the PC. Windows detected the new hardware, created whatever files it needed, and automatically created the USB-connected PSC 1500 series printer in Printers and Faxes. Well, that’s pretty darn close. I opened the printer’s properties, changed the port from USB to Internet Port (as per your configuration guidance), sent a print job….and nothing. Hmmm.

Finally, I recalled seeing someone’s web post –thanks to your point of reference. I followed the poster’s instructions, and, yeah, it prints, and in color. Thanks again, Chu, for your post. It was easy to follow, and importantly, it worked with my setup (despite the color issue).

Patrick McKrell

The solution? Change your CUPS configuration to allow raw printing:

Basically, I saw that every job on the windows side had “error”.

After looking at the error logs for cups on the mac box (/var/log/cups/), I noticed this line repeatedly:

print_job: Unsupported format ‘application/octet-stream’!

Did some googling, and found a post with the answer:

>
> You probably need to uncomment the following lines in
> /etc/cups/mime.types and /etc/cups/mime.convs:
>
>
> /etc/cups/mime.types:
> #application/octet-stream
>
>
> /etc/cups/mime.convs:
> #application/octet-stream application/vnd.cups-raw 0
> -
>
>
> That will allow raw printing.

Then, kill -HUP the cups daemon, and you’re good to go.

instead of

sudo chown -R dick ./

You won’t be able to su – or sudo to chown the ownerships back properly because the sudo executable needs to be owned by root. Some people should never have unfettered sudo rights… like me.