Determining if BASH is interactive

It's sometimes useful to know if the current BASH shell is interactive or not so that you can do things in your .bashrc only when it's interactive. When a script is run as an executable, particularly as part of a command pipeline, you might not want to do things that might mess with the environment or influence the data in the pipeline.

You can use the $- special parameter to know if the shell is interactive. Combine that with substitution in the parameter expansion and you can isolate just the 'i' that indicates it's an interactive shell. Assign the result to a environment variable that will either be the 'i' or empty, and that can be used with the [[, [, or test builtin commands that treat an empty string as false to determine whether or not to do something.

INTERACTIVE_SHELL=${-//[^i]}
if [ $INTERACTIVE_SHELL ]; then
 source ~/my_interactive_stuff.sh
fi

Safari 4 open link in new tab

The new Safari 4 likes to open links in new windows, if you want them to open in new tabs do this in terminal:

defaults write com.apple.Safari TargetedClicksCreateTabs -bool true

Enable SSH into your Mac at home

1. Get an account from dyndns.org

2. Install the DynDns updater on the home Mac

3. Configure the DSL modem to bridged mode (not computer does PPPoE)

4. Configure the router to do PPPoE

5. Configure the router to forward TCP/UDP port 22 to the Mac

6. Configure SSH not to accept passwords in /etc/sshd_config

PasswordAuthentication no

ChallengeResponseAuthentication no

UsePAM no

7. Generate public/private keys for the remote machines

8. Add the remote machines' public keys to ~/.ssh/authorized_keys

9. Enable remote login on the sharing pref panel

Install EasyPeasy via netboot from a Mac

The EasyPeasy distribution is a nice choice for a netbook. The normal install methods depend on either a Linux or Windows computer to copy the distribution to a USB drive and use the unetbootin tool to prep the drive to be bootable.

I thought about porting unetbootin to Mac, but decided I didn't want to after looking at its source. I found that the architecture of the app is little bits and pieces of useful functionality farmed out to system utilities like syslinux, all scattered throughout a bunch of Tk UI code. Indeed, the options that I would expect to find enumerated in nice constants are instead examined as the raw strings presented in the UI. Without a good separation of UI and functionality, I decided that it wouldn't be worth the pain to do a port.

Instead, I opted to put together the pieces to install EasyPeasy over the network, with my MacBook Pro as the host.

The tools we need are:

• CPIO to manipulate the initrd so we can boot
• squashfs tools to extract files from the live image
• PXE boot environment for bootstrapping, consisting of a DHCP server and a TFTP server, as well as a binary from syslinux.
  
• NFS server to make the live image available as a root filesystem.

Alright, let's get started...

Download the easypeasy image

The easypeasy site has you get a .img.iso file by default, but to mount it on the Mac you'll need to get the plain .iso file from sourceforge. This can take a while, so might as well get it started now and setup some other stuff before you need to use the image. As of this writing, 1.5 is the latest version.

Setup the TFTP server

A PXE network boot uses TFTP to get the kernel and initrd for early boot. I've written about that before, but you don't really need to do anything since Apple has provided a TFTP server in OS X 10.5. We just need to make its directory writable to avoid using sudo all the time:

sudo chmod 777 /private/tftpboot

Setup the DHCP server

Follow my instructions for setting up the DHCP server.


Get and setup the PXE netboot binary

The PXE binary comes precompiled as part of the syslinux package, so we just need to download and extract it:

curl -O http://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-3.84.tar.gz
tar xzvf syslinux-3.84.tar.gz syslinux-3.84/core/pxelinux.0
sudo cp syslinux-3.84/core/pxelinux.0 /private/tftpboot/

The PXE code looks for a config file based on the eee PC's MAC address, or falls back to a "default" config if that's not found. PXE allows for boot menus with various config options, but we'll just use a simple default config that's just suitable for loading Easy Peasy. (you could just edit /private/tftpboot/pxelinux.cfg/default in-place if you prefer):

mkdir -p -m 777 /private/tftpboot/pxelinux.cfg
cat > /private/tftpboot/pxelinux.cfg/default <<END
prompt 1
default easypeasy
timeout 100
label easypeasy
kernel casper/vmlinuz
append boot=casper ip=dhcp root=/dev/nfs netboot=nfs nfsroot=192.168.1.50:/private/tftpboot initrd=initrd.gz --
END

Copy the files from the ISO

We need to mount the easypasy image file and copy the live CD files to a place we can share them. The live CD files are in the casper directory in the EasyPeasy ISO, so we'll copy that whole directory to /private/tftpboot. You could probably do this in Finder, but since we're already doing a bunch in Terminal, here's a recipe to do it from the command line (replace the path to the ISO with wherever you downloaded it to):

hdiutil mount -mountpoint /Volumes/easypeasy ~/Downloads/easypeasy-1.5.iso
cp -R "/Volumes/easypeasy/casper" /private/tftpboot/
chmod 777 /private/tftpboot/casper

Fixup the initrd for the eee PC 900

The initrd that's in the image is missing the ethernet network driver needed for the eee PC 900, so we'll grab it out of the squashfs root filesystem. To do that we need the squashfs tools to extract the file, and an updated version of CPIO to manipulate the initrd image. Follow those links and build the two tools, then come back here.

Figure out which driver you need
Boot your netbook with whatever Linux it already has, and in a terminal use modprobe to get a list of all possible network drivers, and lsmod to figure out which of them is being used.

modprobe -l -t drivers/net > /tmp/netdrivers
for d in $(lsmod|cut -d' ' -f1); do grep $d /tmp/netdrivers; done

For my eee 900 running easypeasy 1.0, I get:

/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/wan/hdlc_cisco.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/wireless/ath5k/ath5k.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/atlx/atl2.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/hamradio/6pack.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/acenic.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/macvlan.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/irda/act2001-sir.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/irda/actisys-sir.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/hamachi.ko
/lib/modules/2.6.27-8-eeepc/kernel/drivers/net/wireless/b43legacy.ko

It's not obvious which of those is the ethernet driver, but you can rule out the wan, wireless, hamradio, and irda stuff. With some trial and error, it turns out that it's the atl2 driver I need, so that's the one I'm sticking in the initrd for the netboot.

Figure out the path to the driver in the image
The kernel version your netbook is currently running is probably different than what you're installing, so just grep for the driver in the listing of the contents of the live image.

cd /private/tftpboot
unsquashfs -n -l casper/filesystem.squashfs | grep atl2.ko

reveals that it's at: squashfs-root/lib/modules/2.6.30.5-ep0/kernel/drivers/net/atlx/atl2.ko

Copy the ethernet driver into the initrd
Now that you have the tools, and know which driver you need, it's a simple matter to update the initrd:

Get the driver out of the squashfs

cd /private/tftpboot
unsquashfs -n casper/filesystem.squashfs lib/modules/2.6.30.5-ep0/kernel/drivers/net/atlx/atl2.ko

Extract the contents of initrd

mkdir initrd-root
cd initrd-root
gunzip -c ../casper/initrd.gz | sudo cpio -imdv

Copy the extracted driver

sudo cp -R ../squashfs-root/ .

Put the initrd back together

find . | cpio -ov -H newc | gzip > ../initrd.gz

Setup a NFS share

OS X 10.5 has everything you need to export a NFS share, so we just need to set up the export. The init script will look for the live filesystem in the "casper" directory of the network root, so we'll share /private/tftpboot which is its parent directory. If you already have NFS running, just edit the exports file. If this is your first NFS share, make a new exports file (you could just edit the /etc/exports file if you prefer):

echo "/private/tftpboot -mapall=nobody -ro" > exports
sudo mv exports /etc/exports

Take the NFS server down and bring it back up (sure, you could just use the 'update' verb, but this works even if it wasn't already running):

sudo nfsd disable
sudo nfsd enable

After several seconds you can check that /private/tftpboot is exported:

showmount -e

Setup the eee PC BIOS for network boot

• Hit F2 while booting
• Right arrow to "Boot"
• Down arrow to "OnBoard LAN Boot"
• Enter and select "Enabled"
• F10 to save and exit
• Next reboot, hit F2 again
• Right arrow to "Boot"
• Enter on "Boot Device Priority" and set:
  1st: Removable
  2nd: Network
  3rd: HDD
  4th: CD-ROM
• F10 to save and exit

Boot the Easypeasy ISO via netboot
In case you left them stopped above, now is the time to start all the servers:

sudo ifconfig en0 192.168.1.50
sudo launchctl load -F /System/Library/LaunchDaemons/tftp.plist
sudo /usr/sbin/dhcpd
sudo nfsd enable

Plug an ethernet cable between the eee PC and the Mac, and boot the eee PC.

At the boot: prompt, just wait or hit enter for the default.

Be patient when it seems to hang at any of these boot messages:

[12.345678] sd 2:0:0:0: Attached scsi generic sg1 type 0

[16.324864] eth0: no IPv6 routers present

[32.109876] ACPI: Battery Slot [BAT0] (battery present)

Follow the instructions on screen to install EasyPeasy. You might have to manually run the install app (on the desktop) if it doesn't auto-start after booting. The install will take about 45 minutes, 30 minutes of which is copying files over NFS.

When you're done, you can shut down all the servers

sudo launchctl unload -F /System/Library/LaunchDaemons/tftp.plist
sudo killall dhcpd
sudo nfsd disable

Run a TFTP server on a Mac

Apple has already included everything you need to serve files by TFTP from OS X 10.5. Put files in the directory /private/tftpboot, and start the service with:

sudo launchctl load -F /System/Library/LaunchDaemons/tftp.plist

Stop the service with:

sudo launchctl unload -F /System/Library/LaunchDaemons/tftp.plist

Run a DHCP server on OS X

DHCP will provide an IP address to clients when they need it. It's also the foundation for booting a machine over the network, but I won't get into that too much here.

The DCHP server (dhcpd) source works without any modification on OS X. The DHCP client is already part of OS X, so it's probably better to leave the version that Apple provided alone. To install only the server, we'll specify specific subdirectories.

Before you run it the first time, you need to create an empty leases file to keep it from complaining:

Can't open lease database /var/db/dhcpd.leases: No such file or directory --
check for failed database rewrite attempt!

sudo touch /var/db/dhcpd.leases

We'll setup a basic network and give clients the 192.168.1.70-100 range.

One reason you might want to run DHCPD is to host a network boot, which is called a PXE (Pre-eXecution Environment) boot. We'll throw a couple lines in our config to tell such machines to load the Linux network loader from the file pxelinux.0 on the host at 192.168.1.50 (which is the same Mac running the DHCP server - I'll discuss what I did with PXE in another post).

Here's a complete recipe to download, build and configure dhcpd for OS X:

curl -O http://ftp.isc.org/isc/dhcp/dhcp-3.1.3.tar.gz
tar xzvf dhcp-3.1.3.tar.gz
cd dhcp-3.1.3
./configure
make
sudo make install SUBDIRS='common dhcpctl omapip server'
sudo touch /var/db/dhcpd.leases
cat > dhcpd.conf <<END
default-lease-time 86400;
max-lease-time 604800;
ddns-update-style ad-hoc;
subnet 192.168.1.0 netmask 255.255.255.0 {
 range 192.168.1.70 192.168.1.100;
 filename "pxelinux.0";
 next-server 192.168.1.50;
 option subnet-mask 255.255.255.0;
 option broadcast-address 192.168.1.255;
 option routers 192.168.1.1;
}
END
sudo cp dhcpd.conf /etc/

For my uses, I want this dchpd to serve clients hooked up to my MacBook's ethernet port, so I manually set an IP address that's on the appropriate network range (it's okay if it already has an IP address, this will just give it another one):

sudo ifconfig en0 192.168.1.50

When you start the dhcp server, you'll probably get a warning that there's no subnet declaration for en1 since we didn't configure anything for the AirPort network. That's okay, just ignore that wordy warning:

No subnet declaration for en1 (192.168.10.100).
** Ignoring requests on en1. If this is not what
you want, please write a subnet declaration
in your dhcpd.conf file for the network segment
to which interface en1 is attached. **

Start the dhcp server:

sudo /usr/sbin/dhcpd

Stop the dhcp server:

sudo killall dhcpd

Remove old Eclipse workspace selections

Got stale workspaces in Eclipse's Workspace Launcher dialog? Edit the RECENT_WORKSPACES value in:

$HOME/.eclipse/org.eclipse.platform_3.3.0_1473617060/configuration/.settings/org.eclipse.ui.ide.prefs

(or change the platform to whichever version of eclipse you're using)

Modifying bash $PATH

Here's some functions to edit the $PATH without leaving crud:

function pathremove()
{
 local P=:$1:; shift
 while [ "$1" ]; do P=${P/:$1:/:}; shift; done
 P=${P#:}; P=${P%:}
 echo ${P}
}

function pathprefix()
{
 local P=$(pathremove $@); shift
 local N=""
 for n in $@; do N=${N}:$n; done
 echo "${N#:}:${P}"
}

function pathsuffix()
{
 local P=$(pathremove $@); shift
 local N=""
 for n in $@; do N=${N}:$n; done
 echo "${P}${N}"
}

Example usage:

export PATH=$(pathsuffix $PATH $HOME/bin /opt/bin)
export PATH=$(pathremove $PATH /sbin)
export LD_LIBRARY_PATH=$(pathprefix $LD_LIBRARY_PATH /opt/lib)

Automatically start ssh-agent

The following in my .bashrc starts ssh-agent and adds keys automatically whenever I open a shell, if it's not already running:

# Check PS1 to determine if it's a login shell
if [ "$PS1" ]; then
  SSH_ENV=$HOME/.ssh/env-$HOSTNAME
  if [ -f "$SSH_ENV" ]; then
    source $SSH_ENV > /dev/null
  fi

  ps $SSH_AGENT_PID 2> /dev/null 1>&2
  if [ $? != 0 ]; then
    echo "$(date) : Starting SSH agent on $HOSTNAME" | tee -a $HOME/.ssh/agent.log
    ssh-agent > $SSH_ENV
    chmod 600 $SSH_ENV
    source $SSH_ENV
    ssh-add
  fi
fi

Use tkdiff to view changes in subversion

To use tkdiff to view changed files in an svn repository put the following in a script called svndiff somewhere in your path, and make it executable:

#! /bin/bash
# Script to use a visual differ to view svn changes,
# supporting multiple files in one session

SVN=/usr/local/bin/svn
DIFF=tkdiff

args=""
sep=""
while read pair; do
 args="$args $sep $pair"
 sep=":"
done < <($SVN diff "$@" --diff-cmd echo | \
         awk '/^-L/ {gsub(/\/tmp\/tmp/, "/dev/null"); print $(NF-1), $NF; next}')
$DIFF $args 

Use python to decode base64 data

Whether it's data you copy/paste into the shell, or you want to use piped commands, you can decode base64 data from stdin and send it to stdout with:

python -c 'import sys,base64; base64.decode(sys.stdin,sys.stdout)'

Eclipse/PyDev on a Mac

1. Download "eclipse classic" from http://www.eclipse.org/downloads
2. Unpack the tarball, which will be an 'eclipse' folder
3. Move the 'eclipse' folder to /opt (for the long-term, not just in your downloads folder)
4. Make an alias from the Eclipse app into /Applications
5. Run eclipse
6. Go to Help > Software Updates > Available Software and click "Add Site..."
   enter: http://pydev.sourceforge.net/updates/
7. Check the new site, including "PyDev for Eclipse" among its children
8. Click the "Install..." button, and go through all the prompts including restarting eclipse
9. Open Eclipse > Preferences > PyDev > Interpreter - Python
10. Click the "New..." button
11. Navigate in the finder open dialog to select /System/Library/Frameworks/Python.framework/Versions/2.5/bin/python2.5
    (tip: Use Cmd-Shift-G and paste in the directory, then select python2.5)
    
12. Leave the default checked items for PYTHONPATH and click OK (you can edit it later if you need to)

When you go to create your PyDev project, don't get confused by the diabled "Next >" button... just click "Finish"

If you're doing an AppEngine app, you can add this to your PYTHONPATH:
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine

Rollback a SVN change

Have svn make a diff of the previous revision to the revision you want to rollback, and reverse patch it.

Say you want to rollback r12345...

svn diff -r 12344:12345 | patch -R -p0

Modifying initrd with a Mac

When Linux boots, the bootloader starts the kernel with a parameter telling it what file to load into ram as the initial ramdisk. The initial ramdisk is used as the root filesystem until it's configured enough stuff and loaded enough drivers to mount the real root filesystem.

Normally the initrd is put together with the mkinitrd tool. But, initrd.gz is just a gzipped cpio archive, so you can use the gzip and cpio tools to extract and create it as well. To make sure permissions aren't messed up, run cpio as root (There are probably tools that will twiddle the bits in the cpio archive, but if you have root access then sudo is an easy answer).

mkdir initrd-root
cd initrd-root
gunzip -c ../initrd.gz | sudo cpio -imdv

Examine or modify the extracted contents as you like. Then you can put your modified initrd back together:

cd initrd-root
find . | cpio -ov -H newc | gzip > ../initrd.gz

But, wait... you said this could be done on a Mac...

If you're using Mac OS X, there's a slight glitch - the version of cpio that Apple ships doesn't support creating the necessary newc format archive.

To deal with that, we'll build our own cpio 2.9 from source, which almost works right out of the box. One small glitch is that current versions of gcc shipped with Xcode have a bug that results in:

ld: duplicate symbol _argp_fmtstream_putc in ../lib/libcpio.a(argp-fmtstream.o) and ../lib/libcpio.a(argp-help.o)

The problem is related to Apple's gcc improperly handling extern inline in C99 mode. Since we're just building a binary and therefore don't particularly care about C99, we'll just disable it with a configure flag.

Here's the download-and-build recipe:

curl -O http://ftp.gnu.org/gnu/cpio/cpio-2.10.tar.gz
tar xzvf cpio-2.10.tar.gz
cd cpio-2.10
./configure ac_cv_prog_cc_stdc=no
make
sudo cp src/cpio /usr/local/bin

Build squashfs tools for Mac OS X

To squash/unsquash a Linux squashfs filesystem on a Mac, you can build the squashfs tools.

A few modifications of the source are required for OS X:

• The FNM_EXTMATCH flag doesn't exist in the fnmatch library on OS X. This just means that you can't use glob patterns with mksquashfs.
• The constants to figure out how many CPUs the machine has are in the sysctl header instead of the sysinfo header.
• The C99 compiler has different semantics for the inline keyword, so inline functions should also be static.
• Instead of llistxattr(), lgetxattr(), lsetxattr(), Apple uses a XATTR_NOFOLLOW flag on the non-link equivalent functions.

Here's a complete download-and-build recipe:

curl -OL http://iweb.dl.sourceforge.net/project/squashfs/squashfs/squashfs4.2/squashfs4.2.tar.gz
tar xzvf squashfs4.2.tar.gz
cd squashfs4.2/squashfs-tools

sed -i.orig 's/FNM_EXTMATCH/0/; s/sysinfo.h/sysctl.h/; s/^inline/static inline/' mksquashfs.c unsquashfs.c

cat <<END >> xattr.h
#define llistxattr(path, list, size) \
  (listxattr(path, list, size, XATTR_NOFOLLOW))
#define lgetxattr(path, name, value, size) \
  (getxattr(path, name, value, size, 0, XATTR_NOFOLLOW))
#define lsetxattr(path, name, value, size, flags) \
  (setxattr(path, name, value, size, 0, flags | XATTR_NOFOLLOW))
END

make
sudo cp mksquashfs unsquashfs /usr/local/bin

Usage notes:

The filesystem you're unsquashing probably has root-owned files in it, so you'll have to use sudo to run unsquashfs as root.

The limit of open files is somewhat low on a Mac, so you'll get a bunch of errors if you try to unsquash a whole filesystem:

write_file: failed to create file squashfs-root/path/to/file, because Too many open files

Just change the limit in the shell before running unsquashfs:

ulimit -n 2000
sudo unsquashfs filesystem.squashfs

There seems to be a bug when drawing the progress bar that causes Floating point exception, which you can workaround by using the -n flag.