Biboroku

Setting up MySQL Database from .sql Dump on DotCloud

Taro — Wed, 10 Oct 2012 23:20:17 +0000

Here I assume that MySQL service is running as db for my application named myapp. If I don’t remember, the password can be obtained as follows:

$ dotcloud info myapp.db
... a bunch of info, one of which is passwd ...

If I have a MySQL dump file named somedb.sql, do

$ dotcloud run myapp.db -- mysql -u root -pMYMYSQLROOTPASSWOD < somedb.sql

This is it. It is probably a good idea to create a regular MySQL user.

Customizing & Installing Linux Kernel on Debian Wheezy

Taro — Sun, 30 Sep 2012 09:01:56 +0000

Here is a quickie for customizing and install Linux kernel 3.5.x on Wheezy.

Add yourself to some groups:

# adduser username sudo

You need to logout and login for this change to take effect. You also need to be able to use sudo or su to install the new kernel in the end.

Install some packages and kernel source:

$ sudo aptitude update
$ sudo aptitude install kernel-package pkg-config bzip2 g++ libqt4-dev libncurses5-dev fakeroot

The kernel packaged for Wheezy is at version 3.2. A newer kernel can be obtained from kernel.org and saved somewhere. You just need to expanding the compressed source somewhere (even under your home directory, which is the case in this post). Either way, the customization procedure is similar. Here I just follow the Debian way:

$ sudo aptitude install linux-source-3.2

Extract the source tree:

$ cd ~/src
$ tar -jxf /usr/src/linux-source-3.2.tar.bz2
$ ln -s linux-source-3.2 linux
$ cd linux

Edit the EXTRAVERSION entry in Makefile, as in:

EXTRAVERSION = .20120929.1

for example to add .20120929.1 to the kernel version number. This is convenient for keeping the existing, working kernels around when you need to recompile with different options.

Use xconfig or menuconfig to customize the kernel options. Before the make-kpkg lines, setting concurrency (most likely to the number of cores of my processor) is optinal but having a higher number typically reduces the compilation time.

$ make mrproper
$ make xconfig    # or make menuconfig
$ export CONCURRENCY_LEVEL=2    # this is optional
$ fakeroot make-kpkg clean
$ fakeroot make-kpkg --initrd kernel_image
$ sudo dpkg -i ../linux-image-3.2.20120929.*_amd64.deb

Upon reboot in the GRUB menu you will find the newly installed kernel:

$ sudo reboot

Purging Old Kernel Image from System

For example, if the kernel to be uninstalled is of version 2.6.26 and the extra version that I used was 20091112.1, do:

$ sudo dpkg -P linux-image-2.6.26.20091112.1

That’s it. However it is often a good idea to keep at least one kernel image that I know for sure to work so that when a custom kernel fails, I have something to fall back on. On the other hand, it is also a good idea to purge very old kernel images to save space in /boot.

Power Management on Lenovo T430s with Debian Wheezy

Taro — Sun, 30 Sep 2012 08:58:06 +0000

Since tp_smapi does not work properly on T430s, I am using TLP this time. I still want some traditional tools

$ sudo aptitude install powertop

but not others:

$ sudo aptitude remove --purge laptop-mode-tools

To install, follow the instruction. In short, first update the APT source list by adding the following line to /etc/apt/sources.list:

deb http://ppa.launchpad.net/linrunner/tlp/ubuntu lucid main

and run

$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 02D65EFF
$ sudo aptitude update
$ sudo aptitude install tlp tlp-rdw tp-smapi-dkms smartmontools ethtool acpi-call-tools

Upon reboot, TLP starts running.

I am impressed by the ease of installation and how effective TLP is on configuring T430s to star saving from the get go. According to powertop, T430s is already using just 7 - 8 W on battery. That's several watts below I had been using on other ThankPads!!

Controlling the behavior of TPL is done by modifying /etc/default/tlp. I mostly use the default except the following changes:

...

# Main battery (values in %)
START_CHARGE_THRESH_BAT0=75
STOP_CHARGE_THRESH_BAT0=90
# Ultrabay battery (values in %)
START_CHARGE_THRESH_BAT1=75
STOP_CHARGE_THRESH_BAT1=90

...

# Radio devices to disable on connect
DEVICES_TO_DISABLE_ON_LAN_CONNECT="wifi wwan"
DEVICES_TO_DISABLE_ON_WIFI_CONNECT="wwan"
DEVICES_TO_DISABLE_ON_WWAN_CONNECT="wifi"

# Radio devices to enable on disconnect
DEVICES_TO_ENABLE_ON_LAN_DISCONNECT="wifi"
DEVICES_TO_ENABLE_ON_WIFI_DISCONNECT=""
DEVICES_TO_ENABLE_ON_WWAN_DISCONNECT=""

Note that in this configuration the battery charge thresholds are intentionally set lower than the factory default to prolong the health of batteries. When I wish to fully charge them, do:

$ sudo tlp fullcharge [ BAT0 | BAT1 ]

and the thresholds will be set back to the factory default. The setting reverts to the custom configuration upon next reboot.

Enables RC6 Power Saving

Edit /etc/default/grub and add the option i915.i915_enable_rc6=1 to GRUB:

GRUB_CMDLINE_LINUX_DEFAULT="quiet i915.i915_enable_rc6=1"

After editing, update Grub:

$ sudo update-grub

This can save about 0.5 W of power!

Fan Control

I have not fine tuned this, but the fan can be controlled manually via thinkfan:

$ sudo aptitude install thinkfan

Create (or edit) /etc/modprobe.d/thinkpad_acpi.conf and add the line:

options thinkpad_acpi fan_control=1

Reboot. The fan can be controlled manually following this Wiki article.

Configuring System on Switch between AC & Battery

For regular daemons (like ssh), TLP provides a way to start/stop based on AC status.

For other personal processes, KDE power management can also do some work by running a script upon dis/connecting with AC. In particular, the sound for the KDE system notifications should be disabled entirely, because they cause quite a number of unwanted interruptions according to powertop. Also, I wish to start/stop programs like flux and Dropbox when on battery just to squeeze more juice. I create a script like this:

#!/usr/bin/env bash
##############################################################################
# This is a script to run when AC/battery switches.  Ideally used
# within the power management configuration setting on KDE.  Requires
# tp_smapi to be loaded for the detection of AC/battery.
  
USERNAME=johndoe
LATITUDE=37.78
LONGITUDE=-122.42
  
  
##############################################################################
# DO NOT MODIFY BELOW
 
statuspath="/sys/class/power_supply/AC/online"
  
if [ ! -f "$statuspath" ]; then
    exit 0
fi
  
tag=pmpersonal
status=`cat "$statuspath"`
 
if [ "$status" = "1" ]; then
    logger -t $tag "on ac"
  
    logger -t $tag "start xflux"
    for i in `pgrep ^xflux$` ; do kill -9 $i ; done
    xflux -l $LATITUDE -g $LONGITUDE
  
    logger -t $tag "start dropbox"
    dropbox start
  
    logger -t $tag "enable system notification"
    kwriteconfig --file="/home/$USERNAME/.kde/share/config/knotifyrc" --group=Sounds --key="No sound" false
else
    logger -t $tag "on battery"
  
    logger -t $tag "stop xflux"
    for i in `pgrep ^xflux$` ; do kill -9 $i ; done
  
    logger -t $tag "stop dropbox"
    dropbox stop
  
    logger -t $tag "disable system notification"
    kwriteconfig --file="/home/$USERNAME/.kde/share/config/knotifyrc" --group=Sounds --key="No sound" true
fi

Save this to ~/bin/pmpersonal.sh. Go to System Settings -> Power Management -> Energy Saving Settings. There, for each of “On AC,” “On Battery,” and “On Low Battery,” I can set different configurations. I can also specify “Run Script” option for each profile to enable/disable a few desktop related applications and settings. The script should be run on profile load.

Preload Daemon

$ sudo aptitude install preload

In /etc/default/preload, uncomment the line reading

OPTIONS="-l /dev/null"

This should reduce disk activity.

Disabling Nepomuk, Strigi and Akonadi on KDE

I am not fond of these daemon. Disable them by going to System Settings -> Desktop Search and unchecking “Enable Nepomuk Semantic Desktop.” In ~/.config/akonadi/akonadiserverrc, set:

StartServer=false

so that it doesn’t start at all. On powertop, this shows up as a service running mysqld constantly.

LaTeX on Debian Wheezy

Taro — Sat, 29 Sep 2012 22:11:13 +0000

Install basic packages for LaTeX:

$ sudo aptitude install texlive texlive-latex-extra

Use pdflatex to compile TeX documents.

dvipng may be needed to render LaTeX on Matplotlib:

$ sudo aptitude install dvipng

Installing NixNote on Debian Wheezy

Taro — Sat, 29 Sep 2012 21:20:47 +0000

After downloading the .deb for 64-bit Wheezy, do:

$ sudo aptitude install default-jre
$ sudo dpkg -i nixnote-1.2_amd64.deb

Installing Debian Wheezy on Lenovo T420

Taro — Thu, 27 Sep 2012 22:52:46 +0000

Some loser stole my T410s, so I was needing a quick replacement. With T430s on its way (ordered a month ago since I needed a backup anyways) but with the shipment being delayed for so long with hardware issues, I could wait no longer and decided to get this one on budget. This T420 cost me $650 (actually wanted ThinkPad Edge for a couple hundred bucks less, but it wasn’t available at the store).

By the way, if you see a T410s with serial number R84K78G, it’s mine, stolen at a San Francisco Starbucks on September 9, 2012. Please report to police and have the douche bag who stole the thing well-deserved punishment…

General Hardware Specifications

My Lenovo T420 is 4177X07. This is so-called T420i, a budget version of T420 with Intel Core i3. Not Linux certified but seems to work fine.

Hardware Components	Status under Linux	Notes
Intel Core i3 2310M 2.1 GHz	Works
14.0 HD+ (1600 x 900)	Works
Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)	Works
NVIDIA Corporation GF119 [Quadro NVS 4200M]	Not tested
4 GB Memory	Works
320GB 7200rpm Hitachi HTS723232A7A364	Works
Intel Corporation 82579LM Gigabit Network Connection (rev 04)	Not tested
Intel Corporation Centrino Advanced-N 6205 (rev 34)	Works
HL-DT-STDVDRAM GT33N	Works
Sanyo 42T4791 Battery	Works
Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio	Works
Integrated Speaker & Microphone	Not tested
ThinkLight	Works
TrackPoint & TrackPad	Works
Keyboard	Not fully tested	Haven’t thoroughly tested special keys

The stock kernel (3.2.0-3-amd64) seems to work fine with this configuration.

Installing Debian Wheezy

Recovery Media

Make sure to create factory recovery disks in case the factory default needs to be restored. In order to do this, on Windows 7, launch ThinkVantage and go to Factory Recovery Disks. Check both boot and data. Four blank DVDs were necessary for me (one for boot and three for data). After making the disks, it is recommended to test if they work or not by rebooting off the boot disk just created.

Shrink the Windows Partition

For T420, I decided to dual boot Windows 7 and Linux in case I ever need to run something on Windows (e.g., games). This is actually PITA to do due to “unmovable system files.” I found this article useful for what needs to be disabled before shrinking a partition on Windows (for me no external utilities were necessary to shrink it to the minimum).

Installing Wheezy

I decided to do netinst with wireless firmware already included into the boot media, so that I don’t have to bother with missing firemware during the install process while on WiFi. The latest installer at the writing is still at beta 2, but an ISO image is available here. Burn this onto a blank CD/DVD. The availability of netinst media with firmware made the whole process such a breeze! Simply boot off the DVD and follow the on-screen instruction.

Installation Customization Options

Debian installation has gotten easy enough that I don’t feel the need for this any more, but just for completeness, here goes. Obviously this is just one way to customize. I used the text installer.

Language: English
Country, territory or area: United States
Keymap: American English
Primary network interface should be configured automatically, even if the installer complains about missing firmware.
Hostname: (choice)
Domain name: (choice)
Root password: (choice)
Full name for the new user: (choice)
Username for your account: (choice)
Choose a password for the new user: (choice)
Select your time zone: Pacific
Partitioning method: Manual

I’m doing dual boot, but my Linux partition scheme is very simple:

$ df -Tlh
Filesystem                    Type      Size  Used Avail Use% Mounted on
rootfs                        rootfs    491M  172M  295M  37% /
/dev/disk/by-uuid/cc0...      ext4      491M  172M  295M  37% /
/dev/sda6              　    ext4      101M   24M   72M  25% /boot
/dev/sda7               　   ext4      4.9G  201M  4.5G   5% /tmp
/dev/sda8                　  ext4      3.9G  1.2G  2.6G  32% /var
/dev/sda9                 　 ext4      9.8G  2.8G  6.6G  30% /usr
/dev/sda10                 　ext4      9.8G  318M  9.0G   4% /usr/local
/dev/sda12                 　ext4      168G  5.2G  154G   4% /home

Debian archive mirror country: United States
Participate in the package usage survey: Yes
Choose software to install: (I want a fairly minimal install. I only check ssh, laptop, and standard system utilities.)
Install the GRUB boot loader to the master boot record: Yes

Reboot and a very simple Debian box is ready. Upon the first reboot, I login as root and install a few essential packages for system administration, for example:

# aptitude install sudo ssh rsync wget wireless-tools emacs

and in case I need to run 32-bit applications:

# aptitude install ia32-libs ia32-libs-gtk

I also install basic KDE:

# aptitude install kde-plasma-desktop kmix network-manager-kde

Setting up sudo

At this point I add myself as an sudoer by adding username to /etc/sudoers, just under the entry for root, giving full admin privileges (i.e., just copy the line for root).

# chmod 640 /etc/sudoers
# emacs -nw /etc/sudoers
 ... add a regular user to the list ... 
# chmod 440 /etc/sudoers

Manage Network with Network Manager

After configuring the basic system, I had trouble using the KDE network manager. The installing is simple as usual:

$ sudo aptitude install network-manager-kde

If I don’t see it on the system tray, right-click on the tray, System Tray Settings -> Display -> Extra Items, and check Network Management.

I found that the network manager neither found any WiFi access points nor recognized the ones that I added manually. This seems to happen when I use WiFi to do netinst, in which case my /etc/network/interfaces includes the following lines:

allow-hotplug wlan0
iface wlan0 inet dhcp
    wpa-ssid MYACCESSPOINT
    wpa-psk  MYPASSWORD

Uncomment all these lines and perhaps add a new line instead:

allow-hotplug eth0

This will make the network manager work as usual.

TrackPoint and TouchPad

I can use the red TrackPoint for scrolling. I usually disable TouchPad, since it interferes often with my palms.

Suspend, Resume, and Power Management

I have always been a type who turns off computer when not used, but I am now using suspend/resume frequently. So this hasn’t been tested yet. I have a set of power management tips on separate post, though.

One problem I have been having with T420 is that, when on battery, shutdown operation leads to reboot. This seems to be caused by a kernel bug, but fortunately there is a solution (thanks to this post, which I pretty much duplicate here). Create a file /etc/init.d/shutdownfixes.sh:

#!/bin/sh -e
### BEGIN INIT INFO
# Provides:          shutdownfixes.sh 
# Required-Start:    
# Required-Stop:     
# X-Stop-After:      $network
# Default-Start:     
# Default-Stop:      0
# Short-Description: Fixes shutdown-to-reboot glitch.
# Description:       Fixes shutdown-to-reboot glitch.
### END INIT INFO

PATH="/sbin:/bin"

echo -n "Preparing PCI busses for shutdown..."
for i in /sys/bus/pci/devices/*/power/control; do
  echo on > $i
done
echo "done."

echo -n "Removing e1000e module..."
lsmod | grep -q e1000e && rmmod e1000e
echo "done."

Give this an executable permission:

$ sudo chmod ugo+x /etc/init.d/shutdownfixes.sh

and install it

$ cd /etc/init.d
$ sudo update-rc.d shutdownfixes.sh 8 0 0 .

On the next attempt, shutdown should be successful and not result in reboot.

Miscellaneous

Installing Applications

Here is a list of applications to install on my machine. This is to be done…

Power Management on Lenovo T420 with Debian Wheezy

Taro — Sun, 23 Sep 2012 06:28:31 +0000

NOTE: This is an older method. I have moved on to using TLP and the amount of powersaving is quite amazing (yes, 6 – 7 W is possible!!). I will eventually try to update this post, but for now the solution that I wrote for T430s should be followed.

Here is my continuously growing notes. I use Debian Wheezy (amd64) on T420.

First, install must-have packages:

$ sudo aptitude install powertop acpi acpid tp-smapi-dkms

To load tp-smapi module on boot, add the line

tp-smapi

to /etc/modules.

To monitor the power consumption, use powertop:

$ sudo powertop

This is one of the most useful tool for power management. I can see which processes wake up the computer most, and try to eliminate them one by one.

Laptop Mode Configurations

Install laptop-mode-tools:

$ sudo aptitude install laptop-mode-tools

In /etc/laptop-mode/laptop-mode.conf, make sure to have this line:

ENABLE_AUTO_MODULES=1

(which was set to 0 by default on Squeeze but does not appears to be the case for Wheezy). When on battery this will apply a bunch of power saving settings. This sole step should typically save about 2 – 2.5 W of power consumption according to powertop! Note that wattage information only shows up while running on battery.

The following additional tweaks (overriding the default settings) are optional.

To disable ethernet when on battery, make sure /etc/laptop-mode/conf.d/ethernet.conf has the following lines:

CONTROL_ETHERNET="auto"
DISABLE_ETHERNET_ON_BATTERY=1

For more aggressive power saving on Intel HDA, make sure /etc/laptop-mode/conf.d/intel-hda-powersave.conf has the following lines:

CONTROL_INTEL_HDA_POWER="auto"
INTEL_HDA_DEVICE_TIMEOUT=2
INTEL_HDA_DEVICE_CONTROLLER=1

For more aggressive power saving on the wireless adapter, make sure /etc/laptop-mode/conf.d/wireless-iwl-power.conf has the following lines:

CONTROL_IWL_POWER="auto"
IWL_BATT_POWER=3

After changing the configuration, restart laptop-mode:

$ sudo /etc/init.d/laptop-mode restart

to make the changes effective.

Disabling Nepomuk, Strigi and Akonadi on KDE

I am not fond of these daemon. Disable them by going to System Settings -> Desktop Search and unchecking “Enable Nepomuk Semantic Desktop.” In ~/.config/akonadi/akonadiserverrc, set:

StartServer=false

so that it doesn’t start at all. On powertop, this shows up as a service running mysqld constantly.

Starting & Stopping Services

Some daemons running in the background can be stopped while on battery by laptop-mode. If the init scripts are stored under /etc/init.d, the simplest method is to place a symlink to the init script under /etc/laptop-mode/*-stop directories.

For example, if I want to stop ssh from running while on battery, do this:

$ cd /etc/laptop-mode/batt-stop
$ sudo ln -s /etc/init.d/ssh .

Now, I want ssh to get restarted when on AC:

$ cd /etc/[lm|nolm]-ac-start
$ sudo ln -s /etc/init.d/ssh .

(depending on how laptop-mode is configured, I create a symlink either in lm-ac-start or nolm-ac-start.)

I should create similar symlinks for all services that should be stopped while on battery.

Configuring System on Switch between AC & Battery

For certain things that laptop-mode does not take care of, I create a script which gets run each time the AC is plugged or unplugged. Depending on the state (on battery or on AC), I change some settings.

Such a script may look like this:

#!/bin/bash

if [ ! -f /sys/devices/platform/smapi/ac_connected ]; then
  exit 0
fi

TAG=pmtp
STATUS=`cat /sys/devices/platform/smapi/ac_connected`

if [ "$STATUS" = "1" ]; then
  logger -t $TAG "on ac"
  logger -t $TAG "increasing swappiness"
  sysctl -w vm.swappiness=60
else
  logger -t $TAG "on battery"
  logger -t $TAG "reducing swappiness"
  sysctl -w vm.swappiness=10
fi

Let us save this script as /usr/local/bin/pmtp. (FYI, this script changes the amount of swap disk usage, effectively reducing disk access while on battery. This is a recommended thing to do.)

To let laptop-mode handle this script, modify /etc/laptop-mode/conf.d/exec-commands.conf to have these lines:

BATT_EXEC_COMMAND_0="/usr/local/bin/pmtp"
LM_AC_EXEC_COMMAND_0="/usr/local/bin/pmtp"
NOLM_AC_EXEC_COMMAND_0="/usr/local/bin/pmtp"

This way the script gets run each time the AC/battery state changes.

For personal processes, KDE power management can also do some work by running a script upon dis/connecting with AC. In particular, the sound for the KDE system notifications should be disabled entirely, because they cause quite a number of unwanted interruptions according to powertop. Also, I wish to start/stop programs like flux and Dropbox when on battery just to squeeze more juice. I create a script like this:

#!/bin/bash
##############################################################################
# This is a script to run when AC/battery switches.  Ideally used
# within the power management configuration setting on KDE.  Requires
# tp_smapi to be loaded for the detection of AC/battery.

USERNAME=johndoe
LATITUDE=37.78
LONGITUDE=-122.42


##############################################################################
# DO NOT MODIFY BELOW 

if [ ! -f /sys/devices/platform/smapi/ac_connected ]; then
    exit 0
fi

TAG=pmpersonal
STATUS=`cat /sys/devices/platform/smapi/ac_connected`

if [ "$STATUS" = "1" ]; then
    logger -t $TAG "on ac"

    logger -t $TAG "start xflux"
    for i in `pgrep ^xflux$` ; do kill -9 $i ; done
    xflux -l $LATITUDE -g $LONGITUDE

    logger -t $TAG "start dropbox"
    dropbox stop
    dropbox start

    logger -t $TAG "enable system notification"
    kwriteconfig --file="/home/$USERNAME/.kde/share/config/knotifyrc" --group=Sounds --key="No sound" false
else
    logger -t $TAG "on battery"

    logger -t $TAG "stop xflux"
    for i in `pgrep ^xflux$` ; do kill -9 $i ; done

    logger -t $TAG "stop dropbox"
    dropbox stop

    logger -t $TAG "disable system notification"
    kwriteconfig --file="/home/$USERNAME/.kde/share/config/knotifyrc" --group=Sounds --key="No sound" true
fi

Disabling Bluetooth

To control the on/off status of wireless device on the command line, I need:

$ sudo aptitude install rfkill
... below is an example usage ...
$ rfkill list

I can live without Bluetooth entirely (in fact, my T420 does not have Bluetooth). To disable Bluetooth at boot time, add the following line:

rfkill block bluetooth

to /etc/rc.local before the “exit 0″ statement.

Preload Daemon

$ sudo aptitude install preload

In /etc/default/preload, uncomment the line reading

OPTIONS="-l /dev/null"

This should reduce disk activity.

Changing Swappiness

On the command line,

$ sudo sysctl -w vm.swappiness=10

The system will try to use physical RAM as much as possible instead of swapping. To make this in effect upon boot, add the line

vm.swappiness=10

to /etc/sysctl.conf. However, I opt to change this particular setting dynamically using a custom script described above.

Battery Charge Thresholds

To extend battery life, charge thresholds should be tweaked. My current settings are:

$ echo 90 | sudo tee /sys/devices/platform/smapi/BAT0/start_charge_thresh
$ echo 95 | sudo tee /sys/devices/platform/smapi/BAT0/stop_charge_thresh

Do this for each battery in my system (BAT0, BAT1, etc.)

Chromium over Firefox

My default browser is Firefox. When browsing the web, however, Chromium is much easier on power. I can save about 1 W using Chromium.

Powertop Status

Currently, I typically use 11 – 13 W during typical usage.

Reference

ThinkWiki – A great resource for ThinkPad Linux users.
PC Plus article – See especially the comment section.

Installing Firefox on Debian Wheezy

Taro — Sat, 22 Sep 2012 08:05:42 +0000

I don’t like to get stuck with a very old version of browser, so I install a Firefox binary provided by Mozilla, not the one by Debian.

Make sure to download a 64-bit version (as I use amd64) from here. The tarball expands to a directory named firefox. For example:

$ cd /usr/local/src
$ sudo wget http://releases.mozilla.org/pub/mozilla.org/firefox/releases/latest/linux-x86_64/en-US/firefox-15.0.1.tar.bz2
$ sudo tar -xvjf firefox-15.0.1.tar.bz2
$ sudo mv firefox ..
$ cd /usr/local/bin
$ sudo ln -s ../firefox/firefox .

This makes firefox command available system-wide.

To install flash, do:

$ sudo aptitude install flashplugin-nonfree

To install Java plugin, do:

$ sudo aptitude install icedtea-7-plugin

Installing Skype on Debian Wheezy

Taro — Sat, 22 Sep 2012 06:35:39 +0000

Go to Skype web site and download an appropriate version (Debian 6.0 64-bit for me) of the deb file. Then

$ sudo aptitude install ia32-libs ia32-libs-gtk
$ sudo dpkg -i skype-debian_4.0.0.8-1_amd64.deb

simply works.

Using Japanese on Debian Wheezy

Taro — Sat, 22 Sep 2012 04:23:39 +0000

The goal is to make the system capable for Japanese input, while letting the base system remain English. For the Japanese input method, I had been using Anthy, but I will be using mozc, which is now better supported and presumably much better (it is).

First, set up locales and install im-config. In addition to my base English locale (en_US.UTF-8), install Japnese locales ja_JP.EUC-JP and ja_JP.UTF-8:

$ sudo aptitude install locales im-config
$ sudo dpkg-reconfigure locales

Set the default locale for the system to en_US.UTF-8.

Install some Japanese fonts:

$ sudo aptitude install ttf-kochi-mincho ttf-kochi-gothic ttf-sazanami-mincho

IBus

Install IBus-related packages and configure for KDE:

$ sudo aptitude install ibus ibus-mozc ibus-qt4 ibus-gtk ibus-gtk3 ibus-clutter
$ im-config
... choose IBus ...

Run the IBus setup program:

$ ibus-setup

If asked, start ibus-daemon. Logout the KDE sesssion, restart X session, and login again. Under the Input Method tab, add “Japanese – Mozc”. Configure IBus to my liking, such as making Ctrl + Space (as opposed to Shift + Space) trigger the Japanese input mode. The IBus icon also shows up in the system tray, from which various settings can be changed.

Mozc

Run

$ sudo aptitude install ibus-mozc mozc-utils-gui
$ /usr/lib/mozc/mozc_tool --mode=config_dialog

to configure Mozc to my liking.

Using Japanese in Emacs

Install ibus-el:

$ sudo aptitude install ibus-el

Within emacs, M-x ibus-mode will toggle IBus on and off. Very easy!

NX Client on Debian Wheezy

Taro — Fri, 21 Sep 2012 19:33:57 +0000

My preferred NX client is NoMachine. Simply download the AMD64 version of deb package and do:

$ sudo dpkg -i nxclient_3.5.0-7_amd64.deb

On KDE, the short cuts will be under Applications -> Internet. The installation is under /usr/NX.

Enabling ThinkPad TrackPoint Scrolling on Debian

Taro — Fri, 21 Sep 2012 18:21:42 +0000

One of the major reason for using ThinkPad is that red TrackPoint thing, which allows the control of cursor without leaving the keyboard. However, when I install Linux on a new ThinkPad, scrolling often does not work out of the box. There are several ways to do this, but my preferred way is to use xinput:

$ sudo aptitude install xinput

See what event ID is associated with TrackPoint:

$ xinput list
⎡ Virtual core pointer                          id=2    [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
⎜   ↳ SynPS/2 Synaptics TouchPad                id=11   [slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                     id=13   [slave  pointer  (2)]
⎣ Virtual core keyboard                         id=3    [master keyboard (2)]
    ↳ Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
    ↳ Power Button                              id=6    [slave  keyboard (3)]
    ↳ Video Bus                                 id=7    [slave  keyboard (3)]
    ↳ Sleep Button                              id=8    [slave  keyboard (3)]
    ↳ Integrated Camera                         id=9    [slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard              id=10   [slave  keyboard (3)]
    ↳ ThinkPad Extra Buttons                    id=12   [slave  keyboard (3)]
    ↳ ACPI Virtual Keyboard Device              id=14   [slave  keyboard (3)]

The TrackPoint’s ID is 13 in my current configuration.

$ xinput --set-prop --type=int --format=8 13 "Evdev Wheel Emulation" 1
$ xinput --set-prop --type=int --format=8 13 "Evdev Wheel Emulation Button" 2
$ xinput --set-prop --type=int --format=8 13 "Evdev Wheel Emulation Axes" 6 7 4 5

The change should be noticeable immediately; moving TrackPoint while pressing down the blue middle button right below the space bar acts as scroll, both vertically and horizontally.

I can also do this automatically on start-up by having the same lines in ~/.xsessionrc (create this file if it does not already exist):

#!/bin/bash

# extract device id
s=`xinput | grep TrackPoint`
s="${s#*id=}"
id=${s:0:2}

xinput --set-prop --type=int --format=8 "$id" "Evdev Wheel Emulation" 1
xinput --set-prop --type=int --format=8 "$id" "Evdev Wheel Emulation Button" 2
xinput --set-prop --type=int --format=8 "$id" "Evdev Wheel Emulation Axes" 6 7 4 5

Note that the device ID may change depending on the environment (e.g., attaching USB mouse on boot, disabling graphics card, etc.), so this kind of automation is useful and prevents WTF moments when you notice TrackPoint scrolling suddenly stops working.

Now that TrackPoint can be used for scrolling, I see little reason to enable TrackPad (I think I have a mild case of hyperhydrosis, so I have developed aversion to thermally sensitive touch/track pad). TrackPad can be disabled in BIOS to save power consumption as well as to prevent accidentally touching it with my palms while typing.

Installing Dropbox on Debian Wheezy

Taro — Fri, 21 Sep 2012 17:53:42 +0000

Go to the Dropbox download website and download the .deb package, here dropbox_1.4.0_amd64.deb. Then

$ sudo aptitude install python-gtk2 python-gpgme
$ sudo dpkg -i dropbox_1.4.0_amd64.deb

The program will be listed under the menu Applications -> Internet. On the first launch it will download the daemon and I will need to go through the guided setup process which should be self-explanatory.

Browsing command history on Bash

Taro — Thu, 06 Sep 2012 00:45:44 +0000

This is useful for finding a command that was executed a while ago but still in the history:

$ history | grep search_term
...
$ !history_number

where search_term is the term to search for within my history, and history_number is the number which identifies that command in the history output.

Some control+key commands on Linux shell

Taro — Tue, 04 Sep 2012 21:08:42 +0000

There are a couple I keep forgetting to remember:

ctrl-s
    freezes the screen and stops any display on the screen from continuing 
    (equivalent to a no-scroll key) (sometimes takes a moment to work) 
ctrl-q
    un-freezes the screen and lets screen display continue 
ctrl-c
    interrupts a running program 
ctrl-\
    same as ctrl-c but stronger (used when terminal doesn't respond) 
ctrl-z
    suspends a running program (use the fg command to continue the program, see section Controlling Jobs) 
ctrl-h
    deletes last character typed 
ctrl-w
    deletes last word typed 
ctrl-u
    deletes last line typed 
ctrl-r
    redraws last line typed 
ctrl-d
    ends text input for many UNIX programs, including mail and write.

(Originally from this page.)

Running Tornado on dotCloud

Taro — Mon, 23 Jul 2012 05:56:14 +0000

To brush up my web development skills, I decided to try Tornado. I also wanted to find a hosting service of some sort for my projects, which must be cheap and developer friendly. There, dotCloud seemed very nice and I wanted to give it a shot. From what I gather, the support for Tornado is still on a best-effort basis, so things may still be fluid, but so far I like everything about the platform. Very intuitive and easy to use.

However, I was having a hard time finding a simple working example of using Tornado on dotCloud. I’m totally a beginner in the application stack after all, so I wanted to see a “hello world” all done nicely with Tornado on dotCloud, and keep modifying it to experiment things.

It turns out, there is one, and deploying it is easy. Assuming that I have installed CLI properly already, all I need to do is this:

$ git clone git://github.com/jpetazzo/tornado-on-dotcloud
$ mv tornado-on-dotcloud tornadotest
$ cd tornadotest
$ rm -rf .git
$ dotcloud create tornadotest
$ dotcloud push tornadotest .

In the tornadotest directory, I can modify main.py to do other stuff.

For a dotCloud beginner, the line

$ rm -rf .git

was important! I did not know the dotCloud push process was actually integrated with the existing git repository, and after modifying main.py countless times and pushing the changes only to see them never being reflected on the sandbox, I was this close to taking my own life (and being Japanese, I’m naturally very good at suicide). Alternatively, I could keep logging the changes to the git repo, instead of removing it. Then, every time I make a change to a file, I do the usual add-commit sequence of git, before pushing via dotcloud command. That’s another way of avoiding suicidal thoughts.

All that took a day to figure out! Lack of documentation or unwillingness to read through it is always a source of frustration. Otherwise, I think I’m in love with dotCloud to the extent that I won’t ever need to have a wife.

Searching for Nearest-Neighbors between Two Coordinate Catalogs

Taro — Fri, 29 Jun 2012 00:26:03 +0000

Say I have two catalogs of points, each in two-dimensional space. For each object in a catalog, I want to find the nearest object(s) in the other catalog. I can do this by computing the distances between every single unique pairs of objects and find the ones within a search radius and possibly doing an additional sort. However, there is a much more efficient way using k-d tree, and Scipy has the class for doing this sort of thing fairly easily.

Here is an example Python code:

#!/usr/bin/env python2.6
from cProfile import Profile
import numpy as np
from numpy.random import random
from scipy.spatial import KDTree, cKDTree


def getnnidx(d1, d2, r):
    t1 = KDTree(d1)
    t2 = KDTree(d2)
    idx = t1.query_ball_tree(t2, r)
    return idx


def cgetnnidx(d1, d2, r, k=5):
    t = cKDTree(d2)
    d, idx = t.query(d1, k=k, eps=0, p=2, distance_upper_bound=r)
    return idx


def main():
    # number of points
    np1 = 4000
    np2 = 2000

    # search radius
    r = 0.01

    # prepare coordinates; the input data for the constructor of
    # KCTree needs to be in the form:
    #
    #   data = [[x0, y0], [x1, y1], ... , [xN, yN]]
    #
    d1 = np.empty((np1, 2))
    d2 = np.empty((np2, 2))
    d1[:, 0] = random(np1)
    d1[:, 1] = random(np1)
    d2[:, 0] = random(np2)
    d2[:, 1] = random(np2)

    # profile two versions of KDTree implementations
    p = Profile()

    result = p.runcall(getnnidx, d1.copy(), d2.copy(), r)
    p.print_stats()

    p.clear()
    
    result = p.runcall(cgetnnidx, d1.copy(), d2.copy(), r)
    p.print_stats()


if __name__ == '__main__':
    main()

When I run this script, I get the following output:

         706932 function calls (692470 primitive calls) in 1.663 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1720    0.001    0.000    0.004    0.000 fromnumeric.py:1067(nonzero)
        2    0.000    0.000    0.000    0.000 fromnumeric.py:1147(shape)
    42293    0.046    0.000    0.224    0.000 fromnumeric.py:1314(sum)
        2    0.000    0.000    0.000    0.000 fromnumeric.py:1708(amax)
        2    0.000    0.000    0.000    0.000 fromnumeric.py:1769(amin)
      860    0.001    0.000    0.002    0.000 fromnumeric.py:664(argmax)
    14462    0.008    0.000    0.025    0.000 function_base.py:776(copy)
        2    0.000    0.000    0.045    0.022 kdtree.py:113(__init__)
      862    0.001    0.000    0.001    0.000 kdtree.py:139(__init__)
      860    0.001    0.000    0.001    0.000 kdtree.py:143(__init__)
   1722/2    0.033    0.000    0.045    0.022 kdtree.py:150(__build)
    42293    0.183    0.000    0.992    0.000 kdtree.py:22(minkowski_distance)
    12744    0.085    0.000    0.104    0.000 kdtree.py:36(__init__)
        1    0.002    0.002    1.618    1.618 kdtree.py:437(query_ball_tree)
  12743/1    0.270    0.000    1.616    1.616 kdtree.py:462(traverse_checking)
     6371    0.031    0.000    0.157    0.000 kdtree.py:49(split)
    42293    0.372    0.000    0.682    0.000 kdtree.py:7(minkowski_distance_p)
    12743    0.133    0.000    0.430    0.000 kdtree.py:72(min_distance_rectangle)
     7385    0.052    0.000    0.219    0.000 kdtree.py:76(max_distance_rectangle)
   169174    0.095    0.000    0.213    0.000 numeric.py:201(asarray)
        1    0.000    0.000    1.663    1.663 xmatch.py:8(getnnidx)
    57063    0.036    0.000    0.036    0.000 {isinstance}
     5164    0.001    0.000    0.001    0.000 {len}
      860    0.001    0.000    0.001    0.000 {method 'argmax' of 'numpy.ndarray' objects}
    25488    0.019    0.000    0.019    0.000 {method 'astype' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        2    0.000    0.000    0.000    0.000 {method 'max' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 {method 'min' of 'numpy.ndarray' objects}
     1720    0.003    0.000    0.003    0.000 {method 'nonzero' of 'numpy.ndarray' objects}
    42293    0.146    0.000    0.146    0.000 {method 'sum' of 'numpy.ndarray' objects}
    22165    0.008    0.000    0.008    0.000 {method 'tolist' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 {numpy.core.multiarray.arange}
   183636    0.135    0.000    0.135    0.000 {numpy.core.multiarray.array}
        1    0.000    0.000    0.000    0.000 {range}


         32 function calls in 0.008 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        3    0.000    0.000    0.000    0.000 fromnumeric.py:107(reshape)
        3    0.000    0.000    0.000    0.000 fromnumeric.py:1147(shape)
        1    0.000    0.000    0.000    0.000 fromnumeric.py:1708(amax)
        1    0.000    0.000    0.000    0.000 fromnumeric.py:1769(amin)
        1    0.000    0.000    0.000    0.000 fromnumeric.py:1865(prod)
        1    0.000    0.000    0.000    0.000 fromnumeric.py:32(_wrapit)
        2    0.000    0.000    0.000    0.000 numeric.py:201(asarray)
        4    0.000    0.000    0.000    0.000 numeric.py:314(ascontiguousarray)
        1    0.000    0.000    0.008    0.008 xmatch.py:15(cgetnnidx)
        1    0.000    0.000    0.000    0.000 {getattr}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'max' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'min' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'prod' of 'numpy.ndarray' objects}
        1    0.007    0.007    0.007    0.007 {method 'query' of 'scipy.spatial.ckdtree.cKDTree' objects}
        3    0.000    0.000    0.000    0.000 {method 'reshape' of 'numpy.ndarray' objects}
        6    0.000    0.000    0.000    0.000 {numpy.core.multiarray.array}

from which it is clear that cKDTree is much faster (1.663 vs. 0.008 seconds!). Not surprising given cKDTree is a C implementation of basically the same thing.

These classes are not exactly equivalent and therefore the outputs are different. KDTree.query_ball_tree returns:

[[],
 [216, 1767],
 [317],
 ...,
 [],
 [367],
 [1465, 1899]]

which is an array of indices of nearest neighbors. If an object is not found within the search radius, it simply returns an array of zero element.

cKDTree.query, on the other hand, returns something like

[[2000 2000 2000 2000 2000]
 [ 216 1767 2000 2000 2000]
 [ 317 2000 2000 2000 2000]
 ..., 
 [2000 2000 2000 2000 2000]
 [ 367 2000 2000 2000 2000]
 [1465 1899 2000 2000 2000]]

which is an array of indices of five nearest neighbors. When not found within a search radius, it uses the number of elements as a place holder. The number of nearest neighbors to retain can be specified by the input argument (k).

Installing Ubuntu (12.04 LTS 64-bit) from an ISO Image on Hard Drive

Taro — Mon, 14 May 2012 05:23:21 +0000

Sometimes it’s useful to be able to install Linux off an ISO image on hard drive for whatever reason. My Dell XPS 630i has had some issues when I was installing Debian off an install CD/USB, likely related to how GRUB 2 does not gracefully deals with SATA configuration particular to this machine (the install process doesn’t appear to recognize /dev/sdaX properly at some point, ending up in IO errors or kernel panic). In any case, I could solve this by installing Linux off a pre-partitioned hard drive. This time, I install Ubuntu. (I’m giving my XPS 630i away before moving back to the States!)

Here goes the procedure. I assume that I am installing Ubuntu from scratch and hard drive has been formatted.

First, I need a Linux Live media, whatever from which I can boot the computer. I burn a CD for this particular post. Go to the Ubuntu website, download an ISO image (here ubuntu-12.04-desktop-amd64.iso), and burn a Live CD.

Boot up the computer off Ubuntu Live (do not install at this point!) and prepare the disk partitions, using GParted. It would be best to create full partition scheme at this point, so during the installation I can simply format them without repartitioning. For Ubuntu, the root partition (“/”) must be at least 2.5 GB, otherwise the installer may complain. Typically, a /boot partition needs to be about 300 MB or so, but I store all my ISO files under /boot/iso, so I will use 3 GB for the partition just in case. (For an Ubuntu ISO image, 700 MB may be enough but for Linux Mint, I may need 2 GB, for example; I can also store multiple ISO images if I wish.) Here’s the partition scheme:

Filesystem            Size   Mounted on
/dev/sda5               3G   /
/dev/sda6               3G   /boot
/dev/sda7              10G   /tmp
/dev/sda8               6G   /var
/dev/sda9              10G   /usr
/dev/sda10             20G   /usr/local
/dev/sda11              8G   Linux Swap
/dev/sda12            690G   /home

Then on Terminal I install GRUB 2 on /dev/sda:

$ sudo mount /dev/sda5 /mnt
$ sudo mkdir /mnt/boot
$ sudo mount /dev/sda6 /mnt/boot
$ sudo grub-install --root-directory=/mnt /dev/sda
$ sudo mkdir /mnt/boot/boot-isos
$ sudo cp ~/Downloads/ubuntu-12.04-desktop-amd64.iso /mnt/boot/boot-isos
$ sudo umount /mnt/boot
$ sudo umount /mnt

I assumed that the ISO image was already downloaded to ~/Downloads directory. Now, reboot and on the GRUB prompt, follow this to boot off the ISO image:

$ set prefix=(hd0,6)/grub
$ insmod linux
$ insmod loopback
$ insmod iso9660
$ loopback loop (hd0,6)/boot-isos/ubuntu-12.04-desktop-amd64.iso
$ set root=(loop)
$ linux /casper/vmlinuz boot=casper iso-scan/filename=/boot-isos/ubuntu-12.04-desktop-amd64.iso noprompt noeject toram
$ initrd /casper/initrd.lz
$ boot

The machine should boot into Ubuntu Live. Once in Ubuntu, I should unmount /isodevice:

$ sudo umount -l -r -f /isodevice

Then I can simply run the installer on Ubuntu. In order to prevent the installer from stalling (see this bug report), I had to “format” all partitions except /boot. But otherwise everything should go fine.

Reference

HOWTO: ISO Booting with Grub 2
HOWTO: Boot & Install Ubuntu from the Grub Rescue Prompt

Repeating Failed Linux Command on Bash

Taro — Fri, 04 May 2012 15:33:24 +0000

I was in need of backing up a lot of data in a hurry. I used rsync between the file server and a locally mounted USB drive, but for some reason the command kept failing. It had something to do with checksum calculation, but it may have had something to do with how hardware is set up, so I didn’t wish to go looking for the very root of the problem; I just needed to back things up.

I simply decided to repeat the rsync command when it failed, until it succeeded syncing all files. Here’s a simple example script to do this:

#!/bin/bash

CMD="rsync -av --delete remote:/scrXX/dir /media/usb/storage"

while :
do
    echo $CMD ;
    $CMD ;

    if [ $? -eq 0 ]; then
        break;
    fi

    echo "... redo ..." ;
done

echo "Done." ;

The gist of the Bash trick is to use “$?”, which stores the return code of the last executed process.

Using Linux Shell Command within Python

Taro — Fri, 20 Jan 2012 19:54:34 +0000

If you just want to do some simple task, this would do:

from subprocess import call
cmd = "mkdir -p tmp/some/dir"
call(cmd, shell=True)

Installing Firefox 9 on Debian Squeeze

Taro — Fri, 30 Dec 2011 18:53:39 +0000

The questionable move by Firefox to the rapid release schedule has me disappointed. A little bugs here and there, incompatible add-ons, and so on. Seriously, the major, if not the most important advantage of Firefox is all these add-ons we have come to love, so making them unstable only makes the browser lose the edge.

Yet Firefox remains my default browser (it might change when I have time to give Chrome a serious shot though). On Debian, with its slow release schedule, this means that I need to custom install the latest stable version. Here’s the procedure.

Go to the Firefox download web page. Choose the version you want and download to a temporary directory, here ~/tmp. Then follow this:

$ cd ~/tmp
$ ls
firefox-9.0.1.tar.bz2
$ tar -xvjf firefox-9.0.1.tar.bz2
$ sudo mv firefox /usr/local/firefox-9.0.1
$ cd /usr/local
$ sudo ln -s firefox-9.0.1 firefox
$ cd /usr/local/bin
$ sudo ln -s ../firefox/firefox-bin firefox

This is it! You should be able to launch Firefox with the command firefox anywhere on the system.

Flash

I recommend installing Flash-Aid add-on and re-install Flash using the Firefox just installed.

Installing kcorrect (v4_2) for IDL Locally on Linux

Taro — Wed, 28 Dec 2011 19:46:53 +0000

I usually stay away from IDL but wanted to test the new version of kcorrect which usually runs on IDL. Fortunately a Linux box in the department has IDL installed, so I decided to test it there, installing kcorrect locally. As expected I ran into sneaky problems, so here’s my installation note.

I assume that the user home directory is /home/johndoe and all the stuff will be installed under /home/johndoe/lib. This isn’t my own Linux box and looks like this:

$ uname -a
Linux xxxx.somehost.somewhere 2.6.18-194.el5.centos.plus #1 SMP Tue May 18 17:01:19 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

First, add the following lines to ~/.bashrc:

export IDL_PATH=+/usr/local/rsi:$IDL_PATH

export IDLUTILS_DIR=/home/johndoe/lib/idlutils
export PATH=$IDLUTILS_DIR/bin:$PATH
export IDL_PATH=+$IDLUTILS_DIR/pro:$IDL_PATH
export IDL_PATH=+$IDLUTILS_DIR/goddard/pro:$IDL_PATH

Here, the first line is important and need to be set to the (system) IDL installation directory, here /usr/local/rsi; if this isn’t done anywhere, then IDL will not find all the procedures that are installed by default! (This sneaky caveat caused me quite a headache!)

Next, download idlutils via svn and compile the source code:

$ . ~/.bashrc
$ mkdir -p ~/lib
$ cd ~/lib
$ svn ls http://www.sdss3.org/svn/repo/idlutils/tags
... a list of tags; the latest version v5_4_26 here...
$ svn export http://www.sdss3.org/svn/repo/idlutils/tags/v5_4_26 idlutils
$ cd $IDLUTILS_DIR
$ evilmake all

Make sure there is no error during the compilation.

We are ready to install kcorrect. Add the following lines to ~/.bashrc just below what we just added above:

export KCORRECT_DIR=/home/taro/lib/kcorrect
export PATH=$KCORRECT_DIR/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$KCORRECT_DIR/lib
export IDL_PATH=$IDL_PATH:+$KCORRECT_DIR/pro

Download the tarball and compile the source codes:

$ . ~/.bashrc
$ cd ~/lib
$ wget http://cosmo.nyu.edu/blanton/kcorrect/kcorrect.v4_2.tar.gz
$ tar -xvzf kcorrect.v4_2.tar.gz
$ cd kcorrect
$ evilmake -k all

Make sure there is no error during compilation. That should be it. Also, don’t forget to browse through the Known Problems section of the kcorrect distribution website. In particular, the DEEP2 filters need to be updated by downloading from the most recent subversion repository.

To test if the installation went fine, try the first example on the kcorrect distribution website:

$ idl
IDL Version 8.0.1 (linux x86_64 m64). (c) 2010, ITT Visual Information Solutions
Installation number: XXXXXX.
Licensed for use by: Saint Mary's Univ, Astronomy and Physics

IDL> kcorrect, [1., 4.78, 10.96, 14.45, 19.05],  $
IDL>           [1100., 28., 7.7, 4.4, 2.5], $
IDL>           0.03, kcorrect, band_shift=0.1, chi2=chi2

... bunch of output ...

IDL> print, kcorrect
    -0.381880    -0.342148    -0.191399    -0.158157    -0.132817

g3data

Taro — Wed, 11 May 2011 16:10:32 +0000

$ sudo aptitude install g3data

This is all I needed to use this:

Love Debian.

Half-Light Radii for Various Profiles

Taro — Mon, 14 Feb 2011 22:17:51 +0000

For a radial profile of , the enclosed flux within the radius is given by

I’m only concerned about azimuthal symmetric cases, so . The idea is to solve

for .

Gaussian Profile

Say the Gaussian intensity profile is given by

The enclosed flux within is given by

Hence

Exponential Profile

The radial surface-brightness profile of a disk galaxy is often fitted by the exponential profile:

where is the disk scale length. The flux enclosed within the radius is given by

Solving for the half-light radius leads to the following:

A relevant numerical solution is

Sersic/de Vaucouleurs Profile

The radial surface-brightness profile of a spheroidal galaxy is often fitted by the Sersic (de Vaucouleurs when ) profile:

where (Capaccioli 1989). This expression explicitly parametrizes the profile in terms of half-light radius.

SciPy Weave Bug in Debian Squeeze

Taro — Fri, 04 Feb 2011 20:30:59 +0000

I was trying to remove a bottleneck by converting a portion of Python code into C++ via scipy.weave.inline, and encountered a bug. Here is my code:

#!/usr/bin/env python2.6
import numpy as np
from scipy import weave
from scipy.weave import converters


def compute_unique_pair_dists(xs, ys, ds):
    code = r'''
    int n = xs.size();
    int idx = 0;
    for (int i = 0; i < n; ++i) {
        float x = xs(i);
        float y = ys(i);
        for (int j = 0; j < n - 1 - i; ++j) {
            float dx = xs(i+1+j) - x;
            float dy = ys(i+1+j) - y;
            ds(idx++) = sqrt(dx * dx + dy * dy);
        }
    }
    '''
    weave.inline(code, ['xs', 'ys', 'ds'], type_converters=converters.blitz,
                 compiler='gcc', verbose=2)


def main():
    n = 10
    xs = np.arange(0, n)
    ys = np.arange(0, n)
    ds = np.zeros(n * (n + 1) / 2)
    compute_unique_pair_dists(xs, ys, ds)
    print(ds)


if __name__ == '__main__':
    main()

And this is what I get:


running build_ext
running build_src
build_src
building extension "sc_1eb4d019edb4774720031f15557074a811" sources
build_src: building npy-pkg config files
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'sc_1eb4d019edb4774720031f15557074a811' extension
compiling C++ sources
C compiler: g++ -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -fPIC

compile options: '-I/usr/lib/python2.6/dist-packages/scipy/weave -I/usr/lib/python2.6/dist-packages/scipy/weave/scxx -I/usr/lib/python2.6/dist-packages/scipy/weave/blitz -I/usr/lib/pymodules/python2.6/numpy/core/include -I/usr/include/python2.6 -c'
g++: /home/taro/.python26_compiled/sc_1eb4d019edb4774720031f15557074a811.cpp
In file included from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/applics.h:394,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/vecexpr.h:26,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/vecpick.cc:16,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/vecpick.h:287,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/vector.h:443,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/tinyvec.h:424,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array-impl.h:38,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array.h:26,
                 from /home/taro/.python26_compiled/sc_1eb4d019edb4774720031f15557074a811.cpp:11:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/mathfunc.h: In static member function ‘static long int blitz::_bz_abs::apply(long int)’:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/mathfunc.h:45: error: ‘labs’ is not a member of ‘std’
In file included from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array/funcs.h:23,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array/newet.h:23,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array/et.h:21,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array-impl.h:2509,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array.h:26,
                 from /home/taro/.python26_compiled/sc_1eb4d019edb4774720031f15557074a811.cpp:11:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/funcs.h: In static member function ‘static int blitz::Fn_abs::apply(int)’:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/funcs.h:503: error: call of overloaded ‘abs(int&)’ is ambiguous
/usr/include/c++/4.4/cmath:94: note: candidates are: double std::abs(double)
/usr/include/c++/4.4/cmath:98: note:                 float std::abs(float)
/usr/include/c++/4.4/cmath:102: note:                 long double std::abs(long double)
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/funcs.h: In static member function ‘static long int blitz::Fn_abs::apply(long int)’:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/funcs.h:524: error: ‘labs’ is not a member of ‘std’
In file included from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/applics.h:394,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/vecexpr.h:26,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/vecpick.cc:16,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/vecpick.h:287,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/vector.h:443,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/tinyvec.h:424,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array-impl.h:38,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array.h:26,
                 from /home/taro/.python26_compiled/sc_1eb4d019edb4774720031f15557074a811.cpp:11:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/mathfunc.h: In static member function ‘static long int blitz::_bz_abs::apply(long int)’:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/mathfunc.h:45: error: ‘labs’ is not a member of ‘std’
In file included from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array/funcs.h:23,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array/newet.h:23,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array/et.h:21,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array-impl.h:2509,
                 from /usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/array.h:26,
                 from /home/taro/.python26_compiled/sc_1eb4d019edb4774720031f15557074a811.cpp:11:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/funcs.h: In static member function ‘static int blitz::Fn_abs::apply(int)’:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/funcs.h:503: error: call of overloaded ‘abs(int&)’ is ambiguous
/usr/include/c++/4.4/cmath:94: note: candidates are: double std::abs(double)
/usr/include/c++/4.4/cmath:98: note:                 float std::abs(float)
/usr/include/c++/4.4/cmath:102: note:                 long double std::abs(long double)
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/funcs.h: In static member function ‘static long int blitz::Fn_abs::apply(long int)’:
/usr/lib/python2.6/dist-packages/scipy/weave/blitz/blitz/funcs.h:524: error: ‘labs’ is not a member of ‘std’
Traceback (most recent call last):
  File "bug.py", line 34, in 
    main()
  File "bug.py", line 29, in main
    compute_unique_pair_dists(xs, ys, ds)
  File "bug.py", line 22, in compute_unique_pair_dists
    compiler='gcc', verbose=2)
  File "/usr/lib/python2.6/dist-packages/scipy/weave/inline_tools.py", line 335, in inline
    **kw)
  File "/usr/lib/python2.6/dist-packages/scipy/weave/inline_tools.py", line 462, in compile_function
    verbose=verbose, **kw)
  File "/usr/lib/python2.6/dist-packages/scipy/weave/ext_tools.py", line 365, in compile
    verbose = verbose, **kw)
  File "/usr/lib/python2.6/dist-packages/scipy/weave/build_tools.py", line 288, in build_extension
    raise e
scipy.weave.build_tools.CompileError: error: Command "g++ -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -fPIC -I/usr/lib/python2.6/dist-packages/scipy/weave -I/usr/lib/python2.6/dist-packages/scipy/weave/scxx -I/usr/lib/python2.6/dist-packages/scipy/weave/blitz -I/usr/lib/pymodules/python2.6/numpy/core/include -I/usr/include/python2.6 -c /home/taro/.python26_compiled/sc_1eb4d019edb4774720031f15557074a811.cpp -o /tmp/taro/python26_intermediate/compiler_8430959729cb93b29ebb911a70f70625/home/taro/.python26_compiled/sc_1eb4d019edb4774720031f15557074a811.o" failed with exit status 1

Fortunately there is a fix:

The fix for this bug is simple. As per the debian patch linked to below, one needs to add an include for cstdlib above the BZ_NAMESPACE(blitz) call in the following files:
/usr/share/pyshared/scipy/weave/blitz/blitz/blitz.h
/usr/share/pyshared/scipy/weave/blitz/blitz/mathfunc.h

http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=10;filename=blitz%2B%2B.patch;att=1;bug=455661

This fix works for me by manually editing those files.

The bug appears to be reported a couple years ago but hadn’t been fixed for some reason.

Comparing the Efficiencies of Gaussian Convolution Routines

Taro — Wed, 26 Jan 2011 01:13:00 +0000

These days I quite often need to convolve (i.e., smooth) 2D images by some Gaussian. There are quite a few routines that can do this with varying efficiencies. So I compared a couple of them.

#!/usr/bin/env python2.6
import os
from time import time
import numpy as np
from numpy import exp, pi, sqrt
import pyfits as pf
from scipy.ndimage import gaussian_filter
from scipy.signal import convolve, fftconvolve
from pyraf import iraf
from pyraf.irafpar import IrafParList


KSIZE = 4


def gauss(x, s):
    return exp(-x**2 / (2. * s**2)) / sqrt(2. * pi) / s


def kernel(sx, sy, size):
    dx, dy = int(round(size * sx)), int(round(size * sy))
    xs, ys = np.arange(-dx, dx + 1), np.arange(-dy, dy + 1)
    fxs, fys = np.matrix(gauss(xs, sx)), np.matrix(gauss(ys, sy))
    return fxs.T * fys


def cv_iraf_gauss(data, sx, sy):
    def default_irafpar(task):
        # Return the IrafParList object for default parameters.
        return IrafParList(task.getName(), parlist=task.getDefaultParList())

    size = KSIZE
    ftmpin = 'tmpin.fits'
    ftmpout = 'tmpout.fits'
    pf.PrimaryHDU(data).writeto(ftmpin)

    # Load IRAF packages.
    iraf.image(_doprint=0)
    iraf.image.imfilter(_doprint=0)

    # Get default parameters and override some.
    plist = default_irafpar(iraf.gauss)
    plist.setParList(input = ftmpin,
                     output = ftmpout,
                     sigma = sx,
                     ratio = (1. * sy) / sx,
                     nsigma = size,
                     bilinear = 'yes')
    t1 = time()
    iraf.gauss(ParList=plist)
    t2 = time()

    os.remove(ftmpin)
    os.remove(ftmpout)

    return t2 - t1


def cv_convolve(data, sx, sy):
    k = kernel(sx, sy, KSIZE)

    t1 = time()
    convolve(data, k, mode='same')
    return time() - t1


def cv_fftconvolve(data, sx, sy):
    k = kernel(sx, sy, KSIZE)

    t1 = time()
    fftconvolve(data, k, mode='same')
    return time() - t1


def cv_gaussian_filter(data, sx, sy):
    t1 = time()
    gaussian_filter(data, [sy, sx])
    return time() - t1


def main():
    xsig, ysig = 5, 5
    nrep = 3

    for imsize in [512, 1024, 2048]:
        print('image size = %d' % imsize)
        data = np.zeros((imsize, imsize), dtype=np.float32)
        for func in [cv_gaussian_filter,
                     cv_convolve,
                     cv_fftconvolve,
                     cv_iraf_gauss]:
            ts = []
            for i in range(nrep):
                t = func(data, xsig, ysig)
                ts.append(t)

            print('%s: %f sec' % (func.__name__, np.mean(ts)))
        print('')


if __name__ == '__main__':
    main()

On my laptop, this script gives me the following results:

image size = 512
cv_gaussian_filter: 0.022649 sec
cv_convolve: 2.872223 sec
cv_fftconvolve: 0.096408 sec
cv_iraf_gauss: 0.069467 sec

image size = 1024
cv_gaussian_filter: 0.103350 sec
cv_convolve: 12.024448 sec
cv_fftconvolve: 0.401396 sec
cv_iraf_gauss: 0.253028 sec

image size = 2048
cv_gaussian_filter: 0.451775 sec
cv_convolve: 44.597186 sec
cv_fftconvolve: 1.838378 sec
cv_iraf_gauss: 1.009065 sec

Clearly the winner is scipy.ndimage.gaussian_filter. (This is not a fair comparison for the IRAF/PyRAF’s image.imfilter.gauss, because it includes time that it takes to do the file I/O; I don’t know of a way to avoid this. As is usually the case for IRAF/PyRAF routines, there is no way to get around the “feature,” and it merely shows its limitation as a library.)

Using NTP Server to Synchronize Time on Debian Squeeze

Taro — Sun, 09 Jan 2011 19:20:56 +0000

I want the time of my computer to get synchronized with a reliable time server. To do this, install ntp:

$ sudo aptitude install ntp
$ sudo /etc/init.d/ntp restart

After restarting the NTP server (the second line above) the system time should be synchronized within a couple minutes.

On a laptop, the NTP daemon should be stopped while on battery. With laptop-mode, this can easily be done by creating symlinks under the /etc/laptop-mode directories:

$ cd /etc/laptop-mode/batt-stop
$ sudo ln -s /etc/init.d/ntp .
$ cd /etc/laptop-mode/lm-ac-start
$ sudo ln -s /etc/init.d/ntp .

This way, the daemon runs while on AC, but not when the battery is used.

Suppressing Warnings in PyFITS

Taro — Wed, 29 Dec 2010 02:19:24 +0000

PyFITS is obviously a great Python package if you ever need to deal with data in astronomy.

One thing that consistently annoyed me however was their use of warning messages. For example, whenever you overwrite an existing FITS file, the writeto method issues a warning, saying “overwrite existing file …” This is a very redundant message, as PyFITS doesn’t overwrite files by default, and I’m giving an explicit clobber option…

It’s actually very easy to suppress this sort of warning with the with statement:

import warnings
import pyfits
...
hdus = pyfits.PrimaryHDU(data)
...
with warnings.catch_warnings():
    warnings.simplefilter('ignore')
    hdus.writeto("somefile.fits", clobber=True)
...

Suppressing warning messages is discussed in the PyFITS user manual, but the method using with described here is more elegant.

The Python with statement is a pleasure to use. I should’ve started using this long time ago.

Installing Firefox 4 Beta 9 on Debian Squeeze

Taro — Fri, 24 Dec 2010 16:47:56 +0000

I find the easiest way is to download a precompiled binary from this nightly build repository.

For beta 9 on AMD64:

$ wget http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/firefox-4.0b9pre.en-US.linux-x86_64.tar.bz2
$ tar -xvjf firefox*.tar.bz2
$ sudo mv firefox /usr/local/firefox4-b9pre
$ sudo ln -s /usr/local/firefox4-b9pre /usr/local/firefox4
$ sudo ln -s /usr/local/firefox4/firefox /usr/local/bin/firefox4

That’s it.

For some post install configurations (such as incompatible add-ons), see a previous post.

Mounting an ext4 Partition over NFS on Mac OS X Snow Leopard

Taro — Mon, 20 Dec 2010 19:59:09 +0000

The title says it all. I want to use a USB drive formatted with ext4 on Snow Leopard. I mostly use the drive with Linux but sometimes I wish to mount it on my iMac at work to transfer a large amount of data (the wired network is just much faster there…).

My first attempt to mount an ext4 filesystem on Snow Leopard was to use MacFUSE and ext4fuse, but I quickly found this solution was inflexible (largely the ext4 support is very preliminary).

My solution is to install Debian Squeeze on the iMac using VirtualBox, mount the ext4 formatted USB drive on the guest Linux, and export the partition over NSF to the host iMac. I had shied away from any virtualization up until this point, but it was actually quit trivial to set it up, so no installation note here. Just download and install VirtualBox on iMac, install an OS by popping in an install media, following the standard Debian install procedure, and that was it. The Debian installer has gotten easier to use so this should be very trivial.

Configure VirtualBox for Detect a USB Drive

Here I follow this article.

On VirtualBox, select the Debian I just installed. Go to Settings -> Ports -> USB. Check “Enable USB Controller” and “Enable USB 3.0 (EHCI) Controller.” Add a new USB filter with all fields initially set to empty strings. This is important so that the ext4 formatted USB drive gets detected before OS X does. (When OS X detects such a device, it thinks the device is unreadable and gives me an option to either initialize (omg), ignore, or eject. When this happens, make sure to choose “ignore.”)

Configure Network between Host and Guest

Here I follow this article.

On VirtualBox, select the Debian I just installed. Go to Settings -> Network -> Adapter 2. Check “Enable Network Adapter.” For “Attached to,” choose “Host-only Adapter” and “vboxnet0″ for its name. I may leave the default advanced settings.

On iMac host, find out the IP address of vboxnet0:

host$ sudo ifconfig
...
vboxnet0: ...
    ...
    inet 192.168.56.1 netmask 0xffffff00 broadcast 192.168.56.255

So the host machine is at 192.168.56.1.

Now, login to Debian guest as a regular user. I configure the second network adapter that I created as follows. Edit /etc/network/interfaces to have the following lines:

auto eth1
iface eth1 inet static
    address 192.168.56.10
    netmask 255.255.255.0

so that the guest machine is at 192.168.56.10. Do

guest$ sudo ifup eth1

and try to login to guest from host:

host$ ssh 192.168.56.10

Make sure everything works fine through this point.

Mount the USB Drive in Debian

I just follow a standard procedure for mounting a USB drive, but here goes. Start the guest Linux system. Once logged in as a regular user, plug in the USB drive to iMac. Find out which node the device has been attached:

guest$ dmesg | tail
...
scsi 3:0:0:1: Attached scsi generic sg3 type 13
...
 sdb: sdb1
...

This is how it looks like on my system. Create a mount point and mount it:

guest$ mkdir ~/usb
guest$ sudo mount /dev/sdb1 /home/username/usb
guest$ ls /home/username/usb
... should see the content of the usb drive ...

If this works, the next step is to set up the NFS.

Setting up NFS on Debian Guest

~~On iMac host, find out the uid and gid of myself as a regular user:~~

host$ id
uid=501(username) gid=20(staff) ...

~~Take a note of these numbers.~~

On Debian, install some packages:

guest$ sudo aptitude install nfs-kernel-server nfs-common portmap

In /etc/exports, have the following line:

/home/username/usb 192.168.56.1(rw,sync,all_squash,no_subtree_check,anonuid=0,anongid=0)

[NOTE: THIS ACTUALLY DOESN'T WORK AS I THOUGHT.] Here, use the uid and gid of my regular user on iMac for anonuid and anongid; this is important so that the NFS mounted filesystem can be used as a regular user on iMac host. Then execute:

guest$ sudo exportfs -a

Now /home/username/usb should be mountable on iMac.

Mounting the NFS Partition on iMac Host

host$ mkdir usb
host$ sudo mount_nfs -P 192.168.56.10:/home/username/usb usb
host$ ls usb
... should see the content of the usb drive ...

That’s it.

Power Management on Lenovo T410s

Taro — Sun, 19 Dec 2010 17:57:41 +0000

T410s’s battery life is very short, providing only 2 – 3 hours without a ultrabay module battery. I wish to continually improve the battery life, ideally getting around 5 hours in the end with the main and the ultrabay module combined. Here is my continually growing notes.

I use Debian Squeeze. The general configuration of my T410s box is described in another page.

First, install must-have software:

$ sudo aptitude install powertop acpi acpid

To monitor the power consumption,

$ sudo powertop

I’d say this is the most essential tool for power management. I can see which processes wake up the computer most, and try to eliminate them one by one. Here’s how powertop typically looks like on my T410s.

Laptop Mode Configurations

Install laptop-mode-tools:

$ sudo aptitude install laptop-mode-tools

In /etc/laptop-mode/laptop-mode.conf, make sure to have this line:

ENABLE_AUTO_MODULES=1

which is set to 0 by default. When on battery this will apply a bunch of power saving settings. This sole step should typically save about 2 – 2.5 W of power consumption according to powertop! The following additional tweaks (overriding the default settings) are optional.

To disable ethernet when on battery, make sure /etc/laptop-mode/conf.d/ethernet.conf has the following lines:

CONTROL_ETHERNET="auto"
DISABLE_ETHERNET_ON_BATTERY=1

For more aggressive power saving on Intel HDA, make sure /etc/laptop-mode/conf.d/intel-hda-powersave.conf has the following lines:

CONTROL_INTEL_HDA_POWER="auto"
INTEL_HDA_DEVICE_TIMEOUT=3
INTEL_HDA_DEVICE_CONTROLLER=1

For more aggressive power saving on the wireless adapter, make sure /etc/laptop-mode/conf.d/wireless-iwl-power.conf has the following lines:

CONTROL_IWL_POWER="auto"
IWL_BATT_POWER=5

After changing the configuration, restart laptop-mode:

$ sudo /etc/init.d/laptop-mode restart

to make the changes effective.

Starting & Stopping Services

For example, if I want to stop ssh from running while on battery, do this:

$ cd /etc/laptop-mode/batt-stop
$ sudo ln -s /etc/init.d/ssh .

Now, I want ssh to get restarted when on AC:

$ cd /etc/[lm|nolm]-ac-start
$ sudo ln -s /etc/init.d/ssh .

(depending on how laptop-mode is configured, I create a symlink either lm-ac-start or nolm-ac-start.)

I should create similar symlinks for all services that should be stopped while on battery.

Configuring System on Switch between AC & Battery

Such a script may look like this:

#!/bin/bash

if [ ! -f /sys/devices/platform/smapi/ac_connected ]; then
  exit 0
fi

TAG=pmtp
STATUS=`cat /sys/devices/platform/smapi/ac_connected`

if [ "$STATUS" = "1" ]; then
  logger -t $TAG "on ac"
  logger -t $TAG "increasing swappiness"
  sysctl -w vm.swappiness=60
else
  logger -t $TAG "on battery"
  logger -t $TAG "reducing swappiness"
  sysctl -w vm.swappiness=10
fi

Let us save this script as /usr/local/bin/pmtp. (FYI, this script changes the amount of swap disk usage, effectively reducing disk access while on battery. This is a recommended thing to do.)

To let laptop-mode handle this script, modify /etc/laptop-mode/conf.d/exec-commands.conf to have these lines:

BATT_EXEC_COMMAND_0="/usr/local/bin/pmtp"
LM_AC_EXEC_COMMAND_0="/usr/local/bin/pmtp"
NOLM_AC_EXEC_COMMAND_0="/usr/local/bin/pmtp"

This way the script gets run each time the AC/battery state changes.

KDE

In addition to what I do above, I let KDE do some power management work.

Go to Menu -> Computer -> System Settings -> Advanced -> Power Management -> Edit Profiles. There, I can change various options for different profiles. It is good idea to change the display brightness under the Screen tab.

I also run a script to enable/disable a few desktop related applications and settings. In particular, the sound for the KDE system notifications should be disabled entirely, because they cause a vast amount of interruptions. (If I see knotify4 processes popping up in powertop, this is the cause.) So I create the following script:

#!/bin/bash

if [ ! -f /sys/devices/platform/smapi/ac_connected ]; then
    exit 0
fi

TAG=pmkde
STATUS=`cat /sys/devices/platform/smapi/ac_connected`

if [ "$STATUS" = "1" ]; then
    logger -t $TAG "on ac"

    logger -t $TAG "start xflux"
    for i in `pgrep ^xflux$` ; do kill -9 $i ; done
    xflux -l 44.65 -g -63.6

    logger -t $TAG "start dropbox"
    for i in `pgrep ^dropbox$` ; do kill -9 $i ; done
    /usr/local/dropbox/dropboxd &

    logger -t $TAG "enable system notification"
    kwriteconfig --file=/home/username/.kde/share/config/knotifyrc --group=Sounds --key="No sound" false
else
    logger -t $TAG "on battery"

    logger -t $TAG "stop xflux"
    for i in `pgrep ^xflux$` ; do kill -9 $i ; done

    logger -t $TAG "stop dropbox"
    for i in `pgrep ^dropbox$` ; do kill -9 $i ; done

    logger -t $TAG "disable system notification"
    kwriteconfig --file=/home/username/.kde/share/config/knotifyrc --group=Sounds --key="No sound" true
fi

Save this to maybe ~/bin/pmkde.sh. In this script, I also start/stop flux and Dropbox.

Under the CPU and System tab, specify this shell script for the “When loading profile execute:” entry. This script should be used for all profiles.

Disabling Bluetooth

I can use Fn + F5 to switch on/off just WiFi, not Bluetooth. But I don’t use any Bluetooth device at all. To control the on/off status of wireless device on the command line, I need:

$ sudo aptitude install rfkill
... below is an example usage ...
$ rfkill list
$ rfkill block bluetooth

I can live without Bluetooth entirely.

To disable Bluetooth at boot time, add the following line:

$ rfkill block bluetooth

to /etc/rc.local (before the “exit 0″ statement, of course).

Preload Daemon

$ sudo aptitude install preload

In /etc/default/preload, uncomment the line reading

OPTIONS="-l /dev/null"

This should reduce disk activity.

Changing Swappiness

On the command line,

$ sudo sysctl -w vm.swappiness=10

The system will try to use physical RAM as much as possible instead of swapping. To make this in effect upon boot, add the line

vm.swappiness=10

to /etc/sysctl.conf. However, I opt to change this particular setting dynamically using a custom script described above.

Battery Charge Thresholds

To extend battery life, charge thresholds should be tweaked. My current settings are:

$ echo 90 | sudo tee /sys/devices/platform/smapi/BAT0/start_charge_thresh
$ echo 95 | sudo tee /sys/devices/platform/smapi/BAT0/stop_charge_thresh

Do this for each battery in my system (BAT0, BAT1, etc.)

Chromium over Firefox

My default browser is Firefox. When browsing the web, however, Chromium is much easier on power. I can save about 1 W using Chromium.

Powertop Status

December 19, 2010 — With all this effort, I currently run T410s with about 10 W on idle. That goes up to 13 – 14 W with typical usage (WiFi on, browsing the web, writing on an editor, etc.).

Reference

ThinkWiki – A truly great resource for ThinkPad users.
PC Plus article – See especially the comment section.

Automounting a USB Hard Drive in Debian Squeeze

Taro — Sun, 19 Dec 2010 05:40:41 +0000

I want to automatically mount a USB hard disk drive at a fixed mount point every time it gets plugged in. I also want to automatically unmount the device from the system when it’s unplugged.

With udev, this is easy.

First, attach the USB drive to the computer and find out its serial number:

$ /sbin/udevadm info -a -p $(/sbin/udevadm info -q path -n /dev/sdb)

(Here, I assume the drive is assigned /dev/sdb; check with dmesg if unsure.) This will spew out a lot of information, but it should be easy to track down. In my case, I see lines like this:

...
    ATTRS{manufacturer}=="Western Digital"
    ATTRS{product}=="My Passport 0730"
    ATTRS{serial}=="57383"
...

Take a note of the serial number to identify the device.

Under /etc/udev/rules.d, create a file named like 80-usb-wd.rules, with the following content:

ACTION=="add", SUBSYSTEMS=="usb", ATTRS{serial}=="57583", SYMLINK+="wd%n"
ACTION=="add", SUBSYSTEMS=="usb", ATTRS{serial}=="57583", RUN+="/bin/mkdir /media/wd"
ACTION=="add", SUBSYSTEMS=="usb", ATTRS{serial}=="57583", RUN+="/bin/mount -t auto -o defaults /dev/wd1 /media/wd", OPTIONS="last_rule"
ACTION=="remove", SUBSYSTEMS=="usb", ATTRS{serial}=="57583", RUN+="/bin/umount /media/wd"
ACTION=="remove", SUBSYSTEMS=="usb", ATTRS{serial}=="57583", RUN+="/bin/rmdir /media/wd", OPTIONS="last_rule"

What these instructions do is two things. When the USB drive with the specific serial number is attached, udev creates device nodes like /dev/wd, /dev/wd0, /dev/wd1, …, the exact number of these nodes depending on how many partitions are in the drive. I only have one partition on this drive, and /dev/wd1 points to it. Then it creates a directory /media/wd, and the partition /dev/wd1 gets mounted to the node.

Unfortunately, the device is mounted as root, so in order to unmount the filesystem, I need to unmount as root:

$ sudo umount /media/wd

When the USB drive is removed from the system, it unmounts the partition and the /media/wd directory is deleted as well.

Restart udev and the change will be in effect:

$ sudo /etc/init.d/udev restart

This way I can expect the specific USB drive to be always mounted at /media/wd.

PS3 Media Server on Debian Squeeze

Taro — Tue, 14 Dec 2010 14:49:57 +0000

My music collection consists of Ogg Vorbis files. Rather than re-encoding them again for PlayStation 3, I decided to serve music files via a media server. Fortunately this is very easy to do.

Installation

Mostly following this note:

$ sudo aptitude update
$ sudo aptitude install sun-java6-jre
$ sudo aptitude install mplayer mencoder ffmpeg
$ sudo update-alternatives --config java  # choose Sun Java if multiple Java installs exist

Move to temporary directory and download Linux source tarball:

... download .tgz file ...
$ tar -xvjf pms*.tgz
$ sudo mv pms-generic-linux-unix-1.20.412 /usr/local/pms-1.20.412
$ sudo ln -s /usr/local/pms-1.20.412 /usr/local/pms
$ sudo chmod a+x /usr/local/pms/PMS.sh

To launch the server properly, I have to run the PMS.sh script from the /usr/local/pms directory. Instead of having to cd into that directory each time, I created the following startup script and saved it as /usr/local/bin/pms:

#!/bin/sh
(cd /usr/local/pms && exec ./PMS.sh)

Giving the executable permissions:

$ sudo chmod a+x /usr/local/bin/pms

This way, the pms command will launch the media server anywhere on the system.

Scrobbling with Last.fm

I want music tracks to be scrobbled with Last.fm. Looking at this forum thread, there is a plugin to make this possible.

Download the LastFMScrobbler.zip, unzip the content and move the LastFMScrobbler.jar file to /usr/local/pms/plugins.

Restart the PMS server.

Choose the General Configuration tab. Under the Plugin Systems section, there is an entry for Last.fm Scrobbler plugin. Click on it and I’m asked to provide the username and password for Last.fm. This info gets stored unencrypted in a text file, so be careful about the password…

Restart the PMS server and the music tracks should now be scrobbled. Check on my Last.fm profile page.

Installing Debian Squeeze from Hard Disk

Taro — Tue, 14 Dec 2010 14:37:21 +0000

Unfortunately the Debian installer does not work on my Dell desktop box. I still wanted to do a fresh install of Squeeze, and the only sensible option for me was to install off hard disk. I have been having the same issue for several months and I have not been able to solve it myself, and nobody responds to my bug report… However, this hard drive method may work well even if I have a working install USB stick/CD media.

I assume that a working Linux installation exists on the system already; it will only be used to launch the installer. (In practice, I used an existing Lenny installation to go through this procedure, only to wipe it over with Squeeze.) For this note I use the Debian installer beta 2 for Squeeze (AMD64).

Create a partition large enough to hold a netinst .iso image (~ 170 MB) plus maybe 20 MB for kernel image and whatnot. For illustration, I assume it is at /dev/sda11. Move to a temporary working directory, download some files, mount the partition, and copy files to the install partition:

$ wget http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/hd-media/initrd.gz
$ wget http://ftp.nl.debian.org/debian/dists/testing/main/installer-amd64/current/images/hd-media/vmlinuz
$ wget http://cdimage.debian.org/cdimage/squeeze_di_beta2/amd64/iso-cd/debian-squeeze-di-beta2-amd64-netinst.iso
$ mkdir mnt
$ sudo mount /dev/sda11 mnt
$ sudo cp initrd.gz mnt
$ sudo cp vmlinuz mnt
$ sudo cp debian-squeeze-di-beta2-amd64-netinst.iso mnt/netinst.iso
$ sudo umount /dev/sda11

Now I need to modify the GRUB menu to add an entry for the installer that I just made.

For the new GRUB version 2, an entry in /boot/grub/grub.cfg looks a bit complicated:

menuentry 'Squeeze Installer Beta 2' --class debian --class gnu-linux --class os {
  insmod part_msdos
  insmod ext2
  set root='(hd0,msdos11)'
  search --no-floppy --fs-uuid --set SOMELENGTHYHASH
  echo 'Loading Squeeze installer'
  linux /vmlinuz root=UUID=SOMELENGTHYHASH
  echo 'Loading init ramdisk'
  initrd /initrd.gz
}

Here, the root in which the install partition is the eleventh partition of the first hard drive (hence hd0, msdos11). With the lengthy UUID hash for the hard drive, I’d just copy one of the GRUB entries for working installations.

For an old version of GRUB, an entry in /boot/grub/menu.lst would look like this:

title  Squeeze Installer Beta 2
root   (hd0,10)
kernel /vmlinuz
initrd /initrd.gz

This means the install partition is located at the 11th partition of the first hard drive, and the paths to the kernel and initrd images are located at the topmost level of the partition. This is the sort of entry I copy and paste from other entries in /boot/grub/menu.lst.

After reboot, select the menu entry just added and I will be starting the installer right away!

IMPORTANT: When I partition the drive during the installation process, do not mount the install partition. If I do, the partitioning and the subsequent installation will fail.

USB 3.0 ExpressCard Adapter on Debian Squeeze

Taro — Thu, 09 Dec 2010 19:10:00 +0000

My Lenovo T410s has a rather small 80 GB SSD, and with the amount of data that I need to deal with these days, something like mounting network drive isn’t fast enough for me. So I decided to try USB 3.0, since I have an open ExpressCard slot.

Browsing the web, I found that most USB 3.0 controllers are using NEC uPD720200, which the Linux kernel natively supports. As long as the adapter uses the chip, I should be okay. I got Transcend USB 3.0 ExpressCard Adapter, after making sure on the manufacture web site that it uses the NEC chip.

Kernel Configuration

I have to make sure relevant kernel options are enabled. If not, build a kernel from source, making sure the following options are set:

PM=y
ACPI=y
PCI=y
PCIEPORTBUS=y
HOTPLUG_PCI=m
HOTPLUG_PCI_ACPI=m
USB=m
USB_ANNOUNCE_NEW_DEVICES=y
USB_XHCI_HCD=m

Hope I didn’t leave out other essential options…

Using ExpressCard

If not loaded automatically, I need to load the acpiphp kernel module manually:

$ sudo modprobe acpiphp

When I insert the adapter, it should be recognized:

$ lspci | grep USB
...
05:00.0 USB Controller: NEC Corporation uPD720200 USB 3.0 Host Controller (rev 03)
...

This should be it.

Installing JWM on Debian Squeeze

Taro — Tue, 07 Dec 2010 05:27:36 +0000

To really save battery consumption when needed, I decided to install JWM on my laptop.

As usual, I did

$ sudo aptitude install jwm

and just selected jwm at the KDM login menu.

However, every time I tried to launch JWM, my computer got stack with no response. It turns out the configuration files are not installed properly by default, so I needed to copy and unzip /usr/share/doc/jwm/example.jwmrc.gz, and have it renamed to ~/.jwmrc. After doing this, I was able to login to a JWM session, although the resource file is extremely basic, so I have to learn how to configure things to make it usable.

At the same time, powertop doesn’t show too drastic an improvement on power consumption compared to when I’m using KDE. I wonder why.

SExtractor on Debian Squeeze (AMD64)

Taro — Tue, 30 Nov 2010 14:30:29 +0000

Installing from Debian Repository

$ sudo aptitude install sextractor

This version is rather old (v2.4.4) though.

Installing from Source

I need FFTW and ATLAS installed already, and assume that FFTW was installed at /usr/local/fftw, and ATLAS at /usr/local/atlas following the source install procedures described in those notes. Make sure that you build both double and single precision versions of FFTW by going through the make procedure twice to install both.

Download the SExtractor source into a temporary directory:

$ mkdir tmp
$ cd tmp
... download sextractor-2.8.6.tar.gz
$ tar -xvzf sex*.tar.gz
$ cd sextractor-2.8.6
$ ./configure --prefix=/usr/local/sextractor-2.8.6 \
    --with-atlas=/usr/local/atlas/lib --with-atlas-incdir=/usr/local/atlas/include \
    --with-fftw=/usr/local/fftw/lib --with-fftw-incdir=/usr/local/fftw/include \
    --enable-threads
$ make
$ sudo make install
$ ln -s /usr/local/sextractor-2.8.6 /usr/local/sextractor
$ sudo cp -rp config /usr/local/sextractor-2.8.6

This will install SExtractor at /usr/local/sextractor-2.8.6 (which is symlinked by /usr/local/sextractor) and an executable command sex will be located at /usr/local/sextractor/bin/sex. Make a simlink to that binary /usr/local/bin to make it available system-wide if I wish. I might also want to make a simlink to /usr/local/sextractor/bin/ldactoasc, which is a utility command that might be useful. The last line copies the SExtractor configuration files and convolution masks to /usr/local/sextractor/config. The default convolution masks are often used so it may be a good idea to keep them under the same installation directory.

Error at configure. For version 2.8.4, I might encounter errors at the configure stage. If this happens, inspect config.log file to see what went wrong. Typically I found myself getting stuck because my ATLAS source installation had a linking bug (?) that caused the “hidden symbol” error which is described in the ATLAS installation note mentioned above; this was fixed with rebuilding ATLAS. Another problem that I might encounter is “undefined reference” errors when linking with LAPACK. This can be solved by modifying configure script so that it has “-lf77blas” for every occurrence of “-llapack” in that file. There is a patch to fix this; download to the source directory, rename it to sexconfigfix.patch and then do

$ patch < sexconfigfix.patch

This will make necessary modifications.

FFTW on Debian Squeeze (AMD64)

Taro — Tue, 30 Nov 2010 13:56:40 +0000

Installing from Debian Repository

$ sudo aptitude install libfftw3-3 libfftw3-dev

The second package is only necessary for development.

Installing from Source

The version of FFTW I use here is 3.2.2. Follow this:

$ mkdir tmp
$ cd tmp
$ wget http://www.fftw.org/fftw-3.2.2.tar.gz
$ tar -xvzf fftw-3.2.2.tar.gz
$ cd fftw-3.2.2
$ ./configure --prefix=/usr/local/fftw-3.2.2 --enable-threads \
    --enable-single F77=gfortran
$ make
$ sudo make install
$ sudo ln -s /usr/local/fftw-3.2.2 /usr/local/fftw

This will install FFTW under /usr/local/fftw-3.2.2, symlinked by /usr/local/fftw as well.

Note that the portion in red at the configure line should be modified accordingly. I use GFortran, hence the F77 option. One of the programs that I often ends up building need both double and single precision versions of FFTW, so I run ./configure ; make ; make install once with --enable-single and another without the --enable-single option. This will install both versions under the same lib directory.

ATLAS on Debian Squeeze (AMD64)

Taro — Tue, 30 Nov 2010 13:38:31 +0000

Installing from Debian Repository

$ sudo aptitude install libatlas3gf-base
$ sudo aptitude install libatlas-base-dev

The second line is necessary only for development.

Installing from Source

Disable CPU throttling. Find out how many CPUs I have:

$ cat /proc/cpuinfo
... info for each CPU ...

Check which CPU I have. Mine is Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz and there are four CPUs (really two CPUs but becomes four with hyperthreading). Set each CPU’s governor as performance:

$ for i in {0..3}; do sudo cpufreq-set -g performance -c $i ; done

(Note “{0..3}” should reflect how many CPUs there are; for a dual core machine, it probably should be {0..1}.)

In case, do this first:

$ sudo aptitude install gcc gfortran

I download tarballs for ATLAS and LAPACK from the official repositories. Here I use ATLAS 3.9.32 (still a development version) and LAPACK 3.3.0. I assume the compiled library and header files will be installed at /usr/local/atlas-3.9.32, which will be symbolic linked at /usr/local/atlas.

Prepare the source directory:

$ mkdir tmp
$ cd tmp
$ wget http://www.netlib.org/lapack/lapack.tgz  # at the time of writing this is v3.3.0
... download the tarball for ATLAS from sourceforge ...
$ tar -xvjf atlas3.9.32.tar.bz2
$ mv ATLAS atlas-3.9.32
$ ln -s atlas-3.9.32 atlas
$ mkdir atlas/bld
$ cd atlas/bld
$ ../configure -b 64 --shared \
  --prefix=/usr/local/atlas-3.9.32 --with-netlib-lapack-tarfile=../../lapack.tgz

Hidden symbol in gcc error. Now this may only be relevant to my current configuration, but when I linked some other program against my custom ATLAS build (I was compiling SExtractor), I have the following error:

/usr/bin/ld: prog: hidden symbol `__powidf2' in /usr/lib/gcc/x86_64-.../libgcc.a(_powidf2.o) is referenced by DSO
/usr/bin/ld: final link failed: Nonrepresentable section on output

which prevented me to build programs which link to ATLAS. It took a while to find a fix, but to prevent this error, I need to modify lib/Makefile in the bld directory (after I run ../configure once) so that

liblapack.so : liblapack.a
        ld $(LDFLAGS) -shared -soname $(LIBINSTdir)/liblapack.so \
            -o liblapack.so  -rpath-link $(LIBINSTdir) --whole-archive \
           liblapack.a --no-whole-archive $(F77SYSLIB) -lgcc

“-lgcc” (in red) is added.

Specifying Architecture. The build process might take very long time if I don’t specify right architecture. Do this:

$ make xprint_enums ; ./xprint_enums

and I get a list of enums for different CPU architectures. Mine is Intel Core i5, which isn’t actually listed but I figured it might be close enough to Core i7, so I chose that one (18). I will feed this to the configure script. Earlier I also found out that my CPU runs at 2.4 GHz. I will also feed this info to the configure script as seen next.

Now we do:

$ ../configure -A 18 -b 64 -D c -DPentiumCPS=2400 -Fa alg -fPIC --shared \
  --prefix=/usr/local/atlas-3.9.32 --with-netlib-lapack-tarfile=../../lapack.tgz
$ time make build

(Note the options “-A 18″ and “-D c -DPentiumCPS=2400″ have been added.) This takes about 15 minutes on my Lenovo T410s.

If I wish to do some checking, do:

$ make check
...
$ make time
...

I may wish to compare the outputs with that found in ../doc/atlas_install.pdf. My timing tests look like this:

            single precision                  double precision
            ********************************  *******************************
            real             complex          real             complex
            ---------------  ---------------  ---------------  ---------------
Benchmark   Refrenc Present  Refrenc Present  Refrenc Present  Refrenc Present
=========   ======= =======  ======= =======  ======= =======  ======= =======
kSelMM      718.6   863.4    663.4   798.7    339.3   402.4    334.5   405.6
kGenMM      185.4   216.8    183.8   218.6    179.1   194.9    177.6   204.4
kMM_NT      157.9   186.7    160.7   190.1    170.5   172.2    166.6   186.3
kMM_TN      177.7   206.6    184.3   217.9    171.4   186.0    169.7   196.2
BIG_MM      679.2   796.3    691.1   802.4    322.9   353.8    323.2   357.0
kMV_N       175.7   149.6    353.1   292.1     92.3    74.3    159.5   136.8
kMV_T       182.3   150.2    372.1   296.8     94.8    76.3    161.9   139.4
kGER        137.9   109.1    267.2   213.1     68.9    52.2    121.7   103.1
make[1]: Leaving directory `/usr/local/src/atlas/atlas-3.9.32/bld'

If I don’t find any problem, finally do:

$ sudo make install
$ sudo ln -s /usr/local/atlas-3.9.32 /usr/local/atlas

That’s it.

Installing IRAF version 2.15 on Debian Squeeze

Taro — Sun, 28 Nov 2010 03:05:14 +0000

My way of installing IRAF v2.15, now 64-bit by default.

First create a regular user named “iraf”:

$ sudo adduser iraf
$ sudo aptitude install csh
$ sudo ln -s /usr/lib/libncurses.so /usr/lib/libtermcap.so.2

The second line installing csh is necessary because people using csh don’t go away any time soon… The last line (which is quite a hack) is necessary to avoid a problem of this IRAF version linking to a very old library which no longer exists in Debian Squeeze. Also it is recommended to modify the line in /etc/passwd so that /home/iraf becomes /iraf/iraf/local. Mine looks like this:

iraf:x:1001:1001:,,,:/iraf/iraf/local:/bin/csh

Create the directory hierarchy:

$ sudo mkdir /usr/local/iraf-2.15
$ sudo chown iraf /usr/local/iraf-2.15
$ sudo chgrp iraf /usr/local/iraf-2.15
$ sudo ln -s /usr/local/iraf-2.15 /iraf
$ cd /iraf
$ su iraf
... as user iraf ...
$ chsh
... enter /bin/csh to change to csh ...
$ mkdir bin imdirs iraf

I’m doing it this way so that everything related to IRAF will be installed under /usr/local/iraf-2.15, for ease of future upgrade. Download the tarball and expand it:

$ cd /iraf/iraf
$ wget ftp://iraf.noao.edu/iraf/v215/PCIX/iraf.lnux.x86_64.gz
$ tar -xvzf iraf.lnux.x86_64.gz
$ su
... as root ...
# iraf=/iraf/iraf   # defining $iraf; csh user might do differently...
# $iraf/unix/hlib/install

My responses to the installer queries:

New iraf root directory: /iraf/iraf
Default root image storage directory: /iraf/imdirs
Local unix commands directory: /iraf/bin
Proceed with installation: yes
Proceed to post-install configuration stage: yes
Configure IRAF Networking on this machine: no
Create a default tapecap file: no
Do you wish to delete these unused HSI binaries: yes
Do you wish to strip the system of sources: no

For testing the installation:

$ su iraf
$ cd /iraf/iraf/local
$ mkiraf
$ /iraf/iraf/bin/cl

This should start IRAF.

However, I just realized PyRAF probably isn’t compatible with this IRAF installation, and I only use IRAF via PyRAF these days. Well, I guess I can install IRAF 2.14 and this version side by side. Worth it? No. Hahaha…

Some Physics Units Cheat Sheet

Taro — Fri, 26 Nov 2010 02:42:42 +0000

Before I forget:

In CGS:

where

Estimating Standard Deviation from N Data Points

Taro — Wed, 24 Nov 2010 01:06:20 +0000

In astronomy people often estimate a standard deviation in, say, background counts from the observed data themselves, since it is very difficult to know what true standard deviations are.

Then the question is this: How many measurements (e.g., the number of CCD pixels) do we need to estimate the true standard deviation?

I used the following code to find out empirically:

#!/usr/bin/env python2.6

import matplotlib.pyplot as plt
import numpy as np

def gauss(n, std):
    return np.random.normal(0, std, n)

def uniform(n, std):
    lim = std / 0.57735
    return np.random.uniform(-lim, +lim, n)

def poisson(n, std):
    return np.random.poisson(std * std, n)

def doeach(func, std0, nlogs, repeat=10000):
    percenterrors = []
    for nlog in nlogs:
        n = int(10**nlog)
        stds = []
        for i in range(repeat):
            values = func(n, std0)
            stds.append(np.std(values, ddof=1))

        std_in_measured = np.sqrt(((np.array(stds) - std0)**2).sum() / repeat)

        percenterror = std_in_measured / std0
        print('%6d %.3f %.3f' % (n, percenterror, 1./np.sqrt(n)))
        percenterrors.append(percenterror)
    return percenterrors

def main():
    funcs = [(gauss, 1., 'Gaussian', 'b--'),
             (poisson, 1., 'Poisson', 'r--'),
             (uniform, 1., 'Uniform', 'g--')]

    nlogs = np.linspace(0.5, 4.0, 20)

    for f, std, label, symbol in funcs:
        perrs = doeach(f, std, nlogs, repeat=10000)
        plt.loglog(10**nlogs, perrs, symbol, lw=2, label=label)

    plt.loglog(10**nlogs, 1./np.sqrt(10**nlogs), '-', lw=2, c='gray',
               label='1/sqrt(n)')
    plt.xlabel('Number of measurements')
    plt.ylabel('Std. dev. in measured std. dev. / true std. dev.')
    plt.xlim(2, 2e4)
    plt.legend()
    plt.grid(which='both')
    plt.show()

if __name__ == '__main__':
    main()

If you assume the counts are drawn from three distributions (Gaussian, Poisson, and uniform, of somewhat arbitrary means and widths) then the fractional uncertainty in the estimate of true standard deviation falls off like where is the number of measurements that go into the estimate.

So the rule of thumb is that you need measurements to achieve a fractional uncertainty better than .

Installing mpi4py on Debian Squeeze

Taro — Tue, 23 Nov 2010 04:03:24 +0000

I want to do parallel computing on my desktop to take advantage of quad core. Actually, I just need to test programs to be run on HPC clusters. It is actually easy to set up a parallel computing environment.

Install a couple packages:

$ sudo aptitude install openmpi-bin libopenmpi-dev

The development file is necessary to build mpi4py against it.

Download the mpi4py source and install as usual:

$ tar -xvzf mpi4py-*.tar.gz
$ cd mpi4py-*
$ python setup.py build
$ sudo python setup.py install

That’s it. The command mpirun is available for use.

Installing Firefox 4 Beta on Debian Squeeze

Taro — Mon, 22 Nov 2010 16:53:59 +0000

Update: I think the following note that I made a while ago is no longer applicable. Please visit:

http://mozilla.debian.net/

for a more streamlined installation via apt-get.

Caution: This note won’t work for a version >= Beta 8-1. Unfortunately the new release depends on newer versions of libraries than what Squeeze has. Installing those libraries on Squeeze might defeat the purpose of using Debian for its stability and not recommended. I think it’s best to use a precompiled binary from Mozilla directly.

Firefox 3.5 (a.k.a. Iceweasel in Debian) on Squeeze is a bit old, so I wish to install Firefox 4 (still beta) as well. I’d like to keep an option of running stable Firefox 3.5, so what I do here is to install version 4 as a separate install. Much of this note follows the information found in this article and its comments.

First, download .deb packages from a repository to a temporary directory:

$ wget http://mozilla.debian.net/packages/iceweasel_4.0~b7-2_amd64.deb
$ wget http://mozilla.debian.net/packages/libmozjs4d_2.0~b7-2_amd64.deb
$ wget http://mozilla.debian.net/packages/xulrunner-2.0_2.0~b7-2_amd64.deb

I’ll be installing Firefox under /usr/local/iceweasel4. Do:

$ sudo aptitude install libffi5 libevent-1.4-2
$ sudo dpkg –i libmozjs4d_2.0~b7-2_amd64.deb
$ sudo dpkg –i xulrunner-2.0_2.0~b7-2_amd64.deb
$ sudo mkdir /usr/local/iceweasel4
$ sudo dpkg-deb -x iceweasel_4.0~b7-2_amd64.deb /usr/local/iceweasel4/
$ sudo ln -s /usr/lib/xulrunner-2.0 /usr/local/iceweasel4/usr/lib/

That’s it. The application launch script is located at /usr/local/iceweasel4/usr/bin/iceweasel. I create a symlink in /usr/local/bin so that it becomes available system-wide:

$ sudo ln -s /usr/local/iceweasel4/usr/bin/iceweasel /usr/local/bin/iceweasel4

This way I can launch it via command iceweasel4 from shell. It probably is a good idea to launch Firefox for the first time as

$ iceweasel4 -ProfileManager

to create a separate user profile for the beta. Otherwise the profile needs to be converted each time you switch between version 3.5 and 4 beta.

Making Incompatible Plugins to Bypass Compatibility Test

The biggest advantage of Firefox is plugins (a major reason that I haven’t tried Google Chrome yet). I cannot live without Delicious and Gmail Manager (among others that I don’t bother to mention), but most have not been updated to work with version 4 beta.

Some plugins are not tested on the beta, but some features actually work. In order to bypass version compatibility checks upon installing plugins, visit about:config in the browser, and right-click somewhere the preference list, New -> Boolean, and add a new preference with the name “extensions.checkCompatibility.4.0b” and set the boolean value to “false”. After relaunching, the plugin version checks will be bypassed.

Gmail Manager. The official release does not work, but somebody posted a modified version which works. Download the .xpi file, and open the file with Firefox to install the plugin. I’m not sure of its authenticity though. Use at your own risk…

Delicious Bookmarks. The official release works partially. What I lose on the beta is the Delicious toolbar in which I can make frequently used tags available, but the bookmarks are all available from the Firefox toolbar. I can live with it for now.

Update History

December 24, 2010 – Added a caution for beta 8-1.
April 3, 2011 – Added a note to obsolete this document.

Installing Sage (Math Library) on Debian Squeeze

Taro — Mon, 22 Nov 2010 05:32:46 +0000

First, download the source to some directory (here /usr/local/sage) from one of the official repositories (here it’s at U. of Washington, just for an illustration). Move to the directory and then

$ wget -c http://boxen.math.washington.edu/sage/src/sage-4.6.tar
$ sudo aptitude install gcc g++ gfortran make m4 perl binutils tar
$ tar -xvf sage-4.6.tar
$ sudo mv sage-4.6 /usr/local
$ sudo ln -s /usr/local/sage-4.6 /usr/local/sage
$ cd /usr/local/sage-4.6

If you have a multiprocessor machine,

$ export MAKE="make -j4"

in which I used “4″ because I have a quad core machine. Then

$ make

This is it.

Installing the Python package mpi4py

Go to the Sage installation directory, run Sage:

$ ./sage
...
sage: optional_packages()
...
sage: install_package('openmpi-XXX')
sage: install_package('mpi4py-YYY')

I need to change the versions XXX and YYY of Open MPI and mpi4py depending on the ones found in the list obtained by optional_package().

That’s it.

Setting Time and Time Zone in Debian on a Dual Boot Sytem with Windows

Taro — Mon, 22 Nov 2010 02:02:39 +0000

I noticed that my time is way off on my Debian box. I’m dual booting one of my desktop with Windows XP, so I have to set the CMOS clock to the local time as opposed to the UTC time that’s preferred if the computer purely runs on Linux. If the CMOS clock is not set in UTC, you need to tell Debian by modifying one of the lines in /etc/default/rcS:

... some other stuff ...
UTC=no
... some other stuff ...

This way, Debian can deal with dual boot with an OS like Windows. Note that this won’t work if the real time clock is not supported by your kernel. Make sure that the kernel options RTC_CLASS, RTC_INTF_DEV, and RTC_DRV_CMOS are enabled if for some reason you customized your kernel. (It turned out that my problem was that I inadvertently disabled these options when I last customized kernel…)

To set time zone:

$ sudo dpkg-reconfigure tzdata

To set date and time (to 9:08:52 pm Nov 21, 2011):

$ sudo date -s 2011-11-21
$ sudo date -s 21:08:52

This should be it.

Linux Image Manipulation Cheat Sheet

Taro — Sun, 21 Nov 2010 22:41:35 +0000

This will be my cheat sheet for doing image manipulation on Linux.

Scale Images

ImageMagick has a command mogrify to do this:

$ mogrify -scale 50% *.jpg

This will scale all .jpg images to half the original size.

Create a GIF Animation

$ convert -delay 100 -loop 0 *.png anim.gif

More to be added…

Installing F.lux on Debian Squeeze

Taro — Sat, 20 Nov 2010 19:28:48 +0000

In another feeble attempt to become a “morning person,” I’ve decided to follow the usual advice that I should not be looking at bright LCD screen at night to help myself going into the “sleep mode.”

Hence F.lux.

I want it to launch automatically at the start of a KDE session.

First of all, check the geographical coordinates of your location, perhaps using a search engine like this one. In my case (Halifax, Canada), the latitude and longitude is 44.65 and -63.6 (in degrees).

Download the Linux binary (i.e., the command-line version) from here. As of now, this is just a binary and no source codes. So no compiling is necessary. Do as:

$ wget http://secure.herf.org/flux/xflux.tgz
$ tar -xvzf xflux.tgz
$ sudo cp xflux /usr/local/bin
$ sudo chmod 755 /usr/local/bin/xflux

Now the command xflux is available system-wide.

To make it run at the KDE starup, create a shell script (called here xflux.sh) like this one:

#!/bin/sh
xflux -l 44.65 -g -63.6

Either place this script manually under ~/.kde/Autostart, or go to KDE menu -> Computer -> System Settings -> Advanced -> Autostart, and add the xflux.sh script there.

If for some reason you need to stop this program, do:

$ kill -9 `pgrep xflux`

Running F.lux on 64-bit Debian Squeeze

I may have problem getting xflux to run at all on a 64-bit machine, possibly seeing a “xflux: No such file or directory” error. The xflux binary is built against 32-bit libraries:

$ file xflux
xflux: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
dynamically linked (uses shared libs), for GNU/Linux 2.6.15, not stripped

so if I haven’t done so, I need to install 32-bit environment:

$ sudo aptitude install ia32-libs

After this, I should be able to run xflux without problems.

Installing Exim4 on Debian Squeeze

Taro — Sat, 20 Nov 2010 18:19:01 +0000

My goal is simply to make it possible to send a simple email message from command line via mail, for example to send an email notice when a command-line job finishes, so I do not care about setting up a full-blown mail server. If you care about such a thing, you’d better look elsewhere.

The relevant information about the environment: My host is connected to the Internet via DHCP through an ISP. I’ll use the SMTP server of the ISP. (Apparently there is a way to use Gmail servers as well.)

First, install Exim4 if not already and configure it:

$ sudo aptitude install exim4
$ sudo dpkg-reconfigure exim4-config

I get a series of queries to which I respond as follows:

General type of mail configuration: mail sent by smarthost; received via SMTP or fetchmail
System mail name: (apparently this domain name needs to be valid in a sense that my ISP can “resolve” it. If unresolved, the message does not get sent properly; can see this in a log at /var/log/exim4/mainlog. So I use something like “school.edu” so that when username is prepended, it becomes a valid, existing email address of mine.)
IP-addresses to listen on for incoming SMTP connections: 127.0.0.1
Other destinations for which mail is accepted: localhost
Machines to relay mail for: (blank)
IP address or host name of the outgoing smarthost: smtp.someone.com (check what SMTP server is available with the ISP)
Hide local mail name in outgoing mail: No
Keep number of DNS-queries minimal (Dial-on-Demand): No
Delivery method for local mail: mbox format in /var/mail/
Split configuration into small files: No

At this point exim4 should be restarted. Try sending a test message:

$ echo "test message" | mail -s "test message" myusername@school.edu

(The email address should be changed to yours of course!) To check the log, do:

$ sudo tail /var/log/exim4/mainlog

If there is an error, I should be able to catch it in the log.

I expect the configuration will be more challenging if the SMTP server requires authentication, for example. But for now, it’s working for me.

Installing MacTeX on OS X Snow Leopard

Taro — Thu, 18 Nov 2010 20:21:40 +0000

I was looking for TeX Live distribution for OS X, and MacTeX appears to be the one.

The installation is very easy. Just download a mpkg.zip file, decompress, and click on .mpkg to start the usual OS X install procedure. After the installation, all the familiar LaTeX shell commands will be available. Pretty much the same as on Linux.

A good news for people in astrophysics is that the AASTeX and emulateapj class files are installed by default. The ApJ font style file apjfonts.sty, however, is not. It is easy to install this via command line:

$ wget http://hea-www.harvard.edu/~alexey/emulateapj/apjfonts.sty
$ sudo mv apjfonts.sty /usr/local/texlive/2010/texmf-dist/tex/latex/emulateapj
$ sudo texconfig rehash

That’s it.

Creating an Installation USB Memory Stick for Debian Squeeze (AMD64)

Taro — Wed, 17 Nov 2010 19:35:44 +0000

I think this is mostly a generic procedure for Debian, not specific to Squeeze, but here goes.

If the USB stick is not blank, be careful to create a new bootable partition with cfdisk command in the following procedure, and carefully note the correct path to the partition (/dev/sdb1 in this article).

The line with getting *-netinst.iso may fail once Squeeze goes stable; the particular URL is good only for the beta 2 of Debian installer. The important thing is to get a proper netinst ISO image, which you can find here for non-stable Debian versions. (If you are going to install a stable release, get a proper netinst ISO, but also make sure to get a corresponding version of hd-media/boot.img.gz from the stable repository.)

I assume that you are a superuser. Here is the procedure:

# aptitude install wget mbr
# fdisk -l    # Find the USB stick's device path; here I assume /dev/sdb.
# cfdisk -z /dev/sdb
# wget http://ftp.debian.org/debian/dists/squeeze/main/installer-amd64/current/images/hd-media/boot.img.gz
# wget http://cdimage.debian.org/cdimage/squeeze_di_beta2/amd64/iso-cd/debian-squeeze-di-beta2-amd64-netinst.iso
# zcat boot.img.gz > /dev/sdb1
# mkdir -p /tmp/usbboot
# mount /dev/sdb1 /tmp/usbboot
# cp debian-squeeze-di-beta2-amd64-netinst.iso /tmp/usbboot
# umount /tmp/usbboot
# rmdir /tmp/usbboot
# install-mbr /dev/sdb

That’s it. Boot off the USB stick if the BIOS of your computer allows.

Update History

December 12, 2010 — Changed paths for the beta 2 release.

Looping a YouTube Video

Taro — Fri, 12 Nov 2010 20:42:53 +0000

(Borrowed from this posting, keeping here for my own reference.)

Each YouTube video has a key assigned to it. For example, in this URL

http://www.youtube.com/watch?v=sgjMK2sWgsE

the bold portion is the key. Once you know this, modify the URL as

http://www.youtube.com/v/sgjMK2sWgsE&loop=1&autoplay=1

This will keep the video playing.

Installing Xpdf from Source on Linux

Taro — Mon, 08 Nov 2010 16:02:44 +0000

Some Linux machines that I use at work have older versions of software and Xpdf, at version 3.00, is one of them. The rendering is ugly and it spews out a lot of errors when I’m using on the shell. So I decided to install the newest version manually from source. I cannot be a superuser on those Linux boxes, so I’m doing a local (not system-wide) installation under my home directory, e.g., /home/johndoe.

First, download the libraries and Xpdf to /home/johndoe/src:

Then follow this:

$ cd /home/johndoe/src

$ tar -xvzf t1lib-5.1.0.tar.gz
$ cd t1lib-5.1.0
$ ./configure --prefix=/home/johndoe
$ make; make install

$ cd ..
$ tar -xvjf freetype-2.3.1.tar.bz2
$ cd freetype-2.3.1
$ ./configure --prefix=/home/johndoe
$ make; make install

$ cd ..
$ tar -xvzf xpdf-3.02.tar.gz
$ cd xpdf-3.02
$ export LD_RUN_PATH=/home/johndoe/lib
$ ./configure --with-t1-library=/home/johndoe/lib \
              --with-t1-includes=/home/johndoe/include \
              --with-freetype2-library=/home/johndoe/lib \
              --with-freetype2-includes=/home/johndoe/include \
              --with-Xm-library=/usr/X11R1/lib \
              --with-Xm-includes=/usr/X11R6/include \
              --prefix=/home/johndoe
$ make; make install

That’s it. The executable should now be installed at /home/johndoe/bin/xpdf.

Using LD_RUN_PATH to Set the Run-time Library Search Path

Taro — Mon, 08 Nov 2010 04:46:21 +0000

Sometimes you build a program from source which requires linking to a library located in some nonstandard directory (say, /home/johndoe/lib). Quite often this results in a disappointment as running the executable ends in an error looking like this:

XXX: error while loading shared libraries: YYY.so: cannot open shread object file: No such file or directory

The easiest way to avoid this is to set the environment variable LD_RUN_PATH to point to /home/johndoe/lib. Hence the standard build procedure using make goes something like this:

$ export LD_RUN_PATH=/home/johndoe/lib
$ ./configure --with-YYY-library=/home/johndoe/lib \
    --with-YYY-includes=/home/johndoe/include \
    --prefix=/home/johndoe
$ make
$ make install

(This way you do not need to mess with LD_LIBRARY_PATH, which presumably is less reliable.)

Uncertainty in Flux and Magnitude

Taro — Wed, 03 Nov 2010 17:52:06 +0000

I keep forgetting this sort of simple algebra (aging is no fun), so here’s a note.

Define magnitude to be related to flux as follows:

where is the zero point flux which defines the magnitude scale. Let be the magnitude uncertainty, which is related to the fractional uncertainty in flux, . We wish to find how is related to . This means

Solving for we get

In summary, we have the following relationships:

When or , we may expand the log function about 1 and retain only the first order term:

for . In this limit we can write . Hence

This is why an uncertainty in magnitude, when small, is often an adequate approximation to the fractional uncertainty in flux.

Installing Dropbox without Gnome on Debian Squeeze (AMD64)

Taro — Sun, 31 Oct 2010 19:31:41 +0000

I’m finally trying out Dropbox but didn’t like the way their Linux client depends on Gnome (I use KDE). Following this procedure, it is possible to use Dropbox without Gnome and even on the shell. Here’s my version of the procedure.

First, download to a temporary directory the Linux client from here:

x86 (for 32-bit Linux)
x86_64 (for AMD64 Linux)

The downloaded file is a gzipped tarball which you can explode via

$ tar -xvzf dropbox-lnx*.tar.gz

This will generate a hidden directory named .dropbox-dist in the same directory. Move that directory

$ sudo mv .dropbox-dist /usr/local/dropbox-0.7.110
$ sudo ln -s /usr/local/dropbox-0.7.110 /usr/local/dropbox

(-0.7.110 is a version number of Dropbox I’m using.) I move it under /usr/local just because I like to keep local applications there as much as possible.

The daemon that needs to be running in background is /usr/local/dropbox/dropboxd. If I wish the daemon to run every time I use KDE, let it start automatically by creating a symlink to it under ~/.kde/Autostart:

$ cd ~/.kde/Autostart
$ ln -s /usr/local/dropbox/dropboxd .

This is it. The client icon is accessible in the KDE system tray. The synced files are placed under ~/Dropbox.

Linux Admin Cheat Sheet

Taro — Sun, 24 Oct 2010 00:47:58 +0000

This isn’t intended to be comprehensive at all. I’ll just be listing commands that I sometimes but not often use to do miscellaneous administrative tasks.

Finding CPU Temperature

$ acpi -t

Passwordless SSH

Between trusted hosts, I want to allow passwodless access via ssh. To do this, generate RSA key pair:

$ ssh-keygen

When prompted, accept the defaults unless things need to be changed. Copy the public key just generated in ~/.ssh/id_rsa.pub on the local host. Login to a remote host, and append the key to ~/.ssh/authorized_keys. Next time I login to that host, I will not be prompted for password.

Linux Equivalent of `open` on Mac OS X

One command I like on Mac OS X is open, which open a file with an associated default application. There is a command called xdg-open which does the same thing. Make an alias to it so it acts like the one on Mac:

alias open='xdg-open'

Put this in ~/.bash_aliases.

Find the UUID of Devices

$ sudo blkid

Securely Erase Hard Drive

# fdisk -l                     # check hard drives file systems
# shred -vfz -n 20 /dev/sdb    # secure erase /dev/sdb

Redirect Output to Multiple Processes

Use “tee >(…)”:

$ ls -A | tee >(grep ^[.] > hidden-files) >(grep -v ^[.] > normal-files)

This example comes from this article.

Repeating Same Operation on Many Files

$ for i in *.txt ; do echo $i ; done

Recursively Do Something on Files/Directories

Find all files under a root path:

$ find /path/to/root -print

Change all directory permissions to 755:

$ find /path/to/root -type d -print -exec chmod 755 {} \;

Change all file permissions to 644:

$ find /path/to/root -type f -print -exec chmod 644 {} \;

Change all files and directories with 777 permissions to 755 permissions:

$ find /path/to/root -perm 777 -print -exec chmod 755 {} \;

Show the n-th (e.g., 9th) Line of a Text File

$ sed -n '9p' somefile

Send an Email Message

Provided the mail service is set up properly, a simple message can be sent quickly on command line.

$ echo "message body" | mail -s "subject line" your@mail.com

This is useful when I want to get an email notification when a very time-consuming job finishes. For a time consuming job, sometimes it’s desirable to email the output from the job. I can redirect both standard output and standard error to the email message body by:

$ someprogram 2>&1 | mail -s "job is done" your@mail.com

Keep a Process Running without Hangups

When starting a process on a remote host (e.g., over ssh), that process is killed when you logout of the session. To keep it running there, use nohup:

$ nohup somecommand &

Kill All Processes with a Given Name

$ kill -9 `pgrep someprogramname`

This will kill all the processes with the name someprogramname; see the man page for pgrep for more info.

Change Run-Level Configuration for Init Scripts

$ sudo sysv-rc-conf

This will provide an easy-to-user user interface to configure init scripts. Useful when I want to stop KDM from running at boot, for example.

Find Out How Long the System Has Been Running

$ uptime

Search for Filename in Debian Packages

The Debian packaging system is obviously useful, but I often struggle to find to which package a particular command belongs. I can search for by filename:

$ sudo apt-file search somefilename

To do this, the command apt-file needs to be installed if not already:

$ sudo aptitude install apt-file
$ sudo apt-file update

Clear Apt Package Cache

The /var partition may quickly fill up with downloaded .deb packages and if the partition size is small, occasional cleanup will be necessary. If I wish to just remove .deb files that are no longer relevant, do

$ sudo aptitude autoclean

If I need to remove all .deb files, then

$ sudo aptitude clean

The latter is okay as long as I don’t mind downloading many of them again when I need to update the system via the update-upgrade sequence, for example.

View the CPU Information

$ cat /proc/cpuinfo

Check the Disk Usage by Directories

$ du -h --max-depth=1

The resulting list may not to be sorted. If I want to sort this by name, do:

$ du -h --max-depth=1 | sort --key=2 --ignore-case

Bash Scripting

Get the Absolute Path to the Script’s Current Directory

DIR="`dirname $BASH_SOURCE`"
ABSDIR="`cd $DIR; pwd`"

Get the Host Name

HOST="`hostname -f`"

Parse Space-Delimited String

SEED='a b c d'
set -- $SEED
echo $1
echo $2
echo $3
echo $4

More to come.

Installing Mendeley on Debian Squeeze (AMD64)

Taro — Sat, 23 Oct 2010 01:49:38 +0000

Mendeley does have a Debian package which you can simply apt-get (yay), but it is only available for Lenny, the current stable release. Here I outline the system-wide installation procedure using their tarball.

First, download the tarball from Mendeley to some temporary directory. The file should be named like mendeleydesktop-0.9.8.1-linux-x86_64.tar.bz2.

$ umask 022
$ tar -xvjf mendeleydesktop-0.9.8.1-linux-x86_64.tar.bz2
$ sudo mv mendeleydesktop-0.9.8.1-linux-x86_64 /usr/local/mendeley-0.9.8.1
$ sudo ln -s /usr/local/mendeley-0.9.8.1 /usr/local/mendeley
$ cd /usr/local/bin
$ sudo ln -s /usr/local/mendeley/bin/mendeleydesktop .

This will install all the files under /usr/local/mendeley. The command mendeleydesktop will be available system wide. However, it may be more convenient to add an entry to your application launcher.

Addendum: For some reason my Mendeley installation has had a problem on launch for a while. It always quit after asking me to “repaire” the database, etc. After looking at the log (which is in ~/.local/share/data/Mendeley Ltd./Mendeley Desktop), I leaned that MendeleyDesktop had a problem dealing with SQL database, which is located somewhere deep under /var (I don’t know for sure). It turned out that my /var partition was full. After doing apt-get clean, things are working fine. So the moral of the story is that I should clean up my system more often...



Python Utility for the UVBLUE Stellar Spectrum Library
Taro — Fri, 22 Oct 2010 21:46:37 +0000
I have a need to use spectra from UVBLUE, but they only supply IDL programs (who needs these any more?) to deal with the data.  I want to use the spectra in Python, so I ported their program.  Download it from here.



Installing Epson Perfection V30 on Debian Squeeze (AMD64)
Taro — Thu, 21 Oct 2010 23:57:15 +0000
First, download the following .deb files from here:
esci-interpreter-gt-f720_0.0.1-2_amd64.deb
iscan_2.26.0-3.ltdl7_amd64.deb
iscan-data_1.6.0-1_all.deb
(To get to the download page, you choose “Perfection V30,” “Debian” for distribution, and “others” for version; the exact version numbers may be different.)
Install the SANE library and related packages, and then use the .deb files you just downloaded to install Image Scan ! for Linux:
$ sudo aptitude install libsane xsltproc
$ sudo dpkg -i esci-interpreter-gt-f720_0.0.1-2_amd64.deb iscan_2.26.0-3.ltdl7_amd64.deb iscan-data_1.6.0-1_all.deb
That’s it.  Image Scan! for Linux can be launched from the Graphics menu.  You can also use it from GIMP.  Very simple installation!  I’ve always had good experiences with Epson scanners!
Actually, I find Image Scan! quite lacking in features.  The official SANE page appears to indicate that Epson Perfection V30 is not fully supported by SANE, but after following the above procedure and installed XSANE:
$ sudo aptitude install xsane
the scanner works perfectly.



Installing Bruzual & Charlot 2003 (GALAXEV) on Mac OS X Snow Leopard
Taro — Tue, 19 Oct 2010 18:44:30 +0000
You can follow the same procedure for Linux, except that csh probably is already installed at /bin/csh and gfortran needs to be installed differently (see this post).



Installing Bruzual & Charlot 2003 (GALAXEV) on Debian Squeeze
Taro — Tue, 19 Oct 2010 18:20:03 +0000
The installation procedure is confusing only if you are not using CSH (I use BASH) and if g77 is not available.  Otherwise you can simply follow the installation procedure outlined in the official documentation file (i.e., bc03/doc/bc03.ps.gz).
First, you definitely need csh and a FORTRAN compiler:

$ sudo apt-get install csh gfortran

Download from the B&C 2003 website the following tarballs (assuming here they are saved under /usr/local/SED):

bc03.doc.tar.gz
bc03.models.padova_1994_chabrier_imf.tar.gz
bc03.models.padova_1994_salpeter_imf.tar.gz
bc03.src.tar.gz
bc03.templates.tar.gz

Uncompress and untar all these at once by

for i in *.tar.gz; do tar -xvzf $i; done

This will create the directory bc03 under /usr/local/SED and all the subdirectories (bc03/doc, bc03/models, bc03/src, bc03/templates) should be filled with files.
Unfortunately the setup file only exists for csh (bc03/src/.bc_cshrc), so you need to create one for yourself.  Save the following to a file at bc03/src/.bc_bash:

export FILTERS="$bc03/FILTERBIN.RES"
export A0VSED="$bc03/A0V_KURUCZ_92.SED"
export RF_COLORS_ARRAYS="$bc03/RF_COLORS.filters"
####################################################
alias  add='csh $bc03/add_bursts.sh'
alias  csp='csh $bc03/csp_galaxev.sh'
alias  vdisp='csh $bc03/vel_disp.sh'
alias  cmev='csh $bc03/cm_evolution.sh'
alias  dgr='csh $bc03/downgrade_resolution.sh'
alias  gpl='$bc03/galaxevpl'
alias  zmag='$bc03/zmag'

(This file is actually provided by default in C&B 2007, which you should be using instead of the 2003 version.)  Furthermore, you need to modify a line in bc03/src/Makefile to use gfortran instead of g77:

#FC =  g77 -ffixed-line-length-132  # comment out this line and add:
FC = gfortran -ffixed-line-length-none

Then compile the programs:

$ bc03=/usr/local/SED/bc03/src
$ cd $bc03
$ . .bc_bash
$ make all

Add the following lines to ~/.bashrc and all the commands for GALAXEV will be available (as aliases):

export bc03=/usr/local/SED/bc03/src
. $bc03/.bc_bash

That’s it.



Installing NeverNote on Debian Squeeze (AMD64)
Taro — Thu, 14 Oct 2010 14:40:21 +0000
I still don’t understand what all the fuss is about on Evernote (not to mention their aversion to Linux).  However, some kind of note taking utility and having all the related data in the cloud still relives me from the nuisance of syncing files across different computers.
You never know what people will do with those data, but who gives a fuck unless you have a terror plot or something…
Dale L. suggested NeverNote, a Linux client for Evernote, and this looks promising.  I haven’t used the application thoroughly yet, but this certainly is a much improvement over the web client.
Here’s what I did to install it (version 0.94) on my Debian notebook.
Download nevernote-0.94_amd64.deb from the Sourceforge repository, and then

$ sudo dpkg -i nevernote-0.94_amd64.deb

For me, this is all I needed to do.
All the files, including the startup script, are installed under /usr/share/nevernote. I can launch the client by explicitly running the script:

$ /usr/share/nevernote/nevernote.sh

Perhaps it makes more sense to have a symlink to this script:

$ sudo ln -s /usr/share/nevernote/nevernote.sh /usr/local/bin/nevernote


If I put such a symlink, the file /usr/share/nevernote/nevernote.sh needs to be modified so that the variable NEVERNOTE within the startup script points explicitly to /usr/share/nevernote:

NEVERNOTE=/usr/share/nevernote


Otherwise the application cannot properly link to libraries.



Recipe: Name-dake (なめ筍)
Taro — Sun, 15 Aug 2010 00:50:13 +0000
What you need:

Enoki mushroom 180g
Sake 10 ml
Soy sauce 15 ml
Mirin 15 ml

Put enoki mushroom and sake into a pot.  Shimmer at low to mid heat for a few minutes.  Put soy sauce and mirin a bit later and continue simmering till enoki becomes lightly brown and the sauce becomes thick.  The proportion of ingredients 2, 3, and 4 can be changed according to your taste.



Testing if a Point is Inside a Polygon in Python
Taro — Thu, 24 Jun 2010 17:29:33 +0000
Finally got around to find this out by Googling.  It’s a useful function so I reproduce it here for copy & paste:

def inside_polygon(x, y, points):
    """
    Return True if a coordinate (x, y) is inside a polygon defined by
    a list of verticies [(x1, y1), (x2, x2), ... , (xN, yN)].

    Reference: http://www.ariel.com.au/a/python-point-int-poly.html
    """
    n = len(points)
    inside = False
    p1x, p1y = points[0]
    for i in range(1, n + 1):
        p2x, p2y = points[i % n]
        if y > min(p1y, p2y):
            if y <= max(p1y, p2y):
                if x <= max(p1x, p2x):
                    if p1y != p2y:
                        xinters = (y - p1y) * (p2x - p1x) / (p2y - p1y) + p1x
                    if p1x == p2x or x <= xinters:
                        inside = not inside
        p1x, p1y = p2x, p2y
    return inside





Installing STSCI_PYTHON on Debian Squeeze
Taro — Tue, 15 Jun 2010 02:04:22 +0000
The version I use is 2.10.
First, install IRAF, tables, and stsdas following this article.

$ sudo aptitude install python-tk python-numpy python-pmw python-urwid
$ sudo aptitude install libreadline6-dev python-matplotlib ipython
$ wget http://stsdas.stsci.edu/download/stsci_python_2.10/stsci_python_2.10.tar.gz
$ tar -xvzf stsci_python_2.10.tar.gz
$ cd stsci_python_2.10
$ python setup.py build
$ sudo python setup.py install --home=/usr/local/stsci_python-2.10
$ sudo ln -s /usr/local/stsci_python-2.10 /usr/local/stsci_python
$ cd stscidocs
$ su
# export PATH=/usr/local/stsci_python/bin:"$PATH"
# export PYTHONPATH=/usr/local/stsci_python/lib/python:"$PYTHONPATH"
# python setup.py install --home=/usr/local/stsci_python-2.10

Add the following lines to ~/.bash_profile:

export PATH=/usr/local/stsci_python/bin:"$PATH"
export PYTHONPATH=/usr/local/stsci_python/lib/python:"$PYTHONPATH"

# These two lines are necessary if IRAF was not installed at /usr/local/bin
export iraf=/iraf/iraf  # where IRAF is installed
export IRAFARCH=linux

That’s it.
Update History
November 27, 2010 – Updated for version 2.10.



Installing IRAF on Debian Squeeze
Taro — Mon, 14 Jun 2010 19:45:57 +0000
IRAF doesn’t have a standard installation procedure using make tools, so it is no fun to install in any system.  One solution is to go for Scisoft, but I don’t necessarily want to install a whole bunch of software that I never intend to use in a small-ish SSD disk drive that I now have with Lenovo T410s.
Installing IRAF
The IRAF version here is 2.14.1.  A 64-bit ready IRAF might be coming soon though.
I’m going to install this under /usr/local/iraf-2.14.1, which is eventually symlinked by /iraf.  I now hate overwriting locally installed software.
First prepare my machine for IRAF installation:

$ sudo apt-get install csh xutils-dev ncompress
$ sudo adduser iraf
  ... add IRAF admin user to the system ...
$ sudo nano /etc/passwd
  ... change /bin/bash to /bin/csh for user iraf ...
$ sudo mkdir /usr/local/iraf-2.14.1
$ sudo ln -s /usr/local/iraf-2.14.1 /iraf
$ cd /iraf/iraf
$ sudo chown iraf .
$ sudo chgrp iraf .
$ su iraf
$ mkdir bin extern imdirs iraf irafbin x11iraf
$ mkdir irafbin/bin.linux
$ mkdir irafbin/noao.bin.linux
$ cd /iraf/iraf
$ wget http://iraf.noao.edu/iraf/ftp/iraf/v214/PCIX/as.pcix.gen.gz
$ gunzip -c as.pcix.gen.gz | tar xvf -
$ cd /iraf/irafbin/bin.linux
$ wget http://iraf.noao.edu/iraf/ftp/iraf/v214/PCIX/ib.lnux.x86.gz
$ gunzip -c ib.lnux.x86.gz | tar xvf -
$ cd /iraf/irafbin/noao.bin.linux
$ wget http://iraf.noao.edu/iraf/ftp/iraf/v214/PCIX/nb.lnux.x86.gz
$ gunzip -c nb.lnux.x86.gz | tar xvf -
$ setenv iraf /iraf/iraf
$ cd $iraf/unix/hlib
$ source irafuser.csh
$ ./install -n  # Just go thru the motion to see what will be installed and set

Now we are ready to make changes to the system so that IRAF can be launched.  When actually running install script, I tell it the local bin directory is at /iraf/bin (i.e., /usr/local/iraf-2.14.1/bin).  By default startup scripts (like CL) are installed under /usr/local/bin so that anyone on the system can use, but I now have a policy to avoid system-wide installation unless I am pretty sure that I will not be updating the software in future.  When you are ready,

$ su
# ./install
# exit

Now IRAF should be ready.
Installing xgterm v2.0 Beta
There are better alternatives out there but this is what still is a necessary terminal for pure IRAF.

$ sudo apt-get install xutils-dev byacc flex bison libxaw7-dev
$ su iraf
$ cd /iraf/x11iraf
$ wget http://iraf.noao.edu/x11iraf/x11iraf-v2.0BETA-src.tar.gz
$ gunzip -c x11* | tar xvf -

I need to modify a couple files to get this thing to compile properly (see this thread):
In obm/ObmW/Imakefile, the line containing EXTRA_DEFINES should read

EXTRA_DEFINES = -D_NO_PROTO -DUSE_STDARG

and in xgterm/button.c, comment out the line reading char *malloc():

/* extern char *malloc(); */

The proceed to compile and install:

$ cd /iraf/x11iraf
$ xmkmf
$ make World
$ su
# ./install

Again for the last install script, I specify /iraf/bin as the binary installation directory.
Testing Installation
Next step is to test if I can use IRAF as a regular user.  Login as myself, and do:

$ cd ~
$ mkdir iraf
$ cd iraf
$ mkiraf

(I create ~/iraf just to avoid cluttering my home directory.)  Choose xgterm as your terminal.
With this installation, I actually need to add /iraf/bin to the PATH environment variable to use IRAF commands on a regular shell:

export PATH=/iraf/bin:"$PATH"

Add this to ~/.bash_profile or ~/.bashrc so that you don’t have to do this on each login.  Or I can even invoke CL as /iraf/bin/cl.  Whatever I prefer.
Installing STSDAS and TABLES
Since these may be required for stsci_python, here is the procedure.  Note that I cannot compile them from source due to incompatibility issues with 64-bit architecture.  Assuming I already logged in as user iraf,

$ mkdir /iraf/extern/stsdas
$ cd /iraf/extern/stsdas
$ wget http://stsdas.stsci.edu/download/stsdas_3.11/stsdas3.11.tar.gz
$ gunzip -c stsdas* | tar xvf -
$ mkpkg linux
$ cd bin.linux
$ wget http://stsdas.stsci.edu/download/stsdas_3.11/stsdas3.11.bin.redhat.tar.gz
$ gunzip -c stsdas* | tar xvf -
$ python -m compileall -q .

For tables,

$ mkdir /iraf/extern/tables
$ cd /iraf/extern/tables
$ wget http://stsdas.stsci.edu/download/tables_3.11/tables3.11.tar.gz
$ gunzip -c tables* | tar xvf -
$ mkpkg linux
$ cd bin.linux
$ wget http://stsdas.stsci.edu/download/tables_3.11/tables3.11.bin.redhat.tar.gz
$ gunzip -c tables* | tar xvf -

Add entries for tables and stsdas in /iraf/iraf/unix/hlib/extern.pkg.  An example file looks like this:

# External (non core-system) packages.  To install a new package, add the
# two statements to define the package root directory and package task,
# then add the package helpdb to the `helpdb' list.

reset   noao            = iraf$noao/
task    noao.pkg        = noao$noao.cl

reset   tables          = iraf$../extern/tables/
task    tables.pkg      = tables$tables.cl

reset   mscred          = iraf$../extern/mscred/
task    mscred.pkg      = mscred$mscred.cl

reset   stsdas          = iraf$../extern/stsdas/
task    stsdas.pkg      = stsdas$stsdas.cl

reset   rvsao           = iraf$../extern/rvsao/rvsao-2.5.0/
task    rvsao.pkg       = rvsao$rvsao.cl

reset   xdimsum         = iraf$../extern/xdimsum/
task    xdimsum.pkg     = xdimsum$xdimsum.cl

reset   fuzzy           = iraf$../extern/fuzzy/
task    fuzzy.pkg       = fuzzy$fuzzy.cl

reset   fitsutil        = iraf$../extern/fitsutil/
task    fitsutil.pkg    = fitsutil$fitsutil.cl

reset   gmisc           = iraf$../extern/gmisc/
task    gmisc.pkg       = gmisc$gmisc.cl

reset   gemini          = iraf$../extern/gemini/
task    gemini.pkg      = gemini$gemini.cl

reset   helpdb          = "lib$helpdb.mip\
 ,noao$lib/helpdb.mip\
 ,tables$lib/helpdb.mip\
 ,stsdas$lib/helpdb.mip\
 ,mscred$lib/helpdb.mip\
 ,rvsao$lib/helpdb.mip\
 ,fitsutil$lib/helpdb.mip\
 ,gemini$lib/helpdb.mip\
 ,xdimsum$lib/helpdb.mip\
 "

References
IRAF: iraf-64-bit_step_by_step_installation – CAL Network – Trac
Installing IRAF on Ubuntu Edgy amd64 | MJHutchinson.com



Installing Evernote for Windows on Debian Squeeze
Taro — Sun, 13 Jun 2010 23:40:28 +0000
No Linux version!  Lame!
First, install Wine as usual.
Then download version 3.1 of Evernote for Windows (no, version 3.5 does not install due to its .Net requirement), and run the installer:

$ wine Evernote_3.1.0.1225.exe

That’s it.



Installing KeePass on Debian Squeeze
Taro — Sun, 13 Jun 2010 22:50:49 +0000

$ sudo aptitude install keepassx


This is my favorite username & password manager.



PDF Viewing on Debian Squeeze
Taro — Sun, 13 Jun 2010 22:44:13 +0000
I find Okular to be a superb PDF reader.  Yes, you can even annotate directly onto PDF documents!

$ sudo apt-get install okular poppler-data

The last module is necessary to view Japanese PDF documents.



Installing Skype on Debian Squeeze
Taro — Sun, 13 Jun 2010 00:27:08 +0000
To install Skype under /usr/local on Squeeze (AMD64), do:

$ sudo apt-get install ia32-libs ia32-libs-gtk
... download Skype source (static version) ...
$ bunzip2 -c skype_static-2.1.0.81.tar.bz2 | tar xvf -
$ sudo mv skype_static-2.1.0.81 /usr/local
$ cd /usr/local/bin
$ sudo ln -s /usr/local/skype_static-2.1.0.81/skype .

The program will be accessible at /usr/local/bin/skype.



Installing Kindle for PC on Debian Squeeze
Taro — Sat, 12 Jun 2010 03:27:37 +0000
I sometimes want to run Kindle for PC on Wine.  Amazon is so lame that they don’t even bother making a native Linux version of this software!
Here is the procedure.  Move to a temporary directory and do:
$ sudo aptitude install wine lib32nss-mdns cabextract bzip2
$ wget http://www.kegel.com/wine/winetricks
$ sh winetricks corefonts
$ wget http://okomestudio.net/biboroku/wp-content/uploads/2010/06/KindleForPC-installer.exe.bz2
$ bunzip2 KindleForPC-installer.exe.bz2
$ wine KindleForPC-installer.exe
After going through the above procedure, I need to run Wine Configuration to set Kindle for PC executable to run as a Windows 98 application.  I do this under the Applications tab of the Wine configuration.  It should be self-explanatory.
The program called winetricks is necessary to install necessary MS fonts once Kindle for PC launches.
The old version of Kindle for PC installer, which I just keep for myself here, is necessary to properly install it on Wine.  Unfortunately newer versions won’t work.  Hopefully Amazon releases a new version which runs at least on Wine…
Update History
Jan 26, 2011 – Fixing typos and added a couple packages based on a user comment.



High-performance Computing Tips
Taro — Wed, 09 Jun 2010 14:33:44 +0000
A nice thing about my current institution is that it has access to high-performance computing (HPC) facility.  I get to use so-called supercomputers.  This is a cheat sheet for myself.
Reserving your job:
$ qsub -R y jobscript
Show the information about a job with id JOBID:
$ qacct -j JOBID
Change job priority:
$ qalter -js 100 JOBID
Delete a job:
$ qdel JOBID
or a array job task:
$ qdel JOBID.TASKID
Using remote shell with various settings:
$ qrsh -now no -V -cwd -l h_rt=01:00:00,h_vmem=4G,h_stack=2G,test=true bash
More to be added.



Securely Erasing Hard Disk
Taro — Wed, 02 Jun 2010 13:36:33 +0000
Like when you need to sell your old computer, sometimes you wish to wipe clean a hard disk drive in a secure manner (i.e., not just removing reference to data).
$ sudo fdisk -l

Disk /dev/sda: 80.0 GB, 80026361856 bytes
240 heads, 63 sectors/track, 10337 cylinders
Units = cylinders of 15120 * 512 = 7741440 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xbd3cc0bb

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1       10338    78149633    5  Extended
/dev/sda5               1          46      340992   83  Linux
/dev/sda6              46          52       48128   83  Linux
/dev/sda7              52         698     4881408   83  Linux
/dev/sda8             698        1086     2928640   83  Linux
/dev/sda9            1086        1861     5858304   83  Linux
/dev/sda10           1861        2830     7323648   83  Linux
/dev/sda11           9305       10338     7812096   82  Linux swap / Solaris
/dev/sda12           2830        9304    48949248   83  Linux

Partition table entries are not in disk order

Disk /dev/sdb: 262 MB, 262144000 bytes
16 heads, 32 sectors/track, 1000 cylinders
Units = cylinders of 512 * 512 = 262144 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1         999      255728    6  FAT16

$ sudo shred -vfz -n 3 /dev/sdb
Here, /dev/sdb is the disk to be erased.  Takes a long time, but that’s it.  You can change the “-n” option to repeat more is you really want to be anal.



Installing Debian Squeeze (AMD64) on Lenovo T410s
Taro — Tue, 01 Jun 2010 14:22:09 +0000
So far the only thing that I do not like about T410s is its short  battery life.  I can only get 2 – 3 hours out of “standard” usage.   Hopefully this will improve as I tweak things more.
To Begin With:  A Problem
Unfortunately I had a major issue with my T410s recently (November 17, 2010).  A few months ago, the LCD started showing a vertical band in which graphics was lost.  Fortunately, the issue seems to be fairly prevalent with T410s, which means there was a solution.  So, I just got a warranty repair done, which involved replacing the LCD and motherboard.  It took exactly two weeks for the whole process using the local Lenovo contractor (in Canada), but now the laptop is fixed.  Anyone who got T410s around June 2010 or before should be aware of this LCD issue.
As of November 26, 2010, I decided to reinstall Debian Squeeze from scratch.  So this installation note now reflects the nearly stable version of Squeeze installation procedure.  Things that did not work before now works from the beginning!
General Hardware Specifications
My Lenovo T410s is 2901-CTO.



Hardware Components
Status under Linux
Notes


Intel Core i5 M520 2.4 GHz
Works



14.1 WXGA+ TFT Display, w/ LED Backlight
Works



Intel HD Graphics (IT 5700MHD, i5-520M AMT)
Works



4 GB PC3-8500 DDR3 (2 DIMM)
Works



80 GB SSD (INTEL SSDSA1M080G2LE, 2CV102J6, max UDMA/133)
Works



Integrated Ethernet Card (Intel 82577LM Gigabit Network Connection (rev 06))
Works



Wireless Network Card (Intel Centrino Ultimate-N 6300 (rev 35))
Works
Need to download firmware to /lib/firmware.


Matsushita DVD-RAM Drive (UJ892)
Works



6 cell Prismatic Battery
Works



Integrated Sound Card (Intel 3b57 (rev 06))
Works



Integrated Speaker & Microphone
Works



Integrated Camera
Works
uvcvideo


ThinkLight
Works



Fingerprint Reader
Not tested



TrackPoint & TrackPad
Works



Keyboard
Not fully tested
Haven’t thoroughly tested special keys



Note: Currently my T410s is operating on kernel version 2.26.36.2 from kernel.org which I customized and compiled myself.  I don’t think is necessary to get the basic things working though. On the other hand, I see some problem with kernel 2.26.36.2. When I do dmesg, I get numerous “scsi host1: __pm_runtime_resume()!” errors, and sometimes T410s locks up for several seconds.  In order to alleviate the problem, I have to boot with kernel option pcie_ports=compat.
Why Debian Squeeze?
I have always liked the leanness and stability of Debian, but I admit that I have recently flirted with the idea of migrating to Ubuntu; near the end of Debian release cycle, I have always found myself building from sources various programs for which their Debian versions became obsolete.  When I tried, I also liked Ubuntu for its ease of use; I didn’t have to do whole a lot of configurations myself to get my old laptop (a Dell Latitude) working.  However, it slightly felt like using Windows on a new computer; a lot of software that I don’t care to install are bundled with Ubuntu.  In the end, I decided to stay with Debian for now, but with Squeeze, still a testing distribution (Nov. 17, 2010).
Installing Debian Squeeze
First of all, make factory recovery disks in case you need to restore factory default.  There is little reason to keep Windows 7 around, but it’s always a good thing to have a way to restore default setting.  In order to do this, launch ThinkVantage and go to Factory Recovery Disks.  Check both boot and data.  In my case, three blank DVDs were used (one for boot and two for data; it looks like the boot disk only needs to be of very small capacity).  After making the disks, it is recommended to test if they work or not by rebooting off the boot disk just created.  Mine passed the test and factory reset was completely working.  I also updated BIOS to its latest version on Windows to avoid problems later.
Debian Squeeze will soon become the stable distribution, but till that happens I fetched the installer from the testing distribution site, and made a install USB stick, following this article.  For this installation note, I use the Beta1 release of Squeeze.
Boot off the USB stick by pressing F12 to start the installation.
Installation Customization Options
Debian installation has gotten easy enough that I don’t feel the need for this any more, but just for completeness.  Obviously you should customize to your liking.  I use a graphical installer here.

Language: English
Country, territory or area: (your choice)
Keyboard layout: USA
Primary network interface: wlan0 (for wireless network installation, only WEP or manual appears working, for which you specify the wireless key and password perhaps.) or eth0 (wired network installation is often easier)
Hostname: (your choice)
Domain name: (your choice)
Root password: (your choice)
Full name for the new user: (your choice)
Username for your account: (your choice)
Choose a password for the new user: (your choice)
Select your time zone: (your choice)
Partitioning method: Manual

I’m not doing dual boot, so my partition scheme is very simple:
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda5             323M  149M  157M  49% /
/dev/sda6              46M   27M   17M  62% /boot
/dev/sda7             4.6G  138M  4.3G   4% /tmp
/dev/sda8             2.8G  1.8G  841M  69% /var
/dev/sda9             5.5G  4.4G  914M  83% /usr
/dev/sda10            6.9G  4.4G  2.2G  67% /usr/local
/dev/sda11              8G                  Linux Swap
/dev/sda12             46G   19G   26G  43% /home

Policy for handling keymaps: Select keymap from arch list -> qwerty -> US american -> Standard -> Standard
Debian archive mirror country: (your choice)
Participate in the package usage survey: No
Choose software to install: (I want a fairly minimal install.  I only check Laptop.)
Install the GRUB boot loader to the master boot record: yes

Reboot and a very simple Debian box is ready!
Post-Install Admin Tweaks
Login as root.  First I install a few essential packages for system administration, etc.:
# aptitude install sudo ssh rsync wget wireless-tools
If I need to run 32-bit applications, it is also a good idea to do:
# aptitude install ia32-libs
Setting up sudo
At this point I add myself as an sudoer by adding my regular username to /etc/sudoers, just under the entry for root, giving full admin privileges (i.e., just copy the line for root).
# chmod 640 /etc/sudoers
# emacs -nw /etc/sudoers
... add a regular user to the list ...
# chmod 440 /etc/sudoers
I believe in using sudo rather than becoming root to do admin tasks.
Setting up APT Sources
It is also a good idea to modify /etc/apt/sources.list now.  Mine looks like this:
deb http://ftp.us.debian.org/debian/ squeeze main contrib non-free
deb-src http://ftp.us.debian.org/debian/ squeeze main contrib non-free

deb http://security.debian.org/ squeeze/updates main contrib non-free
deb-src http://security.debian.org/ squeeze/updates main contrib non-free

deb http://www.debian-multimedia.org squeeze main non-free
As I see, the debian-multimedia.org repository is added.  Though this isn’t absolutely essential, I may wish to do
# wget http://www.debian-multimedia.org/pool/main/d/debian-multimedia-keyring/debian-multimedia-keyring_2008.10.16_all.deb
# dpkg -i debian-multimedia-keyring_2008.10.16_all.deb
# aptitude update
in order to avoid GPG error accessing this repository in future.
Setting up DHCP Hostname
Modify /etc/dhcp/dhclient.conf to have the line:
send host-name "yourcomputershostname";
This way, identify this computer on a router, for example, will be easier.
Intel Centrino Ultimate-N 6300
The built-in kernel for Squeeze (2.6.32) appears to have the driver for this WiFi card, but firmware need to be loaded:
$ sudo aptitude install firmware-iwlwifi
Or alternatively I can download firmware, and manually copy the file iwlwifi-6000-4.ucode to /lib/firmware.  For example:
$ wget http://intellinuxwireless.org/iwlwifi/downloads/iwlwifi-6000-ucode-9.221.4.1.tgz
$ tar -xvzf iwlwifi*.tgz
$ sudo cp iwlwifi-6000-ucode-9.221.4.1/iwlwifi-6000-4.ucode /lib/firmware
Either way, WiFi should be working after reboot.
Integrated Webcam
To check if the webcam works, the easiest is to use Kopete on KDE (Menu -> Applications -> Internet -> Kopete).  Go to Settings -> Configure -> Video, and see if the preview works.
KDE
To install KDE,
# aptitude update
# aptitude install kdebase kdenetwork kmix
# reboot
Upon reboot, the KDM login screen appears.
KDE Network Manager
If I don’t see any network manager on the system tray of the bottom bar, click on the system tray (this one is located to the left of the clock by default).  Right click -> System Tray Settings -> Plasma Widgets and check Network Management.  Try to add a wireless access point.
Suspend and Resume
I have always been a type who turns off computer when not used, but I am now using suspend/resume frequently.  Fortunately I have few problems using suspend to RAM/disk on T410s so far.
TrackPoint Scrolling
I can use the red TrackPoint for scrolling as well!  There are several ways to do this, but I’m going to use xinput.  If it is not installed, do
$ sudo aptitude install xinput
See what ID is associated with TrackPoint:
$ xinput list
⎡ Virtual core pointer                          id=2    [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer                id=4    [slave  pointer  (2)]
⎜   ↳ SynPS/2 Synaptics TouchPad                id=11   [slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                     id=13   [slave  pointer  (2)]
⎣ Virtual core keyboard                         id=3    [master keyboard (2)]
    ↳ Virtual core XTEST keyboard               id=5    [slave  keyboard (3)]
    ↳ Power Button                              id=6    [slave  keyboard (3)]
    ↳ Video Bus                                 id=7    [slave  keyboard (3)]
    ↳ Sleep Button                              id=8    [slave  keyboard (3)]
    ↳ Integrated Camera                         id=9    [slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard              id=10   [slave  keyboard (3)]
    ↳ ThinkPad Extra Buttons                    id=12   [slave  keyboard (3)]
    ↳ ACPI Virtual Keyboard Device              id=14   [slave  keyboard (3)]
The TrackPoint’s ID is 13 in my current configuration.  Then add the following lines in ~/.xsessionrc:
xinput set-int-prop 13 "Evdev Wheel Emulation" 8 1
xinput set-int-prop 13 "Evdev Wheel Emulation Button" 8 2
xinput set-int-prop 13 "Evdev Wheel Emulation Axes" 8 6 7 4 5
After restarting X, moving TrackPoint while pressing the blue middle button acts as scroll, either vertically or horizontally.
Now that TrackPoint can be used for scrolling, I see little reason to enable TrackPad, as I never use it.  It may be a good idea to disable TrackPad in BIOS to save power consumption as well as to prevent accidentally touching it with my hands while typing.  Note that after disabling TrackPad, ID (see above) may change, so ~/.xsessionrc may need to be rewritten.
Power Consumption Tips
I now have a separate article for this topic.
Miscellaneous
snd_pcsp Inturrpts
Under certain conditions (which I cannot always reproduce), the system becomes sluggish after playing sounds off Flash on Firefox, for example, and until KDE restarts, sounds are not working.  Looking at the processes with powertop, I found some process using snd_pcsp tends to make frequent interrupts.  A solution to this problem is to ensure that the snd_pcsp kernel module gets loaded after snd_hda_intel gets loaded by feeding “index=2″ option.  Make sure that the file /etc/modprobe.d/snd-pcsp has a line like this:
options snd-pcsp index=2
Installing Applications
Just a list of applications to install on my machine:

Firefox 4 (beta)
Dropbox
Mendeley
F.lux
KeePass
More to come…

Document Update History
June 1, 2010 – First version.
June 6, 2010 – Added a note on suspend and resume.
June 9, 2010 – Added ThinkWiki reference.  Added a section on TrackPoint scrolling.  Created Power Consumption Tips.  Added the preload daemon section.
November 27, 2010 – Major update after a warranty repair of the vertical band issue with LCD.
December 19, 2010 – Moved power consumption tips to another article.



TestDisk
Taro — Mon, 24 May 2010 10:46:18 +0000
Once I fucked up my Win XP/Linux dual boot installation really badly and it appeared all partitions were gone, or at least unrecognizable.
I browsed partition recovery utilities on the web, and found TestDisk.
After following a tutorial (which I lost track of), I could totally recover my old partitions!
Just keeping note for myself so that I don’t forget.



python-mode.el for Emacs
Taro — Thu, 29 Apr 2010 01:38:41 +0000
On Debian, you can simply install the package “python-mode,” but this describes the manual method.
For OS X (I currently use Snow Leopard), I can install the python-mode package via FinkCommander.
First, download python-mode.el and place it under ~/.emacs.d (create this directory if it does not exist).
Then add the following lines to your Emacs configuration file ~/.emacs:
(load “~/.emacs.d/python-mode.el”)

;;python

(setq auto-mode-alist (cons ‘(“\\.py$” . python-mode) auto-mode-alist))

(setq interpreter-mode-alist (cons ‘(“python” . python-mode)

interpreter-mode-alist))

(autoload ‘python-mode “python-mode” “Python editing mode.” t)
(global-font-lock-mode t)

(font-lock-mode +1)
That’s it.



Turning On X Forwarding on Mac OS X Snow Leopard
Taro — Fri, 26 Mar 2010 05:06:16 +0000
Read this article.

sed 's/#X11Forwarding\ no/X11Forwarding\ yes/' /etc/sshd_config > /tmp/sshd_config
sudo mv /tmp/sshd_config /etc/.





Installing Tor on Debian Squeeze
Taro — Wed, 24 Mar 2010 02:34:05 +0000
Install packages:

$ sudo aptitude install tor privoxy


Add the following lines to relevant sections in /etc/privoxy/config:

...
forward-socks4a / 127.0.0.1:9050 .
...
listen-address 127.0.0.1:8118


( See this thread for detail.)  Then

$ sudo /etc/init.d/privoxy restart


to make the change effective.  In my browser proxy setting, use 127.0.0.1 as server and 8118 as port to use it.



Microsoft Natural Ergonomic Keyboard 4000 on Mac OS X Snow Leopard
Taro — Tue, 23 Mar 2010 20:54:47 +0000
I don’t really need to tweak much for the MS keyboard to work with Snow Leopard, as it will be detected and configured automatically.  However, I’m not used to the Mac keyboard bindings, though I can live with it  as long as emacs works as expected.  So here is how I configure, so that at least in the X-like environment keyboard bidings will be similar to Linux.
Go to Apple -> System Preferences -> Keyboard -> Keyb0ard tab -> Modifier Keys.  I configure as follows:

Also, launch X11 and go to X11 -> Preferences -> Input.  Uncheck “Enable key equivalents under X11.”
For now this works for me.



Installing PSFex on Mac OS X Snow Leopard
Taro — Tue, 23 Mar 2010 00:45:11 +0000
There seems to be no stable release for PSFex yet.  So we download source from subversion.
Before doing that, you need to install FFTW and ATLAS first.  (You need a single-precision library for FFTW.)
Download a tarball (for trunk) from the official subversion repository.

$ mkdir /usr/local/src/psfex
$ cd /usr/local/src/psfex
... download tarball ...
$ tar xf *.tar.gz
$ cd trunk.r149
$ ./configure --with-atlas=/usr/local/atlas/lib --with-atlas-incdir=/usr/local/atlas/include \
    --with-fftw=/usr/local/lib --with-fftw-incdir=/usr/local/include --enable-threads
$ make
$ sudo make install

The executable psfex will be installed at /usr/local/bin.



Installing GFortran on Mac OS X Snow Leopard
Taro — Tue, 23 Mar 2010 00:32:12 +0000
Installing from GFortran Wiki (My Current Preferred Way)
GCC Wiki has a link to Mac OS X installer.  As the command gfortran will be installed at /usr/local/gfortran and causes less interferences with other libraries and software, this is my preferred way.
Installing from Fink
I use Fink (as opposed to MacPorts) to manage most Linux-like tools.
$ fink install netcdf-gfortran
This will make gfortran available at /sw/bin/gfortran.  It may take a bit of time to compile binaries though.
Installing Binary from High Performance Computing for Mac OS X
This site has a bunch of pre-compiled binaries useful for scientific computing, and GFortran is one of them.
Installing Binary from the R Project
You may simply use a binary made available by the R people.  Download a .dmg file from here.  This version will install at /usr/local/bin.
Update History
November 24, 2010 – Added different install options.



Installing SExtractor on Mac OS X Snow Leopard
Taro — Tue, 23 Mar 2010 00:14:36 +0000
(November 25, 2010) I screwed up SExtractor installation for no reason and in an attempt to fix it, I’ve found a few things.  After wasting a few hours again, I think this installation note can work for most people who can actually build software from source… I think the usefulness of this note will be limited though, as a new release of SExtractor appears imminent.
Installing from Fink
Fink has a version of SExtractor.  I can install this via the command fink install.
$ fink install sextractor
If Fink is already installed, this may be the most trouble free way of installing SExtractor.
Installing from Source
IMPORTANT: I need FFTW and ATLAS installed already, and assume that FFTW was installed at /usr/local/fftw, and ATLAS at /usr/local/atlas following the source install procedures described in those notes.  Make sure that you build both double and single precision versions of FFTW by going through the make procedure twice to install both.  Build both of them from source, and do not use the libraries from, say, Fink; otherwise dependency hell might arise.  So, if I have Fink or Scisoft installed, put them away during the installation to avoid path problems.  For example, temporarily rename /sw to /sw.tmp to put Fink away, /usr/local/scisoft to /usr/local/scisoft.tmp to put Scisoft away.  (Or alternatively I can “comment out” from your start up shell script those lines that set up Fink and Scisoft, so that environment variables won’t include paths to these things.)  Then restart the computer, and start the whole installation process (that is, including FFTW and ATLAS).
Download the SExtractor source into a temporary directory:
$ mkdir tmp
$ cd tmp
... download sextractor-2.8.6.tar.gz
$ tar -xvzf sex*.tar.gz
$ cd sextractor-2.8.6
$ ./configure --prefix=/usr/local/sextractor-2.8.6 \
    --with-atlas=/usr/local/atlas/lib --with-atlas-incdir=/usr/local/atlas/include \
    --with-fftw=/usr/local/fftw/lib --with-fftw-incdir=/usr/local/fftw/include \
    --enable-threads
$ make
$ sudo make install
$ ln -s /usr/local/sextractor-2.8.6 /usr/local/sextractor
$ sudo cp -rp config /usr/local/sextractor
This will install SExtractor at /usr/local/sextractor-2.8.6 (which is aliased by /usr/local/sextractor) and an executable command sex will be located at /usr/local/sextractor/bin/sex.  Make a simlink to that binary /usr/local/bin to make it available system-wide.  I might also want to make a simlink to /usr/local/sextractor/bin/ldactoasc, which is a utility command that might be useful.  The last line copies the SExtractor configuration files and convolution masks to /usr/local/sextractor/config.  The default convolution masks are often used so it may be a good idea to keep them under the same installation directory.
Hope this is it!
What if Make Stops with an Error?
There may be a bug in ./configure script that makes the --with-fftw-incdir option to be not honored.  If the make stops with errors complaining as in “error: fftw3.h: No such file or directory” despite that I have installed FFTW properly, copy /usr/local/fftw/include/fftw3.h to the sextractor-2.8.6/src directory before doing make.  It will compile fine then.
Update History
November 25, 2010 — Made the procedure description a bit more elaborate where things may fail.



Installing ATLAS on Mac OS X Snow Leopard
Taro — Tue, 23 Mar 2010 00:08:56 +0000
Installing from Fink
Do:
$ fink install atlas atlas-shlibs
That’s it.  They will be installed under /sw/lib.
Installing from Source (Recommended if Building Other Software That Depends on ATLAS)
First of all, I need a FORTRAN compiler, so install it.  I recommend using the Mac installer from the GCC Wiki site, so that GFortran won’t end up having hard-to-track-down dependencies.
I download tarballs for ATLAS and LAPACK from the official repositories.  Here I use ATLAS 3.9.32 and LAPACK 3.3.0.  I assume the compiled library and header files will be installed at /usr/local/atlas.
Here is the procedure:
$ mkdir tmp
$ cd tmp
$ wget http://www.netlib.org/lapack/lapack.tgz
... download the tarball for ATLAS from sourceforge ...
$ tar -xvjf atlas3.9.32.tar.bz2
$ mv ATLAS atlas-3.9.32
$ ln -s atlas-3.9.32 atlas
$ cd atlas
$ mkdir bld
$ cd bld
$ ../configure -b 64 --shared --prefix=/usr/local/atlas-3.9.32 \
    --with-netlib-lapack-tarfile=../../lapack.tgz
$ make build
If I wish to do some checking, do:
$ make check
...
$ make time
...
(I may wish to compare the output with doc/atlas_install.pdf.)
If I don’t find any problem, do:
$ sudo make install
$ sudo ln -s /usr/local/atlas-3.9.32 /usr/local/atlas
That’s it.
Update History
November 24, 2010 – Added the Fink section.  Using the most recent versions of ATLAS and LAPACK.  Made the install directory more explicit to the version installed.



Installing Squid Proxy Server on Mac OS X Snow Leopard
Taro — Mon, 22 Mar 2010 19:52:42 +0000
My goal is to set up a very basic proxy server on my Mac box on  campus, so that I can have full access to subscription-based academic  journals via the proxy on my laptop even when I am off campus.  Some  schools provide such (library) proxies but my school unfortunately does  not.  I want to set it up such that the proxy requires a password  authentication in order not to make it wide open to the public.
Getting and Installing Squid
Download a  tarball for a stable version from the repository.  The version that I  use here is 2.7.  I assume the file is downloaded to /usr/local/src/squid.
$ cd /usr/local/src/squid
$ gunzip -c squid-2.7.STABLE9.tar.gz | tar xvf -
$ cd squid-2.7.STABLE9
$ ./configure
$ make
$ sudo make install
$ cd helpers/basic_auth/NCSA
$ make
$ sudo make install
$ sudo chown -R nobody /usr/local/squid/var
$ sudo /usr/local/squid/sbin/squid -z
Squid will be installed at /usr/local/squid.  (The last  command is necessary to run a daemon as user nobody.)
Configure Squid
First, prepare a NCSA-compliant encrypted password file for a user  (here with username johndoe):
$ cd /usr/local/squid/etc
$ sudo touch squid_passwd
$ sudo chmod o+r squid_passwd
$ sudo htpasswd squid_passwd johndoe
New password:
Re-type new passwod:
Adding passwod for user johndoe
Now, edit /usr/local/squid/etc/squid.conf.  The following  lines need to be added:
# Add this to the auth_param section
auth_param basic program /usr/local/squid/libexec/ncsa_auth /usr/local/squid/etc/squid_passwd

# Add this to the bottom of the ACL section
acl ncsa_users proxy_auth REQUIRED

# Add this at the top of the http_access section
http_access allow ncsa_users
Finally, run the server:
$ sudo /usr/local/squid/sbin/squid -N -d 1 -D
Firewall will prompt me to see if I allow incoming connections to  squid.  Say “allow.”
The IP address or host name of your Mac box at the port 3128 will be  available as a proxy server now.
Launch Squid on Startup with launchd
Under the directory /Library/LaunchDaemons, create a file  named squid.plist with the following content:



  
    Label
    squid
    OnDemand
    
    ProgramArguments
    
      /usr/local/squid/sbin/squid
      -N
      -d 1
      -D
    
    ServiceIPC
    
  

Then issuing
$ sudo launchctl load -w /Library/LaunchDaemons/squid.plist
will launch squid.  On reboot, the proxy  should also be working  automatically.




Installing DS9 on Debian Squeeze
Taro — Mon, 22 Mar 2010 01:29:55 +0000
Download the latest version from the official website:

$ wget http://hea-www.harvard.edu/saord/download/ds9/linux64/ds9.linux64.6.0.tar.gz
$ tar -xvzf ds9*.tar.gz
$ chmod 755 ds9
$ sudo mv ds9 /usr/local/bin

(The wget line is an example.) This will install DS9 under /usr/local/bin.



Installing Science Packages for Python on Debian Squeeze
Taro — Mon, 22 Mar 2010 01:22:17 +0000
Eventually it’s likely I need to build all NumPy, SciPy, and Matplotlib from source, but for now this is good enough:

$ sudo aptitude install ipython python-mode
$ sudo aptitude install python-numpy python-scipy python-matplotlib python-nose

If I need WXAgg as a backend for Matplotlib, I need wxpython:

$ sudo aptitude install python-wxgtk2.8

I need to modify matplotlibrc:

$ cp /etc/matplotlibrc ~/.matplotlib/
... edit ~/.matplotlib/matplotlibrc to have following lines ...
backend : WXAgg

(Debian moves the default matplotlibrc to such a weird location…)
SciPy Might Fail Tests
I test NumPy and SciPy on Python interactive session as follows:

>>> import numpy
>>> numpy.test('full', verbose=10)
...

>>> import scipy
>>> scipy.test('full', verbose=10)
...

I rarely see perfect results (i.e., absolutely no errors), but if there are too many fails then that may be a concern.
Update History
November 27, 2010 – Things have chanced a lot on Squeeze now that a stable release is approaching.  Removed obsolete information and updated everything.



Installing JD on Debian Squeeze
Taro — Mon, 22 Mar 2010 01:00:16 +0000

$ sudo apt-get install jd ttf-konatu ttf-monapo ttf-mona




Installing Freemind on Debian Squeeze
Taro — Sun, 21 Mar 2010 23:42:03 +0000

$ sudo apt-get install freemind




Installing Wine on Debian Squeeze
Taro — Sun, 21 Mar 2010 23:20:21 +0000

$ sudo apt-get install wine lib32nss-mdns




Using Japanese on Debian Squeeze
Taro — Sun, 21 Mar 2010 23:05:30 +0000
The goal is to make the system capable for Japanese input, while the base system remains English.  For the Japanese input method, l use  Anthy.
First, set up locales and install im-switch.  In addition to my base English locale (i.e., en_US.UTF-8), install Japnese locales ja_JP.EUC-JP and ja_JP.UTF-8:
$ sudo aptitude install locales im-switch
$ sudo dpkg-reconfigure locales
Set the default locale for the system to en_US.UTF-8 (or something else if you wish).
Also install at least some Japanese fonts:
$ sudo aptitude install ttf-kochi-mincho ttf-kochi-gothic ttf-sazanami-mincho
$ sudo aptitude install ttf-ipafont-gothic ttf-ipafont-mincho
With IBus
When I started using Squeeze, IBus was unstable and unusable, but things have changed and this is my preferred way now.
Install necessary Debian packages and configure for KDE:
$ sudo aptitude install ibus ibus-anthy ibus-qt4 kasumi
$ im-switch -c
... choose IBus for the current locale ...
Run the IBus setup program:
$ ibus-setup
If asked, start ibus-daemon.  Logout the KDE sesssion, restart X session, and login again.  Under the Input Method tab, add “Japanese – Anthy”.  Configure IBus to my liking.  By default, Ctrl + Space (unlike Shift + Space which used to be the case) triggers the Japanese input mode.  The IBus icon also shows up in the system tray, from which various settings can be changed.
With UIM
This section is largely left for historical purpose, as I now use IBus (see above), which acts more like SCIM used to.
Install necessary Debian packages and configure for KDE:
$ sudo aptitude install uim uim-anthy uim-qt uim-xim kasumi
$ im-switch -c
... choose uim-toolbar-qt ...
$ uim-toolbar-qt4
I also need to register uim-toolbar-qt4 to run at KDE session startup: System Settings -> Advanced -> Autostart.  Add program /usr/bin/uim-toolbar-qt4.  Restart X.  This way, I do not have to issue the uim-toolbar-qt4 command after every boot.
Using Japanese in Emacs
Launching Emacs by changing locale temporarily.  For example:
$ LC_CTYPE=ja_JP.UTF-8 emacs somefile.txt
It turns out this is all I need to make Emacs sensitive to Japanese.  There are other more elegant ways I suppose, but as long as it works for me…  I don’t use Japanese on Emacs much anyways.
Adding Words to Kasumi Dictionary
From command line, do:
$ kasumi
$ kasumi -a   # this one goes directly to new word mode
The same application can be launched from KDE Menu -> Applications -> Settings -> Anthy Dictionary Editor.



Installing K3b on Debian Squeeeze
Taro — Sun, 21 Mar 2010 22:49:24 +0000

$ sudo aptitude install k3b libdvdcss transcode normalize-audio




Sound & Music Applications on Debian Squeeze
Taro — Sun, 21 Mar 2010 22:40:51 +0000
Amarok is my favorite audio player.  For tagging ogg files, I use EasyTAG:

$ sudo apt-get install kmix
$ sudo apt-get install amarok
$ sudo apt-get easytag
$ sudo apt-get install lastfm


It seems replaygain became a built-in feature of Amarok.  Great.



Installing MPlayer on Debian Squeeze
Taro — Sun, 21 Mar 2010 22:30:52 +0000
One easy step:

$ sudo apt-get install mplayer pulseaudio


Should add the debianmultimedia.org repository to /etc/apt/sources.list though.
I had a problem with audio stream when I played a music video (.flv).  I could solve this by installing pulseaudio, and under the audio preferences setting the driver to “pulse.”



Installing Firefox (Iceweasel) on Debian Squeeze
Taro — Sun, 21 Mar 2010 22:29:09 +0000
Note that in Debian, Firefox is called iceweasel.
$ sudo aptitude install iceweasel
Then install some popular plugins.
Java
$ sudo aptitude install sun-java6-plugin sun-java6-fonts
Flash
Do:
$ sudo aptitude install flashplugin-nonfree ttf-mscorefonts-installer ttf-xfree86-nonfree
This should do it.  Try youtube.com to see if Flash is working.
MPlayer
The following apt-get used to work, but apparently not on Squeeze:
$ sudo aptitude install mozilla-mplayer
I wonder where the new module is.
Other Tips
Speeding Up via Pipelining
Visit the URL about:config with Firefox and set the following items:
network.http.pipelining = true
network.http.pipelining.maxrequests = 255
network.http.pipelining.ssl = true
network.http.proxy.pipelining = true



Customizing & Installing Linux Kernel on Debian Squeeze
Taro — Sun, 21 Mar 2010 05:31:57 +0000
Here is a quickie for customizing and install Linux kernel 2.6.x on Squeeze.
Install some packages and kernel source:
$ sudo aptitude update
$ sudo aptitude install kernel-package bzip2 g++ libqt3-mt-dev libncurses5-dev sudo fakeroot
If you do not belong to group “src”, add yourself to the group as we work under /usr/src:
$ sudo adduser username src
You need to logout and login for this change to take effect.  You also need to be able to use sudo or su to install the new kernel in the end.
The kernel packaged for Squeeze is at version 2.6.32.  Alternatively a newer kernel could be obtained from kernel.org and saved to /usr/src.  The customization procedure should be similar.  Here I just follow the Debian way:
$ sudo aptitude install linux-source-2.6.32
Extract the source tree:
$ sudo chgrp -R src /usr/src
$ sudo chmod -R g+w /usr/src
$ cd /usr/src
$ rm linux
$ tar -jxf linux-source-2.6.32.tar.bz2
$ ln -s linux-source-2.6.32 linux
$ cd linux
Edit the EXTRAVERSION entry in Makefile, as in:
EXTRAVERSION = .20100321.1
for example to add .20100321.1 to the kernel version number.  This is convenient for keeping the existing, working kernels around when you need to recompile with different options.
Use xconfig or menuconfig to customize the kernel options.  Before the make-kpkg lines, setting concurrency (most likely to the number of cores of my processor) is optinal but having a higher number typically reduces the compilation time.
$ make mrproper
$ make xconfig    # or make menuconfig
$ export CONCURRENCY_LEVEL=4    # this is optional
$ fakeroot make-kpkg clean
$ fakeroot make-kpkg --initrd kernel_image
$ cd ..
$ sudo dpkg -i linux-image-2.6.32.20100321.1_2.6.32.20100321.1-10.00.Custom_amd64.deb
Upon reboot in the GRUB menu you will find the newly installed kernel:
$ sudo reboot
Purging Old Kernel Image from System
For example, if the kernel to be uninstalled is of version 2.6.26 and the extra version that I used was 20091112.1, do:
$ sudo dpkg -P linux-image-2.6.26.20091112.1
That’s it.  However it is often a good idea to keep at least one kernel image that I know for sure to work so that when a custom kernel fails, I have something to fall back on.  On the other hand, it is also a good idea to purge very old kernel images to save space in /boot.
Giving Kernel Boot Options with Grub (Version 2)
The new Grub has a slightly different way of specifying boot options (/boot/grub/menu.lst no longer exists for version 2!).
$ sudo chmod +w /boot/grub/grub.cfg
... open and edit /boot/grub/grub.cfg ...
$ sudo chmod -w /boot/grub/grub.cfg
If I need to provide a kernel with boot options, find the “menuentry” for the kernel, and change from
linux /vmlinuz-2.6.36.1.3 root=UUID=somehash ro quiet
to
linux /vmlinuz-2.6.36.1.3 root=UUID=somehash ro quiet usbcore.autosuspend=1
in order to set the usbcore.autosuspend option, for example.
Playing with CONCURRENCY_LEVEL
With Core i5 (2 cores but virtually 4 cores with hyperthreading), I get:
$ fakeroot make-kpkg clean
$ export CONCURRENCY_LEVEL=4
$ time fakeroot make-kpkg --initrd kernel_image
...
real    6m50.028s
user    21m45.186s
sys     2m42.246s

$ fakeroot make-kpkg clean
$ export CONCURRENCY_LEVEL=1
$ time fakeroot make-kpkg --initrd kernel_image
...
real    13m55.216s
user    13m23.638s
sys     1m50.359s
I say significant speed up!
What if Kernel Panics after Reboot?
(NOTE: This was true back when I was playing around when Squeeze was still far from stable.  I don’t think this section is relevant as of November 2010, since Squeeze is very close to stable now. Just leaving the note in case for its archival value.)  
Well, it turns out make-kpkg --initrd does not automatically create a /boot/inird.img-* file for my current configuration.  I wondered for a while why my new custom kernel keeps panicking upon reboot but finally realized that was the cause.
Fortunately, I can create the image myself easily:
$ sudo update-initramfs -c -k 2.6.32.20100321.1
$ sudo update-grub
This should work.



Installing Debian Squeeze on Dell XPS 630i
Taro — Sun, 21 Mar 2010 05:17:26 +0000
First, a Big Problem
I wanted to do a fresh install using an install CD or USB stick for  Squeeze (still as testing), but unfortunately these install media have  never worked for me on this desktop machine.  For some reason it always ends in kernel panic,  which I couldn’t solve.  I haven’t been able to get help on the Debian  bug report system.  So this time I decided to do a fresh install off hard drive,  after installing Debian Lenny normally, carefully keeping the disk  partitions that I did not wish to erase.  I don’t know why the Squeeze  installer never works on XPS 630i, when the Lenny installer perfectly  does its job!
Installation
The procedure assumes that I have already been able to start the Debian installer for Squeeze.  When it starts, choose:

Choose a language: English (or whatever)
Choose a country, territory or area: Canada (or whatever)
Keymap to use: American English (or whatever)
Primary network interface: eth0: nVidia Corporation MCP51 Ethernet Controller (I use wired network)
Host name: (whatever)
Domain name: (whatever)
Root password: (whatever)
Full name fo rthe new user: (whatever)
Username for your account: (whatever)
Choose a password for the new user: (whatever)
Select your time zone: Atlantic (or whatever)
Partitioning method: Manual

Here, I do manual partitioning since this is a reinstallation and  there are existing partitions (it is also a dual boot machine with  Windows XP; that’s why the partition scheme is a bit weird below).  I  keep /home, /usr/local, and /install partitions, but can format all the other partitions:
SCSI1 (0,0,0) (sda) - 750.2 GB ATA Hitachi HDS72107
  #1  primary   83.9 GB  B    ntfs
  #2  primary  178.3 GB    K  ext3    /home/media
  #5  logical    6.3 GB    F  ext3    /var
  #6  logical   10.5 GB    F  ext3    /usr
  #7  logical   10.5 GB    K  ext3    /usr/local
  #8  logical    8.4 GB    F  swap    swap
  #9  logical  440.8 GB    K  ext3    /home
 #10  logical  700.0 MB    F  ext3    /
 #11  logical  350.0 MB    F  ext3    /boot
 #12  logical  200.0 MB    K  ext3    /install
 #13  logical   10.3 GB    F  ext3    /tmp
Also, the /install partition should not be mounted during the installation.  When done, write changes to disk.
The following three questions will be asked now; otherwise it will be asked later.  I can check the mirror server status as well.

Debian archive mirror country: United States (or whatever)
Debian archive mirror: ftp.us.debian.org (or whatever)
Proxy: none (or whatever)

The base system will be installed.

Participate in the package usage survey?: no (or yes if preferred)
Choose software to install: Select none; I can install everything later.
Install the GRUB boot loader to the master boot record?: Yes

Should be done!  Reboot into newly installed Squeeze!
Post-Install Admin Tweaks
Login as root.  First I install a few essential packages for system administration, etc.:
# aptitude install sudo ssh rsync wget
If I need to run 32-bit applications, it is also a good idea to do:
# aptitude install ia32-libs
Setting up sudo
At this point I add myself as an sudoer by adding my regular username to /etc/sudoers, just under the entry for root, giving full admin privileges (i.e., just copy the line for root).
# chmod 640 /etc/sudoers
# nano /etc/sudoers
... add a regular user to the list ...
# chmod 440 /etc/sudoers
I believe in using sudo rather than becoming root to do admin tasks.
Setting up APT Sources
It is also a good idea to modify /etc/apt/sources.list now.  Mine looks like this:
deb http://ftp.us.debian.org/debian/ squeeze main contrib non-free
deb-src http://ftp.us.debian.org/debian/ squeeze main contrib non-free

deb http://security.debian.org/ squeeze/updates main contrib non-free
deb-src http://security.debian.org/ squeeze/updates main contrib non-free

deb http://www.debian-multimedia.org squeeze main non-free
As I see, the debian-multimedia.org repository is added.  Though this isn’t absolutely essential, I may wish to do
# wget http://www.debian-multimedia.org/pool/main/d/debian-multimedia-keyring/debian-multimedia-keyring_2008.10.16_all.deb
# dpkg -i debian-multimedia-keyring_2008.10.16_all.deb
# aptitude update
in order to avoid GPG error accessing this repository in future.
Setting up DHCP Hostname
Modify /etc/dhcp/dhclient.conf to have the line:
send host-name "yourcomputershostname";
This way, identify this computer on a router, for example, will be easier.
KDE
To install KDE,
# aptitude update
# aptitude install kdebase kdenetwork kmix
# reboot
Upon reboot, the KDM login screen appears.
Hardware
NVIDIA GeForce 8800 GT
I usually use a custom kernel, so the nvidia module also has to be compiled.  Following this page, here is the summary.  I assume the custom kernel has been compiled and installed following this procedure, and this procedure is done after reboot, while the system is running on that custom kernel.
$ sudo aptitude install module-assistant nvidia-kernel-common
$ sudo m-a auto-install nvidia-kernel-source
$ sudo aptitude install nvidia-glx
Now, logout of KDE and do a console login to modify /etc/X11/xorg.conf.  This file probably does not exist, in which case I can generate one by
$ sudo X -configure
Move the newly generated xorg.conf file to /etc/X11/xorg.conf.  Restart X and I should see the NVIDIA driver working
$ sudo aptitude install mesa-utils
$ glxinfo | grep rendering
direct rendering: Yes
This should be it!



Using Some Python 3 in Python 2.6.x
Taro — Fri, 19 Mar 2010 17:24:01 +0000
This is just a note for myself to start getting used to the Python 3 coding practices.
In order to use some Python 3 features in Python 2.6.x, use this statement in my script:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
It’s not a very good idea to transition to Python 3 right away though, since most packages and modules are still for Python 2.x and it doesn’t seem to change for a while.



Installing pysao on Mac OS X Snow Leopard
Taro — Fri, 19 Mar 2010 02:15:34 +0000
The pysao (or python-sao) version used here is svn rev. 115:

$ mkdir /usr/local/src/pysao
$ cd /usr/local/src/pysao
$ svn checkout http://python-sao.googlecode.com/svn/trunk/ python-sao
$ cd python-sao
$ python xpa_download.py
$ tar xf xpa-2.1.12.tar.gz
$ ln -s xpa-2.1.12 xpalib
$ cd xpalib
$ ./configure CFLAGS="-arch ppc -arch i386 -arch x86_64"
$ make
$ cd ..
$ python setup.py build
$ sudo python setup.py install

That’s it.



Installing pdftk on Mac OS X Snow Leopard
Taro — Fri, 19 Mar 2010 01:56:25 +0000
There is a .dmg for pdftk 1.41 for OS X 10.6.2.  Thanks to fredericiana.com.  What a time saver.



Installing FFTW on Mac OS X Snow Leopard
Taro — Fri, 19 Mar 2010 01:29:38 +0000
No FORTRAN compiler appears to be included in Xcode, so I need to install it if I haven’t done so.  I installed GFortran.  In the configure statement below, set F77 to gfortran.
Installing from Fink
For FFTW version 3.x, do:
$ fink install fftw3 fftw3-shlibs
The library will be available under /sw/lib.
Installing from Source
The version of FFTW we use here is 3.2.2.  Follow this:
$ mkdir tmp
$ cd tmp
$ wget http://www.fftw.org/fftw-3.2.2.tar.gz
$ tar -xvzf fftw-3.2.2.tar.gz
$ cd fftw-3.2.2
$ ./configure --prefix=/usr/local/fftw-3.2.2 --enable-threads \
    CC="gcc -arch i386 -arch x86_64" CXX="g++ -arch i386 -arch x86_64" \
    CPP="gcc -E" CXXCPP="g++ -E" F77=gfortran
$ make
$ sudo make install
$ sudo ln -s /usr/local/fftw-3.2.2 /usr/local/fftw
This will install FFTW under /usr/local/fftw-3.2.2.  If a single precision version also needs to be installed, add --enable-single to the configure statement above and do the configure/make/make install sequence again (it’s a good idea to run “make clean” before the second time around though).
Update History
November 24, 2010 – Added the Fink version.  Changed the installation directory to /usr/local/fftw to avoid potential conflict later.



Installing HOTPANTS on Mac OS X Snow Leopard
Taro — Thu, 18 Mar 2010 18:13:12 +0000
You can’t have enough of hot pants in your life, so you need to reinstall HOTPANTS on Snow Leopard.
First of all, you need a development version of CFITSIO already installed.  See this page if you need to install it yourself.
The version here is 5.1.10.  Download it from Andrew Becker’s distribution web page.

$ mkdir /usr/local/src/hotpants
$ cd /usr/local/src/hotpants
$ wget http://www.astro.washington.edu/users/becker/software/hotpants_v5.1.10.tar.gz
$ tar xf hotpants_v5.1.10.tar.gz
$ cd hotpants_v5.1.10

Now you need to patch the source files for OS X.  The patch that I made is here, which you need to rename it (it is a simple text file, not a real .odt file; it’s a workaround to bypass a wordpress.com limitation):

$ wget http://okomestudio.net/biboroku/wp-content/uploads/2010/03/patch_v5-1-10.odt
$ mv patch_v5-1-10.odt patch_v5.1.10
$ patch -p1 < patch_v5.1.10

You need to modify the paths for CFITSIOINCDIR and LIBDIR in Makefile depending on where your CFITSIO is located.  Then do:

$ make
$ sudo cp extractkern hotpants maskim /usr/local/bin


The last line is only necessary if you wish to install the binaries for system-wide access.
That’s it!



Installing CFITSIO on Mac OS X Snow Leopard
Taro — Thu, 18 Mar 2010 17:41:12 +0000
It’s handy to have CFITSIO against which you may build miscellaneous packages for science computing.
To have this in your system, download the tarball (here it’s version 3.24) and build as follows:

$ mkdir /usr/local/src/cfitsio  # Need a correct file permission for this.
$ cd /usr/local/src/cfitsio
$ wget ftp://heasarc.gsfc.nasa.gov/software/fitsio/c/cfitsio3240.tar.gz
$ tar xf cfitsio3240.tar.gz
$ cd cfitsio
$ export CFLAGS="-arch i386 -arch x86_64 -g -O2"
$ ./configure
$ make

This should build for both i386 and x86_64 architectures, and the header and library files will be at /usr/local/src/cfitsio/cfitsio/include and /usr/local/src/cfitsio/cfitsio/lib directories just underneath the top of the source tree (which you use when building against other libraries).
If you also wish to install those at the standard library location, i.e., /usr/local, then do instead:

...
$ ./configure --prefix=/usr/local
$ make
$ sudo make install


This should be it.



Restoring Hidden Files with Time Machine on Mac OS X Snow Leopard
Taro — Tue, 16 Mar 2010 17:22:55 +0000
Well I had this heart-stopping moment when I did not see all the “hidden” files (on Linux those are files with names starting with a “.”) while browsing Time Machine backups.
It turns out a widget called hiddenfiles can toggle the visibility of hidden files.  Install and set it (on Dashboard) to show all hidden files, you can make those files in Time Machine as well.
The fact that this isn’t a built-in feature is beyond my comprehension…  Apparently Apple is contributing to the increased incidence of cardiac arrests.



Using Logitech MX Revolution on Mac OS X Snow Leopard
Taro — Tue, 16 Mar 2010 16:43:02 +0000
Yup.  Given I need the power of 64-bit computing, I upgraded my Leopard box to Snow Leopard.  So far, so good.
For using Logitech MX Revolution on Snow Leopard, just download and install Logitech Control Center for OS X.  Works flawlessly and it’s even fully 64-bit.



Installing QTScrobbler on Debian Lenny (AMD64)
Taro — Sun, 14 Mar 2010 05:59:53 +0000
Download a tarball and expand it into a temporary directory. $ sudo apt-get install libcurl4-gnutls-dev libmtp-dev pkg-config libqt4-dev $ tar jxf qtscrob-0.10.tar.bz2 $ cd qtscrob-0.10 $ cd src/qt $ qmake $ make $ sudo cp qtscrob /usr/local/bin $ sudo chmod … Continue reading →


Installing NumPy, SciPy, & Matplotlib on Debian Lenny (AMD64) with Python 2.6
Taro — Sun, 14 Mar 2010 02:38:17 +0000
I assume that Python 2.6 has already been installed from source on Lenny.
$ sudo apt-get install libatlas-headers libatlas-base-dev
$ sudo apt-get install libfftw3-dev libsuitesparse-dev
$ sudo apt-get install liblapack-dev gfortran swig
Download NumPy from the official repository.  Then
$ cd /usr/local/src
$ tar -zxf numpy-1.3.0.tar.gz
$ cd numpy-1.3.0
$ python2.6 setup.py build
$ sudo python2.6 setup.py install
Download SciPy from the official repository.  Then
$ cd /usr/local/src
$ tar -zxf scipy-0.7.1.tar.gz
$ cd scipy-0.7.1
$ sudo python2.6 setup.py install
Before installing matplotlib, I want to install wx, since I want to use it as the main GUI backend.  See this installation note if you need to build wxPython from source.  Be careful that when building against wxagg, you need
# export LD_LIBRARY_PATH=/usr/local/lib/wx/2.8/lib
right before running setup.py so that the installer can find the wxWidgets libraries.
Download Matplotlib from the official repository.  Then
$ cd /usr/local/src
$ tar -zxf matplotlib-0.99.1.1.tar.gz
$ cd matplotlib-0.99.1.1
$ emacs setup.cfg   # See README for how to configure.
$ su
# python2.6 setup.py install
This should do it.



Installing Python 2.6 from Source on Debian Lenny (AMD64)
Taro — Sun, 14 Mar 2010 00:38:44 +0000
First, development files need to be installed for some Python packages to be built:
$ sudo apt-get install libreadline5-dev libbz2-dev tcl8.5-dev tk8.5-dev libsqlite3-dev
You need to update the above package list if the Python modules you need are not being built.  You will be told which packages were not built, when make finishes.
Then
$ tar -jxf Python-2.6.4.tar.bz2
$ cd Python-2.6.4
$ ./configure
$ make
$ sudo make install
That’s it.  Nothing different about AMD64.  Note that before doing sudo make install, you may wish to remove /usr/local/lib/python2.6/site-packages directory especially if you have a previous installation of Python 32-bit system.



Installing 64-bit CFITSIO 3.181 on Mac OS X Leopard
Taro — Thu, 11 Mar 2010 19:12:59 +0000
It turns out that the data I’m working on now are too big for 32-bit programs to handle!!
So along side Scisoft OS X which is all 32-bit, I need to install 64-bit version of CFITSIO since many programs depend on this library.
I install from the tarball (located here) downloaded into a temporary directory (e.g., /usr/local/src/cfitsio).  I use version 3.181.
Follow this:

$ mkdir /usr/local/src/cfitsio  # Need right permissions of course...
$ cd /usr/local/src/cfitsio
$ wget ftp://heasarc.gsfc.nasa.gov/software/fitsio/c/cfitsio3240.tar.gz
$ gunzip -c cfits*.gz | tar xvf -
$ cd cfitsio
$ export CFLAGS="-arch i386 -arch x86_64 -g -O2"
$ ./configure
$ make
$ sudo make install

(Key is to use “x86_64″ architecture when compiling.)
This actually installs library and header files under /usr/local/src/cfitsio/cfitsio/lib and /usr/local/src/cfitsio/cfitsio/include.   Read ./configure --help if you wish to install them in different locations; for example, doing ./configure --prefix=/usr/local in the procedure above should get the library installed on the “standard” /usr/local locations.
That should be it!  When building a program linking to this library, you need to specify the “lib” and “include” directory paths when configuring make for the program.



Installing HOTPANTS 5.1.10 on Mac OS X Leopard
Taro — Wed, 10 Mar 2010 22:21:18 +0000
You must love the name, but what the software does is far from sultry; it’s a piece of geekware.
Anyways, I needed to edit files when I install HOTPANTS, photometric alignment utility by Andrew Becker.  Here is my installation note before I forget.
HOTPANTS requires CFITSIO to be already installed.  On Leopard I use Scisoft OS X, so I don’t have to install CFITSIO separately.  Otherwise you have to install CFITSIO first.
Download the tarball and expand it into a temporary directory.
First, you need to delete (or comment out) the line which reads

#include


from all *.c source code files.
Then modify Makefile.  To link CFITSIO properly, modify CFITSIOINCDIR and LIBDIR:

CFITSIOINCDIR=/Applications/scisoft/i386/Packages/cfitsio/include
LIBDIR=/Applications/scisoft/i386/Packages/cfitsio/lib

This is fine if you have Scisoft OS X.  If you installed CFITSIO elsewhere, you have to find the paths yourself.
You should also remove the “-pedantic-errors” option from COPTS so that the line reads like:

COPTS= -funroll-loops -O3 -ansi -Wall -I$(CFITSIOINCDIR)


Then run make within the same directory.
That’s it.  You might wish to copy the binaries hotpants, extractkern, and maskim to one of those bin directories in your PATH.
Building 64-bit HOTPANTS
The 64-bit version of HOTPANTS is twice as sultry, I need to try it.  The change is that you need to first build 64-bit version of CFITSIO.
The procedure is basically similar to the one above, with a couple of changes:

CFITSIOINCDIR=/usr/local/src/cfitsio/cfitsio/include
LIBDIR=/usr/local/src/cfitsio/cfitsio/lib

Basically you need to point them to the locations where you installed the 64-bit CFITSIO; see the CFITSIO installation note using the link above.  Another change is in the compiler options:

COPTS= -m64 -funroll-loops -O3 -ansi -Wall -I$(CFITSIOINCDIR)


These should be the only difference.



Installing SExtractor 2.8.6 on Mac OS X Leopard
Taro — Tue, 09 Mar 2010 23:37:09 +0000
NOTE: This is an old article and there is a newer one which I believe is more thorough.  Please read that one instead.
This was yet another painful experience trying to find out the best way to compile software with minimal effort on OS X.  Yes, it required such an enormous minimal effort; hence my installation note.  Anyways, SExtractor requires ATLAS and FFTW, both programs of which I don’t really wish to build myself.  I wanted to use pre-compiled binaries, but I ended up doing a hybrid installation after attempts to get it compiled with Fink versions of ATLAS and FFTW failed.
First, I installed FFTW from source.  Just download a tarball into a temporary directory, uncompress, and do the usual make routine:
$ wget http://www.fftw.org/fftw-3.2.2.tar.gz
$ gunzip -c fftw*.gz | tar xvf -
$ cd fftw-3.2.2
$ ./configure --enable-threads
$ make
$ sudo make install
Then I installed the MacPort version of ATLAS.  Downloaded a binary for MacPort, installed it, and then from command line:
$ sudo port install atlas
I assume FFTW was installed at /usr/local, and ATLAS at /opt/local.
Download the SExtractor source into a temporary directory:
$ wget ftp://ftp.iap.fr/pub/from_users/bertin/sextractor/sextractor-2.8.6.tar.gz
$ gunzip -c sex*.gz | tar xvf -
$ cd sextractor-2.8.6
$ ./configure --with-atlas=/opt/local/lib --with-atlas-incdir=/opt/local/include
    --with-fftw=/usr/local/lib --with-fftw-incdir=/usr/local/include
$ make
$ sudo make install
This should make and install under /usr/local/bin an executable file called “sex.”  Totally worth it after wasting all these hours.
PSFex
The stable version of the software appears not public yet, but in addition to above, the single precision version of FFTW needs to be installed.  Go to the FFTW source tree, and redo the make proces with different configuration:
$ make clean
$ ./configure --enable-threads --enable-single
$ make
$ sudo make install
This should install libfftw3f.la, libfftw3f_threads.a libfftw3f_threads.la under /usr/local/lib.
Then simply do ./configure; make; sudo make install under the PSFex source tree.  That should do it.  I’m sure that I’m not linking to other optional libraries (since I didn’t specify with ./configure).  But I’ll add to this if I find them necessary later…



Installing wxPython 2.8 on Debian Lenny
Taro — Mon, 08 Mar 2010 04:06:35 +0000
Actually Debian has the python-wxgtk2.8 package which you can simply apt-get if you wish to have it installed as part of Python 2.5 which is the stable version for Lenny.  I’m adding wxPython 2.8 to Python 2.6, which I installed from source.
I mostly followed what the official wxPython installation guide says.  It was not a smooth installation for me, so this is my own installation note.  The version is 2.8.10.1.
First, download the tarball for wxPython from the official repository.  Here I assume the archive is downloaded to /usr/local/src/wx.
Some libraries need to be installed since wxWidgets will be built against them:
$ sudo apt-get install libgtk2.0-dev libglu1-mesa-dev
You need to first build wxWidgets, and then build extension module for wxPython.  Anyways, decompress and untar, and create a working directory for building wxWidgets:
$ bunzip2 -c wxPython-src-2.8.10.1.tar.bz2 | tar xvf -
$ cd wxPython-src-2.8.10.1
$ mkdir bld
$ cd bld
In the bld directory, create a file named .configure with the following content:
#!/bin/sh
../configure \
 --prefix=/usr/local/lib/wx/2.8 \
 --with-gtk \
 --with-gnomeprint \
 --with-opengl \
 --with-libjpeg=builtin \
 --with-libpng=builtin \
 --with-libtiff=builtin \
 --with-zlib=builtin \
 --enable-optimize \
 --enable-geometry \
 --enable-graphics_ctx \
 --enable-sound --with-sdl \
 --enable-mediactrl \
 --enable-display \
 --enable-unicode \
 --enable-rpath=/usr/local/lib/wx/2.8/lib
Here I am assuming the newly build wxWidgets library will be installed under /usr/local/lib/wx/2.8.  You may change the installation location, of course.  For details on what these switches will do, refer to the official installation guide for wxPython.
Create a file named .make with the following content:
#!/bin/sh
make $* && make -C contrib/src/gizmos $* && make -C contrib/src/stc $*
Give both the files an executable permision, and do the usual thing:
$ chmod +x .configure
$ chmod +x .make
$ ./.configure

...

$ ./.make
$ sudo ./.make install
If this process ends without errors, you are ready to install wxPython. Before you run setup.py, however, you need to patch a source file.  Download this diff file to the top installation directory (i.e., wxPython-src-2.8.10.1) and run the patch:
$ cd ..
$ wget http://devide.googlecode.com/svn/trunk/johannes/patches/wxpython28101_gdiwrap.diff
$ patch -p0 < wxpython28101_gdiwrap.diff
After the patch is applied correctly, move to wxPython directory:
$ cd wxPython
Now, in config.py, you need to find the line that starts with WX_CONFIG and set the variable to the path to wx-config script, /usr/local/lib/wx/2.8/bin/wx-config in our case:
WX_CONFIG = '/usr/local/lib/wx/2.8/bin/wx-config'
Then finally install the wxPython package:
$ python2.6 setup.py build_ext --inplace
$ sudo python2.6 setup.py install
Here I’m running setup.py as a Python 2.6 script since I want to install wx into the site-packages directory for Python 2.6.
$ python2.6
$ import wx
If this gives no ImportError, it is a success.  If you receive an error, check the permission of wx.pth undersite-packages; the file must be world readable (i.e., 644).
To avoid ImportError upon importing wx in Python, you need to add the following line in ~/.bashrc (or ~/.bash_profile):
export LD_LIBRARY_PATH=/usr/local/lib/wx/2.8/lib:${LD_LIBRARY_PATH}
That’s it.  Make sure “import wx” works under python2.6.  If not, well…
TODO
I want to have wx loaded without specifying LD_LIBRARY_PATH.  Using the –enable-rpath option is supposed to do this, but it apparently doesn’t for me.



Installing Python 2.6 on Mac OS X Leopard
Taro — Fri, 05 Mar 2010 22:00:19 +0000
It is such a pain in the ass to deal with this weird hybrid system called Mac OS X, but I want to start migrating my tools to Python 2.6 in anticipation of eventually porting to Python 3.  It didn’t go as smoothly as I wanted, so here is my installation note.
First, download the Mac Installer Disk Image from the official Python website.  The version I use here is 2.6.4.  Just double-click the .dmg file and you can follow the GUI instruction.  Easy enough.
Now, one of the packages that I need to install later requires Tkinter.  After installing Python 2.6.4, check if the Tkinter module loads properly:
$ python2.6
>>> import Tkinter
If you get no error, fine.  However you may be unlucky (like myself) and get an error indicating the Python installation has problem linking to Tk.  If that happens, download and install the latest 8.4 version of ActiveTcl.  After that, the problem should go away.  I wasted a few hours trying to figure this out.  (I wish I could just stick to good old Linux systems…  Oh well.)
The default version of Python in Leopard is 2.5, so at this point both versions should be coexisting.  You can still use version 2.5 by explicitly calling the executable:
$ python2.5  # This calls Python 2.5.
$ python2.6  # This calls Python 2.6.
It appears the Python installer adds “/Library/Frameworks/Python.framework/Versions/2.6/bin:${PATH}” to the PATH environment variable in ~/.bash_profile.  If this is not desired, you need to tweak it there, after installing Python packages where Mac binaries exist (I think you need to ensure that Python 2.6 is used by default when installing binary packages via .dmg; otherwise you may overwrite existing installations for Python 2.5).
Basically this should get you going.  It is a very minimal installation but you can keep adding packages you need in standard manner.  The important thing is to explicitly run setup.py scripts via python2.6, so that the packages get installed under the site-packages directory for Python 2.6 and not for Python 2.5.
Packages for Astrophysics
For the packages where binaries do not exist (e.g., ipython, nose, readline, Pmw, urwid, stsci_python), I installed them from source.  Just made sure that I did “python2.6 setup.py install.”
I installed numpy, scipy, matplotlib, and wxPython (v2.8) using binaries available from their official websites.
So far, there has been no problem.



SExtractor
Taro — Fri, 26 Feb 2010 21:23:13 +0000
Just a reminder for myself:  The SExtractor convention is that the first pixel in a coordinate is indexed by one, not zero.



Cooking Japanese Rice
Taro — Sat, 06 Feb 2010 21:16:46 +0000
Just a reminder for myself on how to cook short-grain rice of sticky kind, with a regular pot.  The amount will be good for three servings or four, depending on how much you like rice.  The exact timing is give or take.

Take 360 mL of rice and wash and rinse repeatedly until water becomes clear.
Fill the pot with the rice and 400 mL of water, and cover with a lid.  Let it sit for about 15 minutes.
Bring the rice to boil, with medium to high heat (with my electric stove, this is 6/8 and takes about 6.5 minutes).
Once boiling, turn down the heat to medium low, until “holes” starts to appear (4/8 and takes about 1.5 minutes).
Bring the heat to low (2/8 and 5 minutes).
Turn the heat to max for 10 seconds to blow off excess water.
Turn the heat off and let it sit for about 15 minutes.
Fluff it up and serve!

That’s it.



scipy.interpolate.bisplrep
Taro — Wed, 20 Jan 2010 23:06:27 +0000
Tried using scipy.interpolate.bisplrep to obtained a smooth interpolated image. It turns out that this function is not capable of handling 2D image when one side is more than 256 pixels. This should be documented in SciPy… Just a note for … Continue reading →


Wacom Intuos Tablet on Debian Lenny
Taro — Sun, 17 Jan 2010 01:46:38 +0000
First, install a package:
# apt-get install wacom-tools


Add the following lines to /etc/X11/xorg.conf:
# Device entries for Wacom Intuous:
Section "InputDevice"
	Identifier	"stylus"
	Driver		"wacom"
	Option		"Type"		"stylus"
	Option		"Mode"		"Absolute"
	Option		"USB"		"on"
	Option		"Threshold"	"10"
	Option		"Tilt"		"on"
	Option		"Device"	"/dev/input/wacom"
EndSection

Section "InputDevice"
	Identifier	"eraser"
	Driver		"wacom"
	Option		"Type"		"eraser"
	Option		"Mode"		"Absolute"
	Option		"USB"		"on"
	Option		"Tilt"		"on"
	Option		"Threshold"	"10"
	Option		"Device"	"/dev/input/wacom"
EndSection

Section "InputDevice"
	Identifier	"cursor"
	Driver		"wacom"
	Option		"Type"		"cursor"
	Option		"Mode"		"Relative"
	Option		"USB"		"on"
	Option		"Threshold"	"10"
	Option		"Device"	"/dev/input/wacom"
EndSection

Section "InputDevice"
	Identifier	"pad"
	Driver		"wacom"
	Option		"Device"	"/dev/input/wacom"
	Option		"Type"		"pad"
	Option		"USB"		"on"
EndSection

# Add InputDevice lines in SeverLayout:
Section "ServerLayout"
	...

	InputDevice	"stylus"	"SendCoreEvents"
	InputDevice	"eraser"	"SendCoreEvents"
	InputDevice	"cursor"	"SendCoreEvents"
	InputDevice	"pad"		""
EndSection


Reboot (or insert wacom module).
Using with GIMP
To use cool features like pressure sensitivity and eraser with GIMP, go to: File -> Preferences -> Input Devices -> Configure Extended Input Devices, and set stylus, cursor, eraser, pad to “Screen.” This should be it!



python-sao
Taro — Mon, 04 Jan 2010 04:35:27 +0000
Man, pysao is extremely useful.  Controlling DS9 from Python has never been easier.  Thank you, whoever is developing the software.



"Line in" doesn't work by default on Leopard
Taro — Tue, 29 Dec 2009 20:56:24 +0000
How lame…
Fortunately there is an application called LineIn which makes possible pass through (sounds coming in at line in goes to line out directly).  Just install it and enable “Pass Thru.”  That’s it.



PyRAF Tips
Taro — Fri, 27 Nov 2009 20:55:06 +0000
The Standard Calling Sequence for an IRAF Task within a Python Script
This is just an example of how to run an IRAF task as if it is part of a Python module using PyRAF.  It ensures that the task parameters are set to their default values, and a user overrides only those parameters which need to be customized for a particular task.


#!/usr/bin/env python
from pyraf import iraf
from pyraf.irafpar import IrafParList


def default_irafpar(task):
    # Return the IrafParList object for default parameters.
    return IrafParList(task.getName(), parlist=task.getDefaultParList())


def main():
    # Load IRAF packages.
    iraf.gemini(_doprint=0)
    iraf.gmos(_doprint=0)

    # Get default parameters and override some.
    yes, no = 'yes', 'no'
    plist = default_irafpar(iraf.gprepare)
    plist.setParList(inimages = 'someimage.fits',
                     rawpath = '/astro/data/',
                     fl_addmdf = yes,
                     mdfdir = '../mdf/',
                     logfile = '',
                     verbose = yes)

    # Run the task.
    iraf.gsprepare(ParList=plist)

    return


if __name__ == '__main__':
    main()









Installing Freemind on Debian Lenny
Taro — Sun, 15 Nov 2009 04:15:48 +0000
Download .deb from Freemind’s sourceforge web page, e.g., freemind_0.8.1-2_all.deb.  Then

# apt-get install sun-java6-jre

# apt-get install libcommons-codec-java libcommons-lang-java libjcalendar-java

# apt-get install libjgoodies-forms-java librelaxng-datatype-java

# dpkg -i freemind_0.8.1-2_all.deb

Add the following line to ~/.bashrc:

export JAVA_HOME=/usr/lib/jvm/java-6-sun

(Without the environment variable set, I could not get freemind to run at all.)  You can then launch it by typing freemind on the shell or choose the program in the menu under office.



KCorrect: kcorrect Python Port Project
Taro — Sun, 02 Aug 2009 21:01:18 +0000
By Taro Sato (taro at ap.smu.ca)
NOTE: This is a page for my old side programming project, on astrophysical computing.  I just moved it here since I haven’t been tracking access and therefore I don’t know if there has been any serious interest in doing k correction on Python rather than IDL.  If anyone is seriously interested in doing k correction with Python, this whole thing can be made into an open source project for communal use (if we can obtain an approval from the original library developer).
KCorrect is an attempt to wrap some of the utilities available in kcorrect written by Michael Blanton into an easy-to-use Python package. kcorrect is a collection of C/IDL codes which implements k corrections and photometric redshifts with broad-band photometry (and much more).
Download Version 0.2.1
Motivation
With NumPy quickly maturing into the standard platform for scientific computing with Python, there should be an increasing interest in carrying out astronomical data analysis on Python in near future.  Python is a well-designed programming language offering astronomers a simple-to-use yet very powerful platform with all the advantages of a modern programming language (e.g., object orientation, exceptions, namespace) as well as the simplicity of a scripting language, not to mention it is FREE.  We (actually I…as of now) hope to contribute to the increase in the usage of Python among astrophysicists.
Conditions of Use
Since KCorrect is simply a Python wrapper to kcorrect (v4_1_4), users are requested to follow the conditions of use described in kcorrect distribution website.
No extra citation is necessary on the use of the Python version.  I changed my mind because there are too many people who are not very sincere.  Please acknowledge the use of this Python version clearly, if you ever use a portion of this program in your published work
Also, please do not modify the program yourself.  If you wish to do that, please consider contributing to the project instead, rather than secretly using it to your liking and don’t even acknowledge somebody else’s effort on which you build your work.
Although we do attemp to ensure that users obtain similar results to the IDL version when desired, bugs are a fact of life; please report bugs and comments to Taro Sato (email address given above) if you suspect the problems occur with the Python version.
General Description
The general concepts of k corrections are well documented in Michael Blanton’s kcorrect distribution website; users of KCorrect are urged to learn the principles behind the software by throughly reading his documentation.  There is also an article written by David Hogg which may be of interest for users of kcorrect.
The KCorrect Python port project is totally a volunteer effort by those who love to code in Python, so we have not implemented all the rich functionalities already available in the IDL version of kcorrect (it is slow going without external contributions; please let me know if you can contribute a tiny bit).
Currently Available Functionalities

Core implementation of k correction
High-level interface for some redshift surveys (so far only SDSS and DEEP)

Installing KCorrect
This version of KCorrect requires the following software to be already installed on your machine:

Python 2.5.2 or better (http://www.python.org/)
NumPy 1.3 or better (http://www.scipy.org/Download/)
SWIG 1.3.36 or better (http://www.swig.org/)

Download the snapshot version here. README.txt included in the tarball has a detailed installation instruction.  If you already have a working installation of the IDL version of kcorrect (which is a requirement), the standard python method
 $ python setup.py install
should simply work.
Usage
Here is an example IPython session:
In [1]: import KCorrect as KC

In [2]: maggie = [1., 4.78, 10.96, 14.45, 19.05]

In [3]: maggie_ivar = [1100., 28., 7.7, 4.4, 2.5]

In [4]: redshift = 0.03

In [5]: filters = KC.FilterList(['sdss_u0.par','sdss_g0.par','sdss_r0.par',
   ...:                          'sdss_i0.par','sdss_z0.par'])

In [6]: kc = KC.KCorrect()

In [7]: kc.set_data(redshift, filters, maggie, maggie_ivar)

In [8]: print kc.kcorrect(band_shift=0.1)
[[-0.38402203 -0.33727303 -0.19066639 -0.15796432 -0.13370053]]
The above example is to be compared to the one given at kcorrect distribution website.  What we wanted to do with the Python version is to hide the part of implementation that users really do not want to deal with (e.g., conversions to “AB maggies,” preserving the projection table or the template spectra, etc.).  A more typical usage goes like this:
In [11]: redshift = [1.029, 0.9581]

In [12]: deepmag = [[23.112, 22.363, 21.690],[24.668, 23.878, 23.248]]

In [13]: kc = KC.KCorrectDEEP(cosmo=(0.3,0.7,1.0))

In [14]: kc.set_data(redshift, deepmag)

In [15]: print kc.absmag()
[[-21.2052002  -21.69138336 -21.85865593]
 [-19.35534096 -19.64371872 -19.71680069]]

In [16]: vega2ab = KC.vega2ab(kc.filter_list)

In [17]: print kc.absmag() - vega2ab
[[-21.0925668  -21.93185531 -22.3164852 ]
 [-19.24270756 -19.88419066 -20.17462996]]

In [18]: bessell = KC.FilterList(['bessell_V.par','bessell_R.par'])

In [19]: print kc.appmag(filter_list=bessell)
[[ 22.75250435  22.36059952]
 [ 24.18490219  23.83894157]]
Note that the classes in KCorrect package are designed so that users need to define the minimal set of input data.  Fine tuning of parameters (e.g., assumed cosmology above) can generally be done using optinal input arguments.  We also implement some auxiliary functions such as:
In [28]: KC.get_available_filters()
which returns the list of all available filter file names.
Documentation
At this point, no fancy documentation is available besides this web page.  We strive to document the software using Python’s native docstrings.  For example, doing
In [29]: help(KC.KCorrectDEEP)
should give you more detailed information about KCorrectDEEP class in KCorrect package.



Setting Up Squid Proxy Server on Mac OS X Leopard
Taro — Thu, 30 Jul 2009 20:47:33 +0000
My goal is to set up a very basic proxy server on my Mac box on campus, so that I can have full access to subscription-based academic journals via the proxy on my laptop even when I am off campus.  Some schools provide such (library) proxies but my school unfortunately does not.  I want to set it up such that the proxy requires a password authentication in order not to make it wide open to the public.
Getting and Installing Squid
Download a tarball for a stable version from the repository.  The version that I use here is 2.7.  I assume the file is downloaded to /usr/local/src/squid.
$ cd /usr/local/src/squid
$ gunzip -c squid-2.7.STABLE6.tar.gz | tar xvf -
$ cd squid-2.7.STABLE6
$ ./configure
$ make
$ sudo make install
$ cd helpers/basic_auth/NCSA
$ make
$ sudo make install
$ sudo chown -R nobody /usr/local/squid/var
$ sudo /usr/local/squid/sbin/squid -z
Squid will be installed at /usr/local/squid.  (The last command is necessary to run a daemon as user nobody.)
Configure Squid
First, prepare a NCSA-compliant encrypted password file for a user (here with username johndoe):
$ cd /usr/local/squid/etc
$ sudo touch squid_passwd
$ sudo chmod o+r squid_passwd
$ sudo htpasswd squid_passwd johndoe
New password:
Re-type new passwod:
Adding passwod for user johndoe
Now, edit /usr/local/squid/etc/squid.conf.  The following lines need to be added:
# Add this to the auth_param section
auth_param basic program /usr/local/squid/libexec/ncsa_auth /usr/local/squid/etc/squid_passwd

# Add this to the bottom of the ACL section
acl ncsa_users proxy_auth REQUIRED

# Add this at the top of the http_access section
http_access allow ncsa_users
Finally, run the server:
$ sudo /usr/local/squid/sbin/squid -N -d 1 -D
Firewall will prompt me to see if I allow incoming connections to squid.  Say “allow.”
The IP address or host name of your Mac box at the port 3128 will be available as a proxy server now.
Launch Squid on Startup with launchd
Under the directory /Library/LaunchDaemons, create a file named squid.plist with the following content:



  
    Label
    squid
    OnDemand
    
    ProgramArguments
    
      /usr/local/squid/sbin/squid
      -N
      -d 1
      -D
    
    ServiceIPC
    
  

Then issuing
$ sudo launchctl load -w /Library/LaunchDaemons/squid.plist
will launch squid.  On reboot, the proxy  should also be working automatically.
Launch Squid on Startup with SystemStarter
This method should be ignored in favor of the method with launchd described above.  This one is incomplete anyways…
This is a server so it would be convenient if the proxy starts up upon reboot automatically.  Here is a Mac way to do it:
$ sudo mkdir /Library/StartupItems/squid
$ sudo touch /Library/StartupItems/squid/squid
$ sudo touch /Library/StartupItems/squid/StartupParameters.plist
$ sudo chmod +x /Library/StartupItems/squid/squid
The newly created files should have contents as follows.
squid:
#!/bin/sh

. /etc/rc.common

StartService()
{
 ConsoleMessage "Starting squid"
 /usr/local/squid/bin/RunCache &
}

StopService()
{
 ConsoleMessage "Stopping squid"
 # TODO: add a command to stop squid
}

RestartService()
{
 ConsoleMessage "Restarting squid"
 # TODO: add a command to restart squid
 StopService
 StartService
}

RunService "$1"
StartupParameters.plist:



 
 Description
 squid
 Provides
 
 squid
 
 Requires
 
 Network
 
 OrderPreference
 Last
 

References
Quick HOWTO: Ch32 : Controlling Web Access with Squid – Linux Home Networking
Update History
March 22, 2010: A minor error on the permission of “var” directory corrected.



most (a pager application) Binary for Mac OS X Leopard
Taro — Thu, 16 Jul 2009 20:19:24 +0000
My favorite pager is most, not more or less, so I build a binary. Simply gunzip and put it in your bin directory. Download MOST version 5.0.0 (S-Lang version pre2.2.0-121) for Mac OS X Leopard. No warranty of any kind … Continue reading →


Logitech MX Revolution in Mac OS X Leopard
Taro — Thu, 09 Jul 2009 19:27:10 +0000
I want to use this mouse as if it is a 3-button mouse, such that within X a middle click pastes the content in the clipboard.
I find the Logitech Control Center (LCC) for Mac OS X to work just fine.  After installation, it will show up under System Preferences under the category labeled “Other”.
First, in order to make One-Touch Search button (that’s a button right below the vertical scroll wheel) as the third button, plug in your Apple Mighty Mouse, go to System Preferences -> Keyboard & Mouse -> Mouse, and select the middle button to act as “Button 3″.  Then unplug the Mighty Mouse and plug in your MX Revolution again.  Awkward, but it works…



Miscellaneous Mac OS X Leopard Tips
Taro — Sun, 21 Jun 2009 20:54:07 +0000
Here is a list of miscellaneous tips when configuring my new iMac to my liking.
Disable Automatic Login & Password Protect the System
Apple Menu -> System Preferences -> Security -> General
Check the following: “Require password to wake this computer from sleep or screen saver” and “Disable automatic login.”
Firefall
Apple Menu -> System Preferences -> Security -> Firewall
and choose “Allow only essential services” for the maximum protection.  Use by-application/service option for more flexibility, when allowing remote login via ssh, for example.
Remote Login with SSH
Apple Menu -> System Preferences -> Sharing
Check “Remote Login.”  Don’t forget to reconfigure the firewall to allow ssh (see above).
Right Clicking with Apple Mouse
Apple Menu -> System Preferences -> Keyboard & Mouse -> Mouse
and specify “Secondary Button” for the right side of your Apple Mouse.
Japanese Input Method
Apple Menu -> System Preferences -> International -> Input Menu
and check the box for Kotoeri.  You can change the input method by clicking on a country-flag icon in the menu bar.
Using Clipboard Contents in Terminal
Use pbpaste and pbcopy commands.
For example, once you put text into the clipboard in your OS X application, do:
$ pbpaste > pasted.txt


This will place the content of the clipboard into the file pasted.txt.



Setting Up Astro Research Environment on Mac OS X Leopard
Taro — Sat, 20 Jun 2009 01:27:56 +0000
In my new research environment I have an iMac.  My preferred solution is Linux, but I figured that sometimes learning new things is a good thing (though I have to sacrifice my efficiency which I developed over the years using a Debian-based Linux box).
There are nice existing resources, like

OS X for Astronomers
Setup a New Mac for Astronomy

so what I leave here is a quickie for myself based on the above sites.
X11
X11 comes with Leopard.  Just drag the X11 icon at /Applications/Utilities/X11 to your dock, so that clicking on it you have an X11 environment in which you can open up a shell terminal (Applications -> Terminal).
Xcode Tools
This one includes all the necessary development applications.  Install these from the Mac OS X installation DVD.  Xcode Tools can be found under the Optional Installs directory.
Fink
As an apt-get afficionado, I like the way I can use the APT-based packaging system on Mac OS X.  Basically you can use the apt-get command the way you do on Debian.  Go to the Fink project web site and download and install the software.  You can browse all the available packages as well.  Nice.
Software for Astrophysics
Use Scisoft for Mac OS X to install a bunch at once.  People are lazy but that’s how innovations arise, I suppose.  The package will be installed at /Applications/scisoft.
IRAF
As of this writing, the version of IRAF is 2.14.  Applying 2.14.1 patch is actually easy by overwriting:

Download IRAF binaries from ftp://iraf.noao.edu/iraf/v214/PCIX/ib.macx.x86.gz to /Applications/scisoft/all/Packages/iraf/irafbin/bin.macintel and just extract the archive with gzip and tar.
Download NOAO binaries from ftp://iraf.noao.edu/iraf/v214/PCIX/nb.macx.x86.gz to /Applications/scisoft/all/Packages/iraf/irafbin/noao.bin.macintel and extract the archive with gzip and tar.
Download the patch from ftp://iraf.noao.edu/iraf/v214/PCIX/patch1.tar.gz to /Applications/scisoft/all/Packages/iraf/iraf and extract the archive with gzip and tar.

It is also recommended to check the versions of external packages.  If outdated, you can simply install them under /Applications/scisoft/all/Packages/iraf/extern.
If you are reusing login.cl file from a Linux installation, make sure to update your home environmental variable:
set home = “/Users/username”
(in Linux, it would be “/home/username” of course.)
IMPORTANT: Also, do not forget to do this:

$ cp /Applications/scisoft/all/Packages/iraf/iraf/dev/imtoolrc ~/.imtoolrc

Otherwise you may see DS9 not behaving well…
DS9
Download from here and install the Leopard X11 version.
Others

OpenOffice.org
GIMP

TO BE CONTINUED.  Still need to install quite a few packages via apt-get.
sudo apt-get install wget emacs22 gv less



Microsoft Natural Ergonomic Keyboard 4000 on Mac OS X Leopard
Taro — Fri, 19 Jun 2009 22:54:16 +0000
I just wish the keyboard to work fairly close to iMac keyboard.  Windows (labeled Start) and Alt keys are “flipped,” which is annoying.  Following this article, I set up the keyboard.  Basically
Apple -> System Preferences -> Keyboard & Mouse (under Hardware submenu) -> Keyboard tab -> Modifier Keys
Select keyboard should be “Natural Ergonomic Keyboard 4000″ and you just need to swap the Option and Command modifier key bindings.  That’s it.
However, the above change makes the key locations resemble that of Mac.  I am too used to the PC culture of using Ctrl + some key, and my productivity suffers when I cannot position my left hand that way.  Roughly speaking, the Command key on Mac corresponds to the Ctrl key on PC, so I decided to swap them as well:

This works much better for me.
There are quite a few special keys that do not work the ways they do on Windows, but I don’t use them much anyways so it’s good for now.  I’ll keep adding instructions as I find them on the web.



More Japanese Fonts
Taro — Wed, 22 Apr 2009 20:50:49 +0000
A few collections of web sites listing stylistic Japanese fonts for download:

フリーフォント最前線
脱力系フォントのまとめ
無料で使える日本語フォント11選

Some are free, others are not.
My favorite is 国鉄風フォント, which resembles the characters used by kokutetsu, the Japanese national railway system (now JR and privatized).  It has a very retro feel that I like.



Astro Software in Debian Lenny
Taro — Tue, 21 Apr 2009 00:44:49 +0000
Here my goal is to build a computing environment where Python is the glue of all the astro-related tools (PyRAF, stsci_python, etc.).
IRAF
In order to use PyRAF, IRAF needs to be installed first.
Python
At the very least, I want the basic Python packages related to science/data analysis:
# apt-get install python ipython python-doc
# apt-get install python-numpy python-scipy python-matplotlib

PyRAF
Install several Python packages:
# apt-get install tk8.4-dev python-tk libf2c2-dev
# apt-get install python-dev python-pmw python-urwid

Test if the required Python modules exists by making sure that no ImportError occurs below:
# python
>>> import readline
>>> import Tkinter
>>> Tkinter._test()
>>> import Pmw
>>> import urwid

If you have an installation of previous version, check the section Cleaning An Old Installation of the installation instruction at the STSci web site to make sure things are clean.
Prepare the source directory:
# mkdir /usr/local/src/python_packages/stsci_python
# cd /usr/local/src/python_packages/stsci_python

Download and extract the source:
# wget http://stsdas.stsci.edu/download/stsci_python_2.7/stsci_python_2.7.tar.gz
# gunzip -c stsci_python_2.7.tar.gz | tar xvf -
# cd stsci_python_2.7/

Normally, using python setup.py install will do the job, but it probably ends in compilation error, complaining the header file cfunc.h not found.  In order to force the installation, modify setup.py in the current directory, so that at the very end of the file an additional include directory is defined:
distutils.core.setup(

    # This name is used in various file names to identify this item.
    name="stsci_python",

    # This version is expected to be compared to other version numbers.
    # It will also appear in some file names.
    version="2.7",

    # Apparently, description is not used anywhere.
    description="",

    packages = all_packages,
    package_dir = all_package_dir,
    ext_modules = all_ext_modules,
    scripts = all_scripts,
    data_files = all_data_files,
    include_dirs = ['/usr/lib/python2.5/site-packages/numpy/numarray/numpy']
)

Then we can follow the normal instruction:
# python setup.py install
# cd stscidocs
# python setup.py install

Before running PyRAF for the first time, clear the cache by removing directories under your “pyraf” directory, which is located in where you have your loginuser.cl for IRAF.
That’s it!



IRAF on Debian Lenny
Taro — Fri, 17 Apr 2009 22:54:36 +0000
Here is my installation note for IRAF (ver 2.14.1) on Debian Lenny.
Basically, the installation procedure does not differ much from the one outlined in the official IRAF installation guide.
First, create “iraf” user account to the system if it does not exist already.  Also, install the C shell (csh) as it appears to be the preferred shell for IRAF (people apparently don’t know when to move on).
$ sudo apt-get install csh ncompress
$ sudo adduser iraf
After user “iraf” has been added to the system, change its home directory and the default shell by modifying /etc/passwd to read:
iraf:x:1002:1002:IRAF Admin,,,:/iraf/iraf/local:/bin/csh
Note that the number 1002 may be different depending on how many users already have the accounts on the system.  Only change the last two entries delimited by colon to /iraf/iraf/local and /bin/csh.
We assume IRAF will be installed under /usr/local/iraf, and a symlink /iraf points to that directory:
# mkdir /usr/local/iraf
# chown iraf /usr/local/iraf
# chgrp iraf /usr/local/iraf
# ln -s /usr/local/iraf /iraf
Login as user “iraf,” and create a bunch of directories:
# su iraf
% cd /usr/local/iraf
% mkdir extern imdirs iraf irafbin x11iraf
% mkdir irafbin/bin.linux
% mkdir irafbin/noao.bin.linux
Install x11iraf.  You may use the pre-built binary distribution:
% cd /usr/local/iraf/x11iraf
% wget http://iraf.noao.edu/x11iraf/x11iraf-v2.0BETA-bin.redhat.tar.gz
% gunzip -c x11* | tar xvf -
% su
# ./install
# exit
and select defaults for all the user input upon installation.  (Note that if xgterm quits on an error complaining about “ptys,” then you may need to turn on the kernel configurations CONFIG_LEGACY_PTYS and CONFIG_UNIX98_PTYS; yes, you may need to rebuild the kernel…)
Install the source:
% cd /usr/local/iraf/iraf
% wget http://iraf.noao.edu/iraf/ftp/iraf/v214/PCIX/as.pcix.gen.gz
% gunzip -c as.pcix.gen.gz | tar xvf -
Install the binaries:
% cd ../irafbin/bin.linux
% wget http://iraf.noao.edu/iraf/ftp/iraf/v214/PCIX/ib.lnux.x86.gz
% gunzip -c ib.lnux.x86.gz | tar xvf -

% cd ../noao.bin.linux
% wget http://iraf.noao.edu/iraf/ftp/iraf/v214/PCIX/nb.lnux.x86.gz
% gunzip -c nb.lnux.x86.gz | tar xvf -
% exit
Run the install script:
# su iraf
% cd $iraf/unix/hlib
% source irafuser.csh
% su
# ./install
Follow the instruction for the most part, but the following are not necessary: IRAF networking and tapecap device.
# exit
% cd
% source .login
% rehash
% cl
This should start ecl.
For exposing external packages (which must already be installed if not already) within IRAF, the text file extern.pkg can be reused if these packages are already installed.  Copy it from a previous installation to /usr/local/iraf/iraf/unix/hlib.
# External (non core-system) packages.  To install a new package, add the
# two statements to define the package root directory and package task,
# then add the package helpdb to the `helpdb' list.

reset	noao		= iraf$noao/
task	noao.pkg	= noao$noao.cl

reset	tables		= /iraf/extern/tables/
task	tables.pkg	= tables$tables.cl

reset	mscred		= /iraf/extern/mscred/
task	mscred.pkg	= mscred$mscred.cl

reset	stsdas		= /iraf/extern/stsdas/
task	stsdas.pkg	= stsdas$stsdas.cl
task	apropos		= stsdas$apropos.cl

reset	rvsao		= /iraf/extern/rvsao/rvsao-2.5.0/
task	rvsao.pkg	= rvsao$rvsao.cl

reset	xdimsum		= /iraf/extern/xdimsum/
task	xdimsum.pkg	= xdimsum$xdimsum.cl

reset	fuzzy		= /iraf/extern/fuzzy/
task	$fuzzy.pkg	= fuzzy$fuzzy.cl

set	crrefer		= "/home/astro_data/stsci_data/"

reset   helpdb          = "lib$helpdb.mip\
                          ,noao$lib/helpdb.mip\
                          ,tables$lib/helpdb.mip\
                          ,stsdas$lib/helpdb.mip\
                          ,mscred$lib/helpdb.mip\
                          ,rvsao$lib/helpdb.mip\
			  ,xdimsum$lib/helpdb.mip\
                          "

keep

You can simply comment out lines with “#” if some packages are not installed.
The packages stsdas and tables should be installed following the official instruction.



Favorite Internet Radio Stations
Taro — Fri, 10 Apr 2009 20:50:01 +0000
For funk and soul from 70s & 80s, my favorites are:
www.soulfm.de
Very nice funky music from 70s and 80s.  DJ Joerg spins a variety of stuff from disco to 80s contemporary R&B.  I can always listen to a few songs that I haven’t gotten to know which is cool.
Every Friday, 1pm – 3pm (UTC -7)
Jammin’ A Little Old School (KCSB 91.9 FM Santa Barbara)
A lot of bad jams like Rick James, Cameo, Shock, etc., etc.  It was quite a serendipitous find as it was the college radio station of my old university.
Every Friday, 6pm – 8pm (UTC-7)
Looking to find more of funk, soul, and boogie…



Beautiful (or Cool) Japanese Words?
Taro — Fri, 10 Apr 2009 06:31:47 +0000
I’m currently on a personal mission to find a Japanese word to describe my personal operations.  I want to find a word that sounds and look beautiful and/or cool in Japanese as well as in English (when it’s spelled out in alphabets).
Eventually I hope to use it as a (part of) domain name (which will be my Web portal for my freelance work such as writing, coding, etc.), but the search has actually been quite slow and difficult.  Not surprisingly, many suitable words are already taken by someone else.
Hmmm.  I will keep listing candidates here.

okome (お米)

Just because I love rice…


more to come

Any ideas…?



I Was a Great TA
Taro — Fri, 10 Apr 2009 01:19:19 +0000
and I have a letter from my student to prove it:
Return-Path:
Received: from web10908.mail.yahoo.com (web10908.mail.yahoo.com [216.136.131.44])
	by ilpostino.physics.ucsb.edu (8.11.6/8.11.2) with SMTP id g9RIiBH25703
	for ; Sun, 27 Oct 2002 10:44:11 -0800
Message-ID:
Received: from [128.111.97.165] by web10908.mail.yahoo.com via HTTP; Sun, 27 Oct 2002 10:44:12 PST
Date: Sun, 27 Oct 2002 10:44:12 -0800 (PST)
From: mary watson
Subject: grading our first lab
To: taro@physics.ucsb.edu
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-157626628-1035744252=:40556"

--0-157626628-1035744252=:40556
Content-Type: text/plain; charset=us-ascii


Hello Taro Sato,

     I am a student in one of your labs. Im writing this email to let you know
that I feel you made a huge mistake grading our first lab. You never told us
how you wanted us to write the lab. Most of us were misled into thinking that
all we needed to do was answer the questions in the lab write up. This was not
our fault, and therefore, our grades should not reflect upon your mistake.
Curving our grades in the end is not nearly sufficient enough. At the very
least, you could throw that grade out or let us rewrite our first lab. You are
nothing less than a complete coward and a bastard if you decide to pass even a
shread of the responsibility for your mistake onto us.



---------------------------------
Do you Yahoo!?
Y! Web Hosting - Let the expert host your web site




Updating Flash BIOS off a Bootable CD
Taro — Fri, 10 Apr 2009 00:34:25 +0000
Note: These days, computers ship without floppy drives by default, so there may exist a better generic procedure for updating flash BIOS.  This instruction is just kept for a historical reason.
Flashing BIOS can be dangerous.  A typical disclaimer of the process applies.
First, download a DOS bootdisk image (win98-boot.img — missing for now while I find a semipermanent online disk space — can be useful) and the BIOS update program, e.g., BIOSUPDT.EXT, from the BIOS vendor to a directory (say ~/tmp for an illustration).
Mount the DOS image and copy the BIOS update program as follows:
# cd ~/tmp
# mkdir mnt
# mount -o loop -t vfat win98-boot.img mnt
# cp BIOSUPDT.EXE mnt
# umount mnt

Make a CD ISO image:
# mkisofs -o image.iso -b win98-boot.img win98-boot.img

Burn the ISO image to a blank CD.  Boot from the CD and you should simply run the BIOS update program from DOS.



Search for Files Containing a Text
Taro — Fri, 10 Apr 2009 00:14:20 +0000
Here is a little BASH script (which I name findstr) to search for the files containing a specified string:
#!/usr/bin/env bash
##############################################################################
# findstr
#
# Search for files containing a specified string.  Show the file path,
# line number, and the line itself containing a specified string.
#
# EXAMPLE:
#
# To search for all the files with the file extension .html containing
# a string "body" under the current directory and all its
# subdirectories, do:
#
#   $ findstr . "*.html" "body"
#

find $1 -name \*$2 -exec grep -nH "$3" {} \;



DEEP2 Pipeline Tips
Taro — Fri, 10 Apr 2009 00:03:36 +0000
Custom Extraction
To run the DEEP2 pipeline with boxcar extraction with a specified sigma (4.0 here), do:
$ echo 'boxcar_domask,nlsky=0,nsig_boxcar=4.0' | idl -32 > output.log

Alternatively, use boxsprof_domask to use the boxsprof extraction algorithm.



IRAF Tips
Taro — Thu, 09 Apr 2009 23:59:39 +0000
Printing Out Help
When printing out a help for an IRAF task name iraftask, you may want to save papers by multicolumn printing using a2ps.
cl> help iraftask | type dev=text > iraftask.txt
cl> !a2ps iraftask.txt -2 --medium=letter -o iraftask.ps
The generated .ps file should be printed out.  Note that this doesn’t appear to work well on PyRAF, in which escape sequences do not appear to be parsed.



Changing Page Size of a PostScript Document
Taro — Thu, 09 Apr 2009 23:55:44 +0000
For scaling a postscript page size, do:
$ psresize -Pa4 -pletter src.ps > dst.ps

To generate a letter-sized output as a PDF document, do:
$ dvips -t letter doc.dvi -o doc.ps
$ ps2pdf14 -sPAPERSIZE=letter doc.ps




Reducing the Number of Pages of a PostScript Document
Taro — Thu, 09 Apr 2009 23:53:03 +0000
To save papers, sometimes it is desired to to collect multiple pages into a page on printing a postscript file.  Use mpage command to generate a new postscript file:
$ mpage -2 -c paper.ps > paper_2pg.ps
$ lpr -Pprinter paper_2pg.ps




Merging Two Text Files Line by Line
Taro — Thu, 09 Apr 2009 23:48:28 +0000
Say you have two text files (text1.txt and text2.txt) with data:
# x     y
1.0   1.0
2.0   4.0
3.0   9.0

# x     z
1.0   1.0
2.0   8.0
3.0  27.0

If you wish to merge these files line by line, use paste:
$ paste text1.txt text2.txt > merged.txt

This will create a text file like this one:
# x     y   # x     z
1.0   1.0   1.0   1.0
2.0   4.0   2.0   8.0
3.0   9.0   3.0  27.0




Mirror a website using wget
Taro — Thu, 09 Apr 2009 23:44:33 +0000
To mirror a website (by which I here mean downloading a copy of directory hierarchy) below a certain directory level, use wget.  For example, let’s say you want to mirror everything below some directory in a web site: http://site/a/b/.  Then the directory “b” is two levels below the top most level, so you need to specify “2″ for --cut-dirs option.  Hence
$ wget -m -E -nH -np --cut-dirs=2 http://site/a/b/
should download all the downloadable contents below http://site/a/b/ to the current local directory.



My Favorite Firefox Plugins
Taro — Thu, 09 Apr 2009 22:24:53 +0000
Note: The current version of Firefox on my box is 3.0.6 (that’s the Debian stable as of now).
goo Dictionary
This is an add-on for using dictionaries of English and Japanese.  This will add search interfaces to the navigation toolbar and the context menu .  The dictionaries at goo.ne.jp are quite decent, so highly recommended.



Sawicki & Thompson 2006, ApJ, 648, 299
Taro — Sat, 28 Mar 2009 02:42:39 +0000
Keck Deep Fields. III. Luminosity-dependent Evolution of the Ultraviolet Luminosity and Star Formation Rate Densities at z~4, 3, and 2

Glaxy UV luminosity density peaks at z = 1 ~ 3, sub-L* galaxies dominate the integrated UV luminosity
UV luminosity declines for z > 4
Not enough ionizing photons for reionization?

Madau et al. (1999) criteria may be too stringent?



Check:

Effective volume technique
Evolving faint-end slope confirmed?
Getting rest-frame 1700 A flux




File Manipulation in Linux
Taro — Sat, 28 Mar 2009 02:24:08 +0000
In this page I will keep the list of useful file manipulation commands that I often forget.
Making & Expanding a .tar.gz or .tar.bz2 File
For archiving a directory named foo as foo.tar.bz2 file:
$ tar cvf - foo | bzip2 -9 > foo.tar.bz2

(-9 is the compression level and can be omitted.) To expand this file:
$ bunzip2 -c foo.tar.bz2 | tar xvf -

To use gzip, replace bzip2 with gzip and bunzip2 with gunzip, and change the file extension accordingly (to .gz).



Running Scripts at Login or Logout of a KDE Session
Taro — Sat, 28 Mar 2009 02:14:05 +0000
Here is the situation:  You want to run a customized script when you login or logout of a KDE session.
Here is the solution: You can place a custom script in the directory ~/.kde/Autostart to run when you login to a KDE session. Likewise, you can place the script in the directory ~/.kde/shutdown to run when you logout of a KDE session. The latter directory does not appear to be created by default. If it does not exist, you can just create it by
$ mkdir ~/.kde/shutdown

One thing about running scripts out of ~/.kde/shutdown directory is that it does not recognize all the customization done in your BASH startup scripts, like ~/.bashrc. You may source these files within the scripts in ~/.kde/shutdown, but another approach is to make a script with the contents:
#!/bin/bash
DO SOMETHING BEFORE LOGOUT
# Launch KDE logout interface
dcop kdesktop KDesktopIface logout
DO AFTER LOGOUT

On the KDE panel, select “Add -> Special Button -> None-KDE Application” to create a button pointing to this script. This way, you can run commands assuming all the customization still intact (before logout), and even after you actually logout (as if you put scripts in ~/.kde/shutdown).



Formatting a USB Drive with ext4 for Use with Linux
Taro — Sat, 28 Mar 2009 01:52:13 +0000
I wish to use an external USB drive exclusively on a Linux system (in my case, Debian).  I want to use the ext4 filesystem.  Note that this method wipes out the filesystem on the drive.  
Install hotplug-type package such that upon plugging the USB drive in the device is attached to somewhere in the system.  I use KDE and it automatically prepares such an environment.  When a USB drive is plugged in, I should see a message like this one in the system log:
$ sudo tail /var/log/messages
Dec 16 02:07:51 kernel: [ 3000.544361] scsi11 : usb-storage 3-2:1.0
...
Dec 16 02:07:56 mochi kernel: [ 3005.882879]  sdb: sdb1
...
or I may see something like this instead (I’m using USB 3.0 adapter now):
$ sudo tail -n 100 /var/log/messages | grep sd
...
May  3 12:38:33 mochi kernel: [   12.773057] sd 6:0:0:0: [sdb] Spinning up disk...
May  3 12:38:39 mochi kernel: [   18.782284] sd 6:0:0:0: [sdb] 3906963456 512-byte logical blocks: (2.00 TB/1.81 TiB)
May  3 12:38:39 mochi kernel: [   18.783052] sd 6:0:0:0: [sdb] Write Protect is off
May  3 12:38:39 mochi kernel: [   18.794337]  sdb: sdb1
May  3 12:38:39 mochi kernel: [   18.795569] sd 6:0:0:0: [sdb] Attached SCSI disk
Here, I find the device is at /dev/sdb and there is only one partition, /dev/sdb1, for example.  I can also see all the disk devices as follows:
$ sudo fdisk -l

Disk /dev/sda: 80.0 GB, 80026361856 bytes
240 heads, 63 sectors/track, 10337 cylinders
Units = cylinders of 15120 * 512 = 7741440 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xbd3cc0bb

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1       10338    78149633    5  Extended
/dev/sda5               1          46      340992   83  Linux
/dev/sda6              46          52       48128   83  Linux
/dev/sda7              52         698     4881408   83  Linux
/dev/sda8             698        1086     2928640   83  Linux
/dev/sda9            1086        1861     5858304   83  Linux
/dev/sda10           1861        2830     7323648   83  Linux
/dev/sda11           9305       10338     7812096   82  Linux swap / Solaris
/dev/sda12           2830        9304    48949248   83  Linux

Partition table entries are not in disk order

Disk /dev/sdb: 2000.4 GB, 2000365289472 bytes
255 heads, 63 sectors/track, 243197 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0005f107

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1      243198  1953480704    7  HPFS/NTFS

When the USB drive is new, the partition is most likely formatted in FAT or NTFS unless the drive is preformatted specifically for OS X.  Now, prepare the partition for Linux:
$ sudo fdisk /dev/sdb
See help for detail by pressing “m” on the fdisk command shell.  For example, press “p” to see the list of partitions in the disk.  To create a new one press “n” (probably after deleting one or more partitions by pressing “d”).  Finally press “w” to actually make changes to the drive.  To use ext4 filesystem, do:
$ sudo mkfs.ext4 /dev/sdb1
after exiting fdisk.
If I always mount this drive on boot, add a line like this to /etc/fstab:
/dev/sdc1 /media/usbdrive auto user,noauto 0 0
assuming /media/usbdrive is the mount point.  If I use KDE, this sort of thing is no longer necessary; I can easily manage external drives with Device Notifier.
Update History
May 3, 2012 — Updated for my current system.

December 17, 2010 — Modified to use ext4 filesystem.



Olympus C-750 on Debian Etch
Taro — Sat, 28 Mar 2009 01:42:48 +0000
Note: This may be an obsolete method of using the digital camera with Debian (I would use udev but otherwise let KDE handle it as a simple USB device).  I’m keeping this for record.
The kernel (2.6.x) must have USB_STORAGE (M), USB_HID, and USB_DEVICEFS.  Options SCSI (M) and BLK_DEV_SD (M) under SCSI device support are also needed.
Add usb-storage to /etc/modules file for automatic startup on boot.  Then
# apt-get install sg3-utils hotplug
# tail /var/log/messages

to see where the digital camera is located.  Use that to add an entry to /etc/fstab:
/dev/sda1    /media/camera       vfat     noauto,user,umask=000    0    0

Connect the camera to a USB port, and turns it on.  Reboot the machine. The camera can be mounted using
$ mount /media/camera




GParted, Partition Magic Replacement
Taro — Sat, 28 Mar 2009 00:41:40 +0000
For disk partitioning, my Partition Magic replacement for Linux is GParted.  I rarely need this now that I don’t use Windows much any more.



Renewing My Passport
Taro — Tue, 24 Mar 2009 00:43:49 +0000
All the info appears to be listed here.  For this one I have to visit the consulate general of Japan in person.  How inconvenient.



Getting a Work Permit for Canada
Taro — Tue, 24 Mar 2009 00:39:45 +0000
This page appears to have all the information.
Looks like I don’t have to travel down to LA just to submit my application, which is good.



WBC: Japan defeats USA
Taro — Mon, 23 Mar 2009 07:21:08 +0000
Japan 9, USA 4.
I still think the team USA will come out on top if the game is played in the MLB style: A long season with more than 150 games or so.  At the highest level, the best team in baseball often wins only less than 60% of the time.  The outcome wouldn’t be surprising even if the team USA were in fact the best team.  The effect of chance gets magnified.
However, it is true that you don’t have to watch MLB to appreciate the best of baseball any more.  When I was a kid, MLB seemed out of reach.  It was unthinkable that a Japanese team would play well against MLB players.
Not any more.
And it is actually kind of sad that we have lost a lofty target and goal to follow and pursue.  It is even sadder that this is happening when the popularity of baseball is actually dwindling in Japan.



Simple Firewall with iptables on Debian
Taro — Mon, 23 Mar 2009 01:04:43 +0000
Install iptables:
# apt-get install iptables

First, make a set of iptables rules and save it to a file (say, /etc/firewall-rules.sh):
# BEGIN /etc/firewall-rules.sh
#!/bin/sh

iptables -F

iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
iptables -P FORWARD DROP

iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport ssh -j ACCEPT
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport rsync -j ACCEPT
iptables -A INPUT -j DROP
# END /etc/firewall-rules.sh

Modify /etc/network/interfaces such that the firewall script gets executed when the network connection gets established:
iface eth0 inet dhcp
  pre-up /etc/firewall-rules.sh eth0

Do the security checkup with ShieldsUP! by Gibson Research Corporation to see if the firewall is working.



IDL v6.0.3 on Debian Etch
Taro — Mon, 23 Mar 2009 00:46:10 +0000
(Note: This is an old installation note.)
Install version 6.0 as described in the installation guide (i.e., install in /usr/local/rsi).  Since v6.0 is in the archive, you need to contact the customer support to obtain a link to the download.  Also ask for v6.0.3 patch, which is a separate download.
After installing IDL, you need to install the license manager. lmgrd needs accss to a temporary location, so do:
# sudo mkdir /usr/tmp

Store the license file to /usr/local/rsi/license/license.dat.
Make a script that runs lmgrd:
#!/bin/bash
LD_ASSUME_KERNEL=2.4.1
export LD_ASSUME_KERNEL
/usr/local/rsi/idl/bin/lmgrd > /tmp/idl_lmgrd.log

Copy the above script (call it run_lmgrd) to /etc/init.d, and make a symlink to it in /etc/rc2.d/S20run_lmgrd.  After rebooting (or running the script manually), the license should be honored.
Obtain the IDL Astronomy User’s Library (file: astron.tar.gz) and install .pro files into /usr/local/lib/idl/astrolib/pro
Adding the above path to IDL_PATH enables the use.
Recycle and store .idlenv and .idlstartup into the user’s home directory.
Add lines
. /usr/local/rsi/idl/bin/idl_setup.bash
. /home/taro/.idlenv
export LM_LICENSE_FILE=/usr/local/rsi/license/license.dat

to ~/.bashrc.
Install the v6.0.3 patch following the supplied instruction manual.



Wine on Debian Lenny
Taro — Mon, 23 Mar 2009 00:40:43 +0000
To install Wine, do
# apt-get install wine

To launch a Windows Explorer-like file browser, do
$ winefile

in which you can manage installers and programs just like in Windows.
Installing IEs4Linux
Follow the instruction here.



Microsoft Natural Ergonomic Keyboard 4000 on Debian Lenny
Taro — Mon, 23 Mar 2009 00:34:21 +0000
Unfortunately not all the “special keys” can be enabled at this point; the zoom bar in the middle being notable example.  For my own use it really doesn’t matter since I rarely make use of them.
Download the .deb package file from the xkb-data repository for Squeeze.  Then

$ sudo dpkg -i xkb-data_1.7-2_all.deb

Now under the K menu, go to Control Center -> Regional & Accessibility -> Keyboard Layout -> Layout.  Choose Microsoft Natural Wireless Ergonomic Keyboard 7000 as the Keyboard model. Restart X.
Obsolete Way
An approach here is to use an APT pinning to get a later version of the xkb-data package from the testing repository.  That includes the keyboard layout data for MS Ergonomic Keyboard 7000, which we can use.
First, add a testing repository to /etc/apt/sources.list:

deb http://ftp.us.debian.org/debian lenny main contrib non-free
deb-src http://ftp.us.debian.org/debian lenny main contrib non-free

deb http://www.debian-multimedia.org/ lenny main

deb http://security.debian.org/ lenny/updates main contrib non-free
deb-src http://security.debian.org/ lenny/updates main contrib non-free

# For a few packages including: MS Natural Ergo Keyboard 4000
deb http://ftp.us.debian.org/debian testing main contrib non-free


In /etc/apt/preferences, have the following lines:

Package: *
Pin: release v=5.0*
Pin-Priority: 700

Package: xkb-data
Pin: release a=testing
Pin-Priority: 650


Here v = 5.0* means the stable version (Lenny is 5.0).  So only xkb-data will be fetched from testing, but the rest remains stable.
Then install the xkb-data package:

$ sudo apt-get update
$ sudo apt-get install xkb-data


Now under the K menu go to Control Center -> Regional & Accessibility -> Keyboard Layout -> Layout.  Choose Microsoft Natural Wireless Ergonomic Keyboard 7000 as the Keyboard model.  Restart X.
Most special keys should be working by now.
Update History
March 13, 2010: Using .deb is much easier, so I obsoleted the apt-pinning method.  (Not to mention it was no longer working for me for some reason…)



Mathematica 4.2 on Debian Etch
Taro — Sun, 22 Mar 2009 22:47:49 +0000
Note: I’m yet to successfully install Mathematica 4.2 on Lenny.  I might try again or it is really the time to stop using it.  Symbolic math is so convenient though…  Meanwhile, please let me know if you know of a good open-source alternative!
Mount the install CD:
# mount -o exec -r -t iso9660 /dev/hdc /media/cdrom

where /dev/hdc should be replaced by the device of your CD/DVD drive.
# cd /media/cdrom/Installers/Linux
# ./MathInstaller

Choose default directory, “Linux” install type, Full install, Single machine, and default install directory.
When copying files is finished, you need to create a couple symlinks:
# cd /usr/X11R6/lib/X11
# ln -s /usr/share/X11/XKeysymDB XKeysymDB
# ln -s /usr/share/X11/locale locale

You then need to set things up for dynamically-linked binaries.  The detailed documentation is given here, and here’s a summary:
# SRC=/media/cdrom/Files/SystemFiles
# DST=/usr/local/Wolfram/Mathematica/4.2/SystemFiles
# cp -r ${SRC}/Kernel/Binaries/Linux-dyn ${DST}/Kernel/Binaries
# cp -r ${SRC}/Kernel/SystemResources/Linux-dyn ${DST}/Kernel/SystemResources
# cp -r ${SRC}/FrontEnd/Binaries/Linux-dyn ${DST}/FrontEnd/Binaries
# cp -r ${SRC}/Graphics/Binaries/Linux-dyn ${DST}/Graphics/Binaries
# cp -r ${SRC}/Converters/Binaries/Linux-dyn ${DST}/Converters/Binaries

Now you need to modify some scripts.  In the directory
# cd /usr/local/Wolfram/Mathematica/4.2/Executables/Linux

find scripts named math, mathematica, MathKernel, and Mathematica.  In all these files, replace the line
sysid=Linux

to
sysid=Linux-dyn

Also add a line
export LD_ASSUME_KERNEL=2.4.1

to the same files.
Do the same for the similarly named scripts in the directory /usr/local/bin.
That should be it.  You will need a license number and a password to use the product.



Linksys WMP54G on Debian Etch
Taro — Sun, 22 Mar 2009 22:38:14 +0000
(Note: This is for Etch and may be obsolete.)
Follow the “official” instruction for etch, which is summarized as:
# apt-get install module-assistant
# module-assistant prepare
# apt-get install rt2500-source
# module-assistant auto-install rt2500-source

Reboot or load right away:
# modprobe rt2500

To see which device name has been assigned, do
# ifconfig -a

Then see configuring network to finish off configurinig /etc/network/interfaces.
Unfortunately, the rt2500 driver is still buggy, and may need
# iwconfig wlan0 rate 54M

to make it work at its full capacity.  Write a script which runs the above command somewhere, e.g., ~/.kde/Autostart, so that you do not have to do this manually.



Linksys WET54G on Debian Lenny
Taro — Sun, 22 Mar 2009 22:33:42 +0000
Add the following interface to /etc/network/interfaces:
iface wet54g inet static
  address 192.168.1.10
  netmask 255.255.255.0
Connect the bridge WET54G to the computer.  If an Ethernet card is configured as eth0, do
# ifup eth0=wet54g
The WET54G web interface can be accessed at 192.168.1.226.  Keep the static IP address for later use.  Specify wireless SSID and configure security setting if necessary.  The network type should be infrastructure in normal setting.  Restart the bridge.  (Also for me WPA2 PSK and TKIP for security setting…)
Once the bridge is receiving the wireless connection, having
auto eth0
iface eth0 inet dhcp
in /etc/network/interfaces will configure the network via DHCP on reboot.



Compiz Fusion in Dell XPS 630i (Debian Lenny)
Taro — Sun, 22 Mar 2009 22:24:57 +0000
Note: This is a cursory note when I set up Lenny on Dell XPS 630i (the video card is nVidia GeForce 8800 GT) when Lenny was still testing.  For detail I used this and that.
Install the nVidia video driver from the nVidia web site (use their installer and follow their instruction).  Then install some packages:
# apt-get install compiz compiz-core compiz-gnome compiz-gtk compiz-plugins
                  compizconfig-settings-manager libcompizconfig0
                  python-compizconfig libdecoration0
# apt-get install compiz-fusion-bcop compiz-fusion-plugins-extra
                  compiz-fusion-plugins-main
                  compiz-fusion-plugins-unsupported
                  compizconfig-backend-gconf compizconfig-backend-kconfig
                  compizconfig-settings-manager libcompizconfig0
# apt-get install compiz-kde fusion-icon

Also to let the script modify /etc/X11/xorg.conf, do:
# nvidia-xconfig --composite
# nvidia-xconfig --render-accel
# nvidia-xconfig --allow-glx-with-composite
# nvidia-xconfig --add-argb-glx-visuals

In /etc/X11/xorg.conf, make sure DefaultDepth is set to 24, if it isn’t already, then add the following lines below “EndSubSection” or above the first “SubSection”:
Option         "AddARGBGLXVisuals" "true"
Option         "DisableGLXRootClipping" "true"

Save the file and exit the text editor. Now restart the X-server by pressing Ctrl + Alt + Del or reboot.
Use fusion-icon to configure.  You can use *.profile file to restore saved configuration.
To auto-start compiz fusion at KDE startup, open an editor, and create a file with the following contents:
#!/bin/sh
/usr/bin/fusion-icon

Save the file as ~/.kde/Autostart/compiz_start.sh.



Using NSF Filesystem to Use an Old Debian Box as File Server
Taro — Sun, 22 Mar 2009 22:17:20 +0000
Note: This is an old note.  I tend to use a very old and slow desktop Debian (then Etch) box as a file server for backup and extra storage.  This was my attempt to do it via NSF.  The documentation has not been updated for Lenny.
First, kernel must be built with appropriate modules pertaining NSF, both on client and server sides.
Server Configuration
Make sure to have the following kernel support:
NFSD     (m)
NFSD_V3  (y)
NFSD_TCP (y)

On the server side, install following packages:
# apt-get install nfs-kernel-server nfs-common portmap

Edit /etc/exports such that the directories to be exported and the clients needing access to them are specified, e.g.,
/some/dir client(rw,sync)

Note that “async” option, instead of “sync”, should be used if high performance is desired (the chance of data corruption upon server crash will be higher, though).
Then execute
# exportfs -a

to make the change in effect.
Client Configuration
Make sure to have the following kernel support:
NFS_FS   (m)
NFS_V3   (y)

On the client side, install following packages:
# apt-get install nfs-common portmap

The remote directory server:/some/dir can be mounted at this point. However, most likely you may want to add a line to /etc/fstab:
server:/some/dir /mnt/somer/dir nfs rw,hard,intr,rsize=1024,wsize=1024 0 0

To optimize the transfer rate, rsize and wsize needs to be determined by experimentation.  Use a script like this:
# BEGIN transtest.sh ########################################################
#!/bin/bash

# Specify a mount point.
MNTPT=/some/dir

# Specify a file to be generated.
FNAME=${MNTPT}/testfile

# File size in MB.
let "FSIZE = 1024 * 1024 *  100"

# Block sizes to try.
BSARR=(1024 2048 4096 8192 16384 32768 65536)

# Do the measurements!
for idx in `seq 0 6`;
  do
  for i in `seq 1 3`;
    do

    BS=${BSARR[idx]}
    let "COUNT=${FSIZE}/${BS}"

    echo "##### Iteration $i at bs=${BS} and count=${COUNT} #####"

    sudo umount ${MNTPT}
    sudo mount ${MNTPT}

    echo "Writing to ${FNAME}..."
    time dd if=/dev/zero of=${FNAME} bs=${BS} count=${COUNT}

    sudo umount ${MNTPT}
    sudo mount ${MNTPT}

    echo "Reading from ${FNAME}..."
    time dd if=${FNAME} of=/dev/null bs=${BS}

    echo ""
  done
done

rm ${FNAME}

echo "Done."
# END transtest.sh ##########################################################

Change FSIZE to mean about twice the physical RAM of the server.  Run the script (which should be placed on the client machine, outside the directory tree to be mounted) as a user who can un/mount and has a write permission to the directory.  This takes a long time, so the output should be saved
someone$ ./transtest.sh > testout 2>&1

Pick the lowest rsize and wsize values which gives you the fast transfer rates.



Local Area Networking with Cross-over Cable in Debian
Taro — Sun, 22 Mar 2009 21:55:03 +0000
(Note: This method (which I used often when Debian stable was Woody) is obsolete as there is what I think is a much nicer solution.  I leave this note just for record.)
Easiest way to transfer a large amount of data between two Linux boxes is to use PC-to-PC networking using a cross-over cable.  At least one of the box (i.e., server) has to have sshd running.  First, find out which network interface corresponds to the NIC on the server.
server$ sudo ifconfig -a

Let’s say eth0 is the NIC of the server.  Then assign an IP address to it by
server$ sudo ifconfig eth0 192.168.0.1

Then on the client side, you do similarly:
client$ sudo ifconfig -a

Say eth1 is the NIC of the client.  Then
client$ sudo ifconfig eth1 192.168.0.2

Now the usual ssh session can be used between the two computers using the IP addresses assigned above.  Now with the command
client$ ssh 192.168.0.1

client is connected to the server.
Obviously, going through the above procedure is rather tedius, so automation is in order.  First install guessnet:
# apt-get install guessnet

In /etc/network/interface, have the lines:
allow-hotplug eth0

mapping eth0
  script /usr/sbin/guessnet-ifupdown
  map home_local
  map default: dhcp_fb

iface dhcp_fb inet dhcp

iface home_local inet static
  pre-up if grep -q tg3 /proc/modules; then :; else modprobe tg3; fi
  address 192.168.0.2
  netmask 255.255.255.0
  test peer address 192.168.0.1 mac xx:xx:xx:xx:xx:xx

where tg3 should be replaced by the correct module for your system, and the MAC address in the “test peer” line should correspond to that of the NIC on the server.  In the above, eth0 is assumed to be the NIC on the client.



Using Hostnames in Local Area Network on Debian Lenny
Taro — Sun, 22 Mar 2009 21:45:27 +0000
Background: I have a couple Debian Lenny boxes.  They connect to the Internet via a wireless (or wired) router.  The IPs are assigned dynamically via DHCP.  It is fairly easy to set up a LAN with static IP addresses (e.g., 127.0.0.101, 127.0.0.102, etc.), but it is not so convenient to remember IP addresses for each computer in LAN.  When I ssh into another box, I want to use a hostname rather than an IP address.
There is a very easy solution!
In /etc/dhcp3/dhclient.conf of each computer, specify
send host-name "johndoe";

(for a host named “johndoe”).  This way a host can be identified via a hostname, not via an IP address.



Wacom Intuos Tablet on Debian Etch
Taro — Sun, 22 Mar 2009 21:16:12 +0000
(Note: This is for Debian Etch, not Lenny.  I have not tried this on Lenny.)
The kernel module “wacom” from 2.6.16 should work out of the box.
Install some packages:
# apt-get install wacom-tools

Add the following input devices to /etc/X11/xorg.conf:
Section "InputDevice"
	Identifier	"stylus"
	Driver		"wacom"
	Option		"Type"		"stylus"
	Option		"Mode"		"Absolute"
	Option		"USB"		"on"
	Option		"Threshold"	"10"
	Option		"Tilt"		"on"
	Option		"Device"	"/dev/input/wacom"
EndSection

Section "InputDevice"
	Identifier	"eraser"
	Driver		"wacom"
	Option		"Type"		"eraser"
	Option		"Mode"		"Absolute"
	Option		"USB"		"on"
	Option		"Tilt"		"on"
	Option		"Threshold"	"10"
	Option		"Device"	"/dev/input/wacom"
EndSection

Section "InputDevice"
	Identifier	"cursor"
	Driver		"wacom"
	Option		"Type"		"cursor"
	Option		"Mode"		"Relative"
	Option		"USB"		"on"
	Option		"Threshold"	"10"
	Option		"Device"	"/dev/input/wacom"
EndSection

Section "InputDevice"
	Identifier	"pad"
	Driver		"wacom"
	Option		"Device"	"/dev/input/wacom"
	Option		"Type"		"pad"
	Option		"USB"		"on"
EndSection

Add to ServerLayout:
Section "ServerLayout"
	...

	InputDevice	"stylus"	"SendCoreEvents"
	InputDevice	"eraser"	"SendCoreEvents"
	InputDevice	"cursor"	"SendCoreEvents"
	InputDevice	"pad"		""
EndSection

GIMP
To use cool features like pressure sensitivity and eraser with GIMP, go to: File -> Preferences -> Input Devices -> Configure Extended Input Devices, and set stylus, cursor, eraser, pad to “Screen.”  This should be it!



USB Keyboard on Linux
Taro — Sun, 22 Mar 2009 21:08:41 +0000
To use a USB keyboard, the kernel support needs:
Input core support: yes
Keyboard support: yes
USB Human Interface Device (full HID) support: yes
HID input layer support: yes




Gnuplot (v4.3) from CVS on Debian Etch
Taro — Sun, 22 Mar 2009 21:04:51 +0000
(Note that this is for Debian Etch and written back in 2007.  It may have become obsolete now.  I’m just keeping it for record.)
(See http://www.gnuplot.info/ for detail.)
First, install LaTeX stuff (see other instruction) and others:
# apt-get install cvs groff cm-super texinfo

The summary of building from CVS source:
# mkdir /usr/local/src/gnuplot
# cd /usr/local/src/gnuplot
# export CVSROOT=:pserver:anonymous@gnuplot.cvs.sourceforge.net:/cvsroot/gnuplot
# cvs login
# cvs -z3 checkout gnuplot
# cd gnuplot
# ./prepare

(If ./prepare complains about aclocal or something else, install some packages
# apt-get install automake1.9 autoconf

before you do ./prepare.)
# ./configure --with-readline=gnu
# make
# make install-strip
# apt-get install gnuplot-mode




Matrox MGA G400-TV on Debian Etch
Taro — Sun, 22 Mar 2009 20:51:37 +0000
(Note that this is for Debian Etch, as I have not tried the procedure on Lenny.  It may have become obsolete now.)
(For more detail, see http://marvel.sourceforge.net/)
# apt-get install cvs

This instruction is for kernel version 2.6.18.
# mkdir /usr/local/src/matrox/mgavideo
# cd /usr/local/src/matrox/mgavideo

First, download the driver_fb source from the CVS repository (you may also use an appropriate stable version):
# cvs -d:pserver:anonymous@marvel.cvs.sourceforge.net:/cvsroot/marvel login
# cvs -z3 -d:pserver:anonymous@marvel.cvs.sourceforge.net:/cvsroot/marvel co -P driver_fb

The Linux kernel source must be patched and recompiled.  Follow the README.  Here’s the summary:
# cd /usr/src/linux
# emacs Makefile      # append to kernel version
# make mrproper
# make clean
# patch -p1 < /usr/local/src/matrox/mgavideo/driver_fb/kernel-2.6.19.patch
# make xconfig        # if you need to change kernel options
# make-kpkg clean
# make-kpkg kernel_image
# cd ..
# dpkg -i linux-image-***.deb

Reboot and then
# cd /usr/local/src/matrox/mgavideo/driver_fb
# make

In the module loading script iv, you need to comment out the following line:
#/sbin/modprobe matroxfb_maven

Create a script /etc/init.d/iv with the contents:
#!/bin/sh
cd /usr/local/src/matrox/mgavideo/driver_fb
./iv

Make the script executable and create a symlink in /etc/rc2.d:
# chmod a+x /etc/init.d/iv
# ln -s /etc/init.d/iv /etc/rc2.d/S98iv

Reboot, and should be set.
# apt-get install xawtv scantv xawtv-plugins xserver-xorg-video-mga

You may want to recycle the xawtv config file, /etc/X11/xawtvrc from the previous installation.  Also modify the “Device” section of /etc/X11/xorg.conf to specify “mga” as Driver.
Finally, add a line
xawtv.xv: false

to ~/.Xresources so that DVD viewing with xv is possible.
If you are annoyed by some garbled windows while watching TV, setting
DefaultDepth 16

in the “Screen” section of /etc/X11/xorg.conf should solve the problem.



EPSON Perfection 1260 on Debian Lenny
Taro — Sun, 22 Mar 2009 20:46:29 +0000
Kernel (version 2.6.x) must be configured such that it has /proc and usb device file system support.  (Or more recent info state that you need the kernel modules usbhid, ehci_hcd, and ohci_hcd; see this guide for detail.)
Then install a bunch of software
# apt-get install dbus hal sane
# apt-get install sane-utils xsane libpng-dev usbutils
Verify the scanner is detected:
# sane-find-scanner
should indicate something similar to
found USB scanner (vendor=0x04b8 [EPSON], product=0x011d
[EPSON Scanner], chip=LM9832/3) at libusb:003:005
Add a user who needs access to the scanner to the group scanner:
# adduser username scanner
# exit
$ scanimage -L
Make sure that the last command shows something similar to
device `plustek:libusb:003:005' is a Epson Perfection 1260/Photo USB
flatbed scanner
This confirms that the scanner is usable.  After starting a new session,
$ xsane
should get you going.  Note that hotplugging may not recognize the scanner at times (e.g., after rebooting).  Try unplug and plug again the USB cable if that happens.



Timezone Setting for Windows / Debian Lenny Dual Boot
Taro — Sun, 22 Mar 2009 20:38:17 +0000
In order to coexist with a dumb system like Microsoft Windows, it may be necessary to set up Debian to use localtime instead of UTC to keep track of time.
In /etc/default/rcS, change a line to have UTC=no.  Then reboot.  If you wish, the system time can be then synchronized with the ntp.



Using KDict Locally on Debian Lenny
Taro — Sun, 22 Mar 2009 20:10:03 +0000
Online dictionaries are fantastic, but once in a while an access to a dictionary without picking up wireless becomes necessary.  It may be handy to have KDict set up so that it doesn’t query a remote host.
If you will be using dict.org through the web, just doing
# apt-get install kdict

is enough to start using KDict right away.  If you wish to use it offline, you need a daemon and some database:
# apt-get install dictd
# apt-get install dict-wn
# apt-get install dict-gcide
# apt-get install dict-jargon
# apt-get install dict-moby-thesaurus
# apt-get install dict-vera
# apt-get install dict-foldoc

You may select any of dict-*; they are different dictionaries.  Launch kdict, go to Settings -> Configure Dictionary -> Server, and specify “localhost” (instead of dict.org) as the hostname in DICT Server Configuration.



Ruby on Debian Lenny
Taro — Sun, 22 Mar 2009 20:02:08 +0000
Here I will keep adding Ruby-related reminders for my Debian box.
# apt-get install ruby ruby-elisp




Canon PIXMA iP1800 for Debian Lenny
Taro — Sun, 22 Mar 2009 19:51:21 +0000
This is a cheap inkjet printer that I got for my temporary printing needs.  The cartridges are very expensive (> $16 !!), so for heavy use this isn’t really worth my money.  Anyways…
Install CUPS and printing-related packages:
# apt-get install cupsys cupsys-client libcupsys2
# apt-get install cupsys-driver-gimpprint cupsys-bsd
# apt-get install foomatic-db-engine foomatic-db-gimp-print foomatic-filters
# apt-get install hp-ppd linuxprinting.org-ppds printfilters-ppd
# apt-get install foomatic-filters-ppds
# apt-get install gs-gpl libpng3 ijsgutenprint
(Note that I downloaded a few packages that are not necessary for this particular printer.  This is a generic collection of packages that I find useful to use printers in other places like my office.)
Download deb packages of the printer driver:
cnijfilter-common_2.70-3_i386.deb
cnijfilter-ip1800series_2.70-3_i386.deb
and install them:
# dpkg -i cnijfilter-common_2.70-3_i386.deb cnijfilter-ip1800series_2.70-3_i386.deb
If CUPS has been installed properly, visit the CUPS admin page (http://localhost:631/admin) with your browser.  You will be able to configure the printer in the standard CUPS way.



Using NTP Server to Synchronize Time on Debian Lenny
Taro — Sun, 22 Mar 2009 19:31:06 +0000
# apt-get install ntp
# /etc/init.d/ntp restart
That’s it.  How simple is that!



Python on Debian Lenny
Taro — Sun, 22 Mar 2009 09:32:05 +0000
# apt-get install python ipython python-doc
# apt-get install python-numpy python-scipy python-matplotlib
SciPy from Source
It is recommended the stable versions to be downloaded and installed from www.scipy.org.
# apt-get install subversion swig python2.5-dev python-nose
# apt-get install libatlas-headers libatlas-base-dev
# apt-get install libfftw3-dev libsuitesparse-dev
# apt-get install liblapack-dev gfortran
For matplotlib (UNDER CONSTRUCTION),
# apt-get install libpng12-dev libfreetype6-dev dvipng python-dateutil python-tz

# # for gtk and gtkagg
# apt-get install python-gtk2-dev

# # for tkagg
# apt-get install tk-dev

# # for wxagg
# apt-get install python-wxgtk2.8 libwxbase2.8-dev

and follow the installation instruction of the distribution.
The following instruction is for installing from svn repository.  To install NumPy:
# rm -rf numpy.prev
# mv numpy numpy.prev
# svn co http://svn.scipy.org/svn/numpy/trunk numpy
# rm -rf /usr/lib/python2.5/site-packages/numpy.prev
# mv /usr/lib/python2.5/site-packages/numpy /usr/lib/python2.5/site-packages/numpy.prev
# cd numpy
# python setup.py install
# cd ..
To install SciPy:
# rm -rf scipy.prev
# mv scipy scipy.prev
# svn co http://svn.scipy.org/svn/scipy/trunk scipy
# rm -rf /usr/lib/python2.5/site-packages/scipy.prev
# mv /usr/lib/python2.5/site-packages/scipy /usr/lib/python2.5/site-packages/scipy.prev
# cd scipy
# python setup.py install
# cd ..



Emacs on Debian Lenny
Taro — Sun, 22 Mar 2009 09:02:01 +0000
Install basic packages:
# apt-get install emacs22 aspell aspell-en
# apt-get install css-mode php-elisp html-helper-mode

Changing the Size of Emacs Window
Use xfontsel to select a desired font.  In ~/.Xresources, write lines as follow:
emacs.font:              -misc-fixed-medium-r-semicondensed--13-*-*-*-c-60-iso8859-1
emacs.pane.menubar.font: -misc-fixed-medium-r-semicondensed--13-*-*-*-c-60-iso8859-1

to change font setting in emacs.  To source a modified ~/.Xresources file in the current session, do
$ xrdb ~/.Xresources

and the next invocation of emacs will reflect the change.
JavaScript Mode: javascript.el
First, download javascript.el, and save it to /usr/share/emacs/site-lisp.  Add the following lines to the ~/.emacs file in your home directory:
;; for javascript mode
(add-to-list 'auto-mode-alist '("\.js\'" . javascript-mode))
(autoload 'javascript-mode "javascript" nil t)

This will enable emacs to recognize .js file as a javascript file.
JavaScript Mode: js2
First, download js2.el and save it to /usr/share/emacs/site-lisp as js2.el.  Add the following lines to the ~/.emacs file in your home directory:
;; for js2 mode
(autoload 'js2-mode "js2" nil t)
(add-to-list 'auto-mode-alist '("\.js$" . js2-mode))

This will enable emacs to recognize .js file as a javascript file.  Also, in Emacs, do: M-x byte-compile-file RE js2.el RET
More Reliable Clipboard
Add to ~/.emacs the following lines:
;; for clipboard
(setq x-select-enable-clipboard t)
(setq interprogram-paste-function 'x-cut-buffer-or-selection-value)

Changing Title Bar
Using this tip, the title bar will show the path to the file being modified.  See this tip:
$ wget http://www.emacswiki.org/cgi-bin/wiki/download/frame-cmds.el
$ mv frame-cmds.el ~/.emacs.d

Then add the following lines to ~/.emacs:
;; for frame-cmds.el
(setq frame-title-format '("" "%f"))




LaTeX on Debian Lenny
Taro — Sun, 22 Mar 2009 08:16:12 +0000
Install basic packages for LaTeX:

# apt-get install texlive texlive-latex-extra texlive-math-extra
# apt-get install texinfo cm-super

# apt-get install cjk-latex xdvik-ja dvipsk-ja latex-cjk-japanese ptex-jtex nkf
# apt-get install cmap-adobe-cns1 cmap-adobe-gb1 cmap-adobe-japan1 cmap-adobe-japan2
# apt-get install gs-cjk-resource

The last three lines are for using Japanese.  With \documentclass[a4j]{jarticle}, for example, we can generate a Japanese document with LaTeX. I like to make documents with international characters with UTF-8, but the Japanese documents here need to be encoded with EUC-JP before using platex command, by doing something like this:
$ nkf -e doc.tex > euc_doc.tex

$ platex euc_doc.tex

$ dvipdfmx euc_doc.dvi

$ acroread euc_doc.pdf &
Hopefully UTF-8 can be used in Squeeze…
The rest of the sections are only relevant for astrophysicists.
Installing AASTeX v5.2 LaTeX 2e
# mkdir /usr/share/texmf/tex/latex/misc
# mkdir /usr/local/src/AASTeX
# cd /usr/local/src/AASTeX
# wget http://dopey.mcmaster.ca/AASTeX/aastex52_large.tar.gz
# gunzip -c aastex52_large.tar.gz | tar xvf -
# cd aastex52
# make install
# texconfig rehash

Installing emulateapj.cls (ver. August 13, 2006)
# mkdir /usr/share/texmf/tex/latex/misc
# mkdir /usr/local/src/emulateapj
# cd /usr/local/src/emulateapj
# wget http://hea-www.harvard.edu/~alexey/emulateapj/emulateapj.cls
# wget http://hea-www.harvard.edu/~alexey/emulateapj/apjfonts.sty
# cp emulateapj.cls /usr/share/texmf/tex/latex/misc
# cp apjfonts.sty /usr/share/texmf/tex/latex/misc




PDF Utilities on Debian Lenny
Taro — Sun, 22 Mar 2009 07:45:21 +0000
Programs for handling PDF files:
# apt-get install acroread acroread-plugins mozilla-acroread
# apt-get install xpdf gv
# apt-get install pdftk

To install Japanese fonts for acroread, go to a download site at Adobe get a relevant tarball.  Run the install script and specify the installation direction as /usr/lib.



Utility Programs for Debian Lenny
Taro — Sun, 22 Mar 2009 07:40:44 +0000
Here I simply list the apt-get commands to fetch all the small utility programs that I often use.  I cut and paste these to install complete an installation to my liking!
Archiving
# apt-get install bzip2 zip unzip




Customizing & Installing Linux Kernel 2.6.x on Debian Lenny
Taro — Sun, 22 Mar 2009 07:34:31 +0000
Here is a quickie for customizing and install Linux kernel 2.6.x on Lenny.
Install some packages and kernel source:

$ sudo apt-get install kernel-package bzip2 g++ libqt3-mt-dev libncurses5-dev sudo fakeroot
$ sudo apt-get install linux-source-2.6.26

If you do not belong to group “src”, add yourself to the group as we work under /usr/src.  You also need to be able to use sudo or su.
Extract the source tree:

$ cd /usr/src
$ tar -jxf linux-source-2.6.26.tar.bz2
$ ln -s linux-source-2.6.26 linux
$ cd linux


Edit the EXTRAVERSION entry in Makefile, as in:

EXTRAVERSION = .20100310.1


for example to add .20100310.1 to the kernel version number.  This is convenient for keeping the existing, working kernels around when you need to recompile with different options.
Use xconfig or menuconfig to customize the kernel options.

$ make mrproper
$ make xconfig    # or make menuconfig
$ make-kpkg clean
$ fakeroot make-kpkg --initrd kernel_image
$ cd ..
$ sudo dpkg -i linux-image-2.6.26.20090322.1_2.6.26.20090322.1-10.00.Custom_i386.deb

Upon reboot in the GRUB menu you will find the newly installed kernel:

$ sudo shutdown -r now


Purging Old Kernel Image from System

$ sudo dpkg -P linux-image-2.6.26.20091112.1


Just specifying the kernel image to remove.  That’s it.  However it is often a good idea to keep at least one kernel image that you know for sure to be working, so that when your new custom kernel fails, you have something to fall back on to recompile it again.
Update History
March 13, 2010: Updated the tutorial to use fakeroot and sudo in stead of becoming root.



Sharing an X Session between a Regular User & Root
Taro — Sun, 22 Mar 2009 07:28:55 +0000
When you use su to become a superuser (root) and launch a program that has GUI on KDE, you will notice that you get an error like this:
# konqueror
No protocol specified
konqueror: cannot connect to X server :0

In order to share an X session with a regular user, say johndoe, root needs to share an authorization cookie.  One way to achieve this is to make a BASH script (called “usex” here):
#!/bin/bash
#
# Share the authorization cookie for an X session between a regular
# user and root.
#

# specify the username of X session owner from which the authorization
# cookie is copied
USRNAME=johndoe


##############################################################################
# do not modify below

xauth -f /home/${USRNAME}/.Xauthority extract - :0 | xauth merge -
export DISPLAY=:0

and run this every time you become root.  Make this script accessible at /root/bin/usex and add a line to execute it within /root/.bashrc, somewhere toward the end of the file:
~/bin/usex




More Fonts on Debian Lenny
Taro — Sun, 22 Mar 2009 07:01:55 +0000
Install some commonly used fonts:
# apt-get install x-ttcidfont-conf msttcorefonts ttf-xfree86-nonfree

If you have TrueType font files (.ttf), it is easiest to install fonts via KDE Control Center: Control Center -> System Administration -> Font Installer.



Logitech MX Revolution on Debian Lenny
Taro — Sun, 22 Mar 2009 06:02:17 +0000
For detail, see the instruction for Ubuntu.
Before starting, make sure that the uinput kernel module is available.  This means that you have the kernel options CONFIG_INPUT_MISC=y and CONFIG_UINPUT=m.  If you cannot load uinput with modprobe uinput, you have to recompile the kernel.
Compile from source:
# apt-get install build-essential libgtk2.0-0 libgtk2.0-dev libglade2-0
# apt-get install libglade2-dev pkg-config libdaemon0 libdaemon-dev

$ mkdir /usr/local/src/btnx
$ cd /usr/local/src/btnx

$ wget http://www.ollisalonen.com/btnx/btnx-0.4.11.tar.gz
$ tar -xvvf btnx-0.4.11.tar.gz
$ cd btnx-0.4.11
$ ./configure
$ make
$ sudo make install

$ wget http://www.ollisalonen.com/btnx/btnx-config-0.4.9.tar.gz
$ tar -xvvf btnx-config-0.4.9.tar.gz
$ cd btnx-config-0.4.9
$ ./configure
$ make
$ sudo make install

Now btnx-config should be one of the menu item. In GNOME, Applications->System tools->btnx. In KDE, System->btnx. Press it to launch btnx-config.
Load from a backup archive for your customized configuration, if exists.
It doesn’t seem to be necessary to add anything to /etc/X11/xorg.conf to make MX Revolution work at this point.
If the kernel does not have necessary options, try this: Need the kernel support (“yes”) for Input Core Support (select for mouse as well), USB Human Interface Device (Full HID) Support, and HID Input Layer Support.

star



OpenOffice in Debian Lenny
Taro — Sun, 22 Mar 2009 05:58:12 +0000
This is simple!
# apt-get install openoffice.org

Even using Japanese works out of the box.



Using Japanese in Debian Lenny
Taro — Sun, 22 Mar 2009 04:43:40 +0000
The goal is to make the system capable for Japanese input, while the base remains English.  The software of choice is Anthy and scim.
First, install Japnese locales ja_JP.EUC-JP and ja_JP.UTF-8:
$ sudo apt-get install locales
$ sudo dpkg-reconfigure locales

Set the default locale for the system to en_US (or something else if you wish).
Install necessary Debian packages:
$ sudo apt-get install anthy anthy-el libanthy0
$ sudo apt-get install scim-tables-ja scim-anthy scim
$ sudo apt-get install skim scim-gtk2-immodule scim-modules-socket kasumi
$ sudo apt-get install ttf-kochi-mincho ttf-kochi-gothic


Launch skim (for KDE):
$ skim -d -f

and you will see an icon in the system tray.  Right click on the system tray icon and click on Configure to launch it.  Go to FrontEnd -> General SCIM -> Other and change “Panel Program” to “scim-panel-kde” and “Config Module” to “kconfig” to optimize for KDE.
Then run im-switch -c and choose skim from the list.  This will make it possible to use the input method in any locale.
After restarting X, you should be able to use the input method.  Try visiting Google and place the focus on the search word input form.  Shift + space (and some other key combinations) will switch between the Japanese and English modes.
Some applications still need some environment variable set to display Japanese appropriately.  I make a little script that I will source whenever I wish to type Japanese in an application.  The application launched from the same shell will have Japanese capacity.
#!/bin/bash
export LC_CTYPE="ja_JP.UTF-8"
export XMODIFIERS=@im=SCIM
export GTK_IM_MODULE=scim
export QT_IM_MODULE=scim

Emacs
Install packages:
# sudo apt-get install emacs22 mule-ucs anthy-el emacs21-el emacs-goodies-el
Add the following lines to ~/.emacs:
;; Japanese with Anthy

(set-language-environment “Japanese”)

(set-terminal-coding-system ‘utf-8)

(set-keyboard-coding-system ‘utf-8)

(set-buffer-file-coding-system ‘utf-8)

(setq default-buffer-file-coding-system ‘utf-8)

(prefer-coding-system ‘utf-8)

(set-default-coding-systems ‘utf-8)

(setq file-name-coding-system ‘utf-8)

;; Following is probably not necessary unless using without GUI.

;;(load-library “anthy”)

;;(setq default-input-method “japanese-anthy”)
(See here for Ubuntu documentation.)
Using M+ IPA Fonts
If you want to have more choices on fonts, you might try M + IPA fonts.  The font is slightly anti-aliased and people say it looks better.  Here is the installation summary.
Download the font pack to, say, ~/tmp.  Then
$ cd ~/tmp
$ tar xvf mixfont-mplus-ipa-TrueType-20060520p1.tar.bz2
$ cd mixfont-mplus-ipa-TrueType-20060520p1/opfc-ModuleHP-1.1.1_withIPAFonts_and_Mplus/fonts
$ cp *.ttf ~/.fonts
$ fc-cache

The change takes effect after restarting X.  Choose the fonts with names starting with M+1P+IPAG, etc., to make use of them.
<



Sound Applications (Amarok, Ogg, etc.) in Debian Lenny
Taro — Sun, 22 Mar 2009 04:28:56 +0000
Amarok is my favorite audio player.  For tagging ogg files, I use EasyTAG:

$ sudo apt-get install amarok easytag-aac kmix


In order to make Amarok adjust volume automatically for different volume level at which music are recorded, I use ReplayGain script for Amarok.  It needs to be downloaded manually and installed via script manager.



Installing Firefox (Iceweasel) on Debian Lenny
Taro — Sun, 22 Mar 2009 04:06:33 +0000
Note that in Debian, Firefox is called iceweasel.

$ sudo apt-get install iceweasel


Then install some popular plugins.
Java
For i386:

$ sudo apt-get install sun-java5-plugin sun-java5-fonts


For AMD64, I have not looked up yet.
Flash
For i386:

$ sudo apt-get install flashplayer-mozilla


For AMD64, the flashplayer-mozilla package does not exist.  You have to add the backports.org repository to /etc/apt/sources.list and do

$ sudo apt-get install flashplugin-nonfree


This should do it.  Try youtube.com to see if Flash is working.
MPlayer

$ sudo apt-get install mozilla-mplayer


Other Tips
Speeding Up via Pipelining
Visit the URL about:config with Firefox and set the following items:
network.http.pipelining = true
network.http.pipelining.maxrequests = 30
network.http.proxy.pipelining = true

Send Link Bug
If “Send Link…” in Firefox does not open your email client, visit the URL about:config, and add an entry:
network.protocol-handler.app.mailto : 'icedove'

and restart Firefox.  This sets the default emailer to Thunderbird (icedove in Debian).
Update History:
March 13, 2010: Added infor for AMD64 architecture.



Turning Off Auto-checking of Filesystems on Boot
Taro — Sun, 22 Mar 2009 03:44:11 +0000
By default the filesystems will be regularly checked even if the filesystem is marked clean.  Typically, this happens on every few dozen mounts on boot.  This quite possibly not what you want to happen for laptop.  In fact, one of the advantages of ext3 filesystem is journaling which makes regular checks of filesystems unnecessary.  To disable the automatic checking, do as root:
$ sudo tune2fs -c 0 -i 0 /dev/sdxx
where /dev/sdXX is the partition assigned to the filesystem (e.g., /dev/sdb1).
Update History
May 2, 2012 — Minor updates.



Configuring Devices Using udev
Taro — Sun, 22 Mar 2009 03:24:57 +0000
First, install udev if not already:
# apt-get install udev

In the /etc/udev directory, a text file (e.g., johndoe.rules where johndoe should be something like your username) with custom rules should be created.  For making these rules, a fine tutorial should be referenced.
After making rules, under /etc/udev/rules.d, a symlink named like 010_johndoe.rules should be created and pointed to ../johndoe.rules.



Watching DVD on Debian Lenny
Taro — Sun, 22 Mar 2009 02:56:34 +0000
Install mplayer and some codecs (to install these, you need to add a line:
deb http://www.debian-multimedia.org/ lenny main

to /etc/apt/sources.list, if not already):
# apt-get install mplayer libdvdread3 libdvdcss2 w32codecs

Use gmplayer:
$ gmplayer

and set video driver to “xv” for smooth DVD playback.



K3b on Debian Lenny
Taro — Sun, 22 Mar 2009 02:43:08 +0000
Install k3b and utilities:
# apt-get install k3b cdrdao dvd+rw-tools normalize-audio

For replaygain support in making ogg files:
# apt-get install vorbisgain

If I don’t already belong (use groups), add my user account to the cdrom group:
# adduser username cdrom




First post!
Taro — Sat, 21 Mar 2009 23:42:59 +0000
This is my first post for this blog.  I used to have all my own contents in an old-fashioned “home page” type of thing, but blogs are indeed useful as I read them all the time!
I’ll move much of my content here to call this my new home.  There might be some stuff of interests for others but I realized having them in flat ASCII text files hides them from search engines’ attention…