I choose Python to write mostly small scripts, helper tools, s’il vous plaît. I use it to process text files to extract and transform data (poor man’s ETL). I use it to automate routine manual tasks so I can do bigger and better things with the saved time. I have even (successfully) attempted to write a prototype B2B web application for my employer. I now understand the range of tasks I can accomplish with Python.

But the right tool should always be used for the job. I am not a programmer. I program a bit here and there but that is not my primary task, interest, or goal. I don’t write enterprise software. This is why I write scripts and not software. And Python is a very handy and helpful language to write them in.

I have learned and used C, C++, Java, and Perl. They all do wonderful things on their own but the kind of work I do, they are either too unwieldy or Python beats them in “having fun while creating”. When I choose Python, I don’t choose it because it is the best language in the world. I choose it because it gets the job done for me and saves me time. I am not saying that Python could not be used to write enterprise software and I am not saying it could. But for a person who works with data, database, networking, and voice technologies, Python is a useful tool. Use it where appropriate and don’t try to ram it down a project if it’s not a good fit.

N.B. The idea for this post came from Bad news: Google employees are being discouraged from using Python for new projects.

So I decided to take the plunge and use Linux on the desktop. This post describes how the first ever official Code Ghar Linux desktop was created. As I make changes to the computer, I will keep this post updated.

Distribution

I first installed Fedora Rawhide (nightly build) using ext4 as the filesystem. After a couple days, I decided to add more memory to the system. Once this memory was installed successfully, the OS would not boot at all. I decided to not risk things further and went with Debian testing (Squeeze) instead. I am more comfortable with a Debian based system anyways so the choice wasn’t hard. I chose the netinstall CD and only installed the command line (did not choose the Desktop option during install).

Desktop Environment

I wanted to make best use of the 3GB RAM in my system. So I chose to install LXDE as my desktop environment. Since I was only running command line, I ran the following command to install LXDE.

sudo aptitude install xorg gdm lxde

Wine

I needed to run three main applications. I could either install a virtual machine running Windows to run these apps or install Wine. I decided to try out Wine. To install Wine, I ran the following command.

sudo aptitude install wine

Once installed, all you have to do is run the exe file to install your desired application and Wine takes care of the rest. But make sure you have searched that application on the Wine Application Database.

Virtualization

However much I want to move away from Windows, I just can’t because of some applications. I chose to use VirtualBox although I have been using VMware on Windows and Mac for a couple years now. To install VirtualBox on Debian, I followed their Wiki (VirtualBox Installation). Following are the commands I ran.

sudo aptitude install virtualbox-ose virtualbox-ose-modules-$(uname -r)

sudo invoke-rc.d udev reload

sudo modprobe vboxdrv

sudo addgroup vboxusers

sudo adduser codeghar vboxusers

I then logged out and logged back in for VirtualBox to be ready to be used.

Remote Access

To allow remote access to my machine using VNC, I chose to use X11VNC. To install it, I ran the following command:

sudo aptitude install x11vnc

To invoke it for my user (since I have a single-user machine), I ran the following command once I logged in. This command is all on one line (in case the formatting of the blog makes it appear different)

x11vnc -display :0 -xkb -bg -forever -passwd mypassword

What this does is that it allows you to access your actual desktop on port 5900, runs in the background (bg flag), does not stop after first remote client disconnects (forever flag) i.e. keeps running, and assigns a password (passwd) which must be used to authenticate yourself. The xkb flag was used because for some reason the shift key did not work when I ran a VNC client from another machine into this one. I forgot which resource I used to fix this issue but I will put up the link once I can find it again.

If my computer is restarted, I will have to run the same command again for me to be able to use VNC. This is why you need access to at least SSH to be able to run this command and start VNC. I added the following alias at the end of my ~/.bashrc file. This way I don’t have to retype the command all the time.

alias myvnc="x11vnc -display :0 -xkb -bg -forever -passwd mypassword"

Flash for Iceweasel (Firefox)

To install Adobe Flash, I followed the instructions on How to Install Adobe Flash in Debian Etch/Lenny/Sid. As they were very generic, I took only what I needed and did the following.

First I downloaded the debian-multilemedia-keyring package on the desktop. Then I ran the following command to add it to aptitude.

sudo dpkg -i ~/Desktop/debian-multimedia-keyring_2008.10.16_all.deb

The I modified my /etc/apt/sources.list and added the following line:

deb http://www.debian-multimedia.org testing main

I then ran the following command to update aptitude and then install Flash.

sudo aptitude update

sudo aptitude install flashplayer-mozilla

Remote File Access with SSHFS

I used the guide at Mount a remote file system through ssh Using sshfs to get this working. I strongly recommend that you read it thoroughly for better information. To install SSHFS, you also need to install FUSE, as below.

sudo aptitude install fuse-utils sshfs

Now you need to do two things: load the FUSE module (as below)

sudo modprobe fuse

and also add your user to the fuse group (as below)

sudo adduser codeghar fuse

I then had to log out and log back in for my user to be recognized as part of fuse group.

I then created a mount directory in my home folder, created folders for all remote file systems I need to access, and created aliases for mounting and un-mounting SSHFS “shares”.

mkdir /home/codeghar/.mount/
mkdir /home/codeghar/.mount/server1
mkdir /home/codeghar/.mount/server2

To mount server1, I ran the following (also added this as a bash alias to save typing):

sshfs codeghar@10.10.1.101:/home/remoteuser/ /home/codeghar/.mount/server1

To unmount server1, I ran the following (also added this as a bash alias to save typing):

fusermount -u /home/codeghar/.mount/server1

Miscellaneous Applications

Following are the applications I also installed.

OpenOffice: sudo aptitude install openoffice.org
Wireshark: sudo aptitude install wireshark
gvim: sudo aptitude install gvim
VNC client: sudo aptitude install vncviewer
Grsync: sudo aptitude install grsync
Pidgin: sudo aptitude install pidgin
PDF: sudo aptitude install epdfview
Samba: sudo aptitude install samba samba-client smbfs

One day suddenly I could not copy files to my SMB share. I got an error that “the specified network name is no longer available”. I looked in /var/log/messages and one repeated error was rps.statd can’t decode rpc message. I tried to search for its cause and could not find anything that would solve my problem with not being able to copy any files to the share.

After mucking around the system, I noticed that in Network Access Configuration (in the web GUI), one by one all the entries started to disappear (on each refresh). If I tried to add them back, they would not stick and on refresh would disappear again. Googling revealed a very helpful resource: Network Access Configuration Changes Don’t “Stick”. According to it, the error could be caused by logs filling up the whole filesystem. I looked in /var/log/ and saw that the raa folder was taking in excess of 1.3 GB in a root partition of 2 GB (df -h and du -ah revealed this to me). Navigating to the raa folder showed that two files were approximately 600 MB is size: lighttpd.access.log and web-access. I removed these files and then restarted the machine (thinking on the next boot these files would be generated again).

Upon successful reboot, I was able to add entries to Network Access Configuration and everything was back to normal.

Bonus tips: To display all volume groups in command line, use vgdisplay. To display all logical volumes, use lvdisplay. To remove a logical volume via command line, when they can’t be deleted from the web GUI, use lvmremove. Hat tip to Can’t delete volume group.

This post aims to provide a step-by-step on how to install Trac in Debian and get it working. These “instructions” were created exactly the way I setup my server. So paths and setup information can be modified for your project. The idea was this: there are many guides out there which provide the same information but in a more generic way; this post provides exact path information and such so that there is concrete, real world example of what needs to be done.

CAUTION: This post is a work in progress so there are many things which might not be properly documented and could have adverse effect.

Install Necessary Applications

sudo aptitude install trac bzr trac-bzr trac-spamfilter postgresql python-psycopg2 apache2 libapache2-mod-wsgi

Prepare Database

sudo vim /etc/postgresql/8.4/main/pg_hba.conf
Add following lines:
local tracdb tracuser password
host tracdb tracuser 127.0.0.1/32 md5
host tracdb tracuser 192.168.1.0/24 md5

psql template1 -W postgres
create database tracdb with encoding = 'utf8';
create user tracuser password 'password';
grant all on database tracdb to tracuser;
sudo /etc/init.d/postgresql-8.4 reload

Prepare Directories

cd /media/shared/Admin/
cd /home/codeghar/
mkdir /home/codeghar/trac
cd trac
cd /home/codeghar/trac/
mkdir repo env
cd /home/codeghar/trac/repo/
bzr init

Create Trac Project

cd /media/shared/Admin/env/
cd /home/codeghar/trac/env/
trac-admin $(pwd) initenv
trac-admin /home/codeghar/trac/env/ initenv
Project Name [My Project]> codeghartrac
Database connection string [sqlite:db/trac.db]> postgres://tracuser:password@localhost/tracdb
Repository type [svn]> bzr
Path to repository [/path/to/repos]> /media/shared/Admin/trac/repo

Enable Bazaar with Trac

To get Trac working with bzr, edit the trac.ini file (Hat tip: bzr – trac integration)
vim /media/shared/Admin/trac/env/conf/trac.ini
And add the following two lines, if they don’t already exist
[components]
tracbzr.* = enabled

First Test

Check if everything is working as it should be

tracd --port 8000 /media/shared/Admin/trac/repo/

Trac with WSGI and Apache

cd /media/shared/Admin/trac/www/
trac-admin /media/shared/Admin/trac/env/ deploy /media/shared/Admin/trac/www/
trac-admin /home/codeghar/trac/env/ deploy /home/codeghar/trac/www/

Put the following in your Virtual Host:

WSGIScriptAlias / /home/codeghar/trac/www/cgi-bin/trac.wsgi

<Directory /home/codeghar/trac/www/cgi-bin/trac.wsgi>
WSGIApplicationGroup %{GLOBAL}
Order deny,allow
Allow from all
</Directory>


And then reload Apache. Your site is now available at http://yourip/

Hat tip: View of /packages/trac/trunk/debian/README.Debian; bazaar & trac-bzr; Recipe Installing Trac with PostgreSQL on CentOS4.2; Trac and mod_wsgi;

In HTTP and/or HTML, there are two (main?) types of submissions: GET and POST. I have always had a hard time determining when to use GET and when to use POST. In other words, what is the main difference between GET and POST?

Quite simply, if the submission is reading data, without making any changes, use GET. If your submission will be making changes, or causing side-effects, use POST. For example, when doing a query, such as a Google search, the form should use GET. If you are creating a new account with Yahoo! mail, the form should use POST. The Google search is reading data and new account in Yahoo! mail is changing things.

Simple enough, right? Not so, as there are some situations where either could work. One of the best descriptions of the problem I have read is Methods GET and POST in HTML forms – what’s the difference? And here is what I learned from it (you might learn something else reading the same words):

When doing a submission which will not make changes use GET. When making changes, use POST. But you may want to use POST instead of GET in the following (major?) case: when you don’t want the data you submit to become a part of the URL. When using GET, the form you submit becomes a part of the URL. So if you have a couple fields, say myname and mynumber, using GET your URL might look like http://www.mydomain.com/someform.html/?myname=codeghar&mynumber=1234 after submission. The biggest benefit is that you can bookmark this URL and visit it in future as just another link. The biggest disadvantage, in my opinion, is when you have multiple fields, the fields and their data becomes part of the URL, thus making for ugly URLs. So instead of a neat looking, small URL, you get one huge string.

Don’t get me wrong: a useful URL, even if ugly, should be preferred to an unfriendly, pretty URL. But if you want to pretty-fy your URLs, use POST instead of GET. And with a pretty URL, your user doesn’t have to know the inner workings of your form. They can still look at the source code of the page to see what’s going on, but only if they want to. Encoding the form data into a URL forces them to deal with the data even if they don’t want to.

The second benefit of using POST is that if your form contains non-ASCII data, it doesn’t form a part of the URL, which might be a good thing if your HTTP server is unable to handle this data in the URL. I don’t know, maybe all modern servers or intermediary devices can handle this stuff easily. But better safe than sorry, eh?

So from today my best practice is thus: if the form has a small number of fields, showing the submitted data in the URL is not a problem, and/or URL should be bookmark-able for future reference, use GET if there are no side-effects. If none of these requirements is met, use POST.

One concern I have is: if using HTTPS, is the URL sent in the secure tunnel or is it plaintext for all to see? According to the responses at HTTPS – is URL string itself secure??, the URL should be encrypted before being sent to the server as it is sent as part of the tunnel rather than a separate string. I think it would depend on the implementation of the client and server (I could be wrong in thinking this).

Don’t take my understanding of the situation as the final word. Read as much as you can on the subject to form your own best practice. And if you share your understanding and best practice with us, it would help us as well.

Good reads and hat tips: URIs, Addressability, and the use of HTTP GET and POST; Methods GET and POST in HTML forms – what’s the difference?; Post/Redirect/Get;