When I Choose Python

I choose Python to write mostly small scripts, helper tools, s’il vous plaît. I use it to process text files to extract and transform data (poor man’s ETL). I use it to automate routine manual tasks so I can do bigger and better things with the saved time. I have even (successfully) attempted to write a prototype B2B web application for my employer. I now understand the range of tasks I can accomplish with Python.

But the right tool should always be used for the job. I am not a programmer. I program a bit here and there but that is not my primary task, interest, or goal. I don’t write enterprise software. This is why I write scripts and not software. And Python is a very handy and helpful language to write them in.

I have learned and used C, C++, Java, and Perl. They all do wonderful things on their own but the kind of work I do, they are either too unwieldy or Python beats them in “having fun while creating”. When I choose Python, I don’t choose it because it is the best language in the world. I choose it because it gets the job done for me and saves me time. I am not saying that Python could not be used to write enterprise software and I am not saying it could. But for a person who works with data, database, networking, and voice technologies, Python is a useful tool. Use it where appropriate and don’t try to ram it down a project if it’s not a good fit.

N.B. The idea for this post came from Bad news: Google employees are being discouraged from using Python for new projects.

Code Ghar Official Linux Desktop

So I decided to take the plunge and use Linux on the desktop. This post describes how the first ever official Code Ghar Linux desktop was created. As I make changes to the computer, I will keep this post updated.

Distribution

I first installed Fedora Rawhide (nightly build) using ext4 as the filesystem. After a couple days, I decided to add more memory to the system. Once this memory was installed successfully, the OS would not boot at all. I decided to not risk things further and went with Debian testing (Squeeze) instead. I am more comfortable with a Debian based system anyways so the choice wasn’t hard. I chose the netinstall CD and only installed the command line (did not choose the Desktop option during install).

Desktop Environment

I wanted to make best use of the 3GB RAM in my system. So I chose to install LXDE as my desktop environment. Since I was only running command line, I ran the following command to install LXDE.

sudo aptitude install xorg gdm lxde

Wine

I needed to run three main applications. I could either install a virtual machine running Windows to run these apps or install Wine. I decided to try out Wine. To install Wine, I ran the following command.

sudo aptitude install wine

Once installed, all you have to do is run the exe file to install your desired application and Wine takes care of the rest. But make sure you have searched that application on the Wine Application Database.

Virtualization

However much I want to move away from Windows, I just can’t because of some applications. I chose to use VirtualBox although I have been using VMware on Windows and Mac for a couple years now. To install VirtualBox on Debian, I followed their Wiki (VirtualBox Installation). Following are the commands I ran.

sudo aptitude install virtualbox-ose virtualbox-ose-modules-$(uname -r)

sudo invoke-rc.d udev reload

sudo modprobe vboxdrv

sudo addgroup vboxusers

sudo adduser codeghar vboxusers

I then logged out and logged back in for VirtualBox to be ready to be used.

Remote Access

To allow remote access to my machine using VNC, I chose to use X11VNC. To install it, I ran the following command:

sudo aptitude install x11vnc

To invoke it for my user (since I have a single-user machine), I ran the following command once I logged in. This command is all on one line (in case the formatting of the blog makes it appear different)

x11vnc -display :0 -xkb -bg -forever -passwd mypassword

What this does is that it allows you to access your actual desktop on port 5900, runs in the background (bg flag), does not stop after first remote client disconnects (forever flag) i.e. keeps running, and assigns a password (passwd) which must be used to authenticate yourself. The xkb flag was used because for some reason the shift key did not work when I ran a VNC client from another machine into this one. I forgot which resource I used to fix this issue but I will put up the link once I can find it again.

If my computer is restarted, I will have to run the same command again for me to be able to use VNC. This is why you need access to at least SSH to be able to run this command and start VNC. I added the following alias at the end of my ~/.bashrc file. This way I don’t have to retype the command all the time.

alias myvnc="x11vnc -display :0 -xkb -bg -forever -passwd mypassword"

Flash for Iceweasel (Firefox)

To install Adobe Flash, I followed the instructions on How to Install Adobe Flash in Debian Etch/Lenny/Sid. As they were very generic, I took only what I needed and did the following.

First I downloaded the debian-multilemedia-keyring package on the desktop. Then I ran the following command to add it to aptitude.

sudo dpkg -i ~/Desktop/debian-multimedia-keyring_2008.10.16_all.deb

The I modified my /etc/apt/sources.list and added the following line:

deb http://www.debian-multimedia.org testing main

I then ran the following command to update aptitude and then install Flash.

sudo aptitude update

sudo aptitude install flashplayer-mozilla

Remote File Access with SSHFS

I used the guide at Mount a remote file system through ssh Using sshfs to get this working. I strongly recommend that you read it thoroughly for better information. To install SSHFS, you also need to install FUSE, as below.

sudo aptitude install fuse-utils sshfs

Now you need to do two things: load the FUSE module (as below)

sudo modprobe fuse

and also add your user to the fuse group (as below)

sudo adduser codeghar fuse

I then had to log out and log back in for my user to be recognized as part of fuse group.

I then created a mount directory in my home folder, created folders for all remote file systems I need to access, and created aliases for mounting and un-mounting SSHFS “shares”.

mkdir /home/codeghar/.mount/
mkdir /home/codeghar/.mount/server1
mkdir /home/codeghar/.mount/server2

To mount server1, I ran the following (also added this as a bash alias to save typing):

sshfs codeghar@10.10.1.101:/home/remoteuser/ /home/codeghar/.mount/server1

To unmount server1, I ran the following (also added this as a bash alias to save typing):

fusermount -u /home/codeghar/.mount/server1

Miscellaneous Applications

Following are the applications I also installed.

OpenOffice: sudo aptitude install openoffice.org
Wireshark: sudo aptitude install wireshark
gvim: sudo aptitude install gvim
VNC client: sudo aptitude install vncviewer
Grsync: sudo aptitude install grsync
Pidgin: sudo aptitude install pidgin
PDF: sudo aptitude install epdfview
Samba: sudo aptitude install samba samba-client smbfs
Convert between DOS and Unix text files: sudo aptitude install tofrodos

When to use GET and POST

In HTTP and/or HTML, there are two (main?) types of submissions: GET and POST. I have always had a hard time determining when to use GET and when to use POST. In other words, what is the main difference between GET and POST?

Quite simply, if the submission is reading data, without making any changes, use GET. If your submission will be making changes, or causing side-effects, use POST. For example, when doing a query, such as a Google search, the form should use GET. If you are creating a new account with Yahoo! mail, the form should use POST. The Google search is reading data and new account in Yahoo! mail is changing things.

Simple enough, right? Not so, as there are some situations where either could work. One of the best descriptions of the problem I have read is Methods GET and POST in HTML forms – what’s the difference? And here is what I learned from it (you might learn something else reading the same words):

When doing a submission which will not make changes use GET. When making changes, use POST. But you may want to use POST instead of GET in the following (major?) case: when you don’t want the data you submit to become a part of the URL. When using GET, the form you submit becomes a part of the URL. So if you have a couple fields, say myname and mynumber, using GET your URL might look like http://www.mydomain.com/someform.html/?myname=codeghar&mynumber=1234 after submission. The biggest benefit is that you can bookmark this URL and visit it in future as just another link. The biggest disadvantage, in my opinion, is when you have multiple fields, the fields and their data becomes part of the URL, thus making for ugly URLs. So instead of a neat looking, small URL, you get one huge string.

Don’t get me wrong: a useful URL, even if ugly, should be preferred to an unfriendly, pretty URL. But if you want to pretty-fy your URLs, use POST instead of GET. And with a pretty URL, your user doesn’t have to know the inner workings of your form. They can still look at the source code of the page to see what’s going on, but only if they want to. Encoding the form data into a URL forces them to deal with the data even if they don’t want to.

The second benefit of using POST is that if your form contains non-ASCII data, it doesn’t form a part of the URL, which might be a good thing if your HTTP server is unable to handle this data in the URL. I don’t know, maybe all modern servers or intermediary devices can handle this stuff easily. But better safe than sorry, eh?

So from today my best practice is thus: if the form has a small number of fields, showing the submitted data in the URL is not a problem, and/or URL should be bookmark-able for future reference, use GET if there are no side-effects. If none of these requirements is met, use POST.

One concern I have is: if using HTTPS, is the URL sent in the secure tunnel or is it plaintext for all to see? According to the responses at HTTPS – is URL string itself secure??, the URL should be encrypted before being sent to the server as it is sent as part of the tunnel rather than a separate string. I think it would depend on the implementation of the client and server (I could be wrong in thinking this).

Don’t take my understanding of the situation as the final word. Read as much as you can on the subject to form your own best practice. And if you share your understanding and best practice with us, it would help us as well.

Good reads and hat tips: URIs, Addressability, and the use of HTTP GET and POST; Methods GET and POST in HTML forms – what’s the difference?; Post/Redirect/Get;

State of Linux

Linux’s world has seen very interesting recent developments. But two that stand out the most for me are Debian’s decision about scheduled freezes and the drama surrounding CentOS. Both can be seen positively or negatively but both, in my view, have improved the state of Linux.

First, let’s look at Debian. I had read Mark Shuttleworth’s interview on Gnome 3.0 some time ago. In it he had mentioned talks between Ubuntu and Debian on freeze dates. Now that Debian has confirmed this idea, it’s official: Ubuntu and Debian can now sit down every two years and discuss what to freeze together. It means that Debian can better plan itself by setting hard deadlines on features. Once that freeze occurs, they can focus on making things as perfect as possible before releasing.

Debian has always been known to release when they are ready to release. It was the Debian way, and worked for their users. If someone disagreed, they could use another distro. This wasn’t very helpful to enterprises as they like to know a semi-formal schedule for maintenance and support before committing to a thing as important as an operating system. All users get good quality but that extra bit – a schedule – especially helps enterprises.

Some would see this move as a betrayal of the Debian-ness of Debian. They would point to the idea of time-based freezes as compromising quality. But they would be mistaken. A freeze is not the same as a release. A freeze means for two years developers work on new features before they spend all their time improving the quality of these features.

Debian previously supported a release for roughly three or four years, not because they set out to do so, but because of their release schedule. Now we know they plan to pursue a two-year cycle and I don’t know how long they plan to support a release. If it’s two to two-point-five years, enterprises might not want to invest in Debian for production systems expected to run longer than that. They would rather than Red Hat’s promise of five to seven years. But if Debian can partially match Red Hat’s promise, it now becomes a very good alternative for enterprises.

This also means that Debian stable and Ubuntu LTS could be very closely related, and users get a choice of which goals they want to support. Do they want the Debian goals or Ubuntu goals? I support both because they both provide choices for different set of use cases. But I also love that I can now use Debian when appropriate and Ubuntu when it fits the bill.

I love Debian because of the sheer amount of choice it offers. Red Hat doesn’t match this choice (because it’s not financially feasible?). I am beginning to like Debian with LXDE as GUI (if required). Red Hat doesn’t have it by default. Yes, there are third parties offering this but if you are paying Red Hat you would want Red Hat to provide these things. They don’t and if you are going to third parties, why not just use something which provides these choices under one umbrella?

Combine Debian with the inroads Ubuntu and Canonical are making, and you have a tasty recipe for success. If we continue to support this alliance with our money, it will continue to become a viable alternative to Red Hat for enterprises. Once enterprises begin to support Debian+Ubuntu+Canonical, hardware vendors are more likely to support this alliance; an all-round victory for all.

I am not against Red Hat. In fact, I am a big fan of their success and contribution. I use RHEL and CentOS whenever I can. But I am also a big fan of using the right tools for the job. And for this idea to work, we need choices.

Now let’s look at the CentOS “debacle”. I have always trusted CentOS to provide the best product they can. Recent events have not dented my confidence. In fact, they have reinforced the idea of not relying on one person for anything. At least two or more people should have access and ability to manage all aspects of a group or project. You know, the “hit by the bus” scenario. CentOS can only improve from this moment.

CentOS provides a great service to all Linux users. It builds on the good work of Red Hat and introduces more users to the benefits of using Red Hat, thus supporting them. All contributors to CentOS should be lauded for their voluntary efforts.

And then there are those who had little confidence in open source to begin with and which has decreased significantly because of recent events. They see open source in general, and Linux in particular, as less reliable than proprietary solutions. I think the reason is not technical but commercial: who will support me if I use this product? I would rather not have such people using Linux then. If they don’t believe in open source, they don’t deserve it either.

I know of the fight between “open source” and “free”. I prefer “open source” because “free”, although a laudable goal, is not practical for all use cases. I would rather everything be “free” but if it’s not workable then I would rather support “open source” than proprietary.

In conclusion, I think Linux is better off today than it was a week ago. Debian is changing itself to keep pace with the changing world. CentOS has gone through a rough period to emerge stronger. Things don’t always remain the same but if they are getting better, it’s a victory for all of us.

Choosing Server Names

A very interesting discussion over at reddit (What is a good name for a server?) got me thinking on how to choose server names. In the same discussion someone posted a link to RFC 1178, which contains guidelines on choosing names for computers. I thought why not try to come up with some alternate ideas?

I am of the opinion that one should pick a theme and choose names from within it. For example, a very popular theme is mythical creatures or deities. If you choose Greek gods and heroes, then all your servers are named using their names. Extending this idea, why not pick different themes for different types of servers and devices? For example, Greek deities for servers, colors for printers, classical musicians for network devices, etc. This allows one to identify not only the device in question but also its general classification.

Sometimes it’s just easier to name servers after their function when users are neither very creative nor want to be. For example, if a bunch of servers are for databases only, they could be called DB1, DB2, and so on. Not much fun, obviously, but some situations might warrant this approach. Therefore, we need to keep an open mind.

I mostly prefer lowercase names because they are easier to type. They are not passwords where case sensitivity might be expected or desired. And since almost all commands on Linux/Unix are lowercase, it just flows with it if that’s your platform of choice.

Some people might be consultants and may have to choose names for devices in multiple organizations. In such cases you might want to choose themes for one organization and not repeat them for others. It might be a bit more difficult to know which Hnoss if two or more organizations have the same name for their devices.

This discussion is not limited to device names only. Logins and passwords could also be created using themes and their set of names.

Some suggestions for themes: types of sports (soccer, football, baseball, cricket …); sportspeople (tiger, graff, sampras, jonty, warne, nadal …); rivers (nile, indus, ganges, danube …); poets (keats, angelou, iqbal, bronte …).

What is ‘wheel’?

I came across a very interesting article titled Why GNU su does not support the ‘wheel’ group. I have seen wheel under Red Hat Enterprise Linux but never understood why such a special group existed. I didn’t even bother to find out. But the aforementioned article made me curious. So off to Google. The wheel Group was the one I was looking for. According to it, wheel is a special group which in conjunction with settings for su determines which users are allowed to use the su command. So if you have a wheel group set up then you may configure su so that only members of wheel are able to use the command. As one commentor on the post put it, this creates a circle of trust, and is called wheel.