Such an insightful essay …

Paul Graham has impressed me time and again with his stunning insight. Whenever I read his writings, it’s as if he plucked his ideas out from my own head and then put pen to paper. His latest, Holding a Program in One’s Head, contains several gems that I personally have experienced several times at work.

The danger of a distraction depends not on how long it is, but on how much it scrambles your brain. A programmer can leave the office and go and get a sandwich without losing the code in his head. But the wrong kind of interruption can wipe your brain in 30 seconds.

This is spot-on, and I notice this a lot during my lunch break. Sometimes I can’t get a program or problem out of my head and occasionally I even come up with a solution not 10 minutes into my lunch break and then I can’t wait to get back and finish it. Other times however, especially if I have lunch with my colleagues the whole ‘problem space’ I’ve built up in my head simply vanishes. Due to the amount of work it usually takes (maybe a half-hour to an hour) to re-load my brain with the problem I was working on, a lot of post-lunch time is wasted and sometimes I can never recreate the problem fully again because I tend to be sharper in the mornings than in the lazy afternoons.

Since there’s a fixed cost each time you start working on a program, it’s more efficient to work in a few long sessions than many short ones.

I have often wanted to do this, but it’s almost impossible to do. There is always lunch, some other interruption, going home, eating dinner or something similar. On the weekends, however, I sometimes manage to stay up late and can work uninterrupted for quite a while.

Rewriting a program often yields a cleaner design.

True sometimes, but I agree with him that even the process of rewriting a program can lead to significant insights; even if the rewritten program is not a huge improvement.

Instead of summarizing the whole essay here, I highly recommend that all programmers and their managers go read it. Even non-IT staff, such mathematicians, whose work involves long-stretches of thinking, and constructing problem spaces in their heads will benefit from the advice in this essay.

I haven’t been paying attention to my RSS feeds recently and I forgot just how good some people are at writing and expressing their insights Paul Graham and Joel On Software are two blogs (journals?) that I really enjoy reading.

Technorati Tags: coding programming software

Links for June 3, 2007

Fedora 7 is released!: I really liked the last Fedora release, but I believe it was slightly plagued by problems with some of it’s package management utilities. I have already installed this release and am quite impressed. Wireless now works with WPA out of the box and their new re-spinning feature is something I will try out someday.

XML Parser benchmarks: I have always had my own suspicions of which XML parser model would be faster (Sax or StaX), but I’m glad to see this benchmark done by the O’Reilly folks.

Fear and loathing at Cupertino: Jeremy Allison’s terrible experience while trying to prepare a talk for his Apple WWDC presentation. Jeremy works on Samba, along with Tridge, who they all call “the smartest man in Australia” Jeremy works at Google now. Smart man.

Some web links for today

I’m going to occasionally post links here that I find particularly insightful, interesting or geeky.

Three things that caught my interest today:

PowerTOP: Released by Intel, this utility builds on work done by kernel developers to make the Linux kernel power-efficient. PowerTOP gives you a snapshot of what apps are consuming the most power. Turn off these apps or modify their behavior, and you’ll notice an instant increase in the battery life.

The Linux SLAB Allocator: Traditional heap memory managers suffer from fragmentation, among other issues. The SLAB Allocator in Linux, inspired by a similar implementation for Solaris and various embedded systems, allocates memory as fixed sized objects and uses caches to reduce fragmentation. It also has options to enable hardware cache alignment which allows objects in different caches to share the same cache lines, thus improving performance.

Advanced Linux Programming: After many years of coding mostly Java, I’ve been meaning to brush up on my C, Assembly and general Unix programming skills. I found this excellent book freely available online and it seems to be getting a lot of praise from reviewers on Amazon so I downloaded it. It has a lot of topics that I’m very interested in, like IPC and threads, and it even has a few assembly oriented chapters. I will definitely be reading this one

LinuxWorld 2006

LinuxWorld started yesterday here in rainy London and I had a great time! It was the first time ever that I wasn’t a visitor, but was helping out at the Jokosher stand. I did several demos of the app to tons of people and we managed to distribute more than 130 Jokosher flyers to interested people.

I was quite surprised at the level of interest in the app, and we managed to solicit a great many feature requests from people, some of which will hopefully end up in Jokosher someday, making it rock even harder! I will definitely try to help out at more of these events in the future

You can find the photos I took with my camera phone at my flickr photo page.

Wierd Java error on my Mac

I turned my computer on today to get some work done, started Eclipse and started coding. When I tried to use the command-line though, I got this strange error:


Error: no known VMs. (check for corrupt jvm.cfg file)


I couldn’t run 'java' or 'javac' from the command-line at all! I immediately went to /System/Library/Frameworks/JavaVM.framework and looked for 'jvm.cfg'. I have 3 VMs installed on this machine, 1.3.1, 1.4.2, and 1.5.0. 1.3.1 and 1.4.2 had a proper jvm.cfg file installed but for some reason 1.5.0’s jvm.cfg was a zero-length file. Googling didn’t turn up anything useful except this tip, which wouldn’t work because in my case all my permissions were correct. Fixing permissions using Disk Utility didn’t show any permissions problems at all.

In the end, all I did was copy the 1.4.2 version over to the 1.5.0 directory and all was well. The tip above mentions that Eclipse might have had something to do with this, and there might be some truth to that, as I did update my Eclipse to 3.2 recently, but I have been using it for a week without any problems …

Doing quick tasks in Python

A lot of the time at work I have to come up with quick solutions to various programming problems, and I mean *real* quick. There is usually very little time for finding the right library or open source toolkit that has already solved the problem or some part of it and then to figure out how to integrate that into our own workflow. It is usually much quicker to simply write a few scripts that get the job done.

In such situations I increasingly find myself solving these problems using the Python programming language. I’ve usually whipped up scripts using bash scripts and even Perl but I often find that I have to “re-learn” Perl again and again, or at least the part that I’m temporarily using to solve some problem. Python just seems (and looks) so much natural and cleaner. I’ve been fascinated by Ruby recently after my brush with Ruby on Rails and I am seriously trying to find a project to use it in all the time!

Anyway I wanted to give some examples of Pythons’ ability to solve quick problems. I’ve used it to make remote backups, check if certain services are running remotely and to fetch and delete mail off a remote server. For example, the following snippet of code can be used to send mail:

import smtplib

mailServer = smtplib.SMTP(serverURL)
mailServer.sendmail(sender, to, message)
mailServer.quit()

Really sweet and simple. I’m sure you can do this in only 1 line or even half a line in Perl (or even Python!) but this is really clear and concise. When I needed to write a script to send mail, I simply googled for it and dug this up in literally 20 secs! That’s one other reason I like writing scripts in Python: you can always google for snippets of code to do stuff.

Similarly, the following can be used to check if SSH is running and accepting connections on a remote server:

IDENT_STRING = "SSH"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, int(PORT)))
data = s.recv(1024)
s.close()

if data.find(IDENT_STRING) == -1:
sys.exit (1)
# if everything OK, exit normally
sys.exit(0)

If you put this in a script and run it, you can use the exit code (echo #?) to determine if SSH was running or not. Very useful if you’re writing a script to make backups to a remote server; you need to make sure that the service is up and running on the server and that it is accepting connections from the local machine.

I will try to see if Ruby makes any of these tasks even easier to do or perhaps easier to write and comprehend. I might even give Groovy a try since Java is my day job and Groovy is described as a scripting language for Java programmers.

Ruby on Rails

“Rails is the most well thought-out web development framework I’ve ever used. And that’s in a decade of doing web applications for a living. I’ve built my own frameworks, helped develop the Servlet API, and have created more than a few web servers from scratch. Nobody has done it like this before.” -James Duncan Davidson, Creator of Tomcat and Ant

I tried out the Ruby on Rails (RoR) framework this weekend and I’m quite impressed. I used Apple’s new tutorial for developers which explains how to install and quickly get up and running with a simple Accounts/Expenses webapp. I did encounter one minor glitch while following the install instructions because I was installing on my Debian Linux webserver, instead of an Apple machine. It seems the default ruby install no longer comes with the ruby-zlib library, but that was quickly fixed by following these instructions.

Rails applications all have a consistent directory structure, so you always start by running a command to generate the directory structure for you. This creates directories like app, config, doc, test, etc. The names are very familiar and easy to remember and they reinforce the purpose of the directory, i.e. the test directory is used to hold unit tests and functional tests for our application.

Automatic test creation makes testing an obvious and integral part of the development process, not an afterthought. It encourages you to think of a testing strategy upfront. This is perhaps the single most appreciated aspect of Rails development for me. You are supposed to test your code in other environments, but programmers often write code and if time permits, write tests. Rails tries to make it easy and painless (as much as possible) to test your code.

The next thing I learned about was the fact that Rails, like the Struts framework, tries to explicitly embed the notion of Models, Controllers, and Views within the development workflow. The concept of Actions, which are sent to Controllers in Rails is familiar to Struts. It is very easy to take a Rails URL (as shown in the example) and figure out how the server is going to parse and execute it.

Validation came next and I was suitably impressed. You simply need to add a few lines to the model, describing what each field should validate as, sort of like describing a type for a variable. Simply restart your server and voila, type rubbish in a field and Rails will highlight the field in red and ask you to re-enter it.

After all that, the tutorial shows you how to create a relationship between two models, much like in a relational database. The relation is that ‘one account can have many expenses’. This is accomplished in Rails by adding a ‘belongs to’ field in the Expense model (an expense belongs to an Account) and a ‘has many’ field to the Account model (an Account has many expenses). Simple as that.

Other things covered in the tutorial were:

• adding business logic to total up the expenses for the account
• using helpers to change the view slightly and show the total in red if we are over budget
• writing simple unit tests and running them

Overall an excellent introductory tutorial which has left me wanting more. I will definitely go through the Rails website and find out more about RoR and what it can do.

I know this is a small toy example and I want to know how RoR handles in a real mission critical business app as those are the only kinds of apps developers write these days RoR is easy to code for, but is it easy to maintain? After we get past the simplicity of the example app, how hard is it to write huge apps in it, and how long does it take a new developer to become familiar with and productive with a new codebase? These are all questions that I need answers to and I have the feeling that RoR won’t disappoint.

Microsoft at the PDC

A lot of cool stuff has been coming out of Microsoft recently at the PDC. You can watch the webcast of the event here.

Here is a short summary of some of the announcements, linking to a Channel 9 video of each (if possible):

Sparkle – separating visual components and design of an application from the data representation.

Start.com / Gadgets – nice, clean start/home page which can be extended using “gadgets”. and these gadgets can be re-used through inheritance. The windows vista sidebar will also sport gadgets which help people get to commonly used or needed data or tasks.

Microsoft Max – a kind of a cross between iPhoto and iMovie. allows you to create rich interactive photo albums with slick effects and to export these albums so that almost anybody can view them.

LINQ – a really cool idea of trying to remove the impedance mismatch between object and relational databases. From what I have seen, manipulating XML should be easier. It’s all done in C#. Creating and populating objects after using SQL joins should be easier.

WCF or Windows Communication Founcation (formerly Indigo) – Much more than just another web-services framework, it implements a lot of the plumbing that a lot of developers usually have to manage all the way from SOAP to P2P to some other method of app-to-app or pc-to-pc integration. So if you want 2 machines to talk to one another somehow you don’t have to worry too much about the plumbing going on behind all that communication. At least that’s the easiest way I see to summarize it

One the of absolutely coolest things that Jim Allchin showed off was the ability to increase the available memory of a PC running Vista by simply plugging a USB key in. It’s ingenious and I haven’t heard of anyone doing anything like it unless I am mistaken.

WPF/E or Windows Presentation Foundation / Everywhere. WPF was formerly Avalon, a new way to build rich interactive web / client apps using XML. They showed a very cool Netflix (video rental online) demo that they ran on 4 different machines and it scaled, etc flawlessly: a desktop machine, Media Center PC, Tablet PC and a PDA. Everything being vector based makes this a lot easier. Another cool North Face demo here

A lot more stuff than this has been announced of course but it is good to see at least a few cool things and possibly 1 innovation come out of Microsoft so far! They are pretty gung-ho on security nowadays and let’s hope that Vista finally gets security right (by right I mean a lot better than XP by default). Note that I’m usually *not* very pro-Microsoft, but I do believe in competition, and I hope Apple, Google, Linux et al give Microsoft all they’ve got because it makes things a lot more interesting for the consumer and gives us more choice. That is always a good thing.

WebOS and the future of the Web

[Jason Kottke](http://www.kottke.org) has an interesting write-up about how he envisions the future of the OS and the Web. The gist of his post can be summed up in the following points:

1. The OS can be made irrelevant by people writing for another OS, let’s call it WebOS.
2. People now only need to code for one platform (Java, anyone?)
3. A local web server on the client machine will mean that an app can continue on working even if it’s offline (think local app for Gmail, etc).
4. When users are back online, the app will synchronize itself with its online counterpart. Think adding Flickr pictures locally, and then uploading them.

The OS is still there whichever one it is, but it is no longer going to lock-in developers to only developing for that OS. One of the things Jason is saying is that people will no longer need to write for 3 OSes but for just one OS. This sounds just like “people will no longer have to write for 3 OSes but for just one platform: Java”.

Jason has already outlined some of the problems he envisions with this approach, such as web apps accessing local content on your hard drive. Java applets have solved this problem for a long time by using a sandbox model. The only problem is that these applets have to be downloaded and run locally for them to work. A WebOS app, however, will run both locally and remotely, it will live on your local hard drive and perhaps a sandbox model will help there as well. Are vulnerabilities detected in these local apps any more dangerous than vulnerabilities for remote apps? Perhaps there isn’t much different because at the end of the day, a security hole is the same thing whatever app it affects. However, I suppose a local security hole can do a *lot* more damage than a remote one.

Some other suggestions like: “Read newsfeeds from bloglines locally” sound no different to NetNewsWire, FeedDemon or SharpReader, etc. These are all locally accessible feed readers which, when online, update your subscriptions, etc, otherwise they work fine when offline.

I think, as [Paul Graham](http://paulgraham.com/) said, a WebOS will allow totally new kinds of applications to exist and we don’t know what they will look like at all. So this is definitely an interesting space to watch.
Some great examples of new ways of using the web are: [Backpack](http://www.backpackit.com), [Basecamp](http://www.basecamphq.com/), [Gmail](http://www.gmail.com) and [Google Maps](http://maps.google.com). Another up-and-coming app is [Hula](http://hula-project.org/Hula_Server) which has been open-sourced by Novell and is being actively worked on by many GNOME hackers. If you want to see a demo of how the future of web-calendaring might look like, take a look at [this](http://www.nat.org/2005/august/#Hula-Web-Interface) amazing demo on [Nat's](http://www.nat.org) blog.

In other news, I would love to get my hands on one of [these](http://joi.ito.com/archives/2005/08/25/wearing_firefox.html) [Firefox](http://www.getfirefox.com) T-shirts that [Joi](http://joi.ito.com) managed to pick up from the [Mozilla](http://www.mozilla.org) offices. I have ordered quite a few T-shirts from [ThinkGeek.com](http://www.thinkgeek.com) in the past, and I will order one of these if they stock them someday.

J2EE/EJB Persistence Frameworks

I need to start investigating alternative persistence frameworks whenever I get some free time. I’ve tried looking at Hibernate in the past but just couldn’t find the time to implement a small project properly in it and understand the benefits.

A great write-up about another alternative, iBATIS SQL Maps which seem to have found the sweet spot between full O/R and hand-written JDBC can be found here.

As others have found out entity beans can be overkill in situations and that’s basically why I’m looking for some persistence alternatives. I’ve got to buy this book on J2EE Without EJB as well at some point, I’m sure it’s a great book as I have another one by the same author.

On the Swing/AWT front, SWT seems to be emerging as a good alternative for cases where you need a simpler interface which doesn’t need to have all the customization benefits that Swing gives you. Some evidence of people who like this approach is here and here is Bruce Eckel’s take on what Gosling has to say about SWT. Basically Gosling prefers the flexibility of Swing and the complete emulation and hence customizability it gives you while Eckel and some Smalltalkers think Swing takes it too far and suggest that using the interface objects underlying the platform and wrapping them to give a Java interface is a much better idea and yields better, more lightweight and faster code.

I have to agree on the faster and lightweight issues, even though the only SWT app I use is Eclipse. I know Eclipse is huge but it feels a lot snappier than a pure Swing implementation of it would have been. It has *never* crashed on me and much simpler Swing apps have crashed for me in the past. It is also proof that SWT is indeed suitable for huge projects.

Hiding the menubar on OS X

I’m quite fond of the Apple mailing lists, particularly java-dev and cocoa-dev. They’ve helped me solve quite a few problems on occasion. I once needed to hide the menubar on top of the screen in OSX and I found that it is as simple as this:

[NSMenu setMenuBarVisible:false];

Let’s say you want to pass some arguments to your Cocoa program from the command-line, so how do you do it? If you look at the main.m file that XCode generates you can see that all it does is create an instance of NSApplication and passed the arguments to it, but where do these arguments go? How do I access them from within some other class. Once again, the simply answer, from cocoa-dev is to use NSProcessInfo as so:

NSArray *args = [[NSProcessInfo processInfo] arguments];

This puts them all in a nice array for you. Took a while to figure all these things out, many thanks to the Apple mailing-lists.

Just in case you’re wondering why I didn’t read the API docs, well it isn’t obvious from reading any of the docs that command-line arguments end up in NSProcessInfo. I googled forever but couldn’t find an explanation and read many docs. In the end the lists had already got the solution and I should have looked there in the first place.

Teach Yourself … in 24 hours :)

Nice review of Linux Programming By Example on slashdot today. The book goes through the source of the Unix V7 code for the ls program. The second part then delves into processes, etc. I have a similar book at home, Linux Application Development which is quite an old one, but a good book overall. It also covers the make utility in the beginning which I think the Example book should have covered as well.

The author of the review linked to a good article on how there are so many ‘Teach Yourself in 24 hours’ or ‘21 days’ books out there these days. It does indeed take a long time to learn to program well. You have to make a lot of mistakes, fix them, across multiple languages and generally it does take you several years of exposure to become a competent programmer. I do suppose that if you already know 10 languages, you should be able to pick the next one up in 2 weeks or so? I suppose these books do fill that niche.

Weather Parser

I’ve packaged up the InterceptVector parser and related files into a tar file and you can download it from the software section. There are some examples in there as well to help people get started. Hope whoever uses it likes it, let me know what you think.

Nice article on NewsForge today about the difficulties of being a web developer in Africa. Makes me really wonder about the things I take for granted everyday, such as electricity. Load-shedding, a practice used to save electricity is used often and usually without notice. Adding to the problem is the egotistic attitudes of programmers, who won’t work for anyone not looking like Bill Gates

Here’s a nice excerpt:

After writing yet another program, you just get fed up. I mean, what happened to all those bonuses that you were promised? Why aren’t you riding in a nice flashy car like all the managers? In fact, how come the managers have so many nice cars and you are so poor? I thought I was doing good work!!!

I don’t plan to live my life earning less than $300 a month.

[You hear a little voice in your head ...]

Welcome to the real world, buddy!! Let me explain things to you, you simply have to understand. You’re a big fish in a small pond. There’s not enough water to go round, so sorry if you are feeling a little uncomfortable. After a few more years, your body will get used to it and you will become a small fish, much more comfortable you know …

Code reviews and Peer Programming

Kasia has been trying out code reviews/pair programming at her work and I do tend to agree that code reviews should be done and that they probably result in better code, i.e. code that is less full of bugs We don’t do much code review either at work and after reading her entry it got me thinking that perhaps we should.

When 2 developers at least are working on 2 sides of the same problem I tend to think that pair programming could be very useful. Personally it sometimes does get on my nerves, especially when we are talking about the same thing but thinking about it differently, or there is a subtle difference in what we are actually talking about.

In the end, though it does tend to keep people focused on the task at hand and forces us to quickly work out a solution or pull each others hair out

JavaScript and Motif

Ok, so I kept almost true to my promise today. I studied JavaScript in the morning and I studied Motif in the evening. I also managed to watch a bit of Alien Resurrection in between. That’s an excellent movie. I’m studying up on JavaScript and HTML in order to become a little bit proficient in web page design and programming. I hope I’m not wasting my time.

I was really surprised when my dad scanned one of the pictures from the graduation and brought them home. It’s a picture of me, Nitin, Siddiqi and Christiana standing together. He made it brighter, fixed some red eye problems and I scaled the final pic down to around 20k. It looks quite good now. As soon as I get a few more, I’ll start putting them up. Fixing the picture and setting up my web page once again consumed almost an hour of my time

GTK or QT?

Today was Friday, our holy day, and of course I was sleeping until 12! The prayer was about to start in half an hour and I had to rush to take a shower, eat some breakfast (??) and get dressed (actually, I got dressed before the breakfast). Then off to pray I went.

Afterwards, and after lunch, I once again connected and finally received an email from Evangi!! Great. I sent a reply telling her I’ll have to get back to her later.

After reading the news, I downloaded a tutorial for the GTK toolkit, which Enlightenment and GNOME use for their GUI. Tried and successfully compiled the first example (they were kind enough to supply Makefiles). I like GTK so far, and the programs and the source code looks so sexy! But now I don’t know which one to learn, GTK or Qt? I’ll have to ask someone. Still later on, I read a bit of the Linux Programmer’s Guide, which I downloaded in postscript form. That thing is really incomplete.

Finally today I hope I will be able to go jogging. Keep fingers crossed, yessss???

My friends read something from Nostradamus and they are certain that something BIG is going to happen on the 4th of July. Let’s see. Two days to go