Memorable moments in code reading

Reading through other people's code can be tedious. But sometimes you come across a comment that makes it all worthwhile. Thanks to the httplib2 contributor who added this gem.

Creating HTML5 Cache manifests for GWT Apps

I've been working recently on making Springpad work offline using some of HTML5's new features and want to share a solution to a problem I ran into generating the cache manifest file. The cache manifest is a text file containing the version number of your manifest file and the paths of the files on your site that you wish to cache. For more specifics on how to set up an offline application with this file check out Mark Pilgrim's excellent Dive Into HTML5.

That's pretty simple. Generally, when you make a web application you know all of the resource names and locations and can add those to the manifest. Unfortunately, when you are using GWT, the situation is slightly more complicated. GWT compiles javascript files from your Java source code, one per browser platform you are targeting. It names these files by generating a checksum based on the contents of the file. Then it creates a simple javascript file that loads the correct javascript file based on the browser's useragent. So this means that everytime you make a change to your application and recompile, you will need to specify a different javascript file in your cache manifest.

You could do this by hand, but that's tedious and likely to be forgotten at some point. Alternatively, you could do some regex'ing of the bootstrapping file and figure out from that what javascript file to put into the manifest. This is better than doing it by hand, but seems hacky and painful since that javascript will be obfuscated in production.

The solution that we settled on was to implement a custom GWT linker to generate the manifest file. GWT allows you to create your own linkers to either override their linker or to supplement it. Here's the code for the linker which will find the compiled name for the javascript file that we want in our cache manifest and then output a cache manifest will that file in it. For right now, all we care about is the Safari user agent, so that's all that I'm handling. Other browsers also handle offline, so in that case you would want to modify this create a different manifest file for each user agent and then have your server serve the appropriate one based on the user agent in the GET request. I've added comments inline to explain how our version works. Hope this helps.

Test Driven Development

At this point, I guess Test Driven Development (TDD) is old news but while I've always liked the idea and have even made preachy statements about unit tests in the past, I've found it difficult in practice to actually write in a test driven way. I don't mean that I don't always test first, but that frequently when I'm in a rush, I test-never. This has come back to bite me enough that I've gradually started writing more tests when it was easy to do.

I recently came across this screencast of Gary Bernhardt using TDD to create a simple string calculator. Gary's velocity is super impressive, but more importantly the video shows how fast TDD can be, especially with a good key mappings to run tests and a split screen.

I watched this at a good time because at Springpad we've recently started a new project all in python. After reading some more about TDD, I've been trying to make a habit of this style of development. So far it's been great. I think the big advantages for me so far have been:

Better than a compiler: especially in a dynamic language these are key to make sure you don't have any spelling mistakes or stupid problems. But they're so much better too obviously. Why check that the method exists, when you could check that it behaves as expected before runtime.

Staying out of the debugger is fast: Since starting this, I've tried to write enough tests at the right level such that I don't need to debug the program ever. And I've been mostly successful which for me has really sped things up.

Deeper understanding of the codebase: Since you spend more time thinking about behavior not only of your code but also of libraries that you use so that you can test it right, you end up knowing the code a lot better. Seems obvious in retrospect, but I didn't start TDD for that reason.

I think there's still a lot to figure out with this. Should you use mocks versus stubs? Does a test belong in a separate integration suite versus the unit tests? Etc. So far I haven't created any strong opinions on this, but I'd love to hear anyone's thoughts on how it should be done.

Faster Development with JRebel

Update: In response to the request that I post the problem to forum at Zero Turnaround I was debugging the problem that I had using the intellij run/debug with JRebel buttons some more and found that it was caused by a problem with our code and hibernate. The short version is that there was a String property with getX/setX methods but someone had also added an isX method that returned false if it was null. This usually worked fine, but for some reason when running with JRebel hibernate would get confused and try to set the string value with the boolean from isX() which caused a buried IllegalArgumentException. With that fixed, JRebel works perfectly out of the box. I've changed the post to remove my workaround as it's not necessary any longer.

Why JRebel?
I'm going to walk through the steps I took to get JRebel working with maven and Intellij 8 since I ran into a few snags along the way. But first, I should explain why bother install it all. JRebel is a great tool that promises to increase your productivity in Java by enabling all sorts of HotSwaps not normally allowed by the JVM. While you are running your software in debug mode, you can add or remove methods or classes and JRebel will swap these changes into the JVM. No more restarts when you decide to add a new method. JRebel even support making Spring or SpringMVC configuration changes without a restart and it supports plugins so that similar functionality can be added for most tools in the java eco-system. Beyond the obvious time-savings you get from not having to wait for your server to restart, I find it also helps me because I don't have breaks in concentration while I'm waiting for the server to restart. Check out the video from ZeroTurnAround's website for a quick demonstration of how it works.



My Setup

I get the feeling that the sticking point for JRebel in terms of adoption is that it can be tricky to configure. They actually provide a really good series of how-tos for different setups but it's hard to cover all of the bases. Most developers at my company use this setup:

• The build is defined in a Maven build file


• We debug the application in jetty running it with the maven run-jetty goal


• We write code in IntelliJ 8 using the maven plugin to generate and update the project file


• Debugging is done by running the app from IntelliJ using the maven Run configuration

I tried a few different ways to make this work, and the following setup is what ended up working and being the easiest in terms of development. Much of IntelliJ setup and tips came from here.

Step 1: Install JRebel

Download JRebel from their website and install it where ever your prefer. I put mine in /usr/local/java/jrebel

Step 2: Install the IntelliJ Plugin



In IntelliJ go to Setting->Plugins, click on the Available tab and right click on the JRebel Plugin and install it. You'll have to restart IntelliJ after this is complete.

Step 3: Configure JRebel Plugin

Go to Settings->JRebel in IntelliJ and set JRebel location to where you installed JRebel (e.g., /usr/local/java/jrebel)

Step 4: Make sure IntelliJ and Maven's output directories are the same



You need to ensure that IntelliJ and Maven are using the same build directories, otherwise when IntelliJ compiles a changed class it may put into a different directory from where JRebel expects to find it. Go to Project Structure->Modules->Paths and ensure that the output path is the same as what Maven uses when it builds.

Step 5: Make sure that IntelliJ does not build a WAR

There's no need for IntelliJ to build a WAR file since jetty-run uses an exploded WAR directory. So building one each time that you compile will just slow things down. Go to Project Structure->Modules click on the Web facet and then and under Java EE Build Settings make sure that "Create Web facet war file" is not checked.

Step 6: Turn off IntelliJ's hotswapping



If you don't do this, then everytime that you make a change that the normal JVM cannot hotswap, you will be informed that you've made unsupported changes. It's also a waste of time for it to try and hotswap it. That's why you're installing JRebel. Go to Settings and search for "hotswap". Click "never" for "Reload Classes After Compiliation".

Step 8 (optional): Remap the Compile command

I found it helpful to remap the save key (Command-S) to the compile command. So now whenever I make changes all I have to do is hit Command-S and the changes I've made are compiled into the running instance.


So that's it. Now you can start your application in debug mode. You will see a JRebel startup message in the console immediately. And then you can start adding classes, deleting methods, and when you compile, JRebel will swap it all into the JVM for you... no wait required. It gets you a lot closer to the experience of developing in a dynamic language. Now, if we could only do something about that boilerplate...

Better than chicken soup for the soul

This is the kind of fable that confuses good atheists such as me. A story where it almost seems like the plot is directed by a divine and most assuredly benevolent hand. Switching relious paradigms, it's more likely it was karmic payback for sending me to Iraq... the pat on the back after the kick in the balls. Then again, it was probably just good luck.

So let me paint the picture for you. I'm in the Q-West dining hall. There are many food options, from main line (comfort food) to fast food to healthy choice or stir fry. I've been eating here since in November and the selection has become rather uninspiring of late. It's likely that I've just eaten there too many times, but it really seems like stopped trying and don't use any spices anymore. Maybe they hired a new executive chef? I'm not sure. But I'm not excited as I walk through the serving area and consider the options.

And then a moment that might have caused a religious man to say that the lord helps those who help themselves. The Good Lord may not give you a 24 oz. sirloin steak but he may just provide you with what's necessary and with a little good old fashioned Christian elbow grease and ingenuity you could have a feast.

It happened like this. I approached the healthy food line and saw rice (no surprise there), some shrimp with noodles (sea food in the desert?), stewed tomatos (don't see that often), and asparagus (wait... stewed tomatos?!?!). And then inspiration, the good KBR has provided me with sumptuous rice and stewed tomatos and the fast food line always has hot dogs so.... DAMN RIGHT! I ate hot dog stew for lunch today!

Now that's what I call chicken soup for the soul.

Revisited from a previous blog: Letting Go

My buddy gcole shot me an email pointing to an old blog that I had posted to once and forgotten about. It was funny to read it now almost four years later. It's not embarrassing except for maybe the comparison to an A&F model that I'd probably leave out were I to do it again. More than any of that, it's just really interesting to read it now, a marathon and many assorted miles later. I still have a similar feeling about running now as I did then... I can't be totally away from it for too long for fear of getting too out of shape. But two years of running with Aimee has really matured that impulse and I'm not so in love with the notion of hard core running training or with "great" performance. That said, I run a lot further now then I did back then and it has become a lot more enjoyable. The other thing that I find striking as I read this is the blatant void waiting for climbing to fill it.

Thursday, April 14, 2005
Letting Go
I was a college athelete. For my four years of college, every season was spent in some way preparing for or recovering from track season. At the beginning of the school year was fall track or pre-season. Mostly we ran around the track for various amounts of time and at various speeds. Some of those days I would also lift weights. Then it would snow in Maine and get really cold. Being a sprinter, I would wave good bye to the distance runners as they left for long, cold, snot rocket-filled runs along the roads around Waterville. Meanwhile, we would run around a shorter track for various amounts of time and at various speeds. Occasionally we lifted. Oh and we often would do elaborate warm up drills that involved skipping, hopping, and lunging. These motions were often accompanied by elaborate hand motions. This was indoor track. On saturdays, we ran around the track less then normal, but much faster. Then spring came to Maine and even though there was still snow on the ground, we moved back outside because spring is the time for outdoor track. During this season, we ran around the track, lifted, and accomplished the jumping and hopping warm ups somewhat more gracefully than when we were indoors. On Saturdays, we ran around the track faster than we had on Saturdays during the winter.

So what's my point?

My point is that all of this running around in circles has a weird way of becoming habit forming. It's not something that's easy to explain and if you had spoken to me at the end of my senior year, you might not have predicted it at all. At that time I was declaring to anyone who would listen that after graduation I was not going to run farther than the bathroom -- and then only in cases of dire emergencies. It took all of about two weeks for me to break that promise. And it probably would have taken less if I hadn't been so desperately sick after the rampant binge drinking of the week before graduation.

So why was I addicted? I've had plenty of time to think about it in the five years since. Part of it was probably the look. Sprinting and lifting has a great way of cutting your body fat down to a minimum and since I was already lean it cut me down even more. I was as cut as Abercrombie models -- just not as big or cute, but still its the abs that count, right? That was certainly part of it. Deny it to myself and others as I might, I liked the way I looked.

Then there's the arrogance angle. I certainly hadn't peaked by the time I graduated. Sprinters don't peak until their late twenties or even early thirties. Of course, most sprinters don't continue to train through their late twenties to early thirties. But nevertheless, I knew that I could only get better and now that I was freed of the ridiculous restrictions of my coaches, the sky was the limit. No more worrying about overtraining or any of that crap. Overtraining is for sissy pansies.

So left to my own devices, what happened? It wasn't exactly the mechanical 200 meter workouts run at ever increasing speed. Instead, it was surges of motivation when I would run a lot and lift. Lifting was never a huge problem. I just did it. I never got huge, but I stayed in good shape. Running, on the other hand, presented a more difficult problem.

I was easily caught up in waves of enthusiasm. I would be a fast miler. To be a fast miler, I would run ridiculous amounts of mileage as quickly as possible. I wanted results and I wanted them yesterday. I could easily run ten miles. So I did it. Addicted to the riteous feeling of being an athelete in training, I would run six or seven miles the next day. If it was hard, that was only proof that the training was good. Within a week or two, I would be injured. The giddiness of the images of greatness would be replaced by frustration and impatience. If it wasn't for my knee, I could be running so much faster. Impatience usually led to an early attempt at a comeback - if you can 'comeback' to something that you've only been doing for two weeks consistently.

You might think that a smart person would recognize the pattern after the first or second time that it happened. In fact, a smart person probably would. I've cycled through that pattern at least six or seven times in the last five years. In the midst of all of this, I became an avid reader of workout books and magazines. From Men's Health to Runner's World; Daniel's Running Formula to The Book of Muscle, I read them all. Imagining the effects that each carefully laid out program would have and debating whether it made sense to get too big from lifting if I really wanted to be running a fast 5K and perhaps even throw in a marathon just for show.

I demanded of myself that I be on a workout program. If I was going to be spending time working out, I needed to make sure that I was maximizing it. I needed to get the best possible workout effect, build the most muscle possible, and gain as much speed and endurance as I could so that...

Now let me backtrack for a second. Maybe it makes sense. I wanted to get bigger and faster so that I could compete in races. That's fun, right? There are very few track meets that I ever enjoyed. The races might be fine. But the time leading up the a race was pure torture. It usually began on Friday night, when friends went out and drank themselves into an unthinking stupor like good citizens. During this period, members of the track team usually congregated. It was fun to be with the guys, but sometimes you could feel nerves beginning to show. And by the time, you tried to go to sleep they were in full effect. By Junior year, I was so impatient with the routine that I took Tylenol PM the night before every meet. At breakfast, the food was unappealling and often hard to eat at all. Swallowing almost triggered the gag reflex on a routine basis. And once we had arrived at the meet, being surrounded by the whirl-wind of activity that is a track meet made it much worse. I feared the start of my race and every event that indicated that my event would indeed run and that it was getting closer only intesified those feelings. I was afraid that I would not run well. I was afraid that I would chicken out and not run hard enough. I was afraid that I wouldn't be able to run, because my legs felt completely weakened. All of this introspection led to one thing: self loathing. I hated that I was so nervous. Too inexperienced to realize that everyone gets nervous before major events and there's no way around it, I assumed that it was a character flaw. For a really great explanation of how it feels to waiting for a race read John L. Parker's Once a Runner. Parker's description is amazingly accurate. I get butterflies just reading it.

So I guess the point of that whole digression was to make the point: no, racing is not particularly fun. Or at least, it wasn't for me in college. So what was I getting ready for? Well, I liked the idea of racing and have done it occassionally, but not regularly. Beyond that I think that I have simply been addicted to the idea of being in shape and of being the athelete.

Along the way I have ignored so many things. I tried Tae Kwon Doe for several months and enjoyed it. I was even beginning to get good at being a White Belt with a Yellow Stripe. It was fine until I realized that all of this time spent punching and kicking was really taking away from my running. I hadn't been running in forever. I was getting out of shape. I wouldn't be able to...

I tried yoga once. It was a little new-age for me, but I did enjoy it in spite of myself. I think that it could be a great way to reduce stress, increase flexibility, and just generally feel better. Unfortunately, it couldn't contribute to the bottom line. Would I run faster? Would I be able to lift more? It was cut before it even made the team.

Worst of all, during these years of working out, I even ignored sports. A lot of people join soccer or basketball leagues to hang out with friends, have a good time, and hey-what-the-heck get a little exercise too. I didn't have the need for extra exercise and couldn't see fitting a workout plus a game into my schedule. So I haven't played too many sports in a while.

Most recently I have been following the Core Performance program with a co-worker. It's a great program and really addresses a lot of aspects of fitness. If you need to get in good competetive shape or have the time and just feel like getting in great shape, I would definitely recommend it. For me, it's just another in a long string of paper coaches who I have substituted for the living coaches that I followed in college. Ironically, I left school thinking myself free of the tyranny of organized workouts and then quickly sought them out in books and magazines as soon as I was left to my own devices.

Core performance workouts have left me tired and sore. I feel the same satisfaction that I got five years ago when my hamstring would cramp up after an intense workout. But is all of the time and energy worth it? Exercise is cited as a huge stress reliever, and I believe that it is. But does it really workout that way, when working out is just another stress in my day? Should I rearrange my life to accomodate a demanding workout program?

Today, I decided no. This week I haven't had a chance to workout. And it felt crappy. At the same time I was thinking a lot about how much working out should impact my life. Tonight around 8:30, I knew that I just needed to go for a run. I opted for a short 2.5 mile run around a pond. I didn't time it. And I didn't concentrate on my pace. I enjoyed the feeling of going fast but didn't feel like I needed to press it further. When I was in sight of my car, I stopped running and walked to it.

During that run, I resolved to stop being a slave to working out. Exercise is important. But only because it makes you feel good and live better. It's not an end unto itself. I resolved not to be a slave to programs. Just like eating the same food every night for dinner gets boring, so following a single workout program. The only reason to follow such a program is to 'Maximize your Results.' Who cares? I resolve to run when I feel like it, get into pick up basketball games where I can, and learn to surf for the hell of it.

But most importantly I resolve to ignore those resolutions whenever I feel like it and do what I damn well please.

Is Google Making Us Stoopid?: A proof of concept?

I was in Starbucks today reading about the 1936 Iraqi coup d'etat when an older guy stopped on his way to a nearby table where his daughter was waiting and stared at, I thought, my feet. After a moment's pause, he continued. Then he stopped again and looked back. After a few more beats, he continued to the table. I was self-conciously checking my shoes for unnoticed dog crap or anti-american graffiti or something that would explain a member of the greatest generation's attention. I was drawing a blank and was heading towards an undignified smell-check when he started to say something to his daughter but was interupted by the barista calling him back for his drink. Uncharacterstically patient, I decided to wait for him to come back, hoping he would explain what was wrong with my feet.

When he did come back, he pointed back towards my table and then asked his daughter if she had seen it. As I shifted my feet underneath my chair to avoid further scrutiny, he told her that it was article about whether google was making us stupid. I had the Atlantic on my table and the cover article this month is Nicholas Carr's Is Google Making Us Stoopid? which is followed in smaller print by What the Internet is doing to our brains. It's a well-written and interesting article that voices the author's concerns about the effects of internet usage on the way that we think, particularly in terms of our ability to focus on topics for long periods of time and think deeply about ideas. Mr. Carr worries that the pre-dominance of quick one to two paragraph articles and blog entries have made us incapable of reading longer form work, preferring instead to scan a few paragraphs for information. He means not only that we are no longer able to read books like Gravity's Rainbow (which is a problem I can attest to) but are also incapable of reading longer form articles, ironic, I thought, to deliver such a message in an article weighing in at just over five pages. While I thought the argument was mostly alarmist and even Mr. Carr, himself, admits that he could just be a modern version of those who decried the printing press, I think there is some basis for his concern that is for me best illustrated by the state of cable news and the decline of the newspaper.

I had to wonder though, as the man after explaining the title of an article of which he probably had never even noticed the subtext, went on to argue that the article was baseless. "Does the encyclopedia make me stupid?", he countered.

easier networking with hamachi

Sorry for the crappy formatting. Blogger is doing too much for me and thereby preventing me from doing what I want easily, css-wise. So I've hacked styles into spans manually for now. I'll get this fixed later.

I'm a big fan of hamachi. It's an easy way to access all of your machines whether they have a static IP address or not and it's probably more secure than setting something up with a dynamic IP address and port forwarding.

Hamachi is a "zero-config, centrally managed VPN" that allows you to setup networks and join them with some simple client software. I won't go into the details of how to install and set it up. There's a pretty good description for Mac OS X and for linux. Once you've gotten your machines logged into the same network and given each a name with hamachi set-nick, you should be able to get a list of machines in your virtual network with their names like this:


[aykroyd@entoto ~]$ hamachi get-nicks
Retrieving peers' nicknames ..
[aykroyd@entoto ~]$ hamachi list
* [aykroyd-network]
* 5.71.134.95 electric-monk 71.200.230.164:33000
* 5.22.143.15 entoto 192.160.10.100:52000


You can now use the first IP address to get to your other machines. This is pretty cool, but the obvious next step is to want those nicknames to be in your /etc/hosts file so that you can refer to them by their name rather than the IP. To that end, I've put together a python script that I run in my system crontab every 15 minutes.


refresh-hosts.py
#!/usr/local/bin/python

import re
import subprocess

start_comment = "### start auto-generated hamachi hosts"
end_comment = "### end auto-generated hamachi hosts"

start_pattern = re.compile("^" + start_comment)
end_pattern = re.compile("^" + end_comment)

hostfile = "/etc/hosts"
username = "aykroyd"

def command_output(cmd):
" Capture a command's standard output. "
import subprocess
return subprocess.Popen(cmd.split(), stdout=subprocess.PIPE).communicate()[0]

def get_hamachi_hosts():
" Returns a list of (ip,name) tuples. "
output = command_output("sudo -H -u " + username + " hamachi list")
hosts = list()
for line in output.splitlines():
values = line.split()
if len(values) == 4:
hosts.append((values[1], values[2]))
return hosts

def main():
# make sure that hamachi is started and logged in
status = subprocess.call(("sudo -H -u " + username + " hamachi start").split())
if status != 0 and status != 255:
print "ERROR: problem starting hamachi status " + str(status)
return 1

status = subprocess.call(("sudo -H -u " + username + " hamachi login").split())
if status != 0:
print "ERROR: problem logging into hamachi"
return 1

# populate the nicknames so that we can create host entries
status = subprocess.call(("sudo -H -u " + username + " hamachi get-nicks").split())
if status != 0:
print "ERROR: problem getting nicknames from hamachi"
return 1

buffer = ""
in_hamachi_hosts = False

file = open(hostfile)
for line in file.readlines():
if not in_hamachi_hosts and start_pattern.match(line) != None:
in_hamachi_hosts = True
elif in_hamachi_hosts and end_pattern.match(line) != None:
in_hamachi_hosts = False
elif not in_hamachi_hosts:
buffer += line

file.close()

buffer += start_comment + '\n'

hosts = get_hamachi_hosts()
for host in hosts:
buffer += host[0] + '\t' + host[1] + '\n'

buffer += end_comment + '\n'

file = open(hostfile, 'w')
file.write(buffer)

return 0

if __name__ == "__main__":
import sys
sys.exit(main())


This setup has made life a lot easier for me. I can always get to my server at home and scripts to rsync my development environment now can refer to just a host name, instead of a parameterized IP that I figure out at the time that I run it.

Torture and Complicity

I read and was outraged by Jane Mayer's article in the New Yorker detailing the C.I.A.'s torture program at black sites. It details a scientifically executed program of torture by agency professionals who, called upon to "get answers" by the administration, dusted off information gleaned from behavioral experiments in the 1950s to produce a chillingly exact system for breaking down human subjects.

The C.I.A.’s interrogation program is remarkable for its mechanistic aura. “It’s one of the most sophisticated, refined programs of torture ever,” an outside expert familiar with the protocol said. “At every stage, there was a rigid attention to detail. Procedure was adhered to almost to the letter. There was top-down quality control, and such a set routine that you get to the point where you know what each detainee is going to say, because you’ve heard it before. It was almost automated. People were utterly dehumanized. People fell apart. It was the intentional and systematic infliction of great suffering masquerading as a legal process. It is just chilling.”

The article goes on to detail how suspects were kidnapped, taken to black sites, and then subjected to sensory deprivation, nudity, prolonged periods of loud unbearable noise, hypothermic conditions, waterboarding (which is simulated drowning), among other things. Not surprisingly, those carrying out this program were concerned by the legal ramifications of their actions and so everything inflicted on the prisoners was noted and approved by the chain of command. However, the use of forms, approvals, and other trappings of the banality of corporate America, do not create a morally justifiable program of interrogation. Those who would defend the tactics employed by agency officials would be hard pressed to justify a system whose effects over the course of months are so severe that suicide is frequently attepted by prisoners who, lacking any better means of committing the act, have resorted to trying to beat themselves to death by banging their head aganst concrete walls to the point of unconsciousness.

I knew all of this and I felt sick. I read Frank Rich's excellent column in the New York Times, The ‘Good Germans’ Among Us, in which he points out that we are all complicit in our nation's flagrant abuses of human rights and international laws because we have done nothing. And yet, although the story was nothing new to me, watching Rendition's sadly accurate story of an innocent man abducted by American officials and taken to an unnamed African country to be tortured for information based on a possible link to an Egyptian terrorist this weekend still had a profound effect on me. Maybe it's a credit to the medium that it was so effective, but watching the movie did what none of my news tracking had done before. It made me ask the question, "What am I doing about it?" For some reason, I had never before quite realized that right this moment innocent men, wrongfully and illegally abducted by Americans without the benefit of due process, are shivering in cold darkness without any hope of seeing a familiar face again. All of my self-congratulatory awareness of the problem doesn't do shit for the guy who's slowly losing his mind being asked questions he can't answer by people whose payroll is financed in part by my tax dollars. It had always felt just a bit abstract.

Supporters of what is currently being called enhanced interrogation techniques like to conjure up our war of civilizations against an enemy that is different than any we have ever faced. The defense, and indeed the general motivation behind the program, is basically that terrorists can strike at any moment and as they demonstrated on September 11th have the ability to kill thousands of Americans. Given this new threat, we can't afford to leave any stone unturned in our hunt for terrorists. The immediacy of the danger means that we have to be willing to get tough, be real, and get answers anyway possible. According to this logic, if that means throwing out the Geneva Conventions and Habeus Corpus then that's what we'll do to prevent another catastrophic attack. For me, however, this line of reasoning runs into murky waters for several reasons. First, not everyone who's being subjected to this treatment is guilty. Second, torture has time and time again been shown to be ineffective. More importantly, regardless of the first two points, is this who we are as a people? As a nation, we are outraged by the winning at all costs mentality that leads to blood doping and corporate stock scandals. How then can we not be similarly outraged by an intelligence program founded in the same mentality, which as a consequence shows an absolute disregard for human rights?

At the beginning of the War On Terror, in a moment bit of free market inspiration, we offered cash rewards for tips on members of terrorist groups. This led equally innovative entrepreneurs in countries such as Pakistan to povide tips to U.S. officials about their rivals and enemies. Anger one of these budding free market enthusiasts and you might well find yourself in jail for alleged ties to extremist groups as the NPR show "This American Life" revealed in Habeus Schmabeas in 2006. This episode relates the stories of men who were wrongfully imprisoned in Guantanamo. In stark contrast with the administration's claims that the only people there are "the bad guys" a significant number of Guantanomo detainees are not terrorists. A Seton Hall study cited in the show found that less than 5% of the detainees studied were even captured by Americans on the battlefield. Less than 8% were characterized as al Qaeda fighters. This is not to say that everyone abducted by soldiers in the War On Terrorism are innocent, but how are we to tell? Of course, Guantanamo isn't the only American prison in which people have been unlawfully detained... there are also the black sites that we weren't supposed to know about. So how are we to tell if those were populated mostly by actual terrorists? Amnesty International keeps a list of some people who have wrongfully detained and there have been other cases of people wrongfully imprisoned such as that of Khalid El-Masri. But the complete lack of transparency makes it difficult to make an accurate assessment. The only thing that is clear from these cases is that there are a lot of innocent people being tortured and held indefinitely without reason and no recourse in a court of law.

One actual terrorist who we do know was detained is Khalid Shaikh Mohammed, who was without question a "bad guy". Unfortunately, even in the cases where we have the right guy, torture proves problematic as a means of gathering useful information. Aside from the obvious moral issues in play, it doesn't work. Repeatedly, seasoned interrogators have asserted that torture is an ineffective means of gathering actionable intelligence (Google it). As Mayer's notes in the New Yorker, Khalid Shake Mohammed's confession became confused by what he confessed to,

Perhaps under duress, he claimed involvement in thirty-one criminal plots—an improbable number, even for a high-level terrorist. Critics say that Mohammed’s case illustrates the cost of the C.I.A.’s desire for swift intelligence. Colonel Dwight Sullivan, the top defense lawyer at the Pentagon’s Office of Military Commissions, which is expected eventually to try Mohammed for war crimes, called his serial confessions “a textbook example of why we shouldn’t allow coercive methods.”

And since we are not only interrogating actual terrorists but also innocent men, torture often nets false confessions and fake intelligence designed to please the interrogator and end the pain. So we get inaccurate information from actual terrorists in addition to false confessions from innocent men.

Still, even if we were only capturing guilty men and torture was an effective means of gathering intelligence, I have to echo the fundamental question that Michael Moore asks us in regards to Health Care in Sicko: Who are we? Are we so seduced by our own rhetoric that we are blind to how we look to the rest of the world. In a country that we like to think of as a beacon for freedom, justice, and liberty for all, why are we so quick to throw these virtues out? How can we ignore the fundamental principles upon which our democracy was founded? How, after this display, can we look other countries in the eye and scold them for human rights abuses? We may not be the Chinese government in Tianemen square rolling over a protester with a tank, but how far from a global secret police are we? When Thomas Jefferson said, "Our liberty cannot be guarded but by the freedom of the press, nor that be limited without danger of losing it.", it wasn't a defense of the press's ability to publish trashy gossip columns and make personal attacks on public figures, these were side-effects. Jefferson knew that the only defense against tyranny in a democracy was a press which through its reporting kept the activities of the government transparent to its citizens. And so when our government abducts men in secret and creates secret policies advocating the use of torture at undisclosed locations, and inquiries into the government's practices are prevented by claims of endangering national security, where does that leave our democracy? Is it acceptable because most of the people imprisoned are not American? I hope not.

And all of this brings me back to the question that the movie left me with. What am I doing about this? I wondered if the movie alone was bringing more visibility to torture. So I plotted a report on Google Trends that shows over time the volume of search queries on different search terms: Abu Graihb, torture, and rendition.



As it shows, there was a huge increase in google searches on torture when the Abu Graihb scandal broke, but now although searches on rendition have increased since the release of the movie, there hasn't been a corresponding increase in searches on torture. So, although it's early, the movie doesn't seem to have led to increased interest in the torture issue, at least not on google. To add perspective to the search volumes, I plotted these against another search term to give some context about the level of interest in the torture scandal in the U.S.



And still, where do I go from here? Writing a blog posting isn't going to help. I may write to my congressman, but I'm fairly positive that that won't help. The best answer that I've had so far is donating and/or getting involved with Amnesty Internation, which has setup a campaign specifically for human right's abuses in the War On Terror. Unfortunately, reading through the campaign's website doesn't give me a huge amount of hope that there are many ideas of how to tackle the problem, inspiring as a virtual flotilla is. But I guess getting 30,000 people to board a fake flotilla is a start, and in digitally boarding it each one of those people has done more than I have so far.

The Five Ages of a TSM Office Manager

I wrote this last year about TSM. It became relevant again as I heard through the TSM grapevine that a new office manager was once again doing something about that fridge situation

1. The first age dawns on day one of TSM employment. Fresh faced, filled with enthusiasm and vigor, the new office manager arrives at Black Falcon Ave. in clean and pressed professional attire and is carted around to meet all of the employees. Names are forgotten and awkward pleasantries exchanged—at least that’s how it works in the development office. Still, optimism rules the hour and coffee is promptly brewed and setup with pleasant presentation in the kitchen.

2. The second age comes after the office manager has been on the job for a few weeks. Flushed with the success of the coffee and getting to know co-workers, the office manager sets about to make a difference. Accordingly, the refrigerator is soon cleaned.

3. In the third age, the small details that delighted colleagues have become drab. Routine is the by-word of this age and frankly the routine is getting old. The pleasantly presented coffee pots of earlier ages are forgotten, and the brewing of coffee mostly ceases. Coffee drinkers are left to fend for themselves. They do. No one expects coffee to be made for them. By this point, the office manager is starting to realize his or her true purpose: executive assistant.

4. In the fourth age of the office manager, the last age’s foreshadowings have come to rule the day. Coffee and refrigerator be damned, the office manager is solely responsible to the C and Vs. Cream shortages are rampant with the occasional water shortage. Murmurs echo in the hallway as co-workers speculate about the difficulty of keeping the coffee system running smoothly and noticing that all of the water jugs are empty.

5. The fifth and final age begins as the office manager realizes that in spite his or her aptitude for assisting executives, lowly co-workers circumvent his attempts to be a buffer by speaking directly to the executives. They don’t seem to understand the value of a middleman. They also still seem miffed by the lack of cream. Appearance starts to suffer and cigarette intake increases. The age ends with a mishandled airline ticket, one too many over-important interruptions of the CEO, or a simple disinterested resignation leading to early termination. Life remains largely the same and former co-workers of the once-upon-a-time Office Manager look forward to the cycle renewing itself again. The fridge is starting to get a little grimy, after all.

Record macro and Repeat

I've been slowly updating my IntelliJ keybindings so that unbound keys are bound in ways that my hands expect from using emacs. It's been giving me a lot of "wow that makes me happy" moments from doing stupid things like splitting windows or closing the current editor without leaving the keyboard. But what I really need now is a command that will let me repeat the next command n times as in emacs C-x [n] [command] syntax. I'm probably missing something and am too lazy to research this just yet, but it seems likes it makes macros much less powerful if you can't record a macro that does something to one line and then have it executed an arbitrary number of times.

Zen, C++, IDE reloaded

So I think my initial thought when I started this blog was that within some months of getting started, emacs would have proved itself; I would have gathered helpful hints from around the internet, consolidated them, and turned my emacs work environment into a killer, "have fun developing" environment. Not only that but I would be able to sleep easy at night knowing that I had done it all in emacs. If I was more into being green and less into cars, it would be the same sort of moral superiority that some people derive from owning and operating hybrids... or not. I may just be making random stabs at the hybrid crowd for no good reason, vainly trying to mask my frustration with my own failure to achieve a sense of 'holier-than-thouness'.

Perfect... 1 paragraph and I've insulted a bunch of (oops almost used do-gooders there) people trying to make a difference and help the planet and invented a word... gratuitously.

So I think where I was going with was that there's a lot of functionality in emacs and a lot of people get a lot out of it. I was just hoping to provide a friendly place to pick up some tips on maximizing its use targeted to people with similar (IDE-driven) sensibilities to mine. But, it won't be that way. So, I'll just walk through where I've been from there to now.

First, emacs. Built it for Mac OS X. And started playing. For one thing, it totally kicks ass at one of my basic requirements for work which is to be able to seamlessly develop on a remote machine via ssh. The baseline version TRAMP that ships with emacs 22 allows you to easily load files from a remote machine via ssh just by opening a file with the following syntax: '/ssh:user@machine:/path/to/file.cpp'. What's really nice about TRAMP (given our shady connection to the data center) is that it saves the file in /tmp and all autosaves are performed locally. Only when you make emacs save does it actually go over the network. And if you crash, TRAMP provides a way to restore those backed up files. It also caches directory information enough to make tab complete pretty useable even over a slow connection. The packaged version of TRAMP's only weakness was in building, grepping, and debugging. To do these, you had to use remote versions of the commands that were a bit buggy. However, if you patch to the latest version of TRAMP, the standard commands will 'just work' via ssh. So that's that. TRAMP is totally impressive.

CVS integration is also very good in emacs. It's another feature that seems to just work and is generally pretty good. Diffs and history for files are fairly easy to get at via windowed menus and I don't feel the need to perform those things via the keyboard.

I was also happy with emacs' editing capabilities and general configurability. It's satisfying to envy a simple editing command available in a different editor and just implement it. That said, if the feature you happen to be drooling over is continuous syntax checking, function completion, or god forbid refactoring support, you're in for a ride.

The easiest thing to do for continuous syntax checking seems to be fly-make... a package that finds the Makefile for your current file and runs it with the target check-syntax. With gcc you can set this target up to be something along the lines of: g++ -fsyntax-only $(INCLUDES) $(CPP_FILES) and the module will run in the background highlighting errors. It's pretty useful and really easy to use, but it doesn't always work. Fairly often it would alert me that it was going to crap out... and then it would crap out. At some point, I even got it to take down all of emacs (which incidentally, it how I found out about TRAMP's awesome recovery features).

So that's not great, but then in terms of completion of functions, etc. There's just very little out there. The best thing is the probably CEDET which is collection of .el files to help make emacs into an IDE. Personally, I couldn't get it to even byte-compile the .el files on my mac and even running on the non-compiled version, it didn't do much for me. I've heard similar reviews from friends who have played with it. It may get there. It's an exciting idea. It's opensource, so if you're inclined you can work on it... but it's not there.

Crap... I just checked the length of this in preview and it's a monster. How do people write short blogs?

All of this led me to Eclipse C++. It, too, is opensource, so I figured I could work on problems that I found. And it already supports C++ and better yet I even found a remote plugin for it. But there were some problems. For one, yes it supports C++ but it seems to get tripped up on templates fairly easily and sometimes is just unhappy and won't offer completion hints. It quickly became my opinion that no hints were better than occasional hints. It's just too painful when it frequently doesn't work but you expect it to. I think that ends up being slower than planning to not get help from the IDE. Next, eclipse seems to have been designed with the keyboard forgotten. Everything is mouse centric from navigation to code inspection. Furthermore, I couldn't find any code generation features written (e.g., getters/setters or constructor). Next the code base is way too massive to think about doing anything quickly (as opposed to emacs). Finally, the remote stuff sucked after using TRAMP.

So what now? This is a two part answer, I guess. Answer 1: IntelliJ in textmode. And here's why. It has a lot of the great editing capabilities that I got with emacs including incremental search and macros (although the latter are not as convenient as they are in emacs). It has better file navigation -- I don't think I can live without the cmd-n interface... seriously. It doesn't have remote file editing/building (for C++ obviously) -- but I have handled both of those problems with shell scripts that make it all seamless. It has nicer CVS integration -- it's diffs are great. It has built in syntax highlighting for C++. And it's fast and looks nice.

Part 2 of the answer is that it also supports extension so I'm working on a C++ extension for it. This is a bit of large undertaking since it entails writing a full parser for C++, but at least I can do it in Java, in IntelliJ, be as ghetto about it as I want, and pretty much anything will be better than what I had before.

Why Continuous Syntax Checking Matters

Today, I ran into a great example of why continuous syntax checking matters. Now, I understand that some of the slowness that I experienced in diagnosing this problem is because I'm still getting used to C++ and Boost and the resulting error messages. but still...

Here's the relevant chunk of code with the error in bold and line 69 pointed out for future reference.


--> Line 69

boost::program_options::options_description config( "Config File Options ");
config.add_options()
( "currentTime", boost::program_options::value()->default_value(defaultDateString), "YYYY-MM-DD HH:MM:SS" )
( "df", boost::program_options::value(), "description" )
( "debugDir", boost::program_options::value(), "location for log file" )
( "workingDir", boost::program_options::value(), "all intermediate files will go here" )
( "script", boost::program_options::value(), "script directory (location of func)" )
( "DBName", boost::program_options::value<std::string()>(), "name of database" )
( "DBUserName", boost::program_options::value< std::string >(), "username for database" )
( "DBPassword", boost::program_options::value< std::string >(), "password for database" )
( "DBSchema", boost::program_options::value(), "schema to be used for database" )
( "ASchema", boost::program_options::value(), "schema to access A tables" )
( "PSchema", boost::program_options::value(), "schema to access P tables" )
( "opsEmail", boost::program_options::value(), "operations email address" )
( "researchEmail", boost::program_options::value(), "research email address" );



And I'll just add the error message afterwards, for your enjoyment. I won't belabor the point by actually writing anything afterwards. I think it will be pretty clear. If I had had some kind of auto-syntax checking active, then this would have been a lot easier to find. Probably would have saved 5 minutes of staring at the code and 15 minutes of entering a blog to bitch about it. That's a 20 point swing!

/usr/include/boost/program_options/detail/value_semantic.hpp: In member function `void boost::program_options::typed_value::notify(const boost::any&) const [with T = std::string ()(), charT = char]':
private/SVCExecutable.cpp:69: instantiated from here
/usr/include/boost/program_options/detail/value_semantic.hpp:28: error: function return type cannot be function
/usr/include/boost/program_options/detail/value_semantic.hpp:28: error: function return type cannot be function
/usr/include/boost/any.hpp:172: error: function return type cannot be function
private/SVCExecutable.cpp:69: instantiated from here
/usr/include/boost/program_options/detail/value_semantic.hpp:30: error: assignment of read-only location
/usr/include/boost/program_options/detail/value_semantic.hpp:30: error: cannot convert `std::string ()()' to `std::string ()()' in assignment
/usr/include/boost/program_options/detail/value_semantic.hpp: In function `void boost::program_options::validate(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&, T*, long int) [with T = std::string ()(), charT = char]':
/usr/include/boost/program_options/detail/value_semantic.hpp:149: instantiated from `void boost::program_options::typed_value::xparse(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&) const [with T = std::string ()(), charT = char]'
private/SVCExecutable.cpp:69: instantiated from here
/usr/include/boost/program_options/detail/value_semantic.hpp:81: error: function return type cannot be function
/usr/include/boost/program_options/detail/value_semantic.hpp:81: error: function return type cannot be function
/usr/include/boost/lexical_cast.hpp:185: error: function return type cannot be function
/usr/include/boost/any.hpp: At global scope:
/usr/include/boost/any.hpp: In instantiation of `boost::any::holder':
/usr/include/boost/any.hpp:159: instantiated from `ValueType* boost::any_cast(boost::any*) [with ValueType = std::string ()()]'
/usr/include/boost/any.hpp:167: instantiated from `const ValueType* boost::any_cast(const boost::any*) [with ValueType = std::string ()()]'
/usr/include/boost/program_options/detail/value_semantic.hpp:28: instantiated from `void boost::program_options::typed_value::notify(const boost::any&) const [with T = std::string ()(), charT = char]'
private/SVCExecutable.cpp:69: instantiated from here
/usr/include/boost/any.hpp:125: error: field `boost::any::holder::held' invalidly declared function type
/usr/include/boost/any.hpp: In function `ValueType* boost::any_cast(boost::any*) [with ValueType = std::string ()()]':
/usr/include/boost/any.hpp:167: instantiated from `const ValueType* boost::any_cast(const boost::any*) [with ValueType = std::string ()()]'
/usr/include/boost/program_options/detail/value_semantic.hpp:28: instantiated from `void boost::program_options::typed_value::notify(const boost::any&) const [with T = std::string ()(), charT = char]'
private/SVCExecutable.cpp:69: instantiated from here
/usr/include/boost/any.hpp:159: error: cannot convert `std::string (**)()' to `std::string (*)()' in return
/usr/include/boost/lexical_cast.hpp: In function `Target boost::lexical_cast(Source) [with Target = std::string ()(), Source = std::basic_string, std::allocator >]':
/usr/include/boost/program_options/detail/value_semantic.hpp:81: instantiated from `void boost::program_options::validate(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&, T*, long int) [with T = std::string ()(), charT = char]'
/usr/include/boost/program_options/detail/value_semantic.hpp:149: instantiated from `void boost::program_options::typed_value::xparse(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&) const [with T = std::string ()(), charT = char]'
private/SVCExecutable.cpp:69: instantiated from here
/usr/include/boost/lexical_cast.hpp:185: error: function return type cannot be function
/usr/include/boost/program_options/detail/value_semantic.hpp:81: instantiated from `void boost::program_options::validate(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&, T*, long int) [with T = std::string ()(), charT = char]'
/usr/include/boost/program_options/detail/value_semantic.hpp:149: instantiated from `void boost::program_options::typed_value::xparse(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&) const [with T = std::string ()(), charT = char]'
private/SVCExecutable.cpp:69: instantiated from here
/usr/include/boost/lexical_cast.hpp:191: error: invalid conversion from `std::string (*)()' to `int'
/usr/include/boost/lexical_cast.hpp:187: warning: address of local variable `result' returned
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits: At global scope:
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits: In instantiation of `std::numeric_limits':
/usr/include/boost/lexical_cast.hpp:131: instantiated from `boost::detail::lexical_stream::lexical_stream() [with Target = std::string ()(), Source = std::basic_string, std::allocator >]'
/usr/include/boost/lexical_cast.hpp:186: instantiated from `Target boost::lexical_cast(Source) [with Target = std::string ()(), Source = std::basic_string, std::allocator >]'
/usr/include/boost/program_options/detail/value_semantic.hpp:81: instantiated from `void boost::program_options::validate(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&, T*, long int) [with T = std::string ()(), charT = char]'
/usr/include/boost/program_options/detail/value_semantic.hpp:149: instantiated from `void boost::program_options::typed_value::xparse(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&) const [with T = std::string ()(), charT = char]'
private/SVCExecutable.cpp:69: instantiated from here
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits:290: error: function return type cannot be function
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits:292: error: function return type cannot be function
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits:295: error: function return type cannot be function
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits:297: error: function return type cannot be function
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits:299: error: function return type cannot be function
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits:301: error: function return type cannot be function
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits:304: error: function return type cannot be function
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/limits:308: error: function return type cannot be function
/usr/include/boost/lexical_cast.hpp: In member function `bool boost::detail::lexical_stream::operator>>(InputStreamable&) [with InputStreamable = std::string ()(), Target = std::string ()(), Source = std::basic_string, std::allocator >]':
/usr/include/boost/lexical_cast.hpp:189: instantiated from `Target boost::lexical_cast(Source) [with Target = std::string ()(), Source = std::basic_string, std::allocator >]'
/usr/include/boost/program_options/detail/value_semantic.hpp:81: instantiated from `void boost::program_options::validate(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&, T*, long int) [with T = std::string ()(), charT = char]'
/usr/include/boost/program_options/detail/value_semantic.hpp:149: instantiated from `void boost::program_options::typed_value::xparse(boost::any&, const std::vector, std::allocator<_T2> >, std::allocator, std::allocator<_T2> > > >&) const [with T = std::string ()(), charT = char]'
private/SVCExecutable.cpp:69: instantiated from here
/usr/include/boost/lexical_cast.hpp:149: error: ambiguous overload for 'operator>>' in '((boost::detail::lexical_stream, std::allocator > >*)this)->boost::detail::lexical_stream, std::allocator > >::stream >> output'
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/bits/istream.tcc:87: note: candidates are: std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::operator>>(std::basic_istream<_CharT, _Traits>&(*)(std::basic_istream<_CharT, _Traits>&)) [with _CharT = char, _Traits = std::char_traits]
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/bits/istream.tcc:93: note: std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::operator>>(std::basic_ios<_CharT, _Traits>&(*)(std::basic_ios<_CharT, _Traits>&)) [with _CharT = char, _Traits = std::char_traits]
/usr/lib/gcc/i386-redhat-linux/3.4.5/../../../../include/c++/3.4.5/bits/istream.tcc:102: note: std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::operator>>(std::ios_base&(*)(std::ios_base&)) [with _CharT = char, _Traits = std::char_traits]
make[1]: *** [debug.obj/SVCExecutable.o] Error 1
make[1]: Leaving directory `/data/aykroyd/cvs/Optimization/five0/PredictionAndBidding/SegmentVolumeCalculator'
make: *** [default] Error 2
make: Leaving directory `/data/aykroyd/cvs/Optimization/five0/PredictionAndBidding/SegmentVolumeCalculator'
exit
Connection to master closed.

exit

Compilation exited abnormally with code 2 at Thu Jul 26 10:53:43

emacs and the search for tool zen

NOTE: Originally, this was going to be one post with everything that I've figured out so far about how to use emacs, what tools to extend it with, and what custom extensions to hand-roll based on my current needs. Unfortunately, my tendency towards diarrhea of the mouth being what it is, I've filled more space than I feel comfortable spending on drivel about why I care about this, why tools are important, and why my individual needs are like a unique snow flake that everyone should care about. So, in the interests of optimizing the organization of the information a bit, I'm breaking it up. So this post will contain only introductory and motivation of problem-type content. Future posts will cover what I've actually done, ideas that I like, and hopefully provide some value at least based on the combination of ideas, if not the originality.

So I recently moved from Boston to Northern California. Very shortly thereafter, the startup that I had been at for nearly 2 years was acquired and I was on the job market. I've ended up taking a job with the engineering group in a smallish R&D team that works on optimization problems. The environment is completely different and in many ways I feel like I've stepped back about 20 years into the history of programming. We have to do most of our builds, debugging, etc on remote machines across a connection that is too slow for X forwarding, so most of the coding happens remotely as well. Since I've spent 5 years in Java-based IDEs -- first Eclipse and then IntelliJ, writing C++ ssh'd into a server is unfamiliar ground for me. So I decided to figure out what everyone else was using to do their development. And I was stunned to learn that my new colleagues used vim + grep...

and that was it...

vim and grep ...

To be frank, after having just finished my first warm-up coding assignment during which I was reintroduced to compiler errors that results from misspelled variables and calls to functions that don't exist: m_InitDatabase not m_initDatabase or obj->Run() not obj->Start(), I was hoping for a bit more than that.

So that's the fluffy background... I like tools that make it easier, faster, and more enjoyable to cut code. So I'm in search of the "right" development environment to make me productive with this new set of environmental constraints. I suspect it will be a somewhat lengthy process to configure my environment as well as I'd like and this blog will hopefully reflect that as well other topics that I find interesting.

So why bother?

I have the sense that people who use stock VIM and whatever else is available from bash wonder why I care. The power offered by the standard toolset available on any Unix system is amazing and I think it's important for developers to be comfortable using those tools. Spend any amount of time debugging production issues on servers and your knowledge of UNIX will be rewarded. But I've also been pampered by the amazing development tools offered by IntelliJ and developing without such tools feels needlessly painful. So I'll throw out a random laundry list of things that I love about IntelliJ (not that most of them are exclusive to it. But they are well executed there):

1. Auto Completion: First, you don't make spelling mistakes and the completion list is a crutch to jog your memory of just what you named a variable or method anyway. Second, it serves as an inline method of looking at simple documentation. Often you can avoid a trip to google to search for an API's documentation just by checking completion options on an object. Both of these uses save a lot of tme.
2. Fast Keyboard-based File Access: IntelliJ has this feature that I'm in love with. I think more than anything, this is what won me over to it coming from Eclipse. If you press Command-n, and start typing a class name "mysimpl" it will generate a list of all the classes in the project that match that search string. If there are none it will search in your library. Alone, that's pretty cool. But it also has a number of shortcuts. For example, if you type only capital letters, it will let any lower case letter be in between them (e.g., MSS matches MySimpleServer or ManualSarcasmService). This same tool can also be used to search files (excluding java files) and for symbols.
   
3. Code Generation: One of the things that I love about really expressive languages is how succinctly you can express yourself. But most of day to day use Java or C++ and with these languages there is a lot of boiler plate code: from class definitions to constructors, getters/setters, to equals and hashcode methods. After using IntelliJ for a while it stopped occurring to me that I should ever have to write these methods unless there was something out of the ordinary. Writing boiler plate code by hand is error-prone and a waste of time. What I like about IntelliJ's execution of this versus Visual Studio was that it's not wizard-based. I define a class member in the .java file and then position the cursor where I want the getter/setter to go, hit Ctrl-n, select getter/setter, and I'm done. No wizards, no mouse clicks, no graphic representations of my class... done.
   
4. Finding Uses of Methods/Variables: When debugging a problem, learning a new code base, or thinking about a refactor, it's incredibly useful to be able to jump from a method invocation to its implementation or find all of the uses of a method. Similarly, IntelliJ provides tools to see a graphic representation of an object's hierarchy. By contrast, determining where the method or class is implemented, opening that file, and then moving to the definition is just too slow.
   
5. Source Control Integration: I don't mind using svn or cvs command lines for some operations. But generally when I want to diff a file with a previous version or look at its history, I'm actually looking at the file in my editor. So it's handy (I might start to belabor the point... time saving) to perform that operation in the context of my editor.
6. Integrated Debugging: I'll admit it. I'm a wus. I don't like using command line debuggers. Everything about typing list to see where I am in the file and setting breakpoints by file and line number feels wrong to me. I understand that this is because I'm lacking character. And I'm not hardcore. But I'm not going to change. I like having the option of using a mouse when I debug.
   

So there's a few of the high impact features that I've come to expect from my development environment. Without these, I find coding and code related activities to be markedly slower and I find myself doing work that would be better completed by a computer. Computers are way more patient than I am.

In the posts that I'll be adding soon I'm going to talk about emacs22, tramp, and a number of extensions that I've been testing.