Monday, December 14, 2009

Debugging Python Windows Services

Mark Hammond has a simple example of writing Windows Services in Python using the win32serviceutil.ServiceFramework class in his book Python Programming on Win32.

The problem is, that if you use a breakpoint in your service code and try to debug it, you'll find that it never stops at your breakpoint. The reason is that the win32serviceutil module will actually run PythonService.exe and have that program run your python script. Since PythonService.exe is its own process that has no knowledge of your IDE's breakpoints, it just runs without breaking.

Here is a simple trick for the small intersection of people who:
  1. Write Windows Services
  2. in Python
  3. using an IDE (I use pyDev)
  4. and need to step through their code to debug.

It turns out that win32serviceutil has a reimplementation of the logic in PythonService.exe that it uses to emulate PythonService.exe's debug mode when you convert your python script you an executable (e.g. via py2exe). The implementation is in win32serviceutil's DebugService() function. The trick to being able to set breakpoints and debug your python service is to convince win32serviceutil into calling DebugService rather than spawning PythonService.exe.

Luckily, this is trivially easy: just add the line
sys.frozen = 'windows_exe' # Fake py2exe so we can debug
before you call

Then, you just need to pass the '-debug' command-line argument when you run your service to force it into debugging mode. Your debugger should then control the process so you can debug it.

Monday, November 9, 2009

Government isn't all bad

The big news today is that the U.S. House of Representatives passed a health care reform bill. I'm not particularly impressed with the details of the bill passed nor am I particularly concerned with them...they will most certainly change when reconciling differences with whatever the Senate passes (assuming they pass something).

But, I'll use this is setup to post a link to an interesting counter to nay-sayers scared of government intervention in the health insurance industry.

Update 2009/12/15:
It seems the blog I linked to is no more so I'll go ahead and post the entirety of the message here...

This morning I was awoken by my alarm clock powered by electricity generated by the public power monopoly regulated by the U.S. Department of Energy.

I then took a shower in the clean water provided by a municipal water utility.

After that, I turned on the TV to one of the FCC-regulated channels to see what the National Weather Service of the National Oceanographic and Atmospheric Administration determined the weather was going to be like, using satellites designed, built, and launched by the National Aeronautics and Space Administration.

I watched this while eating my breakfast of U.S. Department of Agriculture-inspected food and taking the drugs which have been determined as safe by the U.S. Food and Drug Administration.

At the appropriate time, as regulated by the U.S. Congress and kept accurate by the National Institute of Standards and Technology and the U.S. Naval Observatory, I get into my National Highway Traffic Safety Administration-approved automobile and set out to work on the roads build by the local, state, and federal Departments of Transportation, possibly stopping to purchase additional fuel of a quality level determined by the Environmental Protection Agency, using legal tender issued by the Federal Reserve Bank.

On the way out the door I deposit any mail I have to be sent out via the U.S. Postal Service and drop the kids off at the public school.

After spending another day not being maimed or killed at work thanks to the workplace regulations imposed by the Department of Labor and the Occupational Safety and Health administration, enjoying another two meals which again do not kill me because of the USDA, I drive my NHTSA car back home on the DOT roads, to my house which has not burned down in my absence because of the state and local building codes and Fire Marshal's inspection, and which has not been plundered of all its valuables thanks to the local police department.

And then I log on to the internet -- which was developed by the Defense Advanced Research Projects Administration -- and post on and Fox News forums about how SOCIALISM in medicine is BAD because the government can't do anything right.

Sunday, October 18, 2009

Two Years of Tokyo

Well, we've now officially been in Tokyo two years now.
The end of this month will represent two years since I started work here. Originally, I was only scheduled to work here for two years with an option to renew for an additional year.

To be honest, these two years have probably been the two most difficult years of my adult life. I'm just glad my wife is here with me; she should probably be sainted for going along with this crazy idea.

Japan is a perfectly fine place to visit. The idiosyncrasies are cute, even fun, when you only have to deal with them for week or two. What visitor to Tokyo can't remember the first time they were packed into a train car? It's pretty memorable and gives you a funny story to tell when you get back home. Now trying doing it everyday for two years -- not so much fun anymore.

When we came, we had intended to travel and see more of the countryside on the weekends. But I've found I just don't have the energy. The daily grind just wears me down. So, with my contract coming to a close, we were looking forward to using the last few months before my wife's contract ended to finally make up for some lost time and see Japan.

Back in July I gave my 3-months notice that I didn't intend to renew my contract come this November. But my employer would have nothing to hear of that. Apparently, they need me to stay on through March. I find it flattering that they value my labor enough to want to keep me on for 4 more months, but it shattered our plans of finally getting out of this soul-sucking city and seeing more of Japan.

Probably the average American would have told them to shove it. We're familiar with a system based on at-will employment and autonomy. It would be unthinkable for an employer to tell you cannot quit. They may ask you to stay, even make offers to entice you to stay. But, at the end of the day, you are free to leave.

However, I am a guest in this country. Like a H1B Visa-holder in the U.S., I would have had to leave the country if I lost my employer sponsorship. Luckily, my wife is still employed, so I could have just changed my status to her dependent and stayed. The problem is that landlords in Japan require a guarantor before they will rent you an apartment -- and my employer has been kindly acting as our guarantor.

Which leads to two complications: a) if I left my job I would lose my guarantor; even if we stayed in the country, we would have to move into the housing provided by my wife's employer and b) I feel an obligation to my employer for having been my guarantor for these 2 (now 2.5) years. Add in the fact that I take pride in my work so, if my employer says they need me to stay for 4 more months, I am inclined to see it through. Result: I'll be working for 4 months longer than originally planned.

So we'll be here a bit longer.

I doubt we'll be seeing too much more of Japan, outside of Tokyo, then. Which is really a shame because, based on our limited experience, life in the country is completely different than life in Tokyo. Even Kyoto felt laid back. But Tokyo, Tokyo will never have a warm place in my heart.

There is an old Soul Coughing song that I never really liked called "The Incumbent". But with a little modification, the refrain has really started to speak to me:

Tokyo, Tokyo, I won't go back
Indelible reminder of a steel I lack
I gave you two years, what did you give me back?
A jaw-grind disposition to a panic attack

Anyway, it has been an experience. I'm glad I had the opportunity to live and work in a foreign country; perhaps aiming for the biggest city in the world was a little too high for me. The pressure of big-city life and the hassles of living in an unfamiliar land have just added up to a less-than-wonderful experience. Work has been fine, but I'm looking forward to getting home.

Just four more months.

Friday, October 16, 2009

Moral Majority

Here I was wishing that more of my fellow Americans would step up and agree that a reasonable baseline of health care should be guaranteed for all of our fellow citizens.

And then I learned about the case of Jamie Leigh Jones. Go ahead, click the link and read about what happened to this poor lady while she worked in Iraq. I'm not going to repeat it here because it makes me sick.

In response, Senator Al Franken offered an amendment to a defense appropriations bill that would ban federal funds to companies that forbid employees from suing for rape.

Not surprisingly, the amendment was added, but it is a sad day that such an amendment is even necessary.

If that was all there was to the story, it would merely be depressing. No, to make things truly sickening you would need to know that 30 Senators voted against the amendment. 30 Senators voted to prevent gang-rape victims from suing their coworkers.

If you would like to know your Senators' stance on defending rape, you can find a list of the 30 pro-rape Senators along with their contact information here. I suppose it shouldn't be surprising that that they are all men.

So now I feel pretty foolish. Here I was hoping my fellow Americans would be Good Samaritans and agree universal health care is a good thing, but apparently we can't even agree rape is wrong.

Tuesday, October 13, 2009

Who Would Jesus Insure?

I'm not a religious man, but I do believe in doing the right thing.
Any way you look at it, I cannot see how anyone could argue that offering health care to every man, woman, and child in the U.S. is morally wrong. So why would someone be opposed to such a proposal?

Two of the more common arguments are:
1) The government would run a public health care option incompetently, squandering money and providing sub-par health care coverage.
2) The government would run a public health care program so well that it would put commercial health care companies out of business.

Proponents suggest a third option:
3) The government program would compliment and co-exist with existing commercial offerings.

I'll contend that argument #1 is obviously false. The "public option" being proposed is intended to provide a baseline level of coverage. In other words, it is intended to cap the lower-bound of coverage options. It does not preclude anyone choosing a private health care option. It does not restrict the maximum level of coverage. As such, it can only raise the level of coverage for those currently most poorly served. One would be a fool to choose the public option if a better private option were available; if no better option is available, the public option cannot be inferior to something that does not exist.

Which only leaves the cost component of argument #1. I would like to see the government run the public option in an efficient way, but if it takes an inefficient organization to provide a baseline level of health care to people who have no better option, than that is simply the cost of doing the right thing. There is no moral high ground in putting money before the health of another human being.

As for argument #2, I cannot see the future so I have no way of guessing whether this is true or false. My gut instinct is that it is false. But, if the government were to be capable of running a program so efficiently that we all received better service at lower cost for all possible treatments, I fail to see the problem. Opposing a public option for fear that it will do a better job that corporate options is analogous to corporate welfare for the less-efficient companies. I do not see the moral high ground is putting the health of companies before the health of citizens.

So our possible outcomes are:
#2 is true - we all get better coverage, albeit from a government-run program.
#3 is true - the public option provides a baseline level of coverage but people/companies can still buy health insurance from private providers for a superior level of coverage.

A number of people seem afraid of a single-payer system, as would result if argument #2 were true. However, I emphasize that the only way for #2 to yield a single-payer system is if that single-payer were superior in the marketplace than all other options. If the government were outlawing the competition, that might be a legitimate concern, but there is currently no proposal to do any such thing. As such, gloomy talk of a single-payer system forcing us to endure inferior service to our current corporate plans is nonsense. The only single-payer system on the table is one so superior (and unlikely to happen) that complaining about it seems neurotic.

In summary, there is small but unlikely possibility that everyone in America gets such superior health care that corporate insurers cannot compete. The expected outcome is that every American is guaranteed a minimum level of health care and those of us fortunate enough to be able to afford better health care can still purchase it just as we do today.

Personally, I do not care what happens to my individual health care package. But I firmly believe that ensuring every man, woman, and child in America has access to at least basic health care is the right thing to do. And I am deeply ashamed that a small but vocal contingent of my fellow Americans so despises their fellow man that they want no part in it.

Monday, October 12, 2009

Sniffle Season

It's fall and the sound of snot is already filling my workplace.

Living in a city the size of Tokyo, you encounter a lot of people on a daily basis. And frankly, people are pretty dirty. It probably doesn't help that there is no custom to cover one's mouth when they cough or sneeze here. I've seen people pick their nose on the train and then immediately re-grab a pole or strap too. As a result, I make it policy to not touch anything.

I'm not ragging on Tokyo here. Frankly, I was appalled at how dirty San Francisco was when I first moved there. I later came to understand it was an "enlightened dirty", something you have to develop an appreciation for. Tokyo, being a far larger city than San Francisco, is correspondingly more enlightened.

So, with the H1N1 virus looming large in the headlines, my workplace has instituted policies including putting hand sanitizers at all of the entrances to "prevent carrying the virus in". We have also been instructed to stay at home if we are running a fever or otherwise suspect we may have contracted swine flu.

Now there is something that has always puzzled me: if you are feeling sick, why would you go into work to get all of your coworkers sick too? This was a complaint I've been harboring for years. Some fool thinks their work is so important that they come into the office and get 5 other people sick. If those five people have any sense, they'll stay at home to prevent further spreading illness, thereby reducing productivity 500% more than if the first person had just stayed home in the first place! But people don't think that way. They think just about their own deadlines or how to make themselves look better to their boss. It is the tragedy of the commons: in trying to sustain their own productivity, the productivity of the company as a whole -- or even the economy as a whole -- takes a hit. In their effort to wring half of their usual productivity in between bouts of coughing or sneezing, they risk spreading their funk to five, ten, a hundred others.

Please stop.

That said, the life of a Japanese salary man is pretty competitive. It is my experience that paid sick days are unheard of -- if you take a day off work, that comes out of your vacation time. Worse, if you take a day off, you "fall behind" your peers. You lose brownie points with your boss. When it comes times for promotion, it is Brownie Point Redemption Day and no one wants to be denied entrance to the pearly gates of management for another year of salary man purgatory.

Which explains why there are so many sick people on the train during the morning commute and why there are so many sick people in my office.

I had one coworker not come in today because H1N1 has been spreading around his daughter's elementary school and apparently made it back to his house. But in my giant open-plan office of 150+ people, there are at least ten more here and sniffling today. And fall is just starting.

I'm actually pretty concerned about coming down with H1N1 myself before the winter is over. It isn't that I'm so much worried about the illness itself -- like the SARS scare of a few years ago, it sounds like H1N1 is more bark than bite. What I'm concerned about is all of the bureaucracy that I would need to deal with. You see, my company has actually instituted a special policy just for H1N1 allowing us to stay home without consuming vacation days if we get a letter from the doctor proving we have H1N1. However, we can only come back to work once we have another letter from the doctor saying we're cured.

I never liked going to the doctor in the U.S., I really don't like doing it in a foreign country. Twice. For a disease they can't cure, no less.

That said, this seems like a nice safe policy to protect the company from H1N1-infected workers thinking they are so important that they need to come in anyway. They are telling us to stay home; even allowing us a special paid leave to do so.

I don't think it'll work, though, for all the same reasons workers always come in to spread their virus-infested cough spew. They should always stay home when the alternative is to risk getting everyone else sick too. But they need the brownie points. It is much easier to put on a mask and just pretend it is a cold than it is to deal with the doctor visits and company bureaucracy. And if they come in, they get the face time and bonus "perseverance" points that managers remember.

My forecast: a train-full of H1N1-carriers dutifully trudging into work, snorting and sniffling all the way. Shoot, come to think of it, even if they were clinic-bound, they'd be on the train sharing their germs.

Thursday, September 10, 2009


Last night I went out with some coworkers to have some monjayaki in Tsukijima. The Tsukijima business association apparently decided some time back to draw people to their town by making it the center for monjayaki restaurants in Tokyo. There are currently over 60 monjayaki restaurants within walking distance of the Tsukijima station. The one we went too was called Oshio.

We ended up ordering 5 different dishes:
  1. mentaiko-mochi monja - Fish eggs and chewy rice.

  2. mapo tofu monja - Spicy chinese-style beef and tofu.

  3. curry seafood monja - Indian dry curry with shrimp and shellfish.

  4. super-spicy monja - Not sure, but had a pretty strong kimchi flavor.

  5. "apache" monja - Butter, potato, and a big slab of fatty beef.
The picture at the top is the super-spicy monja.

This was the first time I had ever had monjayaki. The consistency is fairy liquid with chunks of whatever the main ingredients are. It solidifies just enough on the grill that you can scoop up with a little spatula-like metal utensil.

Yeah, it has the look and consistency of barf.

But surprisingly, it tasted pretty good. The super-spicy monja, was hands-down my favorite of the bunch. The mentaiko wasn't really to my liking, but at least it tasted better than it looked.

Tuesday, July 28, 2009

Politics as Usual

Everyone has their own perspective on the world and their own circumstances influencing their beliefs. What seems right and just to me can seem horrible and unfair to someone else. The founding fathers of the United States of America recognized this truth; in Federalist #10, James Madison wrote that the various competing regional, religious, and economic interests, factions, and parties would be the guarantor of American freedom and justice.

Everyone in America is entitled to their own opinion, representing their own interests. People can try to drum up support for their own ideas by sharing them with others and these ideas compete for mind share with the ideas of others among the American populace.

This is how politics in America works; it is like capitalism applied to the ephemeral world of ideas. And just like capitalism in the market, it should not be surprising that purveyors of ideas often have their own profit motives.

I suppose that statement is somewhat redundant because I believe, to most, the word "politics" is assumed to be the marketing of ideas with underlying profit motives. In which case, to be American is to be constantly embroiled in politics -- to be constantly inundated with ideas, sometimes of dubious motive.

Which is why it is imperative that we each form our own ideas, reflecting our own self-interests. I would dare say that each and every American has their own circumstances, their own obstacles, their own concerns, as well as their own hopes and dreams. And sometimes people's interests conflict.

So ask yourself: if you could change America in a way that would make your life better, what would you do? If you could improve absolutely anything, what would you do? How would you convince others to help you out? If people don't support your idea, what would you do?

Now ask yourself: what are others doing to try to influence you?

Monday, July 27, 2009

Subtle error in RFC 3665 "SIP Basic Call Flow Examples"

RFC 3665 is simply a collection of SIP call flow examples for reference to implementers. However, I noticed what appears to be a small error in section 3.2 "Session Establishment Through Two Proxies" with the digest values.

In step F2 "407 Proxy Authorization Required Proxy 1 -> Alice", the proxy responds with a Proxy-Authenticate header containing the nonce value "f84f1cec41e6cbe5aea9c8e88d359". However, in F4 "INVITE Alice -> Proxy 1", where the user agent should send back the same nonce value to the proxy server along with the calculated digest, it instead sends the nonce "wf84f1ceczx41ae6cbe5aea9c8e88d359". Clearly, this is a typographical error as the nonces are identical except for the leading "w".

A cursory scan of the rest of the examples in RFC 3665 did not turn up any more mismatched nonce values. The "wf84f1ceczx41ae6cbe5aea9c8e88d359" value appears in other call sequences involving proxy authentication while the "f84f1cec41e6cbe5aea9c8e88d359" value never appears again, implying the nonce value with the "w" character prefixed should be the "correct" value.

Interestingly, while the other nonce values used in examples appear to be 32-character hexadecimal representations of 128-bit values, the "wf84f1ceczx41ae6cbe5aea9c8e88d359" value does not fit this pattern as it is both 33-characters long and "w" is not a valid hexadecimal digit. Which leads me to believe that the "f84f1cec41e6cbe5aea9c8e88d359" value is actually the correct nonce value in the "407 Proxy Authorization Required Proxy 1 -> Alice" call flow and that the other instances where the "w"-prefixed value appears are just the same typographical error replicated via copy-and-paste.

What I don't know, and would be interested in finding out, is whether the calculated digest result value examples were calculated with the "f84f1cec41e6cbe5aea9c8e88d359" nonce or with the "w"-prefixed version. Unfortunately, the RFC does not tell us what password values they used to calculate the digests, so I cannot answer that question myself.

Calculating MD5 of binaries without debug symbols

If you compile a binary with gcc with debugging information enabled (-g), the MD5 of the resulting binary will change depending on the name of the directory you compile it in. Which means that if two developers compile the same source code with the same options on the same machine, only they do it in their own home directories, the MD5 of the resulting binaries may differ.

However, as soon as you strip the binaries, their MD5s will be the same.

Which leads me to this little tool I whipped up to compare two binaries without stripping them:

# Display the MD5 of a file, ignoring any debugging symbols in
# binaries.

# The strip(1)/objdump(1) commands for removing debugging
# symbols do not support writing to stdout so we need to
# allocate a temp file to write the stripped binary too.
tempfoo=`basename $0`
TMPFILE=`mktemp -q /tmp/${tempfoo}.XXXXXX`
if [ $? -ne 0 ]; then
echo "$0: Can't create temp file, exiting..."
exit 1

while [ "$1" != "" ]; do
# The following line is a cheezy way to accurately
# reproduce the same error messages as md5(1) when a
# specified file is unreadable.
md5 "$1" > /dev/null
if [ $? -eq 0 ]; then

# Try to strip symbols from the file on the
# assumption it is a binary and, if successful,
# compute the md5 of the stripped file. Note that
# objcopy is the same as the strip(1) command. If
# objcopy failed to parse the file (i.e. because it
# is not in ELF format), simply compute the md5 of
# the whole file since there are no debugging symbols
# to strip.
m=`(objcopy -g "$1" $TMPFILE >/dev/null 2>&1 && \
md5 -q $TMPFILE;) || md5 -q "$1"`

# Output the result in a md5(1)-compatible format.
echo "MD5($1) = $m"


Thursday, July 2, 2009

su Forces Terminal Echo to On

I just discovered an interesting quirk of the su command, at least on FreeBSD: it forces terminal echo to on and you cannot turn it back off. Interestingly, sudo does not have the same behavior. Here is a sample terminal session showing the behavior (input that was not echoed is shown in red):

[14:54:35] myhost:~ $ stty
speed 38400 baud;
lflags: echoe echok echoke echoctl pendin
iflags: -ixany -imaxbel
oflags: -oxtabs
cflags: cs8 -parenb
[14:54:38] myhost:~ $ stty -echo
[14:54:57] myhost:~ $ stty[enter] speed 38400 baud;
lflags: -echo echoe echok echoke echoctl pendin
iflags: -ixany -imaxbel
oflags: -oxtabs
cflags: cs8 -parenb
[14:55:00] myhost:~ $ sudo -s[enter] Password: mypassword[enter]
[14:55:15] myhost:~ # stty[enter] speed 38400 baud;
lflags: -echo echoe echok echoke echoctl pendin
iflags: -ixany -imaxbel
oflags: -oxtabs
cflags: cs8 -parenb
[14:55:20] myhost:~ # exit
[14:55:25] myhost:~ $ su[enter] Password: rootpassword[enter]
myhost# stty
speed 38400 baud;
lflags: echoe echok echoke echoctl pendin
iflags: -ixany -imaxbel
oflags: -oxtabs
cflags: cs8 -parenb
myhost# stty -echo
myhost# stty
speed 38400 baud;
lflags: echoe echok echoke echoctl pendin
iflags: -ixany -imaxbel
oflags: -oxtabs
cflags: cs8 -parenb

As you can see, the -echo flag was preserved by the sudo session, but not be the su session. Furthermore, it was not possible to re-disable terminal echo within the su session -- it simply ignored the directive.

I suspect the cause is somewhere in the PAM libraries that su uses to authenticate the user. However, I would have expected sudo to use the same PAM libraries, so I cannot explain the difference in behavior.

Sunday, June 28, 2009

Not-so-Cool Biz

It is still the rainy season here in Japan but the temperature is slowly creeping up as we head towards summer. Today's high is forecast to be 30 degrees (86 degrees Fahrenheit).

Which wouldn't be too much different than the weather this time of year in my home town in Virginia except for two things:
  1. In Virginia we commute by car and cars have air-conditioning; in Japan most people commute by walking or riding a bike to the train station and taking a crowded train.
  2. Cool Biz
Today I'm going to focus on #2. If you aren't familiar with "Cool Biz", there is a nice article over on Slate about this Japanese campaign to reign in energy usage by making everyone uncomfortable.

The gist of Cool Biz is to crank up the temperature in offices to a balmy 28 degrees Celcius (82.4 degrees Fahrenheit) and allow people to "dress down" to compensate. By "dress down", we're talking not having to wear a tie.

I've heard the figure 4 degrees bantered around as the difference not wearing a tie can make. But when I tried researching where that number came from, I cannot find any research measuring the relative apparent temperature with and without a tie. The number (which I presume is in degrees Celsius) seems to appear only in articles about Cool Biz and only as a number the Japanese government cites.

Not surprisingly, the large company I'm working at here in Tokyo is following the government's recommendation and has the thermostats set for 28 degrees. And I can tell you first-hand that the heat is incapacitating.

I was doing some research on how to deal with the heat and ran across some tips for living without air conditioning. For a wussy American such as myself who has been spoiled by comfortable working conditions for my whole life, 82.4 degrees might as well be no air conditioning. The tips:
  • Turn on fans to keep the air circulating.
We have 2 fans in a room with over 100 people in it.
  • Set ceiling fans to turn clockwise (counter-clockwise looking toward it).
Not really applicable in an office environment.
  • Keep curtains or blinds closed during the day.
Blinds are kept open all day.
  • Use light colored curtains to deflect heat.
The blinds are white. So if we were allowed to close them, it might help.
  • Keep windows and doors closed tightly to keep the house cool.
  • If there is a breeze or if outside air feels cooler than inside, open the window bringing in the cooler.
  • Keep sunny windows closed if there is no breeze.
  • In late afternoon, as soon as outside temperatures feel lower than inside temperatures, open all windows and doors.
  • In the evening, when it’s cool outdoors but still hot indoors, place fans in front of open windows to draw in the cool air.
I work in a fairly old office building that still has windows you can actually open. However, we are forbidden from opening them. Rumor has it that a couple years back someone snapped and try to jump out a window. Since then, we've been strictly forbidden from opening the windows. I suspect it was the heat that drove him to it.
  • Keep as many windows as possible open during night to take advantage of the cool night air to lower inside temperatures.
  • In the morning, close windows as soon as the outside air begins feeling warmer than inside air.
Besides not being feasible in an office environment, see the notes regarding the previous tips.
  • Limit strenuous physical activity until evening.
  • Drink lots of cold water.
And check.
  • When hot, use a water spritzer to spray yourself. The evaporating water will make you feel cool. (For fun, you can also spritz the water upwards and feel it fall down on you. The coldness can be quite shocking.)
I work in an open office floor plan with people sitting 2 feet from me in each direction and no walls. This might work if they are amicable to the idea too.
  • Tie a wet bandanna around your neck (this is something I do for my dog to prevent overheating and heatstroke during the summer).
I think we have a winner. I think I may have to try this one when the summer heat really sets in. Maybe I can start a fashion trend of white buttoned-up dress shirt, dress slacks, and a wet bandana. If I combining it with spritzing water, I can call it "wet biz".

Monday, June 8, 2009

Whats and Whys

I just realized something about myself today: when someone is talking to me, I'm not only listening to what they are saying, but also trying to deduce why they are saying it. Not consciously, of course, or else I would have realized that I'm doing it long before now. :)

In hindsight, I think this has been mostly a good thing. As a software engineer, when a request comes in for feature "X", rather than just understanding that the customer wants feature "X" and immediately starting to work on solving that problem, somewhere in the back of my brain I'm thinking about why the user asked or "X". What are they probably trying to accomplish? Is there perhaps a simpler way to achieve the same result that we can present as a counter-proposal? Is this something other customers might need too? If they are asking for feature "X" now, then they'll probably need feature "Y" at some point in the future too.

Which is to say that I think this has made me a better software engineer. However, sometimes it takes me a night or two of "sleeping on it" before I start being aware of the implications of the request. I've gotten into the habit of splitting my project to-do list into "things needed to do what the customer specifically asked for" and "things needed to do what the customer probably wants" categories. Obviously, the items in the list needed to satisfy the customer's explicit requirements always take priority. But I'll freely add items to the "customer will probably want" list as they occur to me, to be implemented in down time between projects or assigned as introductory tasks to junior engineers to get them familiar with the code base. From experience, I've found that this process results in more generic solutions to problems that make the product more resilient to changing requirements and adaptable to new customers.

On the flip side, I've gotten myself in trouble a few times by reading to much into my wife's words. I don't think I'm unique in that regards either. :)

Today I realized that I am sometimes perplexed by requests/statements from my Japanese co-workers. Of course there is the language barrier which means that it takes a lot more effort on my part to understand what is being said. But sometimes I know I comprehend what they are saying, but I still have an uneasy, discontent feeling. That is when I realized that my brain is subconsciously trying discern the "why" behind the statement and failing, causing the uncomfortable feeling.

As for why it is failing, I'm still unsure. But now that I think I know what is going on, I can work to address it. I suspect it may have nothing to do with the fact that the statements are in Japanese, but rather be a symptom of information asymmetry inherent in working in a large company. In which case, it would explain why I always seem to flourish in small company environments where information is more evenly distributed, hence making it easier to deduce the "why" behind new tasks.

Friday, May 29, 2009

A Score I'm Not Proud Of

I was doing a little research to write a post comparing incentives in the Japanese and U.S. workplace. In the process, I ran across the Zung Self-Rating Depression Scale. I know I'm not happy here in Japan, but I thought I would take the online test just to see how bad it was.

I got a 75 out of 100. That would be "severely depressed"...which is quite an eye opener for me.

The funny thing is that, compared to the people I see on my commute, I think I seem pretty well-adjusted. I mean, Japan has some of the highest incidence rates of suicide and alcoholism amongst industrial countries. I guess I can see why. At least I have something to look forward to: I can go home and get back to a normal life. But I feel for the poor Japanese salarymen I see slumped over on the train platform nursing a beer and staring blankly off into the distance...this is as good as it gets; there is nowhere else to go.

Anyway, I'll try to get back to writing my observations on incentive systems for a future post. For now, I'll leave with the same comment I give everyone who asks about Japan: it is a nice place to visit, but I wouldn't want to live here.

Sunday, May 10, 2009

See Me at Interop Tokyo 2009

In an effort to put a face behind its technology, NTT Communications will be showcasing portraits of some of their engineers at their booth at Interop Tokyo this year. And unless I had spinach in my teeth, there is a good chance my portrait will be there this year....even if I'm not. Six members of the team that runs the servers behind the HOTSPOT service, myself included, were gathered after lunch today to have our pictures taken. It wasn't anything fancy: first, they each of us pose for individual pictures, then did a few pictures of us in groups of three, and finally one big group photo of us at a conference table.

So if you find yourself at Interop Tokyo 2009 this upcoming June, swing by the NTT Communications booth. Above the booth you should see a number of portaits of engineers. I'll be the stupid-looking one.

Wednesday, March 11, 2009

Racism in Japan

I generally try to keep myself upbeat and try to maintain a positive outlook, but the racism in Japan really wears me down. The "帰れ(go home)", "死ね(die)", and host of other comments muttered in passing or said behind my back have really dampened my excitement about living in Japan. While less blatant, the rampant tendency for strangers to clear their throats or spit when they become aware of my presence suggests the racism runs pretty deep.

In hindsight, learning Japanese may have been a mistake; if you can't understand the language, you don't know when people are making rude comments.

Even as a white male, I do my best to not draw attention to myself. I've seen the looks of distrust and fear my fellow commuters give to foreigners of middle-eastern or African decent on the train. I can only imagine the death by the thousand cuts that everyday life in Japan must be for them.

My poor 5'2" wife isn't immune either. She's fed up with the way little old ladies stare at her and awkwardly avoid her like she was some sort of circus freak.

My coworkers often ask me why my wife and I don't go out and see more of Japan on the weekend. Frankly, at the end of a grueling work week, I don't have any urge to stand on the train some more. But more important, the last thing either of us wants is our weekend spoiled by bigots. Just going to the store can be emotionally draining. Trying to rent a room in the country can be outright impossible (70% x 37.8% = ~26.5% of hotels surveyed illegally refuse to serve foreigners; they cite "language barriers" -- apparently confusing nationality with language ability).

The almost-daily little hints, most of which seems trivially minor in isolation, add up to a roar of "you're not welcome here".

Before we came to Japan, we were familiar with the complaints many foreigners living in Japan have. The comment sections of many popular news sites aimed for expatiates living in Japan are filled with disillusionment. I always figured it was because the people hadn't taken the effort to learn the language, or had unreasonable expectations, or were being obnoxious in had to be something they were doing wrong. Maybe it was; maybe I've made the same mistakes.

Since coming to Japan, I have come to appreciate Dave Aldwinckle's complaints and the hard work he has been doing to try and bring the injustices in Japan into the forefront. Whenever I get worked up enough about something that I want to bitch about it on my own blog, I just need to go hit his to commiserate.

When I was sitting in the comfort of the U.S.A., Debito's stories seem farfetched and, frankly, unbelievable. More than once I thought he was making a mountain out of a molehill. However, I now realize that he doesn't have to go digging to find examples of racism, discrimination, injustice, and turns out there is just a lot of material to pull from here in Japan.

Unfortunately, while brave individuals like Debito are trying to recitify the situation, apologists still abound. Guides for foreigners coming to Japan are filled with such insightful advice as:
NOTE: At certain points through this and other articles we note that in some cases foreigners may be refused entry to particular hotels or rental of accommodation. This is not intended to imply any form of prejudice; it is merely a statement of the facts. Almost exclusively this is due to the very low number of foreigners in Japan, and general ignorance among the Japanese regarding foreigners. Smile, persevere, and try to be a good ambassador not just for your own country but for all non-Japanese in general.
(taken from

That's right, denying you access to shops just because you are a foreigner isn't any form for prejudice. Nope. People may be ignorant and racist, but if there is one they are not: they are not prejudiced. So just suck it up and deal.

The implicit message: "no one wants you here and if you don't like it, well go home."

Anyway, I don't want to give the wrong impression: I've met a lot of nice people in Japan (and Japanese living overseas). If it weren't for the great folks I have (and have had) the pleasure to work with, I probably wouldn't still be here. That said, I am looking forward to going home. Honestly, many people I have met seem to have no comprehension of how intolerant their countrymen are or, as the quote above demonstrates, don't even recognize actions as being discriminatory in the first place.

So until I can go home, I guess I'll have to continue to find solice on Debito's blog.

Thursday, March 5, 2009

Happy Nerdigras

I won't go into the origins since I didn't come up with them, but March 5th has been identified as the Square Root of Christmas and March 14th is Pi day. The discoverer of the Square Root of Christmas has expanded on his work to further dub the week between the two geeky dates as Nerdigras.

I guess this may become only an American holiday, as our nerd brethren in the rest of the world insist on silly dd/mm/yyyy or yyyy/mm/dd orderings for representing their dates.

In any event, in celebration, the commenters over on Ned Batchelder's blog are composing an official Nerdigras carol. Here, I reproduce in its entirety, the 012-days of Nerdigras (sung to the tune of the 12-days of Christmas):

On the oh-first day of Nerdigras, my imaginary girlfriend gave to me,
A hash map in a B-Tree.

On the oh-second day of Nerdigras, my imaginary girlfriend gave to me,
Two Turtle functions,
And a hash map in a B-Tree.

On the
oh-third day of Nerdigras, my imaginary girlfriend gave to me,
Three unmatched parens,
Two Turtle Functions,
And a hash map in a B-Tree.

On the
oh-fourth day of Nerdigras, my imaginary girlfriend gave to me,
Four calling stacks,
Three unmatched parens,
Two Turtle Functions,
And a hash map in a B-Tree.

On the oh-fifth day of Nerdigras, my imaginary girlfriend gave to me,
golden ratios!
Four calling stacks,
Three unmatched parens,
Two turtle functions,
And a hash map in a B-tree.

On the oh-sixth day of Nerdigras, my imaginary girlfriend gave to me,
Six trees sp-laying,
Five golden ratios!
Four calling stacks,
Three unmatched parens,
Two turtle functions,
And a hash map in a B-tree.

On the oh-seventh day of Nerdigras, my girlfriend gave to me,
Seven pointers dangling,
Six trees sp-laying,
Five golden ratios!
Four calling stacks,
Three unmatched parens,
Two turtle functions,
And a hash map in a B-tree.

On the oh-tenth day of Nerdigras, my imaginary girlfriend gave to me,
Eight bits a-zeroed,
Seven pointers dangling,
Six trees a-splaying,
Five golden ratios,
Four calling stacks,
Three unmatched parens,
Two turtle functions,
And a hash map in a B-tree.

On the oh-eleventh day of Nerdigras, my imaginary girlfriend gave to me,
A printout of Graham's Number
One corrupted byte
Seven pointers dangling
Six trees sp-laying
Five golden ratios!
Four calling stacks
Three unmatched parens
Two turtle functions
And a hash map in a B-tree.

On the oh-twelfth day of Nerdigras, my imaginary girlfriend gave to me,
[insert lyric here]
A printout of Graham's Number
One cornipted byke
Seven pointers dangling
Six trees sp-laying
Five golden ratios!
Four calling stacks
Three unmatched parens
Two turtle functions
And a hash map in a B-tree.

(Song by Jonathon Duerig, Robert K., et al)

Sunday, February 15, 2009

Procreation Propaganda

The current fashion in Japan is babies.

For the past 2 or 3 years, there has been a concerted effort on the part of the Japanese government to fight Japan's shrinking population by encouraging more women to have kids. At first, they tried the obligation and duty angle: news was that the Japanese people needed to produce more children to maintain Japan's status as the second wealthiest country in the world. Then, the government started providing more services to ease child rearing; some of these were long overdue, such as maternity care programs and day-care centers for working parents.

Let be reiterate that because the concept may be unbelievable to readers from modern nations: until just recently, day-care centers were few and far between in Japan; what few there are were very expensive and, until deregulation in 2000, run by the government. The concept of a babysitter is still foreign here. Of course, things are changing, but Japan has gotten an awfully late start.

I wish I had taken a picture of it, but I recall a poster on the train a few months back that sent the clear, if not implicit, message that if you have kids then people will be nice to you. In a country in which people rarely acknowledge your existence, that's a pretty bold statement. We're going to skip the courteous stage and jump right from the treating-you-as-an-obstacle stage to the being-kind-to-you stage...all you need is a kid.

There has been much internal discourse in Japan about the ramifications of a shrinking population. By and the large, the message is "have more children" because "shrinking is bad". I have not read the multitude of books on the subject, but the message sent by the in-train posters, celebrity magazines, and "talent"-riddled Japanese television shows is "kids are cool".

Don't get me wrong: kids are cool. What is uncomfortable to me is the idea of the government intervening in my personal life to support an official policy to counter the shrinking population.*

The pretext to all this commotion is that a shrinking population is bad. However, Professor Akiko Matsutani disagrees and in 2007 wrote a book titled "Shrinking Population Economics" in which he claims the shrinking isn't bad, but rather provides an opportunity for Japan to improve the quality of life for its smaller population. In his book, he argues that Japan should not focus on producing a new army of alcohol-numbed workaholics, but rather let workers enjoy more time with their family, take up hobbies, and unwind on holidays.

Which actually brings the argument around full circle. Perhaps the 12-hour+ work days, 3+ hour commutes, semi-regimented drinking parties, etc. with little or no time left to spend at home are as much to blame for the current "baby crisis" as anything. You can't be a family if you are never home. If the only time you see your spouse is for a few minutes as your drag your groggy butt back home after a long day at the office, is it really any surprise you don't have more kids.

The government solution: use propaganda to encourage people produce another generation of workaholics with no time or energy to have children themselves. Professor Matsutani's solution: make life enjoyable, kids or not. Life is to be enjoyed, not frittered away in some Orwellian dystopia. And the irony is, happier couples are more likely to have children.

* I use the phrase "my personal life" empathetically with the Japanese people. As a foreigner in Japan, the government couldn't care less whether I have kids or not.


As a timely addendum to my ramblings:
Canon has taken the initiative to send its workers home at 5:30pm (only twice a week, mind you). But it sends them home with homework: to make babies.
I wonder if they get performance reviews.

Wednesday, February 11, 2009

In Love with iTunes Movie Rentals

I've been meaning to try it for a while, but for the first time, on Tuesday night, we rented a movie from iTunes. Yet again, it was another impressively easy-to-use product from Apple.

Living in Japan, we often don't get movies for months after they come out in the states. And when they do finally come out here, they usually either dubbed into Japanese or have subtitles added. Frankly, dubbing sucks in more ways than I care to list. The subtitles aren't so bad, but they do detract from the movie a little. The most important selling point to us, though, is that we don't have to get dressed and walk 30+ minutes to the video rental store (and another 30+ minutes to return it when we're done).

So Tuesday night we basked in the warm glow of my wife's 24-inch iMac and watched WALL-E. Yes, not a new release, but I had only seen it on the airplane and my wife hadn't seen it all yet. It took all of 30 seconds to rent through the iTunes Store and the download began immediately.

On the downside, it took 2 hours to download the movie. While we could have started watching it immediately (while it continued to download in the background), since the movie was only about an hour and 20 minutes long, we didn't want to risk playback catching up to the download and ruining our movie-watching experience with "buffering" breaks. So, we watched a couple of episodes of The Daily Show online to let the download get a little over an hour head start.

With an hour left to finish downloading the movie, we started watching WALL-E full screen on the computer. The image quality was fabulous; there were no visible encoding artifacts at all. The sound was similarly superb. Even with the download continuing in the background, the movie played smooth with no hiccups. It was exactly what I would have expected from renting a title on physical HD-DVD or Blu-ray media.

Overall, the 1 hour lead time before we could start watching the movie was a little disappointing. But being able to sit comfortably at home and not hoof it down to the video store more than made up for that little setback. Next time, we'll probably either download the movie a day or two ahead of time or my wife will start the download when I leave the office so it is ready to play when I get home. Other than that, we were extremely pleased with the convenience, the quality, and the ease-of-purchase.

Apple nailed the user experience aspect again; there wasn't a single moment where I felt like we had to sacrifice something in order to gain convenience by purchasing online. The rental fee was commensurate: $4 to rent a recent release. I seem to recall Blockbuster charging $4-$5 to rent recent releases. So for roughly the same price as an old-school movie rental, I gained convenience but lost nothing. Even without our unusual circumstances, that looks like a decent deal to me.

So Apple iTunes' movie rental feature gets 4 thumbs up. As expatriates, we'll throw in 4 big-toes too, for a total of 8 digits up.

In case you are curious, there are services vaguely similar to NetFlix here, but they all require a) a Japanese credit card and b) postal delivery. I'll save my rant regarding trying to get a credit card in Japan for another day, but suffice to say we don't have one and aren't likely to get one during the remainder of our stay here.

Monday, February 2, 2009

Damn spammers

As much as I don't like it, spammers have forced me to disallow comments from anonymous posters.

Thursday, January 22, 2009

It's the End of the World as We Know It

And I Feel Fine.

Living in Japan for the past year, I largely missed out on the 24-hour coverage of the presidential election. In fact, even though I registered to vote absentee, I never did get my ballot. In any event, I never got that caught up in the presidential election. To be honest, though, while I was impressed with Obama's message of optimism after 8 years of fear, I didn't have a whole lot of faith he would follow through with his promise of change.

So while I was happy the Obama won the election, more for what it said about my fellow Americans than what it said about Obama, I was hesitant to believe in "change" until I saw it. In particular, I wanted to see Obama make his first act as president be to close the Guantanamo Bay prison. That, more than anything to me, would signal the Obama is serious about restoring American ideals.

Well, it wasn't his first act, but my eyes teared up this morning when I read the news: Obama orders Gitmo to close. 3 days into his presidency and he's committed to shutting down that black-eye of injustice and putting America back on the high road of morality.

In the words of the 2nd century philosopher Hierocles,
We ought always to deal justly, not only with those who are just to us, but likewise to those who endeavor to injure us; and this, for fear lest by rendering them evil for evil, we should fall into the same vice.
As an American, today is truly a proud day.

Monday, January 19, 2009

Open Plan Offices

The office that I work at, like many other offices in Japan, is what they call an "open plan office". If you search google images for "open plan office", you'll see page after page of images of hip-looking, airy spaces with lots of natural lighting and no walls. Japanese open plan offices are similar, but without the hipness, airy spaces, or natural lighting. Just long tables full of people sitting at computers under row-after-row of fluorescent lighting. My office consists of over 100 people sitting in 1 giant room. Cozy might be a good word to describe it.

And let me tell you, when the air conditioners shut down at 6pm, it gets mighty toasty -- even in the dead of winter.

I searched google images for some representative samples of Japanese open plan offices. You don't have to look hard, a simple search for "オフィス" ("office" in Japanese) yields almost nothing but open plan office pictures. Oddly, the pictures have tiny 1-foot tall dividers separating facing people for some semblance of personal space; I don't have that luxury at my office.

Anyway, having worked in this environment for over a year now, I took special interest in a recent study from Australia's Queensland University of Technology. In that study, they found that:

In 90 per cent of the research, the outcome of working in an open-plan office was seen as negative, with open-plan offices causing high levels of stress, conflict, high blood pressure, and a high staff turnover.


The high level of noise causes employees to lose concentration, leading to low productivity, there are privacy issues because everyone can see what you are doing on the computer or hear what you are saying on the phone, and there is a feeling of insecurity.

Now that is something I can relate to. I have found it impossible to concentrate on anything in this environment and my productivity has plummeted compared to the comfortable offices (or even cubes) I was used to in the U.S.

I can only hope that the real research and development institutions in Japan have the good sense to give their staff decent working conditions. Otherwise, I cannot fathom how Japan can hope to compete in knowledge industries.