It is a debate, and I debate that I have internally as well...there will never be a perfect answer for this:
I never wore a helmet in Japan, though I had children strapped to my bicycle (also helmet-less), and often in a skirt, which would make riding more of a challenge. BUT! busy roads separate cyclists from cars. And regular community streets saw everyone mushed together, so it was a much slower pace.
In B.C., where we have mandatory helmet laws, it feels strange to not have a helmet on. I ride more aggressively than I did in Tokyo, taking the lane when it's my right and all that stuff. There have been a few times when I am rushing to the store and forget my helmet and then I feel all exposed and rebellious.
One of my guests decided to go for an extended solo ride one day without her group. She was on a lonely country road, with no other cars involved. I got a call from the hospital to go be with her. The last thing she remembers was doing a great descent down Rocky Point Road while eating her apple, and then chucking the apple core to the side of the road. Somehow she regained enough consciousness to crawl to the nearest farmhouse. 2 broken ribs, clavicle, cracked hip. She carried her broken helmet all the way to Massachusetts to show her non-helmet friends what saved her from serious head injury.
I don't think that a properly fitted, quality helmet looks dorky at all. Everything else does look really dorky.
I don't understand why skate boarders and snow boarders often don't wear helmets. I don't know how I figure skated for 14 years with a lot of hard falls and no helmet and never had a head injury.
The debate goes on.