Friday, 29 March 2013

In case you thought this was just about the rules...

Reading up on the wider response to Mackenzie Wilson’s Kickstarter is a really lonely feeling—it seems like the general consensus is that Susan Wilson is a terrible person, possibly a fraudster, ought to have just paid her daughter’s fees, and certainly shouldn’t have used/encouraged her daughter to use Kickstarter to fund it. The more I read, the more I begin to doubt my own point of view, but it’s not out of any sense of actually having been incorrect, but simply that the voices who oppose my viewpoint are incredibly overwhelming.

But then I catch sight of one of the worst of the replies, and I remember just what my viewpoint was.

I’m not passing judgment on the legitimacy of the Kickstarter. I’ve already pointed out that at least one of the arguments against the project itself is clearly unfounded. The fundraising part of the project isn’t even over; if you want to call foul about the project just being about tuition, you really have to wait until it’s clear the finished product—the video game—will never be delivered.

But the active hate and misogyny that’s being unleashed against the Wilsons is, frankly, sickening. Let’s say, just for the sake of argument, that the Kickstarter is an outright scam. The camp doesn’t exist (or Mackenzie isn’t going), and no one will ever see the game or any of the associated swag. Even if that worst-case scenario were, true, it doesn’t change the fact that I want one of those “I code like a girl” mousepads for the office. It doesn’t change the fact that the project is inspiring me to be even more vocal in my support for women in tech.

And it sure doesn’t justify some of these comments. I’ll grant that what’s visible on the Kickstarter thread, as well as the more carefully moderated news articles, is fairly civil. In fact, it’s mostly an argument between various commentors at this point. There’s a great deal of condescension and ad hominem attacks toward Susan, don’t get me wrong, but the actual insults… well, those appear in spades on Reddit. Huffington Post mentions in their coverage that there have been death threats. I grabbed a fairly representative sample. I started out on Kickstarter, then went to Huffington Post, Destructoid, and ended it out on Reddit. Mercifully, I never found the death threats.

What is heartening, however, is this deconstruction of all the arguments against the project, which follows 100% with my line of thinking:

I think some people need a few points clarified.
Point one: This project is not a violation of Kickstarter rules, if you actually read them. According to the KS guidelines:
“A project has a clear goal, like making an album, a book, or a work of art. A project will eventually be completed, and something will be produced by it. A project is not open-ended. Starting a business, for example, does not qualify as a project.”
Mackenzie has proposed a goal of creating a game, and has worked out the project budget based on the training and equipment that will help her do so. Not just for the sake of the training. So clearly, this is not a “fund my life” item.
Point two: the fact that Kenzie is under 18 does not make the project ineligible. According to the KS eligibility guidelines:
“Parents and teachers can launch projects in collaboration with children under 18 only if the adult registers for the Kickstarter and payments accounts and is in charge of running the project itself.”
So, not breaking the rules there either.
Point three: anyone who has been complaining about Susan & Kenzie not ending the project early once the goal was met clearly didn't bother to check this part of the rules:
“If a project reaches its funding goal before time expires, projects continue to accept pledges until the funding deadline. There is no option to end a project early.”
Cancel, yes. End early, no. And really, why should they cancel the whole project just because a few noisy people tell them to? Especially since those same people have already made it clear that they only pledged to be able to post rude, disrespectful and inconsiderate comments, and do not plan to honor their pledges. Perhaps it's worth reviewing what a pledge entails?
Point four: According to the KS Backer Questions section:
“By pledging, you are committing to supporting that person’s project; canceling that commitment is discouraged. If you must cancel, visit the project page and click “Manage Your Pledge.” At the bottom of the next page you’ll see the “Cancel Pledge” button.”
Enough said on that.
Point five: If you are truly still worried about whether this project violates Kickstarter rules, even though they state that projects are reviewed at the start to be sure they meet guidelines, here is what KS says to do:
“If you see a project that you believe violates the Project Guidelines or Terms of Use, the best way to let us know is with the "Report this project" button at the bottom of the project page.”
Funny, I don't see anything in there about faking a pledge in order to post endless comments nagging at the project creator to cancel it, and nagging at backers to cancel pledges.
Point six: because clearly it bears repeating - right below where you need to type in your comment:
“Be respectful and considerate.”
Does anyone need definitions of those terms as well?
And in case anyone is actually wondering:
I am not Susan.
I don't know Susan. Or Mackenzie. But she sounds like a fun kid to know.
I don't plan to cancel my pledge. Because it's a pledge. As in, a promise to support Mackenzie's goal. Not because I desperately wanted to buy a new RPG (though I look forward to seeing what she creates), but because after reading through the project goals & information, and looking at the website for the camp, I felt it was a project worth supporting. I still do.
And if I'm going to be suspicious of anything, it is links to websites that have lots of nasty, hateful information and no actual references except to OTHER websites with nasty, hateful information.
So, please - if you're actually concerned about the project, why not follow the actual Kickstarter policy and contact them directly?
Let's all play nice, shall we?
– Angela Reese

Trigger warning: open misogyny, anti-feminist sentiment, bog-standard MRA bullshit, idle vague threats, comparisons to child prostitution, violent fantasies


  • “This KS was set up by Susan Wilson to cash-in big time USING the dream of her daughter, and by turning it a pseudo-feminist sob story that resonates with naive whiteknights. People fund this KS not because they want the product, they think by throwing money at this KS they fight sexism (when it is actually detrimental to the feminist cause in this case” –Henrik
  • Kenzie is 9 years old and it's against the TOS for someone so young to use this website. The mom is the one in charge of this project and is outing her sons behavior online. To strangers mind you. What kind of parenting is that? She's shaming her own children online and worst of all, profiting off of it!
    The mom is being attacked for multitudes of reasons, all of which are just. Anyone who defends her/supports her behavior is just as guilty as she is for being a scammer/horrible parent.
    Go back to your basement, adults are talking (for the record, white knights never get laid).” –Quellcrist Falconer
  • “I fully expect Susan to go full Anita Sarkeesian now by e-mailing media outlets telling them that she had to take down the Youtube video because of mean comments towards her daughter. That'll skyrocket the amount of money coming to her because internet feminists eat that sort of thing up. It's sad that this is the thing that finally gets me to sign up for Kickstarter which is a great tool for the gaming industry but is getting bogged down by blatantly obvious scams like this. I've been following along at neogaf and a lot of this stuff is just appalling.” –Justin
  • “This was not about it being a worthy cause it was about playing on misandrist hatred of men to make a buck. The bar was set low so that she could pocket all the left over money. She was most likely thinking to clear 100k free with her heinous selling pitch of making her sons demons to get the radical feminist to push her project. As much I like the idea of her scamming non thinking femanazi drone bots its still wrong and should be called out. Let alone the way she is publicly shaming her children over a common children's fight, and for profit at that. This kick-starter is disgusting on so many levels it should make you wretch that this women thought to even try it.
    NO her original paln was to give hats and shirts and mouspads to people. The charity stuff only came when media started to ask questions. She never planned that, it was cause she was getting negative press all the sudden.
    The problem is everything with this project. From how she sold it, to even doing it, to using her children in a disgusting abominable way, without care for the fact she was making her boys out to be monsters with such a charged situation.
    That you cant see that these were wrong speaks volumes about you as a person.” – Brad Donald Lannon
  • “This family is a riot.
    Here's an indiegogo (another kickstarter-like organization) fundraiser from Susan's husband to get his kids off of video games: http://www.indiegogo.com/projects/help-me-get-these-kids-off-video-games-hunting-for-real-treasure?c=home
    Isn't that precious? One fundraiser to get off games, one fundraiser to make them.
    The mother is successful (glorified debt collector) but exploits her children, throws her boys under the bus (PR stunt) under the guise of gender issues. Her daughter has a facebook (violates facebook terms for not being at least 13, but then again this family is no stranger to violating terms) and plays Mature rated games with strong violence sexual themes (great parenting!).
    This family scams. And this article doesn't even scratch the surface of how shady they are.” – Sof Akins
  • “Susan wins. Internet can't win this fight.
    This woman is not an amateur. She knew how to manipulate the system and get all the media attention she needs.
    This is not about the money, it never was. This is about a woman getting tons of media attention and creating a platform to market from in the future. Look at her stretch goal Keep It Up gear. She's playing this for the long haul.
    She's good at it too, she's smart. Who wants to argue against a 9 year old girl? Even KS doesn't want to touch that. Now she gets to look like a hero for women everywhere and a heroic mother.
    She won. I'm sorry internet, we lost this fight.” – frankderr
  • “Wait what? So because you don't know how to raise your children or even control them after you realize you fucked up bad during their adolescence you decide to do this? Judging by every word you have said about your sons it sounds like they should not live with you because you nor your daughter can live with them.” – prettycoolstory
  • “Sadly, if Kickstarter yanks the "project" now, every crazed feminist on the internet will be beating down Kickstarter's doors with torches and pitchforks.” – Ryanne Cross
  • “More to the point, the industry is not sexist, it never has been, the jobs are there for those who want them, men are just sometimes to willing to sacrifice damn everything to achieve it, hell the owner of Obsidian says his resume ratio is 10 men for every 1 women.
    Honestly this whole crap makes it sound like women are being held down in some insane way when women have never had more rights, more than that and like you said, some of the most important positions and jobs in the world are held by women. This whole women in games, women in tech shit is getting really out of hand, its just a tool now for websites desperate for page hits, ironic that much like feminism the message has become distorted as well.” – Kyousuke Nanbu
  • “It's not the point, but she is super duper ugly.” – Phil Hasenkamp, Iowa State U
  • “I wondered who the fark Susan Wilson is, so I clicked to CNN's profile. “Her 10-year-old company finds out where debtors bank, work and own property--and gets 20% of its targets to pay up.” A farking collection agent is one of the most powerful women entrepreneurs. I don't know whether to be afraid of comforted.” – BarkingUnicorn
  • “I'm going to do a kickstarter to fund a marketing campaign to ruin this woman's life. TV ads in her home down, internet ads, etc. At $500 you can help design ads with me.” – m00
  • “I can see her pimping our her daughter to the highest bidder, if the price was right.” – WhippingBoy
  • “saturn badger: And she is a coont.
    I just want to choke this biatch.....” – shortymac
  • “That whore should have never been allowed to squat kinkos.com to make millions.” – FeminismSucks
  • “We have Anita Sarkeesian to thank for these copycats.She demonstrated that there's money in crowdfunding misandry. Now every unemployed feminazi cunt is jumping on the bandwagon.” – stupefyingly
  • “This is sickening. I have never wanted to punch a woman in the face so badly. Not that I would, this lady is just delusional and sexist as hell” – Nimrod41544
  • “This is really an embarrassment to women.
    That gender is the reason why the request received such an enthusiastic response suggests that feminism has succeeded in convincing society that women and girls just don't have what it takes to overcome obstacles facing us on our own... and that a woman who had the mental resources to reach millionaire status felt entitled to use that system instead of funding her own effort to train her daughter in the area of project creation and presentation shows that even some highly successful women aren't completely divorced from the entitlement mentality that abuses the good will of others to fund their lives... taking it even when they don't need the money. How can we expect to be perceived as independent and self-sufficient when people can point to incidents like this?” – oneirosgrip

Like I said, this has been a really lonely feeling, because in order to try to properly understand what’s going on, I’ve had to wade through a whole lot of crap, and the sheer volume of it has left me doubting myself in a few places. I’m glad there are people like Angela who are fighting the good fight.

Tuesday, 26 March 2013

What's wrong with youth interest?

I wanted to write about ducking exceptions and proper coupling tonight. I really did, but I spent all afternoon (while trying to debug an issue with TeamCity) reading about the Internet Hate Parade descending on a nine-year-old’s Kickstarter.

I’m still reading analysis of the shitshow that was Adria Richards’ firing, and then my Twitter firehose starts getting flooded by reports of further asshattery inflicted upon a nine-year-old girl who wants to be a video game developer and her mother. I try not to read the comments when it’s pretty predictable where the threads will go, but I had to read them tonight. I had to find out exactly what was being said; what was being glossed over by the larger media outlets—and I found that it’s unprintable.

So here’s Mackenzie Wilson, a young girl who’s really into role-playing games, Magic: The Gathering, video games, and wants to learn more about how make them herself. Her brothers, like so many other men, have been shitting all over her abilities and intelligence (though, just to play Devil’s Advocate, I recall how much my own older brother did this to me simply because he was older than me). The Kickstarter is asking for enough money to cover her fees at a game-making camp put on by Towson University, and promises a copy of the game to anyone who kicks in more than $10. The original aim was just the $829 single-week fee (which, I might add, doesn’t include the $519 overnight fee), and as of the time I’m writing this, has already come up with $21 910, since the evening of 20 March. Six days, and she can now afford to go to camp all summer, and then some.

The Internet Hate Brigade has descended on this kid and her mother. Between calls to report the project to Kickstarter (which are going unheeded; her mother has said in a couple of updates that the project has Kickstarter’s full support), the following points are being trotted out to fuel the fire:

  • Kickstarter doesn’t permit “pay my tuition” projects.
  • Her mother is, allegedly, a multimillionaire serial entrepreneur who chills out with Warren Buffett and periodically buys $1500 pairs of shoes, and shouldn’t be asking for money to pay for her kid’s camping trip
  • Her mother has been spamming Twitter with retweet requests (this is actually verifiable)
  • There’s no explanation of what will happen with the overflow.

Of those four points, only the Twitter one is actually, specifically, against Kickstarter’s Terms of Service. The first is kind of academic, I feel—the project is clearly titled, “9 Year Old Building an RPG to Prove Her Brothers Wrong!” When you create a Kickstarter project with a given target, you’re estimating what your costs are. In this case, she knew exactly what her costs would be, and clearly declared what they were. Is it going to tuition? Yup. But if you created a Kickstarter for a project, and knew you’d need to learn some more to pull it off, surely you could use the funding to pay for textbooks, as long as there’s a finished product at the end of it… which there will be in this case.

The second point is a series of allegations that have either all been disproven (Warren Buffett photo was a photo op at a meet-and-greet thing, and the shoes were purchased with roulette winnings) or don’t really hold water against reasoned consideration—being a serial entrepreneur doesn’t mean you have a huge bank account; just that you enjoy the thrill of starting something from scratch and are comfortable taking the inherent risks. As to whether or not she should be helping her daughter create a Kickstarter project… what would her daughter learn if everything she asked for, she got? Instead, her mother is teaching her 21st-century entrepreneurship—come up with a cool idea, go to where microinvestors hang out, talk it up, and see what sticks! This is fantastic!

As to the eventual fate of the overflow money, I have two things to say: first of all, there are a collection of material items you can get for different contribution values, and these material goods have costs. So some of the money will go to this overhead. Second, when you give money to any other Kickstarter that has a given target, and it blows past the original target, do you ask for your money back? Hell no, you understand that it will go to the project runners, to be used for the project. Now this girl has the option to not only learn more about video game design, but also, maybe, pay for professional artists and musicians. Who knows? Maybe it will seed her college fund.

But none of this has been considered by the boys’ club that can’t stand that people want in. Everybody seems to be intent on shutting this girl’s dream down, to what end? I’m sure the most likely justification would be something about “honesty” and “fairness”, but Goddamn it, she has been honest, she’s really not breaking the rules, and the way that the software community treats women is anything but fair. Here’s a girl who’s really keen on a lot of traditionally geeky things, and even though $22K is a pretty cool incentive, the words that have been sent her way are by far the cruelest she’ll have to face until she gets interested in sex (at which point, of course, the full force of our misogynist culture will come to bear on her). If you want to encourage kids to get interested in technology, this isn’t the way to do it.

There’s a cynical part of me that wonders what the response would have been if, all else being equal, it had been her father’s Kickstarter account instead of her mother’s, and if Mackenzie had been a boy. I wonder if the project would have faced nearly the same kind of invective and abuse. While I’m not a psychic, somehow I doubt if there were more boys involved, the Internet wouldn’t be shitting nearly so hard on the project.

Thursday, 21 March 2013

The bullied have become the bullies

I’m willing to give SendGrid a pass in this fiasco. A small pass, only, but Daily Dot is reporting that their hand was forced. Beyond the DDOS attack that they, and Richards’ blog, were suffering, Anonymous is/was threatening the utter destruction of the company if she wasn’t removed, and publicly.

I knew there was something strange about publicly firing her, if even just to try to pacify the trolls.

The threat from the Internet’s favourite band of vandals and vigilantes includes the following details of the plan:

“You [sic] client list has … been obtained by Anonymous. They have already begun harassing your customers. These include obnoxious phone calls, emails, denial of service attacks, online vandalism and defamation, and even real-life harassment.”
“Your financial backers have also been targeted for the same harassment. …If any of your backers have something embarrassing or illegal to hide (sexual misconduct, tax fraud, etc), Anonymous WILL find it (they are good at doing this) and make it public.”
“Real life harassment is an escalation that comes into play based on how long this situation is allowed to play out. It is not affected by the effectiveness of the previous forms of harassment. Even if your customers and financial backers are dropping like flies (or the opposite, entirely unaffected), this will still happen if Anonymous still maintains an interest in this situation. …If some of the more talented members of Anonymous take an interest into [doxing], every employee of Sendgrid becomes a target, starting at the top. For your reference, this is already happening to Ms. Richards as per standard protocol. There are also some interesting information about her dentist that was dug up in the process.”
“However, you do have a choice to make at this point: Do nothing, or publicly announce that Ms. Richards will be fired. The opportunity to stop this growing mob in its tracks before it tries to tear Sendgrid apart is as simple as publicly announcing Ms. Richards' firing. Now, you also have the opportunity to be sneaky about it and just publicly announcing the firing but not actually do it. But if Anonymous ever finds out, they will bring the full fury on you and your company. To put it in perspective, not even secure government websites are safe. If you believe you can tough it out, by all means, do nothing.”

I think it’s safe to say that Sendgrid didn’t have a choice in the matter.

They did, however, have a choice in what words to use when they announced Richards’ dismissal, which means that despite the reality that they had to sever their relationship with her, writing the following in their public blog continues to send much of the same message that I decried earlier:

“A SendGrid developer evangelist’s responsibility is to build and strengthen our Developer Community across the globe. In light of the events over the last 48+ hours, it has become obvious that her actions have strongly divided the same community she was supposed to unite. As a result, she can no longer be effective in her role at SendGrid.”

I don’t believe for a minute that Adria Richards divided the developer community. The developer community is already divided, because large numbers of men think nothing of making sexually suggestive jokes in a crowded conference hall. Large numbers of men think that a person who’s been offended by sexually suggestive commentary at a professional event should discuss the issue quietly, and not make a fuss. Large numbers of men believe that Human Resources department policies regarding indirect sexual harassment in a professional setting are stranglings of their free speech.

The software developer community is rife with sexist comments, and sexist thinking. Richards didn’t create a divide, her actions exposed it. The CEO of SendGrid “supports the right to report inappropriate behavior, whenever and wherever it occurs”… but clearly, only as long as it’s kept quiet.

To say that the Internet has exploded with hatred really doesn’t quite do the reality justice.

Let’s review:

  1. A woman—a woman of colour, at that—embarrasses a man for making sexually suggestive comments in a professional setting.
  2. The man is chastised, and apologises, presumably in earnest.
  3. His employer fires him for making these comments.
  4. The public isn’t told specifically why (perhaps this wasn’t his first offense, perhaps it was, we’ll never know, and his employer isn’t accountable to any of us), but he publicly accuses his accuser of getting him fired.
  5. Men across the Internet rally around his cause, and begin attacking her for daring to publicly call out inappropriate comments made in public. Every terrible word you can think of to describe women, and women of colour, is used.
  6. /b/ hears about it, and begins a campaign of harassment, demanding that she be fired in retribution for, ultimately, an HR policy violation firing.
  7. Vigilantes get in on the campaign, and attempt to destroy the woman’s employer, and everyone associated with them, unless she is fired.

This is not the behaviour of an inclusive, welcoming group.

This is the behaviour of a group of misogynists, who are frightened by the thought that the power relationship that they have historically always enjoyed over women (and over people of colour—the English-speaking software development community is not only overwhelmingly male, but overwhelmingly white) is in jeopardy.

I don’t recall where I originally encountered the thought, but it’s becoming more and more clear how true it is, the longer I look around—the nerd community wasn’t born out of a spirit of inclusivity. The community was born in an effort to exclude those who had previously excluded its members—a spirit of “fuck you, now we’re the cool kids.” The community at large claims to be more evolved than the cool kids who rejected them, but that simply isn’t true.

We’re a tribe of hurt little boys, who only ever learned to hurt. When we were overwhelmingly in the minority, it was a support group, and there was no one to hurt. But we never got over the pain of rejection, and we never learned how to rise above the hatred that forged the community.

But now we’re a pop culture, and we don’t know how to deal with that, other than the only way we know how—by lashing out at those who say they don’t like what we’re doing. And those who are most inclined to lash out have the means to do much more damage than a playground fistfight.

This week has dealt a huge blow to the feminist cause in IT, because the public perception of developers being misogynists, and of the community being a boys’ club, has been dramatically reinforced. I’ve seen all kinds of comments confirming that the comments made at PyCon are by no means unusual, or that Richards is alone in finding them highly inappropriate. But for men to demand that women interested in STEM fields simply accept that ribald comments are just part of the environment only serves to keep highly talented women out, because they don’t think having to put up with their colleagues’ shit is worth it.

Regardless of what you think of Adria Richards, or of the picture she tweeted, I would like to think that we can all agree the technical community as a whole needs to stop the cycle of revenge.

Wednesday, 20 March 2013

This is not a meritocracy

UPDATE

An hour ago, SendGrid publicly announced Adria Richards’ termination. They say,

While we generally are sensitive and confidential with respect to employee matters, the situation has taken on a public nature. We have taken action that we believe is in the overall best interests of SendGrid, its employees, and our customers.

In other words, they heard the thousands, if not millions, of people calling for Richards’ termination, and delivered. In an effort to do… what? Save their customer base? This is a hell of a message to send—if you embarrass a man for making tasteless jokes at a technical conference, and he gets fired and complains about it, we’ll throw you to the wolves.

The joker’s behaviour at the conference earned him disciplinary action (whether or not he should have been fired or given sensitivity training is academic)—he was acting as a representative of his company, at an event they had sponsored. Necessarily, he should have been on his best behaviour. Richards was probably representing SendGrid as well, insofar as SendGrid probably paid her to be there, and it was probably all over her nametags—and she may have been wearing company gear too. However, what was Richards’ offense? Saying “that’s not cool” loudly, really.

Rather than go to bat for Richards, and say, “we believe that the software industry is best served by a culture of universal respect, and we don’t condone anyone making inappropriate sexual commentary in the workplace or at a technical conference,” SendGrid has sent the message that they don’t have their employees’ backs. That they either don’t believe that the industry is rife with misogyny, or perhaps that they don’t think it’s a bad thing, or maybe just that it can’t be fixed.

I don’t hold with any of this. I believe that the misogyny that pervades this industry must be confronted head-on. New hire sensitivity training that says little more than, “don’t make dirty jokes around girls” is staggeringly insufficient, and if Human Resources requires this training, then everything that company does in public must reflect the beliefs that that training espouses.

SendGrid has told the world that they believe offensive jokes are okay in the workplace, and that if you call it out, you will be silenced.

Is my calendar right? Is it 2013, or 1963?


This week is a bad week for how I feel about my gender.

We got an early start on Sunday with shockingly insufficient sentences for a pair of teenage rapists, followed up by horrifying apologia from, well, all the major news outlets, CNN included. I’m not going to comment on it here, but I will suggest that you read I Am Not Your Wife, Sister or Daughter, a fantastic article (that my wife Anne wrote) that’s getting a fantastic amount of coverage. She’s absolutely spot-on when she points out that we, as a society, really need to stop trying to humanise rape victims to rape apologists by suggesting, what if it was your wife? Your sister? Your daughter?. It’s not just objectifying, but it also reinforces your audience’s misogynist worldview.

I could really get into it, because it makes me mad… but the way that the professional software community is treating Adria Richards—and, by extension, every woman in the industry, has got me so upset I can hardly see straight.

You probably know where I’m going with this, but let’s review the facts, shall we?

Richards publicly shamed two attendees for cracking sexual jokes about, among other things, “forking his repo” after a suggestion that forking is the highest form of flattery. I understand the pair of them were going on for quite some time, and the PyCon organiser dealt with the situation privately, and the guys were chastised for their behaviour, and that seemed to be the end of it.

Until when they got back to work, when at least one of the pair of jokers was fired. He then posted a strange apology that suggests that he believes Richards was trying to make that happen. Richards sent her own public apology to him and urged his employer to reconsider their decision.

Regardless of this, the male developer community has worked itself into a mouth-foaming rage. People are specifically calling for her dismissal, and there was at least one suggestion that the guy who was fired should sue her. There’s a whole host of men insisting that “dick jokes aren’t harassment”, as though sexual harassment can only occur through individually-directed comments. I’ve lost count of how many people are suggesting that Richards’ fragile female sensibilities caused her to overreact to a “private joke” (one, I’ll point out, was told in a crowded conference hall, and thus is anything but private, unless it was whispered directly into the other person’s ear).

Virtually every comment I read on the thread following the non-apology is coming to his support, and attacks Richards.

Virtually every commenter seems to believe that a man’s desire to make offensive jokes in a public space, while representing his employer, somehow trumps every other person’s basic right to be in a room without being made to feel uncomfortable because of their race, gender, religion, sexual preference, or even no reason at all. That bad jokes are somehow sacrosanct, and that people who are offended by them should simply “grow up and get over it.”

Look, this isn’t the way adults, and professionals, are supposed to talk to each other. This isn’t the way the developer community constantly tries to describe itself to outsiders. We insist, adamantly, that everyone is considered equal, and that the developer community is a meritocracy above all else.

This is, unfortunately, not the reality. Women have never been afforded the respect they deserve within this industry. RADM Grace Hopper invented the compiler, and assembly language, in order to make programming that little bit easier than having to remember and decipher opcodes, and her male peers couldn’t possibly have taken her less seriously… but because of her, I don’t have to have any idea what the x86 instruction set looks like in order to do my work.

And yet marketers at computing events like CES and E3 continue to hire booth babes—in other words, human furniture to make their booth look good. I’ve read of women who have produced games, who later staffed the conference booth for the game, and tech reporters asked her to get the producer, or technical director, as though the idea of a woman being responsible for creating something as complex as a video game was a foreign concept.

I’ve worked in a variety of companies, some larger and some smaller. But the reality is that I’ve worked with far more men in technical roles than I have with women, and that, invariably, when there haven’t been women in the technical group, it’s turned into a boys’ club.

This is unacceptable. The current software developer community is openly hostile to women asking to be treated like human beings, and this shit has to stop. You wouldn’t make racist jokes at a conference (or would you?), so what makes it magically okay to make jokes that objectify sex, and women?

Right, nothing does, because it’s not okay.

It’s not okay to compare an object to a person’s body. It’s not okay to compare a development process to sex. It’s not remotely okay to tell someone who says, “I’m offended”, that nothing offensive happened, and that they’re overreacting.

And it isn’t fucking okay to make death threats against a person who called out inappropriate jokes. Yes, this happened. Yes, the post has been deleted. Yes, I hope YCombinator does the right thing and assists the police in any investigation that might occur, and yes, I hope that investigation happens.

Finally, it’s not even a little bit okay to attack someone for acting on having been offended. That someone got fired for making inappropriate comments while representing his company at a conference shouldn’t be remotely surprising.

Technologists really need to start showing each other a lot more respect, because right now, it really feels like we don’t show each other any.

Sunday, 13 January 2013

On learning good practices the hard way

At the beginning of December, I intended to write an article here about the perceived value of skills certification in the industry, in light of my own recent certification as an Oracle Certified Associate Java SE 7 Programmer. It’s something I’m very glad I did, and it was at my manager’s virtual insistence… but that same manager has also told me that, when perusing resumés to decide who to interview and who to pass over, he places no extra value on applicants with vendor certifications. It’s a bit of a paradox at first, and I promise I will actually publish it.

The problem is that, as usual, life got in the way. Life, this year so far, has also strongly got in the way of any new releases with Project Seshat. There have several bugs that I’ve discovered and fixed, with the help of a couple of good friends, but I haven’t really been able to deploy the most recent work, because of two issues—my son has been sick ever since we returned from our Christmas holiday, and one of the components of this release is turning out to be vastly more complicated than I originally expected.

In retrospect, it’s becoming apparent that I ought to leave out the new feature, deploy the bug fixes for version 0.1.3, and continue on the feature for 0.2.0.

That late realisation aside, like I said, a new component is somewhat complicating matters for me. I became dissatisfied with how I’ve been configuring request mapping a long time ago, and had left a mental note to clean up the technical debt; I was using Zend Framework’s static routes to associate this URL pattern with that controller method. Works reasonably well in most other applications I’ve written, but my desire to use the first part of the URL pattern to differentiate between UIs created a wrinkle large enough that I decided it would (eventually) be more convenient to write a URL decoder to create the route instead.

I worked on the decoder over the holiday, when I had an hour or two here and there after my son had gone to sleep. I implemented the whole thing using test-driven development principles (and fixed a few quirks in my homebrewed test framework while I was at it), and promptly discovered two things:

  1. My original understanding of Zend Framework’s Front Controller and dispatching process was flawed. This may be due to my ever-increase familiarity with Spring Web. I also happen to disagree with how Zend is doing things, but then, that’s largely the point of Project Alchemy—to create my own PHP development framework, based on my needs, by replacing parts of Zend Framework as I find they either aren’t doing what I want, or I just don’t like the API and don’t want to deal with it any more.

  2. The URL decoder could easily be used to fix a hack that I put in place to answer the question of which Notebook to link back to in the “Back” button in the interface. Simply leaving up to the browser Back button is insufficient; I want this button to really be an Up button, the way that the top-left button in iOS and Android apps works. Again, this is part of the point of how I’m writing Project Seshat; I want to write one set of back-end code, and apply an appropriate set of layout, stylesheets and JavaScripts to wrap the application in an idealised wrapper for the usage environment. Whatever device is used, it should work and feel like it was always intended to work on that device.

So, in trying to implement it there—by decoding the Referer URI—I thought I could really easily derive the Correct Value of the Up button. The problem now is that that isn’t remotely the case, based on the navigation paradigm I’m currently using, and intend to use. For the desktop, and probably for more capable mobile interfaces, I want to have the user be able to navigate to a Note either through a Notebook chain or through a Tag. Unfortunately, the way I’ve implemented it so far has created some circular navigation problems, that also run somewhat counter to iOS and Android navigation recommendations. I’m still trying to decide what the best approach is, and it feels like there are several options available to me.

Naturally, this is a pretty big issue that needs some pretty particular and dedicated thought; I really don’t want to just wing it. So, with my son being sick, and I’ve needed to get sle myself, I haven’t especially been able to take the time I want to take. That, in turn, has cost me some development momentum. I also apologise to my testers for leaving a couple of bugs up there for them to deal with while they try it out. I promise, a deployment is coming with bug fixes. Clearly, I got ambitious.

So, at the end of the day, what’s the lesson to be learned here? I think there are several. Stop taking notes in my head, and focus on writing down my thought process, so I can come back to it easily, later (this is valuable in any professional’s working life, particularly software developers, and I feel like being on holiday dropped me off the wagon a bit). Don’t ever combine bug fix releases with feature releases; if it takes longer than anticipated to fix the bugs (assuming you aren’t on a short iteration and deployment schedule), the bugs will remain in production too. Have storyboards for your interface, and understand the style guide(s) you intend to adhere to when you plan out your interaction flow. When designing new features, write down what they will do and how they’ll be used before you get started, instead of making it up as you go; it’s too easy to code yourself into a corner that way.

There are probably others, but I also need to sleep. I’m accepting suggestions. If nothing else, if I can’t be a good example, I can at least be a hell of a warning!

Wednesday, 28 November 2012

Your Encapsulation Is Bad, And You Should Feel Bad

Pass-by-reference is a fantastically powerful tool in object-oriented languages. In Java’s case, it ensures that no argument on a stack frame is longer than a processor word, by only passing along copies of primitives, and the heap locations of objects. It reduces your memory footprint fantastically, because it’s always there, unlike in C, where you had to specifically indicate that this argument is actually a pointer. Java does the same with return values as well—anything that you return from a method that isn’t a primitive is passed by its location on the heap.

And thus are a whole host of encapsulation and coupling issues born—particularly when you work with Collections.

Let’s say, for the sake of a specious example, I run a rental car agency. My agency is represented by an object Location:

According to the Rules and Standards of JavaBeans, I’ve done encapsulation right… until I decide to process a series of updates to my stock in this boneheaded way:

If, at any point after that for loop, I want to work with the list of vehicles, I’ll only have access to those vehicles that haven’t been washed in a week—I removed them from the same List that my Location object refers to.

Like I say, it’s a pretty specious example, but it shows what kind of unintended consequences can crop up when you pass mutable objects around by reference. Fortunately this doesn’t happen with Strings and Numbers (because they create new objects on the heap just about every time you assign a new value), but as soon as you start doing the same thing with more complex objects, you risk loss of data integrity. My Updater needs to know how my Location stores, and returns, the list of Vehicles in order to prevent problems, when it probably shouldn’t. The encapsulation here is bad, because it permits side-effects.

So what’s the fix? Replace the body of Location.getVehicles() with this: return new ArrayList(vehicles); and keep on going as-is. While each individual Vehicle that the lists are backed on will probably point to the same place on the heap (and this may, itself, have attendant problems, depending on what you’re doing, at least you know, for a fact, that whatever changes you make to the list you got back will be self-contained.

This gets even worse when you start throwing around DTOs for different serialisation methods. Because various annotations used by persistence architectures may not necessarily be compatible, often times you need to create three different DTOs for each of your database, XML, and JSON representations. These DTOs should only exist long enough to prepare your problem domain object for serialisation, or to deserialise something into your problem domain.

Say you created an amazing Web service that’s backed on SOAP (I know, I know, JSON geeks. It’s just an example). When I call your Java API’s method to getThing(), I shouldn’t be aware of the SOAP Body. I shouldn’t have to call, say, thing.getAttr(“thing”) to get something that’s an XML attribute, and then thing.getOtherThing().getValue() to get the String value of something that’s stored as an XML element. As a consumer of your API, I shouldn’t be aware of this; it means two things:

  1. You can’t easily move your service away from SOAP, without either...
    1. Forcing all your customers to update their code to use a new API, or
    2. Internally converting your new serialisation into something that can be expressed as a SOAP call,
  2. and you’ve told the world that your service is backed on SOAP.

Whether or not you’re proud of the fact that you’re using SOAP is irrelevant; it’s an implementation detail that I, as a consumer of your API, don’t care about. For all I know or care, you could be using a proprietary binary format, or even passing messages around by carrier pigeon. From my application’s perspective, this is all irrelevant. The encapsulation here is bad, because it exposes implementation details.

So, what’s the take-away from all this? Two things:

  1. Don’t return references to your member collections and arrays. It’s bad for coupling. Return copies, instead.
  2. When designing your API, give the consumers of it a paradigm that makes sense from the problem domain, instead of just blindly representing your storage format.

Thursday, 22 November 2012

LinkedIn, you're fired.

Let’s be clear about something, LinkedIn. You’ve done good work. This has nothing to do with your performance as a professional social network. But your attitude about your job—the lackadaisical attitude toward data security, the fact that your communication with users who have question can take weeks, and your constant suggestions that your customers are mistaken about plain and simple facts—just can’t go on anymore. Your services won’t be required anymore. Someone will pack up your things.

Yup, I’m firing LinkedIn. I almost called this, “LinkedIn, I quit”, until I remembered that they are providing me with a service, so the firing metaphor is more apt. Particularly considering they’re really all about getting people jobs.

And it’s not as though LinkedIn has been totally useless to me. I’ve found a job through LinkedIn, coincidentally interviewing with a former classmate. My long-time tagline, “Minor PHP deity/aspiring system administrator/technorenaissance man” was referred to by my hiring manager at my current job as part of why he hired me. Whether it was because it demonstrated my sense of humour or my confidence, or even just a joke, I’ll probably never know. But the point is, LinkedIn has its uses.

The problem is in their customer service—more accurately, their almost-total lack of customer service. First of all, when you have a problem, you are functionally prevented from submitting a question to their help desk without first making a cursory look in their knowledge base. Seems sort of fair, until you remember that, sometimes, you know going into it that your issue isn’t covered by a question: there are problems that come up that quite simply need a human being to resolve. So there’s a barrier to getting the help you need, and when you consider it a little further, implies that they don’t have enough staff working their help desk. This point, in fact, gets demonstrated later on.

The second barrier to getting help, once you’ve actually made it to their “Contact Us” page, is that, once you’ve typed out your question, a modal dialog pops up, where it performs a keyword search in its knowledge base for you, presumably based on your summary of your issue. JavaScript sets the browser caret on the button labelled “I Found My Answer”, which redirects away from the page. When you click the Back button in your browser, your question is gone. This is yet another attempt to obstruct users from getting the help they need, and another vivid illustration of just how understaffed their help desk is. To describe it as “poor user experience” is a bit insufficient.

So, just in order to get to the point where you have successfully asked a question, LinkedIn has done everything they can to prevent you from doing so. During the most recent occasion when I had to submit a ticket, I actually muttered at the website, “I hate you so much, I wish I could hate you to death.” I know, I know, it’s not really my line, but Goddamn if it isn’t evocative of the particular type of impotent rage you feel when you’re already mad at something that you can’t really take out your anger on, and it thwarts you.

Now, all that being said… what are the problems I’ve had so far with LinkedIn? Leaving out their comically bad response to the infamous Gawker Media password crack—resetting everyone’s password—there have been three:
  1. Recommending connections to contacts from my email
  2. Continuing to send Groups emails to an email address I (thought I had) removed from my account
  3. Automatically connecting me to someone hours after they requested a connection
These are, in my opinion, huge problems in terms of data security. What made them all even worse was the fact that, in every case, the help desk agent flatly contradicted that the problem existed. Because I don’t particularly feel like trying to sum them up, I’m just going to copy and paste the ticket contents, and let LinkedIn speak for themselves. I apologise in advance for my language, and I haven’t changed the agent’s declared names. I see no benefit in protected the guilty.

Support History » Why is LinkedIn accessing my email account?!

Your Question 04/26/2010 23:42
I just received my weekly LinkedIn network update, letting me know that a person I regularly email has just joined LinkedIn.. which seems somewhat strange to begin with, since this person has little reason, that I know of, to use LinkedIn.

What really set off a flag for me, though, is the fact that LinkedIn *knew that I know this person*. I have *never* instructed LinkedIn to import my contacts from my webmail services, and never intend to. So why is it that LinkedIn is suggesting that I add people from my Google Mail contact list to my LinkedIn network?

Before this event, I have been particularly satisfied with LinkedIn’s service; I’ve been able to keep in touch with colleagues, make new connections with recruiters, and in one instance I was able to secure a job because of it. It’s quite valuable to me, but if the service can’t do something simple like respect my privacy, I may have to leave the service and tell everyone I know that uses it precisely why I left.

The contact in question’s email address is [REDACTED]

Thank you. I hope to hear from someone promptly regarding this matter.

LinkedIn Response 05/01/2010 11:21
Dear Matthew,

Thank you for contacting LinkedIn Customer Support.

Please be assured that LinkedIn would never access your address book or import contacts without your permission. Our policies require all members to enter a password any time an address book is imported or other actions are taken on the account. My records show that you have imported some contacts into your LinkedIn account at some point in time. One of those contacts is the one in question. You can see the contacts you have imported if you log into your account and click on “imported Contacts” under “Contacts”. Please know that you can delete imported contacts at any time by selecting contacts and clicking on the "Delete selected contacts" button.

If you have further questions, please feel free to reply to this message.

Roberta
LinkedIn Customer Support

Support History » Remove secondary email

Your Question 01/23/2012 23:47
I attempted to remove [REDACTED] as an email associated with my LinkedIn account a couple of months, preferring [REDACTED] as my email address, due to a brief security compromise of my Google Mail account.

I’ve noticed over the last little while that I’m still receiving email from LinkedIn at the old address. Nowhere in my settings does it indicate that you’re storing the other address, until I came in here to contact you and I found it as a secondary email address.

Remove this address from my account immediately. If this cannot be done, then it must be exposed as my alternate email address within my account settings, so that I might change it to another alternate email address.

LinkedIn Response 01/24/2012 01:06
Hi Matthew,

Thanks for your email letting me know that you still receiving emails to your old email address and I understand how concerned you must be about this matter.

I’m able to locate the account and see that there’s only one associated email address [REDACTED] which have 166 connections. I’ve tried to locate the account with the email address [REDACTED] mentioned, but I couldn't find any account on our records.

[I continue to receive LinkedIn Groups emails at the old address to this day. –ed.]

However, if you want me add your old email address to our “Do Not Contact List” in order to stop receiving emails in future. Please reply to my message so that I’ll be glad to proceed with your request further.

Matthew, I await for your response.

Regards,

Susheel
LinkedIn Customer Service.

Support History » Unauthorized connection

Your Question 11/07/2012 20:44
At 7 November 2012, 12:34 AM EST, I received an email notification from LinkedIn, informing me that “[REDACTED]” would like to connect with me. I received another email at 1:51 AM EST, suggesting that I see what [REDACTED] has been up to, because we had now become connected.

I did not authorize this action. In point of fact, I did not know of either email until I woke up this morning and checked my email. I can only assume that either some device in my house performed the authorization on my behalf, or that this authorization was committed fraudulently. Accordingly, I insist that you divulge all the logs you have respecting this authorization, and with the original request made by [REDACTED] that the two of us connect, so that I can verify that the request did not come from any device in my possession.

Please note that this is not the first problem I have had with LinkedIn. As you can surely see from my support case history, I also have reason to believe that some process of yours gained access to my Google Mail account without my permission. Despite the support representative's insistence that I must have kicked it off myself, the fact is, I disagree with such “helpful” services on general principle and would never have done so. I have also had no end of difficulty purging my other email address from systems. I continue to receive LinkedIn Group notifications at my past email address, despite a support rep’s insistence that the old email address has been purged from your systems.

Your Response 11/13/2012 14:38
And now I’ve just discovered I had apparently followed PwC Consulting?! What the fuck, guys. This shit has to stop.

LinkedIn Response 11/20/2012 14:29
Hi Matthew,

There are a couple of scenarios that could explain the possibility of unauthorized access to a LinkedIn account:

1. If you’ve recently logged into your account from a public computer and didn’t completely sign out of your account, the next person to access the site on that computer may have unintentionally logged into your account.
2. If you share a computer with another person either at your workplace or home and didn’t completely sign out of your account, the next person to access the site on that computer may have unintentionally logged into your account.
3. Your account has been compromised by someone with malicious intent.

In order to secure your account, we have taken the following actions:

1. We signed you out of your account from every computer it has ever been accessed on. This will now prompt a new login for your account.
2. We sent a password reset link to the primary email address listed on your account.

We also like to recommend these best practices for your online privacy:

1. Always completely sign out of your LinkedIn account each time you leave a computer.
2. Don’t use the same password on multiple websites. If fraudsters identify your password, they can use it to access your other accounts.
3. Select passwords that can't easily be guessed. Create one that includes 10 or more characters. Hint: Think of a meaningful phrase or quote and turn it into a complex password using the first letter of each word in the sentence and add more complexity by adding capital letters, punctuation and symbols.
4. Never give your password to others or write it down.

If you continue to see anything suspicious, please report it to us immediately.

Regards,

Andrea
LinkedIn Trust & Safety

Two canned responses! It’s like they aren’t even trying. You can see that I didn’t bother responding to any of the tickets; there was demonstrably no point, since they weren’t putting forth any effort to resolve the issue in the first place. But then, maybe that’s what they’re trained to do.

Like I said above, I’m firing LinkedIn. Exactly what form that will take on LinkedIn’s servers, I have yet to decide. I want to leave a link to this article there, so that anyone who looks me up can read about my awful experience, but I also want to retain the maximum possible visibility—which would mean maintaining all my connections and job history. Decisions, decisions.

LinkedIn, consider this your two weeks’ notice.

Monday, 29 October 2012

Why Do Not Track doesn't really matter to me

It seems that every couple of weeks, a new article crosses Boing Boing or Slashdot about Do Not Track. Not too long ago, it came out that Microsoft was going to launch Internet Explorer 10 with DNT switched on by default, and all the advertisers were up in arms. Now, Yahoo! has announced that, as a response (read: a "fuck you") to Microsoft, they're going to ignore DNT if the visitor is using IE10, because they can't rely on it truly being a reflection of the user's preference. Fine. It's certainly their prerogative whether or not they intend to adhere to it; it's not as though DNT is a required thing. But in looking inito this very public pissing contest about Do Not Track, I discovered that Microsoft isn't actually doing anything that goes against the standard.

So, without further ado, a few things any Web service provider needs to know about Do Not Track:
  1. Do Not Track is not mandatory for providers. DNT is not a requirement for servers. Though a standard is being drawn up for it, respecting that header is purely voluntary. Even indicating that you're respecting the header is voluntary; the W3C draft only defines the response header as something that a server MAY send. Besides, with the standard only in Working Draft status, implementing it at this time may mean that you may have to go back and re-implement Do Not Track... never mind the fact that Yahoo! is being pretty public about intending to ignore it. I really doubt that Web browsers will do much more than say "part of this page didn't return the DNT header", at worst, if a provider decides not to adhere to it, and even that seems unlikely.
  2. Most providers don't have to care. DNT only pertains to tracking done by third-party providers to a Web visit. The canonical example of this is an advertiser, such as DoubleClick. DART ads are damn near everywhere on the Web, and I have to admit it's downright spooky when I look at the Roots Canada online catalogue one day, and for the next three days, every ad I see is suddenly Roots, where they'd never appeared before. Clearly, DoubleClick is watching you. But like I say, unless you're providing content that will be included in another organisation's Web pages, then Do Not Track does not apply to your service, and you can go on ignoring it. An exception is that if your service is forwarding tracking data to a third party on the server side, then you'd actually need to worry about what the DNT header contains, if you're bothering to adhere to it at all. However, most providers prefer to offload as much of that work to the user agent, for the sake of apparent site speed, so, like I say, most providers don't have to care.
  3. Enabled-by-default isn't actually prohibited by the W3C Working Draft. When Microsoft announced that IE10 would switch on DNT by default, this was a valid option, according to the (in-progress) standard. Only in the most recent revision, dated 2 October, was the default specifically stated to be "assume no preference has been expressed." Until then, the standard only stated that intermediary services (such as proxies) may not change what preference is or is not indicated. Currently, the standard states, “A user agent MUST have a default tracking preference of unset (not enabled) unless a specific tracking preference is implied by the decision to use that agent.” Microsoft has publicly stated that their new browser will enabled DNT by default. Certainly there will be clear statements in all the marketing materials to this effect. It’s clearly a safe assumption that use of IE10 implies a specific tracking preference on the part of the user.
  4. The "Acceptable Uses" definition makes a lot of DNT irrelevant for even third-party content providers. My biggest concern about Do Not Track was around maintaining security audit information. Good news! That's one of many acceptable uses of tracking data, that allow a provider to largely ignore the Do Not Track header. The only stipulation made, when claiming "acceptable use" is that you don't pass on that stored data (which you shouldn't anyway), and that you don't use it to personalise ads. That, right there, is the entire crux of Do Not Track: not personalising ads. Track all the information you want, just don't share it and don't expose that you're doing it.
  5. Most, if not all, browsers provide a mechanism to pop up a dialog when a site wants to store a cookie. For the most part, browsers already have the technology to largely prevent effective multi-site tracking by advertising providers. While Do Not Track is a little more comprehensive, simply refusing to allow, say, 112.2o7.net to put a cookie in your browser goes a long way to preventing the, from following you around the web. Granted, it would force users to investigate the options on their browsers (and I am feeling a little cynical right now about the motivations of users), but I don't think it's really asking too much that a person learn about the tools they're using.
So, as someone writing a Web application that I'd love to have other people use, how much do I really need to care about Do Not Track? As it turns out, not much. Not adhering to it, in all likelihood, won't affect interoperability, but if it seems like it is, I'll just need to add one file, in one location, indicating that my service is a first-party service, and that's the end of that.

All in all, I don't really think Do Not Track has much in the way of teeth. The advertisers that it's mostly aimed at are such behemoths in terms of coverage that even without being able to personalise some fraction of the users' ads, that they'll still be making money hand over fist. And the advertisers will continue to be able to track you... they just won't be able to make it obvious.

Saturday, 27 October 2012

On monitors and error detection

Earlier today, a colleague and I were discussing monitoring tools for Web services. He recently joined our team as a systems administrator, and I was filling him in on a homebrew monitoring service I put together a couple of years ago, to cover a gap in our existing monitor’s configuration, done in the spirit of Big Brother. He had praise for its elegance, and we joked a bit about reusing it outside the company, the fact that it would need to be completely rebuilt in that case (since, though it wasn’t composed of original ideas, just a merger of Big Brother and Cacti, it remains the intellectual property of $EMPLOYER$), and whether or not I would even need such a service for Prophecy.

After thinking about it briefly, I realized that not only will Project Seshat deserve some kind of monitoring once I install it on my server—I guess I’ll just add that to the pile of TODOs—but I remembered that I have a WordPress instance running for the Cu Nim Gliding Club, in Okotoks, Alberta. Surely a production install of WordPress deserves monitoring, in order to make sure that Cu Nim's visitors can access the site.

So, while waiting at a restaurant for my wife and our dinner guests to arrive, I took to The Internet to look for any existing solutions for monitoring WordPress with, say, Nagios. I may not be familiar with many monitors, but I know enough about Nagios to know that it works well with heartbeats—URIs that indicate the health of a particular aspect of a service.

The first hit I found that wasn’t a plugin for one of the two was a blog entry describing how to manually set up a few monitors for a local WordPress instance. It explained how to configu Nagios to run a few basic service checks: that the host in question can serve HTTP, that it can access the MySQL server, and that WordPress is configured, a single check on the homepage.

To me, this seems woefully incomplete. A single check to see that anything is returned by WordPress, even if you are separately checking on Apache and MySQL, strikes me as being little more than an “allswell” test. Certainly, success of this test can be reasonably inferred to indicate good health of the system, but failure of this test could mean any number of things, which would need to be investigated to determine what has gone wrong, and the priority of the fix.

When I use a monitoring system, I want it to be able to tell me exactly what went wrong, to the best of its ability. I want it to be able to tell me when things are behaving out of the ordinary. I want it to tell me that, even though the page loaded, it took longer than some threshold that I've set (which would probably warrant a different level of concern and urgency than the page not loading at all, which would be the case with a single request having a short timeout). In short, I want more than just the night watch to call out, “twelve o’clock and all’s well!”.

The options that I could take to accomplish this goal are myriad. First of all, yes, I want something in place to monitor the WordPress instance. But for original products, like Project Seshat, I would definitely like something not just more robust, but also more automatic. Project Alchemy is intended to create an audit trail for all edits without having to specifically issue calls to auditing methods in the controllers. I’d love to take a page from JavaMelody and create an aspect-oriented monitoring solution that can report request timing, method timing, errors per request, and perhaps even send out notifications the first time an error of a particular severity occurs, instead of the way Big Brother does it, where it polls regularly to gather data.

Don’t get me wrong, it’s probably a huge undertaking. I don’t expect to launch Project Seshat with such a system in place (as much as I’d love to). But it’s certainly food for thought for what to work on next. And when Seshat does launch, I will want to have a few basic checks to make sure that it hasn’t completely fallen over. After all, so far, I’ve been adhering to the principle of “make it work, then make it pretty.” May as well keep it up.

Saturday, 22 September 2012

In which the general fear of TDD is discovered

Since I last wrote about test-driven development—since we spent that time at work learning how to do it, I’ve been trying to make use of it in my off-time development. I’ve mentioned before that I’ve been writing an ORM from scratch, to satisfy an itch that I have. In its current incarnation, I haven’t really had many opportunities to write anything using it, other than an aborted attempt to create a tracker for the No-Cry Sleep Solution.

Earlier this year, the note-taking web app that I’ve been using for years made a major overhaul of their user interface…and left mobile web out in the cold. Seriously. If you aren’t accessing the site from something that can fully act like a desktopfull-scale browser, then you’d better be on either an Android or iOS device, because otherwise, you’ve been left out in the cold.

At the time, I was well and truly in the cold. My mobile phone was, and still is, a Palm Centro. My only tablet-like device was my Kobo Touch (my wife owns a Nook Color, but I wasn't about to both commandeer it during the day and install a note taking app), though we’ve since also purchased an iPad with LTE. At work, at the time, I used my Kobo to present myself with my notes during scrums. Since then, I’ve been writing to a static HTML file on those days that I don't bring the iPad to the office, but there’s still a nontrivial issue of synchronisation. While I could probably use Dropbox and a reasonably simple PHP application to read and write to a single note file, that still just doesn’t do it for me.

So, I opted to begin writing my own, using Alchemy and Zend Framework on the back end. The initial progress wasn't so bad, and it isn’t as though I didn’t have alternatives that have worked reasonably well in the meantime. I decided to basically cater to my own use cases, since I could. Mobile Web would be reasonably fully featured, if a degraded experience. My Kobo Touch would get a good interface where I could edit notes, or write new ones, easily. It would all be there.

The problem is that it hasn’t always been smooth sailing. Ignoring the fact that I don’t often have the opportunity to work on it at home, having a toddler, it seems like with every model I implement, I find another thing about Alchemy that needs to be added or fixed. I’ve been trying to adhere to test-driven development to do that, but by God, I made it difficult to do that in some places. Doing the whole “TDD as if you meant it” thing can be particularly tricky when you’re working with an existing codebase that isn't particularly (or even remotely) tested, and particularly when you're writing web application controllers. Controllers are notoriously hard to unit test, if for no other reason that because their very purpose is side effects, which runs somewhat contrary to many of the premises of test-driven development. I’m finding that it’s far more straightforward to perform acceptance testing on your controllers, and actually go through the motions of the task you’re seeking to test.

Where I’ve been running into difficulty with Project Seshat,¹ though, is in code that I not only wrote a long time ago (somewhere on the order of three years), but also works perfectly well in isolation. The Model class, and its database-driven subclass, provide a parent class to all model-like activity in my application. It acts as entity, DAO, and service layer, mainly because that’s what made the most sense to me at the time I started writing it (this was well before I started working with enterprise Java. I still disagree with the notion of the DTO, but have yet to fully articulate why, to my own satisfaction). And that’s fine; it can still work reasonably well within that context. The problem is that, at some point when working with each of the last two Models I’ve added, the logic that stores the information in the database has both succeeded and failed in that regard at the same time.

Huh?

One of the core features of the ORM in Project Alchemy is that every change that’s written to the database with an expectation of long-term persistence (so, basically, everything that isn’t session data) also gets logged elsewhere in the database, so that complete change history is available. This way, if you ever need to figure out who did something stupid, it’s already there. As a developer, you don’t have to create and call that auditing layer, because it was always there, and done for you.

This audit trail, in its current form, is written to the database first—I decided to implement write-ahead logging for some reason that made perfect sense at the time. Not that it doesn’t make sense now, but there are a lot of features that still have to be implemented…like reading from this log and providing a straightforward function for reverting to any previous version. But at least the data will be there, if only, for now, for low-level analysis.

At any rate, because I can see these audits being written, I know that the ORM is at least trying to record the changes to the entities that I've specified; they’re available at the time that I call the write() method in the storage area for uncommitted data. The pain is that when it tries to create a new instance of the Model in the database, the model-specific fields aren’t being written to the entity table, only to the log. The yet-more painful part is that this doesn’t happen in testing, when I try to reproduce it in a controlled environment. This probably just means that these bug-hunting tests are insufficient; that they don’t fully reproduce the environment in which the failure is occurring.

So yeah. TDD, while it’s great for writing new code, is very difficult to integrate into existing code. I’ve had to do what felt like some strange things to shoehorn a test fixture in place around all this code I’ve already written. I recognise that the audit trail makes the testing aspect a little bit more difficult, since it's technically a side-effect. However, I don’t really want to refactor too much, of any, of the Model API, simply because my IDE isn’t nearly clever enough to be able to do it automagically, and because I still really, really want the audit trail to be something that doesn’t have to be specifically called.

I am, however, beginning to understand why so many developers who have never really tried TDD dismiss it, claiming that you end up writing your code twice. At first, you think that the test and the code are completely distinct entities, and that the structure of your tests will necessarily reflect your code. Yeah, this would mean that you’re doing everything twice. But that’s not TDD done properly. But then when you get into it, you realise that it isn’t the new code you have to write twice, but all the existing code that has to be massively refactored (and in some cases, virtually rewritten, so dissimilar is the result from the what you started with), and that’s always a daunting thought. You may even find yourself feeling compelled to throw out things you’ve spent a great deal of time and effort on, purely in order to get it testable.

I get that. That’s where I am right now. But there are two things to remember. First of all, your code is not you. If you want to work effectively in any kind of collaborative environment, whether at work or on an open-source project, you need to be able to write code and leave your ego at the door. Hell, the same thing goes for personal projects. The second is that if you refuse to make something better (whether better means more efficient, more maintainable, or whatever), simply because you invested x hours in it is foolish. You probably made something good, but you can always make it better if you’re willing to put in the effort.

And speaking of persistence issue, I’m sure I’ll fix it eventually. I already did once, though I didn’t properly record how I did it. Gee, if only I had some kind of mechanism for taking notes!


¹ Seshat was the Egyptian goddess of wisdom, knowledge, and writing. Seems appropriate to use a name that means "she who scrivens" for the tool you're going to use for your own scrivening.