Looking back on looking forward

I was just poking around in the BlackBerry PRIV user guide, in an attempt to figure out what USB-based video out standard it uses, when I noticed that certain PRIV models support wireless charging, using the Qi and PMA standards. I got excited (I’m planning on buying one, eventually, once I feel that I’ve got my money’s worth on the spare battery I bought in the fall [read: its lifetime goes for about as crap as the original]) and checked out whether, for example, IKEA’s wireless chargers support either of those standards. Good news for me, they do. I don't have one yet, but we have a bunch of other IKEA furniture, and at least that way I can get a lamp, or and end-table, with a built-in charger for my phone. Hurray!

And I got to thinking about wireless charging, and this thought occurred to me, “great, I see that Android and BlackBerry have caught up to where Palm was seven motherfucking years ago.” Palm, and by extension, HP, were way ahead of the curve when it comes to mobile computing, and for a very good reason. Wireless charging on the Palm Pre. The OS in that phone had real multitasking from day one, beating Apple to the punch by a year. And Palm beat everybody to the punch with their Palm Pilot. Apple was dicking around on the Newton with full handwriting recognition, Palm had a comparatively minuscule processor, and they nailed it. Given their technology constraints, instead of trying to fit the technology to the ideal, they identified the paradigm (pen-like input instead of keys, because the machine was so physically small) they felt would be the killer app, and found a way to apply that paradigm to a pocket-sized device, and won the market.

They kind of reinvented handwriting with Graffiti, but that also allowed them to create an input mode that didn’t actually require user verification the way that even normal handwriting on the page does (e.g. am I staying within the lines? Is this even legible?). Graffiti allowed the Palm user to keep writing away without moving their hand across the screen, and with a simpler gesture set that was still sufficiently similar to real handwriting to be easy to to learn, but also fit inside the tiny 186 processor they had. They revolutionised computer-supported time organisation, and by God if I didn’t want one for myself the first time I laid eyes on it.

Then BlackBerry (well, Research in Motion at the time) comes along with the original BlackBerry. They did the same thing, figured out how to fill a technology gap (simple email where the person was), made it pocket sized, and ran all the way to the bank with it. And like Palm, with something that the user doesn’t actually have to look at in order to input data.

And thinking about my deep and abiding love for both of these technologies reminded me of why I called my “small business” (which I have never really got around to getting off the ground in any meaningful way) Prophecy Newmedia. The notion was that Prophecy would provide its clients with the right technology for their online presence, not just the flashiest, newest stuff. Flash included—it was a pain in the ass from beginning to end, accessibility was notably awful, and after a weird artist love affair with it, it was relegated to streaming video once CSS and AJAX calls were able to do everything Flash could, until Flash itself got end-of-lifed by the vendor, because HTML5 has replaced it.

While I haven't necessarily picked the winning vendors every time (God rest Palm’s soul, and BlackBerry hasn’t been looking very hot since they shit the bed trying to beat Apple at consumer media devices), I feel like when I see an approach to doing things that works… it works. Handheld computing is something I’ve been a devotee of for probably twenty years. Wireless charging, oh my God yes (because wires suck, just plonk the damn thing down and be done with it). Machine readability of websites was part of why I didn’t like Flash in the first place, and Google’s off and running with it, along with everybody else who does big data crunching.

What’s next? I don’t know yet. But it’s going to be exciting.

In which the general fear of TDD is discovered

Since I last wrote about test-driven development—since we spent that time at work learning how to do it, I’ve been trying to make use of it in my off-time development. I’ve mentioned before that I’ve been writing an ORM from scratch, to satisfy an itch that I have. In its current incarnation, I haven’t really had many opportunities to write anything using it, other than an aborted attempt to create a tracker for the No-Cry Sleep Solution.

Earlier this year, the note-taking web app that I’ve been using for years made a major overhaul of their user interface…and left mobile web out in the cold. Seriously. If you aren’t accessing the site from something that can fully act like a desktopfull-scale browser, then you’d better be on either an Android or iOS device, because otherwise, you’ve been left out in the cold.

At the time, I was well and truly in the cold. My mobile phone was, and still is, a Palm Centro. My only tablet-like device was my Kobo Touch (my wife owns a Nook Color, but I wasn't about to both commandeer it during the day and install a note taking app), though we’ve since also purchased an iPad with LTE. At work, at the time, I used my Kobo to present myself with my notes during scrums. Since then, I’ve been writing to a static HTML file on those days that I don't bring the iPad to the office, but there’s still a nontrivial issue of synchronisation. While I could probably use Dropbox and a reasonably simple PHP application to read and write to a single note file, that still just doesn’t do it for me.

So, I opted to begin writing my own, using Alchemy and Zend Framework on the back end. The initial progress wasn't so bad, and it isn’t as though I didn’t have alternatives that have worked reasonably well in the meantime. I decided to basically cater to my own use cases, since I could. Mobile Web would be reasonably fully featured, if a degraded experience. My Kobo Touch would get a good interface where I could edit notes, or write new ones, easily. It would all be there.

The problem is that it hasn’t always been smooth sailing. Ignoring the fact that I don’t often have the opportunity to work on it at home, having a toddler, it seems like with every model I implement, I find another thing about Alchemy that needs to be added or fixed. I’ve been trying to adhere to test-driven development to do that, but by God, I made it difficult to do that in some places. Doing the whole “TDD as if you meant it” thing can be particularly tricky when you’re working with an existing codebase that isn't particularly (or even remotely) tested, and particularly when you're writing web application controllers. Controllers are notoriously hard to unit test, if for no other reason that because their very purpose is side effects, which runs somewhat contrary to many of the premises of test-driven development. I’m finding that it’s far more straightforward to perform acceptance testing on your controllers, and actually go through the motions of the task you’re seeking to test.

Where I’ve been running into difficulty with Project Seshat,¹ though, is in code that I not only wrote a long time ago (somewhere on the order of three years), but also works perfectly well in isolation. The Model class, and its database-driven subclass, provide a parent class to all model-like activity in my application. It acts as entity, DAO, and service layer, mainly because that’s what made the most sense to me at the time I started writing it (this was well before I started working with enterprise Java. I still disagree with the notion of the DTO, but have yet to fully articulate why, to my own satisfaction). And that’s fine; it can still work reasonably well within that context. The problem is that, at some point when working with each of the last two Models I’ve added, the logic that stores the information in the database has both succeeded and failed in that regard at the same time.

Huh?

One of the core features of the ORM in Project Alchemy is that every change that’s written to the database with an expectation of long-term persistence (so, basically, everything that isn’t session data) also gets logged elsewhere in the database, so that complete change history is available. This way, if you ever need to figure out who did something stupid, it’s already there. As a developer, you don’t have to create and call that auditing layer, because it was always there, and done for you.

This audit trail, in its current form, is written to the database first—I decided to implement write-ahead logging for some reason that made perfect sense at the time. Not that it doesn’t make sense now, but there are a lot of features that still have to be implemented…like reading from this log and providing a straightforward function for reverting to any previous version. But at least the data will be there, if only, for now, for low-level analysis.

At any rate, because I can see these audits being written, I know that the ORM is at least trying to record the changes to the entities that I've specified; they’re available at the time that I call the write() method in the storage area for uncommitted data. The pain is that when it tries to create a new instance of the Model in the database, the model-specific fields aren’t being written to the entity table, only to the log. The yet-more painful part is that this doesn’t happen in testing, when I try to reproduce it in a controlled environment. This probably just means that these bug-hunting tests are insufficient; that they don’t fully reproduce the environment in which the failure is occurring.

So yeah. TDD, while it’s great for writing new code, is very difficult to integrate into existing code. I’ve had to do what felt like some strange things to shoehorn a test fixture in place around all this code I’ve already written. I recognise that the audit trail makes the testing aspect a little bit more difficult, since it's technically a side-effect. However, I don’t really want to refactor too much, of any, of the Model API, simply because my IDE isn’t nearly clever enough to be able to do it automagically, and because I still really, really want the audit trail to be something that doesn’t have to be specifically called.

I am, however, beginning to understand why so many developers who have never really tried TDD dismiss it, claiming that you end up writing your code twice. At first, you think that the test and the code are completely distinct entities, and that the structure of your tests will necessarily reflect your code. Yeah, this would mean that you’re doing everything twice. But that’s not TDD done properly. But then when you get into it, you realise that it isn’t the new code you have to write twice, but all the existing code that has to be massively refactored (and in some cases, virtually rewritten, so dissimilar is the result from the what you started with), and that’s always a daunting thought. You may even find yourself feeling compelled to throw out things you’ve spent a great deal of time and effort on, purely in order to get it testable.

I get that. That’s where I am right now. But there are two things to remember. First of all, your code is not you. If you want to work effectively in any kind of collaborative environment, whether at work or on an open-source project, you need to be able to write code and leave your ego at the door. Hell, the same thing goes for personal projects. The second is that if you refuse to make something better (whether better means more efficient, more maintainable, or whatever), simply because you invested x hours in it is foolish. You probably made something good, but you can always make it better if you’re willing to put in the effort.

And speaking of persistence issue, I’m sure I’ll fix it eventually. I already did once, though I didn’t properly record how I did it. Gee, if only I had some kind of mechanism for taking notes!

---

¹ Seshat was the Egyptian goddess of wisdom, knowledge, and writing. Seems appropriate to use a name that means "she who scrivens" for the tool you're going to use for your own scrivening.

A question of prudence

Some time ago (and by “some time”, I mean “roughly eleven years”), I came up with an idea for a strategy game that would be grander in scope than all other strategy games. I don’t recall if it was to be multiplayer (though I doubt it), but I recently realised that, using social platforms like Facebook, MySpace, OpenSocial, &c., it’s quite doable, and with a gentle nod to the strategy games of yesteryear.

Now, I’ve forgotten, in the intervening decade, a lot of the detail. I seem to recall that the original version was supposed to include a first-person mode, which I’m uncertain if I’d be able to create even now. I knew I couldn’t then, and I doubt, highly, that with a social platform, first- or third-person view would even be an option. I’m not exactly what you might call “proficient” with applications like Flash or Silverlight.

I’ve started to write some of the plans down about how the game mechanics might work. I’ve found that, with the proper planning, it could allow players to control not only vast empires, but even small parcels of land, or even just groups of AIs. Players could govern each other under the right circumstances, and a player whose empire was governed by another could feasibly rise up against his former master.

It almost wouldn’t be a game, as much as it would be a simulation, on a grand scale, of the development of intelligent civilization (and yes, I know, there’s already a popular franchise of strategy games with that name… this game would be, in general, a step removed from the necessary micromanagement that I find Mr Meier’s game espouses). The difficulty that I have is that I want to talk about it, but I don’t want to have my ideas stolen by a programmer with more time, or (more likely) a team of subordinates to do the work. Any suggestions for how I could discuss this notion? I'm afraid that if I let my momentum flag, it’ll be another ten years before anything gets done on this!