Symmetry and Code Refactoring

Whenever I make large refactoring changes within a project I think about it in terms of symmetries. I'm not sure anyone else could find that useful, but it's worth a shot.

While getting my Physics undergrad degree there were two things that really held my fascination, quantum physics in general and the role of symmetry in natural laws.

In my opinion the concept of symmetries in nature as the basis of conservation laws was one of the most elegant findings in Physics. We owe Emmy Noether for that almost 100 years ago (between this, Relativity, and Quantum Mechanics, the early 20th century in Physics was just a ridiculously productive period of progress in scientific understanding).

Since then, I've found it interesting, and sometimes useful, to think about things in terms of symmetries. For instance, I think a fully rational ethical philosophy can be built on top of expressing our existence in terms of symmetries (the ideas behind "walking 1000 miles in someone else's shoes" or "The Golden Rule" I see as rooted in this).

But back to coding...

I've sometimes found myself needing to make major changes to a complex system. This can always, as they say, be "fraught with peril." The first (and biggest) of these started with a monolithic Java app that wasn't built on a framework, represented objects with maps, had manual construction of SQL for database access, used CORBA for Java client communication, had little vertical layering of levels of abstraction and little horizontal separations of logically separate spheres of influence.

Certain types of changes to that type of system were just a nightmare to contend with. A particularly obvious example is how changes to the object model that affected the storage in the database might require changes in several seemingly unrelated places in the code where SQL strings were manually constructed.

The team I led transformed the app (which we already started shipping) to be J2EE based, with an EJB interface, employing an ORM layer, with standard Java communication between client and server, and good horizontal and vertical layering, while adding functionality for continued evolution of the product.

To me, the key to successfully making these changes was symmetry. And the reason why is because symmetry essentially boils down to the invariance of some observable for a given type of transformation to a system.

The simple examples of symmetry are those of geometry. A circle is fully rotationally symmetric because no matter how many degrees you rotate it, its appearance remains the same. A square has a rotational symmetry of the 4th order, because you can rotate it 90 degrees and its appearance remains the same.

And this has to do with code refactoring? Sure. The idea was essentially to map out a series of transformations in the code, with a plan for what behavior of the system was going to be preserved for each particular change. This allowed the ability to test each change to make sure that it preserved the behaviors that it was meant to.

I saw each coding change as rotating the square - and the goal was to rotate it the full 90 degrees so that it still looked like a square. Even if what happened behind the scenes involved major structural change.

It's certainly possible (who knows, maybe advisable) not to think of this in terms of symmetry, but that's just how I do it. The key is really to try not to attack everything at once. But instead, to split the changes you're going to make into changes where each can be tested to not have broken the behaviors that it shouldn't have affected.

For me, thinking about it in terms of symmetry provides discipline in terms of defining how much change should be taken on at any given time. But your mileage may vary.

Small suggestion on Python nested code blocks

I've been coding in Python for a while now, and there's one thing that I've found problematic with regard to nested code blocks. It comes down to the age-old Python debate about whether you like Python's "indentation level" approach to determining what block of code a statement belongs to.

In general I actually like the Python approach. For one thing it removes the debates (going back to at least C) when developing in most languages around what style to use for braces. Especially for people who don't realize that K&R is the way to go.

The problem I have is when I have multiple levels of nesting that ends up being like an overhanging cliff that you can fall off of if you're not careful.

if one_thing:
    do_something()
    if another thing:
        do_another_something()
        if third_thing():
            do_yet_another_thing()
    do_thing_after_cliff()

The two problems I have are that it is harder to keep track of what level of nesting a statement is in for some cases. The bigger problem is if the statement do_thing_after_cliff() accidentally gets modified to be at another indentation level it can be very non-obvious. This can come up when moving chunks of code from one indentation level to another.

I've started using a simple trick to help with this, and for me it really helps visually. Basically I use a double comment "##" where I would normally put the "}" if I was working in another language.

For me this provides the visual cue that helps. And it also should make it obvious enough if statements accidentally get modified to the wrong indentation level.

if one_thing:
    do_something()
    if another thing:
        do_another_something()
        if third_thing():
            do_yet_another_thing()
        ##
    ##
    do_thing_after_cliff()
##

I don't use it absolutely everywhere, but it helps in the cases where your eye doesn't immediately feel comfortable with how blocks might line up.

First startup nostalgia

Going through the bootstrap process on our company has brought back memories of starting day at my first startup.

It was back in '96. Not everyone was doing or had been in a startup before so it felt really new. I remember being surprised that we wouldn't be working in a garage or a basement. And that there'd be a 401(k) and health plan. I thought it was supposed to be tough?

Though I signed on before the Series A closed I wasn't to start until it did close. That felt like a looooonnngg wait. Probably about 4 weeks in reality.

The great memory that I have of that time was reporting for the first day of work, which was the first day there was office space. Walked into the office space, and there was - nothing. No cubicles. No chairs. No desks. No computers. A big empty room. (Cool! This WAS going to be a startup after all!)

Our VP of engineering brought in donuts that morning along with some chairs and a whiteboard and we were off and running... By the end of the first week we had cubes, desks, chairs, and some plans.

There's nothing like the feeling of starting from a blank slate. I'm on the 4th time with that feeling and it never gets old. There's also an excitement to getting by with less while you help get everything in place. In my second startup my personal laptop running Windows 98 served as our corporate Internet gateway using dialup while we waited for our T1 install!

The first three times there were network devices to install, desks to help put together, and of course fundamental early decisions to make sitting or standing around a whiteboard.

This time that's all replaced pretty much with services to provision in the cloud. No office space, so no office network. Email and collaboration via Google. Product deployment on servers in far off data centers.

One thing remains the same though. That great feeling of the fresh start. Balancing priorities and trade-offs and trying to maximize the value of what you're going to build.

Very addictive - I highly recommend it.

Unintentional Google Voice Humor

Been using Google Voice for a couple months and it can be somewhat amusing when the voice recognition for the voicemail transcription doesn't quite pick it up right. Had a good one yesterday though.

"If you i just wondering if you could give me a favor. When you come back or when you're on your way back. Can you pick me up like I D cap promo much at all at Starbucks just totally wiped out. I have no idea why I feel so it I think I've been drugs. But anyway, I, I'm gonna trying to the blood down for a nap this morning and I think, up opportunity. Thank you. I mean to you later. Bye"

As a friend pointed out - notice that it didn't have a problem picking out "Starbucks". No surprise, since Google's all about the advertising!

The full corrected message isn't worth it, but for the record, "I D cap promo much at all" is actually "a decaf Carmel Machiato".

The best message transcription fail might be this next one. Although in this case it isn't to be blamed because it was my wife and 17 month old daughter speaking in Portuguese.

"I saw what at the cottage for the dangers visa. Thank you. Bye bye these pages knows. Bye bye. Tom O these Aunt Karen these whoa whoa. Hey, but cos he meant 9 bye bye delivering basis hello."

If the system were a human it would have been pulling its hair out on that one!

"Statistics is the new grammar" - Wired

This article in Wired this month is right along the lines I've been thinking for a while now. Our current education priorities so poorly set our kids up to deal with the world of today.

I truly think it's not useful to have opinions on a large range of topics without understanding how statistics and probability factor in.

If we're clueless about correlation vs. causation, anecdotal vs. statistical, how can we make sense of the world?

This dovetails with the thoughts I put down last month on the Tyranny of Intuition. If we can't make good educated judgments on something, then we're left with making our best uneducated judgments instead. Which I think is setting our selves up for an epic fail.

Serious Uptime

I caught this article (Humming away since 1993) today and it definitely caught my eye and brought back fond memories of the start of my career.

It's about a server shipped by Stratus Computer that has been up and running since 1993. I consider my career as having started at Stratus (first as a Co-op then part-time through the rest of college), and it was in fact 1993. So this computer shipped from the company I first started working for around the time I started working and has been running my entire career so far. Crazy!

The money quote: "Around Y2K we thought it might be time to update the hardware, but we just didn’t get around to it."

You usually think in terms of whether something you worked on might still have the code out there running somewhere (I'm sure I have code dating back to 1996 running on Nortel Contivity switches somewhere out there, and code dating back to 2000 running on CIENA switches). But to think about an actual instance of hardware up and running nonstop for that long just kicks it up to a whole new level.

This made me reflect back on my time at Stratus. It was a great place to start, and back in another era. It was before any real open-source projects, and before you could go online and get answers around your programming problems almost instantly. Everything was in your head, in-house or in a book on your shelf, and all the expertise needed to be inside the company.

From a technology point of view it was great experience. It not only helped ingrain *how* to think about high availability and fault tolerance, but also that it *should* be thought about in the first place. The best lesson is probably that it forces you to think at a full-system level. Everything was redundant in the hardware - power supplies, memories, CPUs, backplane, boards. Anything could fail and be removed and replaced without the system missing a beat.

Now, this is super expensive of course. And around that time (1993-94) Stratus itself was moving away from mainframes with fault-tolerance to high availability clustering approaches. But still - it's cool as hell that there are kids out there driving cars around that were born after this thing first booted.

I worked in the HAL (High Availability LAN) group, mostly around the development of FDDI - itself a fault tolerant networking technology.

I got to work at the application level, in kernel code, and especially enjoyed in an embedded environment - the firmware running on the FDDI board itself.

It was a great kick-start to my career because Stratus had layoffs followed by attrition, which led to my being the sole software engineer running the FDDI project for a good chunk of time at about the time that my Coop stint was over (after which I was a part-time software engineer). Even better, the hardware engineers that had started the project also had left the company. There was great fun to be made of the fact that the project was being led forward by a Coop and a couple lab techs.

I suppose if it had been 2003, then the cool thing would be to drop out of college and start my own company. But in 1993, having responsibility of a full project within a large company was good enough! I loved having a challenge to rise to and doing it.

In the end I can't believe the article didn't say whether this sever is running VOS or FTX though. I bet VOS.

A Strange Anniversary

A couple days ago passed the two year anniversary of my last paid workday. Well, hopefully not last *ever*. I joked with my wife that we'd celebrate by not going out to dinner.

I'm lucky enough that this was voluntary. It has resulted in the best two years of my life, without a doubt.

The plan at the time was to find an idea that would become a company that I would start. At that point I thought it would actually be one of the ideas that I had at the time. I had only worked for startups, 3 of them, since 1996 - the year after I graduated college.

I was looking to get out of networking and telecommunications specialties and work in the more general consumer internet space.

We found out my wife was pregnant before I made the plunge, but that timing was really perfect. It allowed me to be working on my own projects at home through the pregnancy and has allowed me to be working from home through the entirety of my daughter's life so far (17 months).

I worked on a few of my own projects independently, and worried my wife a little bit when I'd move from one to the next - just when she was getting used to the idea that the one I had been working on was going to be "the one".

The whole time I figured the worst case was I was acquiring experience in *lots* of new technologies that I'd put to good use at some point. And as it turned out, that was the perfect warm up for what came next.

Last fall I got introduced to someone looking for a technical cofounder, and the result is now Yieldbot.

Which is a great way to mark the two-year point. We've launched recently, in private Beta, and learning a ton from our first customer experiences, and being pushed by customer demand. Which is good, as this should mean not too much longer before my little family can spend some money again.

It's been a heckuva two years - wouldn't change a thing.