tag:blogger.com,1999:blog-13463674267661068362024-03-13T12:03:51.504-04:00shearicMakin Stuffshearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.comBlogger45125tag:blogger.com,1999:blog-1346367426766106836.post-62229302498945892402015-10-28T01:23:00.002-04:002015-10-28T01:23:54.704-04:002008 CrazinessI was reading <a href="http://www.businessinsider.com/remembering-financial-crisis-tweets-2015-10" target="_blank">this article</a> tonight about a tweetstorm from <a href="http://twitter.com/pmarca" target="_blank">@pmarca</a> about the financial crisis and it sparked a couple thoughts about that time for me.<br />
<br />
First, it's hard to believe I'm at an age where something that seems so recent to me would be relative pre-history for someone of the age that could be a co-worker. But do the math and there you go.<br />
<br />
Second, I was thinking of how crazy that time was and how crazy I was on top of it.<br />
<br />
I was 6 months into a voluntary break from paid work exploring my own ideas/projects that I hoped would lead to my starting my own company. The financial system was collapsing. My wife was 7 months pregnant. I was actually feeling hopeful in that I thought they would let more fail than they did. I was still a little more than a year away from hooking up with <a href="http://twitter.com/jonathanmendez" target="_blank">@jonathanmendez</a> to start <a href="https://www.yieldbot.com/" target="_blank">Yieldbot</a> and more than 2 years away from that bringing my next paycheck. And on the biggest down day of the market I cobbled together what I could to make some strategic stock buys. (For better or worse) I haven't done any stock trading since.<br />
<br />
Reflecting on all of that, I suppose it's no wonder I love the startup world; wherein you get that level of crazy on an ongoing basis.<br />
<br />shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-51684020473691486882015-10-06T21:24:00.000-04:002015-10-06T21:24:58.045-04:00The Song Remains the SameI've generally been way too busy to lift my head up and do any writing, which is probably something that won't change now. But I was amazed that my last post here was more than 4 years ago and I figured I could even say something random just so that wouldn't be true anymore.<br />
<br />
In that time amazing things have happened at Yieldbot, including recently changing to needing 3 digits to count the number of employees in the company. More importantly than the number of people is the quality of the people, the amazing way they work together, and the great product that we've been able to build together.<br />
<br />
A couple key things that have stayed the same over the last 4 years -<br />
<br />
Yieldbot's mission. Everything feels like it's been an evolution, and my (admittedly brief) description of what we do doesn't need updating. This is a tribute to the original vision that <a href="http://twitter.com/jonathanmendez" target="_blank">@jonathanmendez</a> had for what we do and how he's guided the ship the whole way.<br />
<br />
Happily for me another thing that hasn't changed is I'm still the CTO; though the evolution of my role and what I do on a daily basis as we've scaled the company is hard to even believe. Maybe I'll talk more about it when I find the time.<br />
<br />shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-69462905106011395512011-08-17T01:21:00.004-04:002011-08-17T02:05:02.782-04:00UghSQLI spent a good deal of time wrestling with getting MySQL installed on my new MacBook Air (OSX Lion), and in the name of saving someone else from repeating that amount of frustration I want to share the insights gained.
<br />
<br />The problem I was having was that the install all went fine (using Macports) but the post-install setup of the database, using mysql_install_db, was failing. When it seemed to succeed, it left me with an empty users table and inability to gain access (or set the password for the root mysql user).
<br />
<br />I found this: <a href="https://trac.macports.org/ticket/28772">https://trac.macports.org/ticket/28772</a> which perfectly described the situation I was in.
<br />
<br />In the comments of that I got to this <a href="http://stackoverflow.com/questions/4798464/installing-mysql-on-snow-leopard-using-macports/4907071#4907071">stackoverflow answer</a> which was even better.
<br />
<br />The first problem I had, for whatever reason, was related to permissions, and that got me past that.
<br />
<br />However, it continued to fail.
<br />
<br />If you're currently in that situation, I feel your pain. And here is the nugget of info you might need.
<br />
<br />The mysql_install_db script attempts to not overwrite tables that already exist when it is called. I believe the mechanism used to do this is a little too coarse. The scripts use the @@warning_count variable to try and detect if a table already exists, setting other variables, like in this case I hit @had_user_table.
<br />
<br />After having done an uninstall/install to get me back to clean, I ran the mysql_install_db script again, but also got warnings. It probably doesn't matter, but I specifically got this:
<br /><code>
<br />110816 23:42:06 [Warning] '--skip-locking' is deprecated and will be removed in a future release. Please use '--skip-external-locking' instead.
<br /></code>
<br />It may have been these, or it may have been some other warnings that were silent, but I think the (empty) user table got created but the initial entries for the root user didn't get populated. Once the empty table exists, subsequent executions of mysql_install_db then avoid writing new entries into the table. The net result being you never get the initial entries for the root user that you need.
<br />
<br />The simple fix to this was to modify the mysql_system_data_tables.sql file (which for me was in the /opt/local/share/mysql5/mysql/ directory) to change:
<br /><code>
<br />INSERT INTO user SELECT * FROM tmp_user WHERE @had_user_table=0;
<br /></code>
<br />to be:
<br /><code>
<br />INSERT INTO user SELECT * FROM tmp_user;
<br /></code>
<br />This is basically setting it up to ignore the fact that it encountered warnings the table existed and instead you're telling it that you know better.
<br />If you're going to do this, you should make sure your table really is empty. Run the server with the --skip-grant-tables option and run mysql and verify that the following:
<br /><code>
<br />SELECT * FROM mysql.user;
<br /></code>
<br />returns no results.
<br />
<br />That we use MySQL for such a very small part of our platform at <a href="http://www.yieldbot.com">Yieldbot</a> made it that much more painful. That it kept me from what was going to be my focus for the night, working with HBase, was all the more ironic.
<br />
<br />Happy troubleshooting!
<br />shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-74781565996532314662011-07-13T01:15:00.007-04:002011-07-14T02:47:09.809-04:00Are Amazon Spots Struggling?<b>[Update - heard directly from AWS and they report that the graph issue is a known bug and unrelated to the problem we were seeing getting m2.2xlarge instances. Hopeful we'll get to the bottom of the underlying problem]</b><br /><br /><b>[Update #2 - once we shared the info around time and type, AMZN was able to verify that there was a temporary issue involving only the m2.2xlarge, only in our region, and right in the window we happened to want them. Bit unlucky, but we should be all good now :-) ]</b><br /><br />This seems pretty coincidental with the AWS announcement yesterday with per-AZ priced spot instances. At yieldbot we do some heavy lifting with clusters of m2.2xlarge servers, and haven't been able to get any spot instances for hours. Luckily we fall back to reserved instances if we have to, we'd just rather not pay the higher price. :-)<br /><br />Here's a snapshot of the price history taken a few minutes ago for the m1.xlarge instances. Note that us-east-1b and us-east-1d seem to have stopped a while ago. us-east-1a and us-east-1c goes close to current, and our testing verified we could get us-east-1c instances.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyFyKlqR6nbluDa2eTQSw-GstNRBE_5A3h0iedy2UjibbHvVzY-hvm1H9rhM46YI4yegDHUz1nBHqo4b35t2yLuiwZp8TctjP4AM6IHa52ucCIS98F54fv0fpGrKWDEPyHPXy4PTSNqw7g/s1600/aws-m1xlarge-graph.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 143px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyFyKlqR6nbluDa2eTQSw-GstNRBE_5A3h0iedy2UjibbHvVzY-hvm1H9rhM46YI4yegDHUz1nBHqo4b35t2yLuiwZp8TctjP4AM6IHa52ucCIS98F54fv0fpGrKWDEPyHPXy4PTSNqw7g/s320/aws-m1xlarge-graph.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5628702158354177186" /></a><br /><br />Now check out the graph for the m2.2xlarge instances, taken about the same time. Availability seems to have fallen off a cliff hours ago, particularly in us-east-1c which appears to have not had availability for this instance size on the spot market for about 12 hours now.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigKA-DVoFkQo2rucg5CqgrC5mBU4gutA8d32LZpDTm2HJwgQ9hpogZYR0AQhoQS47_k8_XV0x2C0bpu6T-mzsYH5roN_uKA9PlBWCpDtjAgAGuf007JJ5JaJi_TcAFv4CPoUz48prHe-Ge/s1600/aws-m2.2xlarge-graph.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 142px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigKA-DVoFkQo2rucg5CqgrC5mBU4gutA8d32LZpDTm2HJwgQ9hpogZYR0AQhoQS47_k8_XV0x2C0bpu6T-mzsYH5roN_uKA9PlBWCpDtjAgAGuf007JJ5JaJi_TcAFv4CPoUz48prHe-Ge/s320/aws-m2.2xlarge-graph.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5628702541388055810" /></a><br /><br />The AWS health dashboard currently shows all green, which at some level is understandable. Spot instances are, after all, an as-available resource. The interesting thing here though is that the price shows no spikes, no price movement indicating an approaching shortage. The market didn't get expensive, there just stopped being a market.<br /><br />Maybe AWS suddenly needed to add some instances to the pool of available reserved instances? Happily, reserved instances did come up and with no delays.<br /><br />But we can't wait to start saving money with the spot instances again.<br /><br />Anyone out there seeing the same or similar problem?shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-11294844498808540322011-04-22T17:11:00.004-04:002011-04-22T17:46:27.479-04:00Amazon Saved Us a Bunch of Disk SpaceSomeone should start a webpage where people can post the beneficial things that happened to them as a result of the Amazon "cloudocalypse", just for the contrarians out there. Here's our entry.<br /><br />We use MongoDB for a good piece of our platform, and have been meaning to swap in a backup of the database as the primary.<br /><br />Because we had created some large collections a few months back that were subsequently dropped (for which MongoDB doesn't reclaim disk space) we found ourselves in a situation of pretty inefficient disk utilization. We'd recently bumped up to 90% on the database disk volume.<br /><br />At the same time the backup with the same size volume was running at about 40%.<br /><br />It was on our list to swap the backup in as the primary, but we hadn't gotten to it yet. Yesterday Amazon decided for us that it was time. And now we have all this free disk space in our database volume.<br /><br />Thanks Amazon! :^)shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-53506797207658185252010-10-11T19:50:00.003-04:002010-10-11T19:57:17.520-04:00team++ ftw!Today's big news out of the Yieldbot offices is the addition of Soren Macbeth (<a href="http://twitter.com/sorenmacbeth">@sorenmacbeth</a>) to the team. Ok, so we don't have offices. If we did though...<br /><br />Taking a step from being 2 people to being 3 people is huge. Taking that step like we just did, priceless. And now I get to work with <a href="http://twitter.com/jonathanmendez">@jonathanmendez</a> *and* <a href="http://twitter.com/sorenmacbeth">@sorenmacbeth</a> every day. Woot!<br /><br />In my startup experiences the early DNA of the team is the difference. Which is why having Soren join us just has "win" written all over it. From our first conversation it was clear our development philosophies were in alignment. I can't wait for Soren's mark to be made on the code base and on the product. Exciting times ahead!shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-69942403552705407552010-09-23T01:37:00.002-04:002010-09-23T01:40:01.554-04:00A Small Thought on Meaning, Humans, and MachinesI was attending <a href="http://www.10gen.com/conferences/mongoboston2010">Mongo Boston</a> this week when a simple typographical oddity sent me on an interesting (and luckily brief) train of thought.<br /><br />The simple occurence of "mb" on a slide to indiciate "megabytes" (it's in <a href="http://www.slideshare.net/eliothorowitz/mongodb-sharding-mongoboston-2010">this presentation</a>). First amusement because generally 'm' is milli and 'b' is bits. Of course expressing something in millibits is generally useless. So of course a human reading it (unless you're weird like me) doesn't give it a second thought.<br /><br />Then I thought about the difference in how quickly I (and everyone else) make the right assumptions on what 'mb' in this context really is intended to mean.<br /><br />And therein lies the huge challenge on the machine side if you're doing data mining and analytics. How do you infer intention automatically and properly, especially when you don't know ahead of time what type of mistakes or inaccuracies are going to be involved? Just the type of thing I've been having fun figuring out over the past year - at least for a particular problem space.<br /><br />But something else interesting crystallized for me too at a human level. That this auto-correction that we all do so that we don't sit there confused by how one would break something into 200 millibit chunks is also the cause of many of our problems.<br /><br />We need to be able to jump to conclusions in order to get past "mb" without having to ask a presenter for clarification, but we also are prone to jump to conclusions when we shouldn't - particularly in judging attributing some intention to someone else's action.<br /><br />Two side of the same coin, one useful and the other problematic. Mostly we can't have one without the other though. Although I guess with practice we hopefully get better at figuring out when to jump and when not to jump. Though I think way too many people never figure out when not to jump.<br /><br />If there were any typos above I hope your brain autocorrected them for you so you didn't notice.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-29749347594108752812010-09-08T00:54:00.002-04:002010-09-08T01:11:09.071-04:00The Best Wisdom is PortableBy which I mean that it can be put in multiple contexts and keep its status as wisdom.<br /><br />I'm reminded of this over and over reading Seth Godin's book "Linchpin". I was expecting it would be a good book, but only a few pages in I was as excited about reading the book as maybe I've ever been starting off on a book. It just resonated so well with me, gave better voice than I could to vague thoughts that I've had, and went in brilliant directions I would have never thought of.<br /><br />I'm now most of the way through, and I look back and it seems like almost at every page I wanted to stop and jot down at least one quote. Though I haven't (until now) because I couldn't bring myself to stop the flow of reading it.<br /><br />The highly portable quote I hit today:<br /><blockquote><br />"Great bosses and world-class organizations hire motivated people, set high expectations, and give their people room to become remarkable."<br /></blockquote><br />This is one of the thoughts that so obviously resonates with me right now as we're thinking about putting together the team for our startup to take it to the next level.<br /><br />But I feel like its true wisdom is that it resonates just as strongly in the completely different context of being a parent. It just strikes a chord with how I think about how we want to approach raising our almost-2-year-old daughter.<br /><br />All children are motivated learners; are naturally curious. So you get that for free. Setting high expectations and giving her room to become remarkable is where the magic can happen. I get the impression that too many parents forgot that the last 20 years, and I hope that it's changing now.<br /><br />This isn't the first case that this portability of wisdom in Linchpin hit me, and I think that is the real genius of the book.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-19512084810389828202010-09-01T10:44:00.002-04:002010-09-01T10:49:01.174-04:00Mosques, Corporations, and the First AmendmentEarlier this year I agreed with the Supreme Court majority decision on Citizens United that on First Amendment grounds Corporations and Unions could not be limited in their funding of campaigns.<br /><br />The common argument made, which I disagreed with, was the simple one that the ruling made was that "corporations are people" and that this was ridiculous. And that the First Amendment should only, and was only meant to, apply to individuals.<br /><br />I felt that the inclusion of "the press" in the original document indicates that these Rights, while grounded in the rights of the individual, did actually extend to the aggregations of individuals as well.<br /><br />Fast forward to today and we have the "ground zero mosque" issue. Here as well I feel that under the First Amendment, this *organization* has the right to build their Community Center/Mosque wherever they see fit, and many are arguing the same. And it isn't because I/we think "churches are people" (or that "congregations are people"). Note, no single person is building or funding the community center. The first amendment here is getting invoked properly as applied to this group of people, and this should kill the "corporations aren't people" line of attack on the Supreme Court decision on Citizens United.<br /><br />To put it in perspective, imagine the precedent had the Supreme Court ruled the opposite way on Citizens United. The decision would have been that the government can severely restrict the rights of aggregations of people, so long as the rights of the individuals are preserved in some more narrow sense.<br /><br />Using this logic you could then say that the government could have the right to put restrictions on where a mosque get built, perhaps "within reason" or some vague qualifier. For instance, it would be easy to argue using this logic that the government could require the mosque be built 10 blocks away from the proposed location, as there would be little argument that it would put the individual members in a position of not being able to enjoy "free exercise" of their religion - it would be just a little extra commute (for some of them) after all. If you disagree with that, imagine it again but 1 or 2 blocks instead of 10.<br /><br />Now one argument would be that churches exist for the express purpose of the exercise of religion and so it is more obvious that their rights more directly derive from the rights of the individual members. But then in the Citizens United case the Corporation in question was one that was created for the express purpose of putting out a political message. So its rights too were very directly derivative of the rights of the individuals that funded it.<br /><br />Anyway, this is really a complex issue, much more so than "the Supreme Court thinks Corporations are people" line of reasoning.<br /><br />Personally, I definitely do think that there is an overall problem of money and politics mixing. As I <a href="http://shearic.blogspot.com/2010/08/decentralized-beats-centralized.html">wrote about recently</a> I think that transparency is a great part of the solution, as you saw in that recent case of Target dipping its toe into political contributions and the public backlash.<br /><br />If we don't have full transparency yet on the money trail in politics we absolutely should have every measure in place to be fully transparent. The power of transparency in this age of information technology I think should not be underestimated.<br /><br />I also think that it would be healthy to explore options to amend the Constitution to be more specific about rules around elections in particular since they are an aspect of the direct running of government. The bars are somewhat high to get that started, and they should be. But that mechanism is there for us to make changes that we deem important enough (but as Prohibition showed, we can certainly be temporarily pretty stupid about things).<br /><br />But as far as the first amendment goes, I'm happy to have it consistently apply to individuals and to the organizations that individuals choose to create and belong to.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-35101856999097894342010-08-25T14:38:00.002-04:002010-08-25T14:47:33.234-04:00Go Into (Technical) Debt With Eyes Wide Open!This morning Bijan Sabet had a <a href="http://bijansabet.com/post/1008248039/technical-debt">good blog post</a> on being open and honest about "technical debt".<br /><br />I think (and commented there) that it's important to get into such debt with your eyes open and plans for how to address it in the future.<br /><br />If the context isn't clear, what's being talked about here is basically hastily put together technology that accomplishes an early version of what was needed of a product but is considered "debt" because it cannot remain in the state it is in. It is carried as a kind of liability because at some point, either because of new features that are required or it isn't scalable, time and effort will need to be *paid* in order to make it more robust.<br /><br />Having gone through the process of building multiple software systems from the ground up has clarified for me how technical debt should be planned for. And nothing clarified it for me greater than working on a large project with a hefty amount of inherited technical debt that we successfully cleared off the books, and the monumental effort that it took to do so.<br /><br />I find it important as you are putting a system together to consciously make the tradeoffs that are necessary and have a plan for how the code (and technology choices) would be iterated and scaled up when the needs arise.<br /><br />As in monetary debt, this is akin to advising not to take a loan without having a plan for how you're going to pay it back.<br /><br />Along the way there are also very smart choices you can make to help make that debt less too. Layering and using adaptors being one great example. It can be expensive to retrofit an adaptor model to something that was written assuming it would only have to interact with one particular technology. So abstracting early, even if you are in a hurry, will help you with your later "payment plan" at a fairly low cost up-front.<br /><br />Another good example is having an MVC model on your UI, which will help later when you need to put an API in place (and reuse the "M" and the "C").<br /><br />Something to watch out for is piling debt on top of debt (don't charge your mortgage to your credit card!). That is, when the feature comes up that needs to utilize another technology that is in need of repair, instead of making matters worse you should seriously consider at least starting to pay off the debt on the older technology.<br /><br />The important bottom line is take the debt seriously and don't enter into it lightly. Yes, it's necessary, but have a plan. Things will come up you didn't expect, and you'll suddenly find liabilities where you didn't know they existed. But then you'll be freer to deal with those unexpected things since you're in better shape overall. And your future self will likely thank you for it.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-87374372842540601002010-08-19T14:05:00.002-04:002010-08-19T14:11:20.044-04:00Decentralized Beats CentralizedThe current uproar around Target and its political contribution that ended up supporting a Republican candidate whose beliefs many disagree with is, I think, very instructive and holds lots of lessons. (CNN info about it <a href="http://politicalticker.blogs.cnn.com/2010/08/06/retailer-target-now-the-target-of-a-boycott-by-moveon-org-gays/">here</a>.)<br /><br />The biggest for me is that it shows how transparency and decentralized action can be a healthier political force in the country than centralized enforcement.<br /><br />Other than an unfortunate glossing over of some important nuance (more on that below), I think this is playing out exactly as I would have hoped after the Supreme Court ruling earlier this year that corporations could make political contributions.<br /><br />Even though I'm concerned with large corporations unduly influencing our government, I felt the Supreme Court ruling was correct. Previously, the regulations around what corporations could and couldn't do were too difficult to decipher. It's a difficult problem to solve by centralized fiat because there are many subjective dimensions to defining what might be allowed.<br /><br />But what is important is the requirement for transparency. This puts the information out there to be dealt with in a decentalized way, which is exactly what is happening now. And, of course, there's never been a better time to make information available which could lead to decentralized forces making their voices known.<br /><br />Basically the Supreme Court crowd-sourced the problem. They essentially said that instead of saying there will be a central definition of what is acceptable, we will require transparency and let the masses do the job of keeping corporations in check. And the masses, unbeholden to "Mainstream Media" and with the tools of social networks and the virality that comes with it, have never been better prepared to do this.<br /><br />The good lessons for corporations here will be that they can suffer reputational harm by making political donations, even those that might seem innocuous. This should serve as a brake on their making the contributions.<br /><br />In fact, this is where the lack of nuance comes in.<br /><br />The reality of the money flow is that Target contributed to a pro-Business group in Minnesota called "MN Forward" (<a href="http://www.mnforward.com">www.mnforward.com</a>). MN Forward then contributed to the ads for the politician in question. Their issues page lists: Tax Reform, Spending Reform, and Education Reform. Noble goals all.<br /><br />The bad part about this loss of nuance in the story generally is that it means that people are assuming that Target and its principals have an anti-gay agenda. An objective look suggests that they contributed to a pro-business organization, who in turn backed a pro-business candidate who also holds non-progressive social policy positions (specifically, against gay marriage).<br /><br />Now, the *good* part about this loss of nuance is that it will serve as a warning to other corporations. They will realize that their motivations for making a particular contribution won't matter as they can be undone by the perceptions about a candidate that might ultimately benefit from the contribution (even if at least once removed).<br /><br />Let's hope we all keep doing our job and using all of the information we can get our hands on to our advantage, and solving the problem of corporate meddling in politics by being more a part of the process that can solve it.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-29745670537235931042010-08-10T21:55:00.002-04:002010-08-10T22:00:07.186-04:00SE is to CS as ME is to PhysicsThe past week I have had thoughts rolling around spurred by <a href="http://thegongshow.tumblr.com/post/897992236/exploding-computer-science">this post</a> by Andrew Parker. The general theme of which is "how little a computer science education matches up to the real-world building of large scale applications." I thought it brought up a really good issue.<br /><br />For me the issue comes down to whether you see a CS program as vocational in nature. Back when I graduated (in '95) I thought then that the CS program at my school (WPI) was too vocational in nature.<br /><br />At WPI you do a major project your senior year as part of your graduation requirement. I've always leaned to the theoretical and mathematical side, and for my Physics degree my project was in the area of Quantum Mechanical energy degeneracies in multiple dimensions. No, I don't remember the equations in detail. But it landed me a published paper in the American Journal of Physics, so that was cool.<br /><br />When it came time to do the major project for my CS degree I was the only one in my class not to work on some type of coding project. Instead my project was "Complexity Analysis of the Primitive Recursive Functions." I got all the coding experience I would need being in a Coop program and then staying on to work part-time after that, writing embedded firmware code in Assembly and C and Unix (SVR4) drivers and application code in C. And I think that's the right mix.<br /><br />(And yeah, I know, with degrees in those two fields you'd think I would've leaned to Quantum Computation or something, but I'm not *that* theoretically minded :-)<br /><br />It seems to me the argument is around the "S" in CS being "Science". There's a real difference between "Technology" and "Science" that we should recognize.<br /><br />This is obvious when you look at Physics (which I majored in) and Mechanical Engineering (which I also majored in briefly before switching that one to CS). In ME there is more a lean to technology. No technology in Physics, other than around the instruments for running experiments.<br /><br />I'd argue that we need just the same type of split (with cross-over training of course) that exists in Physics and ME in the field of computation.<br /><br />I think there's already been one split that's happened that makes sense, which is in the field of Electrical Engineering with a focus on computers and lower-level programming - I think I've seen this referred to as ECE.<br /><br />If we then split off "Software Engineering" as a full-fledged major, that feels like: SE is to CS as ME is to Physics.<br /><br />Let's be honest, there'd be a *ton* more SE than CS majors, but who cares? At WPI when I was there we probably had at least a 10:1 ratio of ME to Physics graduates. Just like ME majors have to start with some Physics courses, SE majors would have to fulfill the basic CS courses.<br /><br />The upside is that the SE majors would have a clear focus and would probably come out with much stronger industry skills that would enable them to start building things right away. Again, pretty close to the analogy to ME and Physics.<br /><br />I'm sure there must be places where this split of SE off of CS is already happening. If that is the case, it should just be happening even faster :-).<br /><br />I was also motivated to write this by a couple mathematically-oriented CS stories that came out in the past week. The first being around the attempted P!=NP proof, which brought back memories to my complexity theory studies. For the latest on that, <a href="http://rjlipton.wordpress.com/2010/08/09/issues-in-the-proof-that-p%E2%89%A0np/">this blog</a> is following it.<br /><br />The other story that caught my eye was around a proof put forward that any position of the Rubik's Cube can be solved in 20 moves or less (<a href="http://www.cube20.org/">http://www.cube20.org/</a>).<br /><br />They served as reminders to me that a CS education is much more than learning about pointers, or whether you can create anonymous classes in Java easily, or how to work with a source control system, or how to provision and manage a Hadoop cluster.<br /><br />Honestly, compared to when I graduated (it wasn't *that* long ago was it?) all of that can be learned and played with online. With guides, and open source projects looking for you to contribute. And, as a friend reminded me today, things like Google's Summer of Code to really go head down into implementation.<br /><br />I'm sure there's never been a better time to learn about and become a great software developer. And really I think there's more and more proof that on the "technology" side (as opposed to "science" side) it doesn't even take going to school anymore.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-68725661261009427062010-08-03T01:46:00.002-04:002010-08-03T02:18:14.082-04:00Memory LaneHere's to the launch of the TRS-80 back on 8/3/1977 (Wired article: <a href="http://goo.gl/njnJ">http://goo.gl/njnJ</a>). I remember the model well because I had a "pre-owned" one back in the early 80's.<br /><br />At the time it was an awesome upgrade from my first computer, the Timex Sinclair 1000 (although I had the 16K expansion unit for $50 for that baby!)<br /><br />The funny thing about the TRS-80 is that I liked it because it looked like a "real" computer. More like what you could see in the movies. Given everything started with "War Games" for me (like everyone else at the time?) I think it makes sense.<br /><br />I regret that I have didn't have an Apple computer back in the 80's. But I did follow an eclectic path from the TS/1000, to TRS-80, Coleco Adam, Leading Edge IBM clone, and then finally 386, and eventually Pentium.<br /><br />At the same time my online experience started in about 1986 on a teletype we had at school with an acoustic-coupled modem dialed into a small PDP-11 network. High School brought with it a VAX cluster, college was networked Sun workstations (Gopher was all the rage!), and finally the Internet.<br /><br />Some signposts I can measure my life by.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-74889603477671098212010-07-13T11:28:00.004-04:002010-07-13T11:43:19.244-04:00Bootstrapped Travellin'I'm getting ready for a trip to NYC "on a bootstrap" (I'm thinking this makes more sense than "on a shoestring" in this case, given we're still "bootstrapping" our startup).<br /><br />Circumstances have me dropping my mother off at Logan airport during rush hour, with a trip to NYC planned for the next day. Having friends that live in Revere (very close to Logan and Boston) and NYC is making for an interesting itinerary.<br /><br />It won't be "planes, trains, and automobiles" but "trains, buses, and automobiles" (and "air mattresses"). Well, "planes" too if you count dropping my mother off at the airport...<br /><br />It's looking like:<br /><ul> <br /><li>Drive in to airport</li><br /><li>Spend the night near Boston, where car will be left</li><br /><li>Train (T) into Boston in the morning</li><br /><li>Bus from Boston to NYC</li><br /><li>Spend the night over another friend's place in NYC</li><br /><li>Bus from NYC to Boston</li><br /><li>Train (T) back to friend #1's house</li><br /><li>Drive home!</li><br /></ul><br />You can't book that on Kayak.com!<br /><br />Thank goodness for friends, the T, and Boltbus.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-89309115471648314222010-06-25T14:37:00.002-04:002010-06-25T14:51:31.692-04:00Idea vs. ExecutionLast year I was on a flight and the in-flight entertainment switched after the movie to some other types of shows. One of them was around interesting engineering feats and had a feature on a submersible car. It could basically drive into water and then navigate as a submarine (shallow depth naturally) under the water.<br /><br />A few rows ahead of us I heard a woman turn to her friend and exclaim "hey, I thought of that!"<br /><br />This betrays, I think, a common propensity to value an idea over execution. Or at least to underestimate the challenges in the execution. Imagine all of the myriad engineering challenges that had to be overcome in order to execute on this idea of a submersible car. And then someone tries to sum it up with "I thought of that too".<br /><br />For me this story embodies the increasingly common maxim for the entrepreneur that it is the execution that matters. To share your idea. Get feedback. That everyone else has their own ideas. And if somebody pulls off something that you had thought of but didn't execute on - be impressed with them, not with yourself for having the idea too.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-30120107323295095472010-06-11T20:13:00.002-04:002010-06-11T20:23:33.036-04:00Symmetry and Code Refactoring<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8_6cKJYhP0hEWQRzGDsGYxNrWp4J3sxzURp4CsPtQRrApJYcVwfYj-GeCgpWxGKjV7F6KU0lUWgBSe83edEE1WEIGFKpbwKrYfY4P93OYwc8z4IBnPble9OTUlIa_rvR41CEABDqcPikW/s1600/200px-Da_Vinci_Vitruve_Luc_Viatour.jpg"><img style="float:right; margin:0 0 10px 10px;cursor:pointer; cursor:hand;width: 200px; height: 272px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8_6cKJYhP0hEWQRzGDsGYxNrWp4J3sxzURp4CsPtQRrApJYcVwfYj-GeCgpWxGKjV7F6KU0lUWgBSe83edEE1WEIGFKpbwKrYfY4P93OYwc8z4IBnPble9OTUlIa_rvR41CEABDqcPikW/s320/200px-Da_Vinci_Vitruve_Luc_Viatour.jpg" border="0" alt=""id="BLOGGER_PHOTO_ID_5481676071025896402" /></a>Whenever I make large refactoring changes within a project I think about it in terms of symmetries. I'm not sure anyone else could find that useful, but it's worth a shot.<br /><br />While getting my Physics undergrad degree there were two things that really held my fascination, quantum physics in general and the role of symmetry in natural laws.<br /><br />In my opinion the concept of symmetries in nature as the basis of conservation laws was one of the most elegant findings in Physics. We owe <a href="http://en.wikipedia.org/wiki/Noether's_theorem">Emmy Noether</a> for that almost 100 years ago (between this, Relativity, and Quantum Mechanics, the early 20th century in Physics was just a ridiculously productive period of progress in scientific understanding).<br /><br />Since then, I've found it interesting, and sometimes useful, to think about things in terms of symmetries. For instance, I think a fully rational ethical philosophy can be built on top of expressing our existence in terms of symmetries (the ideas behind "walking 1000 miles in someone else's shoes" or "The Golden Rule" I see as rooted in this).<br /><br />But back to coding...<br /><br />I've sometimes found myself needing to make major changes to a complex system. This can always, as they say, be "fraught with peril." The first (and biggest) of these started with a monolithic Java app that wasn't built on a framework, represented objects with maps, had manual construction of SQL for database access, used CORBA for Java client communication, had little vertical layering of levels of abstraction and little horizontal separations of logically separate spheres of influence.<br /><br />Certain types of changes to that type of system were just a nightmare to contend with. A particularly obvious example is how changes to the object model that affected the storage in the database might require changes in several seemingly unrelated places in the code where SQL strings were manually constructed.<br /><br />The team I led transformed the app (which we already started shipping) to be J2EE based, with an EJB interface, employing an ORM layer, with standard Java communication between client and server, and good horizontal and vertical layering, while adding functionality for continued evolution of the product.<br /><br />To me, the key to successfully making these changes was symmetry. And the reason why is because symmetry essentially boils down to the invariance of some observable for a given type of transformation to a system.<br /><br />The simple examples of symmetry are those of geometry. A circle is fully rotationally symmetric because no matter how many degrees you rotate it, its appearance remains the same. A square has a rotational symmetry of the 4th order, because you can rotate it 90 degrees and its appearance remains the same.<br /><br />And this has to do with code refactoring? Sure. The idea was essentially to map out a series of transformations in the code, with a plan for what behavior of the system was going to be preserved for each particular change. This allowed the ability to test each change to make sure that it preserved the behaviors that it was meant to.<br /><br />I saw each coding change as rotating the square - and the goal was to rotate it the full 90 degrees so that it still looked like a square. Even if what happened behind the scenes involved major structural change.<br /><br />It's certainly possible (who knows, maybe advisable) not to think of this in terms of symmetry, but that's just how I do it. The key is really to try not to attack everything at once. But instead, to split the changes you're going to make into changes where each can be tested to not have broken the behaviors that it shouldn't have affected.<br /><br />For me, thinking about it in terms of symmetry provides discipline in terms of defining how much change should be taken on at any given time. But your mileage may vary.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-65667032817817337742010-05-24T13:50:00.003-04:002010-05-24T14:18:46.221-04:00Small suggestion on Python nested code blocksI've been coding in Python for a while now, and there's one thing that I've found problematic with regard to nested code blocks. It comes down to the age-old Python debate about whether you like Python's "indentation level" approach to determining what block of code a statement belongs to.<br /><br />In general I actually like the Python approach. For one thing it removes the debates (going back to at least C) when developing in most languages around what style to use for braces. Especially for people who don't realize that <a href="http://en.wikipedia.org/wiki/Indent_style#K.26R_style">K&R</a> is the way to go.<br /><br />The problem I have is when I have multiple levels of nesting that ends up being like an overhanging cliff that you can fall off of if you're not careful.<br /><pre><br />if one_thing:<br /> do_something()<br /> if another thing:<br /> do_another_something()<br /> if third_thing():<br /> do_yet_another_thing()<br /> do_thing_after_cliff()<br /></pre><br />The two problems I have are that it is harder to keep track of what level of nesting a statement is in for some cases. The bigger problem is if the statement do_thing_after_cliff() accidentally gets modified to be at another indentation level it can be very non-obvious. This can come up when moving chunks of code from one indentation level to another.<br /><br />I've started using a simple trick to help with this, and for me it really helps visually. Basically I use a double comment "##" where I would normally put the "}" if I was working in another language.<br /><br />For me this provides the visual cue that helps. And it also should make it obvious enough if statements accidentally get modified to the wrong indentation level.<br /><pre><br />if one_thing:<br /> do_something()<br /> if another thing:<br /> do_another_something()<br /> if third_thing():<br /> do_yet_another_thing()<br /> ##<br /> ##<br /> do_thing_after_cliff()<br />##<br /></pre><br />I don't use it absolutely everywhere, but it helps in the cases where your eye doesn't immediately feel comfortable with how blocks might line up.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-41083098250062045992010-05-18T02:21:00.002-04:002010-05-18T10:15:35.284-04:00First startup nostalgiaGoing through the bootstrap process on our company has brought back memories of starting day at my first startup.<br /><br />It was back in '96. Not everyone was doing or had been in a startup before so it felt really new. I remember being surprised that we wouldn't be working in a garage or a basement. And that there'd be a 401(k) and health plan. I thought it was supposed to be tough?<br /><br />Though I signed on before the Series A closed I wasn't to start until it did close. That felt like a looooonnngg wait. Probably about 4 weeks in reality.<br /><br />The great memory that I have of that time was reporting for the first day of work, which was the first day there was office space. Walked into the office space, and there was - nothing. No cubicles. No chairs. No desks. No computers. A big empty room. (Cool! This WAS going to be a startup after all!)<br /><br />Our VP of engineering brought in donuts that morning along with some chairs and a whiteboard and we were off and running... By the end of the first week we had cubes, desks, chairs, and some plans.<br /><br />There's nothing like the feeling of starting from a blank slate. I'm on the 4th time with that feeling and it never gets old. There's also an excitement to getting by with less while you help get everything in place. In my second startup my personal laptop running Windows 98 served as our corporate Internet gateway using dialup while we waited for our T1 install!<br /><br />The first three times there were network devices to install, desks to help put together, and of course fundamental early decisions to make sitting or standing around a whiteboard.<br /><br />This time that's all replaced pretty much with services to provision in the cloud. No office space, so no office network. Email and collaboration via Google. Product deployment on servers in far off data centers.<br /><br />One thing remains the same though. That great feeling of the fresh start. Balancing priorities and trade-offs and trying to maximize the value of what you're going to build.<br /><br />Very addictive - I highly recommend it.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-58866950652867428532010-05-03T20:07:00.001-04:002010-05-03T20:08:52.425-04:00Unintentional Google Voice HumorBeen using Google Voice for a couple months and it can be somewhat amusing when the voice recognition for the voicemail transcription doesn't quite pick it up right. Had a good one yesterday though.<br /><br />"If you i just wondering if you could give me a favor. When you come back or when you're on your way back. Can you pick me up like I D cap promo much at all at Starbucks just totally wiped out. I have no idea why I feel so it I think I've been drugs. But anyway, I, I'm gonna trying to the blood down for a nap this morning and I think, up opportunity. Thank you. I mean to you later. Bye"<br /><br />As a friend pointed out - notice that it didn't have a problem picking out "Starbucks". No surprise, since Google's all about the advertising!<br /><br />The full corrected message isn't worth it, but for the record, "I D cap promo much at all" is actually "a decaf Carmel Machiato".<br /><br />The best message transcription fail might be this next one. Although in this case it isn't to be blamed because it was my wife and 17 month old daughter speaking in Portuguese.<br /><br />"I saw what at the cottage for the dangers visa. Thank you. Bye bye these pages knows. Bye bye. Tom O these Aunt Karen these whoa whoa. Hey, but cos he meant 9 bye bye delivering basis hello."<br /><br />If the system were a human it would have been pulling its hair out on that one!shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-9351803520273072002010-05-01T11:27:00.002-04:002010-05-01T11:35:01.333-04:00"Statistics is the new grammar" - Wired<a href="http://www.wired.com/magazine/2010/04/st_thompson_statistics/">This article</a> in Wired this month is right along the lines I've been thinking for a while now. Our current education priorities so poorly set our kids up to deal with the world of today.<br /><br />I truly think it's not useful to have opinions on a large range of topics without understanding how statistics and probability factor in.<br /><br />If we're clueless about correlation vs. causation, anecdotal vs. statistical, how can we make sense of the world?<br /><br />This dovetails with the thoughts I put down last month on the <a href="http://shearic.blogspot.com/2010/03/tyranny-of-intuition-part-one.html">Tyranny of Intuition</a>. If we can't make good educated judgments on something, then we're left with making our best uneducated judgments instead. Which I think is setting our selves up for an epic fail.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-48457657838852284202010-04-29T23:02:00.002-04:002010-04-29T23:37:57.408-04:00Serious UptimeI caught this article (<a href="http://www.boston.com/business/articles/2010/04/26/humming_away_since_1993/">Humming away since 1993</a>) today and it definitely caught my eye and brought back fond memories of the start of my career.<br /><br />It's about a server shipped by Stratus Computer that has been up and running since 1993. I consider my career as having started at Stratus (first as a Co-op then part-time through the rest of college), and it was in fact 1993. So this computer shipped from the company I first started working for around the time I started working and has been running my entire career so far. Crazy!<br /><br />The money quote: "Around Y2K we thought it might be time to update the hardware, but we just didn’t get around to it."<br /><br />You usually think in terms of whether something you worked on might still have the code out there running somewhere (I'm sure I have code dating back to 1996 running on Nortel Contivity switches somewhere out there, and code dating back to 2000 running on CIENA switches). But to think about an actual instance of hardware up and running nonstop for that long just kicks it up to a whole new level.<br /><br />This made me reflect back on my time at Stratus. It was a great place to start, and back in another era. It was before any real open-source projects, and before you could go online and get answers around your programming problems almost instantly. Everything was in your head, in-house or in a book on your shelf, and all the expertise needed to be inside the company.<br /><br />From a technology point of view it was great experience. It not only helped ingrain *how* to think about high availability and fault tolerance, but also that it *should* be thought about in the first place. The best lesson is probably that it forces you to think at a full-system level. Everything was redundant in the hardware - power supplies, memories, CPUs, backplane, boards. Anything could fail and be removed and replaced without the system missing a beat.<br /><br />Now, this is super expensive of course. And around that time (1993-94) Stratus itself was moving away from mainframes with fault-tolerance to high availability clustering approaches. But still - it's cool as hell that there are kids out there driving cars around that were born after this thing first booted.<br /><br />I worked in the HAL (High Availability LAN) group, mostly around the development of FDDI - itself a fault tolerant networking technology.<br /><br />I got to work at the application level, in kernel code, and especially enjoyed in an embedded environment - the firmware running on the FDDI board itself.<br /><br />It was a great kick-start to my career because Stratus had layoffs followed by attrition, which led to my being the sole software engineer running the FDDI project for a good chunk of time at about the time that my Coop stint was over (after which I was a part-time software engineer). Even better, the hardware engineers that had started the project also had left the company. There was great fun to be made of the fact that the project was being led forward by a Coop and a couple lab techs.<br /><br />I suppose if it had been 2003, then the cool thing would be to drop out of college and start my own company. But in 1993, having responsibility of a full project within a large company was good enough! I loved having a challenge to rise to and doing it.<br /><br />In the end I can't believe the article didn't say whether this sever is running VOS or FTX though. I bet VOS.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-13835612048155927292010-04-10T02:12:00.002-04:002010-04-10T02:38:24.475-04:00A Strange AnniversaryA couple days ago passed the two year anniversary of my last paid workday. Well, hopefully not last *ever*. I joked with my wife that we'd celebrate by not going out to dinner.<br /><br />I'm lucky enough that this was voluntary. It has resulted in the best two years of my life, without a doubt.<br /><br />The plan at the time was to find an idea that would become a company that I would start. At that point I thought it would actually be one of the ideas that I had at the time. I had only worked for startups, 3 of them, since 1996 - the year after I graduated college.<br /><br />I was looking to get out of networking and telecommunications specialties and work in the more general consumer internet space.<br /><br />We found out my wife was pregnant before I made the plunge, but that timing was really perfect. It allowed me to be working on my own projects at home through the pregnancy and has allowed me to be working from home through the entirety of my daughter's life so far (17 months).<br /><br />I worked on a few of my own projects independently, and worried my wife a little bit when I'd move from one to the next - just when she was getting used to the idea that the one I had been working on was going to be "the one".<br /><br />The whole time I figured the worst case was I was acquiring experience in *lots* of new technologies that I'd put to good use at some point. And as it turned out, that was the perfect warm up for what came next.<br /><br />Last fall I got introduced to <a href="http://www.linkedin.com/in/jonathanmendez">someone</a> looking for a technical cofounder, and the result is now <a href="http://yieldbot.com">Yieldbot</a>.<br /><br />Which is a great way to mark the two-year point. We've launched recently, in private Beta, and learning a ton from our first customer experiences, and being pushed by customer demand. Which is good, as this should mean not too much longer before my little family can spend some money again.<br /><br />It's been a heckuva two years - wouldn't change a thing.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-56830853244914670492010-04-07T15:52:00.002-04:002010-04-07T16:18:09.778-04:00Long TailThis isn't about the usual "long tail" you hear about, but about something similar that for some reason I find amusing - having just received a check for $39.78.<br /><br />Back in 1998-99 I wrote a book on a VPN networking protocol called L2TP (Layer-2 Tunneling Protocol). I had written (in C/C++) our L2TP (as well as PPTP and L2F) implementations at my first startup, New Oak Communications, and then had gotten involved in the IETF process around the standardization of L2TP.<br /><br />I wasn't looking to write a book, but based on some I-D's I had written on extensions to the base L2TP at the time, I was contacted by an editor in an email and asked if I'd think about writing a book. I had no delusions this would be a best seller, and said yes expecting (correctly) that it wouldn't really pay back in money for the time I spent on it, and that it would be a useful experience.<br /><br />This makes me think of the long tail in two ways. First, this is obviously a niche subject. I wrote it for the audience of software developers who would be implementing the L2TP protocol, and secondarily for those that might be involved in the network planning around deploying its use. Yeah, there's gonna be a lot of those.<br /><br />Honestly, I completely forget I even wrote a book unless one of two things happen. First, someone says (for some random reason) something about "your book" to me. It usually takes me a good 5 seconds to know what they're even talking about. Second, twice a year when I get the royalty statement. I say "statement" because it only becomes a royalty "check" when it accrues > $25 due to me.<br /><br />So that's the second, and most significant, way it seems like a "long tail" to me. Because a full 11 years later the royalty stuff still dribbles in on this seriously niche-subject book. I wonder how much longer the poor publisher will need to keep sending me these statements.<br /><br />It's most interesting to me just to see that 11 copies sold the second half of 2009. It must've been the holiday season. I'm really surprised it is more than zero at this point.<br /><br />It's sold 4046 copies over these 11 years (technically only 10.5 so far), which is pretty cool. All in all it was worth the experience. The most surreal thing was one day years ago coming home to a small package that came in the mail from the publisher that contained a few copies of the Japanese translation. That was worth it just for the joke from my mother that she understood about as much of it as the original in English.<br /><br />The thing I'm happiest about is the positive reviews on Amazon that it got. That's what I was most worried about back then, having direct access, and for all to see, to someone's potential negative take on something that I put some real effort into. That was a new thing back then, and it was scary.<br /><br />Anyway, whenever this statement comes it just makes me chuckle. I was pretty sure I wouldn't break the $25 threshold again and get a check. I've got to imagine though that this one really will be the last one.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-68475232985393105652010-04-03T00:13:00.002-04:002010-04-03T00:46:16.432-04:00MongoDB SequencesI came across an issue today with MongoDB, the first one where SQL would have had a simple answer - sequences.<br /><br />If you're familiar with SQL you know that this is very simple to do by declaring a column an index in a table. With MongoDB there's no built-in capability for this (as of 1.4.0, which I'm using now - I wouldn't be surprised to see some built-in sequence capability in a future release).<br /><br />MongoDB does have a case that guarantees order, a "capped" collection, but it's not really meant for this purpose.<br /><br />I found some hints online about how this could be done. Since none of those were satisfying and I ended up coming up with my own relatively simple way, I thought I'd share it as food for thought.<br /><br />The approach is to have a javascript function saved to the database that can be called from the client to do our bidding. The client calls db.eval() to invoke this function to insert the object for us.<br /><br />To set this up, I created a collection named "sequences" where the "_id" of each entry is the name of another collection in the database that I want to be sequenced. The table just needs to have an entry with an initial value for the sequence (take your pick) before the function to insert ever gets called.<br /><br />For instance, if I had a collection named "foo" I would start with an entry like:<br /><pre>{_id: "foo", seq: 1}</pre><br />To insert an object into the database I invoke db.eval() passing the function the name of the collection to insert into, and the object to be inserted.<br /><br />The function that does the insert is:<br /><pre><br />function(coll, obj) {<br /> var s = db.sequences.findOne({_id: coll});<br /> s.seq++;<br /> db.sequences.save(s);<br /> obj['seq'] = s.seq;<br /> db[coll].insert(obj);<br /> return {'seq':s.seq,<br /> 'error_status':db.runCommand("getlasterror")};<br />}<br /></pre><br />I'm returning the sequence that was allocated (which I keep track of in my use case in the case where the insert was successful) and the error information associated with the insert. That way if the insert failed for some reason (like an index uniqueness constraint violation) I still find out about it.<br /><br />Some caveats are probably in order.<br /><br />I'm not in a sharded environment (yet), and when I am I suspect I will have to revisit this.<br /><br />This also isn't the most efficient approach for high performance because db.eval() monopolizes mongod, so depending on the database usage pattern this could be pretty disruptive. On the other hand, this mongod behavior effectively acts as a lock and means calling this function will be atomic. I'm going to wait and see if this is actually a performance issue in my environment, however, before implementing an approach that brings more complexity into the application.<br /><br />Whatever the case, I thought this was an interesting way to solving the problem to think about as it's a pretty straightforward analog to the SQL sequence functionality.<br /><br />By the time I need to do it differently, maybe there will be a native way to do so in MongoDB.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com0tag:blogger.com,1999:blog-1346367426766106836.post-48660445914802704812010-03-31T18:38:00.009-04:002010-03-31T19:14:21.078-04:00Homemade Anti-siphon Check Valve (zip-tie + latex glove)A tale about bootstrapping your startup.<br /><br />The weather lately has been super rainy, which unfortunately comes with some water in our basement. After our sump pump broke down I was able to switch in a utility pump and move the float-switch from the sump pump to that and it worked well.<br /><br />The one problem I had, though, was that the utility pump uses a standard garden hose and neither I, nor my local hardware stores, had an attachment to make it not siphon when it wasn't running.<br /><br />What this means is that once the utility pump emptied the well it is in and switched off, all the water that was in the hose would flow back down and fill the well half way back up. Obviously this means that half the work the utility pump was doing was put into moving that half-well of water back out. Basically it was working twice as much as it needed to.<br /><br />I put up with this for a bit, but it stayed in the back of my head because this is a problem that should be solvable.<br /><br />Today I hit upon it - using a latex glove to block the end of the hose when the water stopped so that air couldn't enter the end of the hose and let the water flow backward.<br /><br />Here's the recipe:<br /><div><ol><li>cut the tips off all the fingers of a latex glove, except the middle finger;</li><li>put the latex glove over the hose with the end of the hose down as far into the middle finger as you can get it;</li><li>stretch it fairly tight and ziptie the wrist of the glove around the hose to hold it in place.</li></ol></div><br /><br />I'm mainly posting this because I didn't have much luck searching for homemade solutions online. So maybe this method will catch on. Just sent me a $1 royalty any time you use it. ;-) My guess is the "parts" probably total about $0.05 in cost.<br /><br />Here's what happens. When the water turns on, the pressure expands the middle finger part and the water flows up and out the other fingers. When the water stops, the lack of pressure lets the glove contract back into place, with the middle finger back around the hose. When the water wants to flow backward in the hose, the negative pressure sucks the tip of the middle finger back into the hose where it forms a tight seal around the end of the hose and stops it.<br /><br />Of course, you could use duct-tape instead of a zip-tie - but I like to use zip-ties whenever possible.<br /><br />Here's why this is better than store-bought! I was able to do this on the end of a hose that had been cut to length, with an irregular shaped end and no threads.<br /><br />I just couldn't resist a little video to show clearly how it works. This is the one I did around the hose with the cut end. Yes, it is giving you the finger.<br /><br />Direct link: <a href="http://www.youtube.com/watch?v=LXSiZgI2bW8&feature=player_embedded">here</a><br /><br /><object width="320" height="256"><param name="movie" value="http://www.youtube.com/v/LXSiZgI2bW8&hl=en_US&fs=1&"><param name="allowFullScreen" value="true"><param name="allowscriptaccess" value="always"><embed src="http://www.youtube.com/v/LXSiZgI2bW8&hl=en_US&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="320" height="256"></embed></object><br /><br />The funny thing about this whole exercise is that it reminded me of what it's like bootstrapping a startup.<br /><br />There's always a way to solve a problem. The solution you think of first you might not be able to get access to or afford. But remember that you may just have a latex glove and some zip-ties laying around that might even do the job better.shearichttp://www.blogger.com/profile/03201988133381255112noreply@blogger.com1