Paying down organizational debt

When I first learned about the concept of technical debt (from a post on the excellent Coding Horror blog), it was like a light going on in a dark room where I had been tripping on things for years. Technical debt (also known as coding debt, when considering software) is a way of thinking about the cost of shortcuts.  When you are putting together a project and you need to get stuff to work now, we often choose an approach that's faster and easier to implement, but which we know isn't really the right way to do it.  That choice creates a piece of technical debt---an approximation, inelegance, or minor incorrectness.  When we do something else in the future, it may be harder because of the technical debt that we have created, either because we have to work around the poor implementation, because we have to fix the poor implementation, or simply because the poor implementation is confusing.

Technical debt is not necessarily bad, any more than financial debt is necessarily bad.  Taking on technical debt is a way of accomplishing something that you might not have been able to accomplish if you had to do everything "right" the first time.  It's also a way of deferring difficult decisions until we better understand which path is correct, in order to avoid doing something we think is "right" but that later turns out to have been wrong.  Like financial debt, however, technical debt can accumulate and can also lead to taking on additional technical debt, creating all sorts of havoc.  What's important is to keep track of your technical debt and to try to make wise decisions about when to allow it to accumulate and when to pay it off.

Lately, I've been realizing that this same idea can be extended more generally to organization of one's effort on across different projects.  As a scientist who leads my own investigative ventures, I have quite a number of different projects that are "live" at any given point, ranging in scope from an hour or two of effort here and there (e.g., service as an associate journal editor) to complex multi-year ventures (e.g., development of the aggregate programming stack).  When I make choices in managing these projects and my time between them, I often take on organizational debt.  This accumulates in lots of different ways, such as note cards accumulating on my desk, file directories growing large, open browser tabs, email messages on my "need to reply" list, etc.

Just like taking technical debt on in writing software, there are pluses and minuses in taking on organizational debt. If I spent lots of time trying to be hyper-organized and so that I avoid taking on organizational debt, then I will be much slower at actually accomplishing the things I'm trying to organize.  If my organizational debt causes me to overlook a deadline or to end up in a last-minute crisis trying to get things done, however, then my work and my life suffer in various ways. Some of these costs are quite signifiant and spill out of work into costs on the rest of my life, such as losing time with family and friends, lack of sleep, getting sick, and gaining weight.

My struggle of late, then, has been to recognize that paying down organizational debt is a real and legitimate part of my job, just as paying down technical debt is a real and legitimate task in executing particular projects in my job.  I wouldn't say that I've found a clear method of managing my organizational debt yet, but, as they say, the first step to solving a problem is to recognize clearly.  I have, however, made some steps forward that have helped immensely.

For example, I am now both publishing papers frequently and traveling frequently.  Both of those have major time lags involved.  For example, in publishing a paper I first submit a manuscript, then get reviews, send a revision, repeat until accepted or rejected, wait for publication, post and publicize; along the way I also need to obtain release permissions internally and sometimes also from funders.  This process can take years, and if I've got a bunch of papers in flight it's easy to lose track of a deadline and create a failure or crisis where none was needed.  Travel is similar: registration, booking flights, hotels, cars, sending in pre-expenses, actually taking the trip, waiting for expenses to register in the reimbursement system, submitting requests for reimbursement, and actually getting reimbursed often spans many months.  To address these problems, I've created a spreadsheet for each task which lets me have a "dashboard" view of what's going on overall with regards to that area of responsibility.  For example, here is part of the 2015 sheet from my publications spreadsheet:

Publications not yet available have their names tastefully redacted.

It's not a panacea.  Nothing is, for me, when it comes to technical or organizational debt, because I'm not willing to pay the additional cost of not taking on debt.  Moreover, I'm sure that my approaches will have to change periodically, as my career continues to evolve.  Solutions like these help, however, and recognizing the importance of (at least sometimes) tracking these moving targets is, I think, a useful step forward towards more improvements in my quality of life, both at work and also at home.

Signs in the Snow

When I fly for business reasons, I always try to get a window seat.  I have always delighted in the view from the airplane window, at all the shifted perspectives one obtains when looking at the world on high.  As long as I still love these sights, I feel, I know that I have not become too jaded, and my sometimes-strained soul has not yet died (I speak in the metaphorical sense, of course). When I fly with family, it's different now: Harriet almost always claims the window, and despite my longing to displace her, I would rather share the joy than steal it.  When I am alone, however, even on my shortest and busiest flights, I will track the sights at least occasionally, and I often shutterbug my way through takeoff and landing, so happy that our minor electronics are once again officially allowed to be active at those time.

One of the things that always fascinates me most is the way the landscape radically transforms with seasons.  Moreover, despite what one might think, I find that it is winter that most brings out the texture of the land.  When snow is on the ground, its topography is highlighted, leaping out in dark lines on every vertical and slope.  With those thoughts in mind, I present to you dear readers, an album of interesting forms I've seen, which I think of by the title "Signs in the Snow":

Explaining tornadoes to a preschooler

I've had a bit of a nervous evening tonight, thanks to one of Iowa's signature weather events.  I knew there was going to be rain today, but was unprepared for what to do when my phone unexpectedly buzzed with a county emergency alert as I drove to pick Harriet up from school.  I looked down and saw a tornado warning and a small piece of my brain flipped out: "What am I supposed to do if I'm in a car?" "Should I try to pick Harriet up from school or not?" "What happens if I can't pick her up on time?" 

At the next light, as the rain suddenly picked up and began to pound furiously against my windshield, I scrabbled to search on my phone and was rewarded with the official word from NOAA: "There is no safe option when caught in a tornado in a car, just slightly less-dangerous ones."  I scanned channels on the radio and found that while most were doing business as usual, the local country music station had switched over to nothing but live storm reporting.  I cranked the volume to be able to hear it over the rain and inched forward into the pitch black of dark storm and early nightfall, barely able to see the cars packed all around me, trying to figure out whether I was doing the right thing or the wrong thing.

Since moving to Iowa two years ago, I've had two previous encounters with tornado alerts. The first time, the sirens went off while all three of us were at home.  I was actually on a Skype call with colleagues back in Boston at the time, working on a paper, and I think that they were more freaked out than I as we relocated down into the basement for the duration.  The second time, I was in the air, returning from Boston along with Harriet and Ananya, and the plane made three attempts to land in heavy, free-falling turbulence before giving up and depositing us in Peoria, Illinois.  That was a very, very long day.

Today, I splashed through flooding streets and eventually found a big hand-written sign on the school door telling parents to come find after-school down in the basement.  Parents, children, and teachers were all packed in together, and the slightly nervous buzz amongst the adults helped stimulate the children to a high degree of frantic energy, not really understanding why today was different but knowing that it was so.  

After 20 minutes or so, the storms had passed and we began to disperse, but the time had left its imprint on our evening, especially since Harriet wanted to dawdle and hang out at the school and I was very eager to get out of the weather and home, not knowing how long the gap might last.  All the way home, Harriet and I talked about thunderstorms and tornadoes, and I tried to tread a careful line between ensuring that Harriet understood that tornado watches are serious business but also not making her frightened.  Partway there, she had an excellent inspiration and asked me to find a song that would make tornadoes and thunderstorms go away, and I doggerelled up the following:

Tornado, tornado, don't come around our house!
Tornado, tornado, just go away you louse!

Real peace of mind for Harriet, though, didn't come until we were sitting down at the dinner table and got the internet involved.  First, we went searching for other tornado songs, finding some excellent broken-heart-country kinda stuff and a couple of wacky amateur things that Harriet liked even better. As she got into it and began to relax, I offered to show her how we were kept safe from tornadoes, and so we ended up looking at the NOAA radar:

Tonight's line of thunderstorms, observed from home after they passed us by.

The colors help, that's for certain, and I think she really got it, since she started telling me things like "We want to be in the blue and green, not the red."  I showed pictures of doppler radar stations like the one in Davenport and talked about how the people there had really good computers and were always watching to make sure they knew when tornadoes might appear, that could tell us if trouble was coming 15 minutes ahead of time, and that my phone would buzz and let us know if we needed to go to the basement.  We navigated all around the region, looking at different radars (Harriet expressed much sympathy for Chicago, which was going to get hit next), and looked at satellite images from space as well as pictures of weather satellites.

And then Harriet asked to look at the weather radar for Madagascar, which she's been into lately, and we found there wasn't one, or at least not one whose images are posted online.  That made me really think about the infrastructure involved in having weather safety systems like we have, and the powerful role of government and civil society, not just in keeping me physically safe, but also in providing me with peace of mind, enough even to pass it honestly to my three year old daughter.  I did not have to tell any comforting lies tonight, or hope and pray for safety: I know down deep in my heart that men and women are doing simple professional work to establish this remarkable network of protection across the land.  My eyes teared up a little, as they always do when I contemplate the non-glamourous wonders of civic society, and I know why I believe in the importance of good government.

Jack Splat!

Last night the Iowa Children's Museum presented us with a wonderful surprise. We'd made a plan to go there for a Friday evening outing after school, and we walked unexpecting into the remarkable event known as Jack Splat! In the big "main street" open space at the heart of the museum, where normally one could access the music room and the post office, tarps were laid down and a cleanup crew stood below. Above on the balcony lurked a great swarm of over-aged pumpkins, a week past Halloween and ready to meet their end.

Our timing was perfect: Isaac Newton was just explaining to a rambunctious audience how his first law of motion meant that a falling pumpkin, once in motion, would keep moving until it encountered an opposing force---"The ground!" cried the children, and the pumpkin flew down to meet its fate.

Isaac Newton (resplendent in toilet-paper-roll wig) and lab assistant preparing to launch a pumpkin.

The physics lessons continued, accompanied by redolent meaty splats of demonstration. Before each pumpkin flew, the throwers read out its name and the name of its donor, as well as the specified method of execution (e.g., roll from the ledge face first, backflip up in the air).  The children cheered and chanted (though one little boy near us was quite upset, and asked why the people hated pumpkins so much), and the rain of gourds continued for nearly half an hour, quite challenging the cleanup crew to keep up with all the mess.

Getting ready for the next bombardment.

I much enjoyed the unexpected show, and was reminded of a similar but much smaller scale yearly event arranged by the undergraduates at MIT.  A good and rather cathartic end to a Friday evening.

The Power of Ignorance

When my friend and colleague called herself clueless this afternoon, it triggered something in my mind.  We've been working on a paper together, and since I'd taken the first pass I'd put everything in LaTeX, hoping to avoid having to figure out how to manage citations and such in Microsoft Word.  I hate writing in Word, and as a computer scientist I usually get to avoid it, opting instead for the nitpicking and precision control that LaTeX offers me. When I am collaborating with biologists, however, LaTeX might as well be Martian or Haskell to many of them, and we often default back to Word.  Selfishly, I'd avoided that, and just assumed I'd end up taking feedback notes or snippets of text and incorporating them myself.

But I had aimed too low, and here my friend surprised me.  Rather than complain or take the easy route, she asked me to put the document into Overleaf, an online LaTeX interface that she'd just learned about, and she did her editing there, including using the LaTeX todo notes package that I've been using for tracking commentary in the document.

And then she called herself a "clueless biologist," as she asked me questions about this fearsomely complex new technology she's been voluntarily educating herself on.

Those two words say something that is very important and also sad about the way that science often operates, and I think about our larger society as well.  My friend is largely ignorant about LaTeX, in the sense that she lacks knowledge, but "clueless" is a rather negative view of ignorance.  Adding "biologist" lumps her into a category, othering her and tying that to this "clueless" expectation---a stereotype that I'm quite familiar with.  I don't like it, though, because that phrase sets up an expectation of an "us vs. them," Men are From Mars/Women are from Venus, Computer Scientists are from LISP / Biologists are from S. cerevisiae sort of oppositional dichotomy.  That makes us all smaller, I feel, because it divides us and suggests our minds are alien to one another, and therefore that we ought not to attempt to learn from each other so much.

Instead, I think that we should be celebrating and embracing the power of our ignorance.

By this, I do not mean that we should avoid knowledge.  Knowledge is wonderful and empowering, and recognizing one's ignorance is a first step to doing something interesting involving others who are not ignorant in that same area. The Renaissance Man is dead---quite dead---and Joy's law is how we spend our lives, in a world where there are so many interesting things to know about, some important and some just fun for somebody.  We are all more ignorant than we can know, and not just in a snotty Socrates one-upmanship way of looking at it.

One of the hardest things I had to learn while I was a graduate student was how to say, "I don't understand" and "I don't know."  I learned it from Hal Abelson, the professor who always asked me the hardest simple questions I have ever heard.  Hal taught me that saying "I don't understand" did not have to be an admission of weakness.  It could simply be the truth, and then what happens next depends on why you don't know.  When I was talking to Hal, it was usually the case that Hal saying "I don't understand" was an indicator of some fundamental flawed or overlooked assumption in the thing that I was trying to explain to him.  I learned a lot about my own research from Hal's admissions of ignorance, and I also learned to stop being afraid of lacking knowledge.

I'm still learning that. It's easy in our competitive world to fear that admitting ignorance is the first step to losing out to other people who are better at putting up a front, and I still struggle with that. But fearing ignorance is almost as bad as being proud of it, and I prefer to avoid it when I'm not too panicked to consider the bigger picture that I'm living in.

The fear of ignorance is competition, but the power of ignorance is partnership and teamwork.  I'd much rather live in the second world, and I hope that I can be sufficiently wise to help encourage it for both myself and my compatriots.

Whole lotta ruttin' going on

That's what the highway sign said this morning:

WHOLE LOTTA RUTTIN GOING ON WATCH FOR DEER

It brought a smile to my face even as I was duly warned of the frightful danger that I face on the roads of Iowa.  This is one of the most intense states in the nation, when it comes to collision risk: currently third, with a 1 in 68 chance of hitting a deer each year.  Frankly, this number really blows my mind, especially coming from Massachusetts where the odds are an order of magnitude lower, and Boston in particular where you might as well not even bother thinking about the possibility.  I've had a recent close call of my own already, a couple weeks back on a date with my wife, when a deer darted across in front of us and I had to jam my breaks on to avoid an accident.

In Iowa, the distinction between countryside and city is much sharper and closer than New England, and it seems to me that it's a virtually ideal environment for deer.  Deer flourish on the edges of forests and in sparse woodlands, and those are things that Iowa has in great amount.  Drive through the countryside, and you see something that I have never really known before I moved out there into the Midwest: an entirely rural-industrial environment.

Growing up in New England, and with my father's stories of growing up in Colorado, I'm used to the idea of "rural" meaning lands where little or no people live.  Walk through the woods of Maine or New Hampshire or even Massachusetts, and you will first of all find that you've a very hard time walking through those dense pine woods at all, and second that they often extend in all directions for many miles, a fearsome wilderness broken only by the old stone walls of centuries-abandoned farms. The woods of Iowa, by contrast, are linear affairs, in which one would have a rather difficult time getting lost at all.  They curve along contours of land, following streams and rivers, in those narrow places where the land is too steep to be productively arable.  Elsewhere, the land is mostly farms, broken by an apparently arbitrary fractal dispersion of cities, towns, and factories.  Suburbs don't exist they way I knew them in New England, where the city slowly peters out into nothingness over the course of many miles: Iowa City stops about a mile East of our house on a straight-edged line, an instant transition from extremely dense developments to fields of corn and soy (rotating on a 3-year cycle).  Even in the most rural areas I've been, you're never more than a couple of miles from a dense aggregation of a few hundred people clustered together in a tight little town.

Biodiversity is low, in an environment like this, but species that do well on the boundaries with humanity, like deer and rabbits, flourish and expand.  All the halloween pumpkins in our neighborhood are attacked and eaten by marauding squirrels.  And this transplanted specimen still feels for roots, sorting out my place in this rich Midwestern soil.