Why I rejected your scientific paper

A little while back I stumbled across a post by Matt Welsh entitled "Why I gave your paper a Strong Reject," and it's had me thinking since then about my own pet peeves as a reviewer.  One in particular has been frustrating me most of all of late, however, and since it strikes at the heart of one of the most difficult yet simple aspects of science, I thought that I might write about it here.

Ultimately, all of the scientific method boils down to one simple question: "How do you know?"  This is the core of every scientific paper, and when I do not find it, I am guaranteed to recommend rejection rather strongly.  I've had a whole string of papers to review that fail in this regard recently, and it frustrates me to the point where I have changed the way in which I read review papers.  When I begin to review a paper these days, I start by looking at the title and abstract, then immediately flip to the end of the paper to see if there are any results.  Whether it be real-world evidence, lab experiments, simulations, or theorems, I want to know that the authors have delivered something that gives substance to the discussions and assertions that are presented in the rest of the paper.  If so, then I will read the paper with much more enthusiasm and give it a careful review; if not, then I will skim quickly to make sure that I have not missed anything fundamental, but with the pre-judgement that the paper is almost certainly going to fail and be rejected.  Even a review paper would not completely escape this judgement of mine: after all, there should be some synthesis that brings all of the different pieces together to differentiate a good review paper from merely being a keyword search.

So far, it's simple.  You can't claim to know something if you don't present some sort of information to justify that knowledge.  Where it gets difficult is that justification of knowledge is often quite a different matter than people seem to think it is---even scientists.  For example, people often present a number in isolation as though it is meaningful, without giving context to compare it to.  If I tell you a current car can drive 60 miles/gallon, that's a meaningful number because you probably know what other cars tend to get; if I told you that a certain type of car in the 1930s got 12 miles/gallon, however, would that be a lot or a little comparatively?  I certainly don't know, and I get frustrated when people drop context-free numbers like this.

This applies not just to obscure bits of scientific inquiry, but also to all our ordinary lives.  News articles, for example, are full of assertions based on numbers without comparison, as are the labels on foods we eat.  Is $10 million dollars for a bridge a crazy boondoggle or remarkably efficient?  Does "low fat" just mean higher sugar and salt, or that it's actually made in a more healthy way?

At the same time, this doesn't mean one should fetishize any particular classical mode of experimental design or demand "absolute proof" of anything.  There's a lot of ways to lay one's hands on information, and most of them do not need to be double-blind controlled randomized trials.  My own scientific work, for example, almost never involves blind trials, because I work with strong signals and can use computer programs to ensure regularity in how each piece of data is treated.

But dammit, at least give me something to support your castles in the air!

DNA hard drives?

A big change may be coming to computing, and a very strange one at that: sometime in the not too distant future, we may find ourselves storing most of our data in biological DNA rather than our conventional silicon and magnetic storage.  I'm in the midst of a two-day workshop that's investigating this possibility, and there's both an interesting prospect and some very big challenges to overcome.

The core idea and concern is that we're just about at the end of what we can do with silicon. For the past fifty years, the density of transistors in silicon-based computing hardware has doubled approximately once every two years, an observation known as Moore's law.  This observation gave rise to a self-reinforcing market structure and the rapid expansion of cheaply available computing has transformed every aspect of our civilization, binding us ever tighter into a connected global society for better or for worse.  That ongoing increase will soon reach the physical limits of what can be achieved with silicon, however, and one of the interesting directions that people are looking to do better is organic materials like DNA.  The theoretical storage capacity of DNA is about a million times denser than flash drive memory, simply because the molecules involved can pack information that much more densely.  To give a sense of scale, that means that you could (in theory) pack one of the gigantic warehouse-complex data-centers that places like Facebook and Amazon build into a single smallish room.

The goal of this workshop is to figure out just how obstacle-strewn is the path between that theory and reality, and figure out whether it's worth pursuing.  This is another one of those interesting life-on-the-interface meetings, with biologists, chemists, computer scientists, information theorists, and silicon specialists all colliding their different viewpoints; with academic theory, government program-building, and corporate pragmatism all sharing a space.

If these DNA hard drives happen, they'll probably be like tape archives, rather than fast computer drives, but they'll be able to store so much information that would still be very useful to consider. There's a fundamental market challenge in that it's not clear whether DNA synthesis technology has enough pull to support the "writing" part of the hard drives.  There's also a long gap between the density of DNA and the realities of making a reliable storage appliance (error correction, packaging, input/output channels, keeping out biological organisms that would like to eat it, etc.), which I estimate will probably cost about two orders of magnitude, giving only about 10,000-fold improvement over the limits of silicon---which is still quite a big deal.

This is not the main direction that I am going with my own research, which is much more about controlling living cells, but it's quite interesting to be asked to be involved in, and close enough that there may be use for the things we've been developing elsewhere.  Plus it's just plain old fun science fiction and I'll be very interested to see just how it can be developed.

Congratulations to Swati Banerjee Carr!

On Monday at Boston University, I attended the successful Ph.D. thesis defense of Swati Banerjee Carr, who I have been co-advising with Doug Densmore for the past couple of years.

Swati's work is, to my thinking, a classic case of how a lot of science doesn't start with "Eureka!" but with "That's funny..." or, as in Swati's thesis: "Why doesn't any of this work?!?!"  Originally, she was building a simple test system to characterize various different pairs of repressors and promoters in E. coli.  These are some of the most basic and common building blocks in synthetic biology: a repressor acts on a promoter to suppress the gene controlled by that promoter, which means that controlling repressors can switch things on and off in a cell, including other repressors, making it one of the foundational tools for controlling the behavior of cells.  Practically everybody uses them, and in her system she was using some of the most well-known and best understood biological parts around, and yet when she put them together nothing worked.  When she put the parts together in different orders, however, sometimes it worked and sometimes it didn't.  "That's funny..."

With a bit of study, including results of some of my other recent work, the key problem was identified as being the bits of DNA just before the promoters, which changes depending on what else is in the system and how the parts are arranged. And so, Swati's project ended up shifting away from the original plan and focusing instead on solving that problem of context dependence. Now, at the end of her doctoral work, our work together has resulted in a lovely protocol for creating "upstream insulators" that make a promoter behave pretty much the same no matter where you put it.  The data is really beautiful, but I can't share it quite yet: not until Swati has officially deposited the thesis.  What it shows, however, is a remarkably stark contrast:

• Without insulators: a genetic circuit in which every permutation is different, pretty much none of them are "working" by even the most generous definition, and there's not even any real pattern to the chaos
• With insulators: every permutation does almost exactly the same thing, with quite strong and consistent signals for all of them.  

I think this has the potential to be quite a big deal, and don't know why anybody would ever use an uninsulated promoter again once this gets released.

Swati gave a good talk to a packed room, standing room only.  She's got some homework from me and the others on the committee, to improve her actual thesis document some more before it can be considered quite complete, but she's on the home stretch, having done some damned good work, of which I think she can rightly be quite proud.

Reproducibility of Fluorescent Expression from Engineered Biological Constructs in E. coli

I've talked about the interlaboratory studies we've been running through iGEM previously, and while I've been quite excited before, now is the biggest news I have to announce yet: our paper on the results of the studies has just been published in PLOS ONE: Reproducibility of Fluorescent Expression from Engineered Biological Constructs in E. coli.

Fluorescence from iGEM interlab constructs (Credit: Oxford iGEM 2015)

In sum: over the past two years, teams from nearly 100 institutions around the world measured the same simple genetic constructs for expressing green fluorescent protein, and this paper reports their results, crediting all of the more than 600 iGEM authors involved, including lots of undergraduates and even high-school students. In my eyes, the two key results from this study are:

• Ratios between strong fluorescence were remarkably precise.
• Weaker measurements were extremely unreliable, but the problem does not appear to be the biology!  Instead, it appears to be differences in how people use their instruments and handle their data.

These are really good news, because it means that some of the well-known problems in understanding and reproducing biological research might be tackled simply by improving our ability to calibrate our instruments and communicate about our measurements.  That's hard, but it's a lot better than thinking that biology might just be inherently too messy to understand properly.

We've also published all of the raw data submitted by all of the teams, so that people can dig further into the data if they're interested and see what else may be lurking there.

And what will 2016 bring?  That is still in planning, dear reader: you'll just have to wait and see...

How studying synthetic biology has improved my health

A curious thing has happened to me recently, which I would like to share with you, dear readers: over the past year, my study of synthetic biology has significantly improved my personal health. Now, don't worry, I'm not eating poorly tested experimental organisms to improve my gut microbiome or anything like that.  In actuality, the connection is much more indirect: my engagement with biology has simply led me to rediscover something that had fallen from awareness in the course of my modern life.

The story actually begins with the negative impact of science on my life, more specifically the effects of stress and frequent travel.  I am a person who has been overweight for much of my life, and not just by the terrible body-mass-index (BMI) measure that is typically used to judge such things.  At my height, BMI says that I should be carrying at most about 175 pounds, which is simply something that I do not believe.  My ancestry has gifted me with broad shoulders and a solid mass, and even at my skinniest adult weight, early in grad school when I was on a major health kick and running every day, I was just under 200 pounds and think that I was probably a bit too thin.  I am a person who relishes good food, however, and also tend to turn it for comfort, and by the time that I left MIT, I was up to 235 pounds and not so happy about it.

Then my scientific life intensified, and so did my stress and my amount of travel.  Travel is a killer, when it comes to weight, because travel means disrupted eating habits, restaurant dining, and free pastries, cookies, and other treats calling out tempting songs from the coffee breaks of every conference, every working meeting, and every program review.  Restaurants seem to soak everything, even every vegetable, in tasty fats, and I become bloated, weight climbing and energy falling.  As my career progresses, my stress and travel have both increased, and early last year I felt a shock of fear as I saw my weight brush up nearly to 260 pounds.  In that moment, I took a hard look at my sluggish feelings and fatigue, and I thought about my daughter, about the example I set for her and about the fact that I want to live a long, long life as her parent, and I decided that I would have to change my life.

It's been slow and hard and not without its setbacks, but I've been managing my weight much better. As always, there is no secret to it, just a consistent long-term increase of activity and a complementary decrease in my caloric intake.  For activity, my FitBit really helps me, because I am not the sort of person who does well with "heroic" gym time plans: my life is too variable and disrupted, and it works a lot better to just be adding a few more minutes walking here and there throughout the day, taking a circuitous route to get where I am walking, parking at the far end of the parking lot, etc.  For eating, I am a person for whom "eat less" is not a useful option, so I have focused instead on "eat lower caloric density" and have rediscovered in myself a love of many forms of vegetables: my snap peas, brussels sprouts, broccoli, asparagus, cherry tomatoes, carrots, etc. bring the strong tastes and umami satisfactions that can satisfy me and that fill my belly so that I need not suffer for my health.

But still, my travel remained my Achilles' heel, hurling great unhealthy caloric surges every few weeks: I could try to bring some vegetables with me, but without refrigeration I'd start to worry about their lifespan, and often it would take very little time for them to turn quite nasty.  And that's where having spent so much time thinking biologically over the past few years suddenly turned on a light in my mind and made me rediscover something that will no doubt be obvious to anyone who has spent time growing their own vegetables.

You see, vegetables, like us, prefer to live and thrive in a world of open air and oxygen. When we buy them, fresh vegetables are actually still quite alive, and in an open air environment they will try to stay that way for some time, resisting fermentation and rot, and continuing to fight the pathogens that would like to eat them.  My ancestors knew this well, when they stored their root vegetables in the cellar for the winter. When we package vegetables for storage in a refrigerator, though, we seal them in airtight bags, because in a refrigerator the cold slows down the metabolism of their cells and the biggest threat is not decay, but the dehydration of the refrigerator's cooling system.  Take them outside and leave them in their sealed package, though, and in the higher temperature and their enclosed state their respiration is inhibited, the water they emit begins to soak their environment, and they soon begin to spoil.

And so, my friends, once the problem had clearly reappeared within my sight, the solution was also obvious: before each trip, I open up every bag of vegetables, breaking their sterile seal.  The air comes in to help maintain their environment, and in return for this little bit of engineering of my vegetables, I find that they can nourish me in return for many more days.

Sufficiently happy vegetables and fruits, living on the desk in my hotel room.  The rubber bands do not fully close the openings, but keep the vegetables from falling out all over my bags when I travel.

The Value of Lurkers

A little under a year ago, I went to a workshop on synthetic biology standards sponsored by the NIST Synthetic Biology Standards Consortium (and subsequently became a participant in said consortium myself). One of the things that was said at this workshop, which has stuck with me and been rattling around in my head ever since, is about the value of mailing list lurkers.

Lurkers, for anyone who doesn't know, are the people who are subscribed to a mailing list (or other community) but never (or almost never) post to it themselves.  In pretty much every online community that I have been a part of, scientific or otherwise, the lurkers tend to vastly outnumber the active participants, often by an order of magnitude or more.  If you are an active participant, it can feel quite frustrating sometimes, to have all of these people who are theoretically part of your community but do not seem to be contributing anything.  The bulk of the readily measurable "work" in many communities is typically done by a very small group of core participants, i.e., the ones who draft the standards documents, who update the websites, who organize articles for publication, etc.

That view, however, I have come to understand is actually a dangerous illusion.  The lurkers who only watch and rarely if ever speak have a very important role to play in a community, and especially in a community that is seeking to develop a consensus policies (such as scientific standards).  Those lurkers, you see, are a constituency, a user base, and a conduit to the larger intellectual world.  Here are some of the key functions that "inactive" community members perform that are not so visible, yet clearly critical to the success and survival of a community:

• Speaking up when their interests are threatened: Many people in a community will not speak as long as they feel their interests are being sufficiently well represented by the people who are speaking.  They don't really care which of several plausible alternatives are being chosen, because they can work with any of them and they trust the de facto decision makers sufficiently. In this way, silence often does in fact represent consent.  It is for this reason that when somebody who does not usually participate speaks up to voice a concern, it is particularly important to pay attention to them, as they are likely to represent an important and largely silent constituency. Moreover, when they speak up, it is likely because something important is starting to go off the rails and the usual "speaking club" is too blinded in some way to notice this fact (else it would not be happening in the first place).  Unfortunately, it is also particularly easy for such voices to be squashed due to the very fact that they are not part of the usual "speaking club" and likely to be rebuffed due to their less facile and polished presentation of concerns.
• Using the ideas and other products of the community: People generally don't sign up for a community unless they have at least some interest in what's going on in that community.  The people who are not speaking may not be creating much new content for the community, but they are often listening to it.  The thoughts and products of the community are with them still when they go elsewhere, to places in which they are in fact more active.  The silent members of the community, then, are actually most likely to be the conduits by which it actually can affect the outside world, since the more "productive" community members are investing their energy inside the community rather than outside of it.
• Simply being aware of the community: Even if the more silent members of a community are not finding the community's content to be of use to them, they are at least aware that the community exists.  As such, they are still important representatives of the community to other outsiders who may have interest in it, and can help to connect those people to the community, facilitating community growth and mergers with other likeminded communities, as well as helping to prevent others from accidentally trying to reinvent the wheel.

Thus, not only are the quiet members of a community more than dead weight for the community to drag along, but those quiet members also serve functions that are both extremely important and also in many ways quite distinct from those of the louder members.  If you are an active member of a community, I urge you to embrace your lurkers, to value them, and to make it as safe as possible for them to speak up when they are motivated to: their words are likely to be the most important for you to hear.

Sexism on the Radio

Driving to school this morning, my three-year-old daughter made a very difficult request: she wanted to listen to "woman music" on the radio, i.e., music with a female vocalist.  I knew that this would be difficult, from long experience, and so the search itself has become a game, in which we classify each channel as it scans past: "Man music, advertising, advertising, man music, man music, only instruments, advertising, man music..." Eventually, after twice around the dial, we found a song, in which the singer was saying something about making a painting in which she and her man would be trapped in perfect bliss.  The point, however, is that it was remarkably hard to find, as it usually is.

To me, this is a perfect example of a third-wave feminist issue.  I am certain that there is no intentional conspiracy across the various broadcasters to deny female artists a spot on the radio.  Rather, I expect that this is more a case of market optimization, implicit bias, and apathy.  And that's the way in which I understand the three main "waves" of feminism: the first wave was obtaining legal personhood (e.g., voting, property rights), the second wave was removing other formal barriers to entry (e.g., opening up male-only jobs), and the third wave is noticing things and saying: "Hey, some things are still really gender-biased! What's going on?"

At this point, once you've noticed a strong gender-bias somewhere, there are three basic responses:

1. Decide to ignore the issue.
2. Look for a reason why it's "right" that something should be strongly gender-biased.
3. Acknowledge the issue and try to figure out how to respond it.

Right now, the best I know how to do with the lack of women's voices on the radio is to call my daughter's attention to it and help her to become a critic of media, as well as enjoying the parts she likes.  At other times, we simply get our music from other sources, where we have more control of what we hear.  There's no one right solution in parenting, and it's an always shifting ground based on how your child is currently understanding their environment.  I'd prefer if the radio broadcasts could be changed, but I don't have leverage on that problem, and it's not where I'm going to choose to invest my energy right now---thus opening myself, of course, to criticism that I'm choosing to ignore the issue.

If the owners of radio stations chose to, however, reducing gender bias would be quite easy to do: there's lots of awesome female artists out there, just like there's lots of awesome male artists, and it would be pretty easy to simply adjust the playlists to be more balanced.  In fact, two genres already appear be quite balanced: evangelical and club/electronica.  It's strange to me that from such opposite ends of the political spectrum comes a balance, but there it is, as well as clear evidence that it need not be that hard to do. Right now, however, everywhere that I've encountered them, rock, country, alternative, metal, easy listening, and oldies all appear to be quite heavily male in their playlists.

And yet, and yet: still my daughter yearns to hear women's voices, and it's very hard for her to find them on the radio.

How fresh are your bananas? An exploration of functionality in biological computing

A recent conversation got me thinking about how to explain quantification of "function" in biological circuits.  This is a critical issue and one of the key inhibitors to engineering complex biological systems, but it's not easy to explain because it involves a lot of technical electrical engineering / computer science concepts like signal-to-noise ratio, input/output transfer curves, non-linear amplification and threshold matching.  I think, however, that there may be nice biological metaphor that can make this concept easier to understand.

You see, a biological computing device is like a banana.

Let's say that I want something to eat, so I go into my kitchen and find a banana.  Is my banana "functional" as food? Well, it's very important for food to be fresh and healthy, so I'd better make sure my banana is fresh. But just how fresh is "fresh enough"?  Let us consider some different ways that my banana might look:

A spectrum of banana freshness (credits: yellow, spotted, brown, rotted)

The banana on the left is beautiful: no question that it's fresh, and indeed so perfect that I am sure that my three-year-old daughter would eat it without the slightest protest.

The second banana is getting on in age and starting to develop spots.  It might be a bit mushy inside, and my daughter will definitely not eat it, but I'd be happy enough to chow down.

The third banana probably won't taste good to eat on its own, but it's just perfect for making banana bread or other recipes that transform a banana from centerpiece to simply tasty flavoring.

And as for the fourth... I don't think I'd even want to feed that melted mess to livestock.

As you can probably see, the notion of a "fresh banana" is not a fixed concept, but a spectrum, and "fresh enough" depends entirely on what exactly we want to do with that banana.

Biological computing devices are the same way: a device has to be very high-performance in multiple dimensions (uniform yellow) to be safe and useful for complex circuits or applications like precision medical therapy, while simpler and less safety-critical circuits can tolerate some problems (spotted yellow), some applications just need a nudge or two in the right direction (brown), and some devices probably aren't good for anything at all (rotten).

Right now, the vast majority of our available biological devices are metaphorical brown bananas, with just a few spotted bananas available.  Understanding that fact is the first step, and getting on with building some fresher bananas is the second.

A week on the computer science / synthetic biology interface

Back in August, I spent a week in Seattle playing several different roles at the interface between computer science and synthetic biology.  It was all built around a conference that I've been involved with and participating in for a number of years now, the International Workshop on Bio-Design Automation (IWBDA) (amusingly, given my now geographic location, it has only recently displaced the Iowa Wholesale Beer Distributors Association as the top Google hit for that acronym).

This past year, I was the Publication Chair for IWBDA, which meant I ran herd on the paper submission and peer review process, making sure that we actually get clear judgement, as unbiased as possible, of which submissions are strong scientific contributions worthy of putting on stage as talks. In the end, I think we had a quite strong program, with some very exciting results from a lot of good groups (I gave a talk of my own as well, on circuit design using signal to noise ratio, which I will leave to others to judge), and I'm hoping the associated special journal issue will come out strongly as well.

In addition to IWBDA, two other events attached meetings, taking advantage of their overlap in interests with the IWBDA community.  Just before IWBDA (and, in fact, part of its "pre-conference" schedule) was the SBOL community meeting, aiming to disseminate information and support adoption of our data exchange standards for biological designs, as well as to get more input from more different groups into its development.  In that meeting, in my role as an editor of SBOL---one of the community's elected leadership---I presented some material and helped to facilitate discussion and organize plans for the community.  

Before that was a two-day meeting of the SemiSynBio Roadmap project, an effort sponsored by the semiconductor industry, which has gotten keenly interested in synthetic biology as a possible direction of expansion as Moore's law winds down, and is trying to identify the key directions of research that can set up a similar exponential expansion of capabilities and markets in its relationship with the biological world.  It's fascinating, unclear whether it will turn out meaningful or merely hopeful, and I sit on the Executive Committee of this project, trying to help ensure that we end up with a clear and productive vision out of the several working groups studying different aspects of that interface, an invited position that doesn't fit cleanly into any of my well-defined job responsibilities and yet is clearly a good use of my time and effort as a scientist.

In between, in the corners of my time, I pursued yet other pieces of my scientific life on the interface, including standards development with NIST, the 2015 iGEM interlab study, and various relationships and collaborations with other interesting characters who I enjoy and who live in similarly strange niches to myself.

Interesting things happen at interfaces, both in physics and in society, and scientific communities are no different.  It's an uncomfortable and delicate place to stand, when you're not really at the heart of any of the communities that you're trying to participate in and affect, but it also feels like home to me.

An Introduction to SBOL Visual

Following up on our recent publication on SBOL Visual, an important next step is to make it nice and easy for anybody and everybody who wants to illustrate a genetic construct to do so using SBOL Visual.

To that end, I've now posted "Introduction to SBOL Visual," a short set of slides intended to give all you need to know about making genetic construct diagrams in one simple and easy to digest package.

Please share, enjoy, and send feedback on adjustments that you think would improve the document!

Download [673.14 KB]