Monday, January 09, 2017
Now witness the true power of measurement!
Today, one of my friends and colleagues sent me the following image. I am flattered, and notice that I am apparently picking up some kind of reputation, but I am entirely unsure on just how to interpret it. Dear readers, would you be so kind as to provide your own commentary?
Monday, December 26, 2016
Cooler than Magic
Our four-year old daughter doesn't believe in Santa Claus. We didn't push the Santa story, nor did we discourage it: we just let Santa be one of the many enjoyable cultural figures whose stories she knows, along with such other luminaries as Elsa and Anna, Durga, Peppa Pig, Jesus, Cinderella, Shiva, and Team Umizoomi. We've got a "no lies" policy in our parenting, though, so when some time ago she asked straight up if Santa Claus was real, the answer was simple: Santa is not real, but is a game that grownups and kids play, and once she knows the truth, she's playing on the grownup side and it would be very rude to spoil the game for other kids.
I'm certain that some people would say that we have done a terrible thing, stealing one of the pieces of magic from her life and shortening her childhood's period of innocence and wonder. I believe, however, that such concerns stem from a misunderstanding of what is so special about innocence and wonder. The innocence and wonder of children is tied up in many ways with their ignorance: you cannot experience wonder in exploring the geometry of a cardboard tube, for example, unless you are largely ignorant of geometry and cardboard tubes. I find that many people seem to confuse the ignorance, which is anything but valuable, with the innocence and wonder that happen to be associated with it in time.
As a parent, observing my child and other children, my current best understanding is that the importance of innocence is thinking and acting without feeling hemmed in and constrained by fears, and that the importance of wonder is the experience of how remarkable and marvelous our lives can be. It is much easier to achieve these states when we are ignorant, but we need not give them up merely because we become experienced. Rather, I think they are a critically important part of what makes life worth living.
Nor does knowing a truth prevent you from enjoying fiction. In my own experience, holding a very clear understanding of the distinction between fact and fiction can let you enjoy the fiction all the more because there is less need to be worried about whether it is true or how it may be affecting things. I even find it to be a good way to disempowering things that I am concerned about as a parent, like the Disney princess culture. If we try to hide from it or focus on criticism, that just gives it power as something that grownups are worried about. But if we embrace it, and acknowledge how fun the story is, but also help to remind and remember how it's just stories, and that stories are different from the real world, then it can draw the power out and turn literary criticism into a game that we can play together. Elsa is still an awesome queen with seriously cool ice magic, even if she'd be really sick if she was a real person with a waist that is so tiny.
The biggest difference, I find, in talking about fiction versus reality is that the "Why?" questions for fiction quickly ground out in questions of narrative structure. "In 'Moana', why did Te Fiti have her heart on the outside where Maui could steal it?" "They didn't give a reason. We can try to make up some reasons, but mostly it needed to be somewhere that he could steal it so that the story could happen." With reality, however, there is always something more, something outside, and ultimately things always ground out in an answer, and opportunity to look things up and learn, or, eventually, "That's a really good question, and we don't know the answer yet!" And then maybe we can talk about how we might try to find out.
The reality of the world is so intricate, and its systematicity so remarkable that I find wonder is never very far from the surface. Driving through the fields a couple of days ago, I noticed a microwave relay tower and pointed it out, and we talked about communication. I love the great big scoops, posed on top of their rectangular spike of concrete, I love that microwaves interact with metal in such a way that their circuitry often ends up looking more like plumbing than wiring. I love that the waves themselves are inches wide, while an FM radio wave is about three times my daughter's height, and the light that I see is a bit less than a millimeter. I love that our civilization has erected concrete towers standing high across the plains like a high-tech semaphore network, relaying messages in a network little different in principle than moving goods in trucks or messengers riding horses in centuries past. There are thousands and thousands of such edifices out there, and many people have been involving in building them, bringing together on each of those thousands of sites a whole assemblage of heavy machinery, cranes, truckloads of electronic equipment, and so on. The more I think about it, the more wondrous that grey square lump of concrete with the funny metal scoops becomes.
Some of my wonder and joy, I think, comes from the training that I have received as a scientist. Most, however, does not need much specialization or knowledge to appreciate, just an open mind and a willingness to experience one's ignorance and look for answers. I would not say to embrace or celebrate ignorance, but to simply recognize it as a state of knowledge that one can choose what one wants to do with. A lot of people write themselves off from knowledge, saying that they just don't have the right head or the right training. I think that often this may come from a discomfort at being ignorant, and I am saddened by it. I think that we will all be better off, both as individuals and as citizens of our civilization, the more that we are able to embrace our innocence as seekers of knowledge and connoisseurs of the wonder of the world.
Later on Christmas day, as we looked through the window at the rain coming down outside, my daughter mentioned that her teacher had said that it would rain on Christmas. I asked her how she thought her teacher had known, and she told me her teacher must have read a weather forecast. And then we all talked as a family about how weather forecasts use satellites to look down from way above the Earth so that they can see what's going on over a very big distance, and see what weather is coming towards us over the next few days.
My daughter's eyes grew wide with wonder, and she said: "That's even cooler than magic!"
I agree with her most strongly.
I'm certain that some people would say that we have done a terrible thing, stealing one of the pieces of magic from her life and shortening her childhood's period of innocence and wonder. I believe, however, that such concerns stem from a misunderstanding of what is so special about innocence and wonder. The innocence and wonder of children is tied up in many ways with their ignorance: you cannot experience wonder in exploring the geometry of a cardboard tube, for example, unless you are largely ignorant of geometry and cardboard tubes. I find that many people seem to confuse the ignorance, which is anything but valuable, with the innocence and wonder that happen to be associated with it in time.
As a parent, observing my child and other children, my current best understanding is that the importance of innocence is thinking and acting without feeling hemmed in and constrained by fears, and that the importance of wonder is the experience of how remarkable and marvelous our lives can be. It is much easier to achieve these states when we are ignorant, but we need not give them up merely because we become experienced. Rather, I think they are a critically important part of what makes life worth living.
Nor does knowing a truth prevent you from enjoying fiction. In my own experience, holding a very clear understanding of the distinction between fact and fiction can let you enjoy the fiction all the more because there is less need to be worried about whether it is true or how it may be affecting things. I even find it to be a good way to disempowering things that I am concerned about as a parent, like the Disney princess culture. If we try to hide from it or focus on criticism, that just gives it power as something that grownups are worried about. But if we embrace it, and acknowledge how fun the story is, but also help to remind and remember how it's just stories, and that stories are different from the real world, then it can draw the power out and turn literary criticism into a game that we can play together. Elsa is still an awesome queen with seriously cool ice magic, even if she'd be really sick if she was a real person with a waist that is so tiny.
The biggest difference, I find, in talking about fiction versus reality is that the "Why?" questions for fiction quickly ground out in questions of narrative structure. "In 'Moana', why did Te Fiti have her heart on the outside where Maui could steal it?" "They didn't give a reason. We can try to make up some reasons, but mostly it needed to be somewhere that he could steal it so that the story could happen." With reality, however, there is always something more, something outside, and ultimately things always ground out in an answer, and opportunity to look things up and learn, or, eventually, "That's a really good question, and we don't know the answer yet!" And then maybe we can talk about how we might try to find out.
The reality of the world is so intricate, and its systematicity so remarkable that I find wonder is never very far from the surface. Driving through the fields a couple of days ago, I noticed a microwave relay tower and pointed it out, and we talked about communication. I love the great big scoops, posed on top of their rectangular spike of concrete, I love that microwaves interact with metal in such a way that their circuitry often ends up looking more like plumbing than wiring. I love that the waves themselves are inches wide, while an FM radio wave is about three times my daughter's height, and the light that I see is a bit less than a millimeter. I love that our civilization has erected concrete towers standing high across the plains like a high-tech semaphore network, relaying messages in a network little different in principle than moving goods in trucks or messengers riding horses in centuries past. There are thousands and thousands of such edifices out there, and many people have been involving in building them, bringing together on each of those thousands of sites a whole assemblage of heavy machinery, cranes, truckloads of electronic equipment, and so on. The more I think about it, the more wondrous that grey square lump of concrete with the funny metal scoops becomes.
Some of my wonder and joy, I think, comes from the training that I have received as a scientist. Most, however, does not need much specialization or knowledge to appreciate, just an open mind and a willingness to experience one's ignorance and look for answers. I would not say to embrace or celebrate ignorance, but to simply recognize it as a state of knowledge that one can choose what one wants to do with. A lot of people write themselves off from knowledge, saying that they just don't have the right head or the right training. I think that often this may come from a discomfort at being ignorant, and I am saddened by it. I think that we will all be better off, both as individuals and as citizens of our civilization, the more that we are able to embrace our innocence as seekers of knowledge and connoisseurs of the wonder of the world.
Later on Christmas day, as we looked through the window at the rain coming down outside, my daughter mentioned that her teacher had said that it would rain on Christmas. I asked her how she thought her teacher had known, and she told me her teacher must have read a weather forecast. And then we all talked as a family about how weather forecasts use satellites to look down from way above the Earth so that they can see what's going on over a very big distance, and see what weather is coming towards us over the next few days.
My daughter's eyes grew wide with wonder, and she said: "That's even cooler than magic!"
I agree with her most strongly.
Monday, November 28, 2016
Reflections on iGEM 2016
This year was my third year to go to iGEM, the genetic engineering jamboree, and while my big news was the interlab study, in fact we had done most of the work on that in advance and as a result that occupied a relatively small portion of my weekend. Much more of my time was spent fulfilling my responsibilities as a judge, as well as parenting: this year I brought my daughter Harriet along, which was both awesome and challenging.
I also couldn't stay until the end, which I did on both of the previous occasions: the last day of the jamboree was Halloween, and at this age that's very important for me to be at home for. So I left Boston before dawn, and was lucky that my flight got in early enough for me to catch the awards ceremony on a cell-phone broadcast from one of the teams.
 |
| Harriet at the iGEM jamboree opening session |
I also couldn't stay until the end, which I did on both of the previous occasions: the last day of the jamboree was Halloween, and at this age that's very important for me to be at home for. So I left Boston before dawn, and was lucky that my flight got in early enough for me to catch the awards ceremony on a cell-phone broadcast from one of the teams.
Reflecting back, this year was very different for me from my previous two experiences. The first time I went, in 2014, was the first interlab study, my first experience as an iGEM judge, and my first time speaking in front of an audience of thousands. Last year, in 2015, I spent much of my time in an electrifying deep partnership of data analysis with my colleague Markus, as we sorted out bit by bit what the interlab study was telling us, along with the drama of having one of the teams that I had judged becoming a winner of the contest as a whole.
This year was... unexceptional, and I should not complain.
This year was... unexceptional, and I should not complain.
iGEM is a beautiful and unique event, and I'm glad to support and be a part of it. But I'm also a little bit sad to have experienced the expected regression to the mean.
Tuesday, November 22, 2016
Maturity, availability, pacing: the standards release process
The SBOL 2.1 update that I discussed recently had been in the works for nearly a year, practically since we released version 2.0. Today I want to talk about why we released SBOL 2.0 at that time, when we already knew we'd be updating a bunch of things in SBOL 2.1.
One of things that I've observed happens with standards, like any big project, is that there's always a lot of different changes in progress at different levels of maturity. This isn't a problem, but rather a sign of a healthy and growing area of work, where there are a lot of participants with different and evolving needs. This puts a three-way tension into development, between maturity, availability, and pacing:
Given this three-way dilemma, the best thing for us to do seems to be to set some sort of relatively regular schedule on which we plan our releases. We've got something of a natural pacing for this as a community, since there are SBOL workshops twice a year. There are always discussions going on in the community, on its mailing lists, between individual developers, amongst the leadership, etc. At the workshops, the issues raised and the details of proposed solutions get hammered out in more detail, and it there's clear progression toward consensus, these new steps forward can be formally adopted and written into the standard.
About once every six months, then, we're in a good position to assess whether there has been significant enough progress to plan for a new release. Then, when we've decided we need to make a release, every one of those changes-in-progress ends up going through a triage process, where we try to figure out if it's mature enough to actually get put into this particular release or if it needs to get deferred to the next one. Again, maturity versus availability: we end up with a push and pull back and forth between the desire to get more of these corrections and improvements in versus the need to actually make a release so that people can start officially using the things that are already finished.
So when we released SBOL 2.0, back in the summer of 2015, we weren't saying "the standard is done," but "the standard is good enough that we think it will be valuable for a lot of people." And part of making it valuable is actually deciding what to exclude, because including something that's not well developed is inviting problems, incompatibilities, and frustration in developers and users.
Instead, all of that stuff that we know we need to finish, or argue about, or even just plain contemplate, has ended up as issues in our online tracker. Over the next year, members of the community worked on some, argued about some, and ignored others, and over the course of a couple of workshops, we got to the point where SBOL 2.1 made sense---and found a nice external deadline as a forcing function too, which was also useful.
Now SBOL 2.1 is out, and after a short refractory period, the process has continued. People need to talk about permutations and combinations of DNA, circular pieces of DNA raise new and different ambiguities to resolve, nobody's really quite comfortable with certain things about proteins, and so on and on and on and on. Step by grinding little step, we'll work our way to SBOL 2.2, helping to work out these problems of description and communication that people from all around the synthetic biology community are dealing with. Maturity, availability, pacing, and an open invitation for more to join us in the work.
One of things that I've observed happens with standards, like any big project, is that there's always a lot of different changes in progress at different levels of maturity. This isn't a problem, but rather a sign of a healthy and growing area of work, where there are a lot of participants with different and evolving needs. This puts a three-way tension into development, between maturity, availability, and pacing:
- Maturity: the more polished and well-featured a standard, the better that it will serve its users.
- Availability: something that isn't approved and released doesn't actually serve anybody.
- Pacing: unlike a phone app or a web service, which can resolve maturity vs. availability with constant updates, a standard's users are tool developers, and every time we make a new release, we make new work for them.
Given this three-way dilemma, the best thing for us to do seems to be to set some sort of relatively regular schedule on which we plan our releases. We've got something of a natural pacing for this as a community, since there are SBOL workshops twice a year. There are always discussions going on in the community, on its mailing lists, between individual developers, amongst the leadership, etc. At the workshops, the issues raised and the details of proposed solutions get hammered out in more detail, and it there's clear progression toward consensus, these new steps forward can be formally adopted and written into the standard.
About once every six months, then, we're in a good position to assess whether there has been significant enough progress to plan for a new release. Then, when we've decided we need to make a release, every one of those changes-in-progress ends up going through a triage process, where we try to figure out if it's mature enough to actually get put into this particular release or if it needs to get deferred to the next one. Again, maturity versus availability: we end up with a push and pull back and forth between the desire to get more of these corrections and improvements in versus the need to actually make a release so that people can start officially using the things that are already finished.
So when we released SBOL 2.0, back in the summer of 2015, we weren't saying "the standard is done," but "the standard is good enough that we think it will be valuable for a lot of people." And part of making it valuable is actually deciding what to exclude, because including something that's not well developed is inviting problems, incompatibilities, and frustration in developers and users.
Instead, all of that stuff that we know we need to finish, or argue about, or even just plain contemplate, has ended up as issues in our online tracker. Over the next year, members of the community worked on some, argued about some, and ignored others, and over the course of a couple of workshops, we got to the point where SBOL 2.1 made sense---and found a nice external deadline as a forcing function too, which was also useful.
Now SBOL 2.1 is out, and after a short refractory period, the process has continued. People need to talk about permutations and combinations of DNA, circular pieces of DNA raise new and different ambiguities to resolve, nobody's really quite comfortable with certain things about proteins, and so on and on and on and on. Step by grinding little step, we'll work our way to SBOL 2.2, helping to work out these problems of description and communication that people from all around the synthetic biology community are dealing with. Maturity, availability, pacing, and an open invitation for more to join us in the work.
Friday, November 18, 2016
The Living Computing Project
Yesterday, I spent the whole day immersed in the engineering challenges of computing with living organisms. The occasion was an internal team workshop on the Living Computing Project, a really cool NSF-funded project involving a whole bunch of synthetic biologists at MIT and BU.
What makes this particular project so unique is that it's not tied to particular applications, but instead lets us really focus on the foundational questions of how to store, process, and communicate information inside of living cells. In this project, we get to build the engineering models and tools that will enable all sorts of different applications, and I get to play a sort of "metrics czar" role, integrating lots of different elements across the project as a whole, as well as connecting our work on the project with work being performed in other organizations.
We've been working on this project for nearly a year now, and in our meeting yesterday, I saw a number of places where things are really starting to gel intellectually. I'm excited to have people interested in getting their units right, and about getting precise and predictive models that can let us know right away whether something that we want to build is likely to be feasible and to go right to the designs we want. I'm also very excited about the potential in some of the biological devices that we are working on and with.
I like to have my research be applicable and have a clear use story. But sometimes, it's also nice to just be a engineering scientist and work on the tools we get to use in building those applications too. I'm happy that we're working with folks at NSF who share that vision and understanding, and have been willing to give us some rope to go and work on the foundations of the field.
What makes this particular project so unique is that it's not tied to particular applications, but instead lets us really focus on the foundational questions of how to store, process, and communicate information inside of living cells. In this project, we get to build the engineering models and tools that will enable all sorts of different applications, and I get to play a sort of "metrics czar" role, integrating lots of different elements across the project as a whole, as well as connecting our work on the project with work being performed in other organizations.
We've been working on this project for nearly a year now, and in our meeting yesterday, I saw a number of places where things are really starting to gel intellectually. I'm excited to have people interested in getting their units right, and about getting precise and predictive models that can let us know right away whether something that we want to build is likely to be feasible and to go right to the designs we want. I'm also very excited about the potential in some of the biological devices that we are working on and with.
I like to have my research be applicable and have a clear use story. But sometimes, it's also nice to just be a engineering scientist and work on the tools we get to use in building those applications too. I'm happy that we're working with folks at NSF who share that vision and understanding, and have been willing to give us some rope to go and work on the foundations of the field.
Tuesday, November 15, 2016
iGEM 2016 Interlab Presentation Posted
For those who may be interested in more details about our highly successful fluorescent measurement experiment at this year's iGEM: I have now posted the slides that we presented at iGEM in the interlab workshop. These slides contain:
- details on the setup of the experiment
- a discussion of the problems that we encountered and lessons learned
- a sampler of the data and key results
The slides can be accessed at the top of the 2016 iGEM interlab page or via this direct link.
Complete details and raw data will be presented in the paper, once that is available. As of now, it still only just beginning to be written, but I will post news as it becomes available.
Friday, November 11, 2016
A few words on the election, as an American scientist
Although I usually stay strictly away from politics in my online (and thus professional) presence, there is an elephant in the room, and I need to address it before this blog goes back to my regular programming.
On the morning after the election, I had email from my European colleagues expressing sympathy and concern. Others wondered whether the declared plans of our incoming administration meant that organizing and attending conferences in the US was going to become problematic. To them I sent the reply that I feel best describes our situation in general at the moment:
Donald Trump has made a lot of promises, many of them frightening to many people, many of them contradictory, and some of them almost certainly impossible. The presumptive Republican leadership in Congress has done the same. What happens when they actually attempt to govern could go in a lot of different directions, some of them reasonable (even if likely to be things that I disagree with), some of them extremely dangerous and damaging to a large number of people, and some of them potentially existentially threatening to our civilization and possibly even our species. Whether you agreed or disagreed with Hillary Clinton's promises and plans, they certainly offered a much more predictable course than I see playing out before us now, and high degrees of uncertainty have serious dangers of their own, given our species' current technological capabilities.
As a citizen, I intend to take actions to try to ensure that my government effectively supports myself, my family, and every other person in our nation and on our planet, to grow and thrive and live long and stable lives in an environment of sympathy and positively chosen peace. I also intend to make reasonable contingency plans in case some of the bad scenarios that I now see as more likely come to pass.
As a scientist and a professional, my intention is to continue to do my best to contribute in the areas where I have unique or rare skills, abilities, and insights. I will do my best to advise my government and will continue to choose subjects for research and development that I think are likely to have more benefit than not for humanity. Even as people were voting, I was submitting a proposal addressing a subject of potential great concern, and I hope that I will have a chance to execute on that and help to reduce some of the risks that I see out there.
The future is highly uncertain to me right now, and one of the truths that my scientific honesty impels me to acknowledge is that my personal choices and actions are unlikely to make a significant difference to our future, just the same as it was a week ago. Yesterday afternoon, my father and I walked together on a bed of fossils from and ancient sea floor, laid down 375 million years ago, talking about our lives and finding ancient corals, brachiopods, and sponges. Experiences like that are a valuable reminder to me that I am very small and have been here only a very short time. At the same time, as a student of complex and self-organizing systems, I know that individual choices do matter, and that the actions of individual people in networks can indeed have a major impact on the world in which we live.
My dear readers, I urge you to think carefully about your choices and actions, and to act with as much empathy and care for your fellow humans as you can, no matter who you supported, where you live, or what your political inclinations. I urge you this now, as I would have urged this before as well and will continue to do so no matter what may happen. It is only that now, in this time of great chance and uncertainty, that I feel impelled to do so publicly as well as personally.
On the morning after the election, I had email from my European colleagues expressing sympathy and concern. Others wondered whether the declared plans of our incoming administration meant that organizing and attending conferences in the US was going to become problematic. To them I sent the reply that I feel best describes our situation in general at the moment:
As a citizen, I intend to take actions to try to ensure that my government effectively supports myself, my family, and every other person in our nation and on our planet, to grow and thrive and live long and stable lives in an environment of sympathy and positively chosen peace. I also intend to make reasonable contingency plans in case some of the bad scenarios that I now see as more likely come to pass.
As a scientist and a professional, my intention is to continue to do my best to contribute in the areas where I have unique or rare skills, abilities, and insights. I will do my best to advise my government and will continue to choose subjects for research and development that I think are likely to have more benefit than not for humanity. Even as people were voting, I was submitting a proposal addressing a subject of potential great concern, and I hope that I will have a chance to execute on that and help to reduce some of the risks that I see out there.
The future is highly uncertain to me right now, and one of the truths that my scientific honesty impels me to acknowledge is that my personal choices and actions are unlikely to make a significant difference to our future, just the same as it was a week ago. Yesterday afternoon, my father and I walked together on a bed of fossils from and ancient sea floor, laid down 375 million years ago, talking about our lives and finding ancient corals, brachiopods, and sponges. Experiences like that are a valuable reminder to me that I am very small and have been here only a very short time. At the same time, as a student of complex and self-organizing systems, I know that individual choices do matter, and that the actions of individual people in networks can indeed have a major impact on the world in which we live.
My dear readers, I urge you to think carefully about your choices and actions, and to act with as much empathy and care for your fellow humans as you can, no matter who you supported, where you live, or what your political inclinations. I urge you this now, as I would have urged this before as well and will continue to do so no matter what may happen. It is only that now, in this time of great chance and uncertainty, that I feel impelled to do so publicly as well as personally.
Thursday, November 03, 2016
SBOL version 2.1
Version 2.1 of the Synthetic Biology Open Language (SBOL version 2.1) has now been officially released, improving our means of representing biological designs. In addition to a bunch of little technical changes useful for implementation, the key changes with this release are:
I have some musings on the standards release process that I'd like to share later, but for now I just want to cheer and encourage people to update if they're using SBOL and consider adopting if they aren't yet.
- You can describe how the nature of a biological component changes when it's used as part of a larger design.
- It's simpler to mark up features of a design.
- We filled in a bunch of missing terms for describing interactions between components.
- You can describe the topology of components (e.g., circular vs. linear DNA)
I have some musings on the standards release process that I'd like to share later, but for now I just want to cheer and encourage people to update if they're using SBOL and consider adopting if they aren't yet.
Sunday, October 30, 2016
Ladies and Gentlemen, we have our ruler
Dear readers, as promised earlier this week, it is my distinct pleasure to share with you today the headline results of this year's iGEM interlab study. These are preliminary results that we are still writing up for publication, but I feel that it is too exciting to keep quiet, and so I am shouting it from the rooftops today.
In synthetic biology, one of the best tools for studying the behavior of cells is fluorescence. Unfortunately we haven't had a good, accessible way of quantifying fluorescence, so you couldn't compare results from one lab to another, or even necessarily from one experiment to another within the same lab. In short: we've needed a good ruler for measuring fluorescence.
The goal of the iGEM interlab studies has been to understand where the problems in measurement are coming from and then to use that knowledge to produce a good ruler. In the 2014 and 2015 interlab studies, we figured out that the big source of the problems didn't seem to be the biology, but how people were using their instruments and comparing their data. That was actually good news, because we had some ideas for how we might be able to fix that, and this year we tried them out. We gave every team two simple non-living calibration samples to compare their biological samples to, and hoped that this would tighten up the numbers some.
The results we got were beyond my wildest dreams.
Here's what we saw for the precision of measuring fluorescence with plate readers. We compared the standard deviation of the arbitrary unit measurements from 2015 with the calibrated measurements from 2016 before and after using the positive and negative controls for quantitative filtering of problematic tests, and got these standard deviations:
But to really wrap your head around how big an improvement this is, you have to think about the fact that the thing that we are measuring is geometric standard deviation, for which the units are in times multiplied or divided. In general, we would consider the normal range of values to expect from a measurement to be within two standard deviations up or down from whatever the real number is---i.e., 95% of the time, a measurement should lie within that range. With a geometric standard deviation of 35, two standard deviations up is 35*35 = 1,225. That's more than a thousand. Going down is another multiple of more than a thousand, meaning that all told, we would expect measurements to be accurate within a factor of a million or so. Obviously, that's rubbish. It's hard to do anything if you expect your measurements to be wobbling around by a factor of a million.
This year's measurements were more than 100,000 times more precise.
Here's the thing, though: remember that only about half of our improvement came from using the same units, while the rest came from being able to identify failures by the strange behavior of their controls. We would thus expect to see significant improvement in precision this year versus last year, and indeed that is what we saw:
On average, normalized measurements were 70 times more precise.
The flow cytometry results are not quite as dramatic, and not as statistically strong since many fewer teams to took flow cytometry data, but the basic result is the same: orders of magnitude improvement in precision of both individual measurements and of normalized measurement.
There's a lot more to do, to go from initial results to routine and effective usage, but I believe the core results are clear:
- We've got a workable ruler for fluorescence. It's not perfect, but it's orders of magnitude better than current practices.
- If undergraduates and high school students from all around the world can use these methods, there's no reason they can't be adopted in every biology laboratory that measures fluorescence.
Sunday, October 23, 2016
Never work without a net: why units matter
I've been working on measurement and units in synthetic biology for more than five years now, so it would seem that I should have a pretty clear understanding of the landscape. As you, dear reader, may recall, for a long time I've been arguing in favor of getting independently calibrated units into our work in synthetic biology, and working on ways to make this generally accessible. Over the past 24 hours, however, I've come across something that has blown my mind.
The arguments for having units that you can compare across different experiments, devices, and laboratories have been pretty strong and clear, since yes, of course, we want to be able to compare our work. Many people, however, believe that it's good enough to have relative units, where you measure in arbitrary units and then normalize your data by a known genetic construct control. I have not been comfortable with this, because you have no way to know if something goes wrong that affects your control as well.
My arguments there, however, have always felt relatively weak, because: a) how do I know if this actually happens often enough to be a real concern? and b) it sounds like I'm accusing people of doing sloppy lab work, which would be doubly unfair since most scientists I know are quite careful and since I don't do any lab work at all. So while I've had a clear argument against relative units that it persuasive to those who are already basically in agreement, I haven't really had a leg to stand on, scientifically speaking, in my concerns about relative units, and have sort of dismissed it down to a level of secondary concerns.
But in the data from this year's iGEM interlab, we have hard evidence that relative units are not enough, because having a way to catch the results of those little mistakes really matters. I can't give you any more details for a week, not until after we officially unveil the results next week at this year's iGEM Jamboree, but it's a big deal. Like, orders of magnitude big deal.
My world is rocked. It's obvious in retrospect, and I've even made the argument before. There's a difference, however, between making an argument and having data staring you in the face that says that the argument is far more important than you ever had actually realized.
Basically, if you're not using units, then you're working without a net. With units, you have a chance to apply a bit of experience and common sense and realize that something is going wrong with your numbers. You might not know how or why, but usually that's not actually important, because usually it's something small and stupid (dropped a minus sign, got left and right mixed up, grabbed the wrong bottle, etc.), and the best way to fix your mistake is just to do it again, because you probably won't make the same stupid mistake twice in a row. This applies to pretty much everything in life involving numbers and measurements, not just to biological research. Using properly calibrated units gives you a second chance to notice your mistake, and makes all the difference between embarrassment and disaster.
How big a difference, exactly? Well, I'll just go grab a cake to celebrate and I'll see you in about a week.
The arguments for having units that you can compare across different experiments, devices, and laboratories have been pretty strong and clear, since yes, of course, we want to be able to compare our work. Many people, however, believe that it's good enough to have relative units, where you measure in arbitrary units and then normalize your data by a known genetic construct control. I have not been comfortable with this, because you have no way to know if something goes wrong that affects your control as well.
My arguments there, however, have always felt relatively weak, because: a) how do I know if this actually happens often enough to be a real concern? and b) it sounds like I'm accusing people of doing sloppy lab work, which would be doubly unfair since most scientists I know are quite careful and since I don't do any lab work at all. So while I've had a clear argument against relative units that it persuasive to those who are already basically in agreement, I haven't really had a leg to stand on, scientifically speaking, in my concerns about relative units, and have sort of dismissed it down to a level of secondary concerns.
But in the data from this year's iGEM interlab, we have hard evidence that relative units are not enough, because having a way to catch the results of those little mistakes really matters. I can't give you any more details for a week, not until after we officially unveil the results next week at this year's iGEM Jamboree, but it's a big deal. Like, orders of magnitude big deal.
My world is rocked. It's obvious in retrospect, and I've even made the argument before. There's a difference, however, between making an argument and having data staring you in the face that says that the argument is far more important than you ever had actually realized.
Basically, if you're not using units, then you're working without a net. With units, you have a chance to apply a bit of experience and common sense and realize that something is going wrong with your numbers. You might not know how or why, but usually that's not actually important, because usually it's something small and stupid (dropped a minus sign, got left and right mixed up, grabbed the wrong bottle, etc.), and the best way to fix your mistake is just to do it again, because you probably won't make the same stupid mistake twice in a row. This applies to pretty much everything in life involving numbers and measurements, not just to biological research. Using properly calibrated units gives you a second chance to notice your mistake, and makes all the difference between embarrassment and disaster.
How big a difference, exactly? Well, I'll just go grab a cake to celebrate and I'll see you in about a week.
Subscribe to:
Posts (Atom)