Personal comment on the infamous letter to astronomy grad students

If you are in the world of astronomy, you probably have heard or read about the infamous letter written by senior staff of an unnamed “famous” astronomy department sent out to their grad students (though it is not difficult to find out which department sent out the letter). If you don’t know what I’m talking about, please read the “inspirational” e-mail here.

The first reactions and comments were naturally ones of outrage (see e.g. astrobetter). There were tweets and facebook posts that were shocked and indignated, mostly on the fact that we, as astronomers, should be working on the subject all of our waking hours and expressing “Thank you, sir, may I have another“. I think that the main source of the scorn was the claim that you have to work 80-100 hours per week on astronomy as a grad student if you want to have a succesful path as a researcher.

First of all, I don’t think anybody can work that long over prolonged periods of time (there’s even evidence for that). As an exception, because there were so many deadlines, I worked 65 hours last week and feel tired today. I hardly saw my family last week and am pretty unhappy about that. Now add 15 hours to that and I would feel downright atrocious.

However, I feel like that putting a number to the hours you work in a week missed the point completely. I am especially sad that the letter put this point first without any nuance or diplomacy – a great way to discourage your students. I can smugly say that even though my advisor was trying to impart similar wisdom to me about working hard, his way of doing it had a much more positive effect on me. Let me veer a bit into personal anecdote time here:

I found out in 2005 I was pregnant. The timimg is never perfect. I had just gotten the Lawrence Fellowship for Graduate Students. Ironically, while that made my measly grad student salary multiply by about 1.5 times, it carried a pretty horrendous health insurance making me basically pay for a big part of the costs during pregnancy and birth, basically negating all the gains in salary (heh!). Anyway, there I was, freshly married and quite frankly scared of telling my advisor that I was about to devote a large part of my life to a new being growing inside of me.

He couldn’t have been any more supportive. First questions were of the lines of “Are you ok?”, but then he did the smart thing, appealing to my ego: “You’re not thinking of leaving astronomy, are you? That would be a big loss! You can make a great scientist. You can make it happen”. Anyway, I left that meeting with a big sense of confidence, that yes, I could make it happen! He didn’t sugarcoat anything, he never said I could succeed by just being half as productive, but he saw the potential that it was doable.

Fast forward about 14 months. I was on my first trip since having my daughter. We were observing at Keck. Funnily, many people complain that they feel tired all the time while observing, but let me tell you, with an 8-month old at home, I felt refreshed and slept like I hadn’t in months, i.e. contiguous 6 hours. heh! Anyway, it was September of 2006 and I was supposed to be graduating less than a year from that date. I had produced just one paper (my deal with my advisor had been three) and I was hitting a roadblock with the analysis of the two big papers. For the Hubble paper, I feel like I wasn’t getting the fitting program, how to tweak the parameters, etc. I just wanted to scream at the world: “Look at the pictures! They tell all the story, what do I need to analyze here!?!?!”.

So we were driving back from a pretty unsuccessful run. We had just gotten a few hours of data out of 2 or 3 nights, I forget. I was feeling down, it was just too hard, the baby, the roadblocks, the stupid clouds. So somewhere on the road between Waimea and Kona my advisor asked me if I really wanted it, that the time and commitment I was putting in wasn’t gonna cut it. Especially not when I was doing other things than the analysis needed for the papers. “It’s all about decisions, Tanya. The person observing after you tonight went diving today. It is a nice thing to do, especially if you’re in Hawai’i. But to really want that job in astronomy, you will need to give up some nice things sometimes.”

All about decisions. That point really resounded with me. I can’t tell you that suddenly everything worked out fine. But I worked hard and by November, close to Thanksgiving I had *the* analysis, *the* main point of the Hubble paper done. There’s even a funny story about my excitement over that result. I had gotten the correlation with which my most disturbed objects were also the most reddened ones on the Thursday before Thanksgiving. I know because the next morning was a “Baby and me” class that I attended every Friday (those who know me will cringe, since they already know this story). We were asked around the circle to say what we were thankful for. I was to go first and happily blurted out that I had just found out the last piece of the puzzle of the paper and that I would be one step closer to graduating. Oh man, that was the wrong answer in that group and I only realized it slowly after every. single. mom. was thankful for having a healthy baby. Ooops!

I did not get a job for the 2007 job season (but only half a year later), that would’ve been too convenient of a story, wouldn’t it have been?. But I can tell you that 2007 was a year which I did focus on work, very much. I shooed colleagues away wanting to come in and gossip, the hours at work were too precious. It was about decisions. The fact that I did not even know what was playing on the TV during those years (I know my friends were obsessed with Lost!, heh), what was going on in the news. I may not have been the best paper reader, may not have had the best code written, it was sometimes hard going at it a bit alone (not completely alone, of course, thank you, Mark!). But the focus was there, the motivation was sparked that kept me going, I was gonna to whatever it took to get that PhD and hopefully that postdoc job!

Mind you, I didn’t suddenly work some magical fixed number of hours a week or spent my time at home with my baby on one arm and the laptop in the other. Rather, and it sounds weird to say it like that, I became more efficient. I most definitely missed some things. I couldn’t have done it without the enormous help and encouragement of my husband. But I can’t explain it better than I was really into everything astronomy during that time (as I feel like I have been this year, too). And it’s like an avalanche, it just multiplies and gets bigger.

Anyway, boy did I veer off the point. It was about this fixation on the number of hours worked, when in reality you should get away from this thinking. It’s about the focus, the flow and the results which you should measure. And thankfully the posts and blogs that have come out since have been a bit more nuanced and addressed much more eloquently what I was wanting to say. The two articles I liked I liked the most were:

a) The post by Elizabeth Lovegrove over at astrobites. Its main point is that there is no clear line where work ends and when fun begins and that we should step away from the computer once in a while. That doesn’t mean that you suddenly should seek out weird ways of doing science in the cafeteria, but rather that we are not in a factory where some fixed amount of product needs to be produced in a certain amount of time. It is science, insights don’t come at the n-th hour worked on the subject.

b) Another amazing blog on the subject was written by Lucianne Walkowicz. It’s downright inspirational for any field you work in. Right in the beginning she tackles the issue about the number of hours worked with this simple sentence: “If you are counting how many hours a week you are working, you probably don’t like your job very much.” Exactly! I couldn’t have conveyed it better. I would suggest you just sit down and read the whole article, it deals with health, goals and a whole bunch of other things that I was just nodding along to.

Lastly, I do want to say that I hope if anything good comes out of this discussion is: yes, it’s about decisions, but there are a few things that you should not decide away – it is your health, be it physical or mental and your family and close friends! Now go on, keep on truckin’!

Econtalk podcast with Brian Nosek

I have a long commute to work. It takes me about 90 minutes each way if I am taking public transportation, about an hour if I take the car. There are some personal reasons on why I live so far away from work, but I am neither sad or stressed about it. Those 2-3 hours I spend on the road are wonderful times in which I either get caught up on my TED Talks, Jon Stewarts and Colbert Reports or in which I listen to podcasts when I’m in the car.

One of these podcasts is Econtalk with Russ Roberts. It deals with more than the pure money economics that is “Planet Money“, but is way more scientifically rigorous than “Freakonomics“, which is more entertainment than anything else. The podcast is quite libertarian, with a healthy does of skepticism towards the government, but it isn’t preachy or even biased. Russ does a good job of moderating the issues, he’s always well prepared and I like his insights into all aspects of economics. He’s made me be a believer of Hayek (and Mieses to some extent) and appreciate the nuances of Friedman’s policies. His podcasts with Mike Munger are entertaining, but some of the most conflicted I have been when hearing a podcast.

And because econtalk is about the social sciences in general, not just about finance, a few weeks ago he had Brian Nosek on to talk about his latest paper: Scientific Utopia: II – Restructuring Incentives and Practices to Promote Truth Over Publishability. I was very impressed with this podcast not only because of the interesting topic, but because I saw many traits of this among my astronomer friends, even though we are in the hard, natural sciences, not in the social sciences. If you can spare an hour, I implore you to listen to the podcast, I think it is very important to be aware of our unconscious biases when we are doing science and finding publishable results.

The main and salient point that Nosek raises is that we as scientists try to genuinely do good science but that does not mean that we aren’t vulnerable to some of the reasoning and realizations that leverage the incentives to be successful at science. Read: we need to make sure to get published, so we have a hidden bias to pursue publishable results (journals will not publish null results). I mean, if I get data at a telescope, I’m expected to publish a paper on that data. To massage some scientific insight out of there can often be quite difficult, but the incentive to publish is there.

Russ reads aloud two of his quotes and I think they are so good that they bear repeating:

The real problem is that that the incentives for publisheable results can often be at odds with the incentives for accurate results. This produces a conflict of interest. The conflict may increase the likelihood of design, analysis and reporting decisions that inflate the proportion of false results in the published literature.

Publishing is also the basis of a conflict of interest between personal interests and the objective of knowledge accumulation. The reason? Published and true are not synonmyms. To the extent that publishing itself is rewarded then it is in scientist’s personal interest to publish regardless of whether the published findings are true.

The sentence that published and true are not synonyms is the heart of the problem and it is a pretty depressing idea, too. We like to see ourselves as the truthseekers, but personal interests can derail this. Incentives matter, certain kinds of results are valued more than others. We are subtly influenced to take the path that is most beneficial to our career, i.e. seek out results that I can publish. But does that push our analysis in a certain way?

Nosek identifies 9 tricks or “things we do” where our bias shows:

  • – leverage chance by running many low powered studies than a few high powered ones.
  • – Uncritically dismiss failed studies as pilots or due to methodological flaws but uncritically accept successful studies as methodologically sound
  • – selectively report studies with positive results and not studies with negative results every time
  • – stop data collection as soon as a reliable effect is obtained
  • – continue data collection until a reliable effect is obtained
  • – include multiple independent or dependent variables and report the subset that worked
  • – Maintain flexibility in design and analytical models, including the attempt of a variety of data exclusion or transformation methods for to subset
  • – Report a discovery as if it had been the result of an exploratory test
  • – once a reliable test is obtained, do not do a direct replication; shame on them.

Now you may say, Tanya, that is all fine and dandy, but it only applies to the social sciences. Well, unfortunately no. I have seen the above over and over again in astronomy, too. When some observation does not agree with the model the scientists who proposed suddenly dismisses it, many projects are lying around in drawers, because “the discovery” didn’t pan out. But even worse, when an observation begins to even show signs of agreeing with the proposed, it gets sent to a prestigious journal, even if there isn’t critical statistical mass – hey it’s a pilot study, but it’s *really* important!

What do we do? Fix the journal / peer review system? Journals have own sets of incentives that encourage publications. They want attention and prestige, be at the forefront of innovations. So can’t do that. Should we maybe fix the university system? Can’t do that either, because here we face the same challenges. The way that universities gain prestige is the same as journals, so they don’t have the incentive to change either. Nosek suggests to rather start from the bottom up, from the practices of the single scientist to be accountable rather than from the top.

At the boundaries, we will have risk, we will be wrong a lot. That’s what science is supposed to do. But what happens is that by that design the discovery component suddenly outweighs the other side of science which is verification, which is kind of boring. Verification is just as important, but not as exciting! We value innovation more. Just last week this was my facebook status update:

“We had a big discussion at our conference dinner yesterday about science, its history and its impact. Yes, there was wine flowing :). One person in the group stated matter-of-factly “the biggest driver in science is discovery!”. Fortunately, I was not the only one who disagreed and vehemently at that. We need to get away from this thinking – science is incremental! It’s important that we assign glory to the people checking results, improving statistics, interpreting data, improving on some technique etc. as to the people who discover something “first”!” with a link to an ode to incremental research.

By the way, the person in the group is a brilliant scientist, I was rather perplexed by this attitude, but I am finding more and more, that it is prevalent among astronomy. We will face a big challenge is in trying to rebalance this. Can we find some incentives to make verification more interesting and provocative? It’s gonna be difficult. By the way, verification is not attacking the original scientist, disagreements over a subject does not mean that science is broken, that’s actually how science works! You gather some evidence here, somebody tries to disprove it there and then you converge on a result. Well, if that’s how science works, won’t it self correct? Isn’t then discovery the main driver still? Only eventually; it takes a very, very long time! That’s not ok! We can correct and clarify quicker and more efficiently.

So what are the suggestions to counter that thinking. In the podcast they delve into a nice anecdote about a study that was about people acting slower whenever they heard about old age. When the study couldn’t replicate, the original scholar just scowled: “well, you didn’t do it right!”. In the original paper they even reproduced the result, so obviously there was some methodology that was giving them those results. They are still talking about psychology, but even in astronomy, we have our way of doing things. What is a speck, some background variability to somebody is a Lyman alpha blob at a sigma of 2.5 at extremely high redshift to somebody else. Some people have great observing ability, positioning the slit exactly right, some are masters of getting that last little photon out of the data. But when is the science gotten from that photon reproducible? There is often so much subtlety and nuance! Failing to replicate also does not mean that the original result is false!

So the recipe is actually quite simple: be accountable! Describe each detail of the methods. I like the approach that people are putting their scripts now online via the VO or github (e.g. to really make their methods transparent and accessible. Some people provide diaries to their colleagues / collaborators on what they do (I know, I have and I find the ones of others extremely helpful as some sort of cookbook), if we could make this even more open it would be great – documenting your workflow is of real value! Every researcher does more research than he/she actually publishes. There may be diamonds in the rough there if all that data is open as opposed to the biased representation in the published literature.

This way of opening up methods also raises and could correct another thing: when you write a proposal, you already have the expectation of a confirmatory stance. So to correct for the strong expectation of a result you can present the tools you will use to analyze the data. This is why simulations of data in proposals are encouraged. When the data comes in, all you have to do is to run it through the already developed tools and just confirm or deny your hypothesis without fiddling, adjusting or even fudging the output. Register the analysis software in advance, it reduces your degrees of freedom, but makes you fairer, even if the data may not look as pretty. Better yet, have two competing camps work on an analysis script together!

Last, but not least, there is some fame and glory for shooting studies down. But only for the really famous ones. For the simple data analysis paper, we are often met hesitantly: “Why do you feel the need to question that result?”. But it actually is quite doable for high impact studies. This makes it so that one doesn’t constantly replicate things and will actually discover things, too, but it at least makes sure that the most “high impact” results are validated.

These are issues that have been of concern for years. Scientists don’t want to waste their time on things that aren’t true, so obviously they want to get at the heart of the problem. It’s actually great that people are looking critically at the research methods that scientists use. Even just knowing that we might have a skewed view in published results is valuable in it by itself. So if you made it this far, I have now made you aware of even another unconscious bias we carry around and need to account for. I leave you with a funny comic from xkcd, that a commenter on the podcast linked to – very relevant to the discussion.

Astronomy as a profession

This is my first post on my blog, but it definitely entails a few things I’ve always wanted to vent out into world during my astronomy career. It was sparked by a post made by a postdoc into the facebook astronomer group that she was “quitting astrophysics”. In the post she sounded off about the field brutal and contributing little or nothing to science. Of course the work-life balance issues, especially relating to young mothers came up, albeit as a PS. It garnered quite a lot of feedback, most of it sympathetic and I will talk about the problems we have in astronomy a bit in this post. However, I will reserve my views about women in astronomy (kids, two body problem, etc.) for a later discussion, so as to not clutter up my train of though in this post.

So, a job in astronomy… yep, it’s hard to get. There are about 10,000 professional astronomers in the world give or take a few thousands depending on your definition of astronomer. Looking at it from that angle, of course it is difficult, there are far more people interested in astronomy in the world than the 10,000. Collegetimes actually took this argument ad absurdum and published it under “The 11 hardest jobs to get in America“, because there are only about 50 (tenure track) positions advertised every year.

Of course, being president of the United States also ranks as “profession” in their list and herein lies the point. There are some professions in this world that pass as dream professions for some. Many kids want to be professional athletes when they are small, but the ones that actually make it are few and far in between. My daughter at the moment wants to be a rock star / dancer. I do think astronomy is at the boundary of those dream professions, in that for some it’s a dream, they wanted to explore the Universe ever since they were kids and for others they kinda slipped into it during the later years of their studies.

This is why the facebook post irked me so much. The second sentence read: “I find the astronomy world to be a brutal place where dreams go to die.” It has been quite the opposite for me, actually and it is the dream aspect that has been keeping me going (along with encouraging words from mentors and friends, of course). I can always count on astronomy, the idea of learning about the Universe, to keep me going when it gets tough (when the 7th proposal is rejected in a row, when the paper is going nowhere, when you feel there is no job out there for you, …). Videos like this or this make me feel privileged to be part of the endeavour of finding out our place in the Universe and how it came to be. The times when I see the wonder of my students looking through a small telescope or just saying “whoa!” when they got a concept on the blackboard remimd me that I came from the DREAM side to this profession. And the dream is what keeps me going!

But enough about ranting about what makes astronomy so great, otherwise I would never shut up. Let’s get to the bad part. Unfortunately, if you want to someday hold a secure position working in research in astronomy you will have to compete with a lot of people that hold that dream and have already overcome many steps and challenges to get to vie with you for that position. The numbers are roughly the following (this is from memory from an AAS presentation from about 3 years ago, I’m only paraphrasing): Roughly speaking for every qualified applicant, there is a postdoc position. however, the position might not be the position you were originally looking for, so roughly the ratio of *willing and enthusiastic* applicants per position is about 2:1. I don’t know if that is good or bad, scary or not, but it certainly leaves me scrambling on the rumor mill pages during upcoming job seasons a lot – even with a job, heh! The situation becomes bleak when it comes to permanent positions, the ratio rises to about 4:1 and even 10:1 when it comes to top tier research schools tenure-track positions. Those are some daunting odds that rank right up there with Hubble Observing time success chances.

Would you be willing to set your personal life with all that it entails (moving, partner, kids, friends, “stuff”…) for a 10:1 shot? One that isn’t even guarateed to work out (you get denied tenure)? One where you feel that it is a closed club to enter, so why even bother to try?

I don’t know what your answer is to these questions. For me it was something different in that the questions were kinda worded wrong. In the facebook thread somebody mentioned Cecilia Payne-Gaposchkin’s quote on the matter and I felt in summarized my feelings on the matter perfectly: “Do not undertake a scientific career in quest of fame or money. There are easier and better ways to reach them. Undertake it only if nothing else will satisfy you; for nothing else is probably what you will receive. Your reward will be the widening of the horizon as you climb. And if you achieve that reward you will ask no other.” The reward is not to sit in a nifty professor’s University office, the goal is your research and I’m glad I’m part of it.

It’s the same as the xkcd comic on this page. Trying to model yourself after that great professor will most probably lead to disappointment. You do astronomy, because you love astronomy, so do what you love. Even though it’s circular logic, it’s nice to be reminded of it when you are swearing at the computer, because some code is not working. When I can sound excited about the research that I do and for that time forget that I don’t have a permanent position. Who cares about the rumor mill? I get to observe with great telescopes, travel to interesting locations and learn about how stuff works in the Universe. And I do it, because I want to, not because it’s some job

Of course I can’t feel that way always, it would be unnatural, so for those times tell myself: “Yes, it’s hard to win the lottery, but if you never play, then you will never win!” 🙂