We need to become better programmers

 

For a long time programming has been a “means to an end” kind of thing. Like math, I was never really interested in full derivation of the proofs of the theorems we were doing in class. I just did the math and I was damn good at it, if I might say so, but I don’t think I would be happy being a pure mathematician. Similarly, I enjoyed my programming classes in college very much. It was sort of a game for me who would write the best (efficient, working) programs with the least amount of lines. I always enjoyed that challenge.

As such, when it finally came to research, I’m happy to say that I did have the tools to tackle the daily tasks in programming presented to me. I had the techniques of algorithmic approach in me to take on large databases, for example. It was only a few years later that I started to realize that my programs were pretty bad. Oh, they worked perfectly and fast, but no person but me could actually use them (and maybe not even me 5 years from now). They were horribly documented, were unreadable and required the input data to be in an extremely narrow way specified by me. I realized this when I started to publish and people were starting to e-mail me asking me how I calculated the Gini coefficient or Balnicities or similar. It dawned on me that I would be a lousy programmer at a software company or for everything that involved large collaborations.

This was very present in my mind when I came to Germany to work on the Data Reduction Pipeline for MUSE. I was (and sometimes still are) scared that people were going to dismiss my work, since it was documented so lousily. I’m actually quite happy that I have been tasked with the documentation of the pipeline, because that is allowing me to approach this very rigorously keeping in mind that everybody in our collaboration should be able to use the software. In german, we have the acronym DAU – “Dümmster anzunehmender user” (dumbest possible user) as a wordplay on GAU – “Größter anzunehmender Unfall” (greatest possible accident) from nuclear power stations. It relates to program “usability” in general.

I was then quite excited when recently a paper was published on “Best Practices for Scientific Computing” by Aruliah et al. It exactly describes the problems that are facing us astronomers more and more. In a world were author lists are growing, survey data often out-trumps single observations and ever larger and more complex observing instruments, we are increasingly faced with programming – and not easy peasy programming at that.

A lot of my fellow colleagues think that ever astronomer nowadays should take at least one year sabbatical to *really* get into the world of computer programming, not just your quick shell script to modify your ascii table, but really rigorous scientific approach to it. I don’t quite share that extreme of a view, but if you check out the “whatmyfriendsthinkido” meme for astronomy going around at the beginning of the year (see above), it is a quite accurate depiction of my laptop screen most of the time. There are variations of this meme, but the main point remains the same.

Anyway, but back to the article. I had sort of an epiphany last week. I was reducing actual MUSE lab data – lamp tests, pinhole mask tests, geometry issues and the like. If you look at the image below, you’ll see that we still have a long way to go… uff. The important thing is that there were on the order of 50 files and there were not ordered particularly well, you had to look in the headers to find the type of observations, to see which BIAS you had to use – very chaotic. Part of me was screaming to just go through them by hand and just run the pipeline on them one by one, since I would have to use different biases, arcs, flats and the likes, plus they were totally named with different identifiers.

But I remembered that article that I should not keep on doing repetitive tasks. So I soldiered on and 3 days later and many google (stackoverflow and similar) searches later I had a working program on which I could just hit “Enter” and let it run over the weekend. Now, perhaps running it “by hand” might have taken the same 3 days it took me to write that automation, but I had so much fun doing it. My office mate even learned something new relating to bash. I went home that weekend so satisfied. And I was proud of my little program, ready to tackle the next round of testing!

And then I remembered the legend that “DVD Jon” once said (I think it was him that said that, but I’m not entirely sure) that he had way more fun developing DeCSS than later watching the movies derived from that program. In a way, I learned that way of thinking from my father, too. He builds molds for the plastic industry, mostly complicated thermoforming ones. But that challenging work is way more fun than the final product that just gets stamped out by the millions.

I don’t know, it was such a well written and easy to read article, just wanted to give out an endorsement of it again and urge you to read it!

Family-friendly as an institutional culture

I was recently reading an article on the Women In Astronomy blog how the culture in your department has a big impact on how you perceive your work and your role in the organization. (Does Organizational Culture Matter? ). Naturally you start comparing departments and their cultures and what they emphasized. For example, in my time at the Lawrence Livermore National Lab an enormous amount of time was devoted to health and safety, while perhaps a bit of the start-up entrepreneurship was lost. There are always pros and cons on how an institutional culture affects you.

While, for example, the institution’s stance on, outreach or teaching could be an important point worthy of a blog post, this time around, I would like to focus more on family-friendly culture. It was only fitting that a few weeks ago this interesting video by Bryan Gaensler was released on Youtube. Bryan is the director of the ARC Centre of Excellence for All-sky Astrophysics (CAASTRO) at Sydney. His talk deals with family-friendly initiatives at CAASTRO and since it is an astronomy institute I thought it was quite relevant to watch it!

Here are the main points I got from the talk he held:

When advertising a position try to make part-time employment an option. It will leave you money to do something else and even perhaps hire somebody else. The part-time employee will be very dedicated. Don’t exclude a big fraction of very qualified candidates, be flexible in your timelines.
This resonated with me a lot. When I came to Berlin, I was not sure how free I would be with everything. It was not clear how everything would work out with Anna at her new school. My grandmother is not in the best of health. So when the head of my department offered me the half-time option when offering me the position, I was very thankful. I didn’t take it in the end, but I’m sure I would have been *way* more productive than the 20 hours he would’ve been paying me for, it would’ve taken off the guilt we feel that we aren’t working enough and it would’ve left me time for hobbies or other things. Great point and I love that Bryan put it first!

All core meetings should take place between 10-2. Be aware of school holidays. Don’t schedule some important meeting, workshop or similar into those holidays.
This! So much! Look, I can understand that some telecons need to deal with time differences and that some people will be undoubtedly unhappy with set times in large international corporations. But when you would schedule a work event (seminar, team meeting, etc.) at 4pm or later you are basically implying that you do not care about daycare times and are effectively signaling to people with children that those concerns are not valid in your mind.
The same goes for vacations. Here at work we have annual scientific evaluations end of October, which are “strongly recommended” (i.e. mandatory) to attend. For a long time this used to be during school holidays about two months before and it was very hard for people who had school age kids to attend. Don’t assume that conference attendees can just shuffle their kids with them during school holidays. At least be aware!

Welcome the child in the workplace!
I totally get where he’s coming from. He’s not talking about setting up a daycare at work or having toys lying around the office or the workplace suddenly becoming a screaming place. No, it’s that security, that if there’s an emergency, I can let my child come to work with me for some hours, let her play on the iPad a bit or let her draw some things and not feel guilty with other colleagues. It makes it so much easier. When I moved workplaces from the Livermore Lab to IPAC suddenly a world of opportunities opened in that sense, too. It just became a whole lot easier. It’s not about setting up your work to do the babysitting, it’s that even if I just took advantage of it once a year or so, the peace of mind that it is possible is great! Here at work, we are actually in the process of setting up a child-parent room to handle such emergencies, so I am very happy with that.

Discourage working from home, because working from home lets the barrier between work and home fall.
I’m afraid he’s too late with that one with me, heh! But then my computer is a mess of work and personal stuff, too. I am one of those people who does personal stuff in the workplace, too.
Anyway, but I do understand where he’s coming from! As an employer I would definitely not want my employees to be there 24/7 – always available. So there are exceptions when to work from home, e.g. waiting for a repairman, but don’t encourage it! Science is about collaboration and sometimes just walking down the hall to talk to somebody about that crazy idea you just had or that one line of shell script that just isn’t working. You don’t get that at home and many of the interactions (telecons, e-mails, etc.) might seem forced.

Ask yourself if mentioning the parental status of the person is really necessary (this affects women mostly).
I am very happy this doesn’t happen at my institute very much, but I have seen this so often at other places, when a speaker is giving a seminar or so
A friend of mine was thinking of leaving astronomy after having her 2nd child. The relationship with the supervisor was a bit strained because of that – he didn’t want her to leave the job, but he was angry all the time that she left work so early in the afternoon. Ah, but as soon as somebody from outside came to visit the institute, she was the token WOMAN that “yes, it is possible to be a researcher and have two kids”. ugh!
People have children, but it shouldn’t define your professional status! It’s an unconscious stereotype that is easily broken.

Yeah, sure, great science can happen when having a few beers with colleagues. But most of us don’t have that option. Don’t make it a requirement to be part of those social groups to succeed. No after hour social event with work branding!
I really feel for this. When I was a postdoc at Caltech one of the parts that I missed were the social outings that a lot of the other postdocs had. And it was mostly just beer and fun. But I had a family to go home to and wasn’t free in that sense. But that is fine and great and you want to actively encourage socialization outside of the workplace, as long as it doesn’t have an institutional label and it is clear that being part of these social groups is not necessary to be successful at work! Exception: a Christmas party or similar.

Don’t be the boss that is the last to leave! Try to say goodbye to people who work with you.
It sends a signal that it is ok to leave at 5, it makes people feel less guilty about leaving work (early). As a leader don’t feel like you need to hide that you need to leave work for personal reasons. If you don’t have guilt leaving for personal reasons it will make the others feel less guilt, too.

A few things about parking, carer’s leave and flexible culture which I won’t go into deeper, but which lead into an immensely important point.

Talk to people about impostor syndrome, let them know you have it, too.
Oh boy, this is a big one. I think the majority of us suffer from it. Even though *I know* that I am an efficient programmer, I still sometimes fall into the trap of thinking that my current position was gained through a combination of hard work and luck, but not out of inherent skill. Somebody surely will find out soon that I don’t really belong here.
Please, please read JohnJohn’s thoughts on this matter.

Being a family-friendly institution is a choice of culture, but I am definitely happy that I am working at an institution that embraces that choice!

Personal comment on the infamous letter to astronomy grad students

If you are in the world of astronomy, you probably have heard or read about the infamous letter written by senior staff of an unnamed “famous” astronomy department sent out to their grad students (though it is not difficult to find out which department sent out the letter). If you don’t know what I’m talking about, please read the “inspirational” e-mail here.

The first reactions and comments were naturally ones of outrage (see e.g. astrobetter). There were tweets and facebook posts that were shocked and indignated, mostly on the fact that we, as astronomers, should be working on the subject all of our waking hours and expressing “Thank you, sir, may I have another“. I think that the main source of the scorn was the claim that you have to work 80-100 hours per week on astronomy as a grad student if you want to have a succesful path as a researcher.

First of all, I don’t think anybody can work that long over prolonged periods of time (there’s even evidence for that). As an exception, because there were so many deadlines, I worked 65 hours last week and feel tired today. I hardly saw my family last week and am pretty unhappy about that. Now add 15 hours to that and I would feel downright atrocious.

However, I feel like that putting a number to the hours you work in a week missed the point completely. I am especially sad that the letter put this point first without any nuance or diplomacy – a great way to discourage your students. I can smugly say that even though my advisor was trying to impart similar wisdom to me about working hard, his way of doing it had a much more positive effect on me. Let me veer a bit into personal anecdote time here:

I found out in 2005 I was pregnant. The timimg is never perfect. I had just gotten the Lawrence Fellowship for Graduate Students. Ironically, while that made my measly grad student salary multiply by about 1.5 times, it carried a pretty horrendous health insurance making me basically pay for a big part of the costs during pregnancy and birth, basically negating all the gains in salary (heh!). Anyway, there I was, freshly married and quite frankly scared of telling my advisor that I was about to devote a large part of my life to a new being growing inside of me.

He couldn’t have been any more supportive. First questions were of the lines of “Are you ok?”, but then he did the smart thing, appealing to my ego: “You’re not thinking of leaving astronomy, are you? That would be a big loss! You can make a great scientist. You can make it happen”. Anyway, I left that meeting with a big sense of confidence, that yes, I could make it happen! He didn’t sugarcoat anything, he never said I could succeed by just being half as productive, but he saw the potential that it was doable.

Fast forward about 14 months. I was on my first trip since having my daughter. We were observing at Keck. Funnily, many people complain that they feel tired all the time while observing, but let me tell you, with an 8-month old at home, I felt refreshed and slept like I hadn’t in months, i.e. contiguous 6 hours. heh! Anyway, it was September of 2006 and I was supposed to be graduating less than a year from that date. I had produced just one paper (my deal with my advisor had been three) and I was hitting a roadblock with the analysis of the two big papers. For the Hubble paper, I feel like I wasn’t getting the fitting program, how to tweak the parameters, etc. I just wanted to scream at the world: “Look at the pictures! They tell all the story, what do I need to analyze here!?!?!”.

So we were driving back from a pretty unsuccessful run. We had just gotten a few hours of data out of 2 or 3 nights, I forget. I was feeling down, it was just too hard, the baby, the roadblocks, the stupid clouds. So somewhere on the road between Waimea and Kona my advisor asked me if I really wanted it, that the time and commitment I was putting in wasn’t gonna cut it. Especially not when I was doing other things than the analysis needed for the papers. “It’s all about decisions, Tanya. The person observing after you tonight went diving today. It is a nice thing to do, especially if you’re in Hawai’i. But to really want that job in astronomy, you will need to give up some nice things sometimes.”

All about decisions. That point really resounded with me. I can’t tell you that suddenly everything worked out fine. But I worked hard and by November, close to Thanksgiving I had *the* analysis, *the* main point of the Hubble paper done. There’s even a funny story about my excitement over that result. I had gotten the correlation with which my most disturbed objects were also the most reddened ones on the Thursday before Thanksgiving. I know because the next morning was a “Baby and me” class that I attended every Friday (those who know me will cringe, since they already know this story). We were asked around the circle to say what we were thankful for. I was to go first and happily blurted out that I had just found out the last piece of the puzzle of the paper and that I would be one step closer to graduating. Oh man, that was the wrong answer in that group and I only realized it slowly after every. single. mom. was thankful for having a healthy baby. Ooops!

I did not get a job for the 2007 job season (but only half a year later), that would’ve been too convenient of a story, wouldn’t it have been?. But I can tell you that 2007 was a year which I did focus on work, very much. I shooed colleagues away wanting to come in and gossip, the hours at work were too precious. It was about decisions. The fact that I did not even know what was playing on the TV during those years (I know my friends were obsessed with Lost!, heh), what was going on in the news. I may not have been the best paper reader, may not have had the best code written, it was sometimes hard going at it a bit alone (not completely alone, of course, thank you, Mark!). But the focus was there, the motivation was sparked that kept me going, I was gonna to whatever it took to get that PhD and hopefully that postdoc job!

Mind you, I didn’t suddenly work some magical fixed number of hours a week or spent my time at home with my baby on one arm and the laptop in the other. Rather, and it sounds weird to say it like that, I became more efficient. I most definitely missed some things. I couldn’t have done it without the enormous help and encouragement of my husband. But I can’t explain it better than I was really into everything astronomy during that time (as I feel like I have been this year, too). And it’s like an avalanche, it just multiplies and gets bigger.

Anyway, boy did I veer off the point. It was about this fixation on the number of hours worked, when in reality you should get away from this thinking. It’s about the focus, the flow and the results which you should measure. And thankfully the posts and blogs that have come out since have been a bit more nuanced and addressed much more eloquently what I was wanting to say. The two articles I liked I liked the most were:

a) The post by Elizabeth Lovegrove over at astrobites. Its main point is that there is no clear line where work ends and when fun begins and that we should step away from the computer once in a while. That doesn’t mean that you suddenly should seek out weird ways of doing science in the cafeteria, but rather that we are not in a factory where some fixed amount of product needs to be produced in a certain amount of time. It is science, insights don’t come at the n-th hour worked on the subject.

b) Another amazing blog on the subject was written by Lucianne Walkowicz. http://tangledfields.com/2012/10/26/free-advice/ It’s downright inspirational for any field you work in. Right in the beginning she tackles the issue about the number of hours worked with this simple sentence: “If you are counting how many hours a week you are working, you probably don’t like your job very much.” Exactly! I couldn’t have conveyed it better. I would suggest you just sit down and read the whole article, it deals with health, goals and a whole bunch of other things that I was just nodding along to.

Lastly, I do want to say that I hope if anything good comes out of this discussion is: yes, it’s about decisions, but there are a few things that you should not decide away – it is your health, be it physical or mental and your family and close friends! Now go on, keep on truckin’!

MUSE Science Busy Week in Satilleu near Lyon

 

The MUSE instrument is a giant 3D spectrograph consisting of 24 IFUs. It will be put on the VLT next year and be operated by ESO. But it is not built by ESO, it is actually built by a consortium of 7 institutes across Europe. Each institute has its own duties and contributions to the telescope, which is being assembled at CRAL. The AIP has already delivered the Calibration Unit (lamps, masks and other optical elements) and handles the Data Reduction Pipeline, which will also be delivered to ESO next year. This also is the part I am currently mostly working on and why I often talk about the MUSE instrument and data reduction.

As part of building the instrument, the institutes that are part of this consortium that is building MUSE get a significant fraction of guaranteed time observing (GTO) with it. It is about 40 nights per year, with some options to shuffle the nights around in the semester to aleviate pressures on certain RA range. That is a lot of time! And a lot of data! And considering the complexity of the instrument, its 24 IFUs and the variety of lightpaths a large computational challenge. So the consortium wants to make sure that they not only get the best science, but that they are prepared to handle the challenges that come with such a huge and complex dataset.

For that reason every 9 months or so we have scientific busy weeks. These are designed to bring to the table the best science cases possible for the GTO proposals. MUSE is an instrument that will go very deep, covering large fractions of sky at high redshift. One of the main scientific drivers is the unbiased detection of Lyman Alpha Emitters (LAEs) at all redshifts without being constrained by narrow band filters. But there is so much more science to be done with those deep observations (fluorescence, accretion flows, absorption systems, mid-z low luminosity systems, etc.). The same goes for the case in nearby galaxies. The MUSE science busy weeks make sure we are up to date with the newest results that could impact these observations, to prepare the best targets and to double-check that we have all the ancilliary observations and analysis tools to produce the best GTO science possible.

Since my main task is concerned with the data reduction side, for the scientific team that means that people working with the data understand the reduction pipeline and its outputs according to their needs. In the past it was to give them an overview of the recipes and tell them about the process that was being made. But now we are actually at a point that we can do tutorials and let people play with the pipeline themselves on mock or lab data. The past half year I have spent writing an user manual and cookbook for the end user of the pipeline. I don’t know if ESO will take on this manual, but at least in the consortium we will be constantly improving it. In the tutorial I already received enough comments and improvement suggestions so that I have enough to do for the next few weeks. We also envision these sort of tutorials becoming more frequent to potential german users of the MUSE instrument (german taxpayers are paying my salary, after all). Sort of like “MUSE Community Days”, but I am not entirely sure about the format or details.

But besides the technical and the logistical side of these Busy Weeks, the consortium really has gotten together as whole. One of the conditions for a member to put forward a suggestion to tackle a certain observation was that at least aother institute was involved in the project, too. As such, lots of collaborations and even friendships have been formed. People have stated to work together on similar or preliminary science to the proposed MUSE ones.

Another aspect of the Busy Weeks is that we are often in “secluded” locations, so that we don’t get distracted and are only amongst ourselves. There is no possibility of attendees to retire to their office or home. We are all in the same hotel, we all eat together, basically “forced” to talk to each other. On Wednesdays there usually is a hike. All this to say, that by now, the members consortium know each other quite well and there is also happiness of seeing some people again.

This time the Busy Week was at a hotel in Satilleu, near Lyon in the Ardeche district of the Rhone-Alpes region. The area is sort of hilly, but not extreme. The villages are not that abundant, but in the two we stopped (Satilleu and the bigger Annonay) the houses are made out of stone built close to the hill with small and windy roads curling around them. So the scenery was beautiful, but “there was nothing there”, not even some spectacular natural scene. So we spent our free time walking around, playing mini-golf (provided by the hotel), surfing the net or having some beers in the late evenings. The food in the hotel was quite good, as opposed to the last Busy Week and I looked forward to the long conversations over wine and cheese after dinner.

The Busy Week itself went fine, too. I think the tutorial was a success and it finally gave some people the opportunity to work with the pipeline for the first time and take away any initial fears of working with the data. I am very proud of all the things my institute has put out with regards to the MUSE project, so I can say that from our side it was a productive week. There were other tutorials and presentation of analysis software in different stages of execution. There were one and a half days also dedicated to science talks and suddenly it felt like a conference, too. I was looking up papers and listening to people present their results and how it could apply to MUSE observations.

So I would call this week a success. While I was totally beat at the end of it, I also was inspired to keep on working on MUSE stuff here at the institute, knowing that it was being received well.

One funny experience I don’t want to forget either is me driving the AIP people back and forth from the airport in a small bus. Maneuvering the bus through unfamiliar and narrow roads while we argue over the loudness of the radio and the comfort level for the last row participants and whether we should buy goat cheese and which freeway to take and… ah, good times.

German Astronomical Society Meeting in Hamburg

 

Last week (24.9-28.9) was the German Astronomical Society Meeting in Hamburg . It was done in conjunction with the 100th birthday of the observatory near Hamburg (Bergedorf).

This was my first time at such a meeting in Germany. I had been to plenty of AAS (the American version) meetings before with all that it entailed: from the eager first year in 2003 where I knew nobody, attended every session and complained that there was just too much to see to this year where I sat at a table at the entrance with friends, waved people I knew over to talk to them or schedule drinks for later in the evening and “complain” that there are so many grad students around. I was eager to compare and make friends this side of the pond.

The session that was most relevant to my interests concerned the new instrumentation being developed for the VLT. MUSE is part of 3 large instruments that are being installed at the VLT in the next year months along with KMOS and SPHERE, so the organizers of the conference thought it would be relevant to present these instruments and other new things (e.g. improvements to the interferometer) in a large sessions to the German community. As such, I did not attend that many science sessions outside of these, the one about ALMA being the exception which was very nice.

Overall, I can’t tell you that much about the science. A lot of the highlights I already knew – Sandy Faber (Karl Schwarzschild Medal) works in my field of galaxy evolution, Cecilia Scannapieco (Biermann price) works at my institute, so I know of her work. I used to work at IPAC, of which the Herschel Science Center is a part of, albeit in the US, but a lot of science highlights from Herschel were well known, too. So, if you pay attention to your field and the press releases, most of the things at these meetings are already within your scientific knowledge. Oh well, but you’re not going to such a general meeting for that, anyway. The main thing is to see people, network and get a sense of how astronomy is going in general in the country. So it was actually two sessions I attended that had nothing to do with astronomy that were quite interesting.

The first was the workshop of the Astro-Frauen-Netzwerk which is intended to be Germany’s version of the “AAS Committee on the Status of Women in Astronomy“. It featured a really nice presentation on continuing work by J. Fohlmeister on “Career situation of female astronomers in Germany” (Arxiv link), now with more representative surveys also of german men and of scientist (men and women) in the UK. Some highlights of the talk that I took home, because they stood out so much, were how german men kind of stood out with regards to the other three groups (german women, uk women, uk men). I am paraphrasing, but german men hardly found their jobs on the internet (while the other groups did) and they saw opportunities and experienced career growth when they had children (while the other groups saw childbearing as a burden). As for the rest of the workshop, I could wax poetic about matters concerning female german astronomers, but I think it doesn’t fit the scope of this blog. Let’s just say that I see these groups as an opportunity to network and help each other. I was shocked to hear that other male astronomers have problems with such a network even existing (insert appalled emoticon here!).

The second was the general meeting of the members of the German Astronomical Society. It was almost sweet how lovingly and serious everybody took their task and I appreciate all the hard work the board does. So even though this long session had nothing to do with astronomy it was interesting observing the apparent hierarchies within the society.

The last highlight was the conference dinner at the observatory in Bergedorf where they were celebrating their 100 year existence. The dinner experience was well thought out in that you passed various stations of starters, grill, cocktails until you at long last gathered in the library for coffee and dessert. That way, you visited the main stations of the observatory without missing interesting dinner conversations. At the library there was a long speech given by Prof. Dieter Reimers (now retired) explaining the history of the observatory. It was a well done speech interjecting lots of personal anecdotes and funny facts and I was surprised that 45 minutes of the speech had passed so quickly. Overall, an excellent evening and I am happy that I finally visited that observatory.

Lastly I have to mention two things that weren’t so great, but for which the organizers had little wiggle room or fault:

a) The ESO deadline was early on Thursday along with the NOAO deadline on Friday. This made me miss lots of sessions and I was stuck at the hotel writing and reading proposals rather than attending social events.

b) The weather was quite bad, it probably rained for most of the conference. I could sneak away early on Friday to buy toys for my daughter (a boat and a compass, of course) and it was the only morning where it wasn’t pouring. The image at the top of this post was taken at that time. I had been in Hamburg before, but if you ever find yourself there a harbor tour is definitely a must!

If I can, I would gladly attend the conference next year in Tübingen!