Wednesday, February 8, 2017

Sampling Points Proportional to Intensity in an Image

A few months ago Aaron Lun shared some code with me to sample points based on intensity of pixels in an image. With his permission, I thought it would be fun to share it here!


The idea is you start with some image (either in a JPEG or PNG format). Here I'm using Rafa's twitter profile picture in a JPEG format.


I'm using the readJPEG() function from the R jpeg package to read in the JPEG image. Alternatively, you can use the readPNG() function from the R png package to read in a PNG image. If you read the help file of readJPEG(), it says it "reads an image from a JPEG file/content into a raster array". OK, first off what do we mean by raster? You can think of a raster image as a rectangular grid of pixels. In the help file for the readJPEG() function, it states "most common files decompress into RGB channels (3 channels)" where RGB stands for red, green and blue.

So, this means for each pixel in the original image, we should expect 3 values (one for red, green and blue). These values range between 0 and 1 and can be thought of the intensity of each color in the image. Below, I'm showing the intensity values from the three channels colored on the same scale (top row) and colored in the red, green and blue scales (bottom row).

To get the intensity of how light a particular pixel is, we can just average the values across the three channels. Next, we flip the intensity scale by subtracting the intensity scale from 1 to see how dark/black a particular pixel is. This makes the previously dark areas now light and vice versa (top left figure below). To control the contrast of intensity, you can just take powers of the intensity values. The lighter the background, the better the contrast will be between the individual and background on the original scale. Depending on the background in the original image, the skin tone, etc you may need to tinker with the contrast. I've shown the original scale and powers of 2, 4, and 8 below.

Next, we sample points proportional to the (powered) intensity level, add a bit of random noise and then plot the sample points along the x and y axis. In the picture below, you can really see how the power is very important to control the contrast levels.


Happy sampling!


Thursday, January 5, 2017

Women in Statistics and Data Science Conference 2016

Happy New Year everyone! After a wonderful holiday break, I was excited to find my copy of AMSTAT News from the American Statistical Association in my mailbox! Someone pointed out to me that if you look close enough, I can be found standing in the background on the front cover of the Dec 2016 issue. So, apparently I can check off 'being on the cover a magazine' from my bucket list. ;)


The picture was a snapshot of an audience at talk from the Women in Statistics and Data Science Conference (WSDS) 2016 which was held in Charlotte, North Carolina Oct 20-22, 2016. The cover story can be found here. In 2014, I attended the inaugural Women in Statistics Conference and was fortunate enough to attend the conference again this year! Looking back through blogposts this fall, I realized I did not write a blog entry after I returned from the conference, but I did manage keep a few notes in a Github repo of a few of the talks. Here, I want to summarize some of my thoughts and experiences of WSDS 2016 and at the end I describe a few suggestions for future WSDS conferences. I was not able to attend all the talks mostly because there were many concurrent sessions happening at the same time, but I hope this highlights at least a portion of the conference!

The picture above was taken in a talk by one of my favorite speakers from the 2016 conference, Erin Anika Wiley from Westat, titled "Do you Hear What I Hear?: An Examination of Effective Communication". I managed to slip into the back of jammed packed room and furiously write down notes on the results of a survey she conducted asking about opinions of presentations from statisticians. I love that this was the front cover of the magazine because Erin really had the audience laughing and engaged for her entire talk. Also, it was humbling and enlightening to hear the survey responses on how non-statisticians perceive statisticians based on talks and presentations. Definitely motivation for how we can communicate our results more effectively!


The first keynote address was from Cynthia Clark titled "Consider your legacy". She gave inspirational talk discussing what her contributions have been in her personal and professional life. As a new mother, I sincerely appreciated hearing how she prioritized her family throughout her life.



Another great keynote address was from Stacy Lindborg at Biogen with a talk titled "Know your power". Stacy is a natural at connecting with the audience by sharing personal stories from her life. In this talk, she shared five reflections/tips on having a successful career in the face of many challenges (both personal and professional). One of my favorite quotes from her talk was "We love the things that we are good at", which really resonated with me.

Keeping with theme of careers, a topic I found particularly interesting is how to navigate changes in your career. Michelle Dunn, Donna LaLonde, and Nancy Flournoy shared very honest and personal experiences of following non-traditional career paths, recognizing and moving forward when you have failed, and knowing when to leave a job. A central theme from each of their talks was to always be growing your network and find trusted individuals/mentors to help you navigate these experiences.

Similar to the session in 2014, the 2016 conference included a fantastic panel of the past, current and future ASA Presidents. These women were able to persevere in the face of a highly male-dominated field to become ASA Presidents and become role models for women in the early stages of their career like me. Even with all this progress, I agree with Mary Ellen Bock that there is so much more work still to do with supporting more specific minority groups in this field. For example, she eloquently described her hope of one day seeing a non-white woman on this panel of presidents.


In my opinion, one of the coolest sessions was listening to the amazing Mary W. Gray from American University giving a fascinating discussion on what US laws exist to protect women. If you don't know who she is, you can read a summary on wikipedia, which is pretty incredible! One of my favorite things that I learned about her was that she wears a 3/4 euro lapel pin (noting pay inequality is not limited to the US) to promote "equal pay for equal work" referring to the Equal Pay Act in 1963 and Title VII in 1964.  Sadly I didn't get a picture of the lapel pin, but I love the idea!


Wendy Martinez from the Bureau of Labor Statistics gave a great keynote address on what and how federal statistical agencies are thinking about when it comes to data science. Data science is something I am passionate about, so I really enjoyed this talk. I would have loved to see a larger emphasis on data science education though.


Unfortunately Bin Yu from UC Berkeley was unable to join us in person, but the conference organizers were able to set up a video chat and connect to the big screen!  Her talk was titled "A holistic approach to interdisciplinary research" where she began discussing how she has a people centric view of life and research (people are mysteries to unveil just like research). It was amazing to hear a little bit about her background and how she grew up in China during the cultural revolution (1966-1976) where the universities stopped for 10 years. Something I think that cannot be overstated was she noted that she most appreciates intellectual diversity in collaboration and research. Overall, it was an inspiring talk!


The conference ended with a festive dinner filled with new friends, awards, presentations and delicious desserts!


As the conference took place at a hotel, I managed to find a nice location to write down some notes from the talks on my GitHub page. :) Feel free to check them out for more details on the talks that I attended.



Finally, I want to finish this blogpost with a few suggestions for future WSDS conferences. My intention is for these suggestions to be viewed as constructive to help make this conference stronger and more accessible to women in the future.

  1. Fewer talks running concurrently and more posters/speed sessions. I appreciated that the keynote talks did not have other scheduled talks, but at other times I had to choose between 5-6 sessions running concurrently, which led to frustration and disappointment that I missed so many other talks. My arguments for this are similar to the ideas previously described to improve JSM (decreasing the number of contributed sessions and increasing the presence and importance of poster sessions)
  2. More talks from women in data scientist positions in academia, government, and industry. I recognize that the term "data science" is very much being actively debated in many settings, but as this conference now has "data science" in the title, I would love to see a larger discussion of data science and how that relates to statistics, here too. There was such a great representation of perspectives from women who pioneered this field. I think it would be equally beneficial to include the perspective from a new generation of women in data science positions. 
  3. Childcare. I was surprised to find out that no childcare was available at this conference. Considering a good portion of this conference was dedicated towards discussing issues related to balancing careers with families, I found a lack of childcare a bit ironic (?). This suggestion is motivated by my own personal experience of recently having a baby. I saw several other WSDS attendees who brought their children and probably could have benefited from a childcare service. There are great childcare examples broken down by cost, size of conference, childcare agency, fees charged, etc.  I hope this will be incorporated in future WSDS conferences. 
Thank you to all the conference organizers for putting together this conference. I look forward to the next one in La Jolla, CA Oct 19-21, 2017! 

Wednesday, November 9, 2016

Time-Tested French Onion Soup

Today, we have a special surprise! This is our first guest post by a great friend from graduate school. The recipe looks amazing and I can't wait to try it out. Hope you guys enjoy!

This recipe comes from Cook's Illustrated All-Time Best French Recipes. Here’s a tip: go to your local library and look for Cook’s Illustrated cookbooks or America’s Test Kitchen DVDs. I like these recipes because they are generally straightforward in terms of ingredients and cooking techniques. However, if a recipe calls for it, one thing they don’t skimp on is TIME.


Time is the secret to today’s recipe: French Onion Soup. There will, of course, be thyme--but the real secret is time.

Supplies

This recipe can be broken into three stages, so I think it’s helpful to break out the equipment and ingredient list into those three stages as well.

Stage 1: Prep and Oven

Equipment: oven; cast-iron Dutch oven (7 quarts or larger); cutting board; chef’s knife; mandolin (optional); sturdy wooden or high-heat resistant silicone spoon/spatula; measuring spoon;

Ingredients: yellow onions (4 pounds); unsalted butter (3 tablespoons); salt (1 teaspoon); cooking spray;

Stage 2: Stove-top

Additional Equipment: stove-top or burner; measuring cups;

Ingredients: dry cooking sherry (½ cup); beef broth (2 cups); chicken broth (4 cups); fresh thyme (6 sprigs); bay leaf (1 leaf); water (end up using about 3 cups total); salt;

Stage 3: Garnish and Serve

Additional Equipment: oven or broiler; baking sheet; bread knife; cheese grater; oven-safe bowls

Ingredients: baguette; Gruyere cheese, shredded (about 2 cups); salt and pepper; dried thyme or tarragon (optional)

Cooking

We’ll keep a check on time throughout the recipe, so that we can see how long it really takes.
Time-check: 1:05pm

Stage 1: Prep and Oven

Preheat oven

Adjust oven rack to lower position, such that Dutch oven can be placed in center of oven. Preheat oven to 400 F.

Prep onions

Peel and slice 4 pounds onions. Yes, that is a lot of onions! Don’t be alarmed, they will reduce in volume significantly during the cooking process.

What kind of onions to use?

Yellow onions are the winner here. White onions don’t have enough flavor, while Mayan sweet onions are too sweet for this savory application. Red onions would just be kind of weird in French onion soup.



Slicing the onions

How vegetables are sliced affects how they break down during the cooking process. Because this recipe has such a long cooking time, it is very important to slice the onions in the right way. To help peel the onions, I sliced them in half. Notice that I sliced through the root end, or  with the grain. This is important.


To slice the onions for this soup, you want to slice with the grain, or “north-south” through the root. If you’re still unsure, you can think about this as opposite of the way you’d slice onions for onion rings.


To help slice 4 pounds of onions, I recommend a mandolin--and don’t forget the safety guard or safely glove when using a mandolin!

 

Season the onions

Liberally coat the inside walls and bottom of the Dutch oven with cooking spray or vegetable spray. Load all the sliced onions into the pot. Cut 3 tablespoons butter into a few chunks and place on top of onions. Sprinkle over 1 teaspoon salt. Don’t worry if the pot is loaded all the way to the top; that will be fine!


Oven cooking stages

The onions will now cook in the oven in three sub-stages. Time check: 2:04pm.
Place pot in oven with lid secured. Cook for 1 hour.


After 1 hour, the onions will start to wilt and release their moisture. Stir the onions, making sure to scrape the browned bits off the sides and bottom of the pot.


Replace pot in oven with the lid slightly ajar. This will let the moisture cook off in the oven. Cook for 1 hour more.


Remove the pot and stir and scrape the onions again. At this point, they will still be translucent, but starting to brown. Replace the pot in the oven with the lid ajar. Cook for about 35-45 minutes more.


Time check: 4:37pm. Remove the pot from the oven. The onions will have reduced significantly in volume compared to their raw starting point. They should be lightly browned.

(Note: you can leave the oven on)

Stage 2: Stove-top

Transfer the Dutch oven to the stove-top. The secret to this stage is multiple de-glazing. This is where the rich flavor comes from, so please don’t rush any steps on the stove-top!

After the oven stage, the onions will still have some moisture in them. First cook the onions over medium/medium-high heat until they get nice and golden. Keep scraping the bottom and sides of the pot so that the onions don’t stick. After about 15-20 minutes the onions will be dark golden brown and paste-like. Adjust heat as necessary to prevent onions from burning. Time check: 4:53pm


De-glaze

De-glazing is a general technique for re-dissolving food residue. In this recipe, we de-glaze a total of 4 times, which is many more than most recipes call for.
After the initial 15-20 minute cook on the stove-top, you can then let a brown crust start to form on the bottom of the pot. You still want to actively stir the mixture, but don’t scrape so hard as to prevent that crust from forming.



When you get a nice dark crust, but before the bottom of the pan starts to burn, it’s time to de-glaze. Pour ¼ cup water into the pot and use that liquid to scrape the bottom of the pot and re-dissolve the browned bits. In between de-glazings, stir the mixture and allow all the liquid to cook off.


De-glaze 2 more times with ¼ cup water each.



De-glaze a 4th time with ½ cup dry sherry.

Time check: 5:19pm

Simmer

After all the liquid is cooked off from the sherry de-glaze, add beef broth (2 cups), chicken broth (4 cups), and 2 cups water. Scrape the bottom and sides of the pot well.




Toss in 6 sprigs fresh thyme (I have some lovely fresh thyme growing on my kitchen porch!), one bay leaf, and a sprinkle of salt (½ teaspoon).



Bring to a boil, and then reduce to a slow simmer. Cover and let simmer for 30 minutes.
Time check: 5:29pm

Stage 3: Garnish and Serve

While the soup is simmering, you now have time to prepare the garnish.

Slice the baguette and place in the oven to toast. Toast until very lightly golden brown. You can leave the oven on after the onion oven cooking stage and use the hot oven to toast the bread.


Grate the Gruyere cheese. I decided to use a mix of medium (aged 5 months) and intense (aged 12 months) Gruyere. I really like strong cheeses, but you can use a mild Gruyere if you prefer.


Now move the oven rack to the top position, and turn the broiler to High.

After the soup simmers on the stove for about 30 minutes, fish out the thyme sprigs and bay leaf, and season with salt and pepper to taste. Note, you should not need much more salt. In fact, if you feel the broth has reduced a little too much to your taste, you can even add a bit of water.


Ladle the soup into oven-safe bowls. Place bowls on baking sheet for ease in getting them in and out of the broiler.


Float the baguette toasts on top (do not overlap), and cover lightly with cheese. This recipe is all about the rich onion flavor, so you actually don’t want a thick cheese crust. Just use a moderate sprinkle of cheese.



Place sheet with bowls right under the broiler for just a few seconds--just long enough for the cheese to fully melt.



Garnish with fresh cracked black pepper and (optionally) a little dried thyme and/or tarragon.


Time check: 6:04pm. Bon app├ętit!

Epilogue

So was all that time really worth it? Yes, I think so. If you like French onion soup, you owe it to yourself to try out this recipe!

Don’t have a 5-hour chunk of time? You can actually make this soup over multiple days. After the oven stage, let the pot cool down and refrigerate the oven-roasted onions for up to 3 days. Pick the recipe back up at the stove-top stage.

Alternatively, you can let the soup cool after the stove-top simmer, and refrigerate for up to 2 days. When ready to eat, warm the soup to a simmer on the stove (likely need to add a bit of water to the broth and adjust salt and pepper seasoning), prepare the toast and cheese garnish, and serve.

Wednesday, November 2, 2016

Shrimp Etouffee





We've got fine posts about Gumbo, Natchidoches Meat Pies, and Jambalaya... so it's about time we made some etouffee!

As always, the Junior League of Lake Charles has some excellent recipes:

Ingredients:
 - 1.5 lbs shrimp (medium - extra large)
 - 1 onion
 - 1 bell pepper
 - 3 tomatoes
 - 4 stalks of celery
 - 5 green onions
 - 5 cloves of garlic
 - 1/2 cup all-purpose flour
 - 1/2 cup oil (we use canola)
 - 1-1/2 cup of rice

Assemble the veggies!

Chop chop chop~!


The tunics will garnish the dish, and the scales will go in the pot. 



Using 1/2 cup flour and oil, make a roux.



Blonde to light brown is our goal today.

Bell peppers take the longest to cook, so drop them in first.



When the holy trinity has cooked down slightly, stir in the tomatoes and garlic.


Add 4 cups water or stock.

Season the stew with salt, pepper, and red pepper flakes.


Now is the time to prepare the rice.  1.5 cups of long-grain white rice and 2.25 cups of water.

Simmer for 30 minutes and drop in the shrimp. 


Typically, etouffee is served over a big mound of rice, with a garnish of green onion tunics.  Since the spice level is mild, I like to keep the hot sauce nearby.