Wednesday, March 23, 2016

My First Nova Midcycle

The Mitaka Nova Midcycle was held in Bristol, U.K. on January 29-31. I will not be discussing so much of the technical aspects but the experience of the Midcycle from the perspective of a new contributor.

The Midcycle meetings happen in between the larger OpenStack Summits. The Summits happen every 6 months, so the Midcycles happen every 6 months as well. While generally they are based in the U.S., occasionally they take place outside of the U.S. as in this case. When you work on an OpenStack project team, you should expect to have opportunities to meet and interact with your team mates every 3 months. This is really key for teams who span the globe and who have some members that may never be online at the same time. Face time also is important for providing a personal connection and context among team members, which I believe leads to a deeper emotional investment in the project. This pattern may have originated with the Canonical community, which is where many OpenStack folks came from.

Anyone can attend the Midcycles, they are public and open. In fact, I'd highly encourage anyone who is interested in getting more involved with a particular team or project to consider attending. These are, however, fast paced working sessions, so don't expect a lot of context or explanation. It is easy enough to prepare for the Midcycle and get context. I'd argue that even members of the team that have been focused on their own projects might benefit from such preparation as well.

Here are some tips for preparing for the Midcycle (or Design Summit for that matter):
1. Read the priorities document for the current cycle (usually located in an Etherpad, ask in irc)
2. Read over the reviews listed on the priorities document
3. Read the mailing list, particularly paying attention to discussions around high priority items
4. Review recent IRC meeting logs for team and subteam meetings around priority items.

Following these tips will ensure you're up to speed on discussions around things you might not have been particularly focused on, specifically so you can follow the context of the discussions. Note that people are nice and if you ask they will usually explain things. You want to avoid interrupting the whole group discussion however as there is limited time at these meetings and many of your questions can be answered during breaks.

The format for the Midcycle was loosely structured. We had an agenda in an Etherpad with topics to discuss, and people took notes on those topics and items were decided, or follow up items were identified and assigned. Anyone could add an item to the Etherpad. The sessions started at 9:30am and ended around 5pm with several short breaks and an hour lunch break in between. Thursday was more of an unconference/open hack day where people just worked on various items and took the opportunity to pair with each other or to discuss things.

Overall I'd say the Midcycle was an incredibly positive experience. It gives project team members the opportunity to interact outside of the madness of the larger Summit. A key importance of the Midcycle is reinforcing a social bond. The group was friendly and inclusive, going out for dinner and drinks together each evening. Anyone was invited to join but our group tended to stay around 15-20. Many of the Midcycle attendees were locals who had lives and families to attend to. Others were just less interested in the social aspect for their own reasons.

I originally attended the Nova sessions during the Design Summit. This was my first time meeting many folks on the Nova team and I admit I was incredibly shy. This was my second time meeting many of the Nova folks including some who hadn't been at the summit. At the time of the Mitaka Midcycle, I had been working on Nova for 4 months, participating in both IRC discussions and the mailing list. This time when I introduced myself to people they had some idea of who I was and what I was working on.

I'm looking forward to seeing everyone again at the Newton Design Summit in April!

Saturday, January 2, 2016

Groceries without a Car: A Comparison of Options in Portland, OR

I use grocery delivery a lot because I don't have a car. I do use Car2go and Zipcar when I need to, but grocery delivery is a more viable option because the delivery fees tend to be less than the cost of car rental and it doesn't cost me any time. This is a short article where I will compare my experiences with 3 different grocery delivery options in the Portland, OR area.


Years ago when I lived in downtown Seattle, I used as they were the only grocery delivery option at the time (we're talking around 2007). Sadly, their whole online experience hasn't changed a bit. It's still really clunky, specifying substitutions is pretty pointless, and you ultimately have to rely upon the paper receipt the driver gives you for any accurate record of your order history.

You can order from their website online or from your phone. Groceries are arranged by aisle and you can drill down to more specific items, making it very easy to browse. The search functionality is pretty solid as well, I've had no trouble determining if they carry a particular item. You can also access your order history, but note that if any substitutions were made it won't be reflected there. Your order history only shows what you submitted as your order, not what you actually received. You can only access your last order and a complete list of everything you ordered.

Order communication happens via email or the driver will call you on whatever phone number you've provided.

Grocery Selection and Quality

The main reason I continue to use them is they are the only grocery delivery option where you can order beer and wine. In addition, they are the only delivery service that provides their drivers with carts, so I prefer to get bulk heavy items from them.

The only fresh produce I buy from them is bananas and organic berries when they have them in stock. I've ordered organic lettuce, organic tomatoes, and organic cucumbers from them in the past and I've been really disappointed with the quality. The produce just doesn't have any flavor and tends to spoil quickly as it's typically imported from Mexico or Europe. I buy frozen vegetables from them often, the O Organics line offers some good values. Again, the flavor and quality is nowhere near as good as more expensive brands, but in my case I like the lower price.

Besides beer and wine, I typically order soda and mineral water. The price varies, but they sell San Pellegrino sparkling mineral water in glass bottles. I typically order a dozen or so per order. I really wish I could just get a case but really it's no big deal.

I also get household items like laundry detergent that are cheaper through them than through Amazon.

Order Modification and Substitutions

If you need to modify your order, you can do so easily on their website as long as it's prior to the day your order is going to be delivered .If you need to make any modifications to your order on the same day of delivery or if you need to notify the driver of any delivery changes (eg, you won't be home so call a different phone number instead), good luck with that. The customer service is great but slow to respond to changes. I suspect it's their computer software.

You can specify substitutions at the time of ordering but they only have 3 options and no custom text area. The options are "Same Brand, Different Size", "Different Brand, Same Size", or "No Substitution". This has been the same since I first used their service almost 10 years ago.


Groceries are packaged using plastic bags, sometimes with one small item per bag. Depending on what and how much you order, you could end up with anywhere from 5-10 plastic bags. They put stickers on the plastic bags which can make them difficult to re-use.

Delivery Times

They don't do same-day delivery, the soonest you can get your groceries is the next day. They offer 1-hour, 2-hour, and 4-hour windows for different delivery rates.

Delivery Cost

The cost of delivery varies. If you spend over $150 and order 5 items from their special list, you get free delivery. Delivery is $9.95 if your order is over $100. There are time slots that give you discounts as well, for instance a 4-hour time slot will give you a $6 discount off the delivery fee. So if you're spending $100, you can get delivery for $3.95 if your time is flexible. There is no tip option; the drivers are Safeway employees.

Amazon Prime Now (New Seasons)

I love shopping at New Seasons, they offer very competitive prices for locally produced food item and the overall quality is fantastic. I've only shopped at New Seasons through the Amazon Prime Now app, so I can't really compare any of the other stores. My main gripe is you can only order groceries and access your order history from your phone. The user interface leaves a lot to be desired, it's hard to just browse, you end up having to search specifically for the thing you're looking for, or you end up browsing over 100 items in a very broad category. When you submit your order, the status page does not automatically refresh. There is no way to specify substitutions, usually an item is either simply not included if it is out of stock or if a lesser number than what you've requested is available, the shopper may text you and ask if that would work. It varies by shopper. Another gripe I have, sometimes produce shows up in strange quantities that don't seem right. Like tomatoes might say $2.99 each when I think it's supposed to be $2.99 per pound. I tried ordering one one time and ended up with one small tomato for $2.99.

The user experience isn't nearly as bad as Safeway but it does have some issues. I blame part of that on this being fairly new so hopefully they'll improve it over time. I'd really like to see it available on the desktop because the whole phone browsing experience can be a little annoying.

Grocery Selection and Quality

I've always gotten excellent produce from New Seasons via Amazon Prime Now. I've never gotten anything that's brown, too ripe, or smashed. I also order most of my meat through this app. The only gripe I have here is if you order something like 2 1-pound cuts of meat in hopes of getting 1 2-pound cut of meat, you end up with 2 separate packages of meat. I typically order the New Seasons brand of a variety of things including milk, eggs, and butter and the quality and price are comparable to more expensive name brands in the same category, like Organic Valley.

They have a really great selection and probably the best value (price for quality) of all of the grocery options. However, the selection is only a small subset of what you can buy in store.

Order Modification and Substitutions

You can't modify an order once you've submitted it; you have to submit a new order.

You can't specify substitutions in the App UI. The shopper will contact you to let you know if something is out of stock and offer you any substitution options if available. Substitutions really vary by shopper. I've had them only provide the item if it was just a lesser quantity than what I'd originally ordered; when I asked if I could get something else instead, I was told to place a new order. Other shoppers have offered replacements. I think it's because the service is still new, so they are still working out issues.


The shoppers show up in their own car with a pile of paper bags. They do not typically have a cart or anything special to transport your groceries in. New Seasons has the best deal on mineral water, 2 liter glass bottles for $1.50 each. I was buying these in large quantity regularly, but stopped because I had to meet the shopper with a cart and it seemed like it was a big hassle for them.

My biggest gripe with Amazon Prime Now is they consistently use large paper bags. The paper bags are very strong and don't break. I can reuse them for recycling so it's not a huge deal, but I'd love to see a reusable tote option. The shoppers typically put any meat or produce into plastic produce bags instead of using paper produce bags, and they are very bag happy. Meaning, anything they can put into a bag, it's gonna go in a bag. If you get a squash or a watermelon, it will end up in a plastic bag.

Delivery Times

Amazon Prime Now offers same day delivery in a matter of hours. I've never had a delivery come late, but they usually come towards the end of the time range I've specified. You get regular updates of your delivery status via text message and there is a map in the order status page that shows you where your delivery currently is. The only issue is it doesn't automatically update so you have to reload the order status page manually if you want to see any updates.

Delivery Cost

Currently there is no delivery fee but you are expected to tip the delivery person at least 10%. You do it through the app when you order and it shows up as a separate charge. If you made the minimum order of $30, delivery would cost you $3.

Instacart (Whole Foods)

Instacart is my favorite delivery experience. You can order both from your phone or on your computer. The app does an excellent job of updating you of your order status and it's easy to browse order history and groceries both on the web and on your phone.

Grocery Selection and Quality

Overall the quality of the items I've gotten from Whole Foods via Instacart is pretty good but I have gotten some borderline produce before. I strongly suspect it's because it's what Whole Foods had available, since I do shop there myself sometimes and have to pass on some produce. I've gotten potatoes that were rotten in the middle, green beans with a lot of brown spots, and strawberries that started getting a little too ripe within 2 days of purchase. I would argue they have the best selection as far as any of the grocery delivery options go. Pretty much anything you can buy in their store (minus alcohol) you can buy through the app. For meat, I still prefer to go with New Seasons because it's more likely to be local and I generally trust the quality more. As well New Seasons is often a bit cheaper. That being said, it's often a toss up for me because I like the overall user experience with Instacart so much better than the other options that sometimes I just suck it up.

Order Modification and Substitutions

You can modify your order after you place it until a shopper starts working on it. It's really easy to do and you don't have to jump through hoops. The only downside is if you selected a delivery time in the next 1-2 hours, the shopper will pick up the order pretty quickly so chances are your window to modify will be very small. You can, however, leave a note for your shopper and they can modify the order.

Specifying substitutions is really easy, the app lets you specify what you want instead, and the shopper will always contact you with any questions.


Instacart does the best job of minimizing waste. Groceries are delivered in a (really strong!) reusable tote bag and you can give your old tote bags to the delivery person to reuse. The Instacart shoppers typically lean towards putting produce into paper bags and generally limiting extra packaging (for instance if you get a squash, it will just be loose in the bag). After an Instacart grocery delivery, I love not having to deal with a pile of boxes or bags after my Instacart delivery. They are the least wasteful of all the options, and tend to reflect my own habits when I do my own shopping.

Delivery Times

Instacart offers same day delivery in a matter of hours. My deliveries usually come at the earliest part of the time range. The app does the best job of all of the options as far as delivery status updates go.

Delivery Cost

Instacart charges a delivery fee (varies according to store) and you are also expected to tip the driver at least 10% of your total. You can pay a flat yearly fee of $99 for "Instacart Express". It's only worth it if you think you'll spend more than $100 a year in delivery fees. In my case, I order from Whole Foods at least 2x per month, so I would save $140 a year. If you spend the minimum, which is $30, then delivery would cost $13. If you paid for Instacart Express, then it would cost you $3.

Comparisons to Going Myself

To put delivery costs and time frames into perspective, I decided to estimate how much it typically costs me to go to the grocery store myself. I don't own a car, so my options are public transit, car service, or car rental.

Public Transit

From where I live, the total time to get to either New Seasons or Whole Foods is about 20 minutes by transit. The cost is $2.50 and about 60-90 minutes of my time (40 minutes there and back, 20-50 minutes of shopping time). I can only buy as much as I can carry, so no large quantities of water, wine, or beer.

Car Service

I could call a taxi or use a ride sharing option like Uber or Lyft. In those cases, the cost of transit (depending on where I'm going) would be whatever the rates are for 2-3 miles. This is usually anywhere from $5-$7 each way. Another option would be to take transit to the store (20 minutes, $2.50) and take a car home ($5-$7, 10 minutes), which means I could buy large items. 

This comes out to 30 minutes of total transportation time (20 minutes by bus there, 10 minutes by car back) plus 20-50 minutes of shopping time, for a grand total of 50-80 minutes. Cost of transportation is $7.50-$9.50. Hiring a car both ways shaves 10 minutes off the total time so it's 40-70 minutes but brings the transportation total up to $10-$14. 

Car Rental

Finally, car sharing with Zipcar and Car2go are other options to consider. I usually get the car for an hour and a half in case of any issues or delays and then drop the reservation down to an hour if I return it in that time frame. This usually costs me $15. Car2go allows me to park and not incur charges while I'm in the store. Travelling to Whole Foods or New Seasons costs about $5 one way (pretty close to Uber). The only trade off is I need to be able to get a car when I leave or else I'll need to call a taxi or take transit to get home. That could make getting large items complicated. Also traffic is a consideration, if I don't want to be stuck in rush hour or in a crowded grocery store with long lines, I have to avoid shopping during certain time windows.

So with using a car, we're looking at $10-$15 total plus 20 minutes of transportation time (10 minutes each way) plus 20-50 minutes of shopping time, so 40-70 minutes of time.


To get the best value out of grocery delivery, you should order larger quantities of food for a longer period of time. Unfortunately, I don't do that because I find I end up throwing away a lot of food, which is basically throwing away money. I cook from scratch and use fresh ingredients, so things will go bad if I don't eat them in time. This means I order groceries 2-3 times per week.

Based on my preferences, my recommendations for the 3 delivery options I've discussed are as follows:

Safeway: Beer, Wine, Bottled Water, Soda, Bulk Household Items; Avoid produce
Amazon Prime Now (New Seasons): Dairy, Produce, Meat, Eggs
Instacart (Whole Foods): Fresh herbs, Pantry, Convenience Foods, Anything New Seasons is out of


Grocery delivery is a good value in terms of delivery cost and time. The highest delivery cost given my use cases is Instacart and the lowest is Amazon Prime Now. Safeway is the best option for bulk (and the only option for alcohol), if you don't need same day delivery and can be at home for the longer delivery window. The cheapest monetary cost is taking transit to the grocery store ($2.50), but it also has the highest time cost.

Update (2/07/2016):

I recently discovered a feature in Instacart that puts it miles ahead of the other grocery options. You can make a custom request for an item that you don't see listed on the website. I did this recently when I needed some Star Anise. I know for a fact that Whole Foods carries it in its bulk spices section. It, however, wasn't showing up on the website. So, I put in a request and bam, I got my bulk Star Anise. Seriously cool feature I'd love to see more services like this adopt!

Tuesday, December 15, 2015

My First OpenStack Summit: Tokyo 2015

In October I attended my first OpenStack Summit. It's taken me awhile to digest and reflect on my experiences hence this article being published over a month after I got back!

I'm not new to tech conferences or contributing to OpenSource, so much of the "Summit 101" stuff didn't really apply to me. That being said, it's worth noting that the OpenStack Summit has essentially 2 tracks, one for users that's more marketing focused, and one for contributors that's more developer or operations focused. I spent pretty much all of my time in the Developer Summit, so I can't speak to the other sessions.

I think the most important thing to remember when attending something like an OpenStack Summit is you need to have something to focus on if you want to get anything out of it. Sure you can float around and fly by the seat of your pants, but at least I'm the type of person that finds it hard to really ingest things unless I have some kind of context or goal to fit it into.

As a developer, I was hired by my current employer to work on OpenStack Infrastructure upstream projects. After some limited involvement with the Infrastructure team, I started getting involved with Nova (aka Compute). This is largely because I'm more of a traditional Software Engineer and generally found the Nova team to be a better fit for my work style and interests. I feel pretty lucky to be in a work setting where making this kind of transition is so easy compared to other places I've worked! So I guess the first piece of advice I have, if you're hired to work on anything, whether it's upstream on an OpenSource project or on internal proprietary things, if something else catches your interest and seems more relevant to your interests than whatever it is you're currently working on, then definitely look for opportunities to change your focus!

Going into the Design Summit, I had just started working on Nova about 2 weeks prior. My goal for the Summit was to gain as much context as possible, and to meet with my new team members face to face.

What I Learned

Reviews are Important

Even if your goal isn't to become a core member on a project, participating in the review process is really vital to keeping up with what's going on in the project. When you review someone's patch, you have to look at code you might not normally look at and learn how it works in order to provide useful feedback. Reviews help you to understand the code base and to participate in the project. In the long term, if you regularly do useful reviews, you might even be considered for a core role on the project (which could be good or bad depending on how much time you have to spend on the project haha)! I'll write an article that's more in depth on doing reviews.

Development Happens in Cycles

Development typically happens in 6 month cycles with milestones identified by the PTL or other core members. This can include a cut off date for proposed specifications and cut off dates for releases. Additionally sometimes there are special sprints for things like bugs and documentation. Teams typically announce this on the mailing list and may link to an etherpad or two with this information.

Be the Change

The number one thing I heard repeated during these sessions was "propose a spec and we'll discuss it". If you have an issue and you want it to be a priority, the number one way to get the most out of your face time with the development team is to propose a specification for it. Don't plan on having a presentation with slides or a demonstration as media equipment and switching laptops could be tricky. Have an etherpad ready and have links to a blueprint in launchpad and a spec in Even if they are in a draft state, it's an artifact people can look at while you're talking about your thing. Finally, expect to do the work for your change. If you can't do it, then do your best to stir up interest before you present the idea.

Where to Find a Team's Priorities

Priorities are the most important items that need to be completed in the next cycle. The goal of the developer sessions in the summit is to discuss issues that are being proposed and determine if they are a priority. If they are not deemed a priority, they can still make it into a release if they have sponsorship, but priorities are going to get the most attention from team members. The team typically provides a link to their priorities list etherpad on the mailing list but you can also find it during sessions. During the last day of the summit, this priority list becomes the agenda as items discussed during the week are revisited.

Non-priority Items are Important Too

As mentioned before, just because something isn't a priority, it can still get attention and make it into a release. If you are willing to champion the issue and get the work done, the only challenge will be getting core attention to push it through to release. In the worst case scenario, you get all the work done and you don't get the attention you need to get it approved and landed. In that case, you just have to wait until the next summit and pitch it again, reminding everyone that the work is done. The summit will also be a good place to get some face time with core members that can provide feedback that might be more challenging to get remotely with barriers like time zones.

What Could Be Improved

I think the biggest thing that could be improved about the developer summit format is the way communication is facilitated. I was very fortunate and spent most of my sessions with a team that has a PTL who is clearly experienced as a facilitator. Unfortunately that is not true for many of the teams and it creates a very frustrating experience for people whose communication styles do not match an extroverted "high involvement" type but follow a more introverted "high politeness" type [1].

Basically the PTL pulls up an etherpad with a list of items to discuss. People in the room talk when they want to. Some people tried raising their hands to signal they had something to say and in some cases when the discussion got very involved, were passed over. Or worse, when they did get the chance to speak, they were cut off before they could finish. While this happened less frequently in some sessions than in others, it happened more frequently than it should have. Many valuable insights and opinions were basically lost.

The most difficult situations happened when people didn't agree on something. I saw some rude behavior that I found rather surprising. Many of us are working for companies who are paying us to be involved in these projects and we are, however indirectly, representing that company. Making backhanded comebacks might make you feel good about yourself in the moment but ultimately it kills discussion and prevents people from listening. I saw this mainly with cross-team discussions where clearly there was some history, but I did see it in at least one team discussion. It's less important for a person to be *right* than it is for the group to come to a good decision about a thing. At the end of the day no one remembers if someone was right or wrong, we remember what was ultimately decided and go on with our business. People do, however, remember if you were a jerk.

I think some kind of group communication 101 and communication guidelines would be really valuable for both participants and PTL's. Additionally, facilitation training for PTL's and other moderators would help to improve the overall discussion flow a lot. This still surprises me in many industries but in tech especially how we not only tolerate antisocial behavior, but practically celebrate it. A person can be smart and be making valuable contributions but if they are driving away contributors, that's a net loss. Engineers tend to be focused on developing technical skills, not social ones. There are a lot of social basics that we don't seem to possess and this is a problem across the industry, not just in OpenStack.

What Was Good

Everyone I met on the Nova team was super nice and fantastic. I felt as a team they were very welcoming and their sessions were some of the best facilitated that I attended. I actually shared my anxiety about participating in discussions with the PTL. He was very nice about it, admitting he'd felt the same way when he started and was overall very encouraging. Another core contributor I've been working with made a point to call me out when something came up that I'd started helping him with, which also helped to break the ice for me, making it easier for me to introduce myself to others in between sessions.

I got a ton of context from the Design Summit sessions. I left feeling so much more in touch and in tune with what the community is doing that now when I'm in irc or reading in the mailing list, I have a better idea of what a person is referring to. As a new contributor, I found a couple of things to get involved with and left the summit with a ton of enthusiasm and energy to dive in.

For me the best thing about the Summit was getting to meet the people, in person, on the team I recently started working. I got to have lunch with a few of them on the last day and I really enjoyed myself.

In general, everyone was so friendly and I'm really looking forward to the Nova Mid cycle.

Final Thoughts

I know I ranted a lot about some of the downsides of the Summit, but to be fair, this is not at all unique to the OpenStack community. I don't want to discourage anyone who is thinking about participating, but I don't want people to believe any nonsense from anyone trying to say that OpenStack somehow doesn't have these issues. Everyone has these issues, including every company I've ever worked for. Personally I think the road to improvement is a focus on general inclusiveness rather than targeted demographics. I've provided some ideas on how communication could be improved as a step in that direction and I look forward to seeing how things evolve.

It's truly amazing that companies are seeing the value of Open Source enough to pay people to work on it full time. I'm also amazed that so many employees of so many different companies, representing so many different interests (and nationalities) can come together and work together to build a pretty neat thing that so many people are using! That's a real accomplishment and I'm definitely looking forward to my next Summit.

[1] The labels "High Involvement" and "High Politeness" (aka "High Considerateness") originated with the Sociolinguist Deborah Tannen. You can read a book of her theories, Conversational Style, or here's the short version on Wikipedia.

Tuesday, September 22, 2015

My First OpenStack Infra Patch

In August, I started a new job as an upstream contributor on the OpenStack Infrastructure team! The project is so huge, the average on-boarding time is at least 6 months. My goal for this on-boarding time is to document as much as possible while I still have my "new contributor" perspective. I have been focusing on documentation for various Infra projects, including Nodepool and Disk Image Builder.

I had my first OpenStack Infrastructure patch merged on September 2nd, 2015! I'd like to share my experiences writing, submitting, and debugging my first patch.

Step 1: Figure out what to patch

There's a talk I give about how to get started with contributing to the Perl community. Some of the advice I've given there holds true to pretty much any project, but especially any project that is big and overwhelming. Sure, you can submit code as your first patch. For most of us, however, there's a lot of context missing when we interact with a new code-base (and Infra has MANY contexts and code bases). I recommend what I like to call the "Wax-On Wax-Off Paint-the-Fence" approach.

Take on the most ridiculously boring and tedious tasks while you learn how things work.

Good examples are things like adding or improving documentation, adding, fixing, or otherwise improving unit tests, or small code tasks like variable or function renames, formatting changes, whitespace cleanup, or logging or other text output format changes. Don't do anything too invasive until you understand how all the pieces fit together. You can't debug a failure if you don't know where to look, and this is especially true for big projects like Infra!

The team I joined has a policy of assigning an onboarding buddy to new hires. I was lucky to get to work with Greg Haynes. He's done a great job of suggesting tasks like the ones mentioned above that exist in the code bases he's most active in.

My first patch involved reformatting text in a README file for one of the elements in Disk Image Builder. A proposal was made to move this from free text that was all over the place and inconsistent to a formatted table structure with clear items that should be listed for each element. This is a very tedious project and not particularly interesting to someone who is very familiar with the code base. For someone like me, however, it's perfect! So that's my current project, reformatting and improving the documentation for the environment variable overrides in Disk Image Builder's elements.

If you don't have a buddy to make recommendations, just pop into the #openstack-infra channel in IRC and just ask for suggestions! Alternatively, try the #openstack-doc and #openstack-qa channels.

Step 2: Set Up Gerrit and git-review

OpenStack has done an excellent job of documenting the steps for new developers and I recommend following those instructions after reading some of my notes here. There are more in-depth instructions provided by the Documentation team for first time contributors that may be a better fit if you don't have a lot of experience contributing to projects!

OpenStack uses a code review tool called Gerrit hosted at and uses Ubuntu Single Sign On to manage login credentials. When I joined the Infrastructure team, I already had an Ubuntu Single Sign On account via Launchpad from contributions I'd made to Ubuntu a few years ago.

Additionally, to contribute to any OpenStack project (including Infra), you'll need to create a community account and sign the agreement. Make sure the email you provide for your OpenStack email address matches your Ubuntu Single Sign On email address! I ran into this issue myself and I've seen it enough that I'll bet it's one of the top new contributor issues! If you have any issues with your OpenStack community account, the first thing to check is that those email addresses match.

Finally, install git-review on your system. As a side note, for development projects, I generally try to minimize system-wide installs that could affect my entire system. This is especially important when dealing with several projects that may have conflicting version requirements. That being said, git-review is fine to install system-wide, just be aware in the future when some instructions tell you to install something system-wide and think about possible conflicts with any other projects you might be using. See the next section for suggestions on not littering dependencies all over your local file system.

Step 3: Get the Code

Infra code, like all OpenStack code, lives in a git repository hosted on To find the repository for the project you want to patch, look up the project by name at using the Search box at the upper right corner. Copy the “git://” URI to clone the git repo locally.

For your local dev setup on your local machine, I recommend creating a folder for all of your OpenStack related projects. I have a folder called "dev" in my home directory that contains subfolders for various projects. Under that, I created a folder called openstack where I clone my git repos. For OpenStack infra, you will want to create a separate openstack-infra directory because openstack-infra can have different dependencies than other OpenStack projects. It should use its own virtualenv to manage these dependencies.

In general you should be using a local Python rather than a system Python. You know if you're using system Python if you have to type "sudo" anytime you want to do anything beyond running a program (like pip). Virtualenv makes doing this really easy. Learn more about using virtualenv here.

There's another great tool for managing environment variables that's written in bash, called smartcd. It lets you set up custom environment variables on a per directory basis. A lot of OpenStack projects depend on environment variables so having something to customize this without littering your environment is really useful.

I also recommend that you set up a virtual machine with a Devstack. If you're editing docs or unit tests, it's not too big of a deal to run things locally but when you're doing larger code patches, you want to mimic running tools against an OpenStack as much as possible. For the documentation patches I've done, running locally and using a virtualenv has been sufficient.

Step 4: Run the Tests

Once you've cloned the git repository, follow the instructions in the README at the parent level to do any necessary local setup. It will likely involve installing a bunch of python dependencies. See the section above for my recommendation involving using virtualenv for this.

If the project you're working on does not have a README, or the README doesn't mention anything about how to set up the repo, this is definitely an opportunity to add a patch. Every project should have a README and it should contain not only setup information but also bug submission information. These are good first patches as you get to know the infrastructure.

Some projects are complex, require a lot of setup and possess a huge range of dependencies. If your change doesn’t actually require running the project, don’t get overwhelmed by getting the project up and running in order to submit your first patch. You can easily run a subset of the unit tests locally to verify your change without having to locally test the entire project.

After configuring and installing, before touching anything else, I highly recommend running the tests to make sure everything works on your system. It's incredibly hard to troubleshoot when you're not sure if your patch broke something or if it was a setup issue that's causing test failures. To run the tests you'll need to set up tox if you haven't already. It's really straightforward (pip install tox), see the OpenStack Python Developer Docs if you're not sure how.

The tox.ini file at the parent level of the repo defines the different sections that tox recognizes. So if all you really care about is docs, there should be a section called "docs" and it will be clear in the tox.ini what tests are being run as part of that. If you are only touching a specific subset of things (eg, docs), then don't worry about troubleshooting if the entire test suite fails on your local machine.

Unit tests can fail for a variety of reasons, including versions of dependent libraries and OS-level conflicts. This is why it’s important to a) use a tool like virtualenv to separate Python language dependencies and b) to use a virtual machine or other container to avoid system level conflicts that are hard to debug.

Step 5: Hack Hack Hack

Before you start hacking, you should cut a local branch to commit to. However, if you suspect an upstream conflict is going to be merged while you’re working, you can hack on master without committing, and then cut the branch once you're ready to commit. The advantage of just hacking on master (without committing) is you can easily keep the branch current without dealing with merge conflicts. If you accidentally commit to master, Gerrit won't let you merge when you check in your commit. You’ll need to cut a local branch to submit your code upstream anyway, so it's generally just good practice to work on a local branch.

Step 6: Run the Tests

After you're done hack hack hacking, run the tests again to make sure you didn't break anything. Again, only run a subset if that's all you care about. This will just catch anything specific to what you're changing. Jenkins will run the full suite of unit and integration tests once you submit your patch for review.

Step 7: Compose a Commit Message and Rebase

Composing a well thought out commit message is important, even for small changes. The typical OpenStack commit style is to do a short description followed by a longer paragraph that provides more context for the change. If the change is really minor, you can get away with a short description. Learn more about writing good commit messages here.

Gerrit will create one review item per commit. If you have multiple commits, you need to rebase them down to one commit. My workflow is to write my real commit message for the first commit and then do short, less descriptive commits for anything subsequent. Once I'm done developing, I do rebase -i HEAD~n where n is the number of commits (inclusive) to rebase. When you do an interactive rebase, you can tell git what you want to do with each of the commits.

It might be tempting to just constantly rebase your work into one commit to save yourself the trouble. While doing your initial work, I don't recommend this because you might make a mistake that's hard to back out of without your full commit history.

To keep a remote copy of your branch without having it go through testing, you basically push your change to Gerrit but then mark it as a Work in Progress (WIP) through a workflow comment. Run git review, then open the patch's page in Gerrit. Click Review on your patch and mark it as "-1 WIP". This will let everyone (including Jenkins) know that this patch isn't ready for testing just yet.

Note that if you want to keep a remote copy of your work in Gerrit, you'll need to rebase it down to one commit. I haven't figured out an alternative solution that keeps the repo in sync with the master branch but lets you have a remote copy of your revision history without creating a bunch of extra patches in Gerrit. When I figure something out I'll be sure to talk about it in a future article.

Step 8: Submit Your Patch

Before running git-review, make sure the email address in your git config matches the one you have registered in Gerrit. Just type "git review" to submit your patch. If you want to submit the patch as a work in progress, see the above section about doing a workflow commit.

Once you’ve submitted your patch for review, the build system will run a huge suite of tests on the remote branch in Gerrit. Pay attention to your emails for the test results from Jenkins. If you need to troubleshoot a failure, do so by looking at the Jenkins console logs. Ask for help in the #openstack-infra IRC channel if you're not sure what something means, and try to provide as much information as possible including a link to the change in question and specific error messages you think are the problem. For longer pastes, use instead of cluttering up the IRC channel. Check if the failed tests are "non-voting" before digging into things. If the tests are "non-voting" their failure doesn't matter with regards to your patch being accepted because they may have known issues.

If it looks like a fluke, you can re-run the tests by adding a comment with the text "recheck". All automated tests must pass for code to be merged, and typically reviewers won’t even begin reviewing a new patch until the checks have passed.

Step 9: Get Your Patch Reviewed

Once the test have passed, you'll need to get your change accepted by 2 core members in order to get it merged. If your patch has been reviewed and says "Needs Workflow" it means another core needs to approve it before it can be merged.

To find out who the core members are that can +2 or approve your patch, visit the project's page in Click on "Access" at the top of the screen and then click on any of the project-core links to see a list of people. You can add names as Reviewers to your change from your change's url or you can ask in #openstack-infra if those individuals could review your change. Not everyone monitors their emails if you just add them as a Reviewer.

If you get comments and people want you to make changes, generally discussion should happen in Gerrit so there’s some history tied to the change. However, if you’re still uncertain or need more back and forth than Gerrit can offer, feel free to discuss the change publicly in the #openstack-infra IRC channel, or privately with the reviewer on IRC if you’re not yet comfortable speaking in the public channel. Many people in the community are very passionate about wanting the best for OpenStack, and they have strong opinions as a result. If you don't agree with specific feedback because you feel you had good reasons for why you did a thing the way you did, feel free to politely share those reasons. Chances are you'll have a nice discussion and you'll both learn something ;) If for some reason you're not satisfied with the result of the discussion, feel free to seek a second opinion from another core member. It's ok if you get overruled, don't be frustrated. Just try your best to understand why they want it done that way and move on to the next thing. When you're working with groups, sometimes you have to pick your battles. Once you get more established in the community and understand more of the context, you might better understand their reasons or you can be the core person making those decisions. It's all just part of being in a community with other humans :)

Once your patch has been approved, it will be automatically merged and you'll get an email notification (if you've opted for those)!

Step 10: Do the First Patch Dance!

You can watch your progress as an OpenStack contributor at Stackalytics!

Useful Links

Sunday, July 26, 2015

Bike to Work Jeans

A lot of options are entering the market for bicycle commuter clothing, including Levis, Betabrand, ... The main problem I have is the jeans are very low rise and don't come in sizes that fit me. Recently I was browsing L.L. Bean and stumbled across the most amazing thing... "active" jeans! L.L. Bean does a great job of accommodating a variety of shapes and sizes and their quality is pretty good.

I particularly love L.L. Bean jeans because they are made for Pear shaped women. The legs are usually a bit more generous than typical jeans and the waists are more nipped in with a higher rise.

The Performance Stretch Jeans are skinny jeans but they are more of a comfortable slim fit rather than a super squeezed in fit. If you have chunky legs due to either muscle or fat, these will fit you. If your waist is smaller than your hips by a lot, these will fit you.

Finally, they are made of a special performance material and are incredibly durable. I've worn them 3-4 times a week non-stop since I got them earlier this year and they have shown zero signs of wear besides a bit of fading. They also dry faster than regular jeans (although I would not call them "quick drying"). If you tumble dry low for 30 minutes after a spin cycle, they'll easily be done within that time. Hang drying, they are dry within 1-2 hours depending on humidity and air flow. If you're washing in the sink, they'll be dry overnight if you can wring them out with a towel. If you sweat in them or get drenched in a downpour, they'll dry within an hour.

L.L. Bean Performance Stretch Jeans are great for biking!

Another feature I love are the little reflector tabs on the back belt loop and the cuffs!

I have taken these on short trips travelling and I've been impressed at not only the overall durability but their tendency to not retain oders.

Sizing tip, if you want a really tight, skinny fit, then order down. Otherwise, you can safely go by the size chart. It is true to size and if your hips are a little wider than what's listed, don't worry about it because they are stretch jeans and they will stretch out a little bit. My measurements are 34" high waist (38" low waist), 48" hips. No size 20 was available but it would have been too large in the waist. The size 18 fits me perfectly.

L.L. Bean Performance Stretch Jeans, Back View, size 18

L.L. Bean Performance Stretch Jeans, Front View, size 18

If you're looking for an all purpose workhorse jean, something you can do active things in, travel, and still look nice in, I can't recommend these enough!

Sunday, March 22, 2015

Grain Free Cupcake v1

When I went off Paleo last year, I learned a lot about Gluten Free baking and even made a pretty decent sandwich bread thanks to America's Test Kitchen. Gluten Free cooking is rife with processed foods and additives, but there's some sound research on the science of baking and how to mix and match gluten free substitutes to make it work. America's Test Kitchen Gluten Free Cookbook is among the best out there, not so much for the actual recipes, but for understanding how the ingredients work together.

If you're going to engage in Grain Free baking, I find that gluten free recipes are much easier to start from than recipes depending on wheat flour. Besides ATK's book, I have this book by Bruce Fife called "Cooking with Coconut Flour" which I use as the basis for most of my experimentation with baking. I don't think I've yet followed a recipe exactly, but that's kind of my style of cooking.

The basic components you're looking for are:

  • insoluble fiber for bulk
  • starch for lightness
  • protein for rise and browning
  • soluble fiber for binding

The problem with just these components is that they don't quite mimic gluten. You can bake pretty good things with just these, but they won't really rise well or have a nice crumb. Despite the soluble fiber, they will still be a bit dry and crumbly. Many Gluten Free recipes use gums as an extra binder to more closely mimic the structure gluten builds which improves rise. These gums tend to cause me a lot of gastric distress even in small amounts (and I'm not alone in this). But wait! There's hope!

Recently I've been reading on many gluten free cooking blogs that potato flour can substitute quite well for the gums typically used in gluten free baking to replicate gluten's elasticity. I've actually used boiled potato with wheat flour in the past to improve the texture of the final bread. It makes the texture a bit more soft and tender, which is something non-grain flours definitely need. The advantage of using potato flakes (ie, instant potatoes) or potato flour (different from potato starch) is that you don't get water weight variations like you do with fresh potatoes. That being said, fresh potatoes are cheap and don't require a trip to a special grocery store. And, chances are there aren't any additives other than dirt.

I'd been wanting to try making cupcakes for some time but hadn't had the right inspiration. I appreciate coconut flour, however I'm not the hugest fan. It tastes weird, it's dry and crumbly and you have to use crap tons of eggs to even make anything decent with it. Not to mention all that insoluble fiber just goes right through me. After reading about potato flour, I got inspired. So tonight I tried my first attempt at grain-free cupcakes, and I'm quite happy with the results!

If you are looking for a more frequent treat, you can skip the icing and use minimum amount of honey (2T). For a special celebratory treat, I don't see anything wrong with a little refined sugar frosting soaked in butter.

Paleo Cupcake v1 - The secret ingredient is POTATO!

Grain-Free Cupcake v1

The Wet:
2 eggs
2 - 4T honey
3 T melted butter
1/4 t sea salt
1/2 t vanilla
2.5 oz or 72g of cooked white potato (floury or starchy, doesn't matter)

The Dry:
2 T coconut flour
1 T tapioca flour
1/4 t baking powder (homemade with 1t baking powder + 2t cream of tartar)

Beat the eggs and honey together, then add the butter, salt, and vanilla. Mix until well blended. Press the cooked potato through a strainer or ricer and mix in until smooth.

In a separate bowl, mix together the dry ingredients and sift them into the wet ones. Mix until you have a wet, smooth batter.

I used a 12 count USA Pans muffin pan. Line the middle 2 rows (inner 6 spots) with greased paper cups and pour the batter so it fills the up about halfway. You should have enough for 6 cupcakes.

Bake in the upper part of the oven at 400 degrees Fahrenheit for 15 minutes. Let cool completely before icing or eating so the starch has time to set. These should rise to the top of the muffin compartment.

Buttercream Frosting

Any good cupcake of course needs frosting and if you're looking for a simple buttercream, my recipe is adapted from King Arthur Flour's. Granted the use of refined sugar may be offensive to some die-hards out there, but I've always considered cupcakes more of a rare celebration item and not so much of an every day of the week phenomena.

.75 ounces butter
.25 ounces lard
1/4 t vanilla
tiny pinch of salt
4 ounces confectioners sugar (I use 365 Organic with tapioca flour as the starch)
2T milk or cream

Let the fats get to room temperature, then beat them until creamy. Add 2 oz of the confectioners sugar and beat until blended, then mix in the salt and the vanilla. Add 1 T of the milk, then alternate adding in a bit of the confectioners sugar and milk until it's smooth and creamy. I put mine in a plastic sandwich bag and snipped off the corner for an instant mini pastry bag. In my photos, I've added a bit of lime zest.

If you want to add food coloring, I use gel food coloring. For a more natural alternative, you could maybe mix some beet juice and gelatin, but I haven't tried it myself ;) 

Outer crumb of Grain-Free Cupcake v1

Inner crumb of Grain-Free Cupcake v1

This cupcake is a little dense but really not too bad. The most important thing it stays together really well and actually peels off the paper almost like a regular cupcake. The coconut flour makes it a little grainy, possibly soaking the flour for a bit before putting it into the oven could help with that. I also realized I screwed up my homemade baking powder ratio when I was writing this blog and ended up doing 3 baking soda to 1 cream of tartar. I'm going to bet that these would have risen better if that had actually been right ;)

Final Thoughts

Potatoes have been a controversial subject in the Paleo community. My stance on it is my diet is not a religion and I base my diet decisions on science. Do YOUR OWN elimination/reintroduction experiment to determine what to include or exclude from your diet. There's a lot of pseudoscience when it comes to nutrition and the Paleo community is no exception (serious Snake Oil Peddling, amirite?). Recent studies have shown a diet high in starchy tubers does not necessarily lead to diabetes and obesity. The energy in starchy tubers is intended to store food and water for a plant for a long period of time, whereas a grain's starch is intended to be rapidly converted to sugar to feed a germinating seed. Ever done any home brewing? Ever tried it with potatoes? Because potato starch is intended to be released slowly over a long period of time, fermenting grain works a little better. So, logically, it seems reasonable to conclude our bodies would process the starches differently as well.

But really the ultimate test is your own reaction to the foods that you eat. In my case, I ate 1 small white potato. One hour later, I checked my blood sugar - 105 (resting is 90). Within my normal post meal range. No digestive issues or reactions. So much for that whole "glycemic index" nonsense.

Saturday, January 10, 2015

Cream of Mushroom and Leek Soup

My husband really likes Cream of Mushroom Soup. I think it's ok but it's not my favorite. I usually like to add some oomph to it to make it more interesting. Leeks will give the soup a nice sweetness, so make sure you saute them so they are sufficiently caramelized.

Mushroom soup isn't particularly photogenic


4 T Butter
16 oz Mushrooms, separated into caps and stems, caps sliced
2 medium or 1 giant leek, chopped
1 quart (give or take) Chicken Stock (or other light stock)
8 oz Heavy Cream


Salt and Pepper to taste
Nutmeg (to taste)
1/4 Lemon (for juice)


4T Butter
1/4c + 2T Flour


1. Saute the mushrooms and leeks in the butter for about 15-20 minutes on low until they start to emit an aroma and they are soft and shiny.
2. While the mushrooms and leeks are sauteing, add the mushroom stems to the chicken broth and simmer on medium. If using unseasoned or packaged chicken broth, add some aromatics at this time to boost the flavor.
3. Add the chicken stock to the mushrooms and leeks and simmer for 1 hour.
4. In a separate pan, make the roux using the flour and butter. Bring the soup to a boil then add the roux to the soup.
5. Boil the soup for 5 minutes, stirring constantly, until thick
6. Turn off the heat and add the cream. If the soup is too watery, you can turn the heat up to high and boil it until it is thick enough, just make sure you stir it frequently to prevent the bottom from scalding.
7. Add seasonings and lemon juice.

Cooking Tip

Lemon or Lime Juice is my secret ingredient. The citrus adds a wonderful brightness to dense dishes and often it's not obvious that there was lemon specifically added to the dish. I usually use a little lemon juice in stews and vegetables. Orange juice and zest works great with squash. Lime is especially nice with coconut milk or bean-based soups.

If you're worried about seeds or pulp getting into your food, you can squeeze the lemon or lime through a strainer. I do not recommend pre-squeezed juices, they aren't as good as the real deal and getting it straight from the fruit really isn't hard.

For a great refreshing soda-like beverage, squeeze 1/4 to 1/2 of a lemon into a 16 oz glass and top up with cold seltzer water. You can also add the same amount of juice when you refill your water bottle for the gym or what have you with filtered water. I often put my daily dose of liquid magnesium supplement into this mixture because the lemon covers up the bitterness!