Tuesday, September 22, 2015

My First OpenStack Infra Patch

In August, I started a new job as an upstream contributor on the OpenStack Infrastructure team! The project is so huge, the average on-boarding time is at least 6 months. My goal for this on-boarding time is to document as much as possible while I still have my "new contributor" perspective. I have been focusing on documentation for various Infra projects, including Nodepool and Disk Image Builder.

I had my first OpenStack Infrastructure patch merged on September 2nd, 2015! I'd like to share my experiences writing, submitting, and debugging my first patch.

Step 1: Figure out what to patch


There's a talk I give about how to get started with contributing to the Perl community. Some of the advice I've given there holds true to pretty much any project, but especially any project that is big and overwhelming. Sure, you can submit code as your first patch. For most of us, however, there's a lot of context missing when we interact with a new code-base (and Infra has MANY contexts and code bases). I recommend what I like to call the "Wax-On Wax-Off Paint-the-Fence" approach.

Take on the most ridiculously boring and tedious tasks while you learn how things work.

Good examples are things like adding or improving documentation, adding, fixing, or otherwise improving unit tests, or small code tasks like variable or function renames, formatting changes, whitespace cleanup, or logging or other text output format changes. Don't do anything too invasive until you understand how all the pieces fit together. You can't debug a failure if you don't know where to look, and this is especially true for big projects like Infra!

The team I joined has a policy of assigning an onboarding buddy to new hires. I was lucky to get to work with Greg Haynes. He's done a great job of suggesting tasks like the ones mentioned above that exist in the code bases he's most active in.

My first patch involved reformatting text in a README file for one of the elements in Disk Image Builder. A proposal was made to move this from free text that was all over the place and inconsistent to a formatted table structure with clear items that should be listed for each element. This is a very tedious project and not particularly interesting to someone who is very familiar with the code base. For someone like me, however, it's perfect! So that's my current project, reformatting and improving the documentation for the environment variable overrides in Disk Image Builder's elements.

If you don't have a buddy to make recommendations, just pop into the #openstack-infra channel in IRC and just ask for suggestions! Alternatively, try the #openstack-doc and #openstack-qa channels.

Step 2: Set Up Gerrit and git-review


OpenStack has done an excellent job of documenting the steps for new developers and I recommend following those instructions after reading some of my notes here. There are more in-depth instructions provided by the Documentation team for first time contributors that may be a better fit if you don't have a lot of experience contributing to projects!

OpenStack uses a code review tool called Gerrit hosted at review.openstack.org and uses Ubuntu Single Sign On to manage login credentials. When I joined the Infrastructure team, I already had an Ubuntu Single Sign On account via Launchpad from contributions I'd made to Ubuntu a few years ago.

Additionally, to contribute to any OpenStack project (including Infra), you'll need to create a community account and sign the agreement. Make sure the email you provide for your OpenStack email address matches your Ubuntu Single Sign On email address! I ran into this issue myself and I've seen it enough that I'll bet it's one of the top new contributor issues! If you have any issues with your OpenStack community account, the first thing to check is that those email addresses match.

Finally, install git-review on your system. As a side note, for development projects, I generally try to minimize system-wide installs that could affect my entire system. This is especially important when dealing with several projects that may have conflicting version requirements. That being said, git-review is fine to install system-wide, just be aware in the future when some instructions tell you to install something system-wide and think about possible conflicts with any other projects you might be using. See the next section for suggestions on not littering dependencies all over your local file system.

Step 3: Get the Code


Infra code, like all OpenStack code, lives in a git repository hosted on git.openstack.org. To find the repository for the project you want to patch, look up the project by name at git.openstack.org using the Search box at the upper right corner. Copy the “git://” URI to clone the git repo locally.

For your local dev setup on your local machine, I recommend creating a folder for all of your OpenStack related projects. I have a folder called "dev" in my home directory that contains subfolders for various projects. Under that, I created a folder called openstack where I clone my git repos. For OpenStack infra, you will want to create a separate openstack-infra directory because openstack-infra can have different dependencies than other OpenStack projects. It should use its own virtualenv to manage these dependencies.

In general you should be using a local Python rather than a system Python. You know if you're using system Python if you have to type "sudo" anytime you want to do anything beyond running a program (like pip). Virtualenv makes doing this really easy. Learn more about using virtualenv here.

There's another great tool for managing environment variables that's written in bash, called smartcd. It lets you set up custom environment variables on a per directory basis. A lot of OpenStack projects depend on environment variables so having something to customize this without littering your environment is really useful.

I also recommend that you set up a virtual machine with a Devstack. If you're editing docs or unit tests, it's not too big of a deal to run things locally but when you're doing larger code patches, you want to mimic running tools against an OpenStack as much as possible. For the documentation patches I've done, running locally and using a virtualenv has been sufficient.

Step 4: Run the Tests


Once you've cloned the git repository, follow the instructions in the README at the parent level to do any necessary local setup. It will likely involve installing a bunch of python dependencies. See the section above for my recommendation involving using virtualenv for this.

If the project you're working on does not have a README, or the README doesn't mention anything about how to set up the repo, this is definitely an opportunity to add a patch. Every project should have a README and it should contain not only setup information but also bug submission information. These are good first patches as you get to know the infrastructure.

Some projects are complex, require a lot of setup and possess a huge range of dependencies. If your change doesn’t actually require running the project, don’t get overwhelmed by getting the project up and running in order to submit your first patch. You can easily run a subset of the unit tests locally to verify your change without having to locally test the entire project.

After configuring and installing, before touching anything else, I highly recommend running the tests to make sure everything works on your system. It's incredibly hard to troubleshoot when you're not sure if your patch broke something or if it was a setup issue that's causing test failures. To run the tests you'll need to set up tox if you haven't already. It's really straightforward (pip install tox), see the OpenStack Python Developer Docs if you're not sure how.

The tox.ini file at the parent level of the repo defines the different sections that tox recognizes. So if all you really care about is docs, there should be a section called "docs" and it will be clear in the tox.ini what tests are being run as part of that. If you are only touching a specific subset of things (eg, docs), then don't worry about troubleshooting if the entire test suite fails on your local machine.

Unit tests can fail for a variety of reasons, including versions of dependent libraries and OS-level conflicts. This is why it’s important to a) use a tool like virtualenv to separate Python language dependencies and b) to use a virtual machine or other container to avoid system level conflicts that are hard to debug.

Step 5: Hack Hack Hack


Before you start hacking, you should cut a local branch to commit to. However, if you suspect an upstream conflict is going to be merged while you’re working, you can hack on master without committing, and then cut the branch once you're ready to commit. The advantage of just hacking on master (without committing) is you can easily keep the branch current without dealing with merge conflicts. If you accidentally commit to master, Gerrit won't let you merge when you check in your commit. You’ll need to cut a local branch to submit your code upstream anyway, so it's generally just good practice to work on a local branch.

Step 6: Run the Tests


After you're done hack hack hacking, run the tests again to make sure you didn't break anything. Again, only run a subset if that's all you care about. This will just catch anything specific to what you're changing. Jenkins will run the full suite of unit and integration tests once you submit your patch for review.

Step 7: Compose a Commit Message and Rebase


Composing a well thought out commit message is important, even for small changes. The typical OpenStack commit style is to do a short description followed by a longer paragraph that provides more context for the change. If the change is really minor, you can get away with a short description. Learn more about writing good commit messages here.

Gerrit will create one review item per commit. If you have multiple commits, you need to rebase them down to one commit. My workflow is to write my real commit message for the first commit and then do short, less descriptive commits for anything subsequent. Once I'm done developing, I do rebase -i HEAD~n where n is the number of commits (inclusive) to rebase. When you do an interactive rebase, you can tell git what you want to do with each of the commits.

It might be tempting to just constantly rebase your work into one commit to save yourself the trouble. While doing your initial work, I don't recommend this because you might make a mistake that's hard to back out of without your full commit history.

To keep a remote copy of your branch without having it go through testing, you basically push your change to Gerrit but then mark it as a Work in Progress (WIP) through a workflow comment. Run git review, then open the patch's page in Gerrit. Click Review on your patch and mark it as "-1 WIP". This will let everyone (including Jenkins) know that this patch isn't ready for testing just yet.

Note that if you want to keep a remote copy of your work in Gerrit, you'll need to rebase it down to one commit. I haven't figured out an alternative solution that keeps the repo in sync with the master branch but lets you have a remote copy of your revision history without creating a bunch of extra patches in Gerrit. When I figure something out I'll be sure to talk about it in a future article.

Step 8: Submit Your Patch


Before running git-review, make sure the email address in your git config matches the one you have registered in Gerrit. Just type "git review" to submit your patch. If you want to submit the patch as a work in progress, see the above section about doing a workflow commit.

Once you’ve submitted your patch for review, the build system will run a huge suite of tests on the remote branch in Gerrit. Pay attention to your emails for the test results from Jenkins. If you need to troubleshoot a failure, do so by looking at the Jenkins console logs. Ask for help in the #openstack-infra IRC channel if you're not sure what something means, and try to provide as much information as possible including a link to the change in question and specific error messages you think are the problem. For longer pastes, use paste.openstack.org instead of cluttering up the IRC channel. Check if the failed tests are "non-voting" before digging into things. If the tests are "non-voting" their failure doesn't matter with regards to your patch being accepted because they may have known issues.

If it looks like a fluke, you can re-run the tests by adding a comment with the text "recheck". All automated tests must pass for code to be merged, and typically reviewers won’t even begin reviewing a new patch until the checks have passed.

Step 9: Get Your Patch Reviewed


Once the test have passed, you'll need to get your change accepted by 2 core members in order to get it merged. If your patch has been reviewed and says "Needs Workflow" it means another core needs to approve it before it can be merged.

To find out who the core members are that can +2 or approve your patch, visit the project's page in review.openstack.org. Click on "Access" at the top of the screen and then click on any of the project-core links to see a list of people. You can add names as Reviewers to your change from your change's url or you can ask in #openstack-infra if those individuals could review your change. Not everyone monitors their emails if you just add them as a Reviewer.

If you get comments and people want you to make changes, generally discussion should happen in Gerrit so there’s some history tied to the change. However, if you’re still uncertain or need more back and forth than Gerrit can offer, feel free to discuss the change publicly in the #openstack-infra IRC channel, or privately with the reviewer on IRC if you’re not yet comfortable speaking in the public channel. Many people in the community are very passionate about wanting the best for OpenStack, and they have strong opinions as a result. If you don't agree with specific feedback because you feel you had good reasons for why you did a thing the way you did, feel free to politely share those reasons. Chances are you'll have a nice discussion and you'll both learn something ;) If for some reason you're not satisfied with the result of the discussion, feel free to seek a second opinion from another core member. It's ok if you get overruled, don't be frustrated. Just try your best to understand why they want it done that way and move on to the next thing. When you're working with groups, sometimes you have to pick your battles. Once you get more established in the community and understand more of the context, you might better understand their reasons or you can be the core person making those decisions. It's all just part of being in a community with other humans :)

Once your patch has been approved, it will be automatically merged and you'll get an email notification (if you've opted for those)!

Step 10: Do the First Patch Dance!




You can watch your progress as an OpenStack contributor at Stackalytics!


Useful Links

62 comments:

  1. Hiii...
    I have read this blog....very informative above the blog...Thank for sharing the amazing post...
    Openstack Training

    ReplyDelete
  2. Recently download configure guide from link provided, hopefully it works for me. I am also looking for OpenStack alternative
    .

    ReplyDelete
  3. Hello,
    Opensource software platform tools for building and managing the cloud computing solutions for public and private clouds. Thank you for the Article, Your Blog is having Lotzz of Stuff about Openstack... Here i am also Sharing Information on OpenStack Administration Training. It will Definately useful for Everyone.

    ReplyDelete
  4. This is the exact information I am been searching for, Thanks for sharing the required infos with the clear update and required points. To appreciate this I like to share some useful information.sap ehs training in bangalore

    ReplyDelete
  5. I really enjoy reading this article.Hope that you would do great in upcoming time.A perfect post.Thanks for sharing.sap bi training in bangalore

    ReplyDelete
  6. Excellent information with unique content and it is very useful to know about the information.sap hybris training in bangalore

    ReplyDelete
  7. Thanks for sharing it with us. I am very glad that I spent my valuable time in reading this post.sap ui5 training in bangalore

    ReplyDelete
  8. These provided information was really so nice,thanks for giving that post and the more skills to develop after refer that post.sap testing training in bangalore

    ReplyDelete
  9. Really it was an awesome article,very interesting to read.You have provided an nice article,Thanks for sharing.sap successfactor training in bangalore

    ReplyDelete
  10. It will be good if you post here youtube video about your patch too. Some people don't like to read and want to watch video. On this site https://soclikes.com/ I buy youtube likes for my video, just want to share

    ReplyDelete
  11. What To Do If Yahoo Is Not Receiving Emails?
    If you are struggling with Yahoo not receiving emails, check the Yahoo server status by going to DownDetector. If you find the server down, wait for some time till the server goes normal. You should also check your Spam folder as there are chances that your emails, instead of going to your mailbox, are going to the Spam folder.
    Related Blogs -
    How To Resolve if Yahoo Mail Won’t Load Emails?
    Best Ways To Resolve Yahoo Email Search Not Working Issues
    Updating Yahoo Mail on iPhone
    Recover Permanently Deleted Emails From Yahoo
    Trouble signing into yahoo mail

    ReplyDelete
  12. How to troubleshoot Spectrum Email Not Working on Mac?
    If you are facing the issue of Spectrum Email Not Working on Mac, check your internet connection as a poor internet connection could be one of the main reasons for the problem. Also, check if the incoming and outgoing mail server settings are correctly configured and if not, then re-configure the Spectrum account on Mac to fix the issue.
    Related Blogs -
    Gmail Login problems
    Gmail not working
    Google Account Recovery
    Gmail isn't receiving Emails
    AOL Mail Settings
    How To Change AOL Password
    AOL Mail Login Problems

    ReplyDelete
  13. eventprofs. Virtual & Hybrid Event Solution uses a content manager with an intuitive interface for adding event elements and When it comes to designing event pages and a what-you-see-is-what-you-get theme editor makes them easily configurable to your brand esthetic. event reminder wording and work party invitation

    ReplyDelete
  14. If you are finding best location for wedding venues or banquet halls in chattarpur Delhi ,or best farms in chattarpur delhi Please visit official website Fnpgardens.

    wedding venues in chattarpur
    banquet halls in south delhi

    fnpgardens Delhi

    ReplyDelete
  15. What To Do If My Yahoo Account Account Is Temporarily Locked?
    If your Yahoo Account Temporarily Locked, then you need to go to the Yahoo sign-in helper page and enter the information out of the available options. You'll get a code on whatever you've entered the information. Enter the code, and you'll regain access to your Yahoo account.
    Related Blogs -
    Yahoo Mail Problems on iPhone
    Yahoo Mail Not Syncing on Android
    How To Block Email on Yahoo
    Recover Permanently Deleted Emails From Yahoo
    How to Recover Hacked Yahoo Account

    ReplyDelete
  16. Hi, I'm kimjolly. It's my initiative for learning in the field of Mandarin in online chinese course UAE. if you are keen interested in learning Feel free to visit my website also contact our 24*7 assistance for all query related to learning Mandarin.

    ReplyDelete
  17. Hi, I'm kimjolly. It's my initiative for learning in the field of Mandarin in best chinese teachers UAE. if you are keen interested in learning Feel free to visit my website also contact our 24*7 assistance for all query related to learning Mandarin.

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. virtual event ‘Conference’ is a new event type that Zoom plans to make available this fall on Zoom Events. It allows hosts to organize multi-track multi-day events in a more sophisticated format with features that have long been missing from Zoom; until now, features have made the standalone platform unsuitable for more complex business events. Benefits of Work Ethic in the Work Environment, The Meaning of Open Door Policy and its Advantages and Various Types of Leadership Styles That Can Be Used as A Good Leader

    ReplyDelete
  20. This comment has been removed by the author.

    ReplyDelete
  21. Hi, I'm kim jolly. It's my initiative for online learning in the field of Mandarin in dubai UAE. if you are keen interested in learning Feel free to online visit my official website happymandarin also contact our 24*7 assistance for all query related to learn chinese language online .

    online chinese classes.

    ReplyDelete
  22. Good Post! it was so good to read and useful to improve my knowledge as an updated one, keep blogging. After seeing your article I want to say that also a well-written article with some very good information which is very useful for the readers....thanks for sharing it and do share more posts like this. Thank You So Much.!!!

    ReplyDelete
  23. Interesting article, thank you so much for sharing this amazing blog with us.

    ReplyDelete
  24. thanks for sharing a nice blog keep sharing if like to read more visit it https://mulemasters.in/

    ReplyDelete
  25. This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more
    digital marketing agency
    ppc agency for startups
    toronto seo agency

    ReplyDelete
  26. This is nice and informative post that you shared. I like to reading your all blogs. Thanks for posting this amazing blog.
    top 5 builders in noida extension

    ReplyDelete
  27. Do you want to get the best Shades Tensile Structures? If yes then Al mumtaz is the best choice for you. We are located in Dubai, United Arab Emirates. Connect with us for more information.

    ReplyDelete
  28. Thank you so much for your great information, It is too useful for me.
    website designing company in Lucknow

    ReplyDelete
  29. Are you looking for the best SEO Company in andheri that can increase your online business worldwide? If yes then connect with PromoteDial.

    ReplyDelete
  30. Promote Abhi can be the right choice for you if you are looking for the best Website Designing Company in Delhi. We have an expert and dedicated team of Digital Marketing. Connect with us for more information so that we can give you the latest services.

    ReplyDelete
  31. Data Science Course in Chennai with the appropriate tools and techniques at SLA. Learn from fundamentals, Machine Learning Concepts, AI Strategies, SAS, Statistics, Tableau, Hadoop, Apache Spark, and Programming Skills with Python, and R Languages. Promising career after the course completion on these top trending courses with the industry-endorsed certifications.

    Data Science Course in Chennai

    ReplyDelete
  32. It is very interesting! Really useful for me and thank you for this amazing blog.
    Woman Divorce Lawyers Near Me
    Fairfax Divorce Attorney

    ReplyDelete
  33. Dubai Visa Apply Online is an easy process if you connect with E Visa Dubai.

    ReplyDelete
  34. I like the helpful info you provide in your articles. 400 GPD Water Purifier System

    ReplyDelete
  35. Thanks for sharing nice blog.
    Visit my website: https://www.fastprepacademy.com/gre-coaching-in-hyderabad/

    ReplyDelete
  36. If you have more questions about the Saudi Arabia Visa, contact our customer support team at any time via online and visit Our site Visitsaudiarabiavisa

    Saudi arabia visa apply online

    ReplyDelete
  37. Embark on a healing journey with Healing Buddha, where personalized holistic treatments empower you to reclaim your well-being.
    pranic healing

    ReplyDelete

  38. This software platform offers open-source tools designed for constructing and overseeing cloud computing solutions, applicable to both public and private clouds. Thank you for the article; your blog provides a wealth of information about OpenStack. Additionally, I'm sharing insights on AWS vs. OpenStack vs. SoftLayer - helping readers determine the best option. This will undoubtedly prove valuable for everyone.

    ReplyDelete
  39. If you are planning to visit India, applying for a visa online is a convenient and efficient option. With our company, Visits Visa, you can easily apply for an India Visa online. Our streamlined process ensures a hassle-free application experience while saving you time and effort. Visit our website today to seamlessly begin your India visa application process.

    ReplyDelete
  40. Get your visa at an affordable price. You can easily avail it on India Visa. We are a government-certified visa company and ensure to provide your visa at a given period of time.

    ReplyDelete
  41. According to Simon Raven, “Life is short and the world is wide." From this quote, we can understand how wonderful it is to enjoy your life. Do you also want to take a big step in this small life? So, let's start small with Visitsvisa, which provides you with a full visa for your trip. You can travel to any country through a visit visa. Dubai Visa Apply Online, the best destination chosen by tourists by pocket money.

    ReplyDelete
  42. CSS Founder is known for its dedication to delivering high-quality results and meeting client expectations. CSS Founder is a highly reputed and committed website designing company in India. With its exceptional services, CSS Founder has built a strong reputation as a reliable company in the website designing industry in Lucknow.

    ReplyDelete
  43. This comment has been removed by the author.

    ReplyDelete
  44. Hey Dude shoes have taken the footwear market by storm with their comfortable and stylish designs. Whether you're rocking the classic Wally or the sporty Wendy, one thing is certain – your Hey Dudes deserve the best care to maintain their pristine condition. In this comprehensive guide, we'll delve into the art of How to Wash Hey Dudes shoes, providing you with step-by-step instructions and valuable tips to keep your favorite kicks looking and feeling brand new.

    ReplyDelete
  45. Ritchie's visual flair is also on full display in "The Gentlemen," with stylish cinematography, kinetic editing, and vibrant production design capturing the essence of London's eclectic landscape. From the opulent mansions of the elite to the gritty back alleys of the city's underworld, every frame is meticulously crafted to immerse viewers in the film's richly textured world.

    The Gentleman

    ReplyDelete