DevOps Picc12 Management Talk
52 likes•26,089 views
The document outlines the principles and practices of DevOps management at Etsy, emphasizing the importance of culture, collaboration, and transparency within the team. It highlights the role of managers in fostering an environment that encourages creativity, respects time, and promotes open communication. Key practices include hiring smart individuals, accepting failures, celebrating successes, and using tools like Kanban for improved workflow and morale.
DevOps Picc12 Management Talk
- 1. DevOps Management: Herding Cats at Michael Rembetsy Director, Operations Engineering @mrembetsy
- 2. What is Etsy? “Etsy is the marketplace we make together. We enable people anywhere to easily build and directly exchange with independent, creative businesses”
- 4. The Numbers!
- 5. The Numbers! • 15 million members
- 6. The Numbers! • 15 million members • 875,000 active sellers
- 7. The Numbers! • 15 million members • 875,000 active sellers • Sellers in over 150 countries
- 8. The Numbers! • 15 million members • 875,000 active sellers • Sellers in over 150 countries • 2011 sellers grossed 525 million in sales
- 9. The Numbers! • 15 million members • 875,000 active sellers • Sellers in over 150 countries • 2011 sellers grossed 525 million in sales • 1.4 billion page views a month and growing
- 10. Who am I?
- 11. Who am I? • Responsible for day to day collaboration and operations (datacenter / corporate)
- 12. Who am I? • Responsible for day to day collaboration and operations (datacenter / corporate) • I am a mentor
- 13. Who am I? • Responsible for day to day collaboration and operations (datacenter / corporate) • I am a mentor • I enable
- 14. Who am I? • Responsible for day to day collaboration and operations (datacenter / corporate) • I am a mentor • I enable • Remove barriers and encourage shipping
- 16. Culture
- 18. The old Culture didn’t work at Etsy
- 20. So what is the new way?
- 22. Starts with Hiring • Manager != smartest person on the team, hire smart people
- 23. Starts with Hiring • Manager != smartest person on the team, hire smart people • You need to be more than just a technologist
- 24. Starts with Hiring • Manager != smartest person on the team, hire smart people • You need to be more than just a technologist • Experiences count for managers and team members
- 27. Required
- 29. Principles for a DevOps Managers
- 30. Principles for a DevOps Managers • Accept failures not lower standards
- 31. Principles for a DevOps Managers • Accept failures not lower standards • Доверяй, но проверяй
- 32. Principles for a DevOps Managers • Accept failures not lower standards • Доверяй, но проверяй • Celebrate success and failures (blameless post mortems)
- 33. Principles for a DevOps Managers • Accept failures not lower standards • Доверяй, но проверяй • Celebrate success and failures (blameless post mortems) • Be open minded
- 34. Principles for a DevOps Managers • Accept failures not lower standards • Доверяй, но проверяй • Celebrate success and failures (blameless post mortems) • Be open minded • Move fast (collaboration and deployment)
- 35. The Team
- 36. The Team • Flexible. Able to work in a dynamic environment anywhere
- 37. The Team • Flexible. Able to work in a dynamic environment anywhere • There is no single variable to success. There are many.
- 38. The Team • Flexible. Able to work in a dynamic environment anywhere • There is no single variable to success. There are many. • Team members and managers must know they are liaisons for all other teams
- 39. How do I manage my team during...
- 41. Stressful Times
- 42. The Same Way!
- 43. The Same Way! • Failures / outages
- 44. The Same Way! • Failures / outages • Personality conflicts
- 45. The Same Way! • Failures / outages • Personality conflicts • Personal issues
- 46. The Same Way! • Failures / outages • Personality conflicts • Personal issues • Last minute whoopsies
- 47. The Same Way! • Failures / outages • Personality conflicts • Personal issues • Last minute whoopsies • Unexpected context switching (work)
- 48. One on One
- 49. “What’s the point? We talk all the time in IRC?” “I don’t have time, just email me an update.”
- 50. NO!!!!!
- 51. NO!!!!! Provide a scheduled outlet for talk (30 minutes minimum)
- 52. NO!!!!! Provide a scheduled outlet for talk (30 minutes minimum) Respect their time and yours, show up on time and prepared to discuss the last week
- 53. http://www.randsinrepose.com/archives/2010/09/22/ the_update_the_vent_and_the_disaster.html
- 54. Humor
- 58. Freedom
- 59. Engineers are creative. So let them be creative....
- 60. Freedom but with Guidelines • Communication: send everyone updates • Ask for help if you need it • Ask for direction on work • Day to day work is done (queue, job description)
- 61. Open Door Policy
- 64. Problems Managing People • You will have them and each is different
- 65. Problems Managing People • You will have them and each is different • Stay calm
- 66. Problems Managing People • You will have them and each is different • Stay calm • Talk to your peers for advice
- 67. Problems Managing People • You will have them and each is different • Stay calm • Talk to your peers for advice • Think outside the box
- 68. Problems Managing People • You will have them and each is different • Stay calm • Talk to your peers for advice • Think outside the box • Don’t fear having hard conversations. These make you a better manager
- 69. “the job of a manager is to be willing to have a hard conversation, any time during the week with anyone anywhere.” Marc Hedlund
- 70. We also changed our workflow in operations
- 71. Why? We weren’t shipping enough Context switching = low morale “what are you working on?”
- 72. Kanban
- 73. Kanban • Provided the whole organization Transparency into what we were doing
- 74. Kanban • Provided the whole organization Transparency into what we were doing • Less context switching = better morale
- 75. Kanban • Provided the whole organization Transparency into what we were doing • Less context switching = better morale • Better throughput of work
- 76. Kanban • Provided the whole organization Transparency into what we were doing • Less context switching = better morale • Better throughput of work • Better Prioritization
- 77. How Does it flow?
- 82. We use a virtual board
- 83. Give them tools & Encourage building collaborative tools!
- 84. Devops Tools - managers • Jira- tickets All can see • Kanban - workflow that you just saw • but wait theres more.....
- 85. Metrics
- 86. Metrics • We have ~280K now and growing!
- 87. Metrics • We have ~280K now and growing! • Use open source tools (ganglia, graphite)
- 88. Metrics • We have ~280K now and growing! • Use open source tools (ganglia, graphite) • Dashboards are key
- 91. Monitoring • We use tools like Nagios, Pagerduty and Pingdom.
- 92. Monitoring • We use tools like Nagios, Pagerduty and Pingdom. • More than ops are on call
- 93. Monitoring • We use tools like Nagios, Pagerduty and Pingdom. • More than ops are on call • ops on call rotation
- 94. Monitoring • We use tools like Nagios, Pagerduty and Pingdom. • More than ops are on call • ops on call rotation • dev on call rotation
- 95. Monitoring • We use tools like Nagios, Pagerduty and Pingdom. • More than ops are on call • ops on call rotation • dev on call rotation • search on call rotation
- 96. Human escalation
- 97. Asset Management
- 99. Chef deployment is transparent to everyone Deployed Chef Change
- 100. That wasn’t enough collaboration
- 101. Eventinator
- 103. Let’s review
- 105. • Culture • One on One meeting • Humor • Freedom to be creative and explore • Open Door Policy • Encourge the use and building of collabrative tools • DevOps Princples for managers • Be transparent with your work to everyone
- 106. My Influences • Michael Lopp (http://www.randsinrepose.com) • Kate Matsudaira (http://www.katemats.com) • Chad Dickerson (http://blog.chaddickerson.com) • John Allspaw (http://kitchensoap.com/) • Marc Hedlund (http://blog.precipice.org/) • Kellan Elliot-McCrea (https://twitter.com/#!/kellan)
- 107. Thank you for listening Questions?