Sun, 25 August 2013
Roundup ‘13 - Devops and Culture Change
Fully formatted shownotes can always be found at http://javaposse.com
Extra thanks to D.J. Hagberg for the excellent and complete show notes.
- Netflix "Freedom & Responsibility" - Every team can organize as they choose
- "Firemen" who get paged when system goes down. Sets best practices
- "Site Reliability Team" - sets best practices: deployment, monitoring, ...
- "Cloud Performance Team" - sets best practices for cloud performance
- Communication with Devops: Build, Deploy, Release
- Documentation: in many cases, auditors are happy with scripts
- Opportunities for Ops to deploy systems, devs to assist & learn from Ops
- Uptime and Availabitility - metrics
- Wake up the developer at 3am when they break things. Devs need to understand Operational Risk of changes
- PCI Compliance mitigation ideas
. Separate out the SOX/PCI regulated parts of application, pared down to most minimum possible
. Put those parts under separate accounts, machines, audit trails, etc.
- Metrics: what do you use to measure? Customer Happiness, Money, Loyalty? (Netflix: "movie start metric")
- Twitter - measure sentiment (for consumer-facing/high-profile services)
- Automation: bite-size, recoverable pieces
- Eiffel design-by-contract concept: preconditions/postconditions
- Redefining the job of operations
- Executive sponsorship
- QA does the deployment through DevOps-provided tools
- Metric: time to get new feature to the customer
- Visible Ops Handbook - Gene Kim
- The Phoenix Project (book) - Gene Kim
- business Case for improvements? What was cost of the last outage?
- Chase Manhattan - 3 day outage in 2010
- Move developers into Ops teams - Google engineers get extra "hazard" pay/leather jacks - DevOps Culture Hacks
- Practices brought back from Ops to Dev
. JMX: Measure/Instrument *all* the things (esp. consumer facing)
- "Foreign Exchange" (hostage exchange?) program
- "Don't make customers modify XML Files"
- Static Config to Files. Stays the same for release.
- Dynamic Config - looked up from service - Archaius - Netflix
- Feature Switches/Feature Flags - Etsy
. Enable/disable features & settings at runtime
. Hierarchical/tagged/scoped configurations
. Use config changes for rollback
The Java Posse consists of Tor Norbye, Carl Quinn, Chet Haase and Dick Wall
To browse other episodes, use the Archives list on the right >>