Blueflood: A case study in turning a large piece of infrastructure software into an open source project Javazone 2014 • Rackspace • @gdusbabek
Blueflood:" How I learned the hardest problems are not technical Javazone 2014 • Rackspace • @gdusbabek
About Me Rackspace
About Me Cassandra
About Me Monitoring
About Me Metrics
About Me Open Source Veteran
About Blueflood Distributed
About Blueflood Ingest
About Blueflood Rollup
About Blueflood Query
About Blueflood Started 2012
About Blueflood Open source 2013
About You Contemplating releasing a project
About You Having trouble with open sores
About You Having trouble with open source
YOU ARE IN THE RIGHT PLACE
Objectives Devote resources?
Objectives Maybe not?
Objectives Measure success
Objectives Measure failure
Objectives History 101 Business 101
BORING!
We made this thing
We did some things right
We did some things wrong
We Learned Something
We Learned Some things
OK. What things?
HISTORY 101
Software is relatively new
In the beginning it was ALL open source
In 1953 the A-2 shipped with all of its source code
Patches plz
Changed by late 1960s
$$$
1. More Complex
2. Growing Market (Fewer engineers)
Natural Scarcity + More Work To Do ______________________________ Expensive
Protect intellectual property
This trend continues
Not much open source activity
1991" Linux kernel
Modern open source advocacy
Attacked in 1990s
Continued growth well into 2000s
Current State
Open source work is appealing Good for publicity Trendy
Hacker News Cycle Y 1. ▲ Release Something 2. ▲ Post it on HN 3. ▼ See interest in the project 4. ▲ Secondary interest in your business
Hacker News Cycle Not conducive to a healthy ecosystem Y
So Many Ways to Collaborate
I " ♥ " PULL REQUESTS
I " ♥ " PULL REQUESTS
Stimulating Professional Conversations 140 Characters at a time
Mailing Lists IRC
Promote Communities
Releasing a project is easy
Making it successful is very hard
What is “successful” anyway?
“We reviewed and merged 10 pull requests in our OS project this week. It was a lot of work. It was beautiful and awesome!”
“You didn't merge any features we needed to run our business this week. I’m going to eat your family!”
Success is in the eyes of the beholder
Hipster Open Source
Current State Different ways of releasing a project
Code Dump (Sea Turtle Approach)
Lay a bunch of eggs, then run off.
Code Dump (Sea Turtle Approach) What you see: • code • Probably no wiki • Little to no-code activity • "Throw it over the wall"
Code Dump (Sea Turtle Approach) What you get: • Very little community • Infrequent support • No real ownership
worst kind of open source only adds noise
Primary Sponsorship"
Let someone else sit on our eggs.
Primary Sponsorship (Cuckoo approach) One entity does most of the work • Free for everybody to use • Good stewards • Could be a good community and support • Not welcoming to outside contributors
Incubation (Elephant Approach)
Incubation Recruit committers Build community Real Work
Incubation Most difficult Long lasting Provides for lifecycle Meritocracy
Good News: None of these are wrong
Bad News: None of these are wrong
Why is project X open source?
BUSINESS 101
Type 1 It makes money • Profit center • AdWords, Github hosting
Type 2 It doesn't make money • Cost center • Your database • Your queueing software
Reduce the expense of cost centers Increase profitability of profit centers
Nobody will release source for profit centers.
Everybody wants to release the source of their cost centers.
Wrong Mentality
Wrong Mentality
Free doesn’t make it less expensive.
Ok. That was harsh. (End of business lesson)
What kind of future?
Think about what you really want.
Software should be frictionless.
Easy
CAP Theorem?
My Theorem 1. Easy to use 2. Community 3. Quality
CEQ? QEC? It probably will not catch on.
10 minute test Too many projects fail this
Proposition We can improve the experience of using open source software by improving the process of creating it.
Our Humble Goal We can improve the experience of using open source software by improving the process of creating it.
DO THESE THINGS (7)
1. Understand your motivations
Why are you releasing this code?
For the recognition? Don't do it!
You want to help others? Why?
I dunno. Don’t do it!
Because it’s good for business Ok. Do it!
You wan to commoditize something? That’s interesting.
Very interesting.
STOP!
Offload Others will do the work Commoditize Reduce the cost for everybody
Yes, do it, definitely!
2. Then focus on execution
Code dump == Bad
Primary sponsorship: Works, but still expensive for you.
Incubate: Works, but so much initial effort!
Clearly this is where things get Hard™
3. Define how you will measure success
Can be anything But make it explicit And write it down Publicly
WARNING: The following contains controversial opinions. Viewer discretion is advised.
4. Split Your Team Team Foo: Production Team Bar: Open Source
Case α: Bug in Production All hands on deck Open source work grinds to a halt
Case β: Big OS Feature Focus, Focus, Focus Product work becomes a distraction
5. Invest in documentation
More than a day Ok. At least a week. Maybe more
Landing Website Wiki 10 minute guide
And practice the 10 minute guide!
6. Set up communication channels
Mailing lists IRC Twitter
RESPOND TO EVERY BIT OF COMMUNICATION
Black Hole == Bad Nobody is using it Reflection of quality
7. Participate & Advocate
Tell your story Like minds will find you
Difficult for some Just keep practicing You can get good at it even if you don’t like doing it.
Wait, what were all those things? 1. Understand your motivations 2. Focus on execution 3. Measure Measure Measure 4. Split your team 5. Document all the things 6. Communicate with all the things 7. Participate & Advocate
How will we know when we’re there?
Our Humble Goal Change the experience of using software by improving the process of creating it.
Evidence of Success What will our success look like? How shall we measure it?
Evidence of Success When open source no longer hinders us
Evidence of Success When you and others are able to leverage open source projects to run your business with: – More speed – Less resources – Greater focus
Evidence of Success: A Casualty Fewer open source projects.
Evidence of Success: A Casualty What about the bazaar?
Evidence of Success Healthier Ecosystems
Evidence of Success Easily identified lifecycle Young: growing & changing Mature: less risk
Evidence of Success You know how to engage the community and support channels
Evidence of Success Your split team does not feel compromised or stretched They aren't riding two horses Your leadership is comfortable with this.
This all sounds really good, but…
Return on investment Staffing engineers who work on 100% on open source Perception: expensive cost centers
One Approach: Change that perception Dubious proposition
The Problem: This will probably not last long. Not a good approach.
Another Approach: Skunkworks
Split your team, but don’t tell anybody.
Sneaky, but that doesn’t prove ROI.
Measure Everything Prove that that things are better Use science!
Support tickets Monitoring alerts Days spent fixing bugs
Prepare for unexpected results Measurements may indicate that you’re wasting time or other resources.
Be honest with yourself
Measure, Measure, Measure.
History 101 + Business 101 _______________ Undeniable Insight
• Things are changing like they never have • Commoditization of software + hardware • Accelerating pace
• Become adept • Less friction • More time • Time is money • Solid return on investment
Observed Trend! The Open Source Startup Model Venture Backed Great supporters of open source ROI easily justified
Good thing right?
It’s the least bad thing.
Product of prevailing attitudes in non VC companies 1. Contributing to open source is a drain on resources 2. Companies are being short sighted
The Choice Is Ours
Small Steps
Consider carefully your choice to release a project.
Hold yourself accountable
THANKS! @gdusbabek
Some images required attribution boring https://www.flickr.com/photos/phoenixdailyphoto/1467681879 professor https://www.flickr.com/photos/judy-van-der-velden/4468986943 tank https://www.flickr.com/photos/defenceimages/14465175042 tools https://www.flickr.com/photos/75905404@N00/7126147125 businessmen https://www.flickr.com/photos/sgis/6532363 rainbow https://www.flickr.com/photos/liesforaliar/6057583336 trash https://www.flickr.com/photos/en321/14611573210 skunk https://www.flickr.com/photos/i_am_just_a_cloud/1079804050 inspire https://www.flickr.com/photos/mikeyphillips/14654963066

How To (Not) Open Source - Javazone, Oslo 2014

Editor's Notes

  • #19 By the end of this presentation, you should be able to confidently answer several questions.
  • #25 This would be a boring presentation if I only told you about my experience with releasing Blueflood as an open source project. https://www.flickr.com/photos/phoenixdailyphoto/1467681879
  • #28 We did some things wrong
  • #31 First, I want you to be able to see the big picture.
  • #34 In the beginning, just about all software was open source
  • #162 Brings me back to the point of demonstrating that open source is good for business. Still, at some point, you need to prove that either releasing code or contributing to a project yields a return on the investment.