arch bash cakephp dauth devops drupal fosdem foss git golang information age linux lua mail monitoring music n900 netlog openstack photos php productivity python real life thesis travel uzbl vimeo web2.0
Being active as both a developer and ops person in the professional life, and both an open source developer and packager in my spare time, I noticed some common ground between both worlds, and I think the open source community can learn from the Devops movement which is solving problems in the professional tech world.
For the sake of getting a point across, I'll simplify some things.
A commonly used organisatorial idiom used in tech companies is that of developers and operations.
Experience shows this model often falls short. 'Dev' and 'Ops' being too artificially separated from each other, resulting in improper communication, clashing procedures and tools,
resulting in devs disliking ops ("we need to push this out to users, ops are holding us back"), and the other way around ("again new code that will cause trouble, and we will have to figure it out")
It doesn't take a genius to see this is pretty ineffective. There's a better way: integrating and reconciling dev and ops, so that all involved know the hard parts of each others' jobs, and in fact letting each other do the others' job. (developers being responsible for their own checkouts, ops working on the code, etc). Most of all it's about culture over processes. About being smart and nice human beings.
The exact methods are still being experimented with and preached about, and has recently gotten the name "Devops".
There is a really good Devops explanation online, with more details. Read it.
Often enough we're talking about teams working for the same company, usually under the same roof, so it isn't too terribly hard to implement these ideas.
Open source developers ("upstream"):
Like above, the problems stem from both parties not working together enough, and doing things on their own.
..making it hard for downstreams.
Even for each other: unannounced/frequent API changes come to mind.
Nothing pleases an upstream more then complaints from end users running into problems that only happen witch patches applied by the distributor (patches that are deemed necessary to make the app work properly in the distro. The irony..)
Some distributions focus on shipping "only stable software", causing them to be obsolete by definition. (Time to production often extends in the order of years), and are forced to apply so many patches that they are essentially forking their upstreams. Add poor feedback loops to the list and the situation is about as ineffecient as it can get.
Other distributions limit their role to giving you the real open source software experience in it's current state, and that state is not always pretty.
So, in contrast to popular belief, open source is not a magical wonderland where everyone works nicely together.
Tech companies are usually on their way if they understand and can introduce agile and devops, but I think in the open source ecosystem it's much harder to bring unity.
Luckily, some smart people are already working on bridging the gap between up- and downstream, and between each other.
I also think about Fosdem's cross-distro miniconf and the freedesktop.org project, which encourage closer cooperation between different downstreams and desktop projects, respectively.
So, how can we solve this? How can we maximize the end-user experience with more efficient communication and tools?
Some ideas I have:
I don't think we should try to go much beyond some common infrastructure/tools and some best practices. People will always have different opinions on how things should be done. And that's a good thing, it's the very definiton of the open source community: scratch your own itch.
What do you think?
This article is well thought out, well written, and right on point. More cooperation between upstream and downstream would make things easier for devs and end users. If one could just get everyone to agree on this and put it into motion it would be wonderful. A community approach similar to Arch Linux would benefit everyone.
Posted by Myra Nelson on Mon Sep 6 00:44:33 2010
I agree with the sentiment but I think we're unlikely see much change. The open source community by its very nature is heterogeneous. It's deeply rooted in the underlying philosophy. Eric Raymond states: "Rule of Diversity: Distrust all claims for 'one true way'" and you'll often hear "there's always more than one way of doing something". This is no bad thing however. It gives people the freedom to be creative and only then do we see great pieces of work.
Posted by Tom Vincent on Thu Sep 9 02:47:26 2010
Amen, Tom. And if you come up with something genuinly good, people will probably follow.
Since writing this article, I have been reading more about distributed bugtrackers. Seems like there are some nice ones out there.
I especially like the "track bugs in the same branch as the code" approach from bugs everywhere
Posted by Dieter_be on Tue Sep 14 16:46:00 2010