Dieter_be's spot on the web

Being active as both a developer and ops person in the professional life, and both an open source developer and packager in my spare time, I noticed some common ground between both worlds, and I think the open source community can learn from the Devops movement which is solving problems in the professional tech world.

For the sake of getting a point across, I'll simplify some things.

First, a crash course into Devops...

The Community contributions subforum of the Arch Linux forums is awesome.
It is the birthplace of many applications, most of them not Arch Linux specific.
File managers, media players, browsers, window managers, text editors, todo managers, and so on. Many shell scripts, urxvt extensions and dwm patches aswell.
Most of the apps are designed after suckless/KISS principles, but there are also some GUI programs.

If you like to discover new apps and tools, check it out.

I use pm-utils for hibernation support.
It has a hooks system which can execute stuff upon hibernate/suspend/thaw/resume/..., but they run as root.
If you want to run stuff as a regular user you could do something like

su $user -c <command>

The solution is very simple. Use this:

sudo pm-hibernate && do-my-stuff.sh

I'll be at fosdem - 10th edition - again this year.

I'll be presenting a lightning talk about uzbl.
Also, Arch Linux guys Roman, JGC, Thomas and me will hang out at the distro miniconf. We might join the infrastructure round-table panel, but there is no concrete information yet.

More stuff I'm looking forward to:

Xfce/Gnu/Linux works amazingly well on my moms workstation, with one exception: opening files automatically with the correct program.

The two biggest culprits are:

• Gtk's "open file with" dialog: if any Gtk program doesn't know how to open a file it brings up this dialog that is horrible to use. You can search through your entire VFS for the right executable. No thumbnails, no usage of .desktop files, $PATH, autocompletion and not even limiting the scope to directories such as /usr/bin
• Mozilla software such as Firefox and Thunderbird: they only seem to differentiate files by their mimetype, not by extension. There are add-ons to make it easier to edit these preferences, but eventually you're in a dead end because you get files with correct extensions but unuseful mimetimes (application/octet-stream)

Luckily the fd.o guys have come up with .desktop files.

Inspired by Snippy and snippits, I wrote a simple tool called snip.
It helps you to automatically fill in text for you (which can be dynamically created) and/or to perform custom keypresses and operations.

So, the Arch Linux 2009.08 release is now behind us, nicely on schedule.
I hope people will like AIF because it was a lot of work and we didn't receive much feedback. I personally like it to apply my fancy backup restoration approach.
But I'm sure if more people would look at the code we would find quite some design and implementation things that could be improved. (With uzbl I was amazed how much difference it can make if many people all have ideas and opinions about every little detail)

Later this week I'm off to the Counting Cows festival in France, and the week after that (august 22-23) I'm going to FrOSCon in Germany where I will meet some of my Arch Linux colleagues in real life, which I'm really looking forward to.

If anyone wants a ride to froscon let me know. But note I'll try to maximize my time there (leave saturday early and come back late on sunday. I even took a day off on monday so I might stay a day longer if I find more interested people to hang out there)

While setting up Zenoss and reading Mysql documentation about status variables I learned:

• All select_* variables ("Select statistics" graph in Zenoss) are actually about joins, not (all) selects. This also explains why there is no clear relation to com_select (which shows the amount of selects). ("Command statistics:selects" graph in Zenoss)
• Com_select does not denote all incoming select commands. If you have a hit on your query cache, com_select is not incremented. So I thought we were doing less qps while in fact we were just getting more cache hits. Qcache_hits gets incremented on cache hits (but is not monitored by Zenoss)

I've been playing with Zenoss (2.4) for the first time. Here are my thoughts:

I need a browser that is fast, not bloated, stores my data (bookmarks, history, account settings, preferences, ...) in simple text files that I can keep under version control, something that does not reinvent the wheel, something that I can control.

Well, I could not find it.
So I started the uzbl browser project.

I wanted to know how responsive dmenu and awk, sort, uniq are on a 50MB file (625000 entries of 80 1-byte chars each).

generate file:

#!/bin/bash
echo "Creating dummy file of 50MB in size (625000 entries of 80chars)"
echo "Note: this takes about an hour and a half"
entries_per_iteration=1000
for i in `seq 1 625`
do
        echo "Iteration $i of 625 ( $entries_per_iteration each )"
        for j in `seq 1 $entries_per_iteration`
        do

I'm particulary interested in:

• The out-of-the-box concepts of Exherbo. I hope to see more things like this.
• Various talks at OpenSuse about distribution development, their build service, etc.
• tools such as func and puppet.
• syslinux, upstart, ext4 etc.
• Some mysql stuff and the filesystem i/o from a db perspective talk.

At my new job, I'm writing a quite full-featured web application.
I've choosen to use CakePHP.
Why? Well, it may be 2 years since I last used it, but I've followed the project and it's planet, and it seems to have matured and gained even more monumentum.
I want to use something that is widely used so there is plenty of stuff available for it, it's RAD, it's flexible and powerful.
I noticed things such as CLI support and documentation have improved tremendously too.

However, I find that still, the recommended (or at least "most commonly used") practices are not as efficient as they could be, and that emphasis is placed on the wrong aspects.
See, even though the bake tool has come a long way since I last used it, it's still used to "generate some standard models/controllers/views" and the developer can take it from there [further editing the resulting files himself].
Finetuning generated code by editing the templates (in fact, only views have templates; the php code of models and controllers is hardcoded in the scripts that generate them), is still an obscure practice...
Also, there are very few commandline switches (Right now you can choose your app dir, whether you want to bake a model,controller or view, and it's name.)
All other things (validation rules, associatons, index/view/edit/add actions/views, which components, overwrite yes/no etc) are all handled interactively.
There are also some smaller enoyances such as when you specify one option like the name of the model, it assumes you don't want interactivity and produces a model containing nothing more then the class definition and the membervariable $name, which is usually worthless.
One thing that is pretty neat though, If you update $this->recursive in a model, the baked views will contain stuff for the associated things. But so much more could be done...

What better way to launch the new year then starting to work as a System Engineer/Developer for a consulting firm where everyone breathes Linux and Open Source?
Next week I'll start at Kangaroot. Woohoo.

My holidays present for Arch devs and users: AIF alpha-0.6 !

* Changes since alpha 0.5:

Recently I started thinking about writing my own automatic installer that would set up my system exactly the way I want.
(See http://dieter.plaetinck.be/rethinking_the_backup_paradigm_a_higher-level...)

I looked at the official Arch install scripts to see if I could reuse parts of their code, but unfortunately the code was just one big chunk of bash code with the main program and "flow control" (you must first do this step, then that), UI-code (dialogs etc) and backend logic (create filesystems, ...) all mangled up and mixed very closely together.
Functionality-wise the installer works fine, but I guess the code behind it is the result of years of adding features and quick fixes without refactoring, making it impossible to reuse any of the code.

So I started to write AIF: the Arch Linux Installation Framework

I recently had to rename a repo on my Github account. Github has made this very easy but it's just one side of the issue. Obviously you must also update any references to this remote in other clones, otherwise pushes, fetches etc won't work anymore.

You can do this in two ways:

• open .git/config and modify the url for the remote manually
• git remote rm origin && git remote add origin git@github.com:$user/$project.git

That's it! All will work fine again.

I couldn't find a standalone application/script that implements freedesktop compliant (XDG based) autostarting of applications, so I decided to write my own.
The project is at http://github.com/Dieterbe/dautostart .

Right now, all the basics seem to work (except "Autostart Of Applications After Mount" of the spec).
It's probably not bugfree. I hacked it together in a few hours (but it works for me :-). Bugreports welcome!

I'm managing my ~ in svn but using gnome & gconf makes this rather hard.
They mangle cache data together with user data and user preferences and spread that mix over several directories in your home (.gconf, .gnome2 etc).
The .gconf directory is the worst. This is where many applications store all their stuff. User preferences but also various %gconf.xml files, which seem to be updated automatically everytime 'something' happens: They keep track of timestamps for various events such as when you press numlock or become available on pidgin.
I'm fine with the fact they do that. I'm sure it enables them to provide some additional functionality. But they need to do it in clearly separated places (such as xdg's $XDG_CACHE_HOME directory)

DDM v0.4 has been released.
Since the last release many, many things have been changed/fixed/added.

I've put my (somewhat interesting) projects on GitHub.
Git is a damn cool VCS for distributed development, and I think Github integrates with it really nicely, adding some useful aspects for following and collaborating on projects.
The projects I have migrated to my GitHub profile are:

I've been reading GTD lately and it's absolutely a great and inspiring book.
Having made my home office space into a real Zen I want to start implementing GTD in my digital life but it seems very hard to find a good GTD tool that fully implements GTD. (even though there are a lot of tools out there)

The most interesting ones (each for different reasons) I've looked at so far are Thinkingrock, tracks and yagtd (the latter requiring most work before it does everything I need, but it's also the most easy to dive into the code base). I'm keeping my eyes open because there are certainly more things to discover.

Even though there are probably no applications out there that can do everything I want, I just wanted to share my feature-wishlist. These are the requirements I find that a really good tool should comply with:

At work, we've setup the Netlog Developer Pages

It is the place where you can/will find all information around our OpenSocial implementation, our own API, skin development, sample code and so on.
We've also launched a group where you can communicate with fellow developers and Netlog employees.
The page also features a blog where you can follow what is going on in the Netlog Tech team.

UPDATE: this information is outdated. See http://github.com/Dieterbe/ddm/tree/master for latest information.

Introduction

If you have multiple sets of data (e.g.: music, images, documents, movies, ...) and you use these on more then one system ( e.g. a laptop and a file server) then you probably also have some 'rules' on how you use these on your systems. For example after capturing new images you maybe put them on your laptop first but you like to sync them to your file server frequently. On the other hand you also want all your high-res images (stored on the server) available for editing on the laptop, and to make it more complicated you might have the same images in a smaller format on your server (for gallery programs etc.) and want these (or a select few albums of them) available on the road.

The more different types of data you have and the more you have specific work flows the harder it becomes to keep your data as up to date as possible and consistent on your boxes. You could manually rsync/(s)cp your data but you end up in having a mess (at least that's how it turned out on my boxes). Putting everything under version control is great for text files and such, but it's not an option for bigger (binary) files.

I wanted to keep all my stuff neatly organised in my home directories and I want to create good work flows with as minimum hassle as possible, so I decided to write DDM: the Distributed Data Manager.

If you serve lots of (small) files with Lighttpd you might notice you're not getting the throughput you would aspect. Other factors (such as latencies because of the random read patterns ) aside, a real show stopper is the stat() system call, which is a blocking system call ( no parallelism ). Some clever guys thought of a way to solve this : a fastcgi program that does a stat(), so when it returns Lighty doesn't have to wait because the stat information will be in the Linux cache. And in the meanwhile your Lighty thread can do other stuff.

I wish I could put this on my webpage :

Phew! where to start? Probably at this blogpost. It's about making it very easy to work with external encrypted volumes. I'm not going to talk about the article itself but about a great tool i discovered thanks to it: Zenity.

FOSS is written by users, for users, and what I've been doing/experiencing this afternoon is a perfect example of that.

I've been using OpenSSH already for a while and although I've seen mentions of "public key authentication" and "RSA encryption" several times in it's config files, I never decided to figure out what it did exactly, and stuck to password authentication. But now the guys at work explained how it works and after reading more about it, I'm totally hooked on it!

I've always liked Amarok: it does everything I always wanted, and more. It looks perfect in every way ...
But .. it uses the QT library, and although there are tricks to make QT applications more fit in with your gtk desktop/theme it will never fit in perfectly, not only graphically but also because you still need to load the qt libraries when you want to listen to some music and it is built to interact with the KDE desktop environment.

So, I've been looking for an alternative, a GTK application strong enough to actually be able to replace Amarok, the king of all software music players.