Tobi Waves

A round-trip flight from Zurich to Stockholm costs 2280 CHF unless you stay over the weekend or you stop over in Prague then prices drop to 680 CHF. I opted for the weekend solution and arrived day early in Västerås. From Monday to Friday the annual Scandinavian System Management Conference aka NordU/USENIX 2003 will take place here. I am scheduled to give a talk on large scale System management on Thursday.

But I am skipping ahead, the Week has not even started yet. In Sweden, shops are closed on Sunday and on this particular one it was also rather cold outside. So apart from a short walk to lakeside I spent most of the day in the hotel room, writing my own web journal software.

I had hunted the net for a bit, but all that is out there seems to insist on doing all the editing over the web. What a drag. Why should I edit with a crappy text widget in a browser when my beloved editor sits on the very same machine. So I dug in, and finally after many hours a first version of the code is working and I am writing my first entry, I am planning to use this for writing reports on the talks and tutorials I am going to attend this week.

Still early today, and I have learned my first lesson: Always negotiate first and then make a deal, don't try to change it later. I was asked to give a talk at the NordU 2003 conference. My talk is scheduled on Thursday. I assumed now that I am here for the week, that I would get free admission to the tutorials, if there was space. Similar to what I am used to from the SANE conference. Unfortunately it seems that the rules here in Sweden are different. No free admission for speakers ... As I said, I guess I should have asked first, before picking the cheaper flight over the weekend.

Then again, the Internet access is quite decent here, the chairs in the terminal room are comfortable and I got a nice big Apple Mac LCD screen to work with. At the actual conference I can go to all the talks, so I guess I am not missing all that much. If things around here stay as quiet as they are presently I might even get some work done.

Here in the terminal room they have these beautiful new iMacs with the 1440x900 flat panel screen. After I had played around for a bit with all the nice gadgets and got impressed by the sleek performance of their new Safari Web Browser I decided to get to work and installed the new rootless X Server Apple is now offering for download. All went well and I was soon looking at my first xterm. Work could begin, so I thought, but then it hit me. Where the hell are all the keys on this Swedish Mac Keyboard? No {, no ], no * not even a $*. Basically everything I need for programming was missing. Eventually I discovered that while the keys were not labeled all the necessary keys were reachable using combinations of Alt and Shift. I went on and even found how to start the Xquartz server with the swedish keymap applied, only to discover the in this configuration the Alt key was acting as Meta and thus lost all its power to create any of my most wanted characters.

If I ever buy a mac, I would have to get a 3 button mouse and a proper keyboard with it. I wonder how programmers at Apple work? Maybe there is a special high clearance programmers keyboard available with all the secret keys labeled properly and working. This model which would wreck havoc in the Mac user population, if leaked to the unwashed, easily confused, masses. It would leave Apples hot-lines go into meltdown as desperate users try to get help in picking the right keys. What is the world coming to?

Yesterday I have been telling various people that I found it rather annoying to not be allowed to go to tutorials. And guess what, just now, Kristen Nielsen of the program comity was here and told me that it was all a misunderstanding, and that I was welcome to join any of the tutorials I want. So I guess I will have to start writing conference reports sooner than I expected.

Tools for Creating Happy Users by Tom Limoncelli and Christine Hogan

I only joined the class in the afternoon, so I can not report on the material covered in the morning. Below I have taken some notes on the topics I found most interesting.

Handling Support Calls

The first topic was "handling support calls". Tom made a case, that handling support calls can be taught and is not just something one knows or doesn't. On the first level, each call can be broken down in to: Greeting, Whats wrong? Fix It, Verify It.

After the greeting, the second step is to get a complete problem statement. Where, When, Who, What? No special tech knowledge is required here, just social skills, active listening and probably some checklists. A complete problem statement will make the actual problem solving much simpler.

Getting a problem statement is already quite sensitive. Eg. one should never ask things which encourage the user to lie: "Is it plugged in?" is bad. Better: "Lets check both ends of the power cord". When a problem statement has been created the problem has to be verified/reproduced/scripted, because otherwise there is no way of verifying the effectively of an eventual fix.

The third step is where the tech/fixing skills are really important. In trying to find a solution, it can be helpful to involve the user into selecting which approach is taken, to fix the problem, and thereby taking their situation (eg time pressure) into account. Keep in mind that you want to know what caused the problem to go away, so only do One thing at a time.

The forth and probably most important step is, that both the person who fixed the problem and the user who reported it verify and agree that the problem has been fixed.

Unless you monitor something, you can not call it a service!

Customers rely on the services we provide, so if there is a problem with one of the services, we should know before the customers do. "Yes, we are already working on it, we expect to have this fixed by 10am, we will send out mail when it works again." is much better than. "Oh, for how long has the company website not been working anymore?"

Monitoring allows to find patterns in breakage, it assists in capacity planning and if combined with a knowledge base, solutions to known problems can be shipped out with the alarm.

My points: Optimize the monitoring for the case when all is well, because this is the state it will run most of the time. Don't have the monitoring system take 'counter action' because this leads into to the 'nanny trap' where successive layers of software are fixing the problems of the underlying system instead of fixing the root cause of the problem. Check out (isg.ee.ethz.ch ...) and also (people.ee.ethz.ch ...) for two tools in this area.

Want to learn more?

Incidentally Tom and Christine have written a book: The Practice of System and Network Administration (www.amazon.com ...) which expands on the soft topics of system administration.