Tobi Waves

I got up at 4:15 am today. In order to get to Eindhoven on time, I had to take the 7 am flight from Zurich and thus the 5:20 train from Olten. Guess I am lucky to have no problem with getting up early.

In Schiphol I met Rudi and Alexios. Together we rode to Eindhoven and arrived at the TU/e just in time for the meeting. The other members of the program committee were there as well: Walter, Edwin, Brenda, Peter, Bastiaan, Xander, Jos and Fred. And there was also Marielle and Sabina from Iconic who will be doing all the actual organization work for the conference.

This is my first time on a PC. What I found the most difficult part, is to figure out which talks would be relevant 1.5 years from now, as the conference will be held at the end of September 2004.

At the meeting we did not really decide anything fixed, apart from the meaning of SANE: System Administation and Network Engineering Conference.

We tossed around a ton of names of potential invited speakers, had a short discussion about the deadlines, debated whether we should demand 400 word abstracts or extended abstracts like LISA. We also collected places where we wanted to distribute the Call for Papers.

I guess most things will get sorted on our PC mailing list, as we move on.

A week ago, a journalist of the swiss NZZ Newspaper had been directed to me by the universities press office. The guy was looking at writing a story on Open Source software and the press office people knew I had some projects in this area. It was the week of the Blaster worm. So after talking for about 40 minutes to the journalist on Wednesday he calls on Thursday to let me know that the story on Open Source had been shelfed, and he had to write a piece on Blaster. Oh, and by the way, would I know anything about the worm. I had been battling it for the last two days, so I said yes. With the effect that I was quoted in the article on Blaster.

The following week a guy from Swiss national television (SF DRS) called, they were doing a piece on Blaster too, with a 'confront MS angle'. They had read my name in the newspaper, I agreed to the interview. I organized coaching from our press office, as the TV people were from that hard hitting investigative show, and I didn't want to be caught in the middle. I was really careful not to make any too harsh comments. They have not yet broadcast it, so I don't know if I am going to look good or bad.

In the meantime Sobig has come back to haunt us, now in its F incarnation. The press office sent the local TV people straight to me when they inquired about a person to talk to regarding Sobig.

As you can see in the screenshot, I am the IT expert now. I picked that profession after the TV people told me that "Systemmanager" was way too complex and the press office people insisted, that "Security Specialist" was going to cause trouble with the folks from CS or central IT Security. Just in case you ever wondered how you become an expert.

I guess I just had my 15 minutes of fame :-).

I have known it for a about a year. SciFi has not renewed its commitment to Farscape which essentially meant the end of the show. I am sure none of the folks at SciFi who made these decisions had watched the show. How else could it be that they killed the single most captivating, intelligent, humorous, sexy, thrilling, emotional and realistic sci-fi show ever conceived. Almost every episode I watched left me tingling all over, ideally wondering how they were able to come up with these great episodes over and over again.

It's almost a year since the cancellation, the Farscape fan-base still seems to be going strong at (www.savefarscape.co ...) plotting away on a strategy to get the show back on the air. Never loose hope.

Today I also went to the website of SciFi wondering what they were doing regarding Farscape. And indeed, they have this page "Farscape Memories" with articles from different actors remembering their time shooting the series. What hypocrisy first they pull the plug on the show and then they act all bleary eyed.

If I was rich, I mean real rich, I would just order a few more years of Farscape from the Creature Shop ... I guess I should not have gone into OpenSource, but rather have turned commercial at an early age, sucking in big bucks during the Internet Bubble then I could do more than keeping my fingers crossed for another network picking up the show again.

The other night I went to see "Wilbur wants to kill himself" (xrl.us ...) a movie about a guy, his brother, a women kid living in some Scottish town. The guy has a book shop, falls in love with the women and the brother tried to kill himself. The movie is by Lone Scherfig a Danish writer/director who was quite successful with the movie Italian for Beginners. I loved the story and the acting and above all the Scottish accent.

The strangest thing happened to me a few minutes into the film. When Alice (the women) entered the scene for the first time. She looks exactly like a friend of mine. I am quite sure it was not here, because she does not speak with a Scottish accent, and she is not an actress. Never the less I kept comparing her and Alice all through out the film. How would she act in a similar situation? And just to make it more complex, Alice is just a role, played by Shirley Henderson (xrl.us ...) but written by Sherfig, so how can it have any bearing on my friends behavior, or do looks influence how people behave, did Sherfig write the part of Alice for Shirley? Are Alice and my friend twins separated a birth?

Well it was a wired experience, but definitely a great movie. Go watch it!

I admit, I am a soap opera junkie. I just love watching them. My current favorite is The West Wing written by Aaron Sorkin (en2.wikipedia.org ...). Tonight I made it through the final episode of the first season. This time the outlook is great, as the show is till running in the US, now in its fifth season.

The West Wing is about people working in the White House for the President. It's also about the President himself. President Bartlet, played by Martin Sheen is a liberal Democrats dream. What a contrast to the current reality. An American friend told me recently that she finds it rather disturbing to watch the show, seeing what the reality could also be. Michael Moor even nominates Bartlet for President, along with Oprah in his latest book.

I am watching the West Wing on DVD, there was also the 15 minutes Making Of feature on one of the disk. There the actors talk about their parts and some bits of the sets are shown. Hearing the actors talk was odd. They were so different from the roles they play in the show. Not only did they talk differently, but also their body language was altered. I have never noticed something like this before in such a show. The characters in the West Wing make a very authentic impression on me, much more than the actors themselves actually. At least as far as the story line and the dialogs are concerned this has to be credited to Aaron Sorkin. Maybe Michael Moor should be nominating Aron Sorkin along with Oprah and Bartlet, someone has to write their lines after all.

A Talk by Urs Hölzel, Vice President for Technology,

About Google

Mission: TO organize the worlds information and make it universally accessible and useful,

An international company: 250% traffic from outside US

Engine has 4 Billion pages in index

Profitable since q1/2001

23 Office Location Worldwide.

15k boxes, several TB disk storage

There are over 1000 queries a seconds on dec 25th, 2am.

Engineering Offices in the US, Zurich and Bangalore

About the Web

Static web 167 TB in 11 Giga Pages, but dynamic websize 92 PB. (Estimates)

1 in 4 hosts on the net run a webserver.

Problem: All data, users, hosts grow exponentially. This means the problem of finding useful information grows exponentially too which makes for interesting problems.

Google Infrastructure

A high reliable system based on low cost comodity hardware. Redundancy has to be built into the software and hardware. Monitoring, repair and maintain these boxes is a prime problem.

The Google Filesystem GFS

Stripe files across many boxes and replicate them on multiple servers.

Components: Master - keeps directory and plans file layout, ChunkServer - hold the data. Clients - use the data. (Chunksize is 64 MB. Data is cached on client once retrieved. SOSP'03 (www.cs.rochester.edu ...) )

10+ Clusters of 1000+ boxes.

350 TB Filesystem

How to be a Search Engine

Crawling: Recursive Process. Problem: dynamic pages, slow servers, management of the link list, session ids in the URL, how to prioritize the URLs, being nice to the web servers, detection of duplicates, avoiding traps, actively fill forms to pull "hidden" contents, figure out when the page needs to be re-crawled.

Indexing: Words by document and position in the document. One Terra Words in the index.

Ranking: Hard problem. All traditional assumptions on searching like long, coherent, high quality documents are not valid for web documents. Googles idea is to define a PageRank for figuring the importance of the page. The PageRank of a page is the sum of PageRanks of other pages pointing to this page. A page contributes its PageRank divided by number of out-links to each of its target pages. In reality it is more complex. Google has about 100 factors in its real PageRank function like font size, color, proximity to other words.

Serving: Partition the data to different servers and have each solve a sub problem of each query. Query goes to Google Webserver, it queries Index Farm, accesses the Doc Farm for the real data. Additional services from Add Server and Spelling Server. IEEE Micro, 2003 has more on the structure (www.computer.org ...) .

Advertising: Find the best add, relevant to the query. This is a very important problem as this is the main source of revenue. Only show an add which has a chance to be clicked on, if the click-through is low, the add will be dropped. Advertisers only pay for adds actually clicked.

Google Playground

There is lots of data and computing infrastructure at Google. Google pays people who spend their time on figuring new ways to analyze and present this data: (labs.google.co ...)

In the good old times, when men were still men and computer virus writers were still technically brilliant hackers. Viruses used the uncountable holes in Microsofts ubiquitous Outlook eMail software to spread.

But even then, generally the rule was simple. If you don't want to be infected, don't run any code you don't know where it's coming from. If you have Outlook, make sure it's patched and properly configured. In Unix circles, people made fun of the whole situation by sending out mails which claimed to be a solidarity Virus, calling upon the Unix User to copy this mail to all addresses in his address book, to emulate a virus, as a gesture of solidarity with the Windows crowd.

Now, a few years later, Outlook has matured to the point that there have been no major holes for months. Never the less, eMail viruses still crop up and spread. Virus writer started to attack the users mind directly, by writing messages into the body of the virus email with the purpose of confusing the user into clicking the virus attachment, forgetting all the good advice they got. Fortunately the anti-virus software gets updated so quickly, that viruses are normally contained quickly.

Today though, mark the date, the whole matter entered an all new stage. I got the first virus which was contained in a password protected zip file. The password was contained in the accompanying email, so it is easy for a human to unpack, but anti virus software has no chance as it can not decrypt the zip file containing the virus. As a concept this sounds fine, but what totally kills me is that it seems to work. Since this morning, I get an increasing number of encrypted zip file viruses. There must actually be people who get this virus, unzip it using the supplied password and then run the thing in order to get infected.

I wonder how many people would hang themselves if they got a rope in the mail. Warden make sure all the cells are locked.

by Ralf Hildebrandt

How to use Postfix as a crude but cheap filter against spam in front of the more complex filters like spamassassin.

Sources of Spam

An important source of spam these days are miss-configured web proxies which proxy to smtp ports as well and let outsiders connect.

Protection

Use RBL lists for open-proxies, open-relays,

Reject mail from faked sender address (see below).

Insist on RFC conformance (this can make you loose lots of real mail to as there are many missconfigured normal mailservers.

Content Filters: Altermime, SpamAssassin

On Postfix

Use the snapshot version of Postfix as it is realy stable and has all the latest features.

Use a cashing nameserver to speed-up dns lookups.

By default postfix is configured to only accept mail from your local network for external destinations. This has no influence on spam though.

Be careful choosing RBLs because there are many badly maintained blacklists out there. Blacklist must have clear criteria and a delisting procedure.

postmaster@yourdomain and abuse@yourdomain must accept all mail this must be explicitly listed in smtpd_recipient_restrictions.

A good RBL list cbl.abusenet.org recomended by Ralf.

When you are using RBLs make sure that you can quickly add exceptions to your system.

Rejecting mail to unknown users at the smtpd stage is very efficient as it first saves traffic and it also saves you from sending bounces.

Postfix can use various directory services to figure out which users exist. Postfix 2.1 will even cache answers

Use right hand sender black lists may also help. But be care full. Look at =dsn.rfc-ignorant.org, postmaster.rfc-ignorant.org, abuse.rfc-ignorant.org, whois.rfc-ignorant.org=.

RBL/RHSBL are expensive because of all the DNS lookups. Perform them as late in the restrictions list after the cheep mails.

Sender address verification

Check if the sender is either a known valid or can be verified to be valid. Postfix has special support for this as it can send test messages to the sending host. The sender will not notice this as postfix only starts sending mail but aborts before giving any message body.

Make sure you are really careful as this can cause you to loose mail from people who are not able to correctly spell their sender. One option is to apply these sender check restrictions only to suspected domains.

by Patrick Koetter

How to support mobile users to use your server as a mail relay. IP based restrictions do not work as the mobile users will have random IP addresses.

SMTP AUTH

Using Cyrus SASL2 and OpenSSL together with Postfix. You can configure postfix such that it allows relaying access for users who are properly authenticated. Most mail clients support snmtp authentication.

The problematic thing is to properly configure SASL. Get the CVS version as it is less buggy then the official 2.1.17, it even has some minimal documentation.

SASL configuration is governed by a config file called the same as the program using the sasl library. In our case this is smtpd.conf.

If you use SASL with plaintext passwords, make sure it only allows AUTH when TLS is in operation.

Check out Patricks howto on this (postfix.state-of-mind.de ...)

Certificate based Relaying

For people running mobile Unix it is possible to setup a local mailserver which just forwards all mail to the official mailserver of your site. By configuring the postfix smtp daemon to use TLS on the client, and you store the clients cert on the server. Now configure the server to ask clients for a certificate when they connect. If a client submits a vlid (known) certificate it will be allowed to relay even if it has an ip number outside the local network.

The cool thing about this is, that now any program on the mobile unix client can send mail via the local mail server to the company mailserver without further problem.

by Theodore Ts'o

History

VA Linux wanted to be the IBM of the Linux world, but forgot to ask whether maybe IBM wanted to become the IBM of Linux. This was a huge mistake.

What makes Linux significant

One of the first projects done primarily over the Internet without a central group contributing all the code.

Difference to other projects. Many of the people involved are on a mission to distribute the knowledge about the system. Even people who can not bootstrap their own system are welcome.

Open Source and Free Software

The Open Source movement brought the whole "making source available is cool" concept into the industry without the political issues of the GPL.

Intellectual property laws are going to become more and more of a problem, so we have to become involved.

The huge number of Open Source licenses leads to incompatibilities when working with libraries. The important lesson here is to pick the right license from the start, as relicensing is difficult as soon as you have received a lot of code contributions.

Rapid Development and Open Source

Infrastructure work or documentation writing are necessary tasks but they are not very sexy. We need to cater for them. Incremental changes are not the only thing.

Even though there has not been a lot of innovation coming from Redmond. This does not mean they are sitting on their hands. They are working on vertical integration technologies like their new user interface description language which will be used on all their products.

User Interface Testing

Have programmers watch a users working with their software. Do not let them talk to the users and help them. They will see all the problems which are to be fixed.

The Limits

Software with a short shelf life like tax software or games needs a huge investment in development and have to recover the cost quickly, So this will probably never work with opensource concepts.

Binary drivers are a fact of life due to paptent and even legal issues. There is not clear solution for this yet in connection with Open Source.

Closing

It's been a great 10 years, but there is still a lot todo.

by Robert Griessmer

For the technical Information, read my earlier entry about Urs Hoelzels talk.

About cheep hardware: If one writes his software under the assumption that hardware is going to fail anyway. This makes lots of problems much simpler. If the number of machines grows really high, you can employ people who spend their days swapping broken ones.

These days Google designs their own bare-bone boards with only the necessary components for the stuff, the machines are required to-do.

Google Philosophy

Make the worlds information search able.

Do things that matter, and do them algorithmically so that they are scalable.

Every engineer has a free day per week to work on his own fun projects.

Google Tricks

Spell checking happens statistically. Google does not know what is right, but what is most popular.

Google provides an API for programmers.

Experimental projects (labs.google.com ...)

Google Zurich Office

Right in the city center. Entire range of software development. Meant to attract European engineering talent. Tight integration with the Mountain View offices.

Jobs: (www.google.ch ...)

User Questions

Q: What is your crawling bandwidth

A: Well about this (shows with hands) Size.

...

A: oh and ... no comment ... I can't really talk about this.

by David Rosenthal

Open Source in Government

Could government require that only Open Source software be submitted for a new text processing system to be procured? Yes, but only if the fact of it being Open Source was directly tied to the core functionality of the software and requirements in this area. This point may be difficult to make, depending on the nature of the software.

If GPL software is modified by one branche of the governemnt and passed on to a different branch. This is a sort of publication. Accordning to the GPL license the second branche gets a license with the code which allows it to redristribute the modified version without restrictions, takeing it effectively out of the control of the first party.

Copyright

Every program of an "individual nature" is protected by copyright law. "Individual" means, that the code is potentially recognizable as being written by that particular author. This right does not have to be registered or patented or anything. Every author has it.

In Swiss law this means that I as an Author have the exclusive right to decide whether and how my product is used. I can give all or some of my rights up to a third party .. aka grant a license. GPL or any of the other Open Source licenses operate on this principle.

GPL can be enforced. 19.5.2004 the Munich District upheld the GPL in a case of netfilter team vs firewall vendor.

Contract Law

If I modify an Open Source product for a third party (contract) I can become liable for any problems with the whole product if I do not exclude this in the contract. This is not special for Open Source products but it may easily go under the radar. Becareful when drafting your contracts.

Warranty and liability can be limited, but only to the extent allowed in the law. Swiss Law: Any limitation of liability for gross negligence or willful intent is void.

Do not trust in disclaimers. Be open about the limitations of your product. Reduce the expectations in your product.

Software Patents

European patent law is fundamentally different from US patent law. In the US "everything under the sun, made by man" can be patented. In Europe patent are strictly limited to technoligical inventions. And even there. Patents have to be new, not obvious to someone skilled in the 'art' and there must be no prior art. This is not going to change in the next 10 to 20 years at least.

Patents are granted easily, but they may not be valid. In the first 9 Months after publication they are very simple to kill. Greenpeace does this all the time in the biotech area.

The likely hood of a big player (aka MS) going against a small one are slim as the PR damage would be huge, not to speak of all the anti-trust issues.

Conclusion

Open Source doesn't have particular legal problems. But make sure you read the fine print of the licenses.

Legal issues with OSS are very similar to commercial software.

Worrying about patents is of no use.

by Poul-Henning Kamp

Compters are now 50 years old. Unix is 30 years old.

We write code on the screen and not on paper. but thats about it.

Unix has blown more chances at being a big success than any other operating system. It's about making the same mistakes again.

The problem

Programming happens in the brain and not in the computer. Throw your thinking at problems not more hardware. Programmers should have slow machines.

Instead of baking a bigger cake. Unix companies fight about the same piece.

The state today

Uncountable Linux distros, a handfull of BSDs, IBM AIX - IBM the Unix way, Sun Solaris - Unix the Sun Way, HP-UX - Unix cul de sac, Mac OS X - yea, its Unix, but don't worry about it.

Unix Standards

Very weak, Incomplete, Ambigous,

POSIX. Everybody made sure that their product was covered by it. Its not a standard but rather a panorama of things to-do. MVS is POSIX compliant, and so is Windows.

The one good standard is the "POSIX 1E" security extension. Which was never formally adopted, but everybody sticks to it religiously.

The Linux Standards Base will fail because it defines what we have today. It repeats the mistakes of POSIX. Its not about how Unix should be.

Should we save Unix

_No_

Architectural mess.

No significant invention in the last 20 years

Everybody thinks in his box.

_Yes_

You get the source

The only alternative is LongHorn

Can Unix be saved - No

no market model

no cooperation to generate a market

too much politics

Can Unix be saved - Yes

Start thinking outside your box

Stop bickering about irrelevant details (BSD/Linux, Gnome/KDE)

Work on the real problems. Fight for open data. Fight the software patent mafia.

Invent things! Plan9 (namespaces), Sun (Java), Apple (User Interfaces), Your Name could be here,

Find the next Web.

Quotes

The only thing Unix has invented is Unix.

The KDE people sit in the KDE box. They have a little hole in their box to see the GNOME people.

I don't care about your license as long I get your source. I even wrote a license for it.