Tobi Waves

A week ago, a journalist of the swiss NZZ Newspaper had been directed to me by the universities press office. The guy was looking at writing a story on Open Source software and the press office people knew I had some projects in this area. It was the week of the Blaster worm. So after talking for about 40 minutes to the journalist on Wednesday he calls on Thursday to let me know that the story on Open Source had been shelfed, and he had to write a piece on Blaster. Oh, and by the way, would I know anything about the worm. I had been battling it for the last two days, so I said yes. With the effect that I was quoted in the article on Blaster.

The following week a guy from Swiss national television (SF DRS) called, they were doing a piece on Blaster too, with a 'confront MS angle'. They had read my name in the newspaper, I agreed to the interview. I organized coaching from our press office, as the TV people were from that hard hitting investigative show, and I didn't want to be caught in the middle. I was really careful not to make any too harsh comments. They have not yet broadcast it, so I don't know if I am going to look good or bad.

In the meantime Sobig has come back to haunt us, now in its F incarnation. The press office sent the local TV people straight to me when they inquired about a person to talk to regarding Sobig.

As you can see in the screenshot, I am the IT expert now. I picked that profession after the TV people told me that "Systemmanager" was way too complex and the press office people insisted, that "Security Specialist" was going to cause trouble with the folks from CS or central IT Security. Just in case you ever wondered how you become an expert.

I guess I just had my 15 minutes of fame :-).

I have known it for a about a year. SciFi has not renewed its commitment to Farscape which essentially meant the end of the show. I am sure none of the folks at SciFi who made these decisions had watched the show. How else could it be that they killed the single most captivating, intelligent, humorous, sexy, thrilling, emotional and realistic sci-fi show ever conceived. Almost every episode I watched left me tingling all over, ideally wondering how they were able to come up with these great episodes over and over again.

It's almost a year since the cancellation, the Farscape fan-base still seems to be going strong at (www.savefarscape.co ...) plotting away on a strategy to get the show back on the air. Never loose hope.

Today I also went to the website of SciFi wondering what they were doing regarding Farscape. And indeed, they have this page "Farscape Memories" with articles from different actors remembering their time shooting the series. What hypocrisy first they pull the plug on the show and then they act all bleary eyed.

If I was rich, I mean real rich, I would just order a few more years of Farscape from the Creature Shop ... I guess I should not have gone into OpenSource, but rather have turned commercial at an early age, sucking in big bucks during the Internet Bubble then I could do more than keeping my fingers crossed for another network picking up the show again.

The other night I went to see "Wilbur wants to kill himself" (xrl.us ...) a movie about a guy, his brother, a women kid living in some Scottish town. The guy has a book shop, falls in love with the women and the brother tried to kill himself. The movie is by Lone Scherfig a Danish writer/director who was quite successful with the movie Italian for Beginners. I loved the story and the acting and above all the Scottish accent.

The strangest thing happened to me a few minutes into the film. When Alice (the women) entered the scene for the first time. She looks exactly like a friend of mine. I am quite sure it was not here, because she does not speak with a Scottish accent, and she is not an actress. Never the less I kept comparing her and Alice all through out the film. How would she act in a similar situation? And just to make it more complex, Alice is just a role, played by Shirley Henderson (xrl.us ...) but written by Sherfig, so how can it have any bearing on my friends behavior, or do looks influence how people behave, did Sherfig write the part of Alice for Shirley? Are Alice and my friend twins separated a birth?

Well it was a wired experience, but definitely a great movie. Go watch it!

I admit, I am a soap opera junkie. I just love watching them. My current favorite is The West Wing written by Aaron Sorkin (en2.wikipedia.org ...). Tonight I made it through the final episode of the first season. This time the outlook is great, as the show is till running in the US, now in its fifth season.

The West Wing is about people working in the White House for the President. It's also about the President himself. President Bartlet, played by Martin Sheen is a liberal Democrats dream. What a contrast to the current reality. An American friend told me recently that she finds it rather disturbing to watch the show, seeing what the reality could also be. Michael Moor even nominates Bartlet for President, along with Oprah in his latest book.

I am watching the West Wing on DVD, there was also the 15 minutes Making Of feature on one of the disk. There the actors talk about their parts and some bits of the sets are shown. Hearing the actors talk was odd. They were so different from the roles they play in the show. Not only did they talk differently, but also their body language was altered. I have never noticed something like this before in such a show. The characters in the West Wing make a very authentic impression on me, much more than the actors themselves actually. At least as far as the story line and the dialogs are concerned this has to be credited to Aaron Sorkin. Maybe Michael Moor should be nominating Aron Sorkin along with Oprah and Bartlet, someone has to write their lines after all.

A Talk by Urs Hölzel, Vice President for Technology,

About Google

Mission: TO organize the worlds information and make it universally accessible and useful,

An international company: 250% traffic from outside US

Engine has 4 Billion pages in index

Profitable since q1/2001

23 Office Location Worldwide.

15k boxes, several TB disk storage

There are over 1000 queries a seconds on dec 25th, 2am.

Engineering Offices in the US, Zurich and Bangalore

About the Web

Static web 167 TB in 11 Giga Pages, but dynamic websize 92 PB. (Estimates)

1 in 4 hosts on the net run a webserver.

Problem: All data, users, hosts grow exponentially. This means the problem of finding useful information grows exponentially too which makes for interesting problems.

Google Infrastructure

A high reliable system based on low cost comodity hardware. Redundancy has to be built into the software and hardware. Monitoring, repair and maintain these boxes is a prime problem.

The Google Filesystem GFS

Stripe files across many boxes and replicate them on multiple servers.

Components: Master - keeps directory and plans file layout, ChunkServer - hold the data. Clients - use the data. (Chunksize is 64 MB. Data is cached on client once retrieved. SOSP'03 (www.cs.rochester.edu ...) )

10+ Clusters of 1000+ boxes.

350 TB Filesystem

How to be a Search Engine

Crawling: Recursive Process. Problem: dynamic pages, slow servers, management of the link list, session ids in the URL, how to prioritize the URLs, being nice to the web servers, detection of duplicates, avoiding traps, actively fill forms to pull "hidden" contents, figure out when the page needs to be re-crawled.

Indexing: Words by document and position in the document. One Terra Words in the index.

Ranking: Hard problem. All traditional assumptions on searching like long, coherent, high quality documents are not valid for web documents. Googles idea is to define a PageRank for figuring the importance of the page. The PageRank of a page is the sum of PageRanks of other pages pointing to this page. A page contributes its PageRank divided by number of out-links to each of its target pages. In reality it is more complex. Google has about 100 factors in its real PageRank function like font size, color, proximity to other words.

Serving: Partition the data to different servers and have each solve a sub problem of each query. Query goes to Google Webserver, it queries Index Farm, accesses the Doc Farm for the real data. Additional services from Add Server and Spelling Server. IEEE Micro, 2003 has more on the structure (www.computer.org ...) .

Advertising: Find the best add, relevant to the query. This is a very important problem as this is the main source of revenue. Only show an add which has a chance to be clicked on, if the click-through is low, the add will be dropped. Advertisers only pay for adds actually clicked.

Google Playground

There is lots of data and computing infrastructure at Google. Google pays people who spend their time on figuring new ways to analyze and present this data: (labs.google.co ...)

In the good old times, when men were still men and computer virus writers were still technically brilliant hackers. Viruses used the uncountable holes in Microsofts ubiquitous Outlook eMail software to spread.

But even then, generally the rule was simple. If you don't want to be infected, don't run any code you don't know where it's coming from. If you have Outlook, make sure it's patched and properly configured. In Unix circles, people made fun of the whole situation by sending out mails which claimed to be a solidarity Virus, calling upon the Unix User to copy this mail to all addresses in his address book, to emulate a virus, as a gesture of solidarity with the Windows crowd.

Now, a few years later, Outlook has matured to the point that there have been no major holes for months. Never the less, eMail viruses still crop up and spread. Virus writer started to attack the users mind directly, by writing messages into the body of the virus email with the purpose of confusing the user into clicking the virus attachment, forgetting all the good advice they got. Fortunately the anti-virus software gets updated so quickly, that viruses are normally contained quickly.

Today though, mark the date, the whole matter entered an all new stage. I got the first virus which was contained in a password protected zip file. The password was contained in the accompanying email, so it is easy for a human to unpack, but anti virus software has no chance as it can not decrypt the zip file containing the virus. As a concept this sounds fine, but what totally kills me is that it seems to work. Since this morning, I get an increasing number of encrypted zip file viruses. There must actually be people who get this virus, unzip it using the supplied password and then run the thing in order to get infected.

I wonder how many people would hang themselves if they got a rope in the mail. Warden make sure all the cells are locked.

by Ralf Hildebrandt

How to use Postfix as a crude but cheap filter against spam in front of the more complex filters like spamassassin.

Sources of Spam

An important source of spam these days are miss-configured web proxies which proxy to smtp ports as well and let outsiders connect.

Protection

Use RBL lists for open-proxies, open-relays,

Reject mail from faked sender address (see below).

Insist on RFC conformance (this can make you loose lots of real mail to as there are many missconfigured normal mailservers.

Content Filters: Altermime, SpamAssassin

On Postfix

Use the snapshot version of Postfix as it is realy stable and has all the latest features.

Use a cashing nameserver to speed-up dns lookups.

By default postfix is configured to only accept mail from your local network for external destinations. This has no influence on spam though.

Be careful choosing RBLs because there are many badly maintained blacklists out there. Blacklist must have clear criteria and a delisting procedure.

postmaster@yourdomain and abuse@yourdomain must accept all mail this must be explicitly listed in smtpd_recipient_restrictions.

A good RBL list cbl.abusenet.org recomended by Ralf.

When you are using RBLs make sure that you can quickly add exceptions to your system.

Rejecting mail to unknown users at the smtpd stage is very efficient as it first saves traffic and it also saves you from sending bounces.

Postfix can use various directory services to figure out which users exist. Postfix 2.1 will even cache answers

Use right hand sender black lists may also help. But be care full. Look at =dsn.rfc-ignorant.org, postmaster.rfc-ignorant.org, abuse.rfc-ignorant.org, whois.rfc-ignorant.org=.

RBL/RHSBL are expensive because of all the DNS lookups. Perform them as late in the restrictions list after the cheep mails.

Sender address verification

Check if the sender is either a known valid or can be verified to be valid. Postfix has special support for this as it can send test messages to the sending host. The sender will not notice this as postfix only starts sending mail but aborts before giving any message body.

Make sure you are really careful as this can cause you to loose mail from people who are not able to correctly spell their sender. One option is to apply these sender check restrictions only to suspected domains.

by Patrick Koetter

How to support mobile users to use your server as a mail relay. IP based restrictions do not work as the mobile users will have random IP addresses.

SMTP AUTH

Using Cyrus SASL2 and OpenSSL together with Postfix. You can configure postfix such that it allows relaying access for users who are properly authenticated. Most mail clients support snmtp authentication.

The problematic thing is to properly configure SASL. Get the CVS version as it is less buggy then the official 2.1.17, it even has some minimal documentation.

SASL configuration is governed by a config file called the same as the program using the sasl library. In our case this is smtpd.conf.

If you use SASL with plaintext passwords, make sure it only allows AUTH when TLS is in operation.

Check out Patricks howto on this (postfix.state-of-mind.de ...)

Certificate based Relaying

For people running mobile Unix it is possible to setup a local mailserver which just forwards all mail to the official mailserver of your site. By configuring the postfix smtp daemon to use TLS on the client, and you store the clients cert on the server. Now configure the server to ask clients for a certificate when they connect. If a client submits a vlid (known) certificate it will be allowed to relay even if it has an ip number outside the local network.

The cool thing about this is, that now any program on the mobile unix client can send mail via the local mail server to the company mailserver without further problem.

by Theodore Ts'o

History

VA Linux wanted to be the IBM of the Linux world, but forgot to ask whether maybe IBM wanted to become the IBM of Linux. This was a huge mistake.

What makes Linux significant

One of the first projects done primarily over the Internet without a central group contributing all the code.

Difference to other projects. Many of the people involved are on a mission to distribute the knowledge about the system. Even people who can not bootstrap their own system are welcome.

Open Source and Free Software

The Open Source movement brought the whole "making source available is cool" concept into the industry without the political issues of the GPL.

Intellectual property laws are going to become more and more of a problem, so we have to become involved.

The huge number of Open Source licenses leads to incompatibilities when working with libraries. The important lesson here is to pick the right license from the start, as relicensing is difficult as soon as you have received a lot of code contributions.

Rapid Development and Open Source

Infrastructure work or documentation writing are necessary tasks but they are not very sexy. We need to cater for them. Incremental changes are not the only thing.

Even though there has not been a lot of innovation coming from Redmond. This does not mean they are sitting on their hands. They are working on vertical integration technologies like their new user interface description language which will be used on all their products.

User Interface Testing

Have programmers watch a users working with their software. Do not let them talk to the users and help them. They will see all the problems which are to be fixed.

The Limits

Software with a short shelf life like tax software or games needs a huge investment in development and have to recover the cost quickly, So this will probably never work with opensource concepts.

Binary drivers are a fact of life due to paptent and even legal issues. There is not clear solution for this yet in connection with Open Source.

Closing

It's been a great 10 years, but there is still a lot todo.

by Robert Griessmer

For the technical Information, read my earlier entry about Urs Hoelzels talk.

About cheep hardware: If one writes his software under the assumption that hardware is going to fail anyway. This makes lots of problems much simpler. If the number of machines grows really high, you can employ people who spend their days swapping broken ones.

These days Google designs their own bare-bone boards with only the necessary components for the stuff, the machines are required to-do.

Google Philosophy

Make the worlds information search able.

Do things that matter, and do them algorithmically so that they are scalable.

Every engineer has a free day per week to work on his own fun projects.

Google Tricks

Spell checking happens statistically. Google does not know what is right, but what is most popular.

Google provides an API for programmers.

Experimental projects (labs.google.com ...)

Google Zurich Office

Right in the city center. Entire range of software development. Meant to attract European engineering talent. Tight integration with the Mountain View offices.

Jobs: (www.google.ch ...)

User Questions

Q: What is your crawling bandwidth

A: Well about this (shows with hands) Size.

...

A: oh and ... no comment ... I can't really talk about this.

by David Rosenthal

Open Source in Government

Could government require that only Open Source software be submitted for a new text processing system to be procured? Yes, but only if the fact of it being Open Source was directly tied to the core functionality of the software and requirements in this area. This point may be difficult to make, depending on the nature of the software.

If GPL software is modified by one branche of the governemnt and passed on to a different branch. This is a sort of publication. Accordning to the GPL license the second branche gets a license with the code which allows it to redristribute the modified version without restrictions, takeing it effectively out of the control of the first party.

Copyright

Every program of an "individual nature" is protected by copyright law. "Individual" means, that the code is potentially recognizable as being written by that particular author. This right does not have to be registered or patented or anything. Every author has it.

In Swiss law this means that I as an Author have the exclusive right to decide whether and how my product is used. I can give all or some of my rights up to a third party .. aka grant a license. GPL or any of the other Open Source licenses operate on this principle.

GPL can be enforced. 19.5.2004 the Munich District upheld the GPL in a case of netfilter team vs firewall vendor.

Contract Law

If I modify an Open Source product for a third party (contract) I can become liable for any problems with the whole product if I do not exclude this in the contract. This is not special for Open Source products but it may easily go under the radar. Becareful when drafting your contracts.

Warranty and liability can be limited, but only to the extent allowed in the law. Swiss Law: Any limitation of liability for gross negligence or willful intent is void.

Do not trust in disclaimers. Be open about the limitations of your product. Reduce the expectations in your product.

Software Patents

European patent law is fundamentally different from US patent law. In the US "everything under the sun, made by man" can be patented. In Europe patent are strictly limited to technoligical inventions. And even there. Patents have to be new, not obvious to someone skilled in the 'art' and there must be no prior art. This is not going to change in the next 10 to 20 years at least.

Patents are granted easily, but they may not be valid. In the first 9 Months after publication they are very simple to kill. Greenpeace does this all the time in the biotech area.

The likely hood of a big player (aka MS) going against a small one are slim as the PR damage would be huge, not to speak of all the anti-trust issues.

Conclusion

Open Source doesn't have particular legal problems. But make sure you read the fine print of the licenses.

Legal issues with OSS are very similar to commercial software.

Worrying about patents is of no use.

by Poul-Henning Kamp

Compters are now 50 years old. Unix is 30 years old.

We write code on the screen and not on paper. but thats about it.

Unix has blown more chances at being a big success than any other operating system. It's about making the same mistakes again.

The problem

Programming happens in the brain and not in the computer. Throw your thinking at problems not more hardware. Programmers should have slow machines.

Instead of baking a bigger cake. Unix companies fight about the same piece.

The state today

Uncountable Linux distros, a handfull of BSDs, IBM AIX - IBM the Unix way, Sun Solaris - Unix the Sun Way, HP-UX - Unix cul de sac, Mac OS X - yea, its Unix, but don't worry about it.

Unix Standards

Very weak, Incomplete, Ambigous,

POSIX. Everybody made sure that their product was covered by it. Its not a standard but rather a panorama of things to-do. MVS is POSIX compliant, and so is Windows.

The one good standard is the "POSIX 1E" security extension. Which was never formally adopted, but everybody sticks to it religiously.

The Linux Standards Base will fail because it defines what we have today. It repeats the mistakes of POSIX. Its not about how Unix should be.

Should we save Unix

_No_

Architectural mess.

No significant invention in the last 20 years

Everybody thinks in his box.

_Yes_

You get the source

The only alternative is LongHorn

Can Unix be saved - No

no market model

no cooperation to generate a market

too much politics

Can Unix be saved - Yes

Start thinking outside your box

Stop bickering about irrelevant details (BSD/Linux, Gnome/KDE)

Work on the real problems. Fight for open data. Fight the software patent mafia.

Invent things! Plan9 (namespaces), Sun (Java), Apple (User Interfaces), Your Name could be here,

Find the next Web.

Quotes

The only thing Unix has invented is Unix.

The KDE people sit in the KDE box. They have a little hole in their box to see the GNOME people.

I don't care about your license as long I get your source. I even wrote a license for it.

Tutorial by Theodore Ts'o

What to do when data got lost

Don't panic !

Stop and think, figure out what happened. create a backup with

|dd if/dev/hda1 of/dev/hdb1 bs1k convsync,noerrors|

If you have no spare disk, buy one. The disks are way cheaper than the data.

Disks have a life span of 2-3 years, if they are in heavy use ... you might want to swap them preventively just to be sure.

Physical issues

Harddrives can not only experience head crashes but also "high rides". This is the name for incidents when a head flies higher than normal. This condition will only get noticed when data is read back. The new solaris zfs tries to catch this problem by reading back recently written data whenever it has spare time.

Hard drives survive only about 50'000 power downs due to the controlled head crash happening in the landing zone ... This can be a real issues for laptop configurations where frequent disk spin-downs are used to save batteries.

Some harddrives are not designed for continuous use. This will be noted in the spec sheet ... Make sure you check the spec of cheap disks you plan on using in your web-server.

A small head crash will not necessarily cause an immediate disk failure. It could just chip off small amounts of material from the disk surface which will then fly around in the hard-drive case. This condition will cause an increasing number of additional head-crashes which again will chip of material ... This means that it is a good practice to take a full linear image backup of a 'damaged' disk as soon as possible. Errors may well increase as you work on fixing it.

Get a new disk if you find any bad blocks on a disk.

Modern disks

Disks used to have a simple physical geometry. This is still visible in the head, cylinders, sector geometry specifications. Modern disks use constant bit rate and multiple long spiraling tracks to fit more data. This is all hidden by the controller and exposed through a simple linear block number to the OS.

The only thing one can assume about physical disk layout is, that two blocks which are numerically close together will normally have a short seek time.

EFI/GUID partitioning schemes

_Universally Unique IDs (aka GUID)_

A GUID is a 16 Byte number. Either a random number. Collision probability 1/2^64 (birthday paradox). Another method is to take the mac address of the computer plus a hires time stamp. A 3 bit code in the UUID/GUID shows the method used to create the GUID.

The EFI/GUID partitioning scheme uses a GUID to identify each disk as well as each partition. Partition types are "well-known" GUIDs, but still GUIDs (16 Byte) this allows to have unique identifiers for each filesystem type without a central registry.

An EFI/GUID partitioned disk contains an old style MBR patition table in the first sector which claims that the whole disk is covered by a special partition type. This prevents old OSes from messing with an EFI disk. Linux can do EFI partitions on any machine you run linux on. The only special problem is to have a boot loader which is able to deal with it.

About the FAT FS

Because all files are stored as single linked lists, random access is very hard. This also makes file fragmentation very bad. On top of it FAT uses a first free block allocation scheme which again furthers fragmentation.

Inode based Filesystems (FFS)

Stores only the filename and a link to the inode in the directory. The inode then stores all the meta information on the file. This allows to create hard-links.

For short files all blocks are linked directly from the inode. Longer files are created with indirect blocks. Even longer ones are stored with double or even triple indirect blocks.

Inode based filesystems are very fragmentation resistant. This is the reason why there are no defragmenters for Linux.

Old FFS filesystems like UFS allow to specify the physical geometry of the disks to optimize the physical allocation of the filesystem elements. Newer FFS implementations do not bother with this anymore, as there is nothing to be known about disk geometry anyways.

How to recover from accidents

_Overview_

Ask yourself what has happened?

What is the lowest level where you have problems. Always fix the lowest level first.

How important is the data?

When was the last backup performed.

Create a plan of attack before you do anything else.

_Hardware Level_

First indication are often console messages from ide/scsi driver. If you catch a correctable error, you may be able to replace the drive before it actually breaks.

If you see BadCRC errors on a new system it may indicate a simple cabling problem.

The "dev xx:yz" elements in disk errors identify the device file minor/major number affected by the error and thus the partition.

Use |e2fsck -c| to mark bad blocks and see what files are affected.

Check S.M.A.R.T. logs.

In any case make a full image (dd) backup of the disk.

For the image backup you may use |dd_resque| from (www.garloff.de ...) it will alter its block size when it hits a problem to recover as much data as possible without loosing speed while reading is easy, and it has a progress bar.

Partition Table Corruption

If the filesystem can not be found, it may be "only" a problem with the partition table.

|fdisk -l| will show what is there.

Make a backup copy of the MBR. (dd is your friend)

|gpart -W /part.table /dev/hda| can scan the disk for filesystems and reconstruct the partition table. Old filesystems from old partitions still sitting on the disk may confuse gpart.

Filesystem Corruption Problem

Errors may be reported by |e2fsck| during quick boot check or during a full check.

EXT2/3 can also detect errors as it runs ... the actions it should take in this case can be configured at mount time or through tune2fs. For laptops 'remount-ro' is advisable. Servers should better 'panic' as this allows the system to get back into a sensible stat and not limp along. Often such minor corruptions are fixed in the |e2fsck| phase.

In general running |e2fsck| with -y (yes to everything) is fine as you can normally not do anything else than say yes anyway, but |e2fsck| may move orphaned inodes and disconnected directories into 'lostfound' and this should be cleaned up before booting the system fully. The 'file' command can help to identify files. The locate database can help identify the original location of the directory.

e2fsck will not notice blocks with wrong data which are part of a file as it does not maintain any CRCs.

Undeleting Files

In EXT3 unlink will zero out inodes and can thus not be recovered. (this may be 'fixed' at some point')

Undelete on a system level is not possible with EXT3.

Use userspace delete/undelete tools.

Oh and make backups.

|grep -ab regexp /dev/hda1 | awk -F: 'printf(%x\n", ($11023)/1024);}'| (use 4095,4096 for 4k blocks)

Gives the disk blocks where the regexp was found. Then use |lde| to examine the blocks visually.

e2image

The |e2image| tool lets you create a backup of the inode table.

The latest (not released yet) debugfs can use the inode table from an e2image backup, this allows to recover lost files. Even an accidental mkfs can be reverted to a large extent (contents of the root directory will be in lost+found).

It is good practice to run e2image every night.

S.M.A.R.T.

This is the internal health monitoring system of modern hard disks. It will give early warning about disk problems in the waiting.

|smartctl| and |smartd| are your friends here.

Conclusion

Make backups. Save your sanity.

by Greg Koah-Hartmann

Most Unix systems have a device filesystem. So does Linux with devfs. There are three main problems with it.

The code is ugly and beyond repair

The namespace is not LFS compliant

The author of the code has out of the loop for about two years.

A new solution has to be found, as the state of the /dev tree without some automatic management is not tenable. In Debian for example there are 18'000 static entries in there. And on the other hand there are USB plug and play devices which tend to get a different device name every time they are plugged in.

The only thing udev can not do, is to detect a process trying to access a device node that does not exist and then load the relevant driver. This feature of devfs does not seem crutial though.

In the kernel 2.6 there are two main components which make a new and simple solution possible:

The kernel can call a program called |/sbin/hotplug| whenever new devices are connected to the system.

The sysfs filesystem (mounted under /sys) contains all information about devices known to the kernel.

Udev provies a small userspace daemon which manages the /dev tree. It can populate it with a small set of default devices like ttys at boot time and then go on to add all other devices known to the system. It is configurable via simple text file with rules about the naming of the devices. These rules can be pretty sophisticated. Usb devices can be identified according to their vendor or product string as well as through any other property they provide. It is even possible to make udev run an external program which examines the device and then decides how the /dev entry should be called.

All distributions have adopted udev for their linux 2,6 editions. There are some teething problems with distros not using the official udev helper scripts. The author himself maintains the gentoo package.

Udev has to be started VERY early in the boot process, so that other programs can access the devices. Depending on the setup it may be necessary to add udev to initrd. Volume managers and RAID setups are mentioned.

*

by Chris Welti

Motivation

Historically applications running over wide area networks was very bad and people got used to this. Today there is a lot of bandwidth available and the quality is quite good. Still there are performance is not as good as one might expect. Due to the big number of different entities involved in todays complex networks it is very difficult to debug end to end performance issues.

What is PERT

In December 2002 a group called PERT (analog to CERT) was created to help debug such end to end performance issues. PERT is a virtual team of experts. There are cross-discipline experts as well as subject experts. This gets the group both, a good overall as well as deep insight in specific areas.

What do they do

PERT accepts reports about performance problems and analyzes the end to end situation. Information is gathered from all parties involved. Once problems are identified, PERT organizes the implementation the solution as there is often a lot of know-how and know-who involved in getting this fixed.

Performance Hints

TCP buffer size on the sender and the recipient side should be about 2MB on machines which are trying to do fast transfers over wide area links. Default is often 64k. In a real world application this caused file transfers to go from 10 MB/s to 90 MB/s. Rule of thumb: |buffer = bandwidth * round-trip time|.

Duplex miss-match. This is mostly because of hand configured duplex settings on network cards. If one side is configured by hand this normally causes the other side to automatically assume it is on a half duplex link, regardless of the actual manual settings. Rule: always have both sides either manual or auto-negotiated.

Internal buffers of SSH are too small for fast transfer over vast high latency links (see first rule). Needs to be adjusted at compile time! See also (www.psc.edu ...) for more on this.

Small txqueuelen for Giga Bit Ethernet adapters on Linux. Make it larger: |ifconfig eth0 txqueuelen 1000|.

Large round trip times for

Methodology

After identifying the network path, entities and technologies involved PERT applies a divide and conqueror approach to home in on the bottle-neck.

Join the Effort

|pert-discuss@switch.ch|

by Rik van Riel

CKRM (Class-based Kernel Resource Management) is a framework for assigning resources to processes and network connections. Resources can be CPU time, memory, disk IO bandwidth and inbound socket connections. Additional virtual resources like "number of tasks" make the prevention of fork-bombs possible.

In contrast to other Resource Management systems, CKRM comes with a virtual filesystem |/rcfs| which can be used to get information but also t configure the resource classes. The rcfs supports normal permissions to govern access rights. Users can use the system for fine grain control of their own resources on the system. If I have the right to use 50% of the cpu with my processes, I can further define how these 50% are distributed among my processes.

|echo 1234 gt/rcfs/task_class/tc1/targt| moves process 1234 into the task class tc1.

For the automatic classification of processes and network connections there are classification engines available. For example the RBCE (Rule Based Classification Engine).

Inbound connection control can be used to protect server applications from remote denial of service attacks by assigning local addresses to a higher priority resource class.

CKRM is not yet in the mainline kernel. Several people from IBM are working on this full time. The current state of this extension is Beta level. Some interface changes can be expected when mainline kernel integration starts.

More info on (ckrm.sf.net ...)

by Amon Ott

Why

Classic Access Control is not sufficient to define secure access scenarios:

The granularity is way too small (only user group read write execute)

Every user can decide the access permissins for his files.

root can do everything

Solution

Other models (not one, but several appropriate ones) are required for describing security policies according to the requirements of the project.

RSBAC is a framework for implementing access control systems.

It can control individual users and programs as well as incoming and outgoing network connections.

The first stable version has been released in March 2000.

RSBAC can be extended with loadable modules.

Auditing and logging is supported at every level.

Architecture

Subjects - are processes acting on behalf of a user.

Object Types - like FILE, DIR, PROCESS, USERS, NETDEV, ...

Requests - abstraction of what the a subject wants to-do with an object. (eg. R_LINK_HARD, ...)

RSBAC acts on system calls. When a system call is received, the call gets intercepted and passed on to the decision-making facility where a decision is taken if the system call should be performed or not. Once a system call is performed a notification is generated so that the access control system knows what is happening and can take this into account for further decisions.

Models

RSBAC supports number of different access control models.

AUTH - Can be used to restrict which UIDS a process can change to.

Role Compatibiliti (RC) - Subjects and objects are sorted into roles and object types. The rules are then described based on the roles and object types. This makes simple to keep rules stable even though users and objects change.

ACLs - Who may access which object with which rights. RC Roles can be used in this.

File Flags (FF) - Secure Delete, Append Only.

Linux Capabilities (CAP) - lets you control normal Linux capabilities from outside the process.

Process Jails (JAIL) - Like BSD Jails (the better chroot)

Resource Control (RES) - File size, Memory, CPU time, ...

Pageexec (PAX) - anti stack smashing ...

How to get it

Kernel patch from www.rsbac.org

Test ist with the iso images from (www.adamantix.org ...)

Quote

The basic idea of RSBAC is to introduce a second level of security to make sure that errors and mistakes one makes in the first level do not lead to disaster.

by Fredy Künzler

Things I learned

There is no money in ADSL. Swisscom expects the market to be saturated next year.

In summer 2005 Swisscom will offer SDSL (probably 2mb symertical) everywhere. It will be over-booked and not guaranteed availability.

SDSL is like ADSL but without Voice Line on the same wire.

Bluewin 40 million loss per year.

Sunrise can only survive because of their GSM license.

There is no Money in ADSL

End user pays CHF 45.55 (+VAT)

Provider pays CHF 31.20 for the ADSL link to Swisscom. In addition to this the provider has to pay for the network bandwidth between his network and the ADSL backbone (backhaul) this cost CHF 391 for 1 Megabit/s per Month. With moderate overbooking he can fit 40 ADSL customers into one Megabit. This adds another CHF 10 for each of his cutomers.

This means at the end of the day the ISP gets about 4 CHF per ADSL link and month.

The normal wholesale price for 1 Megabit/s connectivity is CHF 100 per Month.

Whats worse, in spring 2003 WEKO got Swisscom to lower the prices they charge for ADSL connectivity by 20%. Swisscom is fighting this decision in court. If they winn, all ISP will have to back the 'missing' 20% back to Swisscom. This will cause a great many of them to go out of buisness.

A tutorial by Joost van Dijk.

About the IPv6 Header

It is much simpler than the IPv4 header. It has a fixed length of 40 Bytes.

Rule 1: Make the frequent case fast

Complex things like fragmentation are handled with extension headers. Because IPv6 routers do not do fragmentation anyway they can do with just looking at the base header. Fragmentation happens in the sending host. The sender uses Path MTU discovery.

With the flow label in the header, The router can see which packets belong together without looking into the packets themselves.

Extension headers can be inserted between the IPv6 header and the payload data. These extension headers come in a predefined order. This order again ensures that the routers only have to look at the first few extension headers because only they can contain information relevant to a router.

Each header has a field called next header which defines the type of the next header.

Implementations

Most current OSes and routers support IPv6. Since Windows XP SP2 there is a production quality IPv6 implementation for Windows as well. The first prototypes were available as a separate downloads from MS.

Enabling it on Windows XP SP2

netsh interface ipv6 install

IPv6 addressing

The first few bits of an IPv6 address define the type of the address. Every IPv6 interface has a link local address (private local address space).

Addresses are written as 8 colon separated tuples of 4 hex digits.

One sequence of zeros can be abbreviated to a double ::

2001:0000:0000:0000:0000:3233:da33:3ad3 becomes 2001::3233:da33:3ad3

The loop back address is ::1

Global Unicast Addresses are built like this

001 - 3 bit
Top Level Aggregator (eg. RIPE) - 13 Bit 
Reserved (these bits could be added the TLA or NLA field in the future) - 8 Bit Next Level Aggregator (ISP) - 24 Bit
Site Level Aggregator (Subnet) - 16 Bit
Interface Address - 64 Bit

RFC 3587 obsoleted this format recently. In the future, the toplevel registrars will decide where the borders are.

Because of the hierarchical nature of addressing the routing tables will become much shorter for IPv6 routing.

Currently only 3 TLAs are defined. Today most new addresses are from the 2001:: Sub-TLA Assignment range. There you get 13 bit sub TLA and 19 bit NLA.

The 48 bit Ethernet addresses can be mapped to the 64 bit IPv6 interface address: (first 24 bit, FFFE, last 24 bit) this is not required though. You can use a random number. Just make sure you get no duplicates.

Addresses starting with 0 ending with an IPv4 address can be used for automatic tunneling.

Current versions of the host dns lookup tool will find IPv6 addresses and it will do reverse lookups automatically when given a numeric address.

Multicast addresses

They start with "FF" there are some well known addresses like

ff02::1 - all nodes on the link

ff02::2 - all routers on the link

ff05::1:3 - All DHCP server at this site

There is a special entry in the routing table for multicast FF00::/8.

ping -c 2 -I eth0 ff02::1 will find all hosts on the local link.

In IPv6 there is a new version of the ICMP protocol (known from ping) it is now also used for ARP and multi cast group membership management.

Get the IPv6 routing table on Linux

route -A inet6

or use the shortcut notation

route -6

Getting an IPv6 address in IPv4 land through tunneling

On way, is to use 6to4. (tldp.org ...) Note that the gateway 192.88.99.1 is a global any-cast address which will automatically go to the closest IPv6 gateway. The 6to4 approach requires you to have a public IPv4 address on your machine or a NAT gateway which can do protocol 41 NAT (protocol 41 is used for 6to4 tunneling)

The new Teredo protocol allows even boxes behind a NAT gateway to get connected to IPv6. Windows XP SP2 has this feature built in. On Linux there is an implementation called miredo which can do the same.

A tutorial by Gerald Carter.

General Things

The big accomplishment of the Samba team is, that they document stuff which MS does not document.

In October 2004, support for Samba 2.x will be dropped.

The configuration parameters parsing and the autoconf files in samba 3 are larger than the whole samba distribution of 13 years ago.

Samba 4 is a complete rewrite from ground up. Don't wait for it!

Samba 3.2 will get backports of some 4.0 features like their RPC code. Better ACL support. Make sure Samba servers look even more like Windows Servers.

There are about 3 people working on Samba 3.x and 3 people working on Samba 4.

A further goal for Samba 4 is to make CIFS protocol as a viable alternative NFS. Unix extensions are being worked on to workout the wrinkles with non unified UIDs.

Samba 3 tibits

The big underestimated tool in samba 3 is net. It is similar to its windows namesake. Unfortunately there is not much documentation on it. But if you start it without parameters it will tell you what it does.

If you run samba without netbios support and you want to use several different configurations on the same server you can add virtual interfaces and then use the %I option for loading different configurations depending on the interface the client connected to.

Samba always tells its version number. This is not a security issue, because if knowing the samba version allows someone to hack into a samba server, this means that there is a bug in samba which needs to be fixed.

Per-service parameters, set in the [global] section will become the default for all services which do not set the parameter explicitly

To reduce the load on your samba server, use the deadtime option in the [global] section. It is set to 0 by default. If you set it to 15 samba will kill seemingly dead connections (happens a lot with print clients) after 15 minutes without negative effects on the client side in general.

In the samba config file you can access environment variables using the %$(ENVVAR) syntax.

SWAT the samba administration GUI will probably be integrated into samba by letting smbd execute swat for connections on port 901.

Windows will not show any shares ending in $. This is only cosmetic though, it does not prevent connections to the share. Using the 'browseable' setting may make more sense as this will prevent listing of the share from the server side (still no security, but you are free to choose the name).

Configuring samba for guest access

[global]
map to guest = bad user
guest ok     = yes
username map = /file
...

And make /file contain

# map everyone to an invalid share 
foo = *

Samba Authentication

Windows uses a challenge response system when authenticating users. This requires both ends to share a common secret. Windows does not store plain text passwords, it does encrypt them, but there is no salt in lanman hashes (windows encrypted passwords). Even worse due to the challenge response system, anyone who is able to get a copy of a encrypted password can use this with a properly hacked smbclient to access the corresponding windows account. Lanman v2 hashes added some measures to prevent 'man in the middle' attacks, but the base problem remains. This means you have to be much more careful to prevent 3rd parties from accessing encrypted windows passwords as they do not even have to be cracked before they can be used.

There is also a positive side to this, because due to the challenge response approach, a hostile (hacked) server will not be able collect passwords from users trying to log on.

Samba can use multiple passdb backends. If several passdb backends are defined in smb.conf, samba will search all backends. If a password gets changed, samba will change it in the passdb backend where the password came from. If a new user is added it gets added to the first passdb backend defined in passdb backend

For storing additional information per user, use at least the tdbsam backend. The text based smbpasswd can only store the most basic information.

Quote: LDAP is not that difficult, but the problem is that people try to walk before they crawl.

Samba needs a Unix account for every user.

Note that smbpasswd does not allow entering the password on the commandline anymore, but it can take input from stdin now:

(echo pass;echo pass)|smbpasswd -a user -s

Access

If users have problems with the fact that they can connect to other users home directories, put the following in your [homes] share.

[homes]
valid users = %S

Instead of using complex mask settings for files and directories, you can set the inherit permissions parameter and manage the permissions on the Unix directory level. This allows to have only one group share with different access permissions down the tree.

Share-level ACLs are done internally in samba, so they do not require any filesystem acl support.

MS-DFS

With MS-DFS, a server can send a transparent referral to a client so that it queries a different server. To make it work the client password must work on both servers.

In smb.conf:

[global]
host msdfs = yes

[dfs]
msdfs root = yes
path = /export/dfs

In /export/dfs do:

ln -s 'msdfs:server1\share1,server2\share2,...' directory

This will cause requests for \\server\dfs to be transparently redirected to \\server1\share1 or \\server2\share2 is the first one is missing.

Smaba can even do DFS proxies. In smb.conf on sever1 do:

[proxyshare]
msdfs proxy = \\server2\anothershare

Printing

On a printer share you can define how much space must be left (in kb) before a new job is accepted:

min print space = 5000

In RPC based printing the %c value contains the number of pages to print.

If using samba as a printer server, you may want to be able to define the default configuration data which is installed together with a printer driver. For this install one printer (lets call it seedprinter) with the driver you are interested in, and change the printer defaults from windows and then call rpcclient with the magic setprinterdata value _p_f_a_n_t_0_m_ this will copy the printer configuration data of 'seedprinter' as the default for all printers who are using the same printer driver as well as for any new printer which is associated with this driver.

rpcclient -U printadmin -c "setprinterdata seedprinter _p_f_a_n_t_0_m_ xxx" server

The xxx argument is ignored, so use just any string ...

The caveat about this is, that when we tried it during the talk it did not work.

NetBIOS

Samba 3 works fine with netbios disabled. Just don't start nmbd, make sure all your servers are in DNS and use the following in your smb.conf file:

[global]
...
name resolve order = host
disable netbios = yes

Several Samba Servers using the same authentication source

To have several Samba servers authenticate against the same user database you can setup one samba server as a PDC and make the other Samba instances into clients of the samba server. Make sure you do not provide winbind with a user id or group id mapping range config so that it falls back to using the user and group ids provided by the Unix host.

Windows Integration

When storing user profiles on samba you may want to use the patch %H/.winprofile/%a as logon path this will store the users profile on a 'per windows release' basis. Note the logon path is not your home!

A PDC requires a machine trust account for each host who is using it. These accounts get created when a machine joins the domain. This means that samba must have appropriate scripts defined to be able to run these scripts, machines must join using the 'root' account of the samba server. This means you need a samba password for root, and the whole setup may make you feel rather edgy :-). The samba folks are working on this.

If you ever want to migrate a Windows NT4 PDC to a Samba domain controller the command net rpc vampire is your friend as it will suck all the account information out of an existing PDC. This relies on the availability of the scripts mentioned in the previous paragraph.