Fri, 30 Sep 2005
Upgrading to the 2.6.13 kernel
I have tried to upgrade our servers (including the IS MU application cluster) to the latest kernel (2.6.13.2), with a mixed results. Odysseus runs 2.6.13.2 without problems, for example.
However, our quad-Opteron HP DL-585 has failed to detect one of its PCI busses including the fibre-channel HBA on this particular bus. No storage, no boot. And two of the servers in our application cluster had failed to boot - one had misdetected the IRQ for the IDE controller, and the other one was not able to receive IRQs on the ethernet NIC. Later I have found that with CONFIG_ACPI=y the 2.6.14-rc2 kernel boots even on these two hosts. However, when I tried to distribute the 2.6.14-rc2 kernel with CONFIG_ACPI=y to the entire cluster, another two hosts failed to boot (this time they were not able to recevie IRQs from the IDE controller).
The strangest thing is that the application cluster consists of two types of hardware only, and for each of the above failures, there were at least four identical computers which worked without problem. I suspect the problem might be in different BIOS settings or something like that.
Tue, 27 Sep 2005
Intrusion control panel and Linux
Yesterday I've been trying to make an intrusion control panel talk to the Linux box. The panel itself is connected to motion sensors via its own cabling, and with the Linux box via a RS-232 serial line. It worked on old Red Hat Linux system with a 2.4 kernel, but after an upgrade to the 2.6 kernel it stopped working - the panel looked as if it received no input from the Linux side (altough the output direction such as status reporting looked fine).
Even with 2.4 kernel the communication was not straightforward. It seems there is some kind of timing issue between the panel and the computer - in 2.4 we had to do "setserial /dev/ttyS0 uart 16450 low_latency" to make it working. With the 2.6 kernel the timing has changed, and the software has stopped talking to the panel. And what was worse, the setserial hack above nor any changes using stty did not help.
After few hours of testing I tried to recompile the kernel with faster system timer (HZ=1000), and insert a 1ms sleep before sending each byte to the panel. It suddenly started to work. I wonder what crappy implementation of RS-232 the panel has, that cannot handle even an input of two conecutive bytes (even with two stop-bits to make the gap longer). A positive outcome is that the 2.6 kernel proved itself to be faster than 2.4, because it can send individual bytes over the RS-232 port in low-latency mode faster than 2.4.
Fri, 23 Sep 2005
Buying a new keyboard
For an unspecified reason, I needed to buy a new keyboard(*). This task has proved to be more difficult than I expected. The problem was that I had some specific (albeit not unusual) requirements:
- USB interface. This is the secondary keyboard in a dual-head X11 setup, USB is a requirement here.
- Wide backspace key.
- No extra "multimedia" keys or any such crap.
- Standard-length shift keys. I wonder why some keyboards have to have two backslash keys at a cost of shortened left shift key.
Pretty simple, isn't it? Quite the contrary. I have browsed few e-shops, but only Orange & Green had the keyboard which met my requirements:
However, they did not have it in their shop in Brno. I have visited few nearby shops, but no luck. In one of them they even had a keyboard with the same picture as the above one printed on the packaging, but there was a different keyboard inside. I suspect that the packaging with that picture is a generic packaging used for various keyboards. So I doubt even O&G would give me exactly this keyboard when I ordered it from their e-shop.
In the end I bought a keyboard at Autocont - they have USB keyboard with long backspace, but with two backslash keys and shortened left shift key. It is sad that all shops are filled with crappy keyboards with unneeded bells and whistles, but they cannot offer a simple USB keyboard with long shift keys and long backspace key.
(*) Did you know that printed circuits inside the keyboard are made of a water-soluble conductor :-)?
Thu, 22 Sep 2005
Inside-out objects in Perl
In Perl, objects are represented as blessed references. The traditional approach is to use the hash reference, with object attributes represented as members of this hash:
package Beverage::Whisky;
sub new {
my $class = shift;
my %args = @_;
my $self = bless { }, $class;
$self->{name} = $args{name};
$self->{age} = $args{age};
return $self;
}
sub get_age {
my $self = shift;
return $self->{aqe};
}
# ... etc ..., and then somewhere else
my $bottle = Beverage::Whisky->new(name => 'Laphroaig', age => 10);
print 'This bottle is aged ', $bottle->get_age(), " years\n";
The main problem with this approach is that when you make a typo (did you notice that I accidentally wrote aqe instead of age in the get_age() method?), new hash member is silently added and no error is reported. Another problem is that nobody can stop the module/class user to access the attributes directly and possibly mess up the internal state of the object.
In Perl Best Pracices Damian Conway suggests that "inside-out" objects should be used instead. I.e. the objects where the attributes are not stored in the object itself, but referenced from the global hash defined in the class itself:
package Beverage::Whisky:
use Class:Std::Utils;
{
my (%name_of, %age_of);
sub new {
my $class = shift;
my %args = @_;
my $self = bless \do { my $anon_scalar; }, $class;
$name_of{ident $self} = $args{name};
$age_of{ident $self} = $args{age};
return $self;
}
sub get_age {
my $self = shift;
return $aqe_of{ident $self};
}
}
Note that this approach is not any longer than the previous version, and a similar typo in get_age() method above leads to an immediate compile-time failure. The object itself is a blessed reference to an empty scalar, which is hard to make a mess of. The per-class attribute hashes are in a local block, so they cannot be accessed directly from the outside of this block.
However, there is something missing from the "inside-out" code, and adding this voids Damian Conway's claim about the same code size:
sub DESTROY {
my $self = shift;
delete $name_of{ident $self};
delete $age_of{ident $self};
}
Without the destructor each object would leak memory used for storing its attributes. But with the destructor the code is longer and thus more error-prone. The "inside-out" approach has also serious drawback in threaded code - while the traditional approach requires no locking in the methods itself, provided that one object is accessed by one thread only, the "inside-out" approach requires access to the per-class global hashes, which implies locking and a potential performance penalty.
In my opinion the inside-out approach is interesting, but it has serious drawbacks which prevent me from using it instead of the traditional "blessed hash reference" approach. I don't need to enforce the encapsulation of objects anywhere else than in the documentation. I just want the typos in the attribute name show up either immediately (preferred) or in the run-time. Is there a better approach to objects in Perl, which would solve this problem? Maybe locked hash keys (as provided by the Hash::Util module) is what I need. Any other recommendations?
Wed, 21 Sep 2005
Car windows polarized
Do you have a polarizing filter or polarizing sunglasses? It is interesting how many unusual things can be seen through the polarizing filter.
I have found that when you look at car windows through the polarizing filter, strips, spots, or chequered pattern can be seen. On most cars only the rear window shows this behaviour, altough I have seen an old Wartburg, which had this pattern on all windows, including the windscreen.
Does anybody know how this pattern is created, and why only the rear windows have this? Does it have something to do with rear window heating/defrosting?
Tue, 20 Sep 2005
Maintaining multiple installations of software - II.
A while ago I wrote about problems with maintainance of multiple independent installations of software with local modifications on some installations. Now I can offer a few practical tips based on my experience with solving this problem (some of them are Perl-related):
- Put the config file name to the command line.
- The original software had the configuration file location hard-coded in the program itself. This means that even when you want to run multiple copies of the software on the same machine, you have to copy the whole program tree, which is a maintainance nightmare.
- Use the FindBin module.
- This module can save you from both hard-coding the module path into your scripts or requiring your scripts to be run from one directory only. This can help especially when the programs are started from various places (such as /etc/inittab), where setting the work directory or PERL5LIB environment variable is not feasible.
- Make reasonable defaults.
- You don't have to repeat all the configuration variables over and over again in each config file. When the defaults are sane enough, only the truly site-specific settings can be set in the config file. The config file is shorter and more readable then.
- Write your config file in Perl.
- When you do this, you don't need to write a special config file parser. And you can even write local hacks, which are specific to one installation only, into the config file itself as anonymous subroutines.
- Use Subversion.
- Especially when refactoring an old code lots of filesystem-related changes (file renaming, directory shuffling, etc.) are made. SVN is excellent for the filesystem-related operations, and provides a full file history including any renames. SVN is more mature than other version control systems, and when fully decentralized operation is not required, it simply rocks. There is only one shortcoming of SVN - you cannot "svn move" a file with uncommited local changes. But this can be worked around using an intermediate commit. Say sayonnara to the hell which is the Attic/ directory with the deleted files in CVS.
- Use different subtrees for different scopes.
- When you have everything in one directory (or subtree), the local hacks or even simple version control are nearly impossible. Use one subtree for things that are shared from everywhere, one subdirectory for local dynamic data, which are not under the version control system (log files, local state data, etc.), and one subdirectory for the site-specific data. I thougt about one more directory for install-specific data (when running multiple copies of software on the same host), but I have found that the separate config file for each copy (with all configs in the site-specific directory) is enough for my purposes.
So on this side things are going well, and I hope I manage to upgrade all my installations to the refactorized version of the software soon. Hope you find these general guidelines useful :-)
Fri, 16 Sep 2005
Skype considered harmful
Root.CZ has an interesting article titled "Ten reasons to not use Skype" (in Czech, sorry). Many arguments seem to be valid - when you have a real Internet connection (i.e. with a public IP address), you are in fact sponsoring Skype with your bandwidth. However, I wonder what better solution we can use?
I tried GnomeMeeting, and it works for calling from home to work. However, it is based on a central registry of users, and I don't want to broadcast my presence to an untrusted site. Maybe it can be switched to a private registry as well, I have not explored the features of GnomeMeeting further. And, it seems that GnomeMeeting has only rudimental support for SIP (it is primarily an H.323 based system). Also, I don't know whether GnomeMeeting can use a strong encryption to make the calls truly private.
I have a real Internet connection at home, so I am thinking about setting up an Asterisk-based PBX for me and my friends. However, time is the constraint here :-(
Also, there is an important drawback of PC-based software phones: the "headphones" problem - when you leave your headphones plugged in, you cannot hear the soft phone ringing, because the speakers are muted then. I have to do some more experiments with virtual ALSA devices, and create a separate virtual device from rear channel outlet of my sound card, which would be permanently connected to a "ring-only" speaker. Or maybe I should buy an USB-connected phone and use it as a voice/phone device.
What VoIP solution would you suggest?
Thu, 15 Sep 2005
Google's Blogsearch
It seems Google started a search engine specialized to searching blogs. I've tried to look up "Yenya".
Two of four results are about myself, which is not so bad. But in fact both of these pages refer to my Linux on Asus M6R page, and strangely enough, both pages are in Japanese. I don't speak Japanese, but it looks like a 302 search engine spam/cloaking, because the contents (as seen by Blogsearch) is exactly the same as my Asus M6R page, but I have not found anything like this on the pages themselves.
So in fact, Blogsearch did not find my blog. Their FAQ suggest about subscribing to weblogs.com, which I still need to figure out how it works. Maybe next time Google will find me. However, one would expect that they implement some heuristics on top of their web pages database to find blogs on the Internet themselves. The "subscribe to weblogs.com" model looks flawed to me.
Update - Thu, 15 Sep 2005: That was fast!
An hour or so ago I registered my blog to weblogs.com, and now the search results for word "Yenya" point to my blog (and the four items which were there before are displayed last :-).
Tue, 13 Sep 2005
Weather forecast
I use meteo.icm.edu.pl pages for a weather forecast - they have all parameters I can think of - temperature, humidity, wind speed and direction, pressure, clouds/visibility, and precipitation amount. And I have found that for our area they have pretty exact forecast, except that usually the amount of precipitation is lower than they forecast. Now Honza pointed me to something better (yet different), which can be used as an immediate overview of the situation:
The Czech Hydrometeorological Institute has on-line map from their radar. This can show the exact areas where it is raining now. They even have an animated version of last three radar snaphots.
Sun, 11 Sep 2005
Java or Perl?
Seen in a local coffee shop. Which one would you choose? Perl seems to have a bit lower Total Cost of Ownership :-)
Fri, 09 Sep 2005
TCP segmentation offload
Last week LWN had an interesting article about TCP offload engines in the Linux kernel. This week they added a follow-up to that article written by a vendor of TOE-equipped hardware. I don't have a TOE-capable NIC myself, but in the comments section people discussed the whether a full TOE gives any significant advantage when compared to a simple TCP segmentation offload. Mingo said that any modern-enough NIC which supports TSO is also supported under Linux, including tg3 which we use in most of our servers.
I wondered whether my version of the tg3 hardware supports TSO, and if so, whether it is better than no TSO at all. I found that according to ethtool -k eth0 the TSO is switched off at Odysseus. So I tried to switch it on (altough I wondered why it is not switched on by default, provided that the hardware supports this feature).
I tried to measure the difference by downloading an ISO image of FC4 i386 CD1 (665434112 bytes) from two hosts connected to the same switch. I did 10 transfers of the same file with each settings, and took the average and maximum of the last five transfers only (to avoid any start-up temporary conditions). The client Alpha was dual Opteron 248 with Tyan S2882 board, and the client Beta was quad Opteron 848 on HP DL-585 board. The FTP server is dual Opteron 244 on Tyan S2882 board, and ProFTPD with sendfile(2) enabled.
| Client | TSO | Average speed | Max speed |
|---|---|---|---|
| Alpha | off | 108.7 MB/s | 110.5 MB/s |
| Alpha | on | 100.9 MB/s | 101.2 MB/s |
| Beta | off | 102.1 MB/s | 102.4 MB/s |
| Beta | on | 93.2 MB/s | 95.5 MB/s |
Surprisingly enough, the tests without TSO were faster than with it enabled. Looking at tcpdump it seems that the system with TSO enabled sends only a 15 KB-sized frames to the NIC instead of full 64 KB-sized ones:
18:45:38.993150 IP odysseus.ftp-data > alpha.33125: P 127424:143352(15928) ack 1 win 1460 <nop,nop,timestamp 2408404 698142942> 18:45:38.993203 IP odysseus.ftp-data > alpha.33125: P 143352:159280(15928) ack 1 win 1460 <nop,nop,timestamp 2408404 698142942>
So I wonder what is wrong with TSO on my hardware. Any clues? I think I should poke the netdev mailing list.
Barcode scanner and Linux
My brother has a CipherLab 8001 hand-held barcode scanner/terminal, and we have been trying to make this beast working under Linux. It has a craddle with RS-232 serial cable, but instead of text protocols it spills out a binary mess to the serial line.
After looking carefully at the craddle we have found that the craddle has in fact a built-in infrared port, and it communicates with the scanner via this port. So maybe they are using the craddle as an IR serial port dongle and communicate via the IrDA protocol. The craddle serial port seems to run at 115200 bps, probably with 8-N-1 port settings.
I was prepared to try all Linux IrDA serial dongle drivers against the craddle, but then my brother found another solution, that surprisingly-enough worked: their DOS/Windows command-line utility for IR transfers works perfectly well under Wine, even without any additional libraries, and even on the command line without the graphical environment.
Nevertheless, I want to try the IrDA communication with this device from my laptop when I get some time ...
Tue, 06 Sep 2005
Guess what has failed!
4:35 am - SMS arrived: the whole is.muni.cz cluster is down. Getting up and going to the computer: strange, our router is perfectly OK, just the interface to that subnet seems to be down. Restarting the transceiver using mii-tool -R. No change. Stopping the interface via ifconfig and starting it up again - no change. mii-tool still reports that the link is down. Bad cable? Extremely improbable. A NIC or driver failure? Should I reboot the router?
A message from Nagios: the switch for that subnet is down. Indeed it does not respond to ping. The neighbour switch reports that the port connected to that switch is down. Hmm, do we have a spare 24-port gigabit switch? Fortunately we do. I am packing up my laptop and leaving home.
In the server room. The switch indeed seems to be dead - no lights. Removing and re-inserting the power cord: no change. Hmm, it looks like the entire rack is dead: none of the computers seem to be powered up. Even the control light on the power outlet strip is off. Maybe the power breaker went off? Trying to put the power cord to another plug in the wall: no change. Bad power outlet strip? It seems so. Replacing it with the spare, and carefully powering up one server after another. Good. The master switch shows that the entire rack takes between 6 and 7 Amps, the outlet strip fuse is rated at 10 Amps. Maybe the fuse was bad or something.
Fixing up few minor problems like that one of the servers was not configured to power itself up after the power loss. 6:12 am: all servers seem to be up. Unfortunately, the main database server cannot start the database. The error message in response to the oracle_start script is strange: "the database instance is already down. Talk about informative messages. Fortunately, our DBA already woke up, so I am leaving this to him. Now if only I can find some time to sleep.
Mon, 05 Sep 2005
Poking at the fossils
Our Oracle 8 client libraries stopped working against the Oracle 10 database after upgrading to UTF-8 internal encoding. Now I am trying to upgrade some of the database clients to a newer version. I came across the following fossil:
# cat /etc/issue
Red Hat Linux release 6.2 (Zoot)
Kernel 2.2.24-6.2.3 on an i586
# free
total used free shared buffers cached
Mem: 15972 15648 324 0 476 6068
-/+ buffers/cache: 9104 6868
Swap: 100760 22612 78148
#
Now the problem is that newer Oracle client libraries (we need 9.2.0 at least) are linked against glibc-2.2, while RH6.2 has glibc-2.1. So the system upgrade seems to be inevitable. However, I doubt FC4 will even boot on a host with 16MB RAM. I am sure there are many stripped-down distros, but the maintainance cost would probably be lower if I simply find a better hardware.
Hmm, my first Linux box was a 386/40 with 8MB RAM running SLS Linux distribution with Linux 0.99.11 kernel. [ Insert a random I-am-so-old or current-software-is-too-bloated rant here ]
Sat, 03 Sep 2005
In the world of grey wings
A quiet, peaceful city of Guri is surrounded by a high wall, which makes
the city a separate world of its own. Aside from humans, there is
a special kind of inhabitants of the city: children with grey wings,
called Haibane. They are born to their world from big, web-like cocoons,
and while they can speak, read, and so on, they do not remember how they
appeared in their cocoon. They usually remember only the last dream from
their stay in the cocoon. Haibane are treated as something special by
humans, as they are said to bring luck to humans. The story starts
when a new Haibane is born in the Old Home, a gloomy building where most
of Haibane live. The girl is given a name Rakka (fall), as her dream
was about falling from the sky. The story then revolves around life
in the Old Home, with the oldest of Haibane, named Reki, and new-born
Rakka as the main two characters.
Haibane Renmei (which can be translated as "Charcoal Feather Federation") is a masterpiece of Yoshitoshi ABe. Unlike his previous works (Serial Experiments: Lain and NieA_7), where he did a character design mostly, he did both the character design and wrote the story for Haibane Renmei. The animation is excellent, yet unusual. Yoshitoshi ABe's style is based on dim and lower-saturated color tones, in order to allow the main subject to stand out.
The story of Haibane is deep and well thought out, yet it leaves opened
questions at the end. Who are Haibane, and how they enter and leave the
world described in this anime? Was their dream related to their previous
life? And did they even have any previous life? The answers are not given
in the series directly, yet there are some clues left for careful watcher.
If you have just finished the story and you are thinking: WTF? Look at
the Within the Wall fan site,
where you can find some theories and speculations about the Haibane Renmei
world.
I liked the peaceful atmospere of the story - there is almost no action, and the story starts very slowly, describing and introducing the life of Haibane. Their world is far from ideal, yet they seem to be sort of happy there. In the second half of the series the story starts to follow Reki's life more closely, and in fact, the whole story is about Reki, as seen by Rakka. Reki's cigarette was the only thing I disliked on the series (I hate smoking/smokers :-). Haibane Renmei has also a decent OST, with an instrumental opening song, and nice ending song.
This series ended up first in the current Top ten animes list at Manga.CZ. So, are you tired of flashy, fast-paced action anime? Do you want to try something really different, yet still belonging to the anime genre? Then Haibane Renmei might be for you.
Links
- Within the Wall
- AnimeNFO page
- Similar Animes (from Anime-Planet)
- AniDB page
- Review at SFK Palantir (in Czech).
- Review at Manga.CZ (in Czech).
Fri, 02 Sep 2005
The adventurous physics
I have found that Brno Planetarium will have a series of lectures with my friend and phycics enthusiast Tom Tyc. The lectures are titled "The Adventurous Physics", and as I know Tom, it will be a spectacular show of a charismatic lecturer. Recommended for everyone who like to explore the inner workings of our world (and for those with math phobia: don't be afraid, no mathematics will probably be involved).