Wed, 30 Aug 2006
Svíčky 2006
On Saturday I took part in another puzzle-solving outdoor game, Svíčky. I was a member of team Abpopa.
We had a slow start as always, in the first three stages we were at about 25th place (out of 53 teams), and we were slowly improving our position. During stages 9 or 10 (I don't remember correctly) we were at the third place.
We have however spent many hours on a not-so-hard stage, which was based on Roman numerals. We knew that the distribution of code sequences matched the distribution of letters in Czech texts, but for many hours we were not able to figure out the encoding.
We have finished at stage 15 (out of 17, I think), which we have obtained as 8th team, well after the time limit, after giving up at two other stages and getting the solution from the organizers. Overall it was a nice game, though the stage with colors was probably unsolvable.
Fri, 25 Aug 2006
A New Router
Some time ago we have bought hardware for a new router, but until few days ago I did not have time to configure it. Yesterday we have finally moved the traffic to the new server, and should everything be OK, I will reconfigure the old one as a hot-spare backup. With a hot-spare backup I will have a testing platform for things like native IPv6 or multicast.
Apparently while the traffic is almost the same (well, Fridays are always weaker in the terms of network traffic), it seems that the CPU usage is significantly lower (the switchover happened before 23:00 yesterday). I estimate we can now route about 250-300kpps, while the old configuration peaked somewhere near 150kpps (this is with ~1700 IP tables rules, including connection tracking, and about 150 classes of HTB traffic shaping, though those do not carry high amounts of load). The new HW has faster CPUs (2.8GHz vs. 1.8GHz) and bus (800 MHz vs. 1 GHz), and the difference is visible. We can now probably match the liberouter performance with purely commodity hardware.
The next level would probably be buying a 10GbE interface. The new server has a spare PCIe x16 slot on an independent bus, conected by a separate HyperTransport channel to the CPU, so I think a 10GbE NIC would not clash with the rest of the system too much.
While preparing for the redundant configuration I have discovered that the new
HW (which also has two 1000baseT NICs, one 1000baseSX, and one 100baseT)
detects the NICs in a different order than the previous one. So for the
first time I have played with interface renaming. Fedora apparently
can locate interfaces by MAC address, and then rename them according to
its config. Now I have interfaces like uplink, trunk2,
etc.
And now a question for my lazyweb: How would you handle stateful data for
two redundant routers? I mean dhcpd.leases, Arpwatch data,
and few other files. So far I have found csync2 - a purely user-space but not near-realtime solution,
and DRBD, replication on a block device
level (which contains a kernel module, but who knows how stable it is,
especially on a bleeding-edge 64-bit SMP hardware).
Thu, 24 Aug 2006
W3C Date and Time
Apparently W3C has defined its own date and time format, incompatible with anything else.
And what is worse, they use the "+HH:MM" format for the timezone
offset, which means W3C time format cannot be formated with
standard library's strftime(3) function.
POSIX (and ISO C99) defines
the "%Z" fromat string for the textual name of the time zone only.
GNU libc (and Perl as well) provide an extension "%z"
which leads to "+HHMM" format, suitable for formating
RFC822-style time zone offsets. But W3C had to invite yet another format,
this time with colon between hours and minutes. Why the standardization
organizations cannot reuse an existing practice, and have always to create
something different and difficult to produce with standard tools?
CPAN has DateTime::Format::W3CDTF,
so at least the Perl side is safe (altough I have not tried it yet, and from
their docs it is not clear how they handle the time zone at all). But
it uses the DateTime object as a time source, instead of
the return value of localtime() or gmtime().
Wed, 23 Aug 2006
3ware 9550SX Versus Faulty Drive
One of the drives in Odysseus
has failed. I have got a replacement today. While installing the new drive,
I have used the 3ware utility tw_cli to handle the configuration
changes. I was surprised how well designed the 3ware controller software
was.
- It is possible to upgrade the controller firmware from Linux, without needing a DOS boot disk or whatever.
- When the new unit is created, Linux immediately can see it (using the hotplug mechanism).
- S.M.A.R.T. tests can be done using standard Linux utilities (which can see the physical drives). Or the controller can be programmed to run self-tests itself on a periodical basis.
- The controller can identify the physical drive by blinking a LED over an I2C bus (provided that the chassis has the actual LEDs - mine unfortunately doesn't have any).
... and some other things as well. This controller also fully supports physical hotplug and hotremove (but again, my chassis does not have the hotplug SATA power connectors). I think with this controller, the features available to the system administrator are as good as the ones of the high-end arrays (or maybe better, see the point about SMART). Good work, 3ware!
Tue, 22 Aug 2006
Haluz
Honza Pomikálek pointed me to an interesting site: haluz.org. It is a web-based puzzle solving game (in Slovak, sorry English-only readers!). The most unusual feature is that it is not a one-evening game, but it is designed to last for two months or so.
The game has been started three weeks ago, so most other teams have
more than two weeks head-start on us. I am in team named (guess what)
coredump,
so far with Pavlína only. The tasks are mostly interesting and inovative.
The most boring and dull one was probably
cracking the 90-digits RSA public key.
You can watch our standings
in their site as well.
A Media Day
On Friday, there was a "media day" here. We have created (read: Mirek has created :-) a system which can discover similar documents in our base of documents (students' homeworks, theses, etc.). Almost all computer-related media and some mainstream media has published the press release, or an article based on it. On Friday, even two TV stations and a radio station Radiožurnál did a coverage of this event. I have even played a student typing into a computer in the N@va TV news :-).
In the afternoon, I did an interview (or something like that) for the Czech TV about an Internet telephony (the journalist said "about Skyping", oh horror). I hope I have managed to pass the main message "do not use proprietary protocols which are not compatible with anyone" through. Altough I suppose they can also edit this footage so that I would look as a compelte fool. Let's see how it will look like. I am glad I managed to decline an offer for another interview for the Czech TV (I have gratuitously passed it to another person; I have yet to watch the result from their web archive :-).
Fri, 18 Aug 2006
Paper Work
I was fixing an UTF-8 related bug in our printing system - the bug report was from some person from the study department. The bugs was too hard to fix (it lead to these two bug reports, BTW), so after few days, they sent an urgency. After resolving the problem, I have re-read the urgency again:
They needed to print the list of students with their personal data, in order to give it to the account department people, who in turn would send payment requests to the students. Apparently they make a dead-tree version of the data, give it to the people in the next door, who then type the same data back to the computer.
When I wondered what was the reason of this unusual workflow, the answer was that it is needed for the inspection authorities. WTF?
Thu, 17 Aug 2006
Perl: The next level
We have ran into a problem that Data::Dumper
module escapes the UTF-8 characters as \x{codepoint}.
This is probably intended, as the result is then usable regardless of
whether the "use utf8" pragma is active or not. But it is not
much readable when the data contains lots of Czech text.
I have solved it by filtering the Data::Dumper output through
the following substitution:
s/\\x{[0-9a-f]+}/"\"$&\""/geexms
This has moved me to the next level of my Perl proficiency, as probably
for the first time I have used the "/ee" in a real-world code.
Wed, 16 Aug 2006
Swimming in Clothes
Last week Iva was attending swimming lessons. The last lesson was dedicated for swimming in clothes - so that children would not be so confused in case they fall into the water in clothes.
I wondered how would Iva react to the fact that she would be swimming in her pyjamas instead of the swimsuit. She was not surprised at all - she just asked why we did not bring the swimsuit, and that was all.
Tue, 15 Aug 2006
Clean Code versus undef
I like to write a clean, readable code. Sometimes even to the point that I am slow to write or fix the code when I think the current code is too ugly, but I have no immediate idea on what would be the best way to fix it cleanly. Recently I've came across an interesting dilemma, which is probably a clash between two measures of a clean code:
- Rule 1:
- The clean code produces no warnings. The
compiler usually knows about dangerous constructs of its language, and can
warn the programmer appropriately. When the programmer really
wants to do something weird, there is often a way to switch the warning off
(such as using two nested pairs of parentheses when an assignment is to be
used as a Boolean value inside the
if (...)statement). - Rule 2:
- The clean code is readable. And by readable, I also mean short. The longer the code is, the more the reader has to keep in a short-term memory in order to grok the code. Perl programmers usually call this principle DWIM - Do What I Mean. So the symbols should be named appropriately, the meaning of the return values should be obvious, etc.
There is an interesting problem in Perl with an undef value.
The Perl interpreter usually considers undef to be something
volatile which should be handled with an extreme care. So it warns of almost
every attempt to use an undef value. Yet undef
has a very clean mental interpretation: it is nothing. So it is natural
to expect it to behave as a neutral element of the operation in question:
The concatenation undef.$string should lead to
$string, length(undef) should be zero,
lc(undef) should remain undef, ++$new_var
should return 1 for a variable with undefined value, etc.
So I am in a situation when I have to put
if (defined $var) { ... }
around various parts of the code, in order to disable warnings
about an unitialized value. But the code is perfectly readable (and thus
clean by Rule 2) even without it. Adding more code makes the readability
by Rule 2 worse.
Moreover, even Perl itself is not consistent with handling undef:
The above ++$new_var construct follows the DWIM principle,
and produces no warning. But something like
print STDERR $log_prefix, $message, "\n"; produces a run-time
warning when $log_prefix is not defined. There are definitely
usages of undef where Perl should complain loudly (such as
calling a method of undef, calling undef as
a function, etc). But I think string concatenation or other string operations
(i.e. everything that can be solved by replacing undef by an
empty string) should not fall to this category.
So what do you think? Should Rule 1 have precedence here? And what about your language of choice? I know, LISP is unreadable by definition :-), but what about other languages?
Thu, 10 Aug 2006
Bluetooth File Transfer
Today I have spent some time trying to upload an image to Pavlína's phone - SonyEricsson T68i. I have succeeded, but it was not straightforward:
The positive side is, that on Fedora Core 5, the bluepin with D-Bus actually works, so it is possible to pair the devices inside the desktop environment, without writing the PIN to a config file. I have then tried gnokii to upload and download files, but without success. Gnokii supports T68i, it can identify it over Bluetooth (via a rfcomm link), but getting the list of files is not supported.
So I have tried OBEX file transfers. After some time I was able
to connect via obex_test, and receive an empty file
telecom/pb/0.vcf (from the examples found in the Web).
However, I still did not know how to send files, and under which names
(to which directories).
The next step was to use obex_push. I have misread the manpage,
and I thought the "channel" parameter is compulsory. But I did not know
the correct channel number. After a while, running obex_push
without any argument helped: it prints that it sends/receives files
on the channel #10. obex_push 10 phone:mac:addr /tmp/file.jpg
then did what I expected.
The phone did receive the image, but refused to display it. After some more googling, I have found the list of T68i supported image formats - the JPEG I sent to the phone was bigger than 640x480 pixels...
I have then moved on to explore the desktop features of Bluetooth.
gnome-bluetooth-manager can discover available devices
(one of my neighbors has a laptop and Nokia 6230i in visible mode ;-),
but did not do anything else, even when choosing "Properties" from the
menu. gnome-obex-send allows to choose the device to which
files are to be sent, but then unsuccessfully attempts to install an icon
and exits. Later I have found, that with
nautilus-sendto-bluetooth it is possible to send files with
Nautilus' right-click menu.
On the other hand, I managed to get the receiving side,
gnome-obex-server, working. It is just necessary to
set up the directory
to which files are to be uploaded.
So I think bluetooth file transfer more-or-less works, but the desktop part still needs to be improved.
Wed, 09 Aug 2006
Odysseus with Ext3
OK, Odysseus is fully restored now. I have also decided to re-create the main RAID-5 volume (with the chunk size increased from the default of 64k to 512k), and running the ext3 filesystem.
So far I think ext3 is not slower or more CPU intensive than XFS was, but we will see what happens during the upcoming release of Fedora Core 6.
According to iostat(8), the average request size on the
raw disks is slightly above 64k - it is bigger than the previous
chunk size, but nowhere near the present chunk size (even while running
a purely sequential load like cat FC3-x86_64-DVD.iso > /dev/null).
Dear lazyweb, which kernel parameter should I change?
I just wonder how long the e2fsck will take, because
even mke2fs ran for 2+ hour (while the RAID-5 volume was being
rebuilt, but still it is too long, I think). Maybe I should implement
a poor man's background fsck - once in three months or so
remount the FTP volume read-only, and run fsck on it.
Tue, 01 Aug 2006
Slow Writes
"There is one thing which is even slower than writing to a RAID-5 volume - it is writing to a RAID-5 volume, which is being rebuilt." -Yenya