Me, on Usenet, in 1998:

Hallo...
 
Nennt mich jetzt ruhig Verräter, aber weil ich leider in der Schule
gezwungen werde, CICS zu programmieren, wollte ich mal fragen ob da
vielleicht mal jemand die Sünde begangen hat, sowas auf Linux/Unix zu
portieren...
Weiss vielleicht jemand was, was mir weiterhelfen könnte? Netzsupport wäre
toll, aber ich wär auch mit einem Single User System zufrieden...
-- 
Thomas Themel
die_glacionauten@haidronic.edv.net
Rennsteinerstrasse 14a/1
A-9500 Villach
++434242210426 

The magic Internets just took a bit more than a decade to cough up the answer: zCobol. Can’t say I miss the days of EIBCALEN and DFHBMSCA, though.

So, my application doesn’t work. Somehow, a file I want to store stuff in always ends up empty. I dig through the code. I debug the code. I strace the code and look for other access to the file. It looks perfectly okay, I never touch the file again after a nice create/write/close without any errors. Standing by as the file is written, I see this:

[root@somewhere somewhere]# while true; do ls -l --time-style=full foo; sleep .5; done
ls: foo: No such file or directory
-rw-r--r--  1 nfsnobody nfsnobody 0 2009-07-24 13:54:36.117293000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 65 2009-07-24 13:54:39.544909000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 65 2009-07-24 13:54:39.544909000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 65 2009-07-24 13:54:39.544909000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 65 2009-07-24 13:54:39.544909000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 65 2009-07-24 13:54:39.544909000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 65 2009-07-24 13:54:39.544909000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 0 2009-07-24 13:54:39.544909000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 0 2009-07-24 13:54:39.544909000 +0200 foo
-rw-r--r--  1 nfsnobody nfsnobody 0 2009-07-24 13:54:39.544909000 +0200 foo

Since this happens even when I kill my application while the file is still in its non-empty state, I start getting suspicious about the underlying filesystem (as you’ve probably noticed by now, NFS). Now, look at this:

[root@somewhere somewhere]# echo "0123456789012345678901234567890123456789012345678901234567891234" > aFile
[root@somewhere somewhere]# cat aFile
[root@somewhere somewhere]# ls -l aFile
-rw-r--r--  1 nfsnobody nfsnobody  0 Jul 24 11:27 aFile

But…

[root@somewhere somewhere]# echo "012345678901234567890123456789012345678901234567890123456789123" > aFile
[root@somewhere somewhere]# ls -l aFile
-rw-r--r--  1 nfsnobody nfsnobody 64 Jul 24 11:27 aFile
[root@somewhere somewhere]# cat aFile
012345678901234567890123456789012345678901234567890123456789123
[root@somewhere somewhere]# ls -l aFile
-rw-r--r--  1 nfsnobody nfsnobody 64 Jul 24 11:27 aFile

Heh. So writing a file of 64 bytes or less works, while anything larger gets reset to zero sooner or later. Of course, that begs the question of what happens when appending to a 64 byte file…

[root@somewhere somewhere]# echo "012345678901234567890123456789012345678901234567890123456789123" > aFile
[root@somewhere somewhere]# cat aFile
012345678901234567890123456789012345678901234567890123456789123
[root@somewhere somewhere]# echo "X" >> aFile
[root@somehwere somewhere]# cat aFile
012345678901234567890123456789012345678901234567890123456789123

Lovely, isn’t it? At this point, I want to investigate this thing a bit more exhaustively and try to write a shell script in that fateful directory. When I’m done, it has of course grown over the magical 64 byte boundary, and vim warns me that

"test.sh" E667: Fsync failed
WARNING: Original file may be lost or damaged
don't quit the editor until the file is successfully written!

An error at last. So I start googling around for failed fsyncs on NFS, and it turns out that they like to crop up when the NFS developers break the kernel code. However, when I strace my vim to find the error, it finally dawns on me what the actual problem is:

fsync(4)                                = -1 EDQUOT (Disk quota exceeded)

Ah yes, quotas.

[root@somewhere somewhere]# quota nfsnobody
Disk quotas for user nfsnobody (uid 65534):
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
nfshome0:/vol/vol1/nfshome0
                2150400* 2048000 2150400             736  102400  122880

Sure enough, this weird behaviour turns up when the user is over its quota. Also, interestingly, the ability to even write the long files in the first place seems to be limited to bash – for example:

[root@somewhere somewhere]# dd if=/dev/zero of=foo bs=65 count=1
1+0 records in
1+0 records out
dd: closing output file `foo': Disk quota exceeded

And sure enough, an strace of this dd shows that it gets an error on close:

close(1)                                = -1 EDQUOT (Disk quota exceeded)

Amazingly, bash’s echo somehow seems to bypass this check, since the appropriate section of its strace reads:

open("foo", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
fcntl64(1, F_GETFD)                     = 0
fcntl64(1, F_DUPFD, 10)                 = 10
fcntl64(1, F_GETFD)                     = 0
fcntl64(10, F_SETFD, FD_CLOEXEC)        = 0
dup2(3, 1)                              = 1
close(3)                                = 0
fstat64(1, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
mmap2(NULL, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7ddb000
write(1, "01234567890123456789012345678901"..., 66) = 66
dup2(10, 1)                             = 1
fcntl64(10, F_GETFD)                    = 0x1 (flags FD_CLOEXEC)
close(10)                               = 0

Huh? I can’t seem to make sense of this. But anyway, I’ll go pester the NFS admins to increase nfsnobody’s quota a bit now and alter my code to check all the fstreams’ failbits again after closing them, then hope that my application doesn’t use whatever magic bash uses to bypass the error. You’re welcome to earn major geek points in the comments by explaining this to me.

M-x phases-of-moon

Of course with optional prefix argument for specifying year and month. Ha, TextPad losers – can your text editor help you find out that your first full moon was three days after your birth?

In the past, I haven’t given much thought to the problem of Internet censorship – lame attempts like the Chinese firewall and Germany’s censorship lists were always easily defeatable by anyone with a server outside their jurisdiction (or even a 37 second YouTube video, in the latter case). Apparently, the current Iranian censorship seems to be more serious in that they actually responded to this by (at least partially) blocking SSH traffic. This, now, annoys me. I need SSH to read my mail (and have unfetterd access to whatever parts of the public internets I want).

How does one get past this crap? My best guess would be that the usual HTTPS dodge still works – blocking HTTPS to “world” is going to break so many legitimate applications that you might as well just shut down your entire Internets, so “SSL to port 443″ is a pretty safe bet as a carrier protocol. Two issues arise:

1. If you have just a single IP address, wasting your port 443 for a rarely-needed redirect is a bit of a pity
2. The censors might discover that you’re not actually running HTTPS on this port and block it as well.

A simple solution relies on the lucky accident that the initial step of the SSH protocol calls for the client to wait for a server message, while the initial step of the HTTP protocol is for the server to wait for a client message. Thus, it’s rather trivial to write a simple redirector that initially sits there like an HTTPS server. When it receives a request (or anything, really), it forwards to the HTTP server. If this phase times out, it forwards to the SSH server, which will immediately send its server prompt. This means that it is totally transparent to HTTPS traffic and just slightly annoying (well, depending on your delay) on connect for SSH sessions. A proof of concept is here.

Of course, that still leaves the censors with the options of

1. Detecting it by waiting for the session timeout.
2. Figuring out what goes on from traffic flow.

Now, number 1 is not much of a technical problem, but I’d still think it hard to find a couple of hidden SSH servers amongst the millions of actual HTTPS out there. Bonus points for doing this on something where the legitimate HTTPS is also popular enough to cause problems when blocked. Number 2 is a threat for me, but it’s less bad if you just want to tunnel your web surfing through it, since it will arguably show quite similar traffic flow behaviour to the actual (direct) HTTPS.

(Inspired by this AskMetafilter thread)

I just tried to run anything.el on the ancient Emacs that comes with CERN Scientific Linux 4, and needed to bend it a little. In the process of looking for an online copy of with-selected-window, I found the delightfully absurd Antinews info page:

Appendix A Emacs 21 Antinews

For those users who live backwards in time, here is information about
downgrading to Emacs version 21.4.  We hope you will enjoy the greater
simplicity that results from the absence of many Emacs 22.1 features.
 

A.1 Old Lisp Features in Emacs 21

[...]

   * Many programming shortcuts have been deleted, to provide you with
     the enjoyment of "rolling your own."  The macros `while-no-input',
     `with-local-quit', and `with-selected-window', along with
     `dynamic-completion-table' and `lazy-completion-table' no longer
     exist.  Also, there are no built-in progress reporters; with
     Emacs, you can take progress for granted.

The most amazing thing about it is that it’s actually super-useful when backporting elisp code that wants to run on a current Emacs…

Another vignette from the jump-through-hoops-to-make-us-take-your-money circus: I wanted to read Arthur C. Clarke’s Superiority. My first search result was unhelpful – reviews, obituaries and quotes. But behold, on refinement, I stumbled on a link to Sony’s eBook store. USD 1.99? An eminently reasonable price! I get a legal copy, content creators (or rather, heirs of content creators) and publishers get paid, everyone is happy! Except for…

Want this eBook?Our eBook Library Software is required to purchase and download eBooks. Download it here.

Meh. Again. For a six page short story. Even opening a PDF feels like overkill on that format. Well sorry, I just had to add the quote that initially triggered my search to the Google terms and find the inevitable free fulltext PDF of dubious legality linked above.

Heise: Stadt Wien begräbt Glasfaserpläne:

Glasfaseranschlüsse für jeden Wiener Haushalt wollte die Gemeinde Wien ohne Subventionen realisieren. Vor drei Jahren hätte ein Pilotprojekt mit 50.000 Haushalten starten sollen, im Endausbau sollten alle 960.000 Haushalte und zirka 70.000 kleine und mittlere Unternehmen (KMU) mit 1 Gbit/s symmetrischer Bandbreite angebunden werden. Damit sollte Wien wieder zur Breitband-Hauptstadt der Welt werden. Aber daraus wird so schnell nichts werden, denn die Stadt hat die Firma Cablerunner Austria verkauft, die die Glasfasern im Kanalnetz der Stadt hätte verlegen sollen. Seit kurzem gehört Cablerunner zu 76 Prozent der Telekom Austria (TA). Die alternativen Provider sind empört und fürchten um ihre Investitionen.

2004 hatte die Stadt Wien den CableRunner vorgestellt: Eine Eigenentwicklung, die selbst in Rohren von nur 25 Zentimeter Durchmesser jene Kabel verlegen kann, in die dann Glasfasern eingeschossen werden. Der Providerband ISPA warnte damals “vor der Gefahr, mit Geldern der Stadt eine neue Monopol-Infrastruktur zu errichten.” Die Stadt etablierte eine eigene CableRunner Austria GmbH und erklärte es zum Ziel, alle am Wiener Kanalnetz angeschlossenen Grundstücke – das sind 99 Prozent – mit Glasfaser zu versorgen. CableRunner Austria erhielt die exklusiven Wegerechte im Kanalnetz.

Adorable. This has been brewing for some time, apparently. We’ll have to see whether this sails past BWB, but even the attempt is despicable enough. I’d love to hear an explanation of why it’s a great idea to create a municipally sponsored entity and basically give away an exclusive right-of-way to create a fiber network so that you can then sell it off to the largest owner of installed copper (and installed fiber). Let me guess – that is why we have a specially created private intermediate owner, who can take the blame in exchange for probably substantial profit.

Incidentally, this explains why nothing has ever been heard of Blizznet expanding its deployment over the last couple of years – I hope they now resume some kind of growth, though of course their standard deployment model means that they’ll probably never manage to hook up the house I live in due to its rather high density of broadband-agnostic older households I share it with…

So, I spent another evening of my precious life-time wrangling with Windows Vista. I wouldn’t even try to bore you to death with the story, but apparently this particular problem has never before bitten anyone who has taken it to the Interwebs, so I might save some poor soul later on from going through the same bullshit.

What happened? Two days ago, the tablet PC functionality on my ThinkPad X61 stopped working. In my Windows session, the pen simply would not do anything. Trying to calibrate the pen through the control panel would do nothing, the button would go grey and no calibration ever appear. Even funnier, Windows Journal would refuse to start – first doing nothing, then sometimes showing an error message

Windows Journal cannot start. There was an error initializing inking components.

I also noticed that the “busy time” before a freshly started Windows session actually became responsive had increased and noted that the mouse cursor went downright choppy for some time after startup. My eventvwr application log was littered with entries of the form

Faulting application WISPTIS.EXE, version 6.0.6001.18000, time stamp 0x47918ff4, faulting module WISPTIS.EXE, version 6.0.6001.18000, time stamp 0x47918ff4, exception code 0xc0000005, fault offset 0x000198d4, process id 0xed8, application start time 01c976a1e13e22eb.

(with a bunch of different fault offsets as well, but exactly one crash per reboot, at the end of the mouse choppiness period)

I first thought that this was some kind of bitchy reaction to the fact that I had incidentally used the pen in Linux for the first time in ages the day before to annotate a plot in GIMP, but after a lot of unsuccessful searching and installation/uninstallation/reinstallation of component drivers and updates, I found out that the problem was in fact caused by attaching a new external monitor the day before – for some reason, in Windows’ multi-monitor numbering scheme that you can see in the display settings dialog, my internal display had ended up as number 2, with the (unattached) external monitor as number 1. This didn’t bother the ordinary course of Windows, since the primary monitor is ostensibly not dependent on that numbering, but apparently something in the tablet layer is convinced that the pen has to be used on display number 1 – as soon as I reattached the monitor and jiggled the primary monitor selection long enough to get my internal display back to number 1 (and did the obligatory reboot), I had myself working tablet services again.

Well, I have high hopes for Windows 7…

My current ThinkPad came with an 80GB hard disk. Early this summer, this started to fill up dangerously under the load of a recovery partition, a Windows Vista, my Debian system and the CERN Scientific Linux I added. Favorable dollar exchange rate and technological progress had conspired to make storage cheap, and so I just got myself a 200GB replacement drive. Once this made its way to Switzerland, I just had to copy the contents of the old disk to a backup drive, plug in the new one and copy everything back. This actually worked and even yielded a bootable version of Vista.

Trouble started a bit later, when I wanted to resize the file systems on the new disk to actually take advantage of all the new space. The old layout had been a bit short-sighted in that it had used all four available primary partition slot for actual partitions, and so I needed to shift things around a bit. No problems for the Linux partitions, just some more copy/writeback (ext3 online resizing amazed me the first time I saw it in action). Amazingly, no problem for the NTFS volume either – ntfsclone even manages to clone volumes onto new partitions of different size, after which I can mount it fine enough, and everything seems to be there.

Of course, having the entire file system back doesn’t mean you can make it boot. Researching this, I stumbled across a program called ntfsreloc, which was supposed to fix the boot sector after moving a NTFS volume. Sadly, the only thing it did for me was corrupt the file system so that it wasn’t even mountable in Linux any more. No problem, I copied it from the backup again, but still no booting Windows.

The ThinkPad doesn’t have an optical drive, but I figured I’d just go out, buy a Vista CD (yay Studenten Software Service!), boot the installer over the network and use it to do ye olde fixboot and fixmbr from there.

After arriving home with my Vista CD, I was in for surprise number 1: For Vista, Microsoft has ditched the old almost-standards-based approach of supplying a PXE boot image on the install CD. Instead, it appears that one now needs a running Windows system to install something called “Windows Distribution Services” on in order to make PXE installations. Back to the drawing board, as they say.

I had noticed a USB CD drive sitting at a client’s office, so I took the ThinkPad and my Vista CD there. Booting from the CD gave me a nice Windowsy GUI repair thing that even recognized my NTFS volume as a Windows partition. A couple of clicks later, I had reached the “repair startup process” option in the installer, and was cheerfully told that Windows could find nothing wrong with my startup process. I cursed and resorted to messing with the command line tools detailed in KB927392 a bit, all the while shaking my head at the idea of replacing the venerable boot.ini with a 32k binary blob in, of all things, registry hive format. It took me a while to figure out that when bootrec /scanos or bootrec /rebuildbcd find 0 operating systems, this is actually a good thing because they only count those which are not already registered in the BCD (a fact that is documented for /scanos, but not for /rebuildbcd, AFAICT). After I had convinced myself that my BCD was in order, I still couldn’t boot, though.

I exhausted the options of bcedit and bootrec, I did a chkdsk /r, but the result was either just a blinking cursor when trying to chainload Windows from grub, or “A disk read error has occurred. Press Ctrl+Alt+Del to reboot.” when trying to boot the Microsoft-supplied MBR. More research was in order. I found the “disk read error” message in the boot sector, so I figured it wasn’t the MBR’s fault (this saved me a lot of time because every time I tried bootrec /fixmbr, I had to restore grub afterwards to get back into my Linux for further research). Via the ntfsreloc page, I found the link to an amazing resource – An Examination of the NTFS Boot Record, where someone has gone through the trouble of documenting every byte of the NTFS boot sector, including a disassembly of the boot code with lovingly detailed comment! Reading it made me wonder why one would want to put all this stuff into a boot sector – number of heads of the hard disk, sector offset of the volume from the start of disk? Stupid. Decoding my BPB, I found out that the number of heads of the hard drive was wrong (00F0 instead of the actually-faked-by-BIOS 00FF), and that the sector offset of the partition was set to some fantasy number (A41000 instead of 1DCDA5). I fixed that using LDE and was “rewarded” with a booting Vista.

On review, it turns out that the problem is not that bootrec /fixboot doesn’t KNOW about these strange values in the boot sector, it just seems to write wrong values there (confirmed by breaking the working boot sector with /fixboot). Armed with the knowledge of what ntfsreloc was supposed to do, I was also able to figure out why it had broken my file system before:

struct ntfs_geometry {
	unsigned short sectors;
	unsigned short heads;
	unsigned long start;
};
[...]
	const int geomsize = sizeof(struct ntfs_geometry);
[...]
	if (read(device, &fs_geom, geomsize) != geomsize) {

Apparently, I was the first idiot to try this code on an AMD64 system… Now excuse me as I wander off to savour the rich irony contained in the fact that the latest stage in Windows evolution has finally succeeded in sending me back to the kind of byte-editing drudgery that I first learned in DEBUG.EXE, a long, long time ago.

At the end of July, I had to leave my previous lodgings in Ornex and move to Genève proper, where I now inhabit an apartement on the fifth floor of la Cité Universitaire with two colleagues. The move caught us ill-prepared in a number of crucial ways, and so the first evening went rather depressingly – the kitchen proved to be spacious, but rather empty (Three spoons, three forks, three butter knives, three plates, three glasses, three cups. One pan, one pot.), and so the first dinner went badly (our only source of salt being a can of anchovis).

Much worse, it turns out that although there is wireless internet avaiable, the charges for short stays are rather steep – two weeks would have been 70 CHF for an unexplained amount of connectivity, and probably limited to one PC. The other available WLAN is provided by the university, but they use MAC filters and a HTTPS secured login form to keep out guests. While pondering this depressing state of things, we chanced upon a third source of network connectivity, named ‘((o)) ville de geneve’ – which turns out to be Genève’s municipal WLAN, available in select spots around the city. It seems that our location was high enough to get stray signal from a stadium a couple of hundred meters away. Sadly, the connection was weak and required users to sit in exactly the right spot in our kitchen’s window bay or lean out of the window with the laptop. To rectify this untenable situation, we semi-seriously acquired a can of Pringles on our Friday shopping trip, and I went to town today in search of a PCMCIA card with an external antenna plug. It was impossible to obtain such a thing, but while browsing available WLAN adapters, I got the much better idea to simply take a USB adapter and build it directly into the antenna… I hopped over to the local Starbucks and verified that of course there was nothing original to that idea and people had done it before all over the Internets, using materials like a strainer, wok lids or fire extinguishers.

I bought a D-Link DWL-G122 USB plug and brought it home. Initially, we hoped to get enough signal advantage from just hanging the USB plug out of the window by its cable, but apparently its antenna is so much smaller that it compensated for the better reception. Some interesting results from that: reflection DOES make a difference – holding the adapter in front of a closed metal shutter or mounting it in our cooking pan definitely increased the signal strength. None of these methods were really practicable, though, so we decided to go the classical Yagi route. Eating the Pringles was pretty hard, but the results are worth it… We now have comfortable signal strength and the network is reexported around the entire apartment via the internal WLAN card in the attached computer, and the setup is even quite stable thanks to the fact that one of my roommates lent his tripod to our noble enterprise.

(Yes, the USB adapter and the Pringles cost just as much as just buying the Internet access would have, but would that have been as much fun?)

Update: On revisiting Edificom’s pricing information, I see that we actually managed to save some money (70 vs 115 CHF) AND gained the fun. Win!