Thursday, November 16th 2006, another normal day for me. OK, that's wrong :) Like any other IT folks nowadays, I spent some part of my free time to browse the Internet. Guess what I found? What? Attending the conference? No no, that just made me feel sleepy and awful. Think again. Yup, I decided to enter the Linux trouble-shooting competition. First thing that came inside my brain was "what the hell I shall face there? A God damn broken ftp server? Compromised host with some sort of 3133t r00k1ts installed there?Awfully broken root filesystem? Or what?" Yeah, that sounds scary, but anything could be the challenge I shall face, so I must be ready for anything. What will I lose anyway? Nothing... except my 50,000.00 rupiahs will fly away :)) hehehehe.
But shit, the end of the registration was near. F**k! So, I picked up my cell phone and dialled the CP (contact person) number. Pheww, the man on the phone said, anybody was still allowed to register 'til the competition was about to begin (that was November 18th, Saturday...). Yeeha. But oh my... registered as a team? Hm, I couldn't ask somebody to join me, so I simply asked again "ehm, I plan to register as a team composed of only one member, is it OK?". Quite funny part here, since I got confusing answer at first. But eventually I got another positive answer again.. Pheww, so the only thing I need to worry is the competition itself.
Saturday, got up earlier, cleaned up myself, got some food and I went to Surabaya. No traffic jam, thank God for that! Arrived at Hi-Tech Mall (a.k.a THR -- Taman Hiburan Remaja), wait for couple minutes, went to 2nd floor and I registered myself. After me, I saw many people coming and did on site registration. And I thought "...surely lack of publication....", but that was just my guess.
Now the hard part... show time baby! :)
In the schedule, the competition should began at 10 AM, but well, that was the theory :) The fact was, it began close to 11 AM :) Surely pumping up your adrenalin? No, quite the opposite, I began to feel sleepy :)))) bwahahahahha. I used this spare time to look around and try to "profile" the other competitors. Holy cow! I was pretty damn sure I was the oldest competitor :) Gee, time flies so fast isn't it? Once you were 17, and now you're suddenly near 28 and your brain is working slower ... shit on me :D
First round. Me and the other 9 time must solved the boot problem. The sympton was quite simply but could be tricky if you didn't understand the real issue. After an item in GRUB was selected, the kernel was booted (NB: the distro used here was Blank On, a localized and modified version of Fedora Core). After some hardware detection, suddenly it went to run-level 6. Did you notice the quirk here? yeap, certainly something was forcing the init stage to go to runlevel 6. It could be the kernel's boot param, or... /etc/inittab. So, I inspected the related GRUB's entry. Clear, nothing was wrong there. Then how to go to the normal runlevel without interrupted by the inittab? Simply provide your own runlevel number as the kernel boot param, e.g linux 3 root=/dev/hda2 and you're done! But since I must made it permanent, I told the kernel to enter single mode (passing "S" as boot parameter), edit /etc/inittab and made 5 as initial run level. Save the file, reboot and voila, I enter the GDM :)
taking a break for about an hour or so, I entered the 2nd round. In this round, 10 from about total 20 team was selected since they solved the first round and wrote the explanation (problems identification, solution) pretty clearly. Quite good judging style, I did like it. So back to the arena. Now what? Sounds simple, X couldn't be started! All I got was a blank shitty screen when I executed "startx". And thanks to this lame head, I forgot where the hell X saved its log! For once, I thought strace could be the life saver here, but .. great... no strace! So it's kinda back to stone age where the tool was just the rock (got my point?)
My mind quickly concluded it could be some sort of permission problem. So I switch to another user ID. What did I got? A message telling me that I had no permission to start X and big chance it was PAM problem (according to the on screen message). So I tried to follow the hint and inspected the PAM configurations. Coming inside xserver PAM setting, changing here and there, no luck. Bummer! What did I wrong here... ??
Almost losing hope here, while one team was announced as the first team correctly solve the problem. Hm, quite fast, so it must be something easy. Minutes later, the announcer told a quite useful hint : "the problem is something related to mouse device!" OK, X and mouse, what could be the link here? Of course, Xorg.conf (small x? big X? I forgot..)! Before that, thanks to "find", I finally found the X log...and yep, it was said that core pointer couldn't be initialized. What's the problem? Could it be the mouse wasn't detected? dmesg confirmed that the PS/2 mouse was indeed correctly detected, so that wasn't the problem. Firing up vi to inspect Xorg.conf, I looked up any directives that could lead to mouse settings. OK, I found the device name to be used by X as the pointer..."/dev/input/mouse". Valid entry? Not really, ls confirmed that device name was wrong because of missing the "0" suffix. The correct one was /dev/input/mouse0. Typing that, saved, fired up startx again and I was done! :)
Third round (final one) was started at 4 PM...oh man, I told you, I was so tired and sweating... and I thought to myself (not a song :) ) "this is it, exhausted...can I win?". But giving up isn't my style... the only style I have is "keep moving!". So that's what I did. This time, 5 team entered the final... and they were all looked skillful, so I planned to give my best here. And the game was started again, this time...three troubles...all at once! F**k!
First, unable to find certain file when GRUB tried to load the selected kernel. What was it? Quick inspection in grub entries revealed it was incorrect initrd's filename. Fixing that and I met another problem... "no users exist" (something like that, I don't remember 100%). Why? No ideas.... but I reset my PC and back to inspect the GRUB entries. I noticed that it passed /dev/hda5 as the root filesystem. Wrong one? Once again no idea, but I guessed it could be the problem. So I changed it to point to another valid one and I used /dev/hda8 since this rootfs was also used by another kernel's entry. OK, booting went smoothly this time. X was also fixed quite similar like round #2. Confidently I raised my hand to let the observers knew I did it. The judging comittee checked my solution and one of them said "oh, you can't change the root filesystem, please change it back". Great.... :(
At almost the same time, another time raise their hands and the judges declared that they did it correctly. Oh my, I lost my chance as the #1 winner. But I won't give up so easily. No change to rootfs? OK, I can do that too. Rolling back the boot loader's settings to the original one, this time I got the "no user" problem again. I was almost sure the root ID wasn't there, so I used another kernel entry to boot the Linux system and inspect /dev/hda5. Got it, no root! So what I did was simply creating it. And since I am lazy, I did :
# grep root /etc/passwd | head -1 >> /mnt/test/etc/passwd
/dev/hda5 was mounted in /mnt/test and I was in the healthy /dev/hda8. Once again, reboot. This time, using runlevel 3, I successfully entered the login prompt. Login and I tried startx. Mouse problem again? Nope... "no screens available". Sounds like driver problem? I checked the Xorg.conf and found the bug, wrong driver! It was written "Sis" as the display driver while obviosly this machine was using Radeon graphic card (I didn't confirmed it using lspci, I purely used instinct since the other machine was using radeon and it worked). While I hacked it, the other team was declared as the 2nd successful solver. OK, I must secure 3rd place then. Finished with Xorg.conf, starting X and it worked! Again, I raised my hand......
Clock ticking, the judges inspected my PC once again... typing here and there, they were trying to confirm something (correct partition? not sure..). But I was pleased to hear they concluded I made correct solution.
Waiting another 2-3 minutes, the official winners were announced. And yep, I got the 3rd place. I was still proud to myself, knowing that I was still quite fast solving the final problem. And the bonus was nice too, Rp 500,000.00 ... not a small cash to carry, right? :)
So, boys and gals, there you go...the story of a man competing for a pride and honour. And I must say, I do like this kind of competition. Not only because it offers competitive atmosphere and quite nice prize, but also because the participant must *solve* a real world problem. Yes, I said it was real world because I met all those problems previously. Compared to the hacking competition, I found this troubleshooting competition is far more educative, challenging and promoting Linux usage in general. Kudos for the organizer who did the quite good job. I hope there will be another similar competition like this and I shall enter it once again. hopefully facing far more mind-juggling problems to solve.
No comments:
Post a Comment