A place where I share my daily experience in both technical and non technical issues. Expect to read Linux kernel related posts too.
20 December 2009
Hopefully these online test results are credible enough to say something about me....
One small question: are odesk.com online tests credible enough to show whether somebody is competent or not in his/her field?
12 December 2009
Contributing to laptop mode tools
As confirmed by laptop mode tools' changelog, my patch is finally merged into their core code. I simply contribute few line to show statistics regarding write and read frequency of every programs recorded during lm-profiler's runtime.
So, how the code looks like?
—- lm-profiler.old2009-06-03 21:12:52.000000000 +0700
+++ lm-profiler2009-06-26 21:08:54.000000000 +0700
@ -209,6 +209,11
@
done
printf ’\r \r’
stop_profiling
+
+echo “Write frequency : ”; cat $WORKDIR/write_accesses_* | sed -e ’s/[ \t]*//;s/[ \t]*$//’ -e ’/$/ d’ | sort | uniq -c | sort -n
+echo “Read frequency : ”; cat $WORKDIR/read_accesses_* | sed -e ’s/[ \t]*//;s/[ \t]*$//’ -e ’/$/ d’ | sort | uniq -c | sort -n
+echo;
+
NETPROFILE=`profilenet`
echo “Profiling run completed.”
Looks awful? I admit it. In short, it grabs files created by lm-profiler during its run time, trims out the blanks and then sort them, while at the same time showing their frequencies ascendingly.NB: Actually it's for my own reminder, but to share with you all. To easily convert text to HTML (and dealing with all those escape characters etc), you could use http://www.textism.com/tools/textile/index.php. Simply paste your text there and click the button, voila..you get the HTML-ized text!
regards,
Mulyadi
25 November 2009
Why so serious? :D
Got this quote from kernelnewbies mailing list....hehehehheheheh :D
"What happens when you read some doc and either it doesn't answer your question or is demonstrably wrong? In Linux, you say "Linux sucks" and go read the code. In Windows/Oracle/etc you say "Windows sucks" and start banging your head against the wall."
-Denis Vlasenko on lkml
PS: s/banging your head*wall/take it for granted/g is better I guess :D I believe many Linux users also bang his/her head when learning Linux for the first time. :))
Moral: just don't bang your head too hard, it hurts, you know? :D
09 November 2009
how to catch white space(s) using grep?
My definition: White space characters are anything that appears as "blank" a.k.a nothing in screen. They include tab, space, carriage return and so on.
As you know, grep provides a way to catch certain characters class or range. Specifically for white spaces, you can use [[:space:]] or [[:blank:]]. Notice the double [[ and ]] !!!
So, suppose you have text file named test.txt that contains:
hehe /var/www/
hehe /var/www2/
ttt hehe /var/www3/
heho /var/www/
Executing:
$ grep ^hehe test.txt
will yield:
hehe /var/www2/
but this:
$ grep -E '^[[:blank:]]*hehe' test.txt
yields:
hehe /var/www/
hehe /var/www2/
In human words, '[[:blank:]]*' will catch zero or more appearance of space or tab before the word "hehe". If you want to catch at least single appearance of any of them, use "+" instead. Oh and let me remind you again, use -E so that "+" doesn't lose its special meaning.
Note: initially, i thought i simply use [:space:] or [:blank:] and end up in something-is-wrong-but-I-dont-know-what land. Turns out, I didn't read the man page carefully (poor me). Since they are built-in classes, I still need to enclose them with another "[" and "]". Valuable experience.....
regards,
Mulyadi.
15 October 2009
Misunderstanding of rate limit concept in iptables
"limit
This module matches at a limited rate using a token bucket filter. A
rule using this extension will match until this limit is reached
(unless the ‘!’ flag is used). It can be used in combination with the
LOG target to give limited logging, for example."
OK, can't wait to get my hand dirty on it. I fired up my VM (Virtual Machine) guest and type this in guest's console:
# iptables -A INPUT -i eth0 -m limit -m icmp --icmp-type echo-request --limit 1/min -j RETURN
To avoid confusion, I assume the default policy of INPUT chain is ACCEPT. Further, there is no other rule in INPUT chain other than what I typed above.
What is the above rule supposed to do? My understanding, at that point, that it will rate limit the ICMP echo request packets up to 1 packet per minute. Thus, only 1 packet during 1 minute interval will be processed. Further packets will be queued in memory awaiting to be processed. My intention to try this feature is simply to find idea to prevent DDoS, but my "assumption of queueing" made me think that this is not really safe. If there 1 million packets waiting to be processed, eventually your machine's memory will be exhausted, no?
But OK, let's put aside that fearness. I flood ping my VM guest (ping -f, if you don't know how to do it. You have to be root to do this). But guess what? 100% of all ICMP packets are responded really fast!!! What's wrong?
Then I did various test. Replace RETURN target with DROP, not using -i, not specificly rate limiting echo-request, etc. Nothing works! tcpdump still showed me that there were lots of echo request - echo reply packets flowing back and forth between my host and my VM guest.
I almost concluded that "limit" was not working as I thought. Perhaps this is a job of iproute tools, something like we do to rate limit packets using CBQ, HTB etc.
But then I smell something fishy:
# iptables -L -n -v | head
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 RETURN icmp -- * * 0.0.0.0/0 0.0.0.0/0
limit: avg 1/min burst 5 icmp type 8
Notice the "pkts" field? It's a counter that denotes how many packets are entering certain rule. Also notice there is global counter displayed in INPUT chain (inside the bracket).
So, what's special with them? When I repeat my flood ping test, I saw both the limit rule and the global INPUT counter increased! Thus, something is wrong in my assumption. If indeed the packets were successfully rate limited, at least ACCEPT counter won't be increased as fast as rate limit counter grew.
The answer? Back on manual page. Looks like my English skill was really tested this time. "A rule using this extension will match until this limit is reached". Uhuh...I see...
Confused? Let me explain it as simple as I can:
Assume you use 1 packet per second as limit. During the first minute interval, if a packet arrives, it will hit INPUT chain and checked against rate limit rule. Does it match? Of course! It is still not beyond our limit, right? How about the 2nd, 3rd, 4th and the 5th? They will match too. Why? Because by default, there is burst limit. It will allow several initial packets to get a match, but not all. The default is 5.
What about the rest? For sure, they won't match our limit rule. Again, why? because the limit has been reached (as stated by manual page), thus the limit rule is passed and netfilter will check the next rule. And since there is no more rule in our scenario and the default is to accept in INPUT chain, then all ICMP request packets are accepted and replied!
Solution? Simple. Since the excessive packets will pass our limit rule, then we need to block them right at the next rule e.g:
# iptables -A INPUT -p icmp -m icmp --icmp-type echo-request -j DROP
Voila! We successfully rate limit the ICMP! Woohoo! Case closed.... phewwww
Lesson taken: do not underestimate manual page. Read it very very carefully and make sure you understand every word in it. Misunderstanding of the meaning even a single word could lead to significant difference between successful or stressful trial-and-error implementation. You've been warned...
regards,
Mulyadi
01 October 2009
A little patch that made into main Linux kernel git repository
Commit-ID: 1ad0560e8cdb6d5b381220dc2da187691b5ce124
Gitweb: http://git.kernel.org/tip/1ad0560e8cdb6d5b381220dc2da187691b5ce124
Author: Mulyadi Santosa <mulyadi.santosa@gmail.com>
AuthorDate: Sat, 26 Sep 2009 02:01:41 +0700
Committer: Ingo Molnar <mingo@elte.hu>
CommitDate: Thu, 1 Oct 2009 10:12:03 +0200
perf tools: Run generate-cmdlist.sh properly
Right now generate-cmdlist.sh is not executable, so we
should call it as an argument ".".
This fixes cases where due to different umask defaults
the generate-cmdlist.sh script is not executable in
a kernel tree checkout.
Signed-off-by: Mulyadi Santosa <mulyadi.santosa@gmail.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <f284c33d0909251201w422e9687x8cd3a784e85adf7d@mail.gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
tools/perf/Makefile | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index b5f1953..5881943 100644
--- a/tools/perf/Makefile
+++ b/tools/perf/Makefile
@@ -728,7 +728,7 @@ $(BUILT_INS): perf$X
common-cmds.h: util/generate-cmdlist.sh command-list.txt
common-cmds.h: $(wildcard Documentation/perf-*.txt)
- $(QUIET_GEN)util/generate-cmdlist.sh > $@+ && mv $@+ $@
+ $(QUIET_GEN). util/generate-cmdlist.sh > $@+ && mv $@+ $@
$(patsubst %.sh,%,$(SCRIPT_SH)) : % : %.sh
$(QUIET_GEN)$(RM) $@ $@+ && \
15 July 2009
Langkah pertamaku di cyberspace.....
pagi ini buka web detik.com, ternyata ada pengumuman lomba nge-blog. So, kenapa tidak saya coba? Itung-itung berbagi pengalaman dengan sesama netters.
Pertama kali ngenet sekitar tahun 1996. Waktu itu yang namanya internet bagai barang mewah bagi masyarakat biasa seperti saya. Jumlah ISP (Internet service provider) juga masih bisa dihitung dengan jari, itupun terpusat di Jakarta atau Surabaya. Kecepatan koneksi? Hehehehe, seingat saya waktu itu modem kebanyakan masih 9600 atau 14400 bps :D Bayangkan saja seberapa lambatnya jika koneksi seperti itu sekarang digunakan untuk ngebrowse web dengan content canggih semacam flash dan berisi beberapa banner plus teknologi AJAX. Bisa-bisa kelenger :)
Jadi kesan pertama waktu itu, berinternet itu menyenangkan tapi juga butuh kesabaran. Ya sabar karena koneksinya sendiri lambat, juga kalau line teleponnya putus. Whahahha, ini yang kadang bikin dongkol. Enak-enak download, telepon masuk, putuslah sudah koneksi. Waktu itu program semacam GetRight belum saya ketahui, jadi ya udah....ulang lagi.
Oh iya lupa, sebenernya saya ini sedikit "teracuni" kakak sepupu saya. Seingat saya dulu punya bisnis BBS (Bulletin Board System) di awal tahun 1990-an. Jangan tanya saya apa itu BBS, yang saya tahu, waktu itu saya diperkenalkan ama semacam layar chatting. Saya sendiri waktu itu coba-coba chat dengan kenalan kakak sepupu. Dari situ saya mikir "wah, hebat juga BBS. mungkin suatu saat gak cuma tulisan, tapi juga suara dan video bisa lewat koneksi semacam ini". Sekarang kita lihat ini sudah jadi kenyataan. VoIP sudah mulai merakyat, video conference berbasis Internet juga banyak dipakai baik oleh perusahaan, pemerintah dan personal.
Begitu lihat layar browser, apa yang dicari? Untungnya search engine seperti Yahoo sudah exist, jadi yang dicari website soal game. Loh? Maklum pecandu game (bahkan sampai sekarang). Nyari move list karakter Mortal Kombat contohnya :)) Ya gimana lagi, waktu itu informasi semacam ini adanya di majalah game luar negeri, tapi gak kebeli. Jadi yang rada murah, ngenet dompleng fasilitas kampus. Tapi yang namanya fasilitas umum, antre nya luar biasa, hehehheheh. Jangan heran kalau kadang yang keluar cuma title halaman terus time out. Jadi ya retry lagi retry lagi.
Kalau soal chat, so pasti dicoba. Seingat saya waktu itu yang populer duluan IRC, baru Yahoo messenger. Apa hayo yang terkenal di IRC? Yup, channel #bawel di DALnet!!! :) Byeuhhhh, tuh channel bener-bener isinya orang yang kayaknya calon MC. Ada aja yang "diomelin". Belum lagi iklan dan quiz yang hilir mudik. Yang mojok? Jangan tanya :)))) Pasti ada, tapi soal ketemunya beda lawan jenis atau ternyata kena yang aspal (asli tapi palsu) ya mana tahu, wong namanya juga dunia maya
OK guys, that's my personal share today. Wanna share yours?
20 June 2009
How to reduce CPU soft lock up in KVM guest
Ever saw something like below messages inside your KVM (Kernel Virtual Machine) guest's console?
" BUG: soft lockup - CPU#0 stuck for 10s! [swapper:0] "
I did and I find it a bit annoying. If you're inside graphical desktop like environment like KDE or GNOME, you probably won't notice it directly. But you will likely suffer the same condition, the guest OS somehow become unresponsive for a few moment. In my case, it manifest into stalled CD/DVD access and "ruins" the console display. I had to press Enter few times before I could get back to normal shell prompt. Before I go further, FYI I use Fedora 9, kernel version 2.6.27.23-xx.x.xx.fc9.i686 on a Core Duo powered laptop.
First, why the kernel shows such message? I use the default CentOS 5.3 kernel, so I check the related kernel config inside /boot directory and here is the related configuration item:
CONFIG_DETECT_SOFTLOCKUP=y
What does it do? Ingo Molnar, the writer of this lockup detection patch describe it as:
"From: Ingo Molnar
This patch adds a new kernel debug feature: CONFIG_DETECT_SOFTLOCKUP.
When enabled then per-CPU watchdog threads are started, which try to run once per second. If they get delayed for more than 10 seconds then a callback from the timer interrupt detects this condition and prints out a warning message and a stack dump (once per lockup incident). The feature is otherwise non-intrusive, it doesnt try to unlock the box in any way, it only gets the debug info out, automatically, and on all CPUs affected by the lockup.
Tested this on x86, both with the feature enabled (in which case a provoked lockup was correctly detected) and with the feature disabled. It is CPU-hotplug aware. Should work on every architecture. "
Pay attention here that the watchdog mentioned in the above description has nothing to do with NMI (Non Maskable Interrupt) watchdog. NMI watchdog deal with hard CPU lockup, while the above mentioned lockup watchdog can't. It's just a kernel thread that will stuck if CPU hangs.
I suspect it might be a bug in KVM driver (or specificly, KVM for Intel VT in my case). I came to this hypothesis because the help section of soft lockup patch says:
"Say Y here to enable the kernel to detect "soft lockups", which are bugs that cause the kernel to loop in kernel mode for more than 10 seconds, without giving other tasks a chance to run."
Great...so IMO KVM is too busy on something, or.... something is delaying KVM guest to run. I almost rush to compile my own kernel image using full preemption, hoping that it could squash the problem. But I was tempted to Google a bit more. Interesting result, a post in a mailing list (I forgot which one) suggest to set the CPU frequency into static. Let's try, I edited /etc/sysconfig/cpuspeed so the related lines become:
MAX_SPEED=1333000
MIN_SPEED=1333000
I pick that frequency because it's the middle number between the three available frequencies: 1833000, 1333000 and 1000000 Hz. So theoritically I still get adequate computing power to most job without draining the battery too soon.
Execute:
# service cpuspeed restart
Make sure it's applied correctly:
# grep '1333000' -r /sys/devices/system/cpu/
/sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq:1333000
/sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq:1333000
...
/sys/devices/system/cpu/cpu1/cpufreq/scaling_min_freq:1333000
/sys/devices/system/cpu/cpu1/cpufreq/scaling_max_freq:1333000
...
Then I ran my KVM guest again. I did few tasks in it, let it went idle, repeat and so on. During my test for about an hour, the result was promising! It became stable. Well, few lock up happened though, but it was far reduced. Thing that I notice is lockup also happens when I switch into another virtual desktop or if fairly heavy swapping in/out is on the way. So, to further reduce lock-ups, I avoid switching to another virtual desktop (ok, that sucks, but I could live with that) and close any unneccesary applications to conserve virtual memory as much as I can.
Why it works? All I can say is by making the frequency static, you also avoid timer interrupt delivery frequency being changed too. It stabilize the kernel timing and also indirectly stabilize the KVM guest timing. Previously, I was using ondemand power governor and as you might be aware of, it adapts the CPU frequency according to the load quite aggresively. So, frequency was juggling between all three available frequencies. Conservative governor didn't lend a help here. Pity, I assume KVM still force conservative governor to switch to highest frequency then drop most of the time because most of the codes could be run natively instead of being translated.
regards,
Mulyadi.
24 May 2009
Double title = "you must read!" :)
Ok, don't worry, we'll read it ASAP :) It was a post in detik.com on May 5th 2009, discussing about police investigation on a murder of a national company's director. It drew so much people's attention during mid April-May, since police has indications that Antasari Azhar, the head of Corruption Eradication Commission, is the mastermind.
Let's see what will the police's conclusion be ....
Wiranto as presidential candidate?
Notice the first sentence? It is written "Presidential candidate Wiranto" and so on. Hehehe, so he swaps position with Jusuf Kalla? :) The complete post is here for clarification.
09 March 2009
kernel panic when calling printk() inside scheduler code...
My analysis, somehow during module initialization, printk() must not be executed to avoid recursively waking up klogd. The original poster said it didn't help. Finally, he found a workaround, that is by incrementing a variable named oops_in_progress. I bet works like a flag. When it's greater than zero, it prevents klogd to wake up. Problem solved!
Note: klogd is a kernel thread that reads kernel ring buffer and send it to user space daemon syslogd if there's any entries there. Those are the messages that we see ending up in /var/log/messages (for non critical kernel messages).
regards,
Mulyadi.
08 March 2009
the way we respond to a question
But when I looked at this blog post, I am a bit intrigued. For those who don't know who that guy is, he's Kir Kolyshkin. One of the main developer of OpenVZ, a container solution for Linux. Container is a kind of virtualization which is done in OS level. System calls are intercepted so every guest thinks he owns the whole system. But unlike User Mode Linux, OpenVZ is not using ptrace, but guest kernel is patched so system call is now routed to a "stub" in host kernel. So certainly, Kir is not average developer and far from mediocre level....
This paragraph is interesting:
"I think the talk was well received and I had about 10 different interesting questions, one is puzzling enough so I was not able to provide a good answer. This is definitely a sign of a good audience."
Most of the time, especially when some (doubtly) educated people receive tough question, he will turn into defending mode or attack the questioner back. But Kir thinks differently. He admitted he can not answer all questions, a normal thing for normal human being. And he appreciated that.
I wish this kind of attitute is something we can learn and practice in our everyday life.....
regards,
Mulyadi.
21 February 2009
simple function intercept in Linux
Without further ado, here's the link:
http://ezine.echo.or.id/ezine20/e20_0x05.txt
I welcome your feedback and comments...regards,
Mulyadi.
18 January 2009
First time I boot my UML guest, I got eth0, then eth1..and so on, why?
Previously, I absolutely had no idea why it happened. But lately I was thinking it could be something with udev (after all, it is the one which deals with device file, right?).
Let's boot the UML guest one more time and find out what we could dig. Note that I use the following parameter when I invoke the guest:
eth0=tuntap,tap0
Guest boots up and login prompt shows up. udev should put its configuration under /etc/udev or something alike, so let's dive. Found it, it's in /etc/udev/rules.d/75-network-devices.rules. Here is the snippet that explains my headache:
KERNEL=="eth?", ATTR{address}=="4e:82:25:ae:b1:bb", NAME="eth0"
I see the correlation now. If I don't specificly declare the MAC address of the ethernet device my UML guest will use, UML will pick one randomly. And udev inside the guest will think it's a different ethernet device.
Finally, simple solution. Put that MAC address as part of "eth" parameter:
eth0=tuntap,tap0,4e:82:25:ae:b1:bb
Now I get consistent naming!
PS: Looks like my "research" insting is back. Hopefully, it will stay inside me for long time....
regards,
Mulyadi.
How to execute multiple commands directly as ssh argument?
Perhaps sometimes you need to do this: ssh user@10.1.2.3 ls It is easy understand the above: run ls after getting into 10.1.2.3 via ssh. Pi...
-
Ever saw something like below messages inside your KVM (Kernel Virtual Machine) guest's console? " BUG: soft lockup - CPU#0 stuck f...
-
Quick summary first: use gcc -save-temps ! Ever dig into Qemu (qemu.org) source code? OK, I assume you ever did that at least once... may ...
-
Dear readers This time I met interesting case, thus I think it is worth sharing it with you all. But first, big disclaimer. I AM NOT FREE ...