Sam’s Network Simulation Cradle Blog

15 Mar 2008

Fun with subversion repository history: a visualisation

Filed under: General — sammydre @ 8:34 pm

Some time ago I hacked up a little script to view my subversion repository version over time. I lost the initial script but I’ve recreated it today. Thought it was interesting to look at as the history dates back to 2003. Creating this is very easy with the help of svn log.

Bash:
  1. svn log svn+ssh://host/repository/ | grep -E "^r[0-9]+" | awk ‘{ print $5, $1 }’ | sed ’s/r//’ > repo.txt

Produces a file like:

2008-03-12 1681
2008-03-12 1680
2008-03-11 1679
... etc ...

Looks good for graphing with a program like gnuplot:

set xdata time
# The format of time found in the file
set timefmt "%Y-%m-%d"

set term png
set output "repository.png"

set xlabel "Date"
set ylabel "Repository revision"
set title "Repository version over time"

# The time format actually shown on the x-axis
set format x "%Y"

# It's tricky to get the tics right in time mode, as you have to specify increments in seconds.
# I find it easier to just specify what dates I want shown
set xtics ("2003-1-1", "2004-1-1", "2005-1-1", "2006-1-1", "2007-1-1", "2008-1-1")
set nokey

plot "repository.txt" using 1:2 with lines

And this produces the following result:

Sam’s research repository version over time

The hickup in the graph for me is due to an import with cvs2svn. Looks like the dates don’t quite increase as one would expect (this can be seen in the svn log output, too). In general the approach appears to work well enough, though.

Upgrading WordPress with SVN

Filed under: General — sammydre @ 5:51 am

One of the recommended ways to upgrade WordPress these days is with SVN. This is a convinient way of updating that is a lot quicker than the previous method. One of the problems with WordPress I’ve found over 4 years of running the software is that it has required many updates — I’ve almost never wanted any additional features, but there always seem to be new security holes. So a quicker way of updating is nice.

One of the things about updating with SVN, which does not appear to be discussed, is that it leaves a plethora of “.svn” directories on your webserver. Stored in these directories is the subversion source control info and backups of the working copy. By default, if you update with SVN, any user could browse the exact code that is running your blog. This sounds like a bad idea to me. While Wordpress is already open source, I don’t see any need to allow potential users/crackers/hackers/whatnot to browse through the exact code I’m running here. .htaccess to the rescue.

You could configure your web servers config to deny access to any directory with “.svn” in it.

If you are only a user on a webserver with .htaccess, it gets a bit more tricky. You could put a .htaccess in each .svn directory. Easy with a script, but tough to maintain. Or you could use mod_rewrite to make a simple rule that denies access to any pages with /.svn in the path:

RewriteEngine on
RewriteRule ^(.*\.svn.*)$ $1 [forbidden]

23 Apr 2007

Silly Perl thoughts…

Filed under: General — sammydre @ 8:44 pm

A simple script to show a table of contents like view of a latex file. I could make a couple of assumptions about the input based on my setup. My first attempt which is probably somewhere on my blog looked like this:

Perl:
  1. if($#ARGV > 0) { $in_file_name = shift; }
  2. else { $in_file_name = "thesis.tex"; }
  3.  
  4. sub read_file
  5. {        
  6.     my $file_name = shift;
  7.     local *IN;
  8.     open IN, "<$file_name";
  9.  
  10.     while(<IN>) {
  11.         if(/^\\chapter.?{(.+?)}/) {print "——————–\n$1\n";}
  12.         elsif(/^\\section.?{(.+?)}/) {print " . . $1\n";}
  13.         elsif(/^\\subsection.?{(.+?)}/) {print " . . . . $1\n";}
  14.         elsif(/^\\subsubsection.?{(.+?)}/) {print " . . . . . . $1\n";}
  15.         elsif(/^\\paragraph.?{(.+?)}/) {print " . . . . . . . . $1\n";}
  16.         elsif(/^\\input{(.+?)}/) {read_file($1);}
  17.     }
  18.  
  19.     close IN;
  20. }  
  21.  
  22. read_file($in_file_name);

A classic recursive solution using a function. I had a quick play today and produced this:

Perl:
  1. #!perl -n
  2.  
  3. /^\\chapter.?{(.+?)}/       and print "-" x20 . "\n$1\n" or
  4. /^\\section.?{(.+?)}/       and print " ." x2 . " $1\n" or
  5. /^\\subsection.?{(.+?)}/    and print " ." x4 . " $1\n" or
  6. /^\\subsubsection.?{(.+?)}/ and print " ." x6 . " $1\n" or
  7. /^\\paragraph.?{(.+?)}/     and print " ." x8 . " $1\n" or
  8. /^\\input{(.+?)}/           and system "perl list_sections2.pl $1";

Which is a lot more compact. It’s almost inefficient, but this doesn’t really matter for this problem (or at least the input I have).

Which one feels more Perl-y?

Or should I just give up on Perl and go back to my Pythonic ways?

12 Sep 2006

The Windows console

Filed under: General — sammydre @ 4:42 am

So Windows (XP and 2k and any reasonable version, anyway) has cmd.exe. cmd.exe mostly does the trick: you can do basic shell-like stuff like use pipes, you can select, copy, and paste text, you can change the font, change buffer length, and search through your command history for a previous command (try F8).

But pretty much every feature I’ve mentioned above is sub-par; some of them are worse than others. For example:

  • You can’t resize the terminal horizontally by dragging the border of the window — you need to set terminal width in the options, then the terminal width is a fixed size and the window size can be changed around it (you end up with a horizontal scroll bar).
  • The command interpreter is pretty basic compared to an alternative like bash.
  • Copy and paste is crap. It is annoying to select stuff in the window – for some reason the only selection model is with a rectangle, which is not inline with any other application on the planet. I don’t believe there is any keyboard shortcut to paste.
  • The fonts you can use are very limited.
  • While there is F8 to search the command history, basic editing of the command-line is nowhere near as powerful as readline-based applications (like you are used to on Linux).

I guess there are actually 2 parts to the problem: the window which does the display, and the console application which runs the actual command interpreter. I’m a little unsure how this is actually broken down in Windows, in Linux it might be xterm+bash.

There are quite a few replacements around:

But none of them quite do what one might want. A simple replacement that doesn’t change things too much, just makes general cmd.exe use nicer is required. So selection needs to be fixed, window resizing could be helped, and with an option for readline-supported command-line editing.

Is that too much to ask?

10 Oct 2005

New laptop

Filed under: General — sammydre @ 2:11 pm

Today I received my shiny new Inspiron 6000 laptop from Dell. It was preinstalled with Windows XP Home.

I chucked an Ubuntu Breezy RC1 CD in, resized my windows partition down with the installed, and let the install do its stuff.

A while later I’m using the laptop and I haven’t manually set up a thing. Everything so far just works.

  • Xorg set up to use 1680×1050 widescreen, works fine.
  • Sound works fine.
  • Intel 2200 wireless works a charm.
  • The Broadcom 10/100 NIC seems to be detected, though I haven’t bothered to use it just yet.
  • Touchpad works fine in X.
  • Speedstep seems to work, most time is spent in 600MHz mode, but when I do something like load The Gimp, it hops up to 1.6GHz.
  • Battery monitor in gnome set up by default, this also works (no idea if its estimate of time remaining is correct just yet).
  • Sound on/off, volume up/down special keys on the front of the laptop work.
  • Plugging my digital camera in came up with a “would you like to download pictures from your camera” dialog box. This worked perfectly.
  • Going into standby works perfectly.

I was expecting to spend the rest of the day (and them some) setting Linux up to work nicely on this laptop. Instead I haven’t spent a second setting it up.

I’m not happy with a default gnome setup, but that is a personal preference thing. I’ll play around a bit and see what I can get working that makes the laptop a bit nicer to work on.

Good work Ubuntu project! Linux feels very accessible.

1 Aug 2005

Something not related to computer science for once

Filed under: General — sammydre @ 7:10 am

For some reason I have the want to “blog” in a what might be described as a more traditional sense. So I am not writing to keep a log of what I am doing in my research, instead I intend to write about my cullinary adventures.

I wanted to cook something Singaporean. Something inspired by Wok Fried Kway Teow, perhaps, but with less frying in pig fat. Certainly noodle based.

I had a look around and found this recipe for Singapore Noodle Curry Shrimp on allrecipes.com. Looked interesting, I thought!

Earlier today I had been shopping and had got some ingrediants I thought would be needed. I was right, but I was also missing a lot of stuff if I go by that recipe. So in normal Sam cooking fashion, I just winged it. Note: the noodles I had were egg noodles, not rice noodles.

I chopped up some ginger and garlic and chucked it in a frying pan (yes, I didn’t even bother with a wok). Got this cooking in some vegetable oil, then chucked the rest of my chopped veges in: red capsicum, 2 spring onions, and one small carrot, all chopped up into thin slices or small pieces. I let this cook for a while, stirring whenever I could, while I formulated a plan for the rest of the recipe. I referred back to the original recipe and found it wanted curry powder. Nope, don’t have any, but I found some Tandoori seasoning. In it goes. Oh look, seasame oil! I’ll chuck a bit of that in too, the veges are looking lonely in there. Just a couple of teaspoons of each.

I got some chicken breast meat and half a punnet of shrimps from the freezer and managed to thaw them out a bit. Also started making the sauce stuff mentioned in the above recipe. Hot water, chicken seasoning, way too much dark soy sauce, salt and sugar. Probably about a cup of water, a couple of teaspoons of chicken stock, about 6 teaspoons of soy sauce, two or three of sugar and a tiny amount of salt. I combined this in a seperate bowl and left it alone. Put the meat in with the veges, turned the heat up, and sauteed to me hearts content. Or did I stir fry? Anyway, once the meat was looking like it was pretty much cooked (doesn’t take long with small pieces of chicken and shrimp. Did I say I cut the chicken up into bits?) I dumped the sauce stuff in. It was supposed to have oyster sauce in it too, but I don’t have any of that.

Let this simmer for a minute or two while I looked at the noodles and wondered what to do with them. They require a bit of boiling in water to become tender, of course. I started getting another pot for them, then thought, “nah, they can go in with the stuff that is currently simmering in water”. I think the original recipe may have called for this. Anyway, I put the noodles in there and the pan was very full. It took a while, but the noodles ate up all the watery saucy stuff and eventually became nice and tender. I mixed it all together once the noodles were unstuck and gave it an extra minute cooking, mixing it about a lot. At some point during this process I also put a teaspoon of chilli powder in.

Then straight into a bowl and I ate it. The noodles absorbed a lot of flavour, which was good, but the soy sauce was too strong I think, they tasted a bit too much like that. Overall it was quite strong and slightly hot, I certainly felt like a drink afterwards, though it was far from hot enough to actually produce any burning sensation in the mouth. Quite enjoyable, less liberal use of soy sauce might make it rather nice.

See the finished product!

28 Jul 2005

Python and my little sync project

Filed under: General, Network Simulation Cradle, Optical Networking — sammydre @ 5:16 am

Problem statement: In my distributed simulation program, I have the problem of keeping the clients synchronised with the server: the clients and the server share a directory hierarchy which contains files that describe a number of simulations.

Previous solution: Previously I had written some code to integrate with subversion. So when you connect as a client, you run svnversion . to figure out what version you are and send that information in your “hello” message. The server checks this, and if it is older than server’s revision, the client is told to update its repository. The client runs an svn update, then attempts to reload its python code. The most important bit is that the simulations are now up to date, or in sync, with those on the server.

One problem with this is that the client machines need subversion installed. Subversion isn’t quite ubiquitous yet, so this is a very real problem. The second is that you need network access for the subversion client and said subversion client needs to be able to authenticate with the server. In the past this has been OK due to the setup of the machines I was working with. However, these two problems are very real and you can’t always get around them easily.

New solution: I no longer care about updating the python code for the simulation client running, this isn’t liable to change (it has existed for some time now and I hardly ever change it) and it already has versioning for the protocol. I concentrate instead on making sure the simulations are up to date. Conceptually, doing this is very easy:

  1. Client is sent the name of a simulation to run by the server (as before).
  2. Client md5sums all files pertinent to the simulation and sends back a list of (file, md5sum) tuples.
  3. Server checks this against its own files. If it matches, it sends back a “md5sum OK” message, else it finds the files that don’t match, gzips them, and sends them back in (file, gzipped file, umask, digest) tuples.
  4. If the client receives an “OK” message, it notes that this simulation is in sync, and requests another simulation.
  5. If the client receives a list of files, it saves them locally into its directory hierarchy. It is able to check their integrity by the digest sent back, but in practice I don’t bother. It then requests another simulation.

Whenever a simulation is started, the clients are alerted of possible new simulations. They then clear their cache of simulations they believe to be up to date.

This works nicely because the files that describe a simulation are fairly small and we don’t care too much about efficiency of the network protocol. A lot of the time this runs over LANs anyway.

So?

Now, the point of this article isn’t to describe the above per se, that is nothing special. It is like a very stunted rsync algorithm that sucks big time.

The cool thing, is that coding the above in Python was so incredibly easy! The standard library does all the things like md5 and gzip. And I already had a framework set up in my system based upon the asyncore/asynchat modules. So plugging this all in was so very easy. I made a few typos and small logical errors, but a bit of testing ironed these out and it all goes now.

Thanks Python!

2 Mar 2005

NZ Cricket team for the next ODI

Filed under: General — sammydre @ 11:04 am

  1. Flemo
  2. McCullum
  3. H. Marshall
  4. Cairns
  5. Vettori (specialist batsman)
  6. Macca
  7. J. Marshall
  8. Mills
  9. Franklin
  10. Hamilton
  11. Martin
  12. Wilson

While brainstorming for the number 10. spot we considered:

  • Andy Bichel (from Australia)
  • M. Hart
  • Simon Doull (Out of retirement)
  • Pasupati
  • Shaw
  • Tuffey
  • Wilson
  • Hamilton
  • Martin
  • Wiseman
  • Franklin
  • Patel
  • B. Walker
  • Canning
  • Aldridge
  • Larsen (Out of retirement)
  • Sir. Richard Hadlee (Out of retirement)
  • Danny Morrison (Out of retirement)

7 Oct 2004

Reverse path simulation/emulation

Filed under: General — sammydre @ 2:25 pm

In simulation we use sim-19. Emulation is emu-9.

Socket buffers need to be set to 64000 bytes.

  • Reverse traffic starts at time=0.0.
  • Forward traffic starts at time=10.0.
  • Test runs for 60 seconds.
  • Bottleneck link is 2Mb/s 50ms, queue-limit 8.

Emulation setup:

machine4:~ $ ipfw pipe 1 config bw 2Mbit/sec delay 50ms queue 8
machine4:~ $ ipfw pipe 2 config bw 2Mbit/sec delay 50ms queue 8

OpenBSD /etc/sysctl.conf:

net.inet.tcp.sendspace=64000
net.inet.tcp.recvspace=64000

Linux /etc/sysctl.conf:

net/core/rmem_max = 64000
net/core/wmem_max = 64000
net/core/rmem_default = 64000
net/core/wmem_default = 64000

FreeBSD /etc/sysctl.conf:

net.inet.tcp.sendspace=64000
net.inet.tcp.recvspace=64000

Except for now Linux is being used for cross traffic so it wont have that setting applied just yet.

Emulations

Filed under: General — sammydre @ 2:16 pm

Setup for emu-7 (5% random loss emulation):

machine4:~ $ ipfw show
01000 449239 668059369 pipe 1 ip from { 192.168.3.0/24 or
192.168.9.0/24 or 192.168.4.0/24 } to { 192.168.7.0/24 or
dst-ip 192.168.8.0/24 or dst-ip 192.168.6.0/24 } in
01001 350569 19550402 pipe 2 ip from { 192.168.7.0/24 or
192.168.8.0/24 or 192.168.6.0/24 } to { 192.168.3.0/24 or
dst-ip 192.168.9.0/24 or dst-ip 192.168.4.0/24 } in
60000 91136675 82707707380 allow ip from any to any
65535 0 0 deny ip from any to any
machine4:~ $
machine4:~ $ ipfw pipe 1 show
00001: 2.000 Mbit/s 100 ms 10 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 udp 192.168.8.2/62134 192.168.9.2/33438 54723956 78862842999 0
0 155714
machine4:~ $
machine4:~ $ ipfw pipe 2 show
00002: 2.000 Mbit/s 100 ms 10 sl. 1 queues (1 buckets) droptail
mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
0 tcp 192.168.7.2/9000 192.168.4.2/1026 36608328 4179547986 0
0 29347
machine4:~ $

Actually, the above was the exact experiment EXCEPT it forgot the packet loss rate in the dummynet pipes. It was turned off for some subsequent tests (the slow start tests in my pam2005 abstract).

23 Sep 2004

Curst

Filed under: General — sammydre @ 11:20 am

Now running on a new machine.

Looks like Wordpress is all up and running correctly on the new server.

Moved from the old dual PII-400Mhz to an Athlon-XP 1800+. Not that is makes any difference to this blog. It was quick enough before…

13 Sep 2004

Subversion

Filed under: General — sammydre @ 1:32 pm

Subversion line endings woes solved

10 Sep 2004

To do

Filed under: General — sammydre @ 2:39 pm

Should set up a set of emulations so I have some results under random loss to compare with ns-2

Want to do it for Linux, OpenBSD and FreeBSD.

Should really upgrade Linux to 2.4.27 first.

2 Sep 2004

Filed under: General — sammydre @ 4:15 pm

MASCOTS: IEEE / ACM International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems looks interesting. I wish I had found out about it earlier. I’ve certainly missed the 2004 date, should remember it’s existance for 2005.

30 Aug 2004

Linux 2.4.27

Filed under: General — sammydre @ 4:21 pm

Linux 2.4.27 is go. The upgrade path wasn’t too bad, but pretty much involved putting my changes to the few headers in asm/ back into the Linux 2.4.27 versions.

Changed file summary: * include/asm-i386/smp.h (smp_processor_id()) * include/linux/sched.h (capable()) * include/asm-i386/system.h (local_irq_*() macros) * include/asm-i386/softirq.h (local_bh_enable(), local_bh_disable(), __local_bh_enable() macros) * include/asm-i386/spinlock.h (spin, read/write locks) * kernel/softirq.c (tasklet_kill, __initcall removal) * include/asm-i386/processor.h (prefetch)

Parser

Filed under: General — sammydre @ 11:09 am

The biggest problem I know of with the parser at the moment is that it doesn’t parse out asm() statements. So if I have:

__asm__ __volatile__ (    "cmpl $0, -8(%0);"    
"jnz 2f;"
"1:;"
".subsection 1nt"
""
".ifndef "
".text.lock."
"clnt"
"nt"
".text.lock."
"clnt"
":nt"
".endifnt"
"2: pushl %%eax; pushl %%ecx; pushl %%edx;"
"call %c1;"
"popl %%edx; popl %%ecx; popl %%eax;"
"jmp 1b;"
".previousnt"
: : "r" (ptr), "i" (do_softirq) );

The ‘ptr’ and ‘do_softirq’ bits will not be “globalised”. Would be nice to have this functionality, though it isn’t of the utmost importance.

26 Aug 2004

Morning wasted

Filed under: General — sammydre @ 12:55 pm

The morning has been wasted with getting my desktop going with a new mainboard and ram. Is up and running now, though I have yet to get accelerated X going properly. And I need to compile a new module for the 1Gb nic. The 10/100 one works perfectly. Sound seems to work nicely as well. Maybe I should consider plugging my cdwriter in at some point as well.

I still plan to change some hardware in vegas as well: need to sort out RAM issues: check why 256MB sticks aren’t working and revert to the 192MB of RAM it had before if they are not going to work. Also want to swap 80GB hdds.

This is probably a mission for this evening.

For now I should do some work.

25 Aug 2004

Parser troubles on FreeBSD

Filed under: General — sammydre @ 2:17 pm

14:00 < @sammydre`_`> a) valgrind shows nothing
14:00 < @sammydre`_`> b) it happens on freebsd and not linux
14:00 < @sammydre`_`> c) the program runs on many other test cases fine
14:01 < @sammydre`_`> d) it happens inside an STL set. I tried changing from
hash_set to set, made no difference
14:01 < @sammydre`_`> e) the bit of code it crashes in is a find(), searching for
'csum`_`fold'. The exact same find had been performed a little earlier
and worked perfectly
14:02 < @sammydre`_`> f) it is reproducable
14:02 < @sammydre`_`> g) the set is not modified between the find() that fails and
the one that works (knowingly, at least)
14:05 < @sammydre`_`> j) I do compile with -Wall

Update:

I found the bug. It was an overflow of a static buffer. I had assumed an error like this would be picked up with valgrind, but I guess not. Very silly error. It only happened when a very specific token was too long. For now I’ve increased the buffer size tenfold. I’m not too sure how to solve this problem in a better way; it would seem I need to understand how flex allocates the buffer size of yytext. I tried a realloc at first, but that didn’t work. Anyway, it works now and Linux almost builds on FreeBSD.

The problem now is that it doesn’t link when I use the linker map (which is quite necessary for operation). It does link fine when I don’t use the linker map. This will need further investigation I guess. Almost there.

Finding bugs like this is never a fun thing. In this case I just iteratively narrowed down my input till I had a small amount of input that broke it, and a minor change to the input that made it work. Once I got it down to this point I could formulate theories on why it was breaking. I added some debug code to show what was happening in what turned out to be the erraneous function and I eventually noticed the small buffer size.

18 Aug 2004

Simulation/emulation findings

Filed under: General — sammydre @ 5:04 pm

Emulation:

01 tcp_write_xmit: skb->len: 1024
02 tcp_write_xmit: skb->len: 1448
03 tcp_write_xmit: skb->len: 1448
04 tcp_write_xmit: skb->len: 1448
05 tcp_write_xmit: skb->len: 1448
06 tcp_write_xmit: skb->len: 1448
07 tcp_write_xmit: skb->len: 1448
08 tcp_write_xmit: skb->len: 1448
09 tcp_write_xmit: skb->len: 704
10 tcp_write_xmit: skb->len: 1448
11 tcp_write_xmit: skb->len: 1448
12 tcp_write_xmit: skb->len: 1448
13 tcp_write_xmit: skb->len: 1448
14 tcp_write_xmit: skb->len: 1448
15 tcp_write_xmit: skb->len: 1448
16 tcp_write_xmit: skb->len: 1448
17 tcp_write_xmit: skb->len: 1448
18 tcp_write_xmit: skb->len: 456
19 tcp_write_xmit: skb->len: 1448
20 tcp_write_xmit: skb->len: 1448

Simulation

printk[0]: 01 tcp_write_xmit: skb->len: 1024
printk[0]: 02 tcp_write_xmit: skb->len: 1448
printk[0]: 03 tcp_write_xmit: skb->len: 1448
printk[0]: 04 tcp_write_xmit: skb->len: 1448
printk[0]: 05 tcp_write_xmit: skb->len: 1448
printk[0]: 06 tcp_write_xmit: skb->len: 1448
printk[0]: 07 tcp_write_xmit: skb->len: 1448
printk[0]: 08 tcp_write_xmit: skb->len: 1448
printk[0]: 09 tcp_write_xmit: skb->len: 1448
printk[0]: 10 tcp_write_xmit: skb->len: 1448
printk[0]: 11 tcp_write_xmit: skb->len: 1448
printk[0]: 12 tcp_write_xmit: skb->len: 1448
printk[0]: 13 tcp_write_xmit: skb->len: 1448
printk[0]: 14 tcp_write_xmit: skb->len: 1448
printk[0]: 15 tcp_write_xmit: skb->len: 1024
printk[0]: 16 tcp_write_xmit: skb->len: 1448
printk[0]: 17 tcp_write_xmit: skb->len: 1448
printk[0]: 18 tcp_write_xmit: skb->len: 1448
printk[0]: 19 tcp_write_xmit: skb->len: 1448
printk[0]: 20 tcp_write_xmit: skb->len: 1448

I did check earlier that there was enough data to send. Or I was fairly certain that that was the case. This needs to be analysed further. Time to go home for now though.

Big screenshots

Filed under: General — sammydre @ 11:09 am

Spend around an hour making BSOD able to generate huge screenshots. Using a tile rendering libary it now generates screenshots of 4000×4000. It could do even bigger if it wanted to.

Next Page »

Powered by WordPress