Sam’s Network Simulation Cradle Blog

Forgot to enable Linux simulation last night. Argh.

Would be useful if I could simulate stuff on curst. Keep forgetting to turn my desktop on.

Need to remember to turn instrumenting functions off and the like. Hopefully will make simulations a bit faster. Really waiting on these results to write the next section of the tomacs paper.

Hmm… I’m sure there is some more work I can do on the existing paper, I just forget what.

Oh yeah, also need to run performance stuff again probably!

After far too much time stuffing around, Linux seems to go now.

I just globalised all variables under the sun (apart from SCTP stuff, which I don’t believe with globalise correctly).

So, I have no idea what global variable was causing things to die. Just one of the many ones I added (mainly ipv6-looking things). Oh well, I guess I was probably going to have to globalise them all at some point anyway.

Remember to run performance tests without -finstrument-functions.

If you are to do any tracing (such as $ns namtrace-all) then you need to turn this on EARLY in your script, like before you set up any links. Or, the link might be in the script but there will be no packets over it.

When writing ns-2 simulation scripts I often have this error and always forget what it means:

--- Classfier::no-slot{} default handler (tcl/lib/ns-lib.tcl) ---
        _o31: no target for slot -1
        _o31 type: Classifier/Hash/Dest
content dump:
classifier _o31
        0 offset
        0 shift
        2147483647 mask
        1 slots
                slot 2: _o44 (Classifier/Port)
        -1 default
---------- Finished standard no-slot{} default handler ----------

It seems to mean I’ve forgotten to an $ns connect $s1 $s2 or similar: I forgot to connect stuff, so when something went to send a packet, it had no destination (though it is connected into the topology correctly).

At some point I’ll need to put FreeBSD 5.3 into simulation.

[20:13][sam@curst:/dump/dump/f]$ uname -a
FreeBSD curst 5.3-BETA5 FreeBSD 5.3-BETA5 #0: Mon Sep 20 06:09:09 UTC 2004
root@mango.internal.hasta.se:/usr/obj/usr/src/sys/GENERIC  i386
[20:13][sam@curst:/dump/dump/f]$ sysctl -a | grep tcp.sack
net.inet.tcp.sack.enable: 1

TCP SACK!

So with Linux in simulation, I have a simple simulation where it will run fine if the length of the simulation is less then 493 seconds. 493 seconds or more, and it seems to go into an infinite loop or something. For now I’ve decreased the length of simulations, but this will need to be fixed in the future.

… Goes now. Running some performance stuff to see how it goes.

Looks remarkably OK.

Should probably actually do other work while I am waiting for simulations!

Working on getting everything compiling on my home system. Probably a good thing, it generally makes me fix lots of silly little mistakes that have crept into things since I have only been working on voodoo for ages.

Got ns-2 compiled, seems alright. Just did a commit and update on voodoo and carceri respectively, hopefully things will work once everything is built now.

Read over my tomacs paper. Isn’t as bad as I thought, but still needs a good amount of work.

Made another diagram for it. I stole my ICNP performance section, chucked it in. That makes the paper 18 pages long. Hrm. The performance section will need a lot of reworking. Wonder if the paper is missing any content now. Not really sure; perhaps more on the validation front? Or should I include some stuff on some larger simulations run? Hmm.

Ok, Solaris is set up: * bash is the shell * homedir is /export/home/sam * make and gcc are installed and work

We don’t have an image for it yet because there is no chance imagezip will compile on it. We’ll have to use a knoppix cd some other time. Can’t be bothered right now.

bash-2.05# uname -a
SunOS sunsucks 5.9 Generic_117172-07 i86pc i386 i86pc
bash-2.05# ./tcpperf -v
./tcpperf: illegal option -- v
Usage: ./tcpperf [-c <client name>] [-p port] [-t seconds] [-s write size] [-w socket buffer size] [-n]

Had to compile like so: gcc -Wall tcpperf.c -o tcpperf -lsocket -lnsl. Also, the headers didn’t define timersub by the look of it.

FreeBSD, OpenBSD and Linux all have varying parameters which can be used to tune TCP. I’ve excluded all sorts of parameters from this because they are not so useful for me. All parameters show their default settings.

FreeBSD 5.2.1:

1. net.inet.tcp.rfc1323: 1 – “TCP Extensions for High Performance”, TCP window scale, RTTs, PAWS.
2. net.inet.tcp.rfc1644: 0 – “T/TCP — TCP Extensions for Transactions”, (not useful)
3. net.inet.tcp.sendspace: 32768 – TCP send buffer size
4. net.inet.tcp.recvspace: 65536 – TCP receive buffer size
5. net.inet.tcp.keepinit: 75000
6. net.inet.tcp.delacktime: 100 – Delayed ACK time in milliseconds
7. net.inet.tcp.delayed_ack: 1 – Whether to enable delayed acks
8. net.inet.tcp.rfc3042: 0 – “Enhancing TCP’s Loss Recovery Using Limited Transmit”
9. net.inet.tcp.rfc3390: 0 – “Increasing TCP’s Initial Window”, increases initial window to 3 or 4 segments.
10. net.inet.tcp.path_mtu_discovery: 1
11. net.inet.tcp.slowstart_flightsize: 1
12. net.inet.tcp.local_slowstart_flightsize: 4
13. net.inet.tcp.newreno: 1 – Newreno congestion control, a good thing

Linux 2.4.27:

1. net/ipv4/tcp_moderate_rcvbuf = 0 – no idea…
2. net/ipv4/tcp_default_win_scale = 0 – hrm
3. net/ipv4/tcp_vegas_cong_avoid = 0 – Probably enables TCP Vegas. Don’t do it.
4. net/ipv4/tcp_westwood = 0 – Enables TCP Westwood. Good.
5. net/ipv4/tcp_low_latency = 0 – not sure
6. net/ipv4/tcp_frto = 0 – Forgotten…
7. net/ipv4/tcp_dsack = 1 – DSACK, a good thing when SACK is turned on.
8. net/ipv4/tcp_ecn = 0 – ECN, leave it off
9. net/ipv4/tcp_reordering = 3 – not sure
10. net/ipv4/tcp_fack = 1 – Forward acks for SACK, might as well leave it on.
11. net/ipv4/tcp_rfc1337 = 0 – “TIME-WAIT Assassination Hazards in TCP”
12. net/ipv4/tcp_retries2 = 15
13. net/ipv4/tcp_retries1 = 3
14. net/ipv4/tcp_retrans_collapse = 1
15. net/ipv4/tcp_sack = 1 – SACK. Might as well leave it on.
16. net/ipv4/tcp_window_scaling = 1 – Should leave this on, window scaling is useful.
17. net/ipv4/tcp_timestamps = 1 – Timestamps – you want these.
18. net/core/wmem_default = 106496
19. net/core/wmem_max = 106496
20. net/core/rmem_default = 106496
21. net/core/rmem_max = 106496

OpenBSD 3.5:

1. net.inet.tcp.rfc1323=1 – as FreeBSD
2. net.inet.tcp.slowhz=2 – no idea
3. net.inet.tcp.recvspace=16384 – TCP receive buffer size, note that the default 16kB is very small
4. net.inet.tcp.sendspace=16384 – TCP send buffer size, should probably be more than 16kB too
5. net.inet.tcp.sack=1 – SACK, leave on.
6. net.inet.tcp.ackonpush=0 – interesting, I guess the name says it all
7. net.inet.tcp.ecn=0 – ECN, leave off.
8. net.inet.tcp.rfc3390=0 – As FreeBSD

sam@carceri ~/900/pam2005 $ make wordcount
Approximate word count: 1970

sam@carceri ~/900/full`_research_proposal` $ make wordcount
Approximate word count: 3739

sam@carceri ~/900/tomacs $ make wordcount
Approximate word count: 5268

Hardly a good indication of the amount of work done or anything, but interesting nonetheless. Total: 10977. I think my honours report had more words than that…

Reviewed all the minor changes from research proposal and integrated these.

Ran reverse path congestion tests at last. Well, ran the Linux ones, the OpenBSD and FreeBSD ones were done. Started writing up some results of this into pam abstract.

Need to do up diagrams for both PAM and research proposal. Bloody diagrams.

In simulation we use sim-19. Emulation is emu-9.

Socket buffers need to be set to 64000 bytes.

• Reverse traffic starts at time=0.0.
• Forward traffic starts at time=10.0.
• Test runs for 60 seconds.
• Bottleneck link is 2Mb/s 50ms, queue-limit 8.

Emulation setup:

machine4:~ $ ipfw pipe 1 config bw 2Mbit/sec delay 50ms queue 8
machine4:~ $ ipfw pipe 2 config bw 2Mbit/sec delay 50ms queue 8

OpenBSD /etc/sysctl.conf:

net.inet.tcp.sendspace=64000
net.inet.tcp.recvspace=64000

Linux /etc/sysctl.conf:

net/core/rmem_max = 64000
net/core/wmem_max = 64000
net/core/rmem_default = 64000
net/core/wmem_default = 64000

FreeBSD /etc/sysctl.conf:

net.inet.tcp.sendspace=64000
net.inet.tcp.recvspace=64000

Except for now Linux is being used for cross traffic so it wont have that setting applied just yet.

Setup for emu-7 (5% random loss emulation):

machine4:~ $ ipfw show
01000   449239   668059369 pipe 1 ip from { 192.168.3.0/24 or 
    192.168.9.0/24 or 192.168.4.0/24 } to { 192.168.7.0/24 or 
    dst-ip 192.168.8.0/24 or dst-ip 192.168.6.0/24 } in
01001   350569    19550402 pipe 2 ip from { 192.168.7.0/24 or 
    192.168.8.0/24 or 192.168.6.0/24 } to { 192.168.3.0/24 or 
    dst-ip 192.168.9.0/24 or dst-ip 192.168.4.0/24 } in
60000 91136675 82707707380 allow ip from any to any
65535        0           0 deny ip from any to any
machine4:~ $
machine4:~ $ ipfw pipe 1 show
00001:   2.000 Mbit/s  100 ms   10 sl. 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 udp      192.168.8.2/62134     192.168.9.2/33438 54723956 78862842999  0
  0 155714
machine4:~ $
machine4:~ $ ipfw pipe 2 show
00002:   2.000 Mbit/s  100 ms   10 sl. 1 queues (1 buckets) droptail
    mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000
BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte Drp
  0 tcp      192.168.7.2/9000      192.168.4.2/1026  36608328 4179547986  0
 0 29347
machine4:~ $

Actually, the above was the exact experiment EXCEPT it forgot the packet loss rate in the dummynet pipes. It was turned off for some subsequent tests (the slow start tests in my pam2005 abstract).

tcpnorm and tcpperf updated on software page. tcpnorm has turned out to be rather useful.

tcpnorm is now fixed up to properly work with SACK. The initial implementation was always buggy.