Sam’s Network Simulation Cradle Blog

Tcpperf shows bandwidth now. It’s getting to the stage where it is more useful to me than iperf. It doesn’t do UDP, but ’sif I care. Any features I want I can easily implement myself, it is a trivial program still.

Um. Periodic loss.

Create some graph that isn’t presented load, because presented load is boring.

Emulation seems to get 1.11Mb/s when I use tcpperf. Modify tcpperf to actually show bandwidth. Its in cvs now and seems to somehow compile on linux, I don’t know why. Make a table again with emulation results, do it with Linux as well, see how well it goes. Do both NS-2 stacks.

Things are very similar now, but there seems to be an extra delay somewhere in simulation that is accumulating to be about 4 seconds out of sync with emulation after 200 seconds. Now, I don’t believe this to be a problem from the modelled application in simulation, but I can’t rule that out just yet. Perhaps a short investigation into a better application model should be done tomorrow to see if it is feasible. I am fairly confident other obvious delays are all accounted for and are correct.

Otherwise I think they are close enough. It should be noted that there is a tiny unexplained difference still, I guess. However it is very small. Very, very small. They look so excrutiatingly close… I’d love to have it perfect. We’ll see.

NB: need to fix wordpress backups

Changed my 100Mb/s 1ms link in simulation so the latency isn’t so high: 1ms is quite a lot for a 100Mb/s link. I did some pings on the emulation network and decided on 0.115ms latency. I thought about serialisation delay quite a lot and figured out both emulation and simulation seem to do that right – as suspected, but worth checking.

I think my current difference (small, but it adds up) is due to application modelling. Because the sending application can select the socket for writing and get woken up the instant (or near enough to the instant) data can be sent (ie. when the socket is connected) in emulation, I am seeing a bit of difference at the start due to when the application is actually writing any data. In simulation I wait a certain amount of time, write data, wait again: a CBR application. I perhaps need to get the socket waking up implemented, but when I last looked at this it looked very complex and tied in with threads and processes deeply. At this point I really need to decide whether it is worth the effort… Quite probably not.

So 0 (zero) is a special value in TCP RTT generation. t_rtttime is set to ticks often, and 0 as a special case. Now, in my simulation I was creating a connection at time 0, which meant the initial value was 0 instead of some other value that ticks would normally be (my actual machine had got up to 30,000, or was that 300,000?).

So, I just start ticks at 1 now. That was RATHER hard to track down.

Make an application tcpnorm whichs normalises a tcp flow in pcap format: seq and ack numbers start at 0 and timestamps start at Jan 1 1970.

Ok, lets a have another look at things:

• Simulation
• Emulation

Ok, they are identifical: except for retransmission timeouts.

(later edit) They aren’t quite identical, there was still small timing differences at this point.

Fix up apps/bsd.cc in various ways: make it an option to write packet dumps. Change this packet dump writing into pcap rather than ERF.

Results show that simulation and our laboratory emulations are almost exactly the same. Take a look at the following:

• Simulation
• Laboratory emulation

The topic says it all: emulation and simulation don’t match up. Now, we are looking at a random loss scenario, which makes tracking why this is happening down very hard.

The actual measurements we are taking are throughput measurements of 5-10 runs of 200 seconds, averaged. Figuring out why this differs but 5-10% between emulation and simulation is not an easy job.

To really check what is going on, we need exactly the same loss happening in simulation and emulation. Then we need to take traces of both and compare. Its easy enough hacking simulation to work like this, but not the same for emulation. It uses a real kernel with uses some random number generator in the dummynet code to check for packet loss.

So I need to hack the kernel to get very predictable random loss. I had thought this would be very hard, but after thinking about this for a couple of minutes I figure its not so bad. It goes like this:

1. run a simulation recording which number packets are dropped. record this in a file or something.
2. make this information into a big C array which we can put directly into the dummynet code.
3. put this information directly into the dummynet code and change what needs changing for packet loss rate.
4. install new kernel, reboot (maybe… its just a module, might not need a reboot!) setup emulation as before
5. run 1 emulation and record results

We can only run 1 emulation because after that the loss is out of sync. Adding any user space way of changing this back is WAAY too much effort. I wouldn’t even know where to begin looking (well I would, but hey, it wouldn’t be easy). Implementing the above is enough to debug.

Linux results are just making things more complex in general.

• On the UDP congestion one: ns-2 is just wrong. Needs to be re-run with Newreno+DelAck ?
• ns-2 5% random loss: doesn’t quite get it right. freebsd and linux are different, but ns-2 doesn’t agree with either. Hmmm…
• currently emulation doesn’t agree with any simulation for the periodic loss one. Not sure why emulation is magically getting better throughput at the moment, surely I’ve set something up wrong. ns-2’s Newreno+DelAck might well be correct (while FullTcpAgent is incorrect I believe).

I’m trying to figure out the answer for 5% random loss once and for all.

I’ve decided to only work with means over 10 runs now, single situations aren’t so interesting when comparing with emulation.

I’ve simulated all of NS-2’s TCP variants to check things. Now running a 10 hour emulation on Linux to get averages values out. Running a lot less than I was before, but its still 10 hours! Should really have looked into running shorter simulations+emulations (not 200 seconds)…

Anyway, should have some results late tonight of the emulation. Need to keep looking at simulation stuff in the meantime I guess…

Should really emulate a lot less points, I dont need to do so much! Only need to do more if the results look like they need it. Its OK in simulation, cause simulation is fast.

Anyway, should get on to doing the last simulation stuff I need to and writing before thinking about the setup of future emulations.

New ipfw lines: before I wasn’t matching all traffic on the left or right, just a single host. Which buggered my emulation where I actually used both hosts on the left and the right:

00010    2370     212888 allow ip from 192.168.2.1 to any
00020    1452     171136 allow ip from any to 192.168.2.1
01000       0          0 pipe 2 ip from { 192.168.3.2 or 192.168.4.2 } to any out
01100       0          0 pipe 1 ip from { 192.168.6.2 or 192.168.7.2 } to any out
65000 7322775 8650569147 allow ip from any to any
65535       0          0 deny ip from any to any

The pipes are still the same, except I might have changed bandwidth for this setup.

• graphs/bsd_loss.gpl looks good. Almost exactly the same: this is the periodic loss simulation.
• graphs/sim-4.gpl is the same. Still boring; it doesn’t interest us in the slightest.
• graphs/sim-5.gpl is similar. If anything, ns-2 is further away from nsc. This is the congestion one, and interesting.
• graphs/sim-9.gpl is almost exactly the same. This is the loss rate vs. throughput graph.
• graphs/sim-6.gpl is different! Pending examinition. It looks like our 5% packet loss one is quite different, which sucks.

Waiting on re-running simulations on Voodoo.

Fixed up two nasty bugs. Don’t like my bzero implementation; using the ‘real’ one would be good. nsc_memset? Try this before running performance measurements again.

Should prolly get on to reading other ICNP papers again.

freebsd/usr/include/machine/*.h wasn’t added to CVS; fixed now.

2744 words now.

TODO:

• Re-run simulations tonight/tomorrow/tomorrow night. This might depend on:
• Commit all changes and make sure it all works on BSD. Perhaps use the freebsd/ directory structure on BSD as well, it makes sense to: then the version of NSC being used isn’t dependant on operating system installed
• Tomorrow: talk to others about ideas in the results section. Maybe email Tony about this if still lost
• Check results vs: previous results, linux vs. bsd.
• Do conclusions section: this might depend on:
• Read more ICNP papers to get the feel of how previous ICNP papers have been written and to get more references perhaps.
• Print out paper and check for lack of flow. It needs more flow! It strikes me as an vaguely sorted collection of thoughts and results at the moment.

That’s about it. Not too bad, once that is done the paper should be in a decent condition.

Really need to re-run every single simulation I am going to talk about in my paper.

Why? Well, I re-ran sim-9-bsd and got every so slightly different results. They were only slightly different, and the trend was the same… But, well, that really means I should re-run everything all the same. Then I can be confident. Then I can also say that I ran all the simulations in the latest version of ns-2, which I think is important.

Future work section mainly done. Conclusions started, but I’m not sure what to say yet. 6.5 pages, going on 7 now.

Wouldn’t it be fun to have an old DOS/other old system TCP/IP stack available for simulation: http://www.ka9q.net/code/ka9qnos/ late 1980’s, early 1990’s. One of the first TCP/IP stacks for personal computers. If I really cared enough I could have this as a simulation target.

It’s not really relevant since I assume nobody is using it anymore. Need to focus my eneries on more important things. Still, it would be funny.

NB:

make wordcount
Approximate word count: 2304

The reason I was getting segfaults sometimes on linux was because when I turn optimisations on, the definition of memset in the BSD headers stuffs up.

The definition of memset they have calls bzero when the byte to set the memory to equals 0. Which would be OK if gcc didn’t decide to optimize bzero by making bzero call memset. So memset calls bzero, which is actually memset, which calls bzero, which is …

I tried defining my own bzero so this wouldn’t happen: it didn’t help.

I’ve come across this before and I have no idea what I did then.

For now I’ve changed both libkern.h headers under freebsd/ and commented out they bit where they call bzero. This is not a good solution, as it is hand modifying code. I would love a better solution. This should really be on my TODO/wish list.