Sam’s Network Simulation Cradle Blog

When a timestamp of 0 is offered TCP negotiation in Linux, it is ignored. So, need to start time from not 0. We do the same in FreeBSD for different reasons anyway.

Not having timestamps explains a few things…

Should rerun random loss situations with varing random seeds. Could also do it with different loss models.

The main difference I am noticing now is that the advertised window grows a lot larger in emulation.

I haved tracked this down to tp->window_clamp. I’m not yet sure how this variable is initialised or modified, but in simulation it is around 30000 bytes and doesn’t seem to change. This limits the receivers advertised window it seems.

Fixed: Needed to set num_physpages to something sensible so the default socket buffer sizes were not so small. Also, needed to fix the implementation of __get_free_pages to be able to return more than 1 page.

Some emulation results with Linux. Currently emulation is getting better throughput than simulation. This is during 5% loss, the seeds are differing in simulation.

stack                    min        q1         mean       q3         max        stddev    number_runs
sim-linux                165.467000 188.048000 198.155825 209.851000 247.616000 17.170473 40
emu-linux-tcpperf        171.664000 192.759000 206.562073 215.740000 257.823000 18.864367 41
emu-linux-iperf          164.000000 197.000000 207.794872 220.000000 254.000000 20.811408 39
emu-linux-nosack-tcpperf 152.182000 175.989000 191.078045 207.662000 236.703000 20.783054 44
sim-linux-nosack         145.416000 175.784000 184.035575 195.835000 220.168000 19.330628 40
sim-linux-nosack-fixed   110.947000 177.122000 188.837925 207.127000 234.543000 21.879117 40

sim-ns2-newreno-delack   44.240000  89.861300  96.399473 108.165000 126.053000 15.957007 40
sim-ns2-vegas-delack     58.000000  96.933300 116.216673 140.400000 166.800000 28.589674 40
sim-freebsd              60.043700 149.627000 160.374192 178.587000 210.250000 27.954632 40
sim-openbsd              78.120500  93.445900 107.328160 121.632000 164.438000 19.873781 40
sim-lwip                 55.090700  58.789300  62.663202  65.602700 81.176000   5.219324 40
sim-linux.freebsd       121.444000 148.858000 163.310550 180.907000 200.599000 20.202567 40
sim-linux-westwood      165.467000 188.048000 198.155825 209.851000 247.616000 17.170473 40

Need to do some more detailed emulation stuff. Remember to check route caching and so on. Perhaps specfic packet traces need to be compared like I did with FreeBSD. Takes quite a bit of time, but is probably worth it.

WSEAS conference on simulation and stuff.

A paper from there is this one about compaing OPNET and ns-2.

WSEAS also has some journals.

On our simple random test: 100ms latency, 2Mbit/s link with bidirectional uniform random loss. When there is no extra random loss the conditions are perfect; no extra congestion or anything. It is not suprising that the forward start is more important in that case.

Weka attribute selection tells us the following:

On the instances with greater than 0% loss: (1% or more)

Evaluator:    weka.attributeSelection.InfoGainAttributeEval 
Search:       weka.attributeSelection.Ranker -T -1.7976931348623157E308 -N -1
Relation:     sam-weka.filters.unsupervised.attribute.Discretize-F-B10-Rfirst-last
Instances:    98990

average merit      average rank  attribute
 1.555 +- 0.001     1   +- 0      13 reverse_loss
 1.555 +- 0.001     2   +- 0       8 forward_loss
 0.162 +- 0         3   +- 0       2 stack2
 0.126 +- 0.001     4   +- 0       1 stack1
 0     +- 0         5   +- 0       5 direction

On the instances with no loss:

Evaluator:    weka.attributeSelection.InfoGainAttributeEval 
Search:       weka.attributeSelection.Ranker -T -1.7976931348623157E308 -N -1
Relation:     sam-weka.filters.unsupervised.attribute.Discretize-F-B10-Rfirst-last
Instances:    2644

average merit      average rank  attribute
 0.923 +- 0.002     1   +- 0       1 stack1
 0.776 +- 0.003     2   +- 0       2 stack2
 0     +- 0         3   +- 0       6 forward_delay

On all the instances:

Evaluator:    weka.attributeSelection.GainRatioAttributeEval 
Search:       weka.attributeSelection.Ranker -T -1.7976931348623157E308 -N -1
Relation:     sam-weka.filters.unsupervised.attribute.Discretize-F-B10-Rfirst-last
Instances:    101634

average merit      average rank  attribute
 0.501 +- 0         1   +- 0      13 reverse_loss
 0.501 +- 0         2   +- 0       8 forward_loss
 0.044 +- 0         3   +- 0       2 stack2
 0.034 +- 0         4   +- 0       1 stack1
 0     +- 0         5   +- 0       5 direction

There is a lot of work to get done on the NSC.

Place of publication search:

• Tony suggested the journal Elsevier Simulation Modelling Practice and Theory might be possible. The special issues at the moment do not seem to be applicable. Worth watching some more.
• Other Elsevier journals:
• Elsevier Computer Communications. General computer networks stuff, doesn’t really apply for simulation as far as I can see.
• Elsevier Computer Networks. Seems to the other type of networking research not covered in the above; except simulation again doesn’t seem to apply well.
• There are also the SCS conferences mentioned earlier.

Documentation:

Could do some documentation for at least the Linux interface. Doxygen would do it here.

Simulation:

There are always lots of simulations I could work on. Should probably note down all the ideas for simulations and progress I have made on existing ideas before starting anything new.

Analysis:

Alan is saying I should do some more advanced analysis. Learning R would be a good start here. Need to sit down and apply myself at this.

lwIP:

People don’t know how to implement TCP. Might be an interesting learning curve to see exactly what their problem is with respect to Fast Retransmit and so on and suggest some changes.

Linux emulation:

Still need to put Linux 2.4.26 on the emulation network and see how close it is to simulation. Really should get around to this sooner or later.

A progress report since the 22nd: * SCons files updated for FreeBSD properly, they also use sdl-config now * COctree has had an almost complete refactor into nicer code. It is now quite a bit more efficient and the code is a lot nicer and follows a consistent naming scheme and so on. Other code such as collision in the tree that was in in bad place (in actor.cpp) is now in the tree. * Entity manager redesigned to have entities in a vector indexed by ID. * Actor/actor collisions work. Actors are now in the octree as well as being in the vector in the entity manager.

The above was quite a bit of work, but ended up in much better code. The octree is now a low happier.

Networking is currently somewhat broken because ID assignment is not sorted out properly. Each client thinks its player is ID 0, which causes problems. Need to sort out how IDs work over networking, especially with respect to creating projectiles and so on.

NB: Subversion repository is at revision 80.

I had fixed sysctls for OpenBSD some time ago, which has a nasty side effect of changing the exact code path of the Linux sysctls because a diff. variable was passed to the sysctl function. This meant Linux called a function which copies memory between user and system space I hadn’t replaced, which returned an EFAULT or something. I had to change this function in asm/uaccess.h. Sysctls now seem to work.

Bugs in setting SACK and so on. Sysctls seem to be broken on Linux: no idea how that happened! I’m sure it used to work.

Alan mentioned analysis; you can see which variables effect your results the most and so on. Gave me a bit of paper describing some of it. I’m gonna need to learn more about data analysis in the future. Need to learn R as well.

The server: * Now works properly, though there is still an issue with IDs. This needs some thought. * Has a simple ncurses interface. Very, very simple so far, but quite useful as it shows when it sends/receives packets from each client. * Works on FreeBSD: it compiles and runs on vegas. Haven’t commited changes for this yet.

I now use SCons for the building on Linux/FreeBSD. This is MUCH nicer than make. The SCons files need to be updated in a better way for FreeBSD, but that isn’t hard.

Couldn’t reproduce a slowing down/jerkyness that Jesse mentioned when the server was remote. Seems ok to me, maybe recent changes have fixed this after all.

Got results from simulations after getting OpenBSD to do SACK. Did the simulation with 10 reverse traffic senders and varied buffer size the the stacks talking to each other. The graphs are almost impossible to interpret. It does look like Linux to FreeBSD is very good, but it is hard to tell which other ones are just as good. Now rerunning this, with forward traffic instead of reverse traffic.

Alright, here’s my bot, named after a spinning top. It’s only mark 1, and could be a lot better. But still, it won this round… (more…)

I played a little Robot Wars 2 last night. My robot ER seems like a winner. There were problems in the initial design. 1. At first I forgot engines and wheels. Whoops. 2. Then my first engines were not enough to make the robot move much. It would sloooowly trickle across the screen. 3. All the weight was at the front and the wheel were too far forward. When adding wheels I had problems getting them the right size. (more…)

OpenBSD sysctls are partly implemented. Not working yet, but hopefully not that far away.

Statistics reporting for all the stacks … works. The entire interface is broken with respect to returning integer pointers and binding TCL variables. It really needs to change. That is something for the future though.

Apparently FreeBSD (Current) now has SACK. How rare.

The parser crashes in a rather random place on FreeBSD at the moment. No idea why! Might have to debug this sometime… It’s just a case of finding that time.

Should run some more interesting simulations tomorrow: random loss one again with sack/nosack/westwood. The one I set up for the westwood paper. Etc.

I can now use NSC stacks in the place of ns-2 tcp senders and sinks and it mostly works. Getting information out is what I am working on at the moment: with FreeBSD you can get things like the smoothed round trip time and so on out, which is neat. I still need to implement this for the other stacks.

TODO: * Implement sysctls in the various stacks, such as OpenBSD, so we can turn sack on and off. * Allow getting at more variables like rtt and so on from each stack. FreeBSD is the only one that works at the moment.

Then: * Use all of the above and run some simulations to compare things.

Later: * Get around to using the emulation network.

BuNg is progressing slowly towards a 1.0 version which will be playable. There has been a fair old amount of code put into the server recently so that it now has the possibility of being much more intelligent. It is now 26+99+332+161=618 lines of code. It now uses a system driver and has an entity manager and so on. This means it will update entities and perform collisions and so on. Currently it’s knowledge isn’t used for anything, but it will be in the future.

Other stuff done over the weekend: * Bung compiles fine in windows. Solution files and precompiled libraries commited to subversion repository. (This was a reasonable amount of work in the end; mainly to get the dependencies compiling OK) One nice thing about working in Windows is TortoiseSVN; it is a VERY nice subversion client. * Bug fixes in movement, the server, level loading. * Jesse added the beginnings of a GUI framework. Console code is in there but currently not working due to there being no input code just yet. Looking good! I’ve been wanting a console for aaaggeesss. This should help debugging no end when we finally get it going. * Completely new server framework as detailed above. * Partly implemented code to allow actors in the octree. This needs more work. The octree could do with some spring cleaning also (bring on spring I say!)

After hours of work there are no new screenshots to show off. However this good work will stand us in good stead in the weeks to come as there is less and less backend code to work on…

NB: Subversion repository revision number: 59

My attempts at optimising OpenBSD by not copying the entire packet (I think I copied the code from my FreeBSD implementation) were actually breaking it. Setting the length of the mbuf longer than it actually is is a bad idea!

That is now fixed. I just do entire packet copies again.

Also fixed an unimplemented OpenBSD function (ppsratecheck, I remember this in FreeBSD as well). Still got some more simulations to do before I know if I’ve fixed everything that came up overnight.

The main thing running all the simulations overnight did was allow me to do some reasonable testing of stacks that I had not got around to testing all that much.

All the simulations run last night were effectively a host of tests which came up with 130 or so simulations failing because of bugs.

I fixed a few of these by just implementing a linux function (it was trying to report SIGPIPE!)

There seem to be some horrible memory problems in the OpenBSD case. I’m not sure what is wrong, but Valgrind crashes after a while. There may still be stuff wrong with FreeBSD, but I think that fairly unlikely.