This simulation seems to have more variance that I had at first thought. After running a preliminary 10 runs on the emulation network (it isn’t all that fair to compare to simulation yet; not until I run more overnight with rebooting of machines), I came up with the following:
9529856 OpenBSD
9367347 FreeBSD
OpenBSD is still leading, but by much less. Anyway, need to run this properly. And run simulation with jitter.
My current findings are this:
* Really need to restart ALL machines in the test. Including the Linux background traffic gen machines.
* Need to start things on time. This means 10 seconds or whatever in, we want to run the actual test.
* To do this nicely, we want something for (reasonably) precise timing. Brendon is going to work on this because its basically a reasonably important part of his 420.
Perhaps I should work on something else for a bit.
The sheer amount of symbols created in the dynamic libraries means starting up ns-2 and loading the libs can be rather slow. An interesting project to work on at home might be to evaluate exactly which symbols need which sort of replacement in the parser. All we need to do is make sure we use the same symbol “mangling” in each file. On a per-file basis this is easy; our symbol table takes care of this. Inter-file is the hard bit. Oh well, something to look at over the weekend or something perhaps. Depends if I find the ever-elusive time.
Reverse traffic congestion.
Topology:
* Bottleneck link: 2Mb 50ms
* Queue-limit: 8
Forward sender:
* Window sizes: 64000 bytes
9802536 Agent/TCP/OpenBSD3 Agent/TCP/OpenBSD3
8948216 Agent/TCP/Linux24 Agent/TCP/Linux24
8421144 Agent/TCP/FreeBSD5 Agent/TCP/FreeBSD5
8334040 Agent/TCP/Sack1 Agent/TCPSink/Sack1
7305040 Agent/TCP/Sack1 Agent/TCPSink/Sack1/DelAck
6547540 Agent/TCP/Newreno Agent/TCPSink/DelAck
6336400 Agent/TCP/FullTcp Agent/TCP/FullTcp
5841040 Agent/TCP/Reno Agent/TCPSink/DelAck
692224 Agent/TCP/lwip Agent/TCP/lwip
Interesting, OpenBSD gets the most bandwidth. Wicked.
So far I can’t make this happen on the emulation network. I suspect I’m not getting the buffers set correctly just yet…
My emulation setup was wrong. OpenBSD now seems correct on the emulation network. Need to set up stuff so I can run ~100 tests.
Checking FreeBSD now.
Simulations seem to run. I should check that the window on FreeBSD is correct.
Research plan pushed back 2 weeks.
Run simulations I sorted out tonight if I can.
Want to fix up building on curst. This could be quite helpful.
Otherwise continue working on validation section I guess. Should do diagrams at some point.
Updated my simulation stuff so I should just be able to run all the reverse path congestion simulations I want. Almost.
Except, I realised FreeBSD doesn’t have sysctl implemented. That is now half-way done.
Made a fairly generic thing to set socket buffer sizes for my stacks. You still need to do a setsockopt, tho.
Run this simulation with the same buffer sizes. maybe 64k, maybe 32k. Something. Definitely needs to be more than 16k!
OpenBSD now is able to support large buggers. Hooray.
OpenBSD doesn’t have many sysctls working. I just added net/inet/tcp/recvspace. I think. And sendspace. Could do with a few more working…
A generic way of setting buffer size would be great!
Doesn’t actually go yet. I seem to be setting lots of variables with no error returned, but the window used in the TCP connection is still small.
Recompiling without optimisation to ease debugging. It is rather frustrating having to implement this now when I just wanted to get a few results.
I really like dummynet as a network emulation tool but I came across a situation today where I noticed it was lacking.
The problem is that with dummynet there is no differentiation between link and router characteristics. To create a virtual link of 2Mb/s+50ms latency I match some packets with a firewall rule and tell them to go to a dummynet pipe. The dummynet pipe specifies these parameters as well as router queue size and packet loss rate. You can also use dummynet to do (gentle)RED or WFQ here.
What I really want to do is emulate a router with 3 interfaces, each of which might have different link characteristics: consider a router with an 2Mb/s HSSI interface, a 10Mb/s card and a 100Mb/s card. With a buffer of some size. We can’t really set this up with a single box using dummynet.
Not getting enough done at the moment. Need to put my head down a bit more. Microgoals are good.
2 things to work on at the moment:
1. Tomacs
2. Research plan
No need to work on (2) any further today. Still need to talk to Tony about it. Consider further work on it tomorrow.
(1) needs further planning. Perhaps I should summarise what I have so far:
1. Abstract – prolly will require an entire re-write, was stolen from ICNP paper
2. Intro. Seems alright so far.
3. Related work section – talks about NCTUns, stolen from ICNP again. BSD Network Stack Virtualization and ATM TN and anything else still needs a discussion here.
4. Arch and design section. Really needs a good diagram. The stuff here probably needs to be moved. I dunno. Lots of writing, really needs to be reviewed.
5. Validation. Talking about packet traces here is quite weak, I think this could all be re-done. Higher level analysis still needs to be written.
5. Reults section. This has nothing. I’m unsure of its import at the moment.
6. Protocol development section. Thorough, not sure where it really fits in. Probably needs more effort to be linked in with the rest of the article.
7. Conclusions+future work not done at all
8. Bibliography exists but I havent cited many other things yet. Need to do citations at some point.
Currently around 11 pages. Would be nice to have 20+.
Need to add a performance (sub?)section in here somewhere!
Validation:
* Should do a cross-traffic validation of some sort.
ENTRAPID is interesting. Or was, back in 1999. ENTRAPID does some nifty “virtualization” to allow protocol development.
However, they say, “so, for example, we can extend our approach to support a virtualized Windows NT kernel…” Now, are they saying that that can be done without the source? It seems like it, but I don’t believe that to be the case. In the case of their FreeBSD kernel, they say that a deep knowledge of the kernel is required and that it is not for the faint of heart. So it sounds like hand modifications to me.
Ahh well, worth talking about, but since nobody uses it/no further research has been done with it it isn’t to important…
Looks like I broke Linux’s sysctl stuff sometime this month. No biggie. Fixed now, I had somehow got rid of my change to include/asm/current.h that I had to make.
Got rid of the spam from the blog and turned on comment moderation. No more comment spam hopefully. Bloody spammers…
Now running on a new machine.
Looks like Wordpress is all up and running correctly on the new server.
Moved from the old dual PII-400Mhz to an Athlon-XP 1800+. Not that is makes any difference to this blog. It was quick enough before…
Working on my literature review.
Got like 26 references now or something. I suddenly know quite a bit more about the state of network simulators, parallel network simulation, and fluid simulation. Still don’t know all that much though, they are large and complex areas (at least the last 2).
Wrote around 1200 words. That’s enough for me for today I think. I think the research plan will end up around 15+ pages or so. Seems like a fair amount of work.
(For little reward).
Working on research plan.
Read a reasonable amount and printed out even more. Some useful stuff there. Seems the BSD network stack virtualization project is actually using it for some research now, and it’s simulation oriented. Need to read their paper soon.
Wrote ~700 words or so on my literature review. Write more tomorrow.
Contacted Vern Paxson about tcpanaly. Very old, had to fiddle around a bit and compile on a FreeBSD machine. Probably not of heaps of use, but I might investigate it a bit more when I get some time. There is a (rather average) technical report about using it to check simulation I have printed out.
Research plan:
* Writing literature review stuff?
* Thesis title?
* Other stuff…
Wednesday “presentation”: prolly not much to say.
Current work:
* TOMACS: emulation stuff. Writing stuff. Work emulating solaris+co?
* Simulations: improve OpenBSD?
* What more results to have
* Validation paper
ATN TN – anything about BSD anywhere?
Run lots of emulations and some simulations.
Wrote quite a bit for tomacs: prolly a bit less than 2000 words over Monday/Tuesday.
tcpnorm needs to understand SACK properly.
Got OpenBSD going on the emulation network properly and set up ssh keys.
Began writing my research proposal.
Need a way to check if we can fire at all. Should probably have like a CPlayer::CanFire() or something. The current framework for firing is somewhat buggered because there is no proper way of saying “you can’t fire.”
More thoughts:
Players are ALWAYS id 0 to 31. We no longer need the player set because of this.
Actors should store their location in CActor. So like m_octree_location or something. No longer need actor map.
Entities should always have a unique ID.
Perhaps all entities, including children, should be in the entity vector. When they are children they can possibly be handled differently.
Get rid of particle manager! If we have the above changes, the particle manager will no longer be necessary to update the particle emitters.
The above is probably quite a bit of work, but worthwhile. The current system is faaar to flaky.
Should set up a set of emulations so I have some results under random loss to compare with ns-2
Want to do it for Linux, OpenBSD and FreeBSD.
Should really upgrade Linux to 2.4.27 first.