The previous post said “look into sim-23″. I did, and it is boring enough. Results with ns-2 and NSC are the same. Which is good for the paper who’s results I was reproducing. I had to update the simulation script a bit. I found it easier this time around to make it work as it says in the paper. Anyway, the result is boring and there is nothing more to say there. Unless we want an example of validation where we don’t find anything extra out.
Oh, and sim-23 does record VmSize for some reason. It is interesting how the memory usage scales, ns-2 and NSC TCP models seem to scale in a nearly equal fashion in that case. Huh.
Look into sim-23 results some more. What is up there?
Put a little bit of stuff into the new ns-2 wiki.
Put my simulation script for the results of my wintersim paper on my NSC pages.
Ended up having to get ns-2.27 to reproduce the results from that. ns-2.29 produces slightly different results. Also had to setup my NSC stuff to use their old defaults. I added a new switch – autoConf_ – which allows automatic setting of MTU and window sizes, by default it is on. Usage: Agent/TCP/NSC set autoConf_ 0 (turns it off, do it before creating and Agent/TCP/NSC/* stacks).
Now need to finish off my presentation and go through it and see how good it is.
Added a tag for the simulation scripts used in the wintersim paper. Should really have done this a long time ago!
Haven’t got around to blogging for a bit. Not for lack of work to blog about though!
lwIP only supports an MTU of 1500. Anything else will break it. It is very hard to support multiple MTU and MSS sizes without recompiling lwIP. Would probably require quite some changing of the internal lwIP code I think. So I put an assertion in for only MTUs of 1500.
On top of this, lwIP seems completely broken when it encounters loss at the moment. I updated it to CVS HEAD, I’m not sure if this made it better or worse. At any rate, for now its results are useless. I’d like to fix this, but I don’t really have the time right now.
Re-ran sim-2. Kind of interesting results. This simulation has periodic loss, like someone pulling out your network cable then putting it back in again after a second. Every 10 seconds, on the dot. Realistic, eh? The main differences really seem to stem from how the stacks handle their own congestion-induced loss on the link, though. That is perhaps the interesting bit. Worth looking into some more at a later date to see what is up.
sim-3 is very uninteresting. sim-5 is completely uninteresting.
Really struggling to reproduce some of my own results right now. Resurrected my old sim-17, which does stuff with random loss. I was very thorough with my earlier runs and I have all the results recorded, but I can’t reproduce them. Going to try an older ns-2 version a bit later, struggling to figure out what else it could be (given that I have SVN history to work with and all).
I spent 10 minutes playing and compiled the OpenSolaris TCP code on my Linux machine. I turned warnings off, and it seemed ok. I did the following:
# cd opensolaris/usr/src/uts/common/inet/tcp
# gcc -nostdinc -D_KERNEL -I ../.. -I ../../../intel -w -c tcp.c tcp_sack.c
# gcc tcp.o tcp_sack.o -o tcp.exe >link_output.txt 2>&1
# grep "undefined reference" link_output.txt |
awk '{ print $5 }' | sed 's/^.//' | sed 's/.$//' |
sort | uniq -c | sort -n > undefined_references.txt
# wc -l undefined_references.txt
291 undefined_references.txt
# tail undefined_references.txt
45 __dtrace_probe_conn__dec(lon
45 ipcl_conn_destroy
45 lbolt
45 tick_per_msec
46 cv_broadcast
51 cmn_err
91 msec_per_tick
125 freemsg
132 mutex_enter
208 mutex_exit
Yes, I know the sh code is horrible. I don’t care, it works.
291 undefined references is about on-par. Note that I haven’t even compiled in the IP code. And I have no idea if there should be other options to the compile line. And I feel that there may have been many warnings from gcc.
I think Sun have their own set of patches to gcc to make it compile OpenSolaris. This download was the special “gcc” version of OpenSolaris too.
Sun has a very nice code browser for OpenSolaris. But I’d really want a command-line way of using this so I could automatically process my list of undefined references and see where they are all defined.
But anyway, this is all a bit of a waste of time, just something I can play around with in my spare time if I’m bored. I would kind of like to prove that it can be done. But anyway.
Title says it all. My very rough word count weighs in at 8000 words or so. It is around 30 pages. Hmm. A bit longer than I wanted it to be perhaps. But maybe not too bad.
Now I have to decide what to do next. First I might go to the gym and have lunch and all those sorts of things.
I looked into sim-34 results more. Again, the graphs don’t really look any different from a greater number of results.
But I did wonder why I only had results for one m_flow_delay and two values of reverse_flows.
Turned out when I set up my simulation, I put the number of simulations as 122500 instead of 1225000 (missed only one order of magnitude).
I could continue running the simulation with the machines and up the number, but um, I don’t really want to. More interested to move onto another simulation. I think I’ve got enough interesting results for this scenario, even if they aren’t quite as good as they could be. There are a lot of other parameters which are varied.
Also, after looking at the data manually in a spreadsheet a little, and sorting on difference, I’ve noticed the following:
- Forward flows is always 55 or 100 (the two highest values it can be) when the difference is large
- Bandwidth is generally low with only the odd one higher than 1Mbit
- Queue sizes do vary, but stay at the low end.
- Add high differences (> 10), but not stupidly high (for example, 14), there is a greater variation of these parameters: the odd big queues (30), bandwidths of 4Mbit commonly, though not at once. Forward flows still the same.
And here is the reason. Taken from here.
Don’t ask—it just is: If you want to be able to refer to a Figure in the text, then you have to give it a label, just like you do for equations, sections, etc. To get things to work, you cannot put the \label line before the \caption line.
I had thought I had checked for that, but I guess not. How incredibly silly.
Was sick on Friday, which didn’t help things. And rather tired yesterday. Yesterday I did get my weekly report done and did a bit of debugging of my reference problems in Latex, which I still don’t have an answer for. Also created a presentation which will one-day form itself into something I’ll present at wintersim I guess. Has no content yet.
Currently updating stats on sim-34. This is probably the last time I’ll bother. I’ll have a little look at some of the stats perhaps and see if there are any obvious conclusions to draw. Probably worth coming up with a simulation with a more complex topology that is otherwise like sim-34 and set it simulating.
Also need to continue working on the implementation chapter: add its intro and conclusion type stuff, and compare and contrast to other theses.
Here’s an interesting conference WNS2 – ns-2 : The IP Network Simulator.
Pisa, Italy, October 11 2006.
So it’s 11 months away. Seems like a place I should really publish something. Would be a shame not to.
CFP is due May 20 2006. Should mark this date down somewhere.
Of course, I don’t know who I’ll be affiliated with come October 2006. That perhaps makes things a little harder, mainly on the “getting-there-and-getting-it-paid-for” front. We’ll see I guess.
Further results from sim-34 don’t show any new trends. Everything is the same, just with more data. This is good.
Weekly report done, email sent to Tony, short discussion with Brendon about comparing numbers statistically.
Need to get into parser issues for the rest of the day I think.
Writing in thesis. Working on implementation chapter. Made a couple of diagrams for the architecture overview. Progress is slow and steady at the moment.
sim-34 analysis continues to get updates. Graphs aren’t changing much with further data, which is good. Need to form a plan for what to do with this simulation.
TODO next: decide what to do with sim-33.
I’ve spent more time analysing sim-34 today, and I’ve come up with the following set of graphs.
A view of the differences in measured goodput for different stacks in otherwise identitcal simulation scenarios.
The graphs on the left show the differences between the stacks visually. Each point on the graph is the difference in one simulation scenario. The y-axis is the difference, ie. a 5 means one stack one 5 times faster than the other. Positive numbers mean the first stack was faster than the second, negative that the second stack was faster than the first. The x-axis is just the simulation number to spread the points out.
On the right is a graph showing the density of the differences. On the y-axis is the percentage (1 == 100%). The x-axis has the difference this time. From these graphs we can see what percentages of the simulation results were how different. If the lines stick around 1 or -1, and cross at y=0.5 or so, then the results were very similar.
I did a basic run-through of my chapters and figured out that my dissertation would end up being around 110 pages long.
I think this might well be a very conservative estimate. I’m having quite a problem estimating how much space actual content will take up. I think I’ll have to start working on a section that doesn’t rely on the previous stuff too much and see how much space it takes up. So probably something in the implementation section.
For now I’ll assume that the current design will create a dissertation of reasonable length. I still have that gut feeling that it will be a bit long (much more than 110 pages).
SP&E still looks like an interesting place for me to write up my parser work. I get the feeling I’ll never get around to it, though.
Results from sim-34 look pretty boring so far. Things look the same. Hmph.
Update, noon: Overall distributions are about the same, but there seem to be lots of individual cases that are different. So there is something interesting going on under the hood. Will look into more after lunch break perhaps.
Update, 2pm: Differences for the same set of parameters looks quite interesting. I’ve made some graphs plotting these differences and it seems a lot of the time there are significant differences found. Need to figure out exactly what percentage of the time this is the case, but it is looking pretty good.
Update, 2:30pm: A bit of data coercing in Python and playing with gnumeric, and I now have the following information. This is obtained by looking at a set of parameters, and the highest and lowest goodput recorded by measured stacks for those parameters. There is no guarantee from this the difference is between an NSC and an ns-2 stack here. And we only have a small amount of the total data we want from this simulation experiment. However we are getting enough data points that what we are looking at is probably representative.
- Only 18% of the simulations are within 5% of measured goodput for the same parameters.
- 40% of the simulation scenarios end with a measured goodput difference of 50% or more.
- Almost 10% of the simulation scenarios end with a difference of 400% or more.
Currently reading some theses. Need to do a bit more of this yet.
Have re-run sim-33 with a fixed “reverse flows” thing. Now have more results to analyse. Looked into this preliminarily. A bit more analysis required perhaps. Don’t think the results are particularly interesting though.
sim-34 is taking forever to run.