Worked on these simd simulations:
test-windowsize — explained in a previous post.sim-29 — looked into multihop wireless networks. Simulation isn’t perfect. Shows that multihop wireless is bad for TCP and that ns-2 is a good estimation of the goodput you get. Simulation could perhaps be fixed so it works with other routing types and such, but not sure if it is worth it?sim-28 — for mattb, very simplesim-27 — not integrated with simd actually, but this is all the simulations like wlan_complex and so on. More on this below.sim-26 — SIP simulation. This needs revisting. This reproduces a simulation from a paper, searching through my blog will find which. The main result was that the paper didn’t describe or analyse its simulations enough. But still, should revisit and see what conclusions can be drawn.sim-25 — looked at different loss models a little bit. Didn’t produce very interesting results. Though I could always go back and experiment a bit more here.
The wireless stuff in sim-27 is the most interesting. This is where I have the simulation scripts for NS2 Simulation Tests for Modeling Wireless Links and Modeling Wireless Links for Transport Protocols. Got some interesting results…
So where to? The idea was to find some interesting simulations. Need to analyse that wlan situation some more to fully understand it. And that’s where I’m at. Running simulations and analysing them to find something nice to write about and eventually put in my thesis. Trying to show all the effort I put in was worth it…
I have been perusing the stuff ns2 code by “Andrei Gurtov”. Looks like there is some stuff possibly worth looking into more there, but I don’t really have time today.
And them I’m away for 3 months! Should have a good summary of where I’m at and where I plan to go before I leave today.
In the linux 2.6 code, we needed to set the addr_limit high, it was zero before which meant some tests were failing when they shouldn’t: specifically putting data back into userspace with the put_user function:
C:
current->euid = 0;
current->thread_info = (struct thread_info *)nsc_malloc(sizeof(struct thread_info));
memset(current->thread_info, 0, sizeof(struct thread_info));
current->thread_info->addr_limit.seg = 0xFFFFFFFF; /* This line is new */
The above bug took a long time to figure out. Largely due to me misunderstanding a small bit of code and a comment. It was actually simple enough, I just interpreted it the wrong way around. Oh well.
Had to implement a few functions and remove __get_user_*. Put getuser.S into the compile. Edited this so it doesn’t check for access stuff.
Implement buffer_size in linux 2.6 like 2.4. Implement trivial __copy_to_user_ll function. Implement some random sysctl handler functions. Add the required sysctl parsing like 2.4.
Window size results
Interestingly, when I run the simulation test-windowsize now that everything is “fixed” there are a couple of obvious observations which one will make when looking at the graph:
- FreeBSD and OpenBSD give the user about the same window that is requested, the function looks to be
y=x as you might expect.
- Linux 2.4 and 2.6 have the same gradient, but it is significantly shallower than the BSD stacks. The function is around
y=0.74*x.
- Linux 2.4 has a huge spike at
x=60.
- Linux 2.6 has a small dip around
30<=x<=45.
The gradiant of the Linux 2.4 line has been verified through measurement on the emulation network. I could not reproduce the spike but the kernel version on the emulation network machines was not exactly the same. I’m still dubious. At least it’s not there for 2.6. I have not looked into verifying the 2.6 “dip” behaviour. I think I’ll leave it as-is, it is “good enough” for now.
Fix up window size in simulation. Want a switch to allow default stack sizes or to use the ns2 window size. Google the Linux problem, Tony said he had the same problem — it doesn’t seem to be specific to NSC.
Look into umts and wlan_compex some more. Want to get a feeling of how different they are. Going to have to decide whether I should continue looking into the wireless realm, it would be possible to spend heaps of team learning about wireless with very little reward.
Today has been spent playing with my page on CiteULike so far. It seems like a nice way of organising publications.
So simple multi-hop wireless stuff isn’t so interesting. Get the same results as ns-2 pretty much. Well, the goodput is a bit lower, but everything is definitely in the right ballpark. At larger hop counts the difference might be more significant. Could look into this some more, but it is not as interesting as I would like.
Random link: ns2 tutorial
Some wireless stuff; looking into something to simulate
TCP Performance in Wireless Multi-hop Networks — uses GloMoSim, from 1999.
Looks very good — The Impact of Multihop Wireless Channel on TCP Throughput and Loss (2003) — must read this some more. And try and reproduce. Could be very interesting… In at least one case they do a real world measurement (7-hop wireless network with 8 notebooks). ns-2 gets the same trend, though it overestimates a fair bit. Interesting.
Then again, might want to look into how the routing protocols effect things. Too much stuff to look at, too little time!
Lots of info on TCP/wireless performance
So, I thought I might try out a Wordpress plugin that uses Geshi.
C:
#include <stdio.h>
/* Herein lines the main function */
int main(int argc, char *argv[])
{
‘c’;
15; // Single line comment.
}
It’s a bit white at this point in time. The colour scheme needs work, but otherwise it’s quite nice. Geshi is good because it has no external dependencies, so it “just works”.
Update: Right, I’ve created some CSS to give colours like the darkblue colour scheme in Vim. The CSS is based on what is output from Geshi, I just got that and hacked it up.
CSS:
/**
* GeSHi Dynamically Generated Stylesheet
* ————————————–
* Dynamically generated stylesheet for c
* CSS class: geshi, CSS id:
* GeSHi (c) Nigel McNie 2004 (http://qbnz.com/highlighter)
* Updated by Sam Jansen, colour scheme based on darkblue.vim (http://www.vim.or
g)
*/
.geshi .de1, .geshi .de2 {font-family: ‘Courier New’, Courier, monospace; font-weight: normal;}
.geshi {}
.geshi .head {font-weight:bold;}
.geshi .foot {}
.geshi .imp {font-weight: bold; color: red;}
.geshi .ln-xtra {color: #cc0; background-color: #ffc;}
.geshi li {
font-family: ‘Courier New’, Courier, monospace;
color: #c0c0c0;
font-weight: normal;
font-style: normal;
background-color: #000040;
}
.geshi li.li2 {
font-weight: bold;
}
.geshi .kw1 {color: #60ff60;} /* keyword. eg. font-style in CSS */
.geshi .kw2 {color: #ffa0a0; } /* keyword. eg. monospace in CSS */
.geshi .kw3 {color: white; text-decoration: underline;} /* Link to function */
.geshi .kw4 {color: #60ff60;} /* keyword. eg. int */
.geshi .co1 {color: #80a0ff; } /* single-line comment */
.geshi .co2 {color: #ff80ff;} /* preprocessor directive */
.geshi .coMULTI {color: #80a0ff; } /* comment */
.geshi .es0 {color: #ff7f00; } /* special; \n in a string */
.geshi .br0 {color: #c0c0c0; font-weight: bold; } /* brackets */
.geshi .st0 {color: #ffa0a0;} /* string */
.geshi .nu0 {color: #ffa0a0;} /* number */
.geshi .me1 {color: #202020;} /* ? */
.geshi .me2 {color: #202020;} /* ? */
Ok, I was hoping to be much further into my ns-2 wireless experiments by the end of the week, but I kept getting distracted.
I should look into multi-hop wireless. Lots of papers have discussed TCP performance in this situation I think? My cursory glance before seemed to show a lack of understanding of TCP in such papers. I’m sure there are good and bad papers, of course.
This should be an interesting field of investigation. Wireless is a realm where a lot of real world testing isn’t done because it’s so hard to do. Simulation is used lots. And they use bad TCP models maybe? Quite probably, they use bad wireless models. Not my place to comment on that, though.
NSC is just about ready to release. Do that on Monday.
Should create a script where I have multiple hops for a TCP performance test, then increase the amount of hops and see how performance is effected.
RTS/CTS? Kinda hard to turn off, it breaks things. What datarate? Still need to know more about this sort of stuff. Probably best starting point is the simple wireless example within ns-2. And the stuff I currently have I guess.
Not sure what to conclude from my wlan findings so far! Hopefully make some real progress next week…
It seems likely you cannot disable delayed ACKs on Windows (like Linux). I looked through information on the Windows 2000 TCP/IP stack and found nothing. It says it has delayed acks, and the delayed ack timer can be set to various multiples of 100ms (seems the “tick” in Windows 2000 is 100ms). Doesn’t mention any way to disable the delayed acks. A quick google search and looking elsewhere on msdn and so-on doesn’t show anything obvious. If it is possible, I have no idea how nor do I know where to find out how I might.
The bug I was encountering where ns-2 would report:
Scheduler: Event UID not valid!
is explained on this page, a quick google found this answer fairly quickly. It seems that the problem was caused by turning RTS/CTS off, setting the datarate high, and some random bug in the code.
The infomation in the link above was posted to the ns-users mailing list and seemed to go unnoticed? This was during ns 2.27 days, the current release is 2.28.
Cleaned up home dir; moved most of my stuff from old to new voodoo. Still waiting on public_html and subversion repository. I’ll move those soon.
Reran sim for mattb, took all of 5 minutes to get results.
Did a bit more research into wireless sims with ns-2 and glomosim. Glomosim hasn’t been updated since 1998 or so? QualNet is a commercial company that continues development on it. Glomosim’s strong point was it’s wireless models. It has a TCP implementation too, which would be vaguely interesting to look into.
You can get a 2 week trial version from QualNet and it also has a university program that allows you to get their simulator for free academic use. I don’t quite care about this enough right now, but might be interesting to look into if I find it is used for many more wireless simulations. I know glomosim has come up a bit in the past, what is the current flavour of the month for wireless simulations? Is it ns-2?
Tidied up a lot of my ns-2 code, finally changed naming of a few things that really required it and also fixed a longstanding hz bug to do with Linux 2.6 and lwIP. Getting ready to release a trial version soon. A few more hours of prep and it’ll be done.
Tried setting data rate to 11Mb, and I get this on many (but not quite all) simulations with that data rate:
Scheduler: Event UID not valid!
I have no idea why that happens. Bug in the wireless code maybe? I should be able to simulation an 11Mb 802.11b link!
CiteULike
Jury is still out on this, looks interesting though. Generated a good bibtex entry on my first go. Haven’t used it for anything serious yet.
This link and the one on the previous post came from Evan Jones’ page about ns2. Some nice looking python scripts there to parse trace files, but only trace files in the new format.
Rather than using powerpoint, can use S5.
Excerpt:
S5 is a slide show format based entirely on XHTML, CSS, and JavaScript. With one file, you can run a complete slide show and have a printer-friendly version as well. The markup used for the slides is very simple, highly semantic, and completely accessible. Anyone with even a smidgen of familiarity with HTML or XHTML can look at the markup and figure out how to adapt it to their particular needs. Anyone familiar with CSS can create their own slide show theme. It’s totally simple, and it’s totally standards-driven.
Looks good.
Wrote a simulation for mattb later in the day. This went well enough, it showed how long it takes Linux TCP to attain maximal link throughput in a specific simulation. Graphed this. Took about 12 minutes to change a simple test simulation script to something that was parametised more and part of my simulation framework.
Looked into wireless sims more; the 2nd wlan_complex situation looks most interesting. Made the scripts output goodput. There wasn’t as much variability in this as I would have liked.
The main problem here is that I don’t know what variables to change to get a better feeling for the variability of the result. Really need to understand this better if I am to draw any conclusions.
And a summary of stuff to get my head around what to talk to Tony about.
- Went over paper, marked what was the same and different and the quality of the different stuff. Emailed Tony about this.
- Found some papers about TCP during non-congestion loss. Nothing too interesting, but worth making note of anyway.
- Found a problem in my parser with static locals. Fixed it.
- Updated to new version of scons, much faster. Yay.
- Convinced myself again using gcc’s C grammar for the parser would have been a bad idea.
- Fixed
__attribute__ support in the parser up. Was not easy to support these properly. Then went on to globalise FreeBSD “linker sets” “properly”. That was hard.
- Turned 23. Globalised some bad variables. Found a 7.5MB long line of C code.
- Fixed Linux 2.6, setting the window size is broken. Not really much point anyway with Linux, disabled.
- Setting MTU in simulation works properly.
- Found a good paper by Allman and someone about evaluation TCP.
- Learnt about the Poisson process and implemented this in TCL. This allowed me to finish off my SIP simulation without any ns-2 hacks. Talked to an author of the SIP paper. Get mostly similar results now, though not quite the same.
- Found my framework doesn’t work for the sort of data I was getting for the SIP simulations. Left alone, did everything “manually” for now.
- In the SIP simulation:
Agent/TCP/Sack does much better than Agent/TCP/NSC/FreeBSD5. That was my basic finding. They are quite different. Experiment was conducted with delayed acks turned off. Very important, that. We can do this for FreeBSD, but not Linux. Unrealistic experiment. So both Linux and FreeBSD get quite different results.
- Found a good paper on wireless simulations by someone and Sally Floyd. Includes ns-2 simulation scripts.
- ns-2 doesn’t even support using FullTcp over its wireless models. But my NSC stuff does work.
- Support packet tracing of sequence number so we can generate nifty graphs with my NSC stuff. It’s a bit of a hack due to assumptions and abstractions ns-2 has, but seems to be basically right.
- Still not too sure about the results of my wireless simulations. Need to form a good comparison, put some graphs on a webpage or something and compare.
Allman and Falk write in their 1999 paper “On the Effective Evaluation of TCP”:
… This mechanism [delayed acks] is widely deployed in real TCP implementations and therefore we recommend that it be used in future TCP experiments. Exluding this mechanism does not necessarily yield useless data and may be quite useful as a comparison to the delayed ack case. However, turning off delayed ACKs without carefully considering the costs and the impact on the conclusions drawn should be avoided.
Good advice. I’ve found that Linux doesn’t even seem to allow turning delayed ACKs off, for example. I have yet to find a reasonable TCP implementation that doesn’t do delayed ACKs.
Gurtov and Floyd’s paper “Modeling Wireless Links for Transport Protocols” published in 2003 has commentary on the type of TCP to be used; when talking about previous researchers using unrealistic models they comment:
Using older deprecated TCP versions. Although vast majority of TCPs in the Internet use either SACK or NewReno for loss recovery, many simulation studies still use Reno or even Tahoe TCP. Possibly, the use of old TCP versions is due to the lack of support for modern TCPs in network simulators.
Quite true. Gurtov and Floyd publish their simulation scripts, in which they use ns-2’s SACK TCP agent. The problem is, it doesn’t have delayed ACKs turned on. Huh. And Sally Floyd and Mark Allman both work at ICIR.