Ok, so setting mtu in simulation basically works. As long as you set packetSize_ on a per-agent basis just after that agent has been created.
Anyway.
Really want a nice way of getting the send time in my SIP simulation. Still got my ugly hack in my ns on voodoo-iii. Ugh. The simulation is working OK tho. MTU effects things a lot. Random seed does the same. These are my biggest questions at the moment.
I guess more work on this will have to wait to Sunday (hah) or Monday (more likely).
For some reason setting the window size is broken. Initially I was trying to set it to something stupidly large (4MB or something). Then I tried a more conservative 32k. Both seemed to have it end up being 96 bytes. I thought maybe the MSS was really low, but it was 1460, it was just the window size that was low.
Need to investigate why this is happening. I’m tempted to remove changing the window size on Linux, it seems like fairly useless functionality given the dynamically growing buffers and such that Linux (2.4 and 2.6) implement.
Allow setting the MTU via TCL/simulation script for NSC stacks.
Clean up FreeBSD NSC code. Remember to name all nsc stuff with a nsc_ prefix. Possibly warn when globalising anything with an nsc_ prefix (Perry’s suggestion, a good one).
Fix simulator < -> cradle interface to be nicer, up to date, consistant, etc.
FreeBSD seems to go into an infinite loop or something. Maybe it panic’d. I dunno. Anyway, there are some errors after it initialises 3 or 4 instances. Heh. Looks like something isn’t globalised. Slowly working on getting this sorted….
Update: No, I had globalised max_stack like I had cur_stack earlier. Should really give these vars a better name. Anyway, once fixing that, it all works nicely (since in hunting for this bug I had gone and made sure everything else was globalised properly).
Getting some interesting results… I should validate that it is working properly now.
I was interested in how long some of the lines output by my parser are.
find . -type f -iname '*.parsed.c' | while read f; do perl -ne 'BEGIN { $m=0;$l=0; } ++$l;
if($m<length ($_)){$m=length($_);$ll=$l;}END { print "$m $ll "; }' < "$f"; echo " $f";
done | sort -n
The output?
... [snip] ...
190100 7566 ./netinet/tcp_usrreq.parsed.c
206430 1696 ./crypto/md5.parsed.c
212942 4916 ./freebsd/usr/obj/usr/src/sys/GENERIC/vnode_if.parsed.c
275951 2257 ./net/zlib.parsed.c
375421 7203 ./netkey/key.parsed.c
451426 3287 ./netkey/key_debug.parsed.c
775952 5858 ./netinet6/esp_core.parsed.c
1320915 7451 ./net/ppp_tty.parsed.c
1835930 496 ./crypto/cast128/cast128.parsed.c
3912425 1878 ./crypto/des/des_setkey.parsed.c
3912932 1858 ./crypto/des/des_ecb.parsed.c
7822597 1914 ./crypto/blowfish/bf_skey.parsed.c
I had to check this.
head -n 1914 ./crypto/blowfish/bf_skey.parsed.c | tail -1 > sam.txt
/home/stj2/900/nsc/freebsd5$ls -lah sam.txt
-rw-r--r-- 1 stj2 wand 7.5M Apr 28 16:28 sam.txt
/home/stj2/900/nsc/freebsd5$wc -l sam.txt
1 sam.txt
/home/stj2/900/nsc/freebsd5$head -c 180 sam.txt
typedef const BF_KEY _GLOBAL_40_T; static _GLOBAL_40_T global_bf_init[NUM_STACKS] = { { {
0x243f6a88L , 0x85a308d3L , 0x13198a2eL , 0x03707344L , 0xa4093822L , 0x299f31d
NB: artificial newlines added here and there for readability
I have made progress with globalising __section__ stuff. It seems to work in my simple test case. No to try it on actual code. It is very limited at the moment, but I seem to have the basic method worked out. I think.
I just ran it over freebsd. It possibly worked. Well, at least enough for it to compile and link. No testing of working code just yet, but compiling is a good start.
Looks like the parser has successfully worked on Linux 2.6 and FreeBSD. Excellent. Hopefully it is also producing valid code! That does seem to be the case.
One more bit of testing, then I’ll endevour to break it again. Before that I’ll do a commit, though.
Debugging parsers is a horrible practice. Try to avoid writing your own Bison-based parser if you can. If you do end up in a situation with shift/reduce conflicts, reduce/reduce conflicts, and inputs that wont parse, your options for “debugging” are as follows:
First of all, make sure you have rudamentary error reporting. I have some code like the following done the bottom of my bison input file:
static void
yyerror(const char *s)
{
fprintf(stderr, "Line %d: Char %d: %s (last token was '%s')\n",
lineno, charno, s, yytext);
exit(1);
}
#if YYDEBUG
extern int yydebug;
#endif
And the all-important:
#define YYERROR_VERBOSE
defined at the top.
Second, at the top of your Bison input file, in the C-code section, add the following code:
#define YY_DEBUG 1
Now run your parser on the input that makes it break. A whole lot of debugging output will be printed on stderr. You can examine this to see what the parser is actually doing, you get to see how it reduces and shifts based upon the input. It is a good idea to try and produce a very small input file for this step else the output will be very long.
These mean that your grammar is wrong. Reduce/reduce mean it is very wrong, shift/reduce might be expected (the “dangling else” is the classic example).
Run bison with the -v or --verbose command-line argument. This tells bison to output the state machine into a file (it will be named something like mygrammar.output, see the man page for details).
At the top of the files it will say which states are erroneous (which have shift/reduce and reduce/reduce conflicts). Go to each state my searching in the file for the regex ^state <number>$. In some cases (generally reduce/reduce conflicts, YMMV) it is impossible to tell what rules are wrong from looking at just the one state. You might want to look at what states lead up to that state. To do this, search for the string go to state <number>$.
Understanding what everything in the output file means is left as a challenge for the reader.
Added in better support for __attribute__ gcc syntax into the parser. This was quite difficult, as the rules are fairly lax as to where attributes can be used. However, gcc’s documentation on the attribute syntax was fairly good; this helped quite a lot.
At one point I resorted to looking at the grammar for the C parser in gcc itself. It is rather complex, which I believe was what I found the first time I looked at it. I am once again convinced that it would not have been a good starting point for my project. I’m sure it would be possible to use and understand eventually, but the learning curve would be rather large.
Then again, it is much more correct and complete than anything I can/will make myself. Anyway, I have largely managed to get the parser working with attributes. I added some test cases to the parser as well. Need more of these, but so far so good.
I am working on actually getting it to compile FreeBSD now. Still doesn’t quite work yet. Maybe it will soon? No idea. Anyway, more stuff to fix right now…
The idea behind all this is full globalisation support for FreeBSD. My goodness, what a horrible word. Ignoring that for the moment, the thing that has been standing in the way of this is linker sets. Globalising such sets is a lofty goal it seems. Or a challenging one. We’ll see how it goes.
Or it seems that way; sim-25 shows that it is getting very little throughput. Should investigate this; might be something to do with it being built on the new voodoo? Anyway, needs to be fixed. Maybe after I’ve got FreeBSD working.
And now my parser is fixed. It was actually a fairly simple fix, all the code to work with self referential static locals was there, it is just that the function name we were in wasn’t being passed through to the global checking code. Anyway, this fix is now committed along with a testing framework for the parser.
Updated scons to a pre-release version (RELEASE 0.96.90) that seems a lot faster. Nice one scons developers. Good work.
Now working on simulating as much as possible, got 6 simulation clients running on voodoo-iii (it’s a holiday, I’m the only one using the computer).
It doesn’t seem to handle static locals as well as it should. This code:
struct test_struct
{
int size;
int num;
};
typedef struct test_struct test_st;
test_st sam = { sizeof(sam), 1 };
void func()
{
int bob;
static test_st saml = { sizeof(saml), 1 };
static char dsa;
}
Will break if you try and “globalise” the symbol func/saml. sam will be globalised correctly. So it seems that the logic we use when doing a global isn’t quite the same for a static local; I would have expected them to be handled in the same way. Also, if the declarations of bob and dsa in func() are not there, it does not seem to pick up saml. And it still checks for globals of name saml and dsa currently, it should not check them as globals, just static locals.
This was testing on carceri, which I believe is up to date.
Research – TCP/IP over Air Links – big list of TCP performance papers over wireless. Should check how much simulation these do when I’ve got some spare time.
“The Effects of Systemic Packet Loss on Aggregate TCP Flows” – talks about systemic packet loss, like 0.01% on the high speed Abilene network. This means they were only getting about 25% of available bandwidth, people use multiple flows to attain better throughput. They analyse this and the fairness of it and propose a modified TCP that is fairer but still ends up using more bandwidth.
“A Stochastic Model of TCP/IP with Stationary Random Losses” – lots of formulas and maths and analytical stuff. Um, yeah.
TCP in Wireless Environments… – summary paper talking about different versions of TCP and how they help for the wireless realm.
Abstract – This changed, but was still very similar. I think the first sentence or two was exactly the same.
Introduction – First few (5, more?) paragraphs are exactly the same in both. Cut out a couple of the later paragraphs and changed the description of the rest of the paper somewhat.
Related Work – Exactly the same, except new paper had the freebsd/omnet bit added (which we want anyway).
Order of sections changes from now.
Architecture and Design – The introduction starts of the same, then has some minor differences. Diagram updated to be better in the section introduction and new paragraph added. Next subsection the same exact diagram from older paper removed (the replacement was a newer one in the section introduction). Next two subsections exactly the same. Old paper still has more though: “Network Stack Integration Process” and “Errors Due to Integration” were removed for the newer paper.
Validation – Section intro the same, most of the section exactly the same apart from minor formatting differences. Last paragraph looks to have been reworked for newer paper.
Performance – Some minor differences in the intro (from Tony’s suggestion) . Largely the same again until we get to memory performance, where I look to have rewritten or reworded a paragraph. Had to rework more because the simulation referred to here no longer exists in the new paper.
These sections don’t exist in the newer paper:
Protocol Development – It’s of dubious importance for a simulation paper. Well…. Maybe that needs some thought. It’s an application of simulation; the debugging and testing field. Hmm.
Simulations – This section was always quite mixed up because there is no firm conclusion and I kind of sit on the fence. Ideally, it wants to show some simulations and conclude something definite (ideally, that you should be using my simulation stuff for more accurate simulation). It may just have to conclude that ns-2 is a good ballpark simulator for most scenarios but if you want to test different tcp implementations in a scenario (perhaps like those papers that compare newreno and reno and stuff in a situation? needs some though) then NSC is great. Maybe. A stronger simulation/set of simulations here would be good.
And then we sum up with the conclusions section in both.
Conclusions – Some of this is removed because we removed two sections; these sections are summarised in the old paper. Otherwise it is exactly the same.
So where does this leave me? That’s the question. Hmmmmmmmmmm………….
Having the electrician turn power off a couple of times wasn’t so fun. Especially when the computer this blog runs on doesn’t come back up both times. It seems that the 80gb disk has just failed. BIOS wont pick it up anymore. Bugger. Oh well, at least it wasn’t the disk with all the important stuff (like this blog!) on it.
On another note, my backups of this blog have been failing for some time. Should fix that ASAP.
Tony pointed me to the paper A simulation-based performance evaluation of Tahoe, Reno and Sack TCP as appropriate transport protocols for SIP
Does some comparisons of Tahoe, Reno and Sack in a specific domain (SIP stuff). Uses ns-2.
They compare the TCP versions to SCTP briefly and say that from their findings TCP with SACK is about the same as SCTP, though more research is required before concluding this. An interesting point here is that they do not seem to address the fact that SCTP allows multiple channels and possibly SCTP-PR (partial reliability) which both could help in their scenario; they are just doing some basic signalling where one signal shouldn’t block another (but does with a byte-stream protocol like TCP). The section on SCTP is so short I am unsure how they used SCTP, it looks like they just used it as a simple TCP replacement (eg. as a byte-stream). In that case, of course it’s the same as TCP with SACK…
Back from the conference and holiday in Boston. Guess I’m going to have to figure out what to work on…
Have a progress report on the 15th: Friday. I had planned to sort that out this week, and I will do so. However, that isn’t about to take 4 days. So I really need to figure out what else I can work on. Hmm.
Powered by WordPress