Had a chat to Richard Sanger about adding web10g data to the AMP
throughput test and got him set up with the current code.

Finished porting the main part of the http test to the new AMP. It runs
fine as a standalone program, but I still need to add the new style of
reporting to be able to send test data back to the collector.

Started packaging the new AMPlet code for Debian to make installation and
distribution easier for testing. It's a little bit more involved than I
thought it would be, as the build process has become a lot more automagic
since I last created a package from scratch.

Finished up the 513 marking (eventually!) and released the marks to the students.

Released a new version of libtrace -- 3.0.17.

Started working on releasing some new public trace sets. Waikato 8 is now available on WITS and the DSL traffic from our 2009 ISP traces will hopefully soon follow. In the process, I found a couple of little glitches in traceanon that I was able to fix before the libtrace release.

Decided that our anomaly detection code does not handle time series that switch from constant to noisy and back again particularly well. A classic example is latency to Google: during working hours it is noisy, but it is constant other times. We detect the switch, but only after a long time. I would like to detect this change sooner and report it as an event (although not necessarily alert on it). I've started looking into an alternative method of detecting the change in time series style based on a pair of sliding windows: one for the last hour, one for the previous 12 hours before that. It is working better, but is currently a bit too sensitive to the effect of an individual outlier.

What happened last week:

• Upgraded our teaching cabinet switches/routers to the latest firmware as recommended by Juniper/Cisco TAC. They are all now running the crypto (domestic) versions which means SSH will work on all of them now.

• Picked up Wand's new toy, a Juniper M320 router from a warehouse in Auckland. It is currently sitting in TSG's storage room until we have arranged space/power from ITS in the G block datacentre to host it.

Completed a one node installation of openstack. Created an instance and brought it down.

Got back into things last week, but I forgot to do my report.

Since then I have had a few issues, Joe has been hard at work merging everything that was worked on during his time at Wellington, and I have had a few issues with keeping up with his updates. I've got stuck with a couple of bugs that he had dealt with in commits I hadnt pulled..

But, rfserver now stores all the routes learned by the switches, making it effectively the third (actually at least the 4th) time this information is stored by RouteFlow. Such is the elegance of the design..

I need this so that if a switch goes down I can add the current set of rules back onto the switch when it comes back up. But at the moment it is just storing them and printing them out every time it learns a new one.

Next step is to actually update the switches in a way so that deletes can never arrive before the route they are deleting, or vice versa.

Spent a large portion of this week getting familiar with OpenFlow version 1.2, taking the simple_switch.py Ryu comes with as an example which is designed for version 1.0 and rewriting with the features of 1.2 in mind. This was mainly to help me get familiar with it.
Otherwise this week has been primarily a "work out what next" week so I had a look into loops in a layer two network and how using OpenFlow I could prevent broadcast loops.

Libtrace 3.0.17 has finally been released.

This release adds some new convenience functions to the libtrace API and fixes a number of bugs, many of which have been reported by users.

The major changes in this release are:
* Added API functions for getting the IP address from a packet as a string.
* Added API functions for calculating packet checksums at the IP and transport layers.
* Fixed major bug where the event API was not working with int: inputs.
* Fixed broken checksum calculations in tracereplay.
* Fixed bug where IP headers embedded inside ICMP messages were not being anonymised by traceanon.
* Added API support for working with ICMPv6 headers.

The full list of changes in this release can be found in the libtrace ChangeLog.

You can download the new version of libtrace from the libtrace website.

Worked with Shane to get smokeping data exported to the web API to graph.
In doing so, found and fixed a few issues with the binned timestamps that
meant the data always looked old, even when it was up to date.

Started working on adding a new graph type to properly show smokeping data
(and any other data type where we have plenty of measurements, such as the
AMP latency tests). While doing this I found that the graphs would never
quite show the most recent data if I zoomed in with the select box, so I
started to investigate that. I don't want to jump to conclusions
and attribute the error to the library, but it appears that envision.js is
possibly off by one when calculating the selected area. This combined with
the fairly aggressive binning of the summary graph means we lose a fair
chunk of data off the end (one bin). Will need to look more into this and
see if it isn't actually a problem with the data or its presentation.

Made a few small updates to AMP for the NLNOG RING and got some good
feedback from them and others who are using AMP.

Completed an initial Internet Simulator run, and began to read up on how to produce graphs from the output data.

Ran daily scamper runs on Yoyo to optimise virtual memory and probes per second usage by modifying the structure of the experiment. I ran UDP and TCP souce port MDA without ICMP echo mode. Hopefully it won't be necessary to run UDP and TCP separately. Limited windows to 60 or unlimited. Limited PPS to 300 or 200. Limited total probes to 15000 or 65000. The aim was to get about 5000 addresses processed per day so that the same can be done on the CAIDA Ar

Fixed the bugs in the anomaly_ts / eventing chain that I introduced last week. We're back reporting events again on the web dashboard.

Wrote ampy modules for retrieving smokeping and munin data from NNTSC so that Brendon could plot graphs of those time series. Doing this showed up some (more) problems in the graphing which Brendon eventually tracked down to being related to how aggregation was being performed within the NNTSC database.

Spent a large chunk of my week marking the 513 libtrace assignment. It is a much bigger class than previous years (over 30 students) so it was pretty time consuming to mark. In general, it was pleasing to see most students had gotten the basics of passive measurement worked out and hopefully they got some valuable experience from it. My biggest disappointment was how many students didn't read the instructions carefully -- especially those who missed the requirement to write original programs rather than blindly copying huge chunks of the example code.