Weekly Report -- 01/06/2012
After a couple of days of mind-numbing mathematics, I managed to nail my implementation of the ARIMA modelling. Instead of taking nearly 40 minutes to run, the ARIMA modelling step now takes less than 4 minutes. This makes it much easier to experiment with a variety of time series as I don't have to sit around waiting for ages when I want to try something new.
Noticed my current setup was producing a few false positive events when testing against a time series of http outbound flows. Spent a decent chunk of time tweaking thresholds and seeing if there was any simple solution. I've managed to make some improvements but still not getting ideal results. Started working on some manual classification of a few time series to see if we can apply some machine learning to a training set and find some suitable thresholds using actual science rather than me just fudging the values until the results look good.