Weekly Report -- 25/05/2012
Finished documenting the current versions of my anomaly detection python scripts. Tested them against a few more noisy data sets, where they performed pretty well, so reasonably happy with the technique so far.
However, the ARIMA modeling component is quite sluggish so starting trying to figure out ways to improve the performance. At the moment, I simply call into R and tell it to reapply the original model to the last N measurements with to little to no control over the math that is performed during that process, e.g. R could be calculating statistics that I don't need.
After carefully poring over lots of R code and papers on ARIMA modelling, I've been able to start developing my own Python version of the ARIMA model function. Managed to get correct values for the residuals when using a simple (1,1,0) model but more complicated models required a surprising amount of additional math. At this stage I'm not sure if my efforts will end up improving the performance of my event detection to any significant degree, but at least I understand the underlying math a lot better!