Impact of the Copyright Amendment Act at NZNOG
At NZNOG 2012, I presented some slides showing a decrease of P2P traffic following the Copyright Amendment Act coming into effect in New Zealand in September 2011. By contrast, the same analysis showed a significant increase in Tunneling, FTP and Remote Access traffic. These results generated a lot of interest, so I am using this blog post to discuss our methodology and results in more detail.
The slides from the talk can be reached from here.
WAND has a passive monitor located within the core network of a New Zealand ISP. We use this monitor to capture traces of all the traffic going to and from the ISP's customers. For privacy reasons, we discard most of the packet contents - we keep the packet headers up to and including the transport header plus four bytes of application payload. This is sufficient for libprotoident to be able to classify most of the traffic correctly.
We performed three captures: one in January 2011, one in mid-September 2011 and the third in January 2012. Each capture was at least a week in length. Storage requirements unfortunately prevent us from taking captures more frequently.
Each of the captures was run through libprotoident, with a filter applied to only include residential DSL traffic. This removed large corporate customers and casual wireless users from the analysis, as these users are likely to have a different traffic profile entirely. In future, we may look at these user classes as well to see if the impact of the CAA is different in such environments.
Using libprotoident, details about each connection that begins during the time covered by the capture are reported and written to a file for subsequent analysis. Amongst other things, libprotoident reports the application protocol being used by the connection (e.g. HTTP, BitTorrent, SSH) and the number of bytes sent and received for the connection.
Many application protocols serve a similar purpose or have similar characteristics. For example, BitTorrent, Gnutella and EDonkey are all peer-to-peer file sharing protocols. In terms of our analysis, it makes sense to group all of these protocols into a single category. Each protocol supported by libprotoident is assigned to a category to support this kind of analysis. All results presented at NZNOG were using the libprotoident categories. Categories that did not produce significant quantities of traffic were ignored in the final analysis.
- All Traffic - total traffic across all categories.
- Web - Web browsing traffic, including YouTube and HTTPS.
- P2P - Peer to Peer file sharing, including BitTorrent, Gnutella and EDonkey
- P2P Structure - P2P network maintenance traffic, including BitTorrent DHTs and peer exchanges
- Streaming - Streaming media that is NOT web-based, including RTMP.
- Unknown - Anything that libprotoident cannot classify. A lot of encrypted P2P traffic falls into this category, as the P2P application uses a custom encryption approach.
- Remote - Protocols for logging into remote machines, including SSH.
- VOIP - Voice over IP, including Skype.
- Tunneling - Any protocol used to create a secure tunnel between two hosts. Includes VPNs and HTTP Tunnels.
- Newsgroups - Usenet, Easynews etc. One of the primary means for sharing torrent files etc.
- Files - Non-P2P file transfer protocols, including FTP.
- Encrypted - Well-defined encryption protocols, such as SSL and TLS, as long as the application does not fall under another category. For example, HTTPS is not included because it fits Web better instead.
So far, we have only looked at the number of bytes downloaded and transmitted by residential DSL users. All numbers are given as values relative to the value observed in January 2011. This helps protect the anonymity of the ISP, as well as enabling us to compare changes across a similar scale.
Overall, the total amount of traffic downloaded has decreased slightly from January 2011 to 2012. Web traffic (which constitutes most of the downloaded traffic) is also reasonably steady - there was a modest increase from 2011 to 2012.
By contrast, P2P, P2P structure, Unknown, Newsgroups and Encrypted have all decreased massively from their January 2011 levels. Interestingly, each of these categories can be tied to the illegal downloading activities targeted by the CAA. P2P and P2P structure are obviously related, Newsgroups are a common source of torrent files and the Unknown and Encrypted categories were strongly suspected of containing a significant quantity of encrypted P2P traffic.
Even more interestingly, Remote, Tunneling and Files experienced similarly large growths in the amount of traffic downloaded by DSL users. This is probably indicative of people changing their approach to downloading copyrighted material. Instead of participating in file sharing on their home machines, it has become more common for people to use machines based in other countries and ship the file back home via another protocol. This might be via SSH, VPN or FTP, for example, which are all covered by the growing categories.
Similar trends are observed when looking at traffic transmitted by the DSL users. Categories associated with P2P file sharing have seen much less traffic compared with January 2011, whereas Tunneling, Remote and Files have soared.
It should be noted that although Tunneling has grown significantly, the overall amount of Tunneling traffic is still much less than the total amount of P2P traffic. But the sudden changes in application protocol usage are still very noteworthy and suggest that the CAA has had a major impact on people's Internet usage.