=========================================================================== IMC'09 Review #52A Updated Thursday 16 Jul 2009 3:27:29pm CEST --------------------------------------------------------------------------- Paper #52: Investigating the Impact of Service Provider NAT on Residential Broadband Users --------------------------------------------------------------------------- Overall merit: 2. Top 50% but not top 25% of submitted papers Reviewer qualification: 5. I am an expert on this topic Novelty: 3. Incremental improvement ===== Paper summary ===== Paper presents an analysis of consumer-ISP traffic over a handful of days in January 2009, from which the authors extrapolate the distribution of inbound and outbound TCP and UDP connections and surmise the likely demands placed on a 'carrier grade' or 'service provider' NAT box. Authors observe that a high proportion of consumer-initiated UDP flows are extremely short-lived, thus overloading the NAT unless shorter timeouts are used in the NAT's session tables. ===== Exciting ===== I like the topic, but there's nothing particularly exciting. This kind of study has been done before in various forms, and the authors do not provide any indication of a truly novel insight. The specific mention of Service Provider NATs provides a currently relevant context, but fundamentally the issues also pertain to any use of NAT and have been discussed before (even if not frequently in peer-reviewed fora). ===== Run-of-the-mill ===== Unfortunately, the paper isn't a novel or unique contribution to interpretation of typical consumer traffic patterns. The authors were unable (or unwilling) to do sufficient deep-packet inspection (or statistical analysis of flows) to identify a lot of their UDP traffic. The authors claim a new "technique" of reducing the NAT session table timeout for UDP flows, but this is an old idea. The problem of NAT tables filling up is well known in e.g. the online gamer communities (typically evidenced by forum threads discussing intermittent internet access when 'server discovery' phase hammers their net connection with thousands of UDP probes over a few minutes, causing their home NAT box to 'fill up' and block new UDP probes for a few minutes). ===== Limitations ===== The analysis of what applications are causing what traffic patterns seems unnecessarily limited and poorly explained. (e.g. you explain how you identify Bittorrent UDP packets, but give no idea how you identified "Steam server browsing protocol" mentioned on p.4) There seems like a lot of "unknown" UDP traffic in Table 3. Couldn't that be due to game traffic - either server discovery or game play? If not, why not?) ===== Comments to author ===== I expect more from a paper on a well-known topic. Submitting in a short-paper category does not free you from the responsibility to research the topic and create a suitably archival paper. The topic is generally reasonable (in so far as you're aiding SPNAT developers and users to provision their devices). But your main conclusion appears to be a relatively trivial "use a short initial timeout on UDP flows, because consumer connections have lots of short-lived UDP flows". (I call it trivial because this observation about consumer UDP flows has been well known in the online gamer community for many years, if not also elsewhere.) You mention that your choice of a 10 second initial timeout is arbitrary - I believe you _ought_ to have actually performed the next step of evaluating a range of possible initial timeout values, and evaluated the consequences. Table 3 seems to have an amazingly high level of "unknown" UDP flows (by bytes and 'servers', even if not by number of sessions). I believe a basic analysis of SPNAT requirements dictates that you explore the possibility of this being due to online game traffic, VPN tunnels, encrypted non-TCP applications, etc. For example, in respect of the impact of online games, Googling "game server discovery NAT" comes up with a number of papers of possible interest, including: - G. Armitage. "Optimising Online FPS Game Server Discovery through Clustering Servers by Origin Autonomous System", in ACM NOSSDAV 2008 http://simula.no/~griff/nossdav2008/03-08.pdf (which appears derived from a range of work at http://caia.swin.edu.au/genius/papers.html) - T. Henderson, "Observations on game server discovery mechanisms", Netgames 2002, http://portal.acm.org/citation.cfm?doid=566500.566507 Similar papers offer insights into how/why consumer connections might see thousands of short-lived UDP flows in intense bursts over multi-minute periods. (There's also the game-play flows -- client-server UDP traffic that can generate MBytes of traffic over longer periods when a player connects to a game server and plays.) I would also suggest considering the impact of SPNAT on the ability of consumers to _host_ online multiplayer games. There are >30K Counter-Strike:Source servers on the Internet today, hosted by people everywhere. They need to set up port-forwarding. You mention that SPNAT would make hosting servers problematic. But I think you ought to make a reasonable estimate of the need for consumers to 'host servers' wrt the online gamer market. =========================================================================== IMC'09 Review #52B Updated Thursday 11 Jun 2009 7:01:24am CEST --------------------------------------------------------------------------- Paper #52: Investigating the Impact of Service Provider NAT on Residential Broadband Users --------------------------------------------------------------------------- Overall merit: 1. Bottom 50% of submitted papers Reviewer qualification: 3. I know the material, but am not an expert Novelty: 3. Incremental improvement ===== Paper summary ===== The paper uses DSL traces to evaluate current best practice in NAT timeout and proposes a change in the timeout mechanisms used to optimize the amount of NAT state necessary. ===== Exciting ===== NAT is being used widely in wired and wireless networks and optimizing NAT gateways is an important task. ===== Run-of-the-mill ===== Straight forward analysis with a straight forward adaptation of best practice to address the discovered issues. ===== Limitations ===== NO ===== Comments to author ===== The paper is well written and sound. The main drawbacks are little surprising results. For me the most surprising information was how slow NAT state gets timed out according to best practice. What would make this a much more interesting paper if the authors could provide some information on how frequently best practice is followed in the Internet today. That would also motivate the importance of the problem more strongly.