1School of Computer Science and Engineering, Northeastern University, Shenyang, China
2Department of ICE, Beijing University of Posts and Telecommunications, Beijing, China
3Institute for Network Sciences and Cyberspace, Tsinghua University, Beijing, China
Copyright © 2017 Fuliang Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Internet Protocol (IP) is used to identify and locate computers on the Internet. Currently, IPv4 still routes most Internet traffic. However, with the exhausting of IPv4 addresses, the transition to IPv6 is imminent, because, as the successor of IPv4, IPv6 can provide a larger available address space. Existing studies have addressed the notion that IPv6-centric next generation networks are widely deployed and applied. In order to gain a deep understanding of IPv6, this paper revisits several critical IPv6 performance metrics. Our extensive measurement shows that packet delay and loss rate of IPv6 are similar to IPv4 when the AS-level paths are roughly the same. Specifically, when the link utilization exceeds a threshold, for example, 0.83 in our study, variation of packet delay presents a similar pattern with the variation of link utilization. If packet delay of a path is large, packet-loss rate of that path is more likely to fluctuate. In addition, we conduct a first-ever analysis of packet reordering in IPv6 world. Few IPv6 probe packets are out-of-order and the reordering rate is , which is much lower than that of 0.79% in IPv4 world. Our analysis consolidates an experimental basis for operators and researchers of IPv6 networks.
As a communication protocol, IP provides an identification and location system for computers on the Internet. Internet Protocol version 4 (IPv4)  has been used successfully for the past years. As the successor of IPv4, Internet Protocol version 6 (IPv6)  was developed by the Internet Engineering Task Force (IETF)  since 1998. Currently, IPv4 still routes most Internet traffic . However, due to the predictable exhaustion of IPv4 addresses , the transition to IPv6 is imperative, because, compared with IPv4, IPv6 can provide a larger available address space.
To try and encourage the adoption of IPv6, more than 400 organizations and institutions participated in the Word IPv6 Day on 8th June 2011 . To further promote the deployment and development of IPv6, World IPv6 Launch was held on 6th of June 2012 . Google has announced that over 10% of users access Google through IPv6 . CAIDA has shown that, from January 2014 to January 2015, the number of IPv6 ASes increased by 23% and the number of links between them increased by 29% . Existing studies have addressed that IPv6-centric next generation networks are widely deployed and applied [10–12]. With the development of IPv6, applications such as video conference, IPTV, VoIP, and online games will become more and more prevalent in IPv6 networks, so keeping network availability as well as good performance is crucial. Investigating the performance of IPv6 network is the basis of guaranteeing the network with a good performance. Therefore, many studies have been performed to analyze IPv6 network performance, as well as figure out the differences between IPv4 and IPv6 [13–17]. However, our understanding of IPv6 network cannot catch up with the speed of its development. A revisiting of current IPv6 network performance is needed.
In this paper, we revisit IPv6 performance and make a comparison with IPv4 from several metrics, including network reachability, packet delay, packet loss, and packet reordering. A preliminary version of this paper has been published in ICNC 2016 . Our contributions and main findings are summarized as follows.
() We design a performance probing method for IPv6 network based on OneProbe , a new TCP tool for reliable and metric-rich path monitoring. Using this method, we can persistently observe and analyze performance of IPv6 network from different angles. Analysis results can help us fully understand IPv6 networks and provide an essential basis for IPv6 network operators and researchers.
() We conduct a comparison study on IPv6 packet delay (round trip time, i.e., RTT). The average RTT of IPv6 is 247.71 milliseconds from our probing host to the remote servers, while the average RTT of IPv4 is 244.48 milliseconds, which is similar to the value of IPv6. However, in most cases, IPv6 RTT is higher than IPv4 RTT in Asia Pacific region. We explore the reasons for the difference by dividing the AS-level paths into three parts, that is, domestic-as, middle-as, and target-domestic-as. We find that the connectivity between middle-as and target-domestic-as and the length of middle-as are the dominant factors that cause the difference in RTT between IPv4 and IPv6. In addition, in our measurement, we find that when the link utilization exceeds 0.83, variation of packet delay presents a similar pattern with the variation of link utilization.
() We make a comparison analysis between IPv4 and IPv6 in packet-loss rate. The average packet-loss rate of IPv6 is 0.25%, while the average packet-loss rate of IPv4 is 0.33%, which is a bit higher than that of IPv6. We also investigate the relationship between packet delay and packet-loss rate. Packet-loss rate of a path is more likely to fluctuate when the packet delay of that path becomes large.
() We conduct a first-ever analysis of packet reordering in IPv6 world. We find that the average packet reordering rate of IPv6 is , while the average packet reordering rate of IPv4 is 0.79%. The reason for the difference is that fragmentation in IPv6 is not encouraged and only the hosts can conduct packet fragmentation in IPv6 network, which could reduce the possibility of packet reordering.
The remainder of the paper is organized as follows. The related work is presented in Section 2. We describe the methodologies used to measure IPv6 network performance in Section 3. Section 4 presents our analysis results, including reachability, packet delay, packet-loss rate, and packet reordering. Our conclusions are presented in Section 5.
2. Related Work
Related work is summarized from two aspects. We begin with discussion on packet delay and packet-loss analysis in IPv6 world. Then we show the studies that have been conducted to characterize packet reordering in IPv4 world.
Wang et al.  compared IPv6 and IPv4 performance from the perspective of end users and found that it still has headroom to improve IPv6 performance, including increasing its connectivity and reducing its packet-loss rate. Muniyappa  carried out a simulation-based study comparing the use of IPv4 versus IPv6 on simple campus networks and found that performances of IPv4 and IPv6 are almost the same and the difference is negligible. Shiwani et al.  measured throughput, delay, and jitter of IPv6 on a test-bed implementation. Zhou et al.  found that IPv6 paths have a higher delay and loss than their IPv4 counterparts, mainly due to its poor performance in IPv6-in-IPv4 tunnels. Nikkhah et al.  and Dhamdhere et al.  found that IPv6 delay is similar to IPv4 performance when the AS-level paths are identical. Czyz et al.  concluded that IPv6 RTT is getting better according to a long-term observation. Livadariu et al.  revealed that IPv6 routing system is less stable than IPv4, and IPv6 performance is comparable to that over IPv4. Bajpai and Schönwälder  found that TCP connection establishment times of IPv6 are relatively higher than that of IPv4 in the cases when CDN caches are present for IPv4, but are largely absent for IPv6. Lin et al.  designed and implemented a high-performance IPv6 address lookup engine in GPU-accelerated software routers. In this paper, we revisit several critical performance metrics of IPv6 network, including reachability, delay, and packet-loss rate. In addition, we also explore the reasons for the differences between IPv4 and IPv6. Furthermore, we analyze the relationship between delay and link utilization, as well as capture the variation pattern and find a fitting model for the delay when it fluctuates greatly.
Packet reordering in IPv6 world is less understood. However, packet reordering has great influence on transmission efficiency. Stevens  found that when the TCP receiver gets out-of-order packets, it sends duplicate ACKs to trigger fast retransmission algorithm at the sender, which may cause unnecessary retransmission in transport layer. Laor and Gendel  found that packet reordering results in limited speed of packet transmission as well as throughput degradation. Wang et al.  proposed a single-point reorder-judging algorithm to measure packet reordering and found a threshold to help distinguish reordering and loss on some heavily reordering paths. Zhou and Van Mieghem  analyzed the end-to-end packet reordering by tracing UDP packets between 12 test-boxes in RIPENCC and found that packet reordering has a significant influence on the UDP performance but does not have a significant impact on UDP delay. Bohacek et al.  mentioned that packet reordering is generally ascribed to transient conditions, pathological behavior, and erroneous implementation. Bennett et al.  refuted the widely held belief that packet reordering in the Internet is an uncommon behavior caused by incorrect or malfunctioning network components. Przybylski et al.  discussed the factors affecting ordered packet delivery and revealed that the main protocols of modern gigabit networks cannot cope well with reordering.
In addition, packet reordering also greatly affects network intrusion detection and prevention systems that need to maintain the state of each flow [32–34]. Out-of-order packets must be buffered until all the packets are reassembled in the correct order. However, the buffer may overflow when the packet reordering occurs frequently. The attackers may also craft ambiguous traffic to breach the defense of such systems. For example, an attacker may send TCP segments with different payloads for the same sequence number space. In this paper, we conduct a first-ever study on IPv6 packet reordering. This can not only let us know exactly whether IPv6 packets are disordered, but also provide a basis for investigating TCP behaviors in IPv6 world.
In this section, we first take a look at the probing tool, OneProbe. Then we describe our probing method.
3.1. Overview of OneProbe
OneProbe is a reliable and metric-rich path monitoring method based on TCP . We use HTTP/OneProbe, which sends legitimate HTTP GET request in the TCP data probes to induce HTTP response messages to measure delay, packet-loss rate, and packet reordering rate. Each probe of OneProbe consists of two customized back-to-back packets, applied to measure the performance of the forward link. When probes arrive at remote-ends, they will induce remote endpoints to send back two back-to-back packets, which are used to measure the performance of the reverse link. In this study, the values of the forward link and reverse link are merged to evaluate packet-loss rate and packet reordering rate. When using OneProbe, we should set some parameters with appropriate values. As shown in Table 1, refers to the size of the packets that will be captured for performance analysis. represents the size of the probing packets, while represents the size of the response packets. Parameter of is used to control how long the probing will last. We denote the sending rate of the probing packets. In our experiment, we send 2 probing packets in one second. In addition to these primary parameters, we also need a source address, a destination address, and its corresponding URL (Uniform Resource Locator) to launch a OneProbe process.
Table 1: Parameters of OneProbe.
3.2. Probing Method
Based on OneProbe, we design a probing method, which can persistently evaluate the interdomain performance of an IPv6 network. Our probing method mainly contains three phases, that is, obtaining URLs, classifying URLs, and probing.
(1) Obtaining URLs. We first find the URLs that meet the requirements of OneProbe. To guarantee the accuracy of probing, the object of each URL is well over 10 Kbytes . Because of probing such kind of URLs, we can induce sufficient number of response packets. We download top 1 M websites from Alexa  and obtain desirable URLs by crawling these websites. In addition, to make comparison between IPv4 and IPv6, only the URLs supporting both IPv4 and IPv6 access are chosen in our study.
(2) Classifying URLs. For the sake of analysis, we divide the URLs into five groups in terms of the five Regional Internet Registries (RIRs (RIR is an organization that manages the allocation and registration of IP addresses and autonomous system numbers. RIR divides the world into five RIRs, including African Network Information Center (AFRINIC), American Registry for Internet Numbers (ARIN), Asia-Pacific Network Information Centre (APNIC), Latin America and Caribbean Network Information Centre (LACNIC), and Réseaux IP Européens Network Coordination Centre (RIPENCC))), that is, APNIC, ARIN, AFRINIC, LACNIC, and RIPENCC. To this end, we explore the addresses of these destination URLs, and match these address to the prefixes that have been assigned to the five RIRs .
(3) Probing. Our probing host locates in CERNET2 . To avoid cross impact caused by too many synchronous probing packets, we choose no more than 15 URLs from each RIR randomly and launch no more than 15 OneProbe processes synchronously at each time. Note that these URLs belong to different websites and servers. For each RIR, we use OneProbe to probe the destination URLs enabled with both IPv6 and IPv4 for 10 minutes (5 minutes for IPv6, 5 minutes for IPv4) in one polling cycle. That is to say, it costs 50 minutes to probe all the five RIRs one after another. Therefore, we denote an hour as a polling cycle.
4. Performance Analysis
According to the probing methods depicted in Section 3, we conduct a case study on IPv6 performance from four aspects, that is, reachability, delay, packet loss, and packet reordering. In this study, one-week probing data is gathered and used. There are 168 polling cycles during the one-week observation.
Reachability is a basic performance metric for network operation and management. There are many reasons that may cause network interruption, such as routing policy adjustment, security policy control, physical link failure, and intermediate node failure. We use interrupts and interrupt rate to describe the reachability. Interrupts denotes the number of disconnects between the probing client and probed destination URLs. Interrupt rate = interrupts/(the number of polling cycles the number of URLs).
As shown in Table 2, RIPENCC of IPv6 has the most number of interrupts. Interrupt rates of APNIC and LACNIC are lower than those of the other three RIRs, while, as shown in Table 3, interrupt rates of IPv4 in APNIC and LACNIC are higher than those of the other three RIRs. In APNIC and LACNIC, IPv6 performs better than IPv4. However, the total Interrupts of IPv6 are 676, while the number of IPv4 is 437. Overall, the reachability of IPv4 is better than that of IPv6.
Table 2: Reachability of IPv6.
Table 3: Reachability of IPv4.
4.2. Packet Delay
We use round trip time (RTT) to describe the time overhead (i.e., delay) of a packet that goes to the destination and returns to the source.
4.2.1. Results Analysis
Figure 1(a) shows the variation of IPv6 RTT across the five RIRs. RTT of ARIN is approximately 160 milliseconds, and RTT of AFRINIC is around 180 milliseconds. RTT of APNIC and LACNIC are similar to each other. However, RTT of APNIC is more stable than that of LACNIC. The reason for the instability is that some URLs of LACNIC with high RTT were interrupted during our measurement; thus their RTT values cannot be measured in some polling cycles. These unreachable URLs were excluded when calculating the average RTT. As a result, the average RTT of LACNIC in some polling cycles decreases. RTT of RIPENCC fluctuates at 280 milliseconds and it is relatively stable. Results show that RTT of LACNIC, APNIC, and RIPENCC are higher than that of ARIN and AFRINIC. Through further analysis, we find that the number of hops between probing client and destination URLs in ARIN and AFRINIC is smaller than that of the other three RIRs. This is in accordance with the rule that less number of the hops usually causes smaller RTT.
Figure 1: RTT variation across the five RIRs.
According to the aggregation analysis results, we make a comparison between IPv6 and IPv4 in RTT. As depicted in Figure 1(b), in the regions of AFRINIC, ARIN, RIPENCC, and LACNIC, RTT of IPv6 is similar to that of IPv4. This is because, for a destination URL in these regions, the AS-level paths of IPv6 and IPv4 are similar. However, we find that IPv6 RTT is higher than IPv4 RTT in APNIC. We try to explain the reason for the differences by exploring AS-level paths between the probe client and the destination URLs.
4.2.2. Reason Analysis
As shown in Figure 2, we divide AS-level paths into three parts, that is, domestic-as, middle-as, and target-domestic-as. Domestic-as denotes the ASes which locate in the same country/area as the probe client, while target-domestic-as denotes the ASes locating in the same country/area as the destination URLs. The remaining ASes that may locate in any countries or areas are middle-as.
Figure 2: Division of AS-level paths.
We calculate the RTT for each destination URL in APNIC. As depicted in Figure 3, for most destination URLs, such as URL , IPv6 RTT is larger than IPv4 RTT. For 80% of these URLs, the length of IPv6 middle-as is the same as that of IPv4 middle-as. For the rest of the URLs, IPv6 middle-as has one more AS in comparison with IPv4 middle-as. When a probing packet begins to enter middle-as, IPv6 RTT is similar to IPv4 RTT. However, when the probing packet enters target-domestic-as from middle-as, the growth of IPv6 RTT is more obvious than that of IPv4 RTT. After the probing packet enters target-domestic-as, the increase of IPv4 RTT is similar to that of IPv6 RTT. Therefore, the path between middle-as and target-domestic-as causes the difference between IPv6 RTT and IPv4 RTT. That is to say, the connectivity between middle-as and target-domestic-as in IPv4 network is better than that in IPv6 network, resulting in IPv4 RTT being smaller than IPv6 RTT for most destination URLs.
Figure 3: RTT distribution in APNIC.
As shown in Figure 3, IPv6 RTT of URL is smaller than IPv4 RTT. () For URL 7, the length of IPv6 middle-as (i.e., the number of ASes in middle-as) is the same as to that of IPv4 middle-as. However, middle-as contributes 120 milliseconds to IPv6 RTT, while it adds 150 milliseconds to IPv4 RTT. This is the reason for the smaller RTT of IPv6 in comparison with IPv4. () For URL 13, the length of IPv6 middle-as is 1, while the length of IPv4 middle-as is 3. The increase of IPv4 RTT in middle-as is much larger than IPv6 RTT. Thus, for this destination URL, the length of middle-as causes the difference between IPv4 RTT and IPv6 RTT.
Wang et al.  analyzed the delay of more than 3600 websites in APNIC, ARIN, and RIPENCC. We conduct a similar analysis across all the five RIRs. We use scatter diagram to describe the RTT distribution of IPv4/IPv6 in one polling cycle. As depicted in Figure 4, for each destination URL, IPv6 RTT is represented across the -axis and IPv4 RTT is represented across the -axis. We find that RTT of ARIN almost locates between 150 milliseconds and 200 milliseconds for both IPv6 and IPv4. The values of ARIN are most clustered among the five RIRs. Results imply that the paths between the probing client and the destination URLs of ARIN are relatively simplex. RTT of APNIC shows the most dispersive distribution, which is caused by the diversified paths between the probing client and the destination URLs.
Figure 4: RTT distribution of IPv4/IPv6 in one polling cycle.
We use Traceroute to explore AS-level paths between the probing client and the destination URLs. The total number of ASes between the probing client and the destination URLs is denoted as . Note that there are some duplicated ASes in . The number of unique ASes in is represented as . denotes the ratio of and ; that is, = /. We calculate the of the five RIRs. The results are 91.7% (APNIC), 66.7% (RIPENCC), 40% (LACNIC), 27.3% (AFRINIC), and 13.3% (ARIN). Among the five RIRs, the of APNIC is highest, which reveals that paths between the probing client and the destination URLs of APNIC are most diversified. This could explain why RTT distribution of APNIC is most dispersive. The of ARIN is lowest, which could explain that RTT of ARIN is most clustered.
4.2.3. Stability Analysis
In addition, we conduct a comparison analysis between IPv4 RTT and IPv6 RTT without distinguishing regions. During the one-week observation, the average RTT of IPv6 is 247.71 milliseconds, while the average RTT of IPv4 is 244.48 milliseconds. As depicted in Figure 5, IPv6 RTT is relatively stable over time, while IPv4 RTT fluctuates greatly. This can be confirmed by the standard deviation, which is calculated based on the average RTT of each hour. The standard deviation of IPv6 is 1.4 milliseconds, while the standard deviation of IPv4 is 3.3 milliseconds.
Figure 5: RTT comparison between IPv4 and IPv6.
We also utilize weighted moving average to further compare the stability and predictability of IPv6 RTT with IPv4 RTT. The average RTT of the five RIRs in each polling cycle is regarded a data point. That is to say, there are 168 data points during the whole observation. For a window size , we predict the next data point by computing moving average for the previous consecutive data points. We compute the moving average error for different window sizes. We find that when the window size equals to 2, the moving average error of IPv6 RTT is below 1% in most cases, while the moving average error of IPv4 fluctuates greatly with a maximum of 4.5%. Results reveal that stability and prediction accuracy of IPv6 RTT are much better than those of IPv4.
The more obvious fluctuation of IPv4 RTT may be caused by some overutilized links in IPv4 networks. In order to investigate the relationship between delay variation and link utilization, we probe the RTT of two backbone nodes of CERNET2. These two nodes and directly connect another node where we deploy the probing host. The link from to is labeled as , and the link from to is labeled as . Then we calculate the link utilization for and . Note that the link utilization denotes the ratio between the average traffic rate (in each five minutes) and the bandwidth of the physical link. For the clarity of description, we calculate and present the average link utilization of each five minutes.
As shown in Figures 6(a) and 6(b), RTT of shows a similar pattern with the utilization of . RTT of is high and fluctuant when the utilization of is large, while RTT of is low and stable when the utilization of is small. However, not all the links present the strong relationship between the delay and link utilization, such as and . Although the link utilization of has obvious fluctuations, RTT of is relatively stable. Therefore, we infer that the RTT fluctuates greatly only when the link utilization exceeds a threshold.
Figure 6: Relationship between the delay and link utilization.
As depicted in Figure 7(a), we find that RTT of is concentrated around 20 milliseconds except two abnormal points. The maximum link utilization of is 0.8223 during the one-week observation. As shown in Figure 7(b), when the link utilization is below 0.7, RTT is pretty steady around 13 milliseconds except one abnormal point. When the link utilization varies between 0.7 and 0.83, RTT begins to fluctuate and generates some high values. Once the link utilization is 0.83 or above 0.83, RTT of greatly fluctuates from 20 to 80 milliseconds. In addition, when the link utilization exceeds 0.83, most points are distributed from 50 to 60 milliseconds. As depicted in Figure 7(b), the number of points in the right of the gray line accounts for 49.4% of the total points. RTT of is relatively stable because the maximum link utilization of is 0.8223, which is smaller than 0.83. According to the above analysis, we could conclude that when the link utilization exceeds a threshold, for example, 0.83 in our case study, the RTT fluctuates greatly.
Figure 7: RTT distribution across link utilization.
4.2.4. Model Analysis
If the RTT fluctuates significantly when the link is heavily used, we should consider the influence on the real-time applications, such as video conference, IPTV, VoIP, and online games. If we can predict the changes of RTT, we can make some reliable routing policies for these delay-sensitive applications. We take as an example and try to find a fitting model to depict and predict RTT changes.
We apply R-square, which is an important metric to evaluate the goodness of a fitting model or accuracy of a predication model. As depicted in (1), denotes the average RTT of each polling cycle during the one-week observation. Then, denotes the modelling or prediction value of , and represents the average value of . The value of R-square is between 0 and 1. The more closely the value approaches to 1, the more accurately the model can explain the RTT changes. Through repeating attempts, we find that RTT variation of follows the model of Fourier. We also find that when the number of Fourier functions equals to 7, we could obtain the best fitting results. The value of R-square is 0.92. The general fitting model is shown in (2). The specific coefficient parameters of this model are shown in Table 4.
Испанские власти обещали придержать информацию - столько, сколько смогут. Мы узнали об этом лишь благодаря оперативности КОМИНТа. - Стратмор внимательно посмотрел не.
- Я должен найти его партнера, прежде чем он узнает о смерти Танкадо. Вот почему я тебя вызвал.