RAW DATA: TrueNet August 2015 Broadband Report - Congestion Appears on Fibre Networks

(TrueNet is contracted to the Commerce Commission to monitor broadband performance. It uses a network of probes attached to 400 volunteers' internet connections to rate ISPs — Editor.)

Many ISPs are now showing signs of congestion in their fibre products.  Only Spark was immune in August 2015.  Early users of Fibre connections (supplied via the Ultra Fast Broadband rollout sponsored by Government) found that their usage was consistently close to full speed at all times of day, despite the product being sold as delivering "up to" the speed they bought.  Now that demand is growing and many connections are sharing the same backhaul, congestion is reducing speeds in the evenings on many ISPs networks.  Chart 7 shows the impact.

Vodafone Cable has a similar network structure to Fibre, where all users share a backhaul from a local node.  Vodafone Cable continued to have serious congestion in the evenings right throughout August after having insufficient capacity to support the major increase in on-line video that started with Netflix launch in March.

Summary of Performance Measures

Due to an acquisition, the ISP formerly reported as Snap is now labeled  2Degrees from this month forward.

FIBRE

Fibre performance from Spark is remarkable for it's consistency and speed, not only is their upload and download speed the best across a range of Fibre products, but the average speed does not change at all during the busy hour.  No other ISP can match that.  Unfortunately, they  offer only the 20Mb/s upload product while Snap and Orcon offer 50Mb/s upload achieving almost double that of Spark.

Both Download and Upload fibre speeds are starting to show congestion with many ISPs - see charts 7 and 11.   Speed results this month include MyRepublic where we now have seven panelists with reliable and consistent results.  They top our Australian speed test during the busy hour, but were well below the rest for USA and NZ based testing, with a very long period of low speed during daylight hours, suggesting congestion impacts from video.  

ADSL & VDSL

The market has become quite competitive for performance measures with consistency in performance now standard for copper products ADSL & VDSL, a major improvement over previous years. The speed variation by time of day for both Vodafone & Slingshot on copper connections remains higher than other ISPs.  The time taken to download our sample webpages on copper from the USA continues to be dominated by Vodafone & Slingshot though Orcon is not far behind, but most other ISPs take close to 50% longer.  This difference seems consistent independent of technology.  The range of times to download NZ, or Australian pages is too small to differentiate.  

QUARTERLY REPORT - RURAL

The Quarterly Report again looks at the performance in Rural parts of NZ, 3 previous rural reports in our series are here.  RBI funded copper connections (both ADSL and VDSL) are significantly better than connections not yet upgraded, showing the impressive impact of the RBI project.  Improvements in Time of Day performance for Rural connections are now evident and approaching those for urban connections.

(Click to zoom)

Webpage Download Time

Responsive website browsing is valued by most Internet users, and conversely, slow-loading sites can be extremely frustrating.

TrueNet tests internet browsing by downloading a selection of Live Webpages from NZ, Australia, and the USA, measuring the time to fully download all files on the page.  These pages are changed from time-to-time. This month there are 7 NZ pages, 4 Australian, and 4 USA pages.

The charts in this section have three points for each ISP entry showing the Min Hour (minimum load time), the Max Hour (maximum time), and the Median of all hours.

NZ Live Webpages

We download 7 popular NZ websites from each connection and compare the average time taken to download all websites.  This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the websites into regions, NZ, Australia and USA.  

Chart 1 deals with NZ website comparisons by technology and ISP.   

FIbre results are consistent and very similar between ISPs, with the exception of Vodafone's 100Mb/s result.  

Vodafone Cable shows more variation than last month while their upgrade is awaiting completion.  

 iiNet performance is closer to NZ ISPs this month.

Slingshot DSL & Vodafone Cable have large differences in Median v Max hour, suggesting significant levels of congestion.  Median results are all much the same by technology.

Australian Live Webpages

We download 4 popular Australian websites from each connection and compare the average time taken to download all websites.  This test also replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the websites into regions, NZ, Australia and USA.  

Chart 2 deals with Australian webpage results by technology and ISP. 

iiNet has now reached the middle of the ADSL pack, and has been improving slowly over a number of months.  

Slingshot and Flip stand out as best for ADSL when downloading from Australia, VDSL, Fibre and Cable results are all much the same.

US Live Webpages

We download 4 popular USA  websites from each connection, and compare the average time taken to download all websites.  This test replicates daily activity for many people, but to provide the ability for readers to compare between ISPs based on their own preferences, we group the websites into regions, NZ, Australia and USA.

Chart 3 shows USA webpage results by technology and ISP. 
Orcon/Vodafone/Slingshot were best of the ADSL & VDSL providers this month, where the smaller ISPs in ADSL "Other" were notably worse in terms of average and min/max variation.  All other measures were much the same.

Vodafone Cable had the greatest variation between Min and Max hour, with Chart 3 showing a big difference between 50Mb/s and 100Mb/s Cable not seen in the NZ and Australian pages.

Speed (File Download Performance)

For TrueNet's speed tests each panelist's probe regularly downloads a 1MB or 2MB file from Auckland, Wellington, Dallas and Sydney.  The faster the connection, the larger the file we need to download to ensure that the maximum speed is reached during our test. 100Mb/s connections easily reach full speed within 2MB of data.  Slower connections can test accurately with much smaller files. Our tests on ADSL can tolerate a file size of just 300k.

New Zealand tests take the best of each test run from Auckland or Wellington.   International tests take the result from each test run from Dallas, or Sydney. 

We compute a median hourly speed for each panelist's connection, then take the average of all probes to report the actual hourly speed by ISP.   Our reported measure is the average hourly speed as a percentage of the best hourly speed in the month.

International File Download Speed

TrueNet tests download speeds from our servers in Dallas & Sydney to measure performance to our international neighbours.

We ensure the download file is not held in New Zealand (cached), so that the test truly measures international performance.  The results are selected from the 8pm to 10pm evening busy period, when congestion is likely to be observed.

Fibre & Cable

Chart 4: MyRepublic 100Mb/s service had the best speed from Australia this month, with Orcon down from almost 60Mb/s last month.  The range of speeds from Australia is very large, with the best ISP achieving over 50Mb/s, while "Other", a group of ISPs with many panelists, was well below 40Mb/s.  The critical USA speeds vary a lot also, with Vodafone Fibre best with over 15Mb/s and Other again last on less than 10Mb/s

The ISP with the best results for  30Mb/s Fibre seeems to change each month. 2Degrees did best this month, but Spark - best last month - now 3rd.

Vodafone 100Mb/s Fibre was fastest to the USA, with 30Mb/s and 100Mb/s services achieving similar results for the remainder of providers.  The range of speeds on 30Mb/s fibre connections is relatively small for both USA and Australia.

Vodafone Cable had the lowest average speed from each country compared to all fibre services, this is most likely due to their major problems with congestion in the wake of online Video growth starting in March.  Cable services became extremely congested in March and the situation has continued to worsen to the point that most of our 100Mb/s panelists consistently get less than 5Mb/s at some stage during the evening busy hours of 8 - 10pm.  Vodafone have publicly promised that work is proceeding as quickly as possible (we know it is a big task) with a target completion in September.  Most of our monitored connections are now (mid September) showing serious improvement, but there is still improvement to go, with the average mid-peak speed being about 50-70Mb/s.

ADSL & VDSL

Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem and the exchange equipment (also a modem) which means that ISPs do not have any influence on the peak speed of each connection.  To overcome this limitation, the Australia and USA speeds are referenced to the average NZ download speed of the respective ISP.  The speed reported is from the 8pm-10pm evening period.

Vodafone & Slingshot ADSL continue to show a consistently better speed to the USA than other ISPs.  

Vodafone again registered the best speed to Australia, as well as best to the USA of the ADSL providers.  ADSL results to Sydney slightly improved this month; VDSL largely the same.

Within the VDSL group, Bigpipe improved on last month, and Slingshot moved from 2nd to last, although remained by far the best for speed from the USA.

NZ File Download Speed

Comparing performance by time-of-day is important as it shows the service degradation when everyone is using the internet during the evening hours of 8pm to 10pm.  TrueNet uses the best of a pair of Auckland and Wellington download tests to calculate the median NZ results by hour over the month for each monitored connection.  We take the average of all median results with each ISP for each hour.

Technology Comparison

The Time of Day performance of DSL, Cable and Fibre services are compared in Chart 6. The result is indexed to the hour with maximum speed for each technology.

For the first time we are seeing consistent congestion on Fibre and DSL (copper) services with a notable "dip" between 8pm to 10pm.  There is a more general reduction in 100Mb/s fibre throughout the daylight hours starting at 9am, just after school has started.  Monitoring to show the differences between ISPs is shown below. 

Vodafone has started work to ease congestion issues on their cable network this month. This work has not progressed sufficiently to show in August results, we expect a partial improvement in September. 

Fibre - Cable ISP Comparison

Time of Day comparison of Fibre and Cable providers by ISP, Chart 7, shows very different results this month between providers.  

Spark fibre was consistent by Time of Day this month, unique amongst our sample of ISPs providing fibre.   

2Degrees and Orcon have an evening dip staying well below the advertised speed.   MyRepublic (now with sufficient panelists for long term reporting) and Vodafone fibre were well below advertised speed for most of the day. The remaining providers in "Other" having the lowest overall result.

Chart 7: Fibre & Cable Performance as Percentage of Advertised Speed

DSL Performance by Time of Day (ToD)

High Speed Copper (VDSL) File Download Speed

Almost all ISPs stayed above 95% of Max hour speed this month, highly consistent by Time of Day.   Slingshot results had a shorter, deeper speed reduction in the busy evening period.

Low Speed Copper (ADSL) File Download Speed

Performance most hours is very consistent, with a few ISPs showing contention around 8-9PM. Slingshot, Flip & Vodafone slowed during the evening busy period - more so than last month, while iiNet (Australia) has improved.  We have 5 ISPs achieving better than 95% - BigPipe, 2Degrees, Spark, iiNet and Orcon

There may be residual evening capacity issues catering for the growth of online video, which future results will help clarify.

Upload Speed - Performance

Upload speed is important to users sending large amounts of data through the internet, or loading files to the Cloud.   TrueNet's upload test sends a 1MB file to our Wellington server, and records the results using a similar method to the download tests, but measured from Wellington only.

By Technology

All technologies achieve very close to advertised speeds.  Fibre achieves the fastest Upload speeds, the average 50Mb/s service speed was down a small amount this month, with 20Mb/s and 10Mb/s Fibre exceeding Advertised Speed on average. 

Upload Time of Day Performance - by ISP

Spark fibre upload speeds are not only rock solid, but also exceed their advertised speeds. The 10Mb/s product was measured at 12Mb/s and the 20Mb/s product at 26Mb/s.   Vodafone Cable results also show very consistent ToD performance. 

In Chart 11, the 50Mb/s services from 2Degrees and Orcon have a dip in the evening period, as does Vodafone 20Mb/s fibre services. This represents a change from last month. 

Unfortunately, we have an almost even split of 50 or 20Mb/s products for MyRepublic, so cannot report upload speeds yet, so too with Orcon 20Mb/s product.  Orcon & MyRepublic are the majority of the 20Mb/s "Other" category, while Spark, MyRepublic, and Voyager make up the majority of 50Mb/s "Other".

Quarterly Feature - Rural Broadband Performance

This month we review Rural Broadband performance for the fourth time, based on the 3 month period June-August 2015. Rural Broadband services with poor internet access has been a persistent issue over a number of years. The need to improve the rural sector has become increasingly obvious as UFB and other improvements see much faster speeds in Urban areas. The Government sponsored Rural Broadband Initiative has invested in improving rural broadband using both a special Wireless solution, and an increase in investment in copper.   This, and other technologies do offer choices for improved speeds in some areas. As the results below show, the alternative technologies offer better service than Rural ADSL that has not yet been upgraded on RBI funding.  Our panelists have a number of technologies:  Satellite (under-represented this quarter, Wireless RBI, RBI-funded ADSL & VDSL services from Chorus-designated RBI cabinets; and standard ADSL in the rural environment, typically with long copper lengths. New to reporting this month are RBI-funded VDSL and Wireless RBI panelists.

Live Webpage Performance

The average time to load Live Webpages from NZ, Australia, and the USA are shown in Chart 12 below. The webpages are the same group as used with Urban panelists.

Compared to the previous quarter, the min/max variation is less, with the exception of Rural ADSL.  

RBI-funded VDSL had the best results for both NZ and Australian Webpages, while Rural ADSL had both the longest load times, and the greatest variation between best and worst hour of the day.  Wireless RBI was similar to RBI-funded ADSL for NZ Webpages (the only webpage group where there was sufficient sample size because those connections use low caps for testing)

RBI-funded ADSL average times were more or less equal to Urban ADSL, but had a slightly larger Time of Day difference.  RBI funded copper connections (both ADSL and VDSL) are significantly better than connections not yet upgraded, showing the impressive impact of the RBI project.

Rural File Download Speed

Averaged over the full 24 hours, the speed (Up/Down) by Technology is given in Chart 13.   RBI-funded VDSL and Wireless achieve good download speeds, and reasonable upload speeds. Satellite is shown as indicative -  sample too small this quarter, but is in line with speeds reported last quarter. RBI-funded ADSL on average achieves somewhat faster Download and Upload speeds compared to non-RBI Rural ADSL, though less than the average of Urban ADSL (9-10Mb/s)

Time of Day in Rural

In the past Time of Day variation has been an issue in Rural. While not ideal, the results in Chart 14 are fairly good for rural broadband. Rural ADSL and RBI-Funded ADSL show slight improvement over last quarter. RBI-based VDSL overall achieves similar or better results compared to Urban ADSL, which is good news. Wireless RBI in our sample was slightly better in the evening than Rural ADSL, where both appear to have extended periods with below peak speeds compared to the other technologies available.

Chart 14: Time of Day Performance of Rural Technologies v Urban ADSL

(Click to zoom)