The chase is on for high speed cabling solutions
Fri 11 Apr 2014
Erwin Deeben, technical consultant at Brand-Rex, looks at how infrastructure technology should be deployed in order cope with the ever-growing demand for data
The amount data centre traffic predicted over the coming years is truly astonishing. It is forecast to hit 4.1 Zettabytes (ZB) next year and by 2016 it will be up to 6.6ZB. (A Zettabyte is 1015 Megabytes).
What’s more, by 2017 not only will there be a lot more servers but, according to Intel and Broadcom, two-thirds of them will ship with 40 Gb Ethernet connectivity. When we reach 2020 there will be almost double today’s number of servers and all of them will be capable of either 40 or 100 Gb Ethernet.
Such massive traffic rises cannot occur without the data centre sector rising to the challenge. And that doesn’t just mean more data centres but also changes to the interconnecting infrastructure – optical fibre and twisted pair copper – deployed within them.
Early adopters with high performance computing requirements will need 40 and 100Gb/s first, and they will use fibre and short reach coax solutions in advance of a BASE-T cabling solution being available. Meanwhile, the solutions for high-density connectivity are likely to use multi-lane coax, but only to the top of rack (ToR) as the maximum reach is only 7m. The 40Gb/s implementation is effectively 4x10Gb/s ‘lanes’ side-by-side and is available now.
However, the electronics to multiplex such high data rates are complex and the cables are bulky and very expensive. This is particularly so if you take into account that every rack must have a ToR switch with high-speed fibre connections to the higher level switches.
Having to install a switch-per-rack is obviously expensive in terms of capital costs and it is also difficult to manage. The attraction of higher utilisation of the ports in centralised switches using a fibre implementation is obvious. A number of 40Gb/s fibre solutions are already available for longer links, but the cost benefits of a BASE-T copper solution is still likely to make it the solution of choice, over fibre connections, for server to switch.
Meanwhile, the IEEE’s next generation BASE-T study group has concluded with a watertight case to form a new project for 40GBASE-T. It is reasonable to suggest that we may see propriety pre-standard 40Gb/s category solutions in late 2014 followed by a standardised 40GBASE-T in 2015. Given that the majority of data centres will only need 40Gb/s server connections in large volumes from 2016 to 2022, 40GBASE-T is likely to be the solution that will make most economic sense.
When it comes to topologies, one possible change is that centralised switching will not be as popular as it was – unless single mode optical fibre is used – and deployment of end of row (EoR) and middle of row (MoR) switches will be the preferred architectures.
Trunk cabling from EoR and MoR switches to the central MDF and core switches could be 40/100Gb/s OM4 multimode fibre – though this requires eight fibres per link for 40 Gigabit Ethernet and 20 fibres for each 100 Gb Ethernet link. This may well be the preferred choice for colocation data centres keen to contain capex since the multimode optical interfaces are significantly cheaper than single mode.
The downside of this is that there is a need to run-in a lot more fibre to move from 10 to 40 Gb Ethernet and then to 100 Gb Ethernet – with ensuing disruption and danger of service interruption. With this in mind it might be better to go straight to single mode solutions based on duplex connectivity with high-density panels providing cross-connect to server and switch connections.
Of course, as data centre infrastructures become more complex, the need to monitor and manage them becomes imperative. An intelligent infrastructure management system will streamline documentation processes and substantially reduce the operational costs of network ownership – bringing a much needed focus to quality service provision. IIM plays an important role in the move towards a complete data centre infrastructure management (DCIM) solution, which should also be a high priority as it offers the potential for huge savings in terms of maximising resources and minimising downtime and lost business output.
For data centre owners and operators, whose future-proofing goal and forecasting task have never been easy, I’m sorry to say it’s just got harder. I strongly advise them to discuss their plans with the top-tier cabling manufacturers at a very early stage to de-risk their projects and investments as much as possible.
