Direct Attach Cables for Data Center Interconnection

Direct attach cables are gaining more and more popularity among data center operators, for they use the same port as optical transceivers but with significant cost savings and power savings in short reach applications. These cable assemblies are mainly utilized to support high transmission rates between servers, switches and storage devices in data centers. This article will focus on the types and usage of direct attach cable assemblies for data center interconnection.

Types of Direct Attach Cable

According to the transceiver-style terminated on the end of the direct attach cables assemblies, they have these categories: SFP+ cable, SFP28 cable, QSFP+ cable, QSFP28 cable, etc. Using the same port as transceiver optics, direct attach cables can support Ethernet, Infiniband and Fibre Channel but with independent protocols. In general, direct attach cable assemblies are divided into three families—direct attach passive copper cable, direct attach active copper cable and active optical cable (AOC).

• Passive Direct Attach Copper Cable Passive direct attach copper cables are without active circuitry component. They can achieve interconnections up to 7m at 10 Gbps or 40 Gbps with low power. Direct attach passive copper cable assemblies offer high-speed connectivity between equipment. They are compatible with hubs, switches, routers, servers, and network interface cards (NICs) from leading electronics manufacturers like Cisco, Juniper, etc.
• Active Direct Attach Copper Cable Active copper cables are designed in the same cable type as the passive one, but they contain low power circuitry in the connector to boost the signal and are driven from the port without additional power requirements. The active version provides a low cost alternative to optical transceivers, and are generally used for end of row or middle of row data center architectures for interconnect distances of up to 15 meters.
• Active Optical Cable Active optical cable (AOC) incorporates active electrical and optical components. It can achieve longer distance than the copper assemblies. In general, active optical cable can reach more than 100m via multimode fiber. Compared to direct attach copper cable, AOC (eg. Cisco SFP-10G-AOC10M) weighs less and can support longer transmission distance. It is immune to electromagnetic energy since the optical fiber is dielectric (not able to conduct electric current). And it is an alternative to optical transceivers and it can eliminate the separable interface between transceiver module and optical cable. However, it costs more than copper cable.

In addition, with the fan-out technology, both direct attach copper cable and active optical cable can be designed as breakout direct attach cable assemblies, like 40G QSFP+ to 4x10G SFP+ AOC, which can better satisfy the demands on network migration.

Direct Attach Cables for Data Center Interconnection

Direct attach cable assemblies are ideal choices for short-reach direct connection applications. Generally, they are used in the EDA (Equipment Distribution Area) where cabinets and racks house end equipment (servers) and where horizontal cabling from the HDA (Horizontal Distribution Area) are terminated at patch panels.

For interconnection in racks and between rows of racks, direct attach cable assemblies are used to connect server to switch, storage to switch or switch to switch. Depending on different interconnect applications and distance requirements, direct attach copper cables, passive or active, active optical cable, or breakout direct attach cable assemblies with various length options can be used.

Conclusion

Different types of DAC assemblies have different applications. For example, the 10G SFP+ DACs are commonly used in interconnect applications below 15m, such as server to switch or storage to switch interconnection in the same rack. And now 25GbE is popular and 25G direct attach cable assemblies, such as SFP28 DACs, are already available in the market. For 40GbE, 40G QSFP+ DACs and AOCs are used. Of course, higher speed and more bandwidth are needed for spine switches. Thus, 100G DACs, like QSFP28 DACs are used in this case.

Things You Should Know About Direct Attach Cables

Direct attach cable, short for DAC, is a form of high speed cable with “transceivers” on either end. They can be used to connect switches to routers or servers. They are becoming increasingly popular in the network industry and widely applied in storage area network, data center, and high-performance computing connectivity etc. The DAC has the advantage of cost-effectiveness compared with the regular optics. Another reason for its increased popularity is that RJ-45 10G is not widely adopted, and most high density 10G switches are delivered with 48x SFP+ ports in 1 Rack Unit. Here are some basic knowledge you may not know but should know about the DACs.

1. No matter in the related article or in product description of some DACs vendors, there is always a point saying that the active DACs are lighter and thinner than passive DACs. Actually, it is a wrong view of point. If you have an using experience of both active and passive DACs, you may find that there is no difference between them in the weight and appearance.

2. As the first point mentioned, the thickness and weight do not depend on the active or passive function. In fact, they are divided according to their characteristic of AWG (American Wire Gauge). For instance, AWG24 DAC is thicker and heavier than the AWG30. Because of the difference of wire diameter, the longer the cable, the more inconvenience AWG24 DAC is, compared with the AWG30. Moreover, there is also limited bend radius. Thus, the longer the distance, the higher the AWG rating should be.

3. When the transmission distance is over 5 metres, active DAC is more suitable than passive DAC. Because it will cause the signal issue when using passive DAC for long-haul transmission.

4. DACs are more cheaper than the regular optics. Because the “transceivers” on both ends of DACs are not real optics. Compared with the regular optics, they are without the real components and just used to transmit the optical signals. And of course, without those expensive optics components, the cost of DACs is much lower. Thus, though using the same port as an optical transceiver, DACs are with significant cost savings and power savings in short reach applications.

5. The main difference between active DAC and passive DAC is that there is a driving chip in the design of active DAC.

6. AOC (Active Optical Cable) is one of the form of DAC which integrates multimode optical fiber, fiber optic transceivers, control chip and modules. With the benefits of AOCs, such as lighter weight, high performance, low power consumption, low interconnection loss, EMI immunity and flexibility etc. AOCs are now considered as the rising star of telecommunications and datacom transceiver markets.

Direct attach cable assemblies, including copper (DAC, direct attach copper cable) and fiber (AOC, active optical cable) types, are mainly used as media to support high transfer rates between servers, switches and storage devices intra rack or inter rack in a data center. FS.COM offers a variety of high speed interconnect DAC assemblies including 10G SFP+ cable, 40G QSFP+ cable, and 120G CXP cable to satisfy the demands from 10G to 100G interconnection. All these direct attach cables can meet the ever growing need to cost-effectively deliver more bandwidth, and can be customized to meet different requirements.

AOC—the Ideal High-Speed Interconnect Solution

What is the Ideal High-Speed Interconnect Solution?

The ideal high-speed interconnect solution should have such features as optimized for short distances, low cost, low power consumption, small cable bend radius, low cable weight, high density, and low link latency. The only one solution which can meet all the requirements is the direct attach active optical cables.

Direct attach active optical cables, or active optical cables for short, are direct-attach fiber assemblies with optical transceiver (SFP+, XFP, QSFP+, CXP etc.) connectors. They are suitable for short distances and offer a cost-effective way to connect within racks and across adjacent racks. Nowadays the Active Optical Cable (AOC) is accelerating data connectivity for storage, networking, and HPC applications. It leverages fiber optic technology for the transmission of data while reducing the weight, density and power consumption of traditional copper solutions. For example, the 40G active optical cable (AOC) is a type of active optical cable for 40GbE applications that is terminated with 40GBASE QSFP+ on one end, while on the other end, besides QSFP+ connector, it can be terminated with SFP+ connectors, LC connectors, etc. 40G active optical cables have great advantages over 40G copper QSFP+ cable when transmission distance reaches up to 7 meters. Moreover, 40G AOC has lower weight and tighter bend radius, which enables simpler cable management.

Advantages of Active Optical Cables

The AOC assemblies provide the lowest total cost solution for data centers by having the key advantages as following:

• Low weight for high port count architectures;
• Small bend radius for easy installations;
• Low power consumption enabling a greener environment.

Compared to Active & Passive Copper Cable Assemblies

1. Longer reach (> 7 meters)

2. Lower weight and tighter bend radius enable simpler cable management

3. Thinner cable allows better airflow for cooling

4. Lower power consumption

5. No need for power-hungry conditioning ICs on the host board

6. Can be used in architectures with challenging cable routing

Compared to Optical Transceivers

1. Datacenter/Consumer friendly: No cleanliness issues in optical connector

2. Cost-optimized: Not constrained by optical interface specifications driven by longer reach applications

(Note: However, the active optical cables cannot be routed through fiber patch panels.)

Ideal High-Speed Interconnect Solution

Today’s enterprise data centers and networking environments are undergoing an infrastructure transformation, requiring higher speeds, greater scalability, and higher levels of performance and reliability to better meet the demands of business. As speed and performance demands increase, the AOC assemblies have become an integral part of the overall system design. However, AOC design margins and parameters vary widely, and can be the difference between an optimized, highly reliable fabric and the incompatibility issues that drive up support costs. There are various types of AOC assemblies for 10G, 40G, and 100G applications on the market. Judging from the cost performance, FS.COM can provide the most ideal high-speed interconnect solution of AOC assemblies including 10G SFP+ AOC, 40G QSFP+ AOC, 40G QSFP+ to 4×SFP+ AOC, 40G QSFP+ to 8×LC AOC, and 120G CXP AOC.

What's the Difference Between Direct Uplink and Fan-Out Uplink?

Fan-out cable, also called breakout cable is referred to be one of the latest enabling technologies to increase port densities and reduce the costs. Taking one (large bandwidth) physical interface and breaking it out into several (smaller bandwidth) interfaces, it has been highly recommended to be used in network migration. In this post, I will compare the 40G direct uplink and the fan-out 4x10G uplink configurations and their inherent differences in maximum server scalability.

For a leaf/spine network architecture, the number of connections used for uplinks from each leaf switch determines the total number of spine switches in the design. Meanwhile, the number of ports on each spine switch determines the total number of leaf switches.

Now, imagine that we’re building a network that supports 1200 10G servers in one fabric with 2.5:1 oversubscription. The network must seamlessly expand to over 5000 10G servers without increasing latency or oversubscription in the future. Next, I will compare two different approaches.

40G Leaf/Spine Fabric

The following picture shows us a 40G fabric to us. Each of the spine switches has 32 ports of 40G. Each leaf ToR (Top of Rack) switch is connected to the spine with 4 ports of 40G using the leaf switches 40G QSFP uplink ports. There will be 40 servers per rack connected to each leaf switch. Namely, it allows a maximum is 1280 x 10G servers at 2.5:1 oversubscription. This meets the initial scale target of 1200 servers however, it cannot scale larger. Before the network can achieve the 5000 server design goal, the 40G design will have to be re-architected.

10G Leaf/Spine Fabric

As mentioned above, a direct 40G uplink is not an ideal configuration for a cost-effective server scalability. So, what level of server scalability can be achieved when using fan-out the leaf switch uplink ports with 4 x 10 G each?

The picture below shows a 10G leaf/spine fabric to us. With a QSFP+ to SFP+ optical fan-out cable, each QSFP leaf port is now fanned out to 4 x 10G interfaces each, for a total of 16 x 10G uplinks. Each of the spine switches now has 128 ports of 10G. Each leaf switch is connected to the spine with 16 ports of 10G. In this case, the maximum scale is 5120 x 10G servers at 2.5:1 oversubscription. Obviously, with the same bandwidth, latency and oversubscription, this fabric is better. It can not only be built to 1200 servers for present demand but also can seamlessly scale to over 5000 servers in the future.

Through the above analysis, we can see that the 10G leaf/spine fabric design offers four times greater scalability than the 40G fabric design when they have the same hardware, bandwidth, latency and oversubscription. These two configuration scenarios show us how fan-out technology are used to scale up a data center fabric. As the 10G network is widely deployed in today’s data center, the 10G direct attach cables (like Cisco SFP-H10GB-ACU10M) are preferred by many designers and companies. But higher speed demand such 40/100G is also needed. The fan-out technology not only enables new levels of server scalability, but also promotes a time-saving and cost-effective network migration.

Why 40G Active Optical Cables So Popular?

Nowadays, 40 Gigabit Ethernet is the main trend in higher data transmission. The 40G optical devices are gaining popularity, including 40G QSFP+ transceivers and 40G direct attach cables. 40G direct attach cable (DAC) provides a cost-effective solution for high-density network connectivity. It is a kind of high-speed cable which has transceivers on either end used to connect switches to routers or servers. According to the media type, DAC can be classified into direct attach copper cable and active optical cable.

40G active optical cable (AOC) is a type of active optical cable for 40GbE applications that is terminated with 40GBASE-QSFP+ on one end, while on the other end, besides QSFP+ connector, it can be terminated with SFP+ connectors, LC connectors, etc. 40G active optical cables have great advantages over 40G direct attach copper cables (like QSFP+ cable) when transmission distance reaches up to 7 meters. Moreover, 40G AOC has lower weight and tighter bend radius, which enables simpler cable management.

Actually, 40GBASE-SR4 QSFP+ transceivers also have the above advantages. So why we highly recommend 40G AOC rather than 40GBASE-SR4 QSFP+ optics? The following section will tell the answer.

Firstly, 40G AOC has lower cost than 40GBASE-SR4 QSFP+ modules and does not need to use with extra fiber patch cables. Especially, 40G breakout active optical cables, such as 40GBASE QSFP+ to 8 x LC or 40GBASE QSFP+ to 4 x SFP+ are cost-effective solutions to achieve 40G migration. What’s more, if using AOC, there will be no cleanliness issues in optical connector and there is no need to do termination plug and test when troubleshooting, which can help end users save time and money.

Insertion loss and return loss are the second factor. Under the same case of transmission distance, the repeatability and interchangeability performances of 40GBASE-SR4 module interface are not good as 40G AOC. Furthermore, when different fiber optic cables plug into the module, it will have the different insertion loss and return loss. Even for the same module, this issue is existed. Of course, the related metrics, such as the testing eye pattern, will have no significant changes so long as the variation in and conformed to the scope. In contrast, an AOC with good performance is more stable and has better swing performance than SR4 modules in this situation. The following table shows the result of the repeatability test of SR4 module. From the data, it is clear to see that the repeatability performance of SR4 module is not stable.

Thirdly, four-quadrant test, a testing under four combinations of input voltage and signal amplitude, is used to ensure the product to keep better performance even under the lowest and highest voltage and temperature situation. Four-quadrant test in wide temperature range is used to test MTP/MPO interface and optical cable of AOC to ensure them not to be melted in high temperature. Generally, the current products of AOC can satisfy this demand. Moreover, the performance of AOC is more stable than SR4 module which should be used with indeterminacy-performance MTP/MPO connectors. Unlike 40GBASE-SR4 module, the quality index of AOC is judged by electric eye pattern but not by light eye pattern.

Fourthly, digital diagnostic monitoring (DDM) can help end users to monitor real-time parameters of the modules. Such parameters include optical output power, optical input power, temperature, laser bias current, and transceiver supply voltage, etc. 40GBASE-SR4 QSFP+ transceiver with DDM function can ensure its optimal coupling by the ADC (analog to digital converters) value of real-time monitoring receiver when receive coupling. Thus, 40GBASE-SR4 modules have better receiving sensitivity than 40G AOC. However, at present, the 40GBASE-SR4 and AOC cannot reach the function of real-time power monitoring.

Transmission distance is the last factor. When transmitting over OM3 fiber, there is no significant difference between 40GBASE-SR4 and 40G AOC. But 40GBASE-SR4 has better performance control than 40G AOC. Moreover, proposals for transmission distance that is longer than 300 meters will be SR4 module in order to ensure a good performance.

Through the above analysis, we can see that 40G AOC has better consistency and repeatability cabling performance. Moreover, it can avoid the influence of environment and vibration, even for troubleshooting, AOC is more easier to manage. So active optical cable is highly recommended to use in data center interconnection. Apart from 40G AOC, FS.COM also supplies 10G DAC (eg. SFP-H10GB-ACU10M) which can meet the ever increasing need to cost-effectively deliver more bandwidth, and can be customized to meet different requirements.