Fast Data Transfer In and Out of Microsoft Azure

The Microsoft Azure cloud platform continues to gain traction for a wide range of applications across many industries. As businesses adopt Azure, one of the initial decisions they face is the choice of a mechanism for transferring data from their on-premises locations to the cloud.

Data ingest presents a particularly significant challenge for data-intensive verticals such as Media & Entertainment, Life Sciences, Geospatial Imaging, and Gas & Oil, and for use cases that include ‘big data’ analysis or large-scale data migration. The sheer volume of data can be enormous, and it often arrives from multiple locations and is needed for time-critical operations.

For these situations, Microsoft has partnered with Signiant to provide fast, flexible and reliable solutions for accelerated data movement into and out of Azure.

Choosing the right approach to Azure data ingest

There are a number of options for moving data into Azure, including:

• Shipping physical storage devices (Azure’s disk import/export service and Azure Data Box),
• Private direct connectivity via Azure ExpressRoute,
• Microsoft software utilities, including the AzCopy command line interface or other applications built on the Azure SDK,
• Signiant’s cloud-native Flight, Manager+Agents, and Media Shuttle offerings.

Choosing the optimal solution for each use case depends on the amount of data, the frequency of data movement, the number of source locations, and specific attributes of those locations.

Here are some things to consider during the process of selecting the best tool for the job.

Physical Media: If you’re planning a one-time move of several Petabytes of data and don’t have a tight timeline, shipment of physical storage devices can be a good choice. Remember that you will need in-house resources at the data center to load the drives, and security concerns may prohibit physical shipments in some situations.

Private connectivity: If most of your data is in a single location and you’re moving it to a single Azure data center, buying a dedicated network pipe such as ExpressRoute can provide an efficient means of data transfer. If your data is widely dispersed, however, it’s generally impractical to procure dedicated connectivity for all locations. Utilization is also an important consideration – depending on your data flow, you may end up paying for a connection that is often under-utilized.

Microsoft Software Utilities: Many routine data transfer needs can be accommodated with applications like AzCopy. Because these applications rely on TCP, however, they are susceptible to degraded performance in the presence of network congestion and/or when transmitting over longer distances (high latency). It’s therefore important to consider specific attributes of the source locations – including network capacity, network conditions and distance from the destination Azure data center.

Signiant acceleration for fast Azure data transfers

Signiant Flight with Microsoft Utilities: If you are experiencing unpredictable or slow performance with utilities like AzCopy, pairing the Microsoft client software with Signiant Flight is likely to solve the problem. This is particularly true if you’re transferring from multiple locations around the world.

Flight can be easily deployed with any HTTP-based application and uses Signiant’s UDP acceleration technology to minimize the impact of latency and congestion – thereby enabling full utilization of the available bandwidth. Because Flight can be deployed anywhere and you will only be charged when you’re transferring data, it can be thought of as “Software-Defined Express Route”.

Signiant Flight with Signiant Manager+Agents: For advanced enterprise use cases, the combination of Manager+Agents and Flight provides a complete solution for managing and orchestrating the large-scale movement of data in and out of Azure. Data can be transferred over standard Internet connections or private Azure ExpressRoute connections, and you will have unified visibility and control of all transfers. This combination also provides a highly reliable mechanism as interrupted transfers are automatically restarted and the bandwidth used for the transfer can be controlled by the administrator.

Signiant Media Shuttle: Many cloud use cases involve data transfer by people, either as an ongoing part of the workflow or on an occasional, ad hoc Signiant’s Media Shuttle SaaS offering meets this need via a simple Web interface that enables the transfer of large files to and from your Azure blob storage.

Benefits of Signiant for Azure data ingest

In summary, Signiant solutions provide unique value in situations where large volumes of data, long distances, multiple source locations, or flaky networks are involved. Unlike standard transfer mechanisms, our software enables cloud adoption by providing:

Speed: With Signiant’s patented transfer protocol and scale-out architecture, you can take advantage of multi-Gbps transfer rates to Azure – even in the presence of latency and congestion.

Cloud-Native Implementation: Both Flight and Media Shuttle are cloud-native solutions, so there’s no need to procure or manage any cloud infrastructure. We handle all of that, and our 24/7 DevOps team monitors the system for thousands of customers worldwide.

Security: All Signiant solutions are designed from the ground up to be highly secure. Our software is trusted by most of the world’s top media enterprises to move their valuable content around the world.

For more on Signiant Flight for large file Azure uploads, visit the Flight product page.