AzureCopy

Update (24/12/2016)

A newer version of AzureCopy is being developed and IT’S CROSS PLATFORM! 🙂

I’ve had numerous requests to have a Linux and/or a MacOS version, well things are being developed. Currently it’s VERY alpha but is slowly having features added. I’ll start adding binaries to the github repo (note, different repo to the regular AzureCopy).

I’ll be posting some blog posts about the upcoming changes. Both versions of AzureCopy will be developed (regular dotnet for Windows and the Golang crossplatform version).

Now, back to your regular content 🙂

 

Updated, new, improved and generally neatened up AzureCopy documentation!

AzureCopy is a both a command line tool and a Nuget package which can be used to copy blobs (files) between Azure, S3, Skydrive Onedrive, Sharepoint Online, Dropbox and your local filesystem.

Command

Once the command is configured (see below) copying between cloud providers is simple and easy.

Examples:

To copy a blob from Amazon S3 to Azure Blob storage:

azurecopy –i https://mybucket.s3.amazonaws.com/ –o https://myaccount.blob.core.windows.net/mycontainer/myblob

To copy a blob from Azure to Dropbox:

azurecopy –i https://myaccount.blob.core.windows.net/mycontainer/myblob –o https://dropbox.com/mydirectory/

To copy a blob from Azure to Onedrive:

azurecopy –i https://myaccount.blob.core.windows.net/mycontainer/myblob –o one://myfolder

To copy a blob from Onedrive to Sharepoint Online:

azurecopy –i one://myfolder/myblob –o “https://testuser.sharepoint.com/Shared Documents”

and finally… to copy from my local drive to Azure:

azurecopy –i “c:\temp\myfile” –o https://myaccount.blob.core.windows.net/mycontainer/

Any combination of the above is allowed as well. So although AzureCopy has “Azure” in the name, it can certainly be used for many other purposes!

If the target is Azure then AzureCopy is able to make use of the CopyBlob API. This allows Azure itself to perform the copying between the source URL and the Azure target URL. This means that although you may execute the command on your own machine in your own network, the bandwidth (say for copying from S3 to Azure) is purely between Amazons and Microsofts datacentres. Your bandwidth is NOT used!  To enable this feature, just make sure the target destination (specified with the –o) is an Azure URL and that you also add –blobcopy onto the command parameters.

For example:

azurecopy –i –https://mybucket.s3.amazonaws.com/ –o https://myaccount.blob.core.windows.net/mycontainer/myblob –blobcopy

Configuration

Configuration of the command line tool is primarily done through the azurecopy.exe.config ( aka app.config for any .NET followers out there), although any parameter that can be configured in the config file can also be passed on the command line if so desired. But be warned, most cloud storage providers like long alphanumeric tokens/keys/secrets so having these in a config file is usually less hassle.

An example of the config file is (well the interesting parts):

<add key=”AzureAccountKey” value=””/>

This is the Azure Account Key taken from the Azure portal. This will be used as the default source or target azure keys. This key needs to match the Azure blob URL used in the copying command.

<add key=”SrcAzureAccountKey” value=””/>

This is the Azure Account Key taken from the Azure portal. This will be used as the source azure key. This key needs to match the source Azure blob URL used in the copying command.

<add key=”TargetAzureAccountKey” value=””/>

This is the Azure Account Key taken from the Azure portal. This will be used as the target azure key. This key needs to match the target Azure blob URL used in the copying command.

<add key=”AWSAccessKeyID” value=”” />

This is the Amazon S3 Access Key ID taken from the S3 Portal. This will be used as a default if the source/target versions of this parameter are not supplied.

<add key=”AWSSecretAccessKeyID” value=””/>

This is the Amazon S3 Secret Access Key taken from the S3 Portal. This will be used as a default if the source/target versions of this parameter are not supplied.

<add key=”AWSRegion value=””/>

This is the Amazon S3 Region for the account specified above. This will be used as a default if the source/target versions of the parameter are not supplied.

Valid regions can be read taken from Amazons S3 region page.

<add key=”SrcAWSAccessKeyID” value=”” />

This is the Amazon S3 Access Key ID taken from the S3 Portal. This will be used as credentials when S3 is the source of the copying command.

<add key=”SrcAWSSecretAccessKeyID” value=””/>

This is the Amazon S3 Secret Access Key taken from the S3 Portal. This will be used as credentials when S3 is the source of the copying command.

<add key=”SrcAWSRegion value=””/>

This is the Amazon S3 Region for the account specified above. This will be used as the region when S3 is the source of the copying command.

Valid regions can be read taken from Amazons S3 region page.

<add key=”TargetAWSAccessKeyID” value=”” />

This is the Amazon S3 Access Key ID taken from the S3 Portal. This will be used as credentials when S3 is the target of the copying command.

<add key=”SourceAWSSecretAccessKeyID” value=””/>

This is the Amazon S3 Secret Access Key taken from the S3 Portal. This will be used as credentials when S3 is the target of the copying command.

<add key=”TargetAWSRegion value=””/>

This is the Amazon S3 Region for the account specified above. This will be used as the region when S3 is the target of the copying command.

Valid regions can be read taken from Amazons S3 region page.

<add key=”SkyDriveCode” value=””/>

This is the Onedrive code retrieved after authenticating with Onedrive through a browser. Please execute: “azurecopy –configonedrive” for more details. Yes, this is currently a little confusing due to the settings being called Skydrive as opposed to Onedrive.

This is simply due to the Microsoft renaming of the product. This may eventually get changed here.

<add key=”SkyDriveRefreshToken” value=””/>

This is the Onedrive refresh token which is required to be valid for access to Onedrive . This is populated when executing azurecopy and does not require manual configuration.

<add key=”DropBoxAPIKey” value=”” />

This is the DropBox API Key used to authenticate against Dropbox. For the binary AzureCopy command this parameter is not in the config file but is hardcoded into the executable. This is due to “AzureCopy” being the registered app against Dropbox. If you are using the source/Nuget package for your own app development then you’d populate this value in the config file.

For the command please execute: “azurecopy –configdropbox” for more details.

<add key=”DropBoxAPISecret” value=””/>

This is the DropBox API Secret used to authenticate against Dropbox. See above….

<add key=”SharepointUsername” value=”test@test.onmicrosoft.com” />

This is the Sharepoint username used to authenticate against Sharepoint Online.

<add key=”SharepointPassword” value=””/>

This is the password for the above.

Nuget Package

See Nuget for details.

Will document more soon.

Github

See Github for details.

Will document more soon.