Recently, we released a new version of AzCopy with a set of new features.
You can always download latest version from aka.ms/AzCopy.
New features added
- /DestSAS and /SourceSAS: This option allows access to storage containers and blobs with a SAS (Shared Access Signature) token. SAS token, which is generated by the storage account owner, grants access to specific containers and blobs with specifc permissions and for a specified period of time. Permissions include LIST, READ, WRITE or DELETE.
Currently, we support SAS token while you upload data to Azure Storage or download data from Azure Storage. For blob copy, we support SAS Token for the source location.
Please refer to this blog post for more information on How to create a Shared Access Signature. - Enhancement of re-startable mode: In previous version, the re-startable mode is supported for interruption caused by network or other issues during file transfer but it only supports restarting the transfer from the beginning of the interrupted file. Imagine if the interrupted file is a large file (e.g. a VHD file) and most part of it has already been transferred to Azure Storage, it would be time-consuming to restart the transfer of entire file again. So to improve this scenario, we enhanced re-startable mode and made it restart the transfer from the point of interruption. For block blob, we chose 4Mb as the size of data chunk.
Example1: Upload all files from a local directory to a container using SAS token which offers permits for list and write
AzCopy C:\blobData https://xyzaccount.blob.core.windows.net/xyzcontainer /DestSAS:”?sr=c&si=mypolicy&sig=XXXXX” /s
/DestSAS here is for you to specify the SAS token to access storage container, it should be enclosed in quotes.
Example2: Download all blobs from a container to a local directory and then delete them from source (by using /Mov). The SAS token offers list, read and delete permission.
AzCopy https://xyzaccount.blob.core.windows.net/xyzcontainer C:\blobData /SourceSAS:”?sr=c&si=mypolicy&sig=XXXXX” /MOV /s
Example3: Upload all files from a local directory to a container in re-startable mode.
AzCopy C:\blobData https://myaccount.blob.core.windows.net/mycontainer /destkey:key /Z:restart.log /s
/Z is the parameter to turn on the re-startable mode. ‘restart.log’ is the customer-defined name of the journal file which will be saved in local.
For instance, if “C:\blobData” folder contains four files below and the size of file2.vhd is 1GB.
C:\blobData\file1.docx
C:\blobData\file2.vhd
C:\blobData\file3.txt
C:\blobData\file4.txt
Assume an interrupt occurs while copying the “file2.vhd”, and 90% of the file has been transferred to Azure already. Then upon restarting, we will only transfer the remaining 10% of “file2.vhd” and another two remaining files.
Jason Tang
Where does the file actually get installed to? I've ran the .msi but can't find azcopy…
Please check C:Program Files (x86)Microsoft SDKsWindows AzureAzCopy
Hi Jason,
Thanks a lot. Tnis "Please check C:Program Files (x86)Microsoft SDKsWindows AzureAzCopy" should be inside the article.
Why are we getting errors such as "The remote server returned an error: (403) Forbidden"
I've used the code to generate the policy, and still receive an error. it works fine with the SourceKey parm. Also tried different variants of the Storage Library, including the 2.1 version as some posts on Stack Overflow indicate.
msdn.microsoft.com/…/jj721951.aspx
Here's the command:
AzCopy zzz.blob.core.windows.net/vhds zzz.blob.core.windows.net/vhds /SourceSAS:"?sv=2012-02-12&sr=c&si=unpolicy&sig=kdTPj7BTVtvTUqqbWUAfNCwSFLgr5hgQuF9FluPU%2F5k%3D" /DestKey:"Zg6yIu0dreMpIUiJTnaJbHShBuY6riYYYZ8ckOZ5D5iS35FOXrLzM/tfMf5587B9/u+NufFfl9bsTicMhy3HJw==" /Z:AzCopy-journal.log /V:AzCopy-verbose.log diab4n13.v32201402281848330226.vhd
OK, poor mistake on my part:
The command, if put in a BAT or CMD file, the command processor (cmd.exe) will replace the %3 as an example with what it expects was passed on the command line.
So, to make it work from a BAT/CMD file, any % needs to become %% instead…
For example if the SIG was /SourceSAS:"?sv=2012-02-12&sr=c&si=unpolicy&sig=%2BTus%2BfFNJqxMpFqYDX9%2Fve09AH7IibPb8SGlkvY4ONk%3D"
Need to make it this: /SourceSAS:"?sv=2012-02-12&sr=c&si=unpolicy&sig=%%2BTus%%2BfFNJqxMpFqYDX9%%2Fve09AH7IibPb8SGlkvY4ONk%%3D"
Hi Shawn,
Thanks for raising the question.
Windows handles command line and bat file differently. In the command line, add double quotation is enough for SASToken string, but for bat file, user need add another % before % as you said.
Beside adding %, you may also try to use azcopy response file together with bat file.
E.g. create several response txt files, each one contains one command, like this
zzz.blob.core.windows.net/vhds zzz.blob.core.windows.net/vhds /SourceSAS:"?sv=2012-02-12&sr=c&si=unpolicy&sig=kdTPj7BTVtvTUqqbWUAfNCwSFLgr5hgQuF9FluPU%2F5k%3D" /DestKey:"Zg6yIu0dreMpIUiJTnaJbHShBuY6riYYYZ8ckOZ5D5iS35FOXrLzM/tfMf5587B9/u+NufFfl9bsTicMhy3HJw==" /Z:AzCopy-journal.log /V:AzCopy-verbose.log diab4n13.v32201402281848330226.vhd
Then create a bat file with below lines,
Azcopy.exe /@:response1.txt
AzCopy.exe /@:response2.txt
AzCopy.exe /@:response3.txt
Execute this bat file, it will behave same as you input three commands in the command line window directly, and you can see that there is NO need to add % before % in the response file.
Zhiming
How do I download only files created in the blob since a specific date so I can limit the download to only recent files added to the blob?