We are excited to introduce a new blob type called Append Blob (alongside our existing Block and Page blobs) that will be publically available in Q3 2015. In this blog, we will provide an overview of Append Blob as well as highlight its most common usage scenarios. We will share more information and details as we get closer to release.
Overview
Append Blob is a new blob type that will be available with an upcoming storage service version. All writes to an Append Blob happen at the end of the blob. Updating and deleting existing blocks is not supported. To modify an Append Blob, you add blocks to the end of the blob via the new Append Block operation. Each appended block is accessible immediately.
Append Blob is optimized for fast append operations, making it ideal for scenarios where the data must be added to an existing blob without modifying the existing contents of that blob (Eg. logging, auditing). In addition, Append Blob supports having multiple clients writing to the same blob without any need for synchronization (unlike block and page blob). An Append Blob has the same scalability targets as a block blob. See Azure Storage Scalability and Performance Targets for details.
Preparing for Append Blob
Most customers can simply wait for AppendBlob to be available and then upgrade their applications to use the latest version of our client libraries, at which point they can begin using AppendBlob. Customers that will want to plan for Append Blob are likely to be tooling providers (Eg. Storage explorers, cloud storage management tools) that operate on storage accounts that are managed independently of the tooling.
Once Append Blob is released, customers who want to use Append Blob will have to upgrade their applications to use the latest version of the storage service, client libraries, and tools in order to handle the new blob type. There are three possible issues that you may encounter with Append Blob:
- A container that contains one or more Append Blobs may be accessed only with a version of the storage service and client library that supports Append Blobs. If you attempt to list from a container that contains an Append Blob using an earlier version of the service, the service returns error 409 [FeatureVersionMismatch]. For a tooling provider, this means that until you upgrade your application to the new version, your tool may receive an exception when listing a container in a customer's storage account that contains an Append Blob.
- If you are using an AzCopy version that does not support Append Blob, the download and copy operations will fail when the source container includes an Append Blob.
- If you attempt to access an Append Blob using a PowerShell version that does not support Append Blob, the Get-AzureStorageBlob operation, it will fail. An update to the PowerShell libraries will be coming shortly.
As always, your feedback is welcome and appreciated!
Perry Skountrianos
Program Manager, Azure Storage
Responding to Jenny. You can of course append using block blobs but it requires three operations – getblocklist, putblock, putblocklist – which isn't that performant and becomes difficult for synchronizing if you have multiple clients. Regarding logging and table storage you have to ensure you distribute writes across partitions – take a look at the "log anti-pattern" described here: azure.microsoft.com/…/storage-table-design-guide.
Can it also support TrimEnd(int numberOfBytes)? That would enable the scenario of storing sorted streams/logs – when you have a refresh window in a stream, which supports sorted append of late data.
Responding to Michael. Thank you for your interest in Append Blob. Adding support for TrimEnd(int numberOfBytes) is not currently on our roadmap for Append Blob.