Post
Ask Question

Road To AZ 204 - Developing Solutions That Use Blob Storage

Introduction

 
This article's intention is to explain the main skills measured in this sub-topic of the AZ-204 Certification. Azure Storage and Blob Storage are the main components that will have their fundamentals explained here, alongside a practical example of its usage.
 
This certification is very extensive and this article approaches only the main topics, make sure you know deep down those components before taking the exam. Another great tip is doing exam simulators before the official exam in order to validate your knowledge.
 

What is the Certification AZ-204 - Developing Solutions for Microsoft Azure?

 
The AZ-204 - Developing Solutions for Microsoft Azure certification measures designing, building, testing, and maintaining skills of an application and/or service in the Microsoft Azure Cloud environment. It approaches, among others, those components:
  • Azure Virtual Machines
  • Docker
  • Azure Containers
  • Service Web App
  • Azure Functions
  • Cosmos DB
  • Azure Storage
  • Azure AD
  • Azure Key Vault
  • Azure Managed Identities
  • Azure Redis Cache
  • Azure Logic App
  • Azure Event Grid
  • Azure Event Hub
  • Azure Notification Hub
  • Azure Service Bus
  • Azure Queue Storage
 
Target Audience
 
Any IT professional willing to improve his knowledge in Microsoft Azure is encouraged to take this certification. It is a great way to measure your skills within trending technologies. But, some groups of professionals are keener to take maximum advantage of it:
  • Azure Developers, with at least 1 year of experience with Microsoft Azure
  • Experienced Software Developers, looking for an Architect position in a hybrid environment
  • Software Developers, working to move applications to the cloud environment
Skills Measured
 
According to today's date, the skills that are measured in the exam are split as follows:
  • Develop Azure compute solutions (25-30%)
    • Implement Azure functions
  • Develop for Azure storage (10-15%)
  • Implement Azure security (15-20%)
    • Implement user authentication and authorization
    • Implement secure cloud solutions
  • Monitor, troubleshoot, and optimize Azure solutions (10-15%)
    • Integrate caching and content delivery within solutions
    • Instrument solutions to support monitoring and logging
  • Connect to and consume Azure services and third-party services (25- 30%)
    • Develop an App Service Logic App
    • Implement API Management
    • Develop event-based solutions
    • Develop message-based solutions
Updated skills can be found in the AZ - 204 Official Measured Skills Website.
 

Benefits of Getting Certified

 
The main benefit here is having a worldwide recognized certification that proves that you have knowledge of this topic. Among intrinsic and extrinsic benefits, we have:
  • Higher growth potential, as certifications are a big plus
  • Discounts and deals in Microsoft products and partners, like PluralSight and UpWork
  • MCP Newsletters, with trending technologies
  • Higher exposure on LinkedIn, as recruiters usually search for specific certifications
  • Higher salary, you will be more valuable to your company
  • Unique happiness when getting the result and you were approved, knowing that all your efforts were worth it
 

Main Skills Measured by this Topic

 
What is a Blob Storage?
 
Blob Storage is part of Azure Storage where we can only store Blobs. It is optimized for storing massive amounts of unstructured data. Blobs are stored inside blob containers and those blob containers are inside an Azure Storage Account.
 
Azure charges a Blob Storage for storing and accessing its data. It is great for:
  • Sharing unstructured files, as far as each file has its own URI and security permissions
  • Streaming audio or video, Azure Blob Storage has excellent performance and availability;
  • Storing highly accessible files, using its hot storage
  • Storing big and not so frequently used files, using its cool or archive storage. Backup files and logs, for example
Blob Storage Access Tiers
 
In order to store and access your data in the most cost-effective manner, Azure offers Blob Storages in three different tiers, whereas their cost calculations differ from the usage storage and access frequency, as follows:
  • Hot tier, ideal for highly frequently accessed small files. Highest storage cost and lowest accessibility cost.
  • Cool tier, ideal for large files that are not so frequently accessed. Medium storage cost and medium accessibility cost.
  • Archive tier, ideal for large files that are rarely accessed. Lowest storage cost and highest accessibility cost.
Blob Storage Lifecycle Management
 
Azure Blob Storage has a very important feature that allows you to move your items between access tiers according to their usage. Azure Blob Storage lifecycle management policies make the transition of an item between tiers automatically in order to optimize costs and performance, those policies can be configured to run once a day and at the level of containers and subset of blobs. Those policies allow your Blob Storage to:
  • Transition from cold to hot, for better performance
  • Transition to cooler tiers in case of not being used
  • Delete items if not accessed or modified, according to the retention period
System Properties and Metadata
 
Blob Storage allows us to set and retrieve information from the Container level or the Blob level as follows:
  • System properties allow you to read the Blob Container Properties. Those properties can be updated by calling their own methods, not by updating their properties directly.
  • Metadata allows you to write and read key-value attribute pairs assigned to each Blob or Blob Container.
Blob Storage SDKs
 
Azure has a wide range of SDKs in order to manage and manipulate your Blobs, as follows:
  • Azure Portal
  • PowerShell
  • Azure CLI
  • Azure Storage Explorer
  • .Net
  • Java
  • Python
  • Javascript
  • C++
  • Go
  • PHP
  • Ruby
  • Xamarin
Practical Samples
 
In this study of the case, we are going to store images with Azure Blob Storage. Images that are going to be categorized as more frequently used, others not so often used and even others rarely used.
 

Creating a Blob Container in an Azure Storage Account

 
Using Azure CLI
 
Setting variables 
  1. $resourceGroup = "your resource group here"  
  2. $storageAccountName ="samplestorageaccountblob"  
  3. $location ="West Europe"  
  4. $containerName="sampleblobcontainer"  
  5. $subscription="your subscription id here"  
Creating the storage account
 
 
Creating the Blob Container
 
 
  1. az storage account create --name $storageAccountName --resource-group $resourceGroup --location $location  
  2. az storage container create --account-name $storageAccountName --name $containerName --auth-mode login  
Validating through Azure Portal
 
 
Using Azure Portal
 
From your Azure Portal, search for the Storage Account resource and click on Add. Fill the Basics, Networking, Data Protection, and Advanced forms then Create the Azure Storage Account.
 
 
 
After successfully deploying, go to your resource. Inside Blob Service, go to Containers and click on "+ Container", set your container name, its access level, and click on Create
 
 
Success
 

Manipulating Blobs

 
Using .NET
 
Prerequisites
  • Nuget Azure.Storage.Blobs
  • Azure Storage Account previously created
First, retrieve your connection string. From your Storage Account, go to Setting and then Access Keys.
 
 
Setting Local Variables
  1. private static readonly string connectionString = "DefaultEndpointsProtocol=https;AccountName=samplestorageaccountblob;AccountKey=FyJw8U6tORM7DVBxw4G5nxUlbTy8l5so8lAGfsr1AEf6fSN63A6llM8kQfQO9bF1qABHkNHfCWtL/l/j9zk/WQ==;EndpointSuffix=core.windows.net";  
  2. private static readonly string containerName = "sampleblobcontainer";  
  3. private static readonly string imagesPath = @"C:\~~~~~~\AzureBlobStorage\AzureBlobStorage\Pictures\";   
Connect to your Blob Container, create it if does not exist yet.
  1. BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);  
  2.            BlobContainerClient containerClient = null;  
  3.   
  4.            if (blobServiceClient.GetBlobContainers().Where(x => x.Name == containerName).Any())  
  5.            {  
  6.                Console.WriteLine("Blob Container Client found.");  
  7.                containerClient = blobServiceClient.GetBlobContainerClient(containerName);  
  8.            }  
  9.            else  
  10.            {  
  11.                Console.WriteLine("Blob Container Client not found, creating a new one.");  
  12.                containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName);// Create the container and return a container client object  
  13.                Console.WriteLine("Blob Container Client created.");  
  14.            }  
Upload the images as Blobs
  1. // Get a reference to a blob  
  2.           BlobClient blobClient = containerClient.GetBlobClient(filename);  
  3.   
  4.           Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);  
  5.   
  6.           // Open the file and upload its data  
  7.           using FileStream uploadFileStream = File.OpenRead(Path.Combine(imagesPath, filename));  
  8.           await blobClient.UploadAsync(uploadFileStream, true);  
  9.           uploadFileStream.Close();  
List the Blobs in your container
  1. await foreach (BlobItem blobItem in containerClient.GetBlobsAsync())  
  2.           {  
  3.               Console.WriteLine("\t" + blobItem.Name);  
  4.           }  
 
Result:
 
 
Azure Portal
 
 
Using Azure CLI
 
You need your key to manipulate your container, you find it at the same place where you found the connection string in the .NET sample.
 
Uploading the blob:
 
 
Validating in Azure Portal
 
Listing Blobs
 
 

Setting and Retrieving properties and Metadata

 
Using Azure CLI
 
Getting Container Properties:
  1. az storage container show --name $containerName --account-name $storageAccountName --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==  
 
Setting and Getting Container Metadata, the attention that the update command overwrites the existing container metadata 
  1. az storage container metadata update --account-name $storageAccountName --name $containerName --metadata creationType=AzureCli --auth-mode key --account-key "P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA=="  
  2. az storage container metadata show --account-name $storageAccountName --name $containerName --auth-mode key --account-key "P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA=="  
 
 
Setting and Getting Blob Metadata, attention that the update command overwrites the existing blob metadata
  1. az storage blob metadata update --name "frequent.jpg" --account-name $storageAccountName --container-name $containerName --metadata creationType=azurecli --auth-mode key --account-key "P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA=="  
  2. az storage blob metadata show --name "frequent.jpg" --account-name $storageAccountName --container-name $containerName --auth-mode key --account-key "P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA=="   
 
Using .NET
 
Getting Container Properties
  1. private static readonly string connectionString = "DefaultEndpointsProtocol=https;AccountName=samplestorageaccountblob;AccountKey=NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==;EndpointSuffix=core.windows.net";  
  2.       private static readonly string containerName = "sampleblobcontainer";  
  3.       static async Task Main(string[] args)  
  4.       {  
  5.   
  6.           Console.WriteLine("Program started. Connecting to Blob Client");  
  7.           BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);  
  8.           BlobContainerClient containerClient = null;  
  9.   
  10.           if (blobServiceClient.GetBlobContainers().Where(x => x.Name == containerName).Any())  
  11.           {  
  12.               Console.WriteLine("Blob Container Client found.");  
  13.               containerClient = blobServiceClient.GetBlobContainerClient(containerName);  
  14.           }  
  15.           else  
  16.           {  
  17.               Console.WriteLine("Blob Container Client not found, creating a new one.");  
  18.               containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName);// Create the container and return a container client object  
  19.               Console.WriteLine("Blob Container Client created.");  
  20.           }  
  21.           var properties = containerClient.GetProperties();  
  22.       }  
Setting and Getting Container Metadata, pay attention that the SetMetada method overwrites the existing metadata.
  1. private static readonly string connectionString = "DefaultEndpointsProtocol=https;AccountName=samplestorageaccountblob;AccountKey=P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA==;EndpointSuffix=core.windows.net";  
  2.         private static readonly string containerName = "sampleblobcontainer";  
  3.          
  4.         static async Task Main(string[] args)  
  5.         {  
  6.   
  7.             Console.WriteLine("Program started. Connecting to Blob Client");  
  8.             BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);  
  9.             BlobContainerClient containerClient = null;  
  10.   
  11.             if (blobServiceClient.GetBlobContainers().Where(x => x.Name == containerName).Any())  
  12.             {  
  13.                 Console.WriteLine("Blob Container Client found.");  
  14.                 containerClient = blobServiceClient.GetBlobContainerClient(containerName);  
  15.             }  
  16.             else  
  17.             {  
  18.                 Console.WriteLine("Blob Container Client not found, creating a new one.");  
  19.                 containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName);// Create the container and return a container client object  
  20.                 Console.WriteLine("Blob Container Client created.");  
  21.             }  
  22.             Dictionary<string, string> containerMetadata = new Dictionary<string, string>();  
  23.             containerMetadata.Add("creationType", ".Net SDK");  
  24.             containerMetadata.Add("seccondMetadata", ".Net SDK");  
  25.             containerClient.SetMetadata(containerMetadata);  
  26.             var containerProperties = containerClient.GetProperties();  
  27.         }  
Setting and Getting Blob Metadata, pay attention that the SetMetada method overwrites the existing metadata.
  1. private static readonly string connectionString = "DefaultEndpointsProtocol=https;AccountName=samplestorageaccountblob;AccountKey=P3o/5qMhTrI8rJgt34SPSaeLuv7ED6Szft5kAf7hixh2UiSwltBW52CG4FejDs3nhV4t2lAE/2XEAIAsUvKorA==;EndpointSuffix=core.windows.net";  
  2.         private static readonly string containerName = "sampleblobcontainer";  
  3.         static async Task Main(string[] args)  
  4.         {  
  5.   
  6.             Console.WriteLine("Program started. Connecting to Blob Client");  
  7.             BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);  
  8.             BlobContainerClient containerClient = null;  
  9.   
  10.             if (blobServiceClient.GetBlobContainers().Where(x => x.Name == containerName).Any())  
  11.             {  
  12.                 Console.WriteLine("Blob Container Client found.");  
  13.                 containerClient = blobServiceClient.GetBlobContainerClient(containerName);  
  14.             }  
  15.             else  
  16.             {  
  17.                 Console.WriteLine("Blob Container Client not found, creating a new one.");  
  18.                 containerClient = await blobServiceClient.CreateBlobContainerAsync(containerName);// Create the container and return a container client object  
  19.                 Console.WriteLine("Blob Container Client created.");  
  20.             }  
  21.   
  22.   
  23.             Dictionary<string, string> blobMetadata = new Dictionary<string, string>();  
  24.             blobMetadata.Add("creationType", ".Net SDK");  
  25.             blobMetadata.Add("seccondMetadata", ".Net SDK");  
  26.             BlobClient client = containerClient.GetBlobClient("frequent.jpg");  
  27.             client.SetMetadata(blobMetadata);  
  28.             var blobMetadataValues = client.GetProperties().Value.Metadata;  
  29.         }  
 
Result from Azure Portal 
 
 
Moving items between Blob Storage and Containers
 
You can copy a single Blob or a Batch of Blobs, here we will be moving a batch of Blobs from one container to another inside the same Blob Storage Account. 
 
The error here is due to the Rare.jpg being rehydrated. 
 
  1. az storage blob copy start-batch --account-name $storageAccountName --destination-container "anothercontainer" --source-container $containerName --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==   
Moving from one container to another inside another storage account.
  1. az storage blob copy start-batch --account-name "cloudshell325844558" --source-account-name $storageAccountName --source-container "anothercontainer" --destination-container "containeranotheraccount" --account-key Oc3mEd/80DqJ65+nxMUJb/C9nq/hre+YLdVEfXeZ7M3gvcwJ7zqB7SAzNaiRtPyOrEtYXBub9Wn4xVUgTv5BTw== --source-account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==   
 
Hot, Cool, and Archiving Storage
 
By default, when you create a Blob its access tier is set automatically to the one configured when creating Azure Storage. And by default, Blob Storages are created with the Hot Access Tier, in order to change the default access tier of the Storage Account, check the configuration tab under the settings sector.
 
 
To change the access tier of a blob you have to individually set its new access tier. We are going to organize the blobs according to their necessities: frequent image stays in the hot tier, not so frequent image stays in the cool tier and the rare image is going to be placed in the archiving tier.
 
 
  1. az storage blob set-tier --account-name $storageAccountName --container-name $containerName --name "not so frequent.png" --tier Cool --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==  
  2. az storage blob set-tier --account-name $storageAccountName --container-name $containerName --name "rare.jpg" --tier Archive --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==   
Result in Azure Portal
 
 
To move one item from Archive Access Tier to another Access Tier you can also set the rehydrate priority to high. This feature is still in preview mode.
 
  1. az storage blob set-tier --account-name $storageAccountName --container-name $containerName --name "rare.jpg"  --tier Cool --rehydrate-priority high   --account-key NSs2H9xg8DIpKH8y3EeYMiIpM1zYl/K9oAlBvDnPJaOpwUuS1GIwnbllIBuWngVpUYkpzmiRHdJiybgmGR3tag==  
 

Implementing Archiving and Retention - Lifecycle Management

 
In order to manage the lifecycle of our Blobs inside containers, let's create a new lifecycle management rule. From Azure Storage Account go to Lifecycle Management, under Blob Service, and Add a rule.
 
You may apply this rule to all blobs inside the storage account or filter the blobs to have this rule applied in this storage account. 
 
 
Now you can configure how many days without being modified those blobs are going to be moved to other access tiers. 
 
 
For now, that is it!
 
Thanks for reading this article, if you enjoyed it and it was helpful do not forget to like it. If you did not like it, do not forget to comment and tell me what I can improve in future articles. 
External References:

Similar Articles