Software Engineering Life

Storage Accounts

Properties1
aliases

5 min read

Up:: Microsoft Azure Developer Associate AZ-204 2023

Storage Accounts

Introduction to Storage Accounts

  • 05:02:37 Introduction to Storage Accounts
    • 5 Core storage services
      • Azure Blob - Scalable object store - big data
      • Azure Files - Managed File shares for cloud or on premises deployments
      • Azure Queues - NoSQL Store for schemaless storage of structured data
      • Azure Tables - reliable messaging between application components
      • Azure Disks - Block-level storage volumes for Azure VMs
    • Performance Tiers (Blob Storage)
      • Standard
        • HDD
        • Use cases
          • Backup
          • Media
          • bulk data processing
      • Premium
        • SSD
        • Use Cases
      • Think about IOPS (Input output operations per second)
    • Access Tiers (Blob Storage)
      • 3 Types for standard storage
        • Hot
          • Data that is accessed frequently
          • Highest storage cost, lowest access cost
        • Cool
          • Data that is infrequently accessed and stored for at least 30 days
          • Lower storage cost, higher access cost
        • Archive
          • Data that is rarely accessed and is stored for at least 180 days
          • Lowest storage cost but highest access cost
      • Account Level Tiering
        • Assigned to blobs that does not have explicit tier assigned to it, inherits from the Storage Account access tier setting
      • Blob-level Tiering
        • Can upload blob to tier of your choice
        • Changing tiers happens instantly with the exception of moving out of archive
      • Rehydrating a blob
        • When moving a blob out of archive to a different tier. This is known as “rehydrating”
      • Blob lifecycle Management
        • Create rule-based policies to transition data to different tiers
        • After 30 days move to cool storage
      • When uploaded or moved to another tier, it is charged at the new tiers rate immediately upon tier change
        • When moving from a Cooler Tier
          • operation is billed as a write operation to the destination tier
          • write operation (per 10000) and data write (per Gb) charges of the destination tier apply
        • When moving from a Hotter tier
          • operation is billed as a read from the source tier
          • read operation (per 10,000) and data retrieval(per Gb) charges of the source tier apply
          • Early deletion changes for any blob moved out of the cool or archive tier may apply as well
        • Cool and Archive early deletion
          • Any blob that is moved into the cool tier is subject to a cool early deletion period of 30 days
          • blob moved into archive tier is subject to a archive early deletion period of 180 days. This charge is prorated.
    • AZCopy
      • Command line utility to copy blobs or files to and from Storage Account
      • Need the following authorization level attached via roles
        • To Download
          • Storage Blob Data Reader
        • To Upload
          • Storage Blob data Contributor
          • Storage Blob data Owner
      • Gain Access
        • Azure Active Directory
        • Shared Access Signature
      • azcopy copy 'source' 'destination'
        • azcopy copy 'c:\test.txt' 'https://enterprise.blob.cor.windows.net/mycontainer/test.txt' - Upload
        • azcopy copy 'https://enterprise.blob.cor.windows.net/mycontainer/test.txt' 'c:\test.txt' - Download
    • Lifecycle Management
      • Rule-based policy to transition blob data to the appropriate access tiers or to expire data at the end of the data lifecycle
      • You can
        • transition blobs from cool to hot when they are accessed to optimize for performance
        • transition blobs, blob versions and blob snapshots to a cooler storage tier if the object has not been accessed for a period of time to optimize for cost
        • delete blobs, blob version and blob snapshots if it has reached the end of its lifecycle.
        • Define rules to be run once per day at the storage account level.
        • Apply rules to the container or to a subset of blobs, using name prefixes or blob index tags as filters
    • Setting and Retrieving properties and Metadata
      • Using Azure CLI
      • Getting container properties(show)
        • az storage container show --account-name $storageAccountName --name $conatinerName --account-key $accountKey
      • Setting container metadata (update)
        • az storage container metadata update --account-name $storageAccountName --name $containerName --metadata creationType=AzureCli --auth-mode key --acount-key $accountKey
      • Getting the container metadata (show)
        • az storage container metadata show --account-name $storageAccountName --name $containerName --metadata creationType=AzureCli --auth-mode key --account-key $accountKey
      • Setting the Blob Metadata
        • az storage blob metadata update --name "TestImage.png" --account-name $storageAccountName --container-name $containerName --metadata creationType=AzureCli --auth-mode key --account-key $accountKey
      • Getting the Blob Metadata
        • az storage blob metadata show --name "TestImage.png" --account-name $storageAccountName --container-name $containerName --metadata creationType=AzureCli --auth-mode key --account-key $accountKey
    • AZCopy Follow Along
      • copy command works
        • Could not log into azcopy using Azure AD
        • SAS did not work in console but did work when I copied in windows
    • Interact with Storage Account with SDK Follow Along
      • Github
        • GitPod
          • export ACCOUNT_NAME=""
          • gp env ACCOUNT_NAME=""
          • export ACCOUNT_KEY=""
          • gp env ACCOUNT_KEY=""
        • JS
          • npm init -y
          • npm i @azure/storage-blob
    • 05:58:47 Modify metadata of blob storage objects Follow Along
      • Install in local PowerShell: Install-Module -Name Az
PS /home/username> $ResourceGroup = "my-blob-metadata"
PS /home/username> $StorageAccount  = "myblobmetadata8282"
PS /home/username> $StorageKey = (Get-AzStorageAccountKey -ResourceGroupName $ResourceGroup -Name $StorageAccount)    
        
PS /home/username> $Context = New-AzStorageContext -StorageAccountName $StorageAccount -StorageAccountKey $StorageKey[0].Value
PS /home/username> $Context
 
 PS > $Blob = Get-AzStorageBlob -Blob kavisfajo.png -Container container1 -Context $Context
 PS > $Blob.BlobClient.GetProperties().value
 
# Create Metadata
PS /home/username> $Metadata = New-Object System.Collections.Generic.Dictionary"[String,String]"
PS /home/username> $Metadata.Add("Author","username")                                            
PS /home/username> $Metadata.Add("Department","IT") 
PS /home/username> $Blob.BlobClient.SetMetadata($Metadata, $null)
  • 06:08:54 Copy blob storage between containers
    • Get the file
      • Get-AzStorageBlobContent -Blob kavisfajo.png -Container container1 -Context $Context -Destination .
    • Upload to next container:
      • Set-AzStorageBlobContent -File ./kavisfajo.png -Container container2 -Context $Context -Properties @{"ContentType"="image/png"}

Additional Metadata

Graph View

Backlinks