Databases & DBA  

Creating a Multi-Tier Attachment Storage System (Hot, Warm, Cold Storage)

Introduction

Modern enterprise applications handle thousands of files: invoices, images, compliance documents, user uploads, engineering drawings, reports, and archives.
Over time, these files grow so large that:

  • storage becomes expensive

  • performance degrades

  • backups become slow

  • access latency varies

  • old files are rarely used but occupy prime (expensive) storage

To control cost, improve performance, and maintain compliance, organisations adopt multi-tier storage .
This article explains how to build a Hot–Warm–Cold tiered document system using Angular (front-end) and ASP.NET Core (backend) , with support for:

  • dynamic tier selection

  • automated tier migration

  • lifecycle policies

  • leases, locks, and write protection

  • secure URL generation

  • unified retrieval across tiers

  • cost-optimised storage architecture

This is a production-grade guide targeted at senior full-stack developers.

What is multi-tier attachment storage?

A multi-tier storage system splits file storage into logical tiers:

Hot Storage

  • stored locally or fast object storage (SSD-backed)

  • used for recently uploaded, frequently accessed files

  • low latency, high cost

Warm Storage

  • stored in medium-cost storage (HDD, slower buckets)

  • used for files with moderate access frequency

  • slightly slow, medium cost

Cold Storage

  • archival, long-term storage

  • very cheap (S3 Glacier, Azure Archive, GCP Coldline)

  • slower access and retrieval delays possible

Key goals of the architecture

  • Reduce storage cost per file

  • Provide seamless retrieval regardless of tier

  • Automatically migrate files based on lifecycle rules

  • Support metadata-driven access patterns

  • Implement domain-level rules: lease locks, expiry, policies

  • Ensure compliance and auditability

System architecture overview

Below is the high-level architecture.

components

  • Angular Front-End

    • Upload component

    • Versioning and metadata panel

    • Preview and download module

    • Tier badge: Hot/Warm/Cold

    • Fetching signed URLs

  • .NET API

    • Unified File Controller

    • Tiered Storage Provider

    • File Metadata Store (SQL Server)

    • Migration Engine (Hosted Service)

    • Security: Permission + Time-limited URLs

    • Lock Manager

  • Storage Providers

    • Hot: Local Disk / Azure Blob Hot / S3 Standard

    • Warm: Medium-tier bucket

    • Cold: Archive storage

Workflow diagram (text-based)

  
    +------------------------+
                        |        Angular UI      |
                        | Upload / Preview / DL  |
                        +-----------+------------+
                                    |
                                    v
                      +-----------------------------+
                      |         .NET API            |
                      | File Controller / Metadata  |
                      +-----------+-----------------+
                                  |
                 +----------------+-------------------+
                 |                |                   |
                 v                v                   v
      +----------------+ +----------------+ +----------------+
      |   Hot Tier     | |   Warm Tier    | |   Cold Tier    |
      |   Fast Store   | | Medium Storage | |  Archive Bucket |
      +----------------+ +----------------+ +----------------+
                                  |
                                  v
                     +-----------------------------+
                     | Migration Engine (Hosted)   |
                     | Lifecycle Rules & Policies  |
                     +-----------------------------+
  

Flowchart: upload → store → migrate

  
    Start
  |
  v
Receive Upload Request
  |
  v
Store Metadata in DB
  |
  v
Save File to Hot Tier
  |
  v
Assign Tier = HOT
  |
  v
Scheduled Migration Job Runs
  |
  +--- If last accessed < 30 days? --> Keep in Hot
  |
  +--- If last accessed > 30 days --> Move to Warm
  |
  +--- If last accessed > 180 days --> Move to Cold
  |
  v
Update Metadata Tier & Location
  |
  v
End
  

Detailed database design (SQL Server)

Table: Attachment

ColumnTypeDescription
AttachmentIdbigintPK
FileNamenvarchar(255)Original name
ContentTypenvarchar(200)MIME
CurrentTiertinyint1=Hot,2=Warm,3=Cold
StoragePathnvarchar(max)Actual path or bucket key
FileSizebigintBytes
CreatedOndatetime2
LastAccessedOndatetime2Updated during read
VersionintOptional versioning
LockedBybigintLease owner
LockExpiresOndatetime2
IsDeletedbitSoft delete

Indexes recommended:

  • IX_Attachment_LastAccessedOn

  • IX_Attachment_CurrentTier

  • IX_Attachment_StoragePath

Lifecycle policy examples

  1. Hot → Warm after 30 days inactivity

  2. Warm → Cold after 180 days inactivity

  3. Cold → Delete after 5 years (optional)

  4. Locked documents never migrate until lock expires

Angular front-end implementation

A production-ready solution contains:

Modules

  • AttachmentModule

  • AttachmentUploadComponent

  • AttachmentListComponent

  • AttachmentPreviewer

  • StorageBadgeDirective

Upload component (Angular)

  
    uploadFile(event: any) {
  const file = event.target.files[0];
  if (!file) return;

  const formData = new FormData();
  formData.append('file', file);

  this.http.post('/api/attachments/upload', formData)
    .subscribe(res => {
      this.loadList();
    });
}
  

Display tier badge

  
    <span [ngClass]="{
  'tier-hot': row.currentTier === 1,
  'tier-warm': row.currentTier === 2,
  'tier-cold': row.currentTier === 3
}">
  {{ getTierName(row.currentTier) }}
</span>
  

Badge styles

  
    .tier-hot { color: #e53935; font-weight: bold; }
.tier-warm { color: #fb8c00; font-weight: bold; }
.tier-cold { color: #1e88e5; font-weight: bold; }
  

Preview/download workflow

Angular retrieves a signed URL .

  
    download(id: number) {
  this.http.get(`/api/attachments/${id}/signed-url`)
    .subscribe((url: string) => {
      window.open(url, '_blank');
    });
}
  

This hides the storage tier from the user.

.NET backend implementation

Controller: unified entry point

  
    [ApiController]
[Route("api/attachments")]
public class AttachmentController : ControllerBase
{
    private readonly IAttachmentService _service;

    public AttachmentController(IAttachmentService service)
    {
        _service = service;
    }

    [HttpPost("upload")]
    public async Task<IActionResult> Upload(IFormFile file)
    {
        var id = await _service.UploadAsync(file);
        return Ok(new { id });
    }

    [HttpGet("{id}/signed-url")]
    public async Task<IActionResult> GetSignedUrl(long id)
    {
        var url = await _service.GetSignedUrlAsync(id);
        return Ok(url);
    }
}
  

Service: upload into hot tier

  
    public async Task<long> UploadAsync(IFormFile file)
{
    var id = await _repository.CreateMetadataAsync(file);

    string path = await _hotStorage.SaveAsync(id, file);

    await _repository.UpdateStoragePath(id, path, StorageTier.Hot);

    return id;
}
  

Storage provider interface

  
    public interface IStorageProvider
{
    Task<string> SaveAsync(long id, IFormFile file);
    Task<Stream> GetAsync(string path);
    Task<string> GenerateSignedUrl(string path, TimeSpan expiry);
    Task MoveAsync(string oldPath, string newPath);
}
  

Three provider implementations

  • HotStorageProvider : Local directory

  • WarmStorageProvider : Azure Blob Standard

  • ColdStorageProvider : Azure Archive / S3 Glacier

For example:

  
    public class HotStorageProvider : IStorageProvider
{
    public async Task<string> SaveAsync(long id, IFormFile file)
    {
        var path = Path.Combine("hot", $"{id}_{file.FileName}");
        using (var stream = new FileStream(path, FileMode.Create))
        {
            await file.CopyToAsync(stream);
        }
        return path;
    }

    public Task<string> GenerateSignedUrl(string path, TimeSpan expiry)
    {
        // return API URL, not direct file path
        return Task.FromResult($"/api/attachments/stream?path={path}");
    }
}
  

Migration engine (background service)

This is the heart of the system.

  
    public class TierMigrationService : BackgroundService
{
    private readonly IAttachmentRepository _repo;
    private readonly IStorageProvider _hot;
    private readonly IStorageProvider _warm;
    private readonly IStorageProvider _cold;

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            await RunMigrationCycle();
            await Task.Delay(TimeSpan.FromHours(6), stoppingToken);
        }
    }

    private async Task RunMigrationCycle()
    {
        var candidates = await _repo.GetMigrationCandidatesAsync();

        foreach (var file in candidates)
        {
            if (file.CurrentTier == StorageTier.Hot &&
                file.LastAccessedOn < DateTime.UtcNow.AddDays(-30))
            {
                await Move(file, _hot, _warm, StorageTier.Warm);
            }
            else if (file.CurrentTier == StorageTier.Warm &&
                     file.LastAccessedOn < DateTime.UtcNow.AddDays(-180))
            {
                await Move(file, _warm, _cold, StorageTier.Cold);
            }
        }
    }

    private async Task Move(Attachment file,
        IStorageProvider source,
        IStorageProvider target,
        StorageTier newTier)
    {
        var newPath = file.StoragePath.Replace("hot", "warm")
                                      .Replace("warm", "cold");

        await source.MoveAsync(file.StoragePath, newPath);
        await target.MoveAsync(file.StoragePath, newPath);
        await _repo.UpdateTier(file.AttachmentId, newTier, newPath);
    }
}
  

Unified retrieval logic

  
    public async Task<string> GetSignedUrlAsync(long id)
{
    var meta = await _repo.GetAsync(id);

    IStorageProvider provider = meta.CurrentTier switch
    {
        StorageTier.Hot => _hot,
        StorageTier.Warm => _warm,
        StorageTier.Cold => _cold,
        _ => throw new Exception("Unknown tier.")
    };

    await _repo.TouchLastAccessed(id);

    return await provider.GenerateSignedUrl(meta.StoragePath, TimeSpan.FromMinutes(10));
}
  

Implementing leases and locks

Prevent overwriting/migration during editing.

  
    public async Task<bool> AcquireLease(long id, long userId)
{
    var item = await _repo.GetAsync(id);

    if (item.LockExpiresOn > DateTime.UtcNow)
        return false;

    return await _repo.SetLock(id, userId, DateTime.UtcNow.AddMinutes(30));
}
  

Locks automatically release after expiry.

Security and compliance best practices

  1. Never expose raw storage paths

  2. Always use time-limited signed URLs

  3. Encrypt sensitive documents

  4. Log every read attempt

  5. Do not allow migration of leased files

  6. Maintain audit trails: who uploaded, who accessed

  7. Keep SHA-256 hash to detect tampering

Cost-optimization strategies

  • Store thumbnails in hot tier; originals in warm

  • Use compression for warm and cold storage

  • Pre-migrate old documents during off-peak hours

  • Use file batching during migration

  • Avoid frequent warm → hot movement

  • Cold tier retrieval should require explicit approval

Testing strategy

Unit tests

  • Tier selection logic

  • Metadata repository tests

  • Storage provider mocks

  • Lock acquisition and expiry

Integration tests

  • Upload and GET signed URL flow

  • Migration workflow across tiers

  • Permission enforcement

Load tests

  • Bulk upload under high concurrency

  • Migration of 10k documents

  • Cold storage retrieval simulations

Real-world production considerations

  • Disaster recovery replication for hot tier

  • Object lock retention for cold tier compliance

  • Policy-based access control using RBAC

  • Versioning of attachments

  • Handling extremely large files using chunked uploads

  • Global CDN caching for public documents

  • Multi-region warm and hot storage

Summary

A Multi-Tier Attachment Storage System is essential for modern enterprise applications that demand:

  • cost-efficient long-term storage

  • fast access for recent documents

  • seamless tier-aware retrieval

  • automated lifecycle management

  • compliance, auditability, and security

In this article, we implemented a complete end-to-end solution using:

  • Angular (UI)

  • ASP.NET Core (API, storage controller, migration engine)

  • SQL Server (metadata storage)

  • Hot, Warm, Cold storage providers

  • Lifecycle and migration policies

  • Locking and lease protections

This architecture is ready to be applied to ERPs, CRMs, DMSs, invoicing systems, workflow platforms, or any enterprise system that manages large volumes of documents.