Skip to content
Cloudbricks.dev logo

Apache CloudStack — Storage

Part 4 of the Apache CloudStack series: primary and secondary storage, volumes, templates, snapshots, and object storage.

Kishore 4 min read

CloudStack separates storage into two roles, and keeping them straight explains most of the behaviour that otherwise looks odd.

Primary storage holds the disks that running instances use. It is fast, local to the compute cluster, and measured in IOPS as much as capacity.

Secondary storage holds the artefacts: templates, ISOs, snapshots, and volume backups. It is shared, usually larger and slower, and every zone needs one.

Primary storage

A primary storage pool is presented to a cluster. CloudStack supports several backends:

BackendProtocolNotes
NFSNFSv3/v4Simplest for a lab; one export per pool
iSCSIiSCSIRequires multipath configuration on the host
Fibre ChannelFCNeeds zoning and multipath
Local storageLocal diskNo live migration; tied to one host
Ceph RBDRADOSShared, scalable; requires the Ceph plugin
PowerFlexSDCDell/EMC-specific

For a first lab, NFS is the least painful choice. On the KVM host, export a directory and mount it, then add it to CloudStack as a primary storage pool with the same path.

# On the storage server
sudo mkdir -p /export/primary
echo "/export/primary *(rw,async,no_root_squash)" | sudo tee -a /etc/exports
sudo exportfs -ra
# On the KVM host
sudo mkdir -p /mnt/primary
sudo mount -t nfs storage.lab:/export/primary /mnt/primary

Add the pool in the UI under Infrastructure → Primary Storage, or with the API:

cmk create storagepool \
  name=primary-nfs \
  zoneid=<zone-id> \
  podid=<pod-id> \
  clusterid=<cluster-id> \
  url=nfs://storage.lab/export/primary

Storage tags let you steer volumes to a particular pool. Tag the pool (for example ssd), then create a disk offering with the same tag. A volume whose offering requests ssd will only be placed on a pool carrying that tag.

Secondary storage

Secondary storage is added per zone and holds:

  • System VM templates
  • Guest templates and ISOs
  • Volume snapshots
  • Volume backups
  • Instance snapshots

NFS is the common choice; S3-compatible object storage and Swift are also supported. The secondary storage must be reachable from the management server and from the system VMs.

cmk add secondarystorage \
  zoneid=<zone-id> \
  url=nfs://storage.lab/export/secondary

CloudStack copies templates to secondary storage once, then provisions instances from that copy. If secondary storage is slow or full, template registration and instance deployment both stall.

Disk offerings

A disk offering describes the size and performance characteristics of a volume. It can be:

  • Custom — the user chooses the size at creation time.
  • Fixed — the size is set by the offering.
  • Tagged — restricted to storage pools with a matching tag.
cmk create diskoffering \
  name=ssd-50gb \
  displaytext="50 GB SSD" \
  disksize=50 \
  storagetype=shared \
  tags=ssd

The storagetype value matters: shared for NFS/iSCSI/Ceph, local for host-local disks. A mismatch between the offering and the pool is a common reason a volume stays in the Allocated state.

Volumes, snapshots, and backups

These three are related but not interchangeable:

  • A volume is a block device attached to an instance.
  • A snapshot is a point-in-time copy of a volume, stored in secondary storage. It is tied to the volume and is the basis for restoring or cloning.
  • A backup is a copy of a volume that can be used to create a new volume independently. Backups are the disaster-recovery artefact.

Create and attach a volume:

cmk create volume \
  name=data-01 \
  zoneid=<zone-id> \
  diskofferingid=<disk-offering-id>

cmk attach volume \
  id=<volume-id> \
  virtualmachineid=<vm-id>

Snapshot it:

cmk create snapshot volumeid=<volume-id>
cmk list snapshots volumeid=<volume-id>

Create a template from a volume (useful for golden images):

cmk create template \
  displaytext="Ubuntu 24.04 base" \
  name=ubuntu-2404-base \
  ostypeid=<os-type-id> \
  volumeid=<volume-id>

Templates and ISOs

Templates are the images instances boot from. You can:

  • Register an existing qcow2/raw/vmdk image.
  • Create a template from a volume or an instance.
  • Copy a template to another zone.
cmk register template \
  name=ubuntu-2404 \
  displaytext="Ubuntu 24.04" \
  url=https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img \
  zoneid=<zone-id> \
  format=qcow2 \
  hypervisor=KVM \
  ostypeid=<os-type-id>

CloudStack downloads the image to secondary storage, then makes it available to the zone. Watch the template status: Downloading → Downloaded → Ready. A template stuck in Downloading is almost always a URL, DNS, or secondary-storage problem.

Object storage

CloudStack’s object storage feature exposes S3-compatible storage to users, backed by a Ceph RADOS Gateway or another supported provider. It is separate from primary/secondary storage and is configured as a zone-level service. Enable it only if you need S3 semantics; block volumes and secondary storage cover most lab use cases.

Troubleshooting

  • Volume stuck in Allocated. The disk offering tag does not match any pool, or the pool is full.
  • Snapshot fails. Check secondary storage capacity and NFS permissions.
  • Template download fails. Test the URL from the management server and from a system VM; DNS is a frequent culprit.
  • Live migration fails. Local storage cannot be migrated. Use shared storage or cold-migrate.
  • Instance disk fills the pool. Thin provisioning reports allocated size, not physical usage. Monitor the storage backend directly.