Apache CloudStack — Storage
Part 4 of the Apache CloudStack series: primary and secondary storage, volumes, templates, snapshots, and object storage.
CloudStack separates storage into two roles, and keeping them straight explains most of the behaviour that otherwise looks odd.
Primary storage holds the disks that running instances use. It is fast, local to the compute cluster, and measured in IOPS as much as capacity.
Secondary storage holds the artefacts: templates, ISOs, snapshots, and volume backups. It is shared, usually larger and slower, and every zone needs one.
Primary storage
A primary storage pool is presented to a cluster. CloudStack supports several backends:
| Backend | Protocol | Notes |
|---|---|---|
| NFS | NFSv3/v4 | Simplest for a lab; one export per pool |
| iSCSI | iSCSI | Requires multipath configuration on the host |
| Fibre Channel | FC | Needs zoning and multipath |
| Local storage | Local disk | No live migration; tied to one host |
| Ceph RBD | RADOS | Shared, scalable; requires the Ceph plugin |
| PowerFlex | SDC | Dell/EMC-specific |
For a first lab, NFS is the least painful choice. On the KVM host, export a directory and mount it, then add it to CloudStack as a primary storage pool with the same path.
# On the storage server
sudo mkdir -p /export/primary
echo "/export/primary *(rw,async,no_root_squash)" | sudo tee -a /etc/exports
sudo exportfs -ra
# On the KVM host
sudo mkdir -p /mnt/primary
sudo mount -t nfs storage.lab:/export/primary /mnt/primary
Add the pool in the UI under Infrastructure → Primary Storage, or with the API:
cmk create storagepool \
name=primary-nfs \
zoneid=<zone-id> \
podid=<pod-id> \
clusterid=<cluster-id> \
url=nfs://storage.lab/export/primary
Storage tags let you steer volumes to a particular pool. Tag the pool (for example ssd), then create a disk offering with the same tag. A volume whose offering requests ssd will only be placed on a pool carrying that tag.
Secondary storage
Secondary storage is added per zone and holds:
- System VM templates
- Guest templates and ISOs
- Volume snapshots
- Volume backups
- Instance snapshots
NFS is the common choice; S3-compatible object storage and Swift are also supported. The secondary storage must be reachable from the management server and from the system VMs.
cmk add secondarystorage \
zoneid=<zone-id> \
url=nfs://storage.lab/export/secondary
CloudStack copies templates to secondary storage once, then provisions instances from that copy. If secondary storage is slow or full, template registration and instance deployment both stall.
Disk offerings
A disk offering describes the size and performance characteristics of a volume. It can be:
- Custom — the user chooses the size at creation time.
- Fixed — the size is set by the offering.
- Tagged — restricted to storage pools with a matching tag.
cmk create diskoffering \
name=ssd-50gb \
displaytext="50 GB SSD" \
disksize=50 \
storagetype=shared \
tags=ssd
The storagetype value matters: shared for NFS/iSCSI/Ceph, local for host-local disks. A mismatch between the offering and the pool is a common reason a volume stays in the Allocated state.
Volumes, snapshots, and backups
These three are related but not interchangeable:
- A volume is a block device attached to an instance.
- A snapshot is a point-in-time copy of a volume, stored in secondary storage. It is tied to the volume and is the basis for restoring or cloning.
- A backup is a copy of a volume that can be used to create a new volume independently. Backups are the disaster-recovery artefact.
Create and attach a volume:
cmk create volume \
name=data-01 \
zoneid=<zone-id> \
diskofferingid=<disk-offering-id>
cmk attach volume \
id=<volume-id> \
virtualmachineid=<vm-id>
Snapshot it:
cmk create snapshot volumeid=<volume-id>
cmk list snapshots volumeid=<volume-id>
Create a template from a volume (useful for golden images):
cmk create template \
displaytext="Ubuntu 24.04 base" \
name=ubuntu-2404-base \
ostypeid=<os-type-id> \
volumeid=<volume-id>
Templates and ISOs
Templates are the images instances boot from. You can:
- Register an existing qcow2/raw/vmdk image.
- Create a template from a volume or an instance.
- Copy a template to another zone.
cmk register template \
name=ubuntu-2404 \
displaytext="Ubuntu 24.04" \
url=https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img \
zoneid=<zone-id> \
format=qcow2 \
hypervisor=KVM \
ostypeid=<os-type-id>
CloudStack downloads the image to secondary storage, then makes it available to the zone. Watch the template status: Downloading → Downloaded → Ready. A template stuck in Downloading is almost always a URL, DNS, or secondary-storage problem.
Object storage
CloudStack’s object storage feature exposes S3-compatible storage to users, backed by a Ceph RADOS Gateway or another supported provider. It is separate from primary/secondary storage and is configured as a zone-level service. Enable it only if you need S3 semantics; block volumes and secondary storage cover most lab use cases.
Troubleshooting
- Volume stuck in
Allocated. The disk offering tag does not match any pool, or the pool is full. - Snapshot fails. Check secondary storage capacity and NFS permissions.
- Template download fails. Test the URL from the management server and from a system VM; DNS is a frequent culprit.
- Live migration fails. Local storage cannot be migrated. Use shared storage or cold-migrate.
- Instance disk fills the pool. Thin provisioning reports allocated size, not physical usage. Monitor the storage backend directly.
