Quick Summary
Fastest path: Vultr — paste an ISO URL, boot, install via VNC. Done in 10 minutes. No limits: Hetzner — rescue mode + dd, any image, any size. VM migration: DigitalOcean — upload raw, qcow2, vdi, or vmdk images directly. Out of luck: Contabo, InterServer, Hostinger, RackNerd — none of them support it.
Table of Contents
What Is Custom ISO Upload?
Same concept as burning a disc and booting a physical machine from it, except the "disc" is a file and the "machine" is a virtual server in somebody else's datacenter. You provide an ISO image — the same .iso file you would download from freebsd.org or nixos.org — and the provider mounts it as a virtual CD-ROM. Your VPS boots from it, you run through the installer via a web-based VNC console, and when you are done, you detach the ISO and reboot from disk.
One hard requirement: KVM virtualization. OpenVZ containers share the host kernel, so there is no concept of booting a custom OS — you get whatever the host runs. If a provider does not mention their virtualization type, ask before you buy.
Why Custom ISO Matters
If you are happy with Ubuntu 24.04 and Debian 12, skip this page entirely — the provider templates work fine. But custom ISO becomes non-negotiable the moment you need any of these:
- Firewall/router OS: pfSense, OPNsense, or VyOS for building a cloud firewall or VPN gateway
- Niche Linux distributions: NixOS, Gentoo, Arch Linux, Alpine, or custom-compiled kernels
- BSD variants: FreeBSD, OpenBSD, or NetBSD that may not appear in the provider's template list
- Windows with your own license: Install Windows Server from your own ISO to avoid the provider's licensing markup
- Security-hardened images: Pre-configured images with your organization's security baseline baked in
- Recovery tools: Boot into SystemRescue or Hiren's Boot CD for diagnostics
- Specialized appliances: Sophos UTM, Proxmox VE nested inside a VPS, or TrueNAS for cloud storage
Provider Support Comparison
I checked every major US VPS provider. The results are split almost evenly — and the dividing line is price tier, not technical capability:
| Provider | Custom ISO | Upload Method | Max Size | Supported Formats | API Support |
|---|---|---|---|---|---|
| Vultr | Yes | URL upload | 10 GB | ISO | Yes |
| Hetzner | Yes | Rescue mode + dd | Unlimited | Any | Partial |
| DigitalOcean | Yes | Custom Images API | 100 GB | raw, qcow2, vdi, vmdk | Yes |
| Kamatera | Yes | Support ticket | 50 GB | ISO | No |
| Linode | Yes | Custom Images | 6 GB | raw, img | Yes |
| Contabo | No | N/A | N/A | N/A | N/A |
| InterServer | No | N/A | N/A | N/A | N/A |
| Hostinger | No | N/A | N/A | N/A | N/A |
| RackNerd | No | N/A | N/A | N/A | N/A |
| BuyVM | Yes | Upload via panel | 10 GB | ISO | No |
The pattern is obvious: every budget provider (Contabo, InterServer, Hostinger, RackNerd) says no. They run simplified provisioning stacks that only handle templates. If custom ISO is a hard requirement for your project, this table eliminates four providers before you even look at pricing.
Provider-by-Provider Breakdown
Vultr — Best Overall for Custom ISO
Vultr nailed this. Paste a URL, they download the ISO to their storage, and you can boot any instance from it in under 5 minutes. I have uploaded pfSense, NixOS, FreeBSD 14, and a Windows Server 2022 ISO without a single issue. The 10GB limit has never been a problem — even the largest Linux installers are under 5GB. API support means you can script the whole process, and ISOs persist in your account until you explicitly delete them. If custom ISO is your primary reason for choosing a provider, Vultr is the obvious answer.
Hetzner — Most Flexible (No Size Limit)
Hetzner does not have an "Upload ISO" button. Instead, you boot into their rescue system — a minimal Linux environment — and use dd or wget to write whatever you want directly to the disk. No size limits. No format restrictions. If it boots on x86, it works. The price you pay is complexity: you need to know what you are doing with disk images and partition tables. But for power users, this is the most flexible option on the market, and Hetzner's prices ($4.15/mo for a CX22) make it even more appealing.
DigitalOcean — Most Formats Supported
DigitalOcean is the right choice if you are migrating an existing VM. Export your VirtualBox image as .vdi, your VMware image as .vmdk, or your QEMU image as .qcow2 — DigitalOcean accepts all of them, plus raw disk images. The 100GB limit is absurdly generous. The catch: this is a custom *image* system, not an ISO boot system. You upload a pre-installed disk, not an installer. That means no interactive installation, but also no installation step at all — your VM just starts running.
Linode — Limited but Functional
Linode technically supports custom images, but the 6GB compressed size limit makes it frustrating. A full Ubuntu server image compresses to about 1.5GB, so lightweight distributions fit fine. But try uploading a Windows image or anything with a desktop environment and you will hit that wall fast. I have used it for deploying pre-built Alpine and NixOS images, and for that narrow use case, it works. Just do not expect the same flexibility as Vultr or Hetzner.
BuyVM — Simple Panel Upload
BuyVM flies under the radar, but their Stallion control panel has a clean ISO upload feature. Direct upload, 10GB limit, and the VNC console works well enough for interactive installations. No API, so you cannot automate it, but for a one-off pfSense or FreeBSD install on their $3.50/mo KVM plans, it does the job. BuyVM also offers unmetered bandwidth, which is a nice bonus if you are running a VPN gateway from your custom OS.
Kamatera — Via Support Ticket
Kamatera supports it, but you have to ask. Submit a support ticket with the ISO URL and their team mounts it for you. I waited about 4 hours for my last request, which is fine if you are not in a hurry but annoying if you are iterating on a custom image. The 50GB limit is generous. Kamatera's real strength is their per-hour billing and global datacenter spread — if you need a custom OS in Tel Aviv or Hong Kong, this is one of your few options.
Common Use Cases for Custom ISO
pfSense / OPNsense Cloud Firewall
This is the single most common reason people need custom ISO, and it is the use case I have the most experience with. A $6/mo Vultr instance running pfSense gives you a cloud firewall with IDS/IPS, site-to-site VPN, and traffic shaping — capabilities that would cost $50/mo from a managed firewall provider. OPNsense works equally well. Neither appears in any provider's template list, so custom ISO is the only path. I run my pfSense instance on Vultr specifically because their VNC console handles the installer without any quirks, and the 1GB RAM plan is enough for a basic firewall serving 10-20 VPN clients.
NixOS / Guix / Immutable Linux
Once you have managed a NixOS server, going back to mutable Linux feels reckless. Your entire server config lives in one file, version-controlled in Git, and you can rebuild the identical system from scratch in minutes. The problem is that NixOS requires its own installer — there is no converting an Ubuntu box after the fact. Upload the NixOS minimal ISO (about 900MB), partition the disk, and run nixos-install. Here is what a basic VPS config looks like:
# NixOS configuration.nix snippet for a VPS
{ config, pkgs, ... }:
{
boot.loader.grub.device = "/dev/vda";
networking.hostName = "my-vps";
services.openssh.enable = true;
services.nginx.enable = true;
system.stateVersion = "24.11";
}
FreeBSD & OpenBSD
FreeBSD shows up as a template on Vultr and DigitalOcean, but OpenBSD almost never does. And OpenBSD is the one you actually want for security-critical work — its default install is tighter than most hardened Linux distributions. Custom ISO lets you run the official OpenBSD installer, set up full-disk encryption with softraid, and configure the system from the ground up. I have run OpenBSD firewalls on Vultr for years. The attack surface is laughably small compared to an Ubuntu server. See our security hardening guide for more on securing your VPS regardless of OS.
Windows with Your Own License
The math on this one is simple. Vultr charges $16/mo extra for a Windows license. If you have an MSDN subscription, a Volume License, or a spare retail key, uploading your own ISO saves you $192/year per server. Vultr's VNC console handles the Windows installer without any issues — just remember to load VirtIO drivers during the disk detection step or you will stare at an empty "Where do you want to install Windows?" screen. I made that mistake once. It cost me 45 minutes of confusion. For more on Windows VPS options, see our Windows VPS guide.
Custom Hardened Linux
If you work in a regulated industry — HIPAA, PCI-DSS, SOC 2 — your auditor does not care what the provider's default Ubuntu template looks like. You need a golden image: CIS benchmarks applied, unnecessary services stripped, audit logging wired up, approved package repositories locked in, and SSH hardened beyond the defaults. Build it once as a disk image, upload it to your provider, and every new server starts from an identical, auditable baseline. DigitalOcean's custom image system is perfect for this workflow.
Alternative Methods When Custom ISO Is Not Available
Stuck on Contabo or RackNerd and need something other than stock Ubuntu? You are not completely out of luck. These workarounds range from "hacky but effective" to "the professional way to do it":
debootstrap / chroot Method
This is the "I know what I am doing and I accept the consequences" method. You can replace the running OS with a different Debian-based distribution from within the terminal. It works. I have done it. I also bricked a server the first time I tried it because I forgot to configure the bootloader. Have a recovery plan ready:
# Install a minimal Debian from within Ubuntu
apt install debootstrap
debootstrap --arch amd64 bookworm /mnt http://deb.debian.org/debian
# Configure and pivot root...
Advanced technique. Do not attempt this on a production server unless you have a snapshot to roll back to.
Cloud-Init / User Data Scripts
Honestly, most people who think they need a custom ISO actually need cloud-init. It runs on first boot and can install packages, configure services, create users, set up firewall rules, and apply your entire security baseline automatically. It works on almost every provider. It covers about 80% of what people use custom ISOs for. The only things it cannot do: change the OS entirely or install a non-Linux system.
#cloud-config
packages:
- nginx
- fail2ban
- ufw
runcmd:
- ufw allow 22/tcp
- ufw allow 80/tcp
- ufw allow 443/tcp
- ufw --force enable
- systemctl enable fail2ban
Packer for Building Custom Images
This is how infrastructure teams at real companies do it. Packer builds a custom image from a definition file, runs your provisioning scripts against it, and uploads the finished artifact to DigitalOcean or Linode. Your server image becomes version-controlled code that any team member can rebuild identically. It takes an afternoon to set up and saves weeks of "but it worked on my server" debugging:
# packer.pkr.hcl for DigitalOcean
source "digitalocean" "base" {
api_token = var.do_token
image = "ubuntu-24-04-x64"
region = "nyc1"
size = "s-1vcpu-1gb"
ssh_username = "root"
}
build {
sources = ["source.digitalocean.base"]
provisioner "shell" {
script = "setup.sh"
}
}
Ansible/Terraform Post-Provisioning
If your goal is a consistently configured server rather than a specific OS, Ansible is the pragmatic choice. Deploy a standard Ubuntu template, then run an Ansible playbook that transforms it into exactly the configuration you need. This works on every provider regardless of custom ISO support, and the playbook serves as living documentation of your server configuration. Combined with Terraform for provisioning, you get reproducible infrastructure without touching a custom ISO.
Step-by-Step: Upload Custom ISO on Vultr
I have done this enough times that it takes me about 10 minutes end to end. Vultr makes it almost too easy:
Step 1: Host Your ISO File
Vultr fetches the ISO from a URL — it must be publicly accessible over HTTP or HTTPS. Easiest approach: grab the direct download link from the OS vendor's website. For pfSense, that is the AMD64 ISO from pfsense.org. For NixOS, the minimal ISO from nixos.org. If you built a custom image locally, toss it in an S3 bucket or on any web server temporarily.
Step 2: Upload the ISO
In the Vultr dashboard: Products → ISOs → Add ISO. Paste the URL, hit Upload. Takes 2-10 minutes depending on file size. Or skip the GUI entirely and use the API:
curl -X POST "https://api.vultr.com/v2/iso" \
-H "Authorization: Bearer ${VULTR_API_KEY}" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/my-custom.iso"}'
The status will show "Pending" until the download completes. Refresh the page after a few minutes.
Step 3: Deploy a Server from the ISO
Create a new instance and look for the Upload ISO tab in the OS selection screen. Your uploaded ISO appears in the list. Pick your plan (1GB RAM minimum for most OSes, 2GB for Windows), choose a region, deploy. The server boots straight into your ISO's installer or live environment.
Step 4: Complete Installation via Console
Click "View Console" in the Vultr dashboard to open a VNC session. You will see the OS installer exactly as if you had a monitor plugged into a physical machine. Step through the installation, accept Vultr's DHCP for networking (it works out of the box on most OSes), and reboot when the installer finishes.
Step 5: Detach the ISO
Critical step that people forget: detach the ISO after installation. Otherwise your server will boot back into the installer on every reboot. Go to Server Settings → Custom ISO → Remove ISO. The server reboots from its local disk. API version:
curl -X POST "https://api.vultr.com/v2/instances/{instance-id}/iso/detach" \
-H "Authorization: Bearer ${VULTR_API_KEY}"
Tips & Warnings
Network Configuration
This is where most custom ISO installations fail. Your shiny new OS boots, and it has no network connectivity. Vultr and DigitalOcean provide DHCP — most Linux installers pick it up automatically. Hetzner requires static IP configuration, and they give you the details in the server's admin panel. Write down the IP, gateway, and DNS servers *before* you start the installer. Debugging network issues through a VNC console with no clipboard support is miserable.
VirtIO Drivers
Warning: Your custom OS must include VirtIO drivers for disk and network access. Most Linux distributions include them by default. Windows requires loading VirtIO drivers during installation — download the VirtIO driver ISO from Fedora and attach it as a secondary ISO. FreeBSD and OpenBSD include VirtIO support in their kernels.
Disk Partitioning
The disk is /dev/vda on most KVM providers. Use the whole thing — there is no reason to get fancy with partitioning on a VPS. One root partition and an EFI partition if your OS needs it. Skip the swap partition entirely on SSD-based VPS plans. If you need swap later, a swap file is more flexible and does not lock up disk space you cannot reclaim.
Firewall Rules
Your fresh custom OS install has zero firewall rules. Every port is open. The clock starts the moment your server gets an IP address — bots will find it within minutes. Before you do anything else, lock down the firewall. SSH on port 22, your application port, and nothing else. On Linux, ufw or nftables. On FreeBSD, pf. On pfSense, it is the whole point. See our security hardening guide for a proper checklist.
GRUB Configuration
This one has bitten me twice. Some OS installers default to installing GRUB on /dev/sda, but KVM providers use /dev/vda. If GRUB targets the wrong disk, your server will not boot after you detach the ISO. You will just see a blinking cursor. Verify this *before* you reboot:
# Ensure GRUB targets the correct disk
grub-install /dev/vda
update-grub
Serial Console Access
VNC consoles are laggy, do not support clipboard, and occasionally stop responding entirely. Serial console is better in every way for text-based work. Hetzner and Linode both offer it. Add console=ttyS0,115200 to your GRUB kernel parameters and you get a clean, responsive terminal you can paste commands into. Do this during installation, not after your VNC session freezes during a critical configuration step.
Troubleshooting Common Problems
Problem: "No bootable device" After Detaching ISO
The most common issue. The OS installed to the wrong disk or GRUB was not configured correctly. Re-attach the ISO, boot into recovery/rescue mode, mount your installed system, and run grub-install /dev/vda. On Hetzner, use rescue mode instead of the ISO.
Problem: No Network After Installation
Check three things in order: (1) Is the network interface detected? Run ip link — if you do not see an ethernet device besides lo, VirtIO network drivers are missing. (2) Is DHCP running? Most Vultr and DO installs should get an IP automatically. (3) Is static configuration correct? Hetzner requires manual IP setup — check the admin panel for the correct IP, gateway, and netmask.
Problem: Windows Cannot Find Disk During Installation
Windows does not include VirtIO drivers by default. You need to load them during installation: at the "Where do you want to install Windows?" screen, click "Load driver," browse to the VirtIO ISO (mounted as a secondary drive), and select the appropriate driver for your Windows version. After loading the driver, the disk appears normally.
Problem: ISO Upload Stuck at "Pending" on Vultr
Usually means the URL is inaccessible from Vultr's network. Verify the URL works from your browser, check that it is HTTPS or HTTP (not FTP), and ensure there are no authentication requirements or geographic restrictions on the download. Some OS vendor mirrors use Cloudflare protection that blocks automated downloads — try a different mirror.
Problem: Custom Image Rejected by DigitalOcean
DigitalOcean requires specific image formats and the image must be compressed with gzip or bzip2. The disk image must use MBR or GPT partitioning, include a bootloader, and have cloud-init or a similar agent for networking. If your image was built for VMware, convert it: qemu-img convert -f vmdk -O raw input.vmdk output.raw then compress with gzip.
Frequently Asked Questions
Can I install Windows from a custom ISO?
Yes, on providers that support custom ISO (Vultr, Hetzner, BuyVM). You need to supply your own Windows license and VirtIO drivers. Vultr has the smoothest experience — their VNC console lets you step through the Windows installer interactively. Budget about 2GB RAM minimum for Windows Server.
Why do some providers not support custom ISO?
Custom ISO support requires KVM virtualization with console access and ISO mounting capabilities in the hypervisor. Budget providers like Contabo, RackNerd, and Hostinger use simplified provisioning systems that only support pre-built templates. InterServer uses SolusVM which has limited ISO support. It is primarily a cost and complexity decision by the provider.
Is custom ISO the same as custom image?
Not exactly. A custom ISO is a bootable installer (like a CD/DVD image) that you install from. A custom image (supported by DigitalOcean and Linode) is a pre-installed disk image that gets written directly to the VPS disk — no installation step needed. Custom images are faster to deploy but require you to prepare the disk image in advance.
Can I upload an ISO larger than the provider's limit?
Not through the standard upload process. If you need to install an OS whose ISO exceeds the limit, consider using Hetzner (no size limit via rescue mode) or DigitalOcean (100GB limit for custom images). Alternatively, check if the OS offers a minimal/netinstall ISO that fits within the limit — most Linux distributions offer versions under 1GB.
Do I lose the custom ISO if I delete my server?
On Vultr, no — uploaded ISOs are stored in your account independently of servers. You can reuse the same ISO for multiple deployments. On Hetzner, there is no stored ISO (you write directly via rescue mode), so you would need to repeat the process. On DigitalOcean, custom images persist in your account until you delete them.
Can I use custom ISO to run a firewall appliance like pfSense?
Yes, and this is one of the most popular use cases for custom ISO. Upload the pfSense or OPNsense ISO to Vultr, deploy a $6/mo instance with 1GB RAM, and install through the VNC console. The result is a cloud firewall with IDS/IPS, VPN gateway, and traffic shaping that would cost $50+/mo from a managed firewall provider. Vultr's networking (private networks, floating IPs) makes it the best provider for firewall appliances.
What happens if my custom OS does not boot after installation?
The most common causes are wrong GRUB target disk (should be /dev/vda not /dev/sda on KVM), missing VirtIO drivers, or incorrect network configuration. On Vultr, re-attach the ISO and boot into the installer's recovery mode. On Hetzner, use rescue mode to access the disk and fix GRUB. Always verify the bootloader targets the correct disk before detaching the ISO. If network is the issue, check whether the OS needs static IP configuration (Hetzner) or DHCP (Vultr, DigitalOcean).
Can I automate custom ISO deployments with an API?
Yes, on Vultr and DigitalOcean. Vultr's API lets you upload ISOs, create instances from them, and detach ISOs programmatically. DigitalOcean's API supports custom image uploads and droplet creation from custom images. This enables fully automated infrastructure provisioning with custom operating systems. Hetzner's API supports rescue mode but not direct ISO mounting. For large-scale automation, consider building custom images with Packer and deploying them via API rather than using interactive ISO installations.
Ready to Install Your Own OS?
Vultr is where I do all my custom ISO work. Paste a URL, boot from it, install through VNC, done. Their $100 free credit gives you plenty of room to experiment with different OSes before committing to a plan.