although it is most likely best to run most things directly on an OpenBSD host, compatibility issues and other pains may arise depending on the service.

this is a guide on how to host web content from a virtual machine running inside OpenBSD's vmm and serve it through a reverse proxy with relayd.

in the setup this guide shows, relayd will handle http and https requests to the OpenBSD machine and separate them based on domain (yonderly.org, example.com, …), then send them to either the host or the virtual machine. httpd will also listen on the domain for the virtual machine in order to force https; the virtual machine's http server will only be accessible through https.

vmd is the daemon that interacts with vmm to make virtual machine creation and management easy. its config file is at /etc/vm.conf.

what I like to do is create /etc/vm.conf.d and use “include” in vm.conf to load the configs within it. this allows you to create individual config files and enable or disable them by simply commenting their include line.

for example:

/etc/vm.conf

include "/etc/vm.conf.d/alpine0.conf"
#include "/etc/vm.conf.d/alpine1.conf"
include "/etc/vm.conf.d/openbsd0.conf"
...

here is the vm.conf used for this guide:

/etc/vm.conf

include "/etc/vm.conf.d/alpine0.conf"

also, I create /var/vm, /var/vm/disk, and /var/vm/iso for the disk and installation images for the VMs.

# mkdir -p /var/vm/disk /var/vm/iso

now, set secure permissions for the disk directory.

# chmod -R 600 /var/vm/disk

for convenience, I create the vm group and allow its members to control virtual machines (in next section). this allows you to start, stop, monitor, and stuff without needing to be root.

# groupadd vm
# usermod -G vm user

pf.conf needs to be modified so that networking works properly for the VMs.

/etc/pf.conf

...
match out on egress from 100.64.0.0/10 to any nat-to (egress)
pass in proto { udp tcp } from 100.64.0.0/10 to any port domain \
	rdr-to 1.1.1.1 port domain
...

for this guide, Alpine Linux will be used. download the installation ISO.

# cd /var/vm/iso
# ftp https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/x86_64/alpine-extended-3.21.3-x86_64.iso

create the disk image. a size of 64G will be used here.

# cd /var/vm/disk
# vmctl create -s 64G ./alpine0.qcow2
vmctl: qcow2 imagefile created

make sure that the disk image has the correct permissions!

# ls -l ./alpine0.qcow2
-rw-------  1 root  _vmd    256K Feb 1 12:00 alpine0.qcow2

here is the vm block for the VM. read vm.conf(5) for more information.

/etc/vm.conf.d/alpine0.conf

vm alpine0 {
        owner   :vm # allow group vm to control this virtual machine.
 
        boot device cdrom
 
        cdrom   /var/vm/iso/alpine-extended-3.21.3-x86_64.iso
        disk    /var/vm/disk/alpine0.qcow2
 
        memory  8192M
 
        local interface
}

if you haven't already, start vmd.

# rcctl start vmd

if you have, run the following to have vmd reload its config.

# vmctl reload

the virtual machine will automatically start, use the following command to connect to it.

# vmctl console alpine0

if you see nothing, wait, and then press Enter. the login prompt should be visible.

see Alpine Linux's installation guide for more information.

• the hostname doesn't matter.
• initialize interface “eth0”. the IP 100.64.1.3 will be used for this guide.
  • you can either use dhcp or figure out a valid IP by looking at the tun device being used by the VM.
• you should still enter secure root and user passwords, as a vulnerability in your web stack could still be present (we will only forward port 80; no other ports will be accessible publicly).
• probably choose openssh as the ssh server. I have little experience with dropbear. you should at a minimum enable an sshd server as it will make interacting with the VM much more convenient, no longer needing to use slow cu(1) after installation.
• you can do a crypt install, but it will mean you'll need too cu into the VM and enter the disk password each time it starts up.

this is optional, but it is very beneficial in debugging a VM which isn't booting. by default, Alpine Linux's output will not be visible in cu until it has reached the login prompt. you can change this by appending “console=…” to the kernel options.

the installer will have unmounted the installation, so it must be remounted.

# mount /dev/vda3 /mnt
# mount /dev/vda1 /mnt/boot
# mount /proc /mnt/proc
# mount /dev /mnt/dev

chroot into the install.

# chroot /mnt

now, edit /etc/update-extlinux.conf and add “console=ttyS0,115200” to 'default_kernel_opts'.

/etc/update-extlinux.conf

...
# default_kernel_opts
# default kernel options
default_kernel_opts="... quiet console=ttyS0,115200"
...

now, run update-extlinux.

# update-extlinux

if it mentions the configuration was unchanged, you did not edit its config properly.

exit the chroot and unmount the installation.

# exit
(no longer in chroot)
# cd /
# umount /mnt/dev
# umount /mnt/proc
# umount /mnt/boot
# umount /mnt

after running the commands, press Enter, then ~, then Ctrl+D to exit cu.

power off the virtual machine.

# vmctl stop alpine0

change the VM's configuration so it boots from disk.

/etc/vm.conf.d/alpine0.conf

vm alpine0 {
        ...
        boot device disk
        ...
}

have vmd reload the configuration.

# vmctl reload

start the virtual machine, and, using the -c flag, immediately connect to it.

# vmctl start -c alpine0

look through the output and make sure everything looks correct, then exit cu using the keys earlier.

if you enabled sshd, you can now login to the VM. it should have told you the IP earlier. if it is your first VM, the IP should be 100.64.1.3. your second would be 100.64.2.3, and so on. 100.64.1.3 will be used in this guide.

$ ssh 100.64.1.2

you cannot login as root.

become root and install nginx (from Alpine Linux Wiki).

# apk add nginx
# mkdir /www
# chown -R www:www /var/lib/nginx
# chown -R www:www /www

edit the default server.

/etc/nginx/http.d/default.conf

server {
        listen 80 default_server;
        listen [::]:80 default_server;
 
        location / {
                index index.html;
                root /www;
        }
}

create /www/index.html.

/www/index.html

<html>hi</html>

start nginx.

# rc-service nginx start

it should be possible to fetch the html page.

# wget localhost
Connecting to localhost ([::1]:80)
saving to 'index.html'
# cat index.html 
<html>hello</html>

on the host, make sure you can access the VM's http server.

# ftp http://100.64.1.3/index.html
Requesting http://100.64.1.3/index.html

now that an http server is running inside the virtual machine, we can make it publicly accessible through relayd.

this section will use 10.0.0.4 as the IP of the server running OpenBSD, and alpine0.yonderly.org as the domain meant for the VM.

/etc/relayd.conf

ext_ip="10.0.0.4"
 
table <self> { "127.0.0.1" }
table <alpine0> { "100.64.1.3" }
 
http protocol wwwtls {
        tls keypair "yonderly.org"
        #tls keypair "alpine0.yonderly.org"
 
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" \
            value "$SERVER_ADDR:$SERVER_PORT"
        match request header set "Connection" value "close"
 
        block
 
        pass request header "Host" value "yonderly.org" \
                forward to <self>
        ...
        pass request header "Host" value "alpine0.yonderly.org" \
                forward to <alpine0>
}
 
http protocol www {
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" \
            value "$SERVER_ADDR:$SERVER_PORT"
        match request header set "Connection" value "close"
 
        block
 
        pass request header "Host" value "yonderly.org" \
                forward to <self>
        ...
        pass request header "Host" value "alpine0.yonderly.org" \
                forward to <self>
}
 
relay www {
        listen on $ext_ip port http
        protocol www
 
        forward to <self>       port 80
}
 
relay wwwtls {
        listen on $ext_ip port https tls
        protocol wwwtls
 
        forward to <self> port 8080
        forward to <alpine0> port 80
}

make sure that the config is sane and start or restart it.

# relayd -n
# rcctl restart relayd

httpd will serve the acme-challenge directory for TLS certificate generation and redirect http to https.

/etc/httpd.conf

...
server "alpine0.yonderly.org" {
        listen on * port 80
 
        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
 
        location * {
                block return 301 "https://$HTTP_HOST$REQUEST_URI"
        }
}

make sure that the config is sane and start or restart it.

# httpd -n
# rcctl restart httpd

/etc/acme-client.conf

...
domain alpine0.yonderly.org {
        domain key "/etc/ssl/private/alpine0.yonderly.org.key"
        domain full chain certificate "/etc/ssl/alpine0.yonderly.org.crt"
        sign with letsencrypt
}

generate the certificate.

# acme-client -v alpine0.yonderly.org

now that the certificates exist, the “tls” line for alpine0.yonderly.org can be uncommented in relayd.conf.

/etc/relayd.conf

...
        tls keypair "alpine0.yonderly.org"
...

# rcctl restart relayd

the virtual machine's nginx server should now be accessible at https://alpine0.yonderly.org.