wiki.yonderly.org

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
**notice**: I have been having issues with vmm/Alpine Linux, where, after a VM shutdown, Alpine Linux will become somehow be corrupted and I must boot into the installation iso to repair the system. if this happens to you, all I had to do was update Alpine.

======running webservices from OpenBSD's vmm======
although it is most likely best to run most things directly on an OpenBSD host, compatibility issues and other pains may arise depending on the service.

this is a guide on how to host web content from a virtual machine running inside [[https://openbsd.org|OpenBSD]]'s [[vmm]] and serve it through a reverse proxy with [[https://man.openbsd.org/relayd.8|relayd]].

in the setup this guide shows, relayd will handle http and https requests to the OpenBSD machine and separate them based on domain (yonderly.org, example.com, ...), then send them to either the host or the virtual machine. httpd will also listen on the domain for the virtual machine in order to force https; the virtual machine's http server will only be accessible through https.
=====vmd=====
vmd is the daemon that interacts with vmm to make virtual machine creation and management easy. its config file is at ///etc/vm.conf//.

what I like to do is create ///etc/vm.conf.d// and use "include" in //vm.conf// to load the configs within it. this allows you to create individual config files and enable or disable them by simply commenting their include line.

for example:
<file bash /etc/vm.conf>
include "/etc/vm.conf.d/alpine0.conf"
#include "/etc/vm.conf.d/alpine1.conf"
include "/etc/vm.conf.d/openbsd0.conf"
...
</file>

here is the //vm.conf// used for this guide:
<file bash /etc/vm.conf>
include "/etc/vm.conf.d/alpine0.conf"
</file>

also, I create ///var/vm//, ///var/vm/disk//, and ///var/vm/iso// for the disk and installation images for the VMs.
<code>
# mkdir -p /var/vm/disk /var/vm/iso
</code>

now, set secure permissions for the disk directory.
<code>
# chmod -R 600 /var/vm/disk
</code>

for convenience, I create the vm group and allow its members to control virtual machines (in next section). this allows you to start, stop, monitor, and stuff without needing to be root.
<code>
# groupadd vm
# usermod -G vm user
</code>
=====pf.conf=====
pf.conf needs to be modified so that networking works properly for the VMs.
<file bash /etc/pf.conf>
...
match out on egress from 100.64.0.0/10 to any nat-to (egress)
pass in proto { udp tcp } from 100.64.0.0/10 to any port domain \
	rdr-to 1.1.1.1 port domain
...
</file>
=====the virtual machine=====
for this guide, [[https://alpinelinux.org|Alpine Linux]] will be used. download the installation ISO.
<code>
# cd /var/vm/iso
# ftp https://dl-cdn.alpinelinux.org/alpine/v3.21/releases/x86_64/alpine-extended-3.21.3-x86_64.iso
</code>

create the disk image. a size of 64G will be used here.
<code>
# cd /var/vm/disk
# vmctl create -s 64G ./alpine0.qcow2
vmctl: qcow2 imagefile created
</code>

**make sure that the disk image has the correct permissions!**
<code>
# ls -l ./alpine0.qcow2
-rw-------  1 root  _vmd    256K Feb 1 12:00 alpine0.qcow2
</code>

here is the //vm// block for the VM. read [[https://man.openbsd.org/vm.conf.5|vm.conf(5)]] for more information.

<file bash /etc/vm.conf.d/alpine0.conf>
vm alpine0 {
        owner   :vm # allow group vm to control this virtual machine.
 
        boot device cdrom
 
        cdrom   /var/vm/iso/alpine-extended-3.21.3-x86_64.iso
        disk    /var/vm/disk/alpine0.qcow2
 
        memory  8192M
 
        local interface
}
</file>

if you haven't already, start vmd.
<code>
# rcctl start vmd
</code>
if you have, run the following to have vmd reload its config.
<code>
# vmctl reload
</code>

the virtual machine will automatically start, use the following command to connect to it.
<code>
# vmctl console alpine0
</code>
if you see nothing, wait, and then press Enter. the login prompt should be visible.
====Alpine installation====
see [[https://wiki.alpinelinux.org/wiki/Installation|Alpine Linux's installation guide]] for more information.

  * the hostname doesn't matter.
  * initialize interface "eth0". the IP 100.64.1.3 will be used for this guide.
    * you can either use dhcp or figure out a valid IP by looking at the tun device being used by the VM.
  * you should still enter secure root and user passwords, as a vulnerability in your web stack could still be present (we will only forward port 80; no other ports will be accessible publicly).
  * probably choose openssh as the ssh server. I have little experience with dropbear. you should at a minimum enable an sshd server as it will make interacting with the VM much more convenient, no longer needing to use slow [[https://man.openbsd.org/cu|cu(1)]] after installation.
  * you can do a crypt install, but it will mean you'll need too cu into the VM and enter the disk password each time it starts up.

===improving console===

this is optional, but it is very beneficial in debugging a VM which isn't booting. by default, Alpine Linux's output will not be visible in cu until it has reached the login prompt. you can change this by appending "console=..." to the kernel options.

the installer will have unmounted the installation, so it must be remounted.
<code>
# mount /dev/vda3 /mnt
# mount /dev/vda1 /mnt/boot
# mount /proc /mnt/proc
# mount /dev /mnt/dev
</code>
chroot into the install.
<code>
# chroot /mnt
</code>
now, edit /etc/update-extlinux.conf and add "console=ttyS0,115200" to 'default_kernel_opts'.
<file bash /etc/update-extlinux.conf>
...
# default_kernel_opts
# default kernel options
default_kernel_opts="... quiet console=ttyS0,115200"
...
</file>
now, run update-extlinux.
<code>
# update-extlinux
</code>
if it mentions the configuration was unchanged, you did not edit its config properly.

exit the chroot and unmount the installation.
<code>
# exit
(no longer in chroot)
# cd /
# umount /mnt/dev
# umount /mnt/proc
# umount /mnt/boot
# umount /mnt
</code>
after running the commands, press Enter, then ~, then Ctrl+D to exit cu.

power off the virtual machine.
<code>
# vmctl stop alpine0
</code>

change the VM's configuration so it boots from disk.
<file bash /etc/vm.conf.d/alpine0.conf>
vm alpine0 {
        ...
        boot device disk
        ...
}
</file>

have vmd reload the configuration.
<code>
# vmctl reload
</code>
====setting up Alpine====
start the virtual machine, and, using the -c flag, immediately connect to it.
<code>
# vmctl start -c alpine0
</code>
look through the output and make sure everything looks correct, then exit cu using the keys earlier.

if you enabled sshd, you can now login to the VM. it should have told you the IP earlier. if it is your first VM, the IP should be 100.64.1.3. your second would be 100.64.2.3, and so on. 100.64.1.3 will be used in this guide.
<code>
$ ssh 100.64.1.2
</code>
> you cannot login as root.
===simple nginx setup===
become root and install nginx (from [[https://wiki.alpinelinux.org/wiki/Nginx|Alpine Linux Wiki]]).
<code>
# apk add nginx
# mkdir /www
# chown -R www:www /var/lib/nginx
# chown -R www:www /www
</code>

edit the default server.
<file bash /etc/nginx/http.d/default.conf>
server {
        listen 80 default_server;
        listen [::]:80 default_server;

        location / {
                index index.html;
                root /www;
        }
}
</file>

create /www/index.html.
<file html /www/index.html>
<html>hi</html>
</file>

start nginx.
<code>
# rc-service nginx start
</code>

it should be possible to fetch the html page.
<code>
# wget localhost
Connecting to localhost ([::1]:80)
saving to 'index.html'
# cat index.html 
<html>hello</html>
</code>

on the host, make sure you can access the VM's http server.
<code>
# ftp http://100.64.1.3/index.html
Requesting http://100.64.1.3/index.html
</code>
=====relayd=====
now that an http server is running inside the virtual machine, we can make it publicly accessible through relayd.

this section will use 10.0.0.4 as the IP of the server running OpenBSD, and alpine0.yonderly.org as the domain meant for the VM.
<file bash /etc/relayd.conf>
ext_ip="10.0.0.4"

table <self> { "127.0.0.1" }
table <alpine0> { "100.64.1.3" }

http protocol wwwtls {
        tls keypair "yonderly.org"
        #tls keypair "alpine0.yonderly.org"
        
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" \
            value "$SERVER_ADDR:$SERVER_PORT"
        match request header set "Connection" value "close"
        
        block
        
        pass request header "Host" value "yonderly.org" \
                forward to <self>
        ...
        pass request header "Host" value "alpine0.yonderly.org" \
                forward to <alpine0>
}

http protocol www {
        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" \
            value "$SERVER_ADDR:$SERVER_PORT"
        match request header set "Connection" value "close"
        
        block
        
        pass request header "Host" value "yonderly.org" \
                forward to <self>
        ...
        pass request header "Host" value "alpine0.yonderly.org" \
                forward to <self>
}

relay www {
        listen on $ext_ip port http
        protocol www

        forward to <self>       port 80
}

relay wwwtls {
        listen on $ext_ip port https tls
        protocol wwwtls

        forward to <self> port 8080
        forward to <alpine0> port 80
}
</file>

make sure that the config is sane and start or restart it.
<code>
# relayd -n
# rcctl restart relayd
</code>

=====httpd=====
httpd will serve the acme-challenge directory for TLS certificate generation and redirect http to https.
<file bash /etc/httpd.conf>
...
server "alpine0.yonderly.org" {
        listen on * port 80

        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }

        location * {
                block return 301 "https://$HTTP_HOST$REQUEST_URI"
        }
}
</file>

make sure that the config is sane and start or restart it.
<code>
# httpd -n
# rcctl restart httpd
</code>

=====acme-client=====
<file bash /etc/acme-client.conf>
...
domain alpine0.yonderly.org {
        domain key "/etc/ssl/private/alpine0.yonderly.org.key"
        domain full chain certificate "/etc/ssl/alpine0.yonderly.org.crt"
        sign with letsencrypt
}
</file>

generate the certificate.
<code>
# acme-client -v alpine0.yonderly.org
</code>
=====finalizing=====
now that the certificates exist, the "tls" line for alpine0.yonderly.org can be uncommented in relayd.conf.
<file bash /etc/relayd.conf>
...
        tls keypair "alpine0.yonderly.org"
...
</file>
<code>
# rcctl restart relayd
</code>
the virtual machine's nginx server should now be accessible at https://alpine0.yonderly.org.
{{tag>openbsd www vmd relayd httpd}}