jardim do jojo

Firecracker

// Publicado em: 28 de março de 2023

Firecracker is a secure and fast microVM for serverless computing. Enables services like AWS Lambda and AWS Fargate. Written in Rust.

What is Firecracker?

Needs a Linux Kernel, it doesn’t run natively on OSX.

On VirtualBox, you need to enable the nested virtualization. You can configure on the GUI VM Settings > System > Processor > Enabled Nested VT-x/AMD-V or on the CLI:

vboxmanage list vms # get the VM name you want to enable
vboxmanage modifyvm "<vm-name>" --nested-hw-virt on

Check if KVM is enabled:

sudo apt install cpu-checker -y

kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

Give the user read/write access to KVM:

sudo apt install acl -y

sudo setfacl -m u:${USER}:rw /dev/kvm

Installing on Ubuntu:

# adjust to latest version
wget https://github.com/firecracker-microvm/firecracker/releases/download/v1.1.2/firecracker-v1.1.2-x86_64.tgz
tar xfz firecracker-v1.1.2-x86_64.tgz
sudo cp release-v1.1.2-x86_64/firecracker-v1.1.2-x86_64 /usr/local/bin/firecracker
sudo chmod +x /usr/local/bin/firecracker

Kernel and rootfs:

To create a Firecracker VM (or any Linux VM), we need a compiled kernel image vmlinux and roots fo boot on.

Building a kernel image:

# adjust the version if needed
wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.137.tar.xz
tar xf linux-5.10.137.tar.xz
cd linux-5.10.137

# get Firecracker recommended kernel configuration
wget https://raw.githubusercontent.com/firecracker-microvm/firecracker/main/resources/guest_configs/microvm-kernel-x86_64-5.10.config -O .config

# install dependencies to build the kernel
sudo apt install bison build-essential flex libssl-dev libelf-dev -y

# build it
make vmimage

Creating a rootfs (with OpenRC):

dd if=/dev/zero of=rootfs.ext4 bs=1M count=50 # or `touch rootfs.ext4 ; truncate rootfs.ext4 -s 50MB`
mkfs.ext4 rootfs.ext4

# mount
mkdir /tmp/my-rootfs
sudo mount rootfs.ext4 /tmp/my-rootfs

# running Alpine, configuring the system and copying the data to the rootfs
# all inside the Alpine container
docker run -it --rm -v /tmp/my-rootfs:/my-rootfs alpine

passwd -d 'root' root

apk add openrc
apk add util-linux

ln -s agetty /etc/init.d/agetty.ttyS0
echo ttyS0 > /etc/securetty
rc-update add agetty.ttyS0 default

rc-update add devfs boot
rc-update add procfs boot
rc-update add sysfs boot

for d in bin etc lib root sbin usr; do tar c "/$d" | tar x -C /my-rootfs; done

for dir in dev proc run sys var; do mkdir /my-rootfs/${dir}; done

exit

sudo umount /tmp/my-rootfs

Start Firecracker:

firecracker --api-sock /tmp/firecracker.socket

On another shell:

# set the kernel as a boot source
curl --unix-socket /tmp/firecracker.socket -i \
    -X PUT 'http://localhost/boot-source'   \
    -H 'Accept: application/json'           \
    -H 'Content-Type: application/json'     \
    -d "{
        \"kernel_image_path\": \"/home/vagrant/linux-5.10.137/vmlinux\",
        \"boot_args\": \"console=ttyS0 reboot=k panic=1 pci=off\"
    }"

# set the rootfs
curl --unix-socket /tmp/firecracker.socket -i \
  -X PUT 'http://localhost/drives/rootfs' \
  -H 'Accept: application/json'           \
  -H 'Content-Type: application/json'     \
  -d "{
        \"drive_id\": \"rootfs\",
        \"path_on_host\": \"/home/vagrant/linux-5.10.137/rootfs.ext4\",
        \"is_root_device\": true,
        \"is_read_only\": false
   }"

# run this command and the microVM will start where
# you started the Firecracker API
curl --unix-socket /tmp/firecracker.socket -i \
  -X PUT 'http://localhost/actions'       \
  -H  'Accept: application/json'          \
  -H  'Content-Type: application/json'    \
  -d '{
      "action_type": "InstanceStart"
   }'

To run with a configuration file, create a file named config.json with the content below:

{
  "boot-source": {
    "kernel_image_path": "/home/vagrant/linux-5.10.137/vmlinux",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off",
    "initrd_path": null
  },
  "drives": [
    {
      "drive_id": "rootfs",
      "path_on_host": "/home/vagrant/linux-5.10.137/rootfs.ext4",
      "is_root_device": true,
      "is_read_only": false
    }
  ],
  "machine-config": {
    "vcpu_count": 1,
    "mem_size_mib": 128
  }
}

As run the binary with the option below. The VM will start as soon as you run the command:

firecracker --api-sock /tmp/firecracker.socket --config-file $HOME/config.json

Now, if you instantiate again with a different socket, you have more than one VM running.

Adding network connection. Basically we forward the traffic between the host and guest using network interfaces and iptables rules.

First, prepare the host:

sudo ip tuntap add tap0 mode tap

# make sure the address is different than the host's network
# enp0s3 is my hosts network to the internet
sudo ip addr add 172.16.0.1/24 dev tap0
sudo ip link set tap0 up
sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
sudo iptables -t nat -A POSTROUTING -o enp0s3 -j MASQUERADE
sudo iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i tap0 -o enp0s3 -j ACCEPT

Start Firecracker with the configuration below:

{
  "boot-source": {
    "kernel_image_path": "/home/vagrant/linux-5.10.137/vmlinux",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off",
    "initrd_path": null
  },
  "drives": [
    {
      "drive_id": "rootfs",
      "path_on_host": "/home/vagrant/linux-5.10.137/rootfs.ext4",
      "is_root_device": true,
      "is_read_only": false
    }
  ],
  "machine-config": {
    "vcpu_count": 1,
    "mem_size_mib": 128
  },
  "network-interfaces": [
    {
      "iface_id": "eth0",
      "guest_mac": "AA:FC:00:00:00:01",
      "host_dev_name": "tap0"
    }
  ]

}

Once booted, finish the process on the guest:

ip addr add 172.16.0.2/24 dev eth0
ip link set eth0 up
ip route add default via 172.16.0.1 dev eth0

If you want to run multiple guests, you’ll need to create multiple tap interfaces and configure then accordingly.

To log to a file, add the following to the configuration file:

  "logger": {
    "log_path": "logs.file",
    "level": "Debug",
    "show_level": false,
    "show_log_origin": false
  }

To generate metrics, add the following to the configuration file:

  "metrics": {
    "metrics_path": "metrics.file"
  }

Questions

What is the ‘jailer’ process?

Resources