How I set up Poudriere

For those following along at home, I have a server which I use to learn how to do silly computer things, like this. It runs FreeBSD-14.1 and I wanted to get my website on it, rather than hosting it locally on my laptop. Since I used Caddy, I need to use Frankenphp as the PHP-part of the server. From getting that to work on my laptop, I knew that meant building Frankenphp/Caddy from source, with a correctly configured PHP 8.4 binary hanging around. Right. It was a right pain on my laptop, and it was even worse on my server.

The point of this post is to document how I did this so I could do it again, and maybe seeing the mistakes I made/steps I took can help someone else. Good luck!

I don't make things easy for myself.

My first problem was that I don't like to do things the easy or simple way, I enjoy making things needlessly complex. Additionally, I needed to build caddy from source anyway, so I may as well build everything else from source as well. Hahahahahaaaa! Somewhere along building all the doc tools you get for free when you build git(?) from soure, you build Python 3.11, and when you build PHP from source, you build a slightly different version of Python 3.11 and they conflict? and the whole thing goes kaplut. Well this happened a couple times and I got pretty annoyed.

I figured my options were as follows:

  1. Modify the Makefiles for PHP or whatever dependence of git is causing the issue and unify/change the Python versions.
  2. Use portmaster to magically fix everything?
  3. Use Poudriere to magically fix everything?

Well, number 1 is too hard for me, number 2 was easiest so I tried that but it didn't solve the problem. I figured I would try number 3. Three hours later I have Poudriere set up and installed, and I am able to install pkgs from my local Poudriere jail to my users (ie, install it with `pkg` but it's built by me at home)... okay. That was complicated.

First you have to build the Poudriere port, which is easy enough. Then you create a BSD Jail with the `poudriere jail` command, and make sure you put it at an OS version that's behind that of the running system! Then, you put a copy of the Port tree in that jail, ah... you need git for this (at least for a new version, I guess I could've copied the base OS's port tree, but I didn't think of that at the time). Welp. Time to build git from source! (Halfway through agreeing to all the default configurations for documentation generation tools, I wondered why I didn't just install the binary with `pkg` but there we go...). Once that is done, you can write a list of the Ports you want Poudriere to build, configure them all with the `poudriere config` command and build them all with the `poudriere build` command, passing them all the correct jail name, correct port tree name and correct package list.

Oh, you have to configure Poudriere to use your ZFS system correctly, and also tell it where to download FreeBSDs from.

A short-ish while later, you have all green and Poudriere has built all the packages. Now you need to install them in the base OS from the jail. You do this using the `pkg` command. Ohhh, okay, Poudriere builds packages which you then have to install. So only sort of like ports. Wow, everyone wants to avoid using ports for actually putting binaries on disks. Okay, to do this, you need to add the output file of the jail, which you will identify by seeing its full of `.pkg` files, as a repository for pkg.

I did this by following the instructions linked below, except I used a local file rather than a remote server (saving me from faffing around with authentication), and the man page for `pkg_conf`, which was surprisingly detailed and helpful :)). Then I had to configure the package directory as a package repository that `pkg` could use to install things. The config files I used are here:

In `/usr/local/etc/pkg/repos/FreeBSD.conf`:

FreeBSD: { enabled: no }

which disables the default repository. Now `pkg` doesn't work!

In `poudriere.conf` in the same directory:

            
Poudriere: {
    enabled: yes,
    url: "file:///path/to/output/directory/of/poudriere"
    mirror_type: "none",
    signature_type: "none"
}

which is what sets up the poudriere output as a package repository (which is just a directory full of `.pkg` files). Finally, you run `pkg update -f` and it updates everything, including its repositories, and now it knows about your new, freshly built packages! Gosh. That was a lot of work. At least I have `curl` and `vim` and `go` and so on now.

Here are some resources I found helpful:

Wait, how do I install a Package if you removed the default repository?

Hahahahahaha &emdash; you don't! Go outside instead. Well, you can still:

  1. Go to the Poudriere config directory in `/usr/local/etc/poudriere.d` and add the port names you want to the `14amd64-[hostname]-pkglist` file you left there
  2. Run `poudriere options` with the relevant `-j` name and Port tree and so on to do all the configuration (this runs `make config` for each port in your list, and its dependencies)
  3. Run `poudriere bulk` to actually build all the packages. Need the `-z workstation` argument to make it match up with the set you configured this with. This will take a while, especially if they've updated.
  4. Run `pkg update -f` to refresh the repo list or whatever. Now you can install your new package!

Is that it?

For now! Now I just have to install PHP and get xcaddy to do it properly!

Additional dependencies needed?

Okay, FrankenPHP also wants Brotli, which I couldn't get to build earlier... and also wants a library called watcher-c for seeing when files change, which is useful functionality I'd like to keep. Issue is there's no BSD compatible version of that :/ yeah it uses linux kernel stuff. Welp, I guess I have to figure out how to pass the relevant argument to xcaddy to stop it complaining. If I "got good" at C++/C, making a compatible version would be a pretty fun project, I think.

Ah, and PHP wanted sqlite3. And then it complained there was no iconv giving error numbers, when there is. Y'know, once I've got this working I'm going to submit it as a pull request to the FrankenPHP people so they can have some instructions on how to do this. Okay, the PHP `.configure` script can't find the FreeBSD `iconv` program unless you also pass the argument `--with-iconv=/usr/bin` and then it will work. But then... it just built! No problems! And passes all the tests? Except one... ah well.

Then it was time to build frankenphp. I navigated to `/usr/local/bin` (where PHP and xcaddy live) and tried the supplied command. It fetched everything and then crashed when trying to include a file. Googling indicates this is because CGO is not enabled... but it is. Ah, the issue is because the FrankenPHP instructions use `php-config` to populate some environment variables, but this wasn't on my path, nor is it executable. So I rectified that and added it to the path by copying it into `/usr/local/bin/`. Okay it gave other errors, the solution was to add the php/main directory to the path for the build command.

Much screaming later

Okay, turns out that despite hours and hours of my trying, FrankenPHP will not build on my FreeBSD machine. I end up with weird Go/Cgo dependency problems that I'm certain shouldn't happen, but do, and I can't troubleshoot it. Instead, I installed php-fpm, and just used the Caddy fast_gci directive to proxy .php pages through it. That worked great and only took 30 minutes to set up, haha. Only gotcha is you can't use hyphens in rc.d scripts or service names, so `service php-fpm XXXX` doesn't work. You have to use `service php_fpm XXXX`.