FreeBSD Ports Annoyance

Ivan Voras’ article on FreeBSD’s pkgng prompted me to try pkgng. pkgng works exactly as advertised, with a couple of minor annoyances. But this brought to head a problem with FreeBSD that I’ve had for a while. I’ve talked to various ports guys about it over the years. It’s an engineering problem that’s begging for someone to solve.

Before folks in other Unix-like operating systems start snickering at “Lucas turning like a rabid dog on the community he came from”: you guys have your own problems. Go deal with them.

First, some context:

When the Ports Tree started it was pretty durn good. It might have started life as a temporary workaround, but it came out all right. Ports was the envy of many operating systems, and it got dragged into other BSDs and copied elsewhere. And in many ways, it’s actually aged pretty well. I run all three “Big BSDs” and a DragonFly box. I administer CentOS and Ubuntu machines. I have a couple of OpenSolaris servers slated to be replaced with FreeNAS. I feel qualified to offer an opinion on how things compare. FreeBSD trumpets its performance as a network operating system, as well it should. It passes packets like “whoa.” It makes lots of complicated stuff easy.

But some simple tasks are harder than they should be.

I’m standing up a new server to replace my old Web server. The old server originally ran 6-current, and has been updated to -current every few months. Now and then, Apache hangs up. I’m fairly confident that some lingering cruft from a previous aeon is responsible, and while I could exhaustively search the old server for the problem, I’d rather send it off to a well-deserved retirement and put some new hardware in play.

So, I have a virgin server. I want sendmail, mutt, and a WordPress server. (Your choices might differ, and that’s fine.) I don’t care which Web server WordPress runs on, anything vaguely modern would do. I’d like to install all software from packages, as I don’t have the time or energy to build an artisan server. Ideally, I won’t even need to install the Ports Collection. This is an extremely common set of software.

I can do this easily on OpenBSD, because OpenBSD has One True Web Server. It’s Apache 1.3, yes, which makes my bowels churn a bit, but the OpenBSD Project stands behind it, so: fine. I can do this with CentOS or Ubuntu or OpenSolaris — I have to install new repositories, which brings up all kinds of trust issues, but when you run those operating systems you decide to evaluate those risks on an ongoing basis. Fine. (I must confess, I’ve never installed a Web server on my NetBSD palmtop. That’s too geeky even for me.)

But I just can’t do this with FreeBSD.

The Ports System is massively flexible. If I’m willing to build from source, I can configure my system in dang near any way I like. That’s an awesome feature. It really is. The Apache 2.2 port has about eighty configuration knobs. I can tweak them all. I can build Apache exactly as I want and be highly confident that it will Just Work.

This is amazing.

But I can’t install a WordPress server with precompiled packages.

The problem is in the PHP port. PHP doesn’t know which Web server you’re running. The Ports System gives you the flexibility to build PHP with or without the Apache module. You can build with or without every dang module in the world. The PHP port doesn’t tie your hands.

But without these — comparatively tiny — bits of precompiled glue, I can’t make FreeBSD Just Work.

And it’s not just WordPress. I run an LDAP server at work. Apache 2.2 can be compiled to include the OpenLDAP module. But I cannot install this common corporate configuration from packages, I must compile the software myself. If I want OpenLDAP and Postgres, well, that’s a different build. And if an application absolutely requires MySQL, it’s back to the Ports Tree.

Multiply this by the thousands of combinations of options in over 23,000 ports, and… wow. Building a package for every possible combination would take a long time.

I can — and do — build my own package repositories. But this is an ongoing annoyance. And if I need an Apache 2.2 server that supports OpenLDAP, MySQL, and Postgres, I’m back to building it myself.

The Ports System makes the difficult manageable. I’d much rather maintain my own FreeBSD package repo than a CentOS or Ubuntu/Debian repo. But I really shouldn’t have to.

I’m not saying “copy Linux.” By all that’s sacred, no. NO. But they’ve solved this problem without building umpteen bajillion packages. We have some really brilliant people in the community. Surely there’s a realistic solution to this?

I pimp BAMP (BSD, Apache, Mysql, PHP) as an alternative to LAMP. This particular example of this issue stops those efforts cold.

Just in case you’re one of the brilliant people in our community, and this motivates you to do something, as a sysadmin who wants to run a lot more FreeBSD, here’s what I’d like to see:

  • Don’t rely on external package repositories. Every time I need to add a new repository to make some Linux package work, my blood turns to icewater and the voices in the back of my head start gibbering again. FreeBSD has a centralized model. I trust FreeBSD’s central package builds; they’re highly consistent, and if they get compromised, I will hear about it.
  • I’d like to be able to set variables in, say, /etc/make.conf or /usr/local/etc/pkg.conf that say what combination of software I use. Say, WEBSERVER=apache22 and DATABASE=postgres91. There are ports that will configure the ports-building process for me, such as ports-mgmt/portconf; I’d like something similar for packages.
  • When I install a new server, I want to be able to copy the config file in, run pkg add wordpress, and have the package system say “Aha! The Boss wants to run this webserver, and this database, but the software package doesn’t support his chosen database, so I’ll install MySQL anyway.” It should then just Do The Right Thing.
  • Why don’t I fix this? I’m a sysadmin, not a programmer. Hogs, when they look at my code, vomit. My spare time is spent writing, most often in support of the various BSD communities. I could learn to fix it, but I believe that others are better qualified to do it first. But those most likely to be able to fix it are also those most likely to just build the software themselves.

    We run operating systems to use applications. (If that wasn’t true, I would have truly mastered my Sinclair ZX80 by now.) The new user, confronted with a requirement to install the Ports Tree to build a simple PHP webserver, might well say “ick” and install something else instead. It’s a serious barrier to entry.

    So, if you’re looking for a new problem to solve, here it is.

    7 comments to FreeBSD Ports Annoyance

    • Fredrik Ludl

      This is right.
      Computers are used to automate processes.

    • Erwin Lansing

      You’re absolutely right. For years, the ports system has had precompiled packages available, but the whole mindset around the ports system was based on the assumption that users compile things from source. It’s enormously flexible if you do so, but completely inflexible of you want to use binary packages. As you mention, we talked about this at many conference over the past few years and the first step is to change that mindset, then start on the engineering. PKGNG is laying the groundwork to move into that direction and make precompiled packages all the more useful. Decoupling package releases from src releases will be the next step, then we can start working on this and a lot of other intricacies of the current system. Stay tuned…

    • Erwin, I’m really glad to hear that. Stop listening to me and get on with your work.

    • [...] FreeBSD pkgng and suggestion for improvement Posted on July 31, 2012 by Gerard Ivan Voras’ wrote a post with his thoughts on FreeBSD’s pkgng. Michael W. Luchas thinks there’s still room for improvement: FreeBSD Ports Annoyances. [...]

    • [...] also recently talked about a problem with port installation on FreeBSD.  What he says there I think applies to pkgsrc as [...]

    • Lars Engels

      Thanks for the article, Michael.

      IHMO port maintainer should enable more OPTIONS by default, so the packages that are built support more features the users expect from the software they install.
      As Erwin states pkgng will hopefully solve those issues.

      In the meantime you can try PC-BSD (>= 9.1), which offers you to install “TrueOS”, their name for FreeBSD servers with PBI support on the CLI.
      See http://wiki.pcbsd.org/index.php/Warden%C2%AE#Packages_Tab for a list of packages (the screenshot is from PC-BSD’s jail management tool, but you can also install the PBIs without a jail and GUI). There’s also Apache and WordPress in it.

    • Tim

      I agree with most of what you are talking about, especially that it would be nice to have packages of the most common configurations for various popular ports. However, the main problem for me is that they don’t build packages often enough. Building packages every 2 weeks is okay, (the -stable repositories, which as far as I can tell are the most frequently updated) but not great. It really should be twice a week, if possible. While I am ok with building ports for the most part, I can imagine that it would seem tedious to a Windows or Debian user.

      As far as I can tell, for many ports, the project as a whole is still holding off on packages since the package server incident last year, which isn’t practical at all for me.

      Then, there are ports which, through no fault the of FreeBSD project, have constant security vulnerabilities, and need to be updated (firefox, chromium, etc). The problem is, that almost every time, I have to compile them from source, (I don’t want to wait something like 2 weeks for a urgent security fixed package to be built) which takes hours on my computer. It is especially annoying at work.

      Plus, they are too obsessed about quality control sometimes – they still haven’t included some ports like TeXlive, which even NetBSD has in pkgsrc. So basically I have to compile from scratch ports for TeXlive, which are kindly provided by Romain Tartière. There are dozens of ports where I have to make custom versions to avoid the annoying teTeX dependency. I’m sure others have similar situations that are annoying to deal with about ports. I am seriously considering using pkgsrc again in addition to ports to deal with this problem, as well as future similar issues.

      That said, I still like the ports system a lot, in part because it works very smoothly with portupgrade, but as a user and port contributor, I hope that packages are treated as a top priority.