NFSv4 and UIDs on OpenSolaris and Ubuntu

NFS clients and servers negotiate to use the highest NFS version they both support. NFSv4 usually performs much better than NFSv3, but requires a little more setup. Here I get NFSv4 working between an OpenSolaris file server and a diskless Ubuntu client. In theory, a plain mount(8) gives us a NFSv4 mount.

# mount server:/data1/opennebula/on22 /mnt/
#

Use nfsstat -m to see what kind of mount they negotiated

# nfsstat -m
...
/mnt from server:/data1/opennebula/on22
Flags: rw,relatime,vers=4,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.0.2.2,minorversion=0,addr=192.0.2.1

We have NFSv4, huzzah! Go look at the files.

# ls -lai /mnt/
total 12K
5 drwxr-xr-x 8 4294967294 4294967294 8 2011-04-19 11:50 .
3 drwxr-xr-x 21 root root 26 2011-03-17 10:22 ..
6 drwxr-xr-x 2 4294967294 4294967294 24 2011-04-19 11:50 bin
29 drwxr-xr-x 16 4294967294 4294967294 21 2011-04-19 11:50 etc
74 drwxr-xr-x 2 4294967294 4294967294 2 2011-04-19 11:50 include
75 drwxr-xr-x 7 4294967294 4294967294 7 2011-04-19 11:50 lib
296 drwxr-xr-x 5 4294967294 4294967294 5 2011-04-19 11:50 share
332 drwxr-xr-x 5 4294967294 4294967294 10 2011-04-19 11:50 var

A UID of 4294967294? That’s awesome. Wrong, but awesome. 4294967294 is -1 on a 64-bit system. Many modern Linuxish systems assign nobody and nogroup (the standard unprivileged NFS accounts) a UID and GID of -1. While my files are owned by uid 1003 on the server, and the client’s mount point is owned by uid 1003, NFSv4 defaults to mapping all UIDs to nobody. Use rpc.idmapd to map UIDs between systems. Go to /etc/default/nfs-common and enable idmapd.

NEED_IDMAPD=yes

Lower case seems to be required: I originally set this to YES and the process didn’t start.

Reboot the client, and the files are now owned by nobody. Well, at least that’s a legitimate system user, one originally created for NFS. The files are owned by UID 1003 on the server, however.

Here’s where NFSv4 gets interesting. In NFSv3 and earlier, file ownership over NFS is controlled by UID. Systems administrators worked hard to keep UIDs synchronized across their systems so that NFS permissions would be consistent. You can remap UIDs over NFS, of course, but maintaining those maps is vastly annoying.

NFSv4 maps file permissions by UIDs, but uses usernames for ACLs and ownership. Both must be correct, or common operations won’t work as expected. I have an OpenSolaris NFS server that contains lots of files for lots of diskless systems with lots of different usernames. Some of those usernames do not exist on the fileserver. While I keep user accounts in LDAP, I (mostly) don’t bother with system or program accounts. To share files over NFSv4, though, the accounts must exist on both client and server.

NFSv4 uses helper programs to map usernames and UIDs: nfsmapid on OpenSolaris, rpc.idmapd on Ubuntu, and nfsuserd on FreeBSD. (Please insert a screaming rant here: these are all basically the same program. Why, why, why change the name? We don’t give ping(8) different names even though it has completely different under-the-hood implementations on each program, do we? Sheesh.)

NFSv4 maps usernames within a domain, generally (but not necessarily) the machine’s domain name. If the NFSv4 client and server domain names doesn’t match, all the usernames will show up as “nobody.” OpenSolaris’ nfsmapid pulls the domain name from the machine’s domain name. I had to set the domain name on Ubuntu 10.10 in /etc/idmapd.conf.

NFSv4 now works in my environment.

Note that NFSv4 also has a variety of other changes. All exports are part of a single unified namespace. OpenSolaris handles that for you. If you use a different NFSv4 server, you might need to manage that namespace yourself. But that’ll be a topic for another post, when I get my FreeBSD/ZFS/iSCSI/NFSv4 server working.

5 comments to NFSv4 and UIDs on OpenSolaris and Ubuntu

  • Hi, I’m in trouble with the uid mapping of an NFS4 server on opensolaris, accessed by a Solaris 10 machine.
    No problem with files who’s uids have no matching on the server, the uid is passed to the client and the client shows the correct user.
    Some of the files have low uids that map to different names on server and client (e.g. 101=pf on server, 101=clamav on client), and these are mapped as nobody….
    Is there any way I can force UIDs instead of names in these cases? I don’t want the server and clients to have same passwd….NFS3 did this by default…
    Thanx for any help,
    Gabriele.

  • @Gabriele: The UID’s numeric value should not matter if rpc.idmapd is working correctly. It should be matching the username strings, not the numeric UIDs — though I have had some weird behavior before, usually when upgrading bits of NFS or network services, and restarting the daemons on the server was necessary (and waiting for some elements like nsswitch to time out on the client side).

    The best solution to all of this is simply to use a directory service of some type to unify your names/UIDs/GIDs once and for all. I say “simply”… but remember pulling some serious hair before I figured LDAP out (and heaven forbid my notes disappear in a fire).

  • Cliff

    4294967294 is -1? Surely it is -2?

    Cheers,

    Cliff

  • Wow. Apparently I cannot do math.