Can't start Jail due to ifconfig: ioctl (SIOCAIFADDR): File exists

[tobybryans]

I had to increase storage on a FreeBSD 14.1 VM last night which needs a powerdown. When I bought it up again one of my Bastille jails refused to come up with the following error:

[spamd]:
ifconfig: ioctl (SIOCAIFADDR): File exists
jail: spamd: /sbin/ifconfig bastille0 inet 10.13.0.8 netmask 255.255.255.255 alias: failed

All other jails started fine. I tried manually staring and stopping, but it still wouldn't come up with the same error.

I destroyed the jail, tried to recreate it and the same thing happened on jail creation.

I recreated it on another IP (10.13.0.25) and it worked fine.

an ifconfig returns the following:

ifconfig bastille0
bastille0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.13.0.1 netmask 0xffffffff
        inet 10.13.0.7 netmask 0xffffffff
        inet 10.13.0.10 netmask 0xffffffff
        inet 10.13.0.9 netmask 0xffffffff
        inet 10.13.0.12 netmask 0xffffffff
        inet 10.13.0.5 netmask 0xffffffff
        inet 10.13.0.14 netmask 0xffffffff
        inet 10.13.0.4 netmask 0xffffffff
        inet 10.13.0.19 netmask 0xffffffff
        inet 10.13.0.11 netmask 0xffffffff
        inet 10.13.0.6 netmask 0xffffffff
        inet 10.13.0.21 netmask 0xffffffff
        inet 10.13.0.2 netmask 0xffffffff
        inet 10.13.0.3 netmask 0xffffffff
        inet 10.13.0.16 netmask 0xffffffff
        inet 10.13.0.17 netmask 0xffffffff
        inet 10.13.0.18 netmask 0xffffffff
        inet 10.13.0.22 netmask 0xffffffff
        inet 10.13.0.24 netmask 0xffffffff
        inet 10.13.0.20 netmask 0xffffffff
        inet 10.13.0.23 netmask 0xffffffff
        inet 10.13.0.15 netmask 0xffffffff
        inet 10.13.0.25 netmask 0xffffffff
        inet6 fe80::1%bastille0 prefixlen 64 scopeid 0x3
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

and 10.13.0.8 is not on that list (or on any other interfaces).

I am at a bit of a loss of how to debug this further, so any advice would be greatly appreciated, and I fully understand that this may not be a Bastille issue, so if that is the case happy to take the ticket elsewhere.

[bmac2]

@cedwards any ideas on what hung him up????

[tschettervictor]

At the beginning of the "start.sh" and "stop.sh" files, add "set -x" to enable debug mode. Then run the commands again and you can see what it is hanging on.

[bmac2]

@tobybryans can you add the set -x as @tschettervictor mentioned to help us zoom in on what is hanging?

[JRGTH]

Hello, I've added a quick debug option for the start/stop commands to track down issues for the developers and power users, please test here: #791 before I request a merge.

Regards!

[tobybryans]

Hi all, first of all: apologies for the delay on this, thank you for the comments.

Here's the output of doas bastille create test 14.1-RELEASE 10.13.0.8 2>&1 | tee /tmp/output.txt

Valid: (10.13.0.8).

Creating a thinjail...

+ . /usr/local/share/bastille/common.sh
+ COLOR_RED=''
+ COLOR_GREEN=''
+ COLOR_YELLOW=''
+ COLOR_RESET=''
+ [ -z '' -a -t 1 ]
+ . /usr/local/etc/bastille/bastille.conf
+ bastille_prefix=/usr/local/bastille
+ bastille_backupsdir=/usr/local/bastille/backups
+ bastille_cachedir=/usr/local/bastille/cache
+ bastille_jailsdir=/usr/local/bastille/jails
+ bastille_releasesdir=/usr/local/bastille/releases
+ bastille_templatesdir=/usr/local/bastille/templates
+ bastille_logsdir=/var/log/bastille
+ bastille_pf_conf=/etc/pf.conf
+ bastille_sharedir=/usr/local/share/bastille
+ bastille_bootstrap_archives=base
+ bastille_tzdata=''
+ bastille_resolv_conf=/etc/resolv.conf
+ bastille_url_freebsd=http://ftp.freebsd.org/pub/FreeBSD/releases/
+ bastille_url_hardenedbsd=https://installers.hardenedbsd.org/pub/
+ bastille_url_midnightbsd=https://www.midnightbsd.org/ftp/MidnightBSD/releases/
+ bastille_zfs_enable=YES
+ bastille_zfs_zpool=u01
+ bastille_zfs_prefix=bastille
+ bastille_zfs_options='-o compress=lz4 -o atime=off'
+ bastille_compress_xz_options='-0 -v'
+ bastille_decompress_xz_options='-c -d -v'
+ bastille_compress_gz_options='-1 -v'
+ bastille_decompress_gz_options='-k -d -c -v'
+ bastille_export_options=''
+ bastille_network_loopback=bastille0
+ bastille_network_pf_ext_if=ext_if
+ bastille_network_pf_table=jails
+ bastille_network_shared=''
+ bastille_network_gateway=''
+ bastille_network_gateway6=''
+ bastille_template_base=default/base
+ bastille_template_empty=''
+ bastille_template_thick=default/thick
+ bastille_template_clone=default/clone
+ bastille_template_thin=default/thin
+ bastille_template_vnet=default/vnet
+ [ 1 -gt 1 ]
+ [ 1 -lt 1 ]
+ bastille_root_check
+ id -u
+ [ 0 -ne 0 ]
+ TARGET=test
+ shift
+ [ test '=' ALL ]
+ [ test '!=' ALL ]
+ bastille list jails
+ awk '/^test$/'
+ JAILS=test
+ [ ! -d /usr/local/bastille/jails/test ]
+ /usr/sbin/jls name
+ awk '/^test$/'
+ [ '' ]
+ /usr/sbin/jls name
+ awk '/^test$/'
+ [ ! '' ]
+ bastille config test get vnet
+ [ 'not set' '!=' enabled ]
+ bastille config test get interface
+ _interface=bastille0
+ ifconfig
+ grep ^bastille0:
+ bastille config test get ip4.addr
+ ip=10.13.0.8
+ [ -n 10.13.0.8 ]
+ ifconfig
+ grep -wF 10.13.0.8
+ pfctl -q -t jails -T add 10.13.0.8
+ info '[test]:'
+ echo -e '[test]:'
[test]:
+ jail -f /usr/local/bastille/jails/test/jail.conf -c test
ifconfig: ioctl (SIOCAIFADDR): File exists
jail: test: /sbin/ifconfig bastille0 inet 10.13.0.8 netmask 255.255.255.255 alias: failed
+ [ -s /usr/local/bastille/jails/test/rctl.conf ]
+ [ -s /usr/local/bastille/jails/test/rdr.conf ]
+ echo

[test]: Not started. See 'bastille start test'.
[test]: Not started. See 'bastille start test'.
+ . /usr/local/share/bastille/common.sh
+ COLOR_RED=''
+ COLOR_GREEN=''
+ COLOR_YELLOW=''
+ COLOR_RESET=''
+ [ -z '' -a -t 1 ]
+ . /usr/local/etc/bastille/bastille.conf
+ bastille_prefix=/usr/local/bastille
+ bastille_backupsdir=/usr/local/bastille/backups
+ bastille_cachedir=/usr/local/bastille/cache
+ bastille_jailsdir=/usr/local/bastille/jails
+ bastille_releasesdir=/usr/local/bastille/releases
+ bastille_templatesdir=/usr/local/bastille/templates
+ bastille_logsdir=/var/log/bastille
+ bastille_pf_conf=/etc/pf.conf
+ bastille_sharedir=/usr/local/share/bastille
+ bastille_bootstrap_archives=base
+ bastille_tzdata=''
+ bastille_resolv_conf=/etc/resolv.conf
+ bastille_url_freebsd=http://ftp.freebsd.org/pub/FreeBSD/releases/
+ bastille_url_hardenedbsd=https://installers.hardenedbsd.org/pub/
+ bastille_url_midnightbsd=https://www.midnightbsd.org/ftp/MidnightBSD/releases/
+ bastille_zfs_enable=YES
+ bastille_zfs_zpool=u01
+ bastille_zfs_prefix=bastille
+ bastille_zfs_options='-o compress=lz4 -o atime=off'
+ bastille_compress_xz_options='-0 -v'
+ bastille_decompress_xz_options='-c -d -v'
+ bastille_compress_gz_options='-1 -v'
+ bastille_decompress_gz_options='-k -d -c -v'
+ bastille_export_options=''
+ bastille_network_loopback=bastille0
+ bastille_network_pf_ext_if=ext_if
+ bastille_network_pf_table=jails
+ bastille_network_shared=''
+ bastille_network_gateway=''
+ bastille_network_gateway6=''
+ bastille_template_base=default/base
+ bastille_template_empty=''
+ bastille_template_thick=default/thick
+ bastille_template_clone=default/clone
+ bastille_template_thin=default/thin
+ bastille_template_vnet=default/vnet
+ [ 1 -gt 1 ]
+ [ 1 -lt 1 ]
+ bastille_root_check
+ id -u
+ [ 0 -ne 0 ]
+ TARGET=test
+ shift
+ [ test '=' ALL ]
+ [ test '!=' ALL ]
+ bastille list jails
+ awk '/^test$/'
+ JAILS=test
+ [ ! -d /usr/local/bastille/jails/test ]
+ /usr/sbin/jls name
+ awk '/^test$/'
+ [ '' ]
+ /usr/sbin/jls name
+ awk '/^test$/'
+ [ ! '' ]
+ bastille config test get vnet
+ [ 'not set' '!=' enabled ]
+ bastille config test get interface
+ _interface=bastille0
+ ifconfig
+ grep ^bastille0:
+ bastille config test get ip4.addr
+ ip=10.13.0.8
+ [ -n 10.13.0.8 ]
+ ifconfig
+ grep -wF 10.13.0.8
+ pfctl -q -t jails -T add 10.13.0.8
+ info '[test]:'
+ echo -e '[test]:'
[test]:
+ jail -f /usr/local/bastille/jails/test/jail.conf -c test
ifconfig: ioctl (SIOCAIFADDR): File exists
jail: test: /sbin/ifconfig bastille0 inet 10.13.0.8 netmask 255.255.255.255 alias: failed
+ [ -s /usr/local/bastille/jails/test/rctl.conf ]
+ [ -s /usr/local/bastille/jails/test/rdr.conf ]
+ echo

The contents of the jail.conf file is as follows:

test {
  devfs_ruleset = 4;
  enforce_statfs = 2;
  exec.clean;
  exec.consolelog = /var/log/bastille/test_console.log;
  exec.start = '/bin/sh /etc/rc';
  exec.stop = '/bin/sh /etc/rc.shutdown';
  host.hostname = test;
  mount.devfs;
  mount.fstab = /usr/local/bastille/jails/test/fstab;
  path = /usr/local/bastille/jails/test/root;
  securelevel = 2;
  osrelease = 14.1-RELEASE;

  interface = bastille0;
  ip4.addr = 10.13.0.8;

  ip6 = disable;
}

[tschettervictor]

Did you reboot your server before this? I'm assuming so.

[tobybryans]

Yes - this started after a disk space upgrade (it's a VM), and when it first happened I rebooted a couple of times to try to clear it. The server has been up since then, but I'm about to do the 14.2 upgrade so will report after that if the issue still stands.

[bmac2]

Update on this issue?