[zfs-discuss] kernel oops removing empty fs

[Douglas J Hunley]

I've got an empty fs called data/repos that I can't seem to get rid of.
Running 'zfs destroy' on it throws a kernel oops and the command never
returns:
[614835.025699] BUG: unable to handle kernel NULL pointer dereference at
0000000000000018
[614835.025974] IP: [<ffffffffc0329564>] zap_count_write+0x157/0x314 [zfs]
[614835.026422] PGD ca772067 PUD ca5cd067 PMD 0
[614835.026956] Oops: 0000 [#1] SMP
[614835.027465] Modules linked in: r8169 mii zfs(PO) zunicode(PO) zavl(PO)
zcommon(PO) znvpair(PO) spl(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O)
[614835.028181] CPU: 1 PID: 609 Comm: txg_sync Tainted: P O
3.17.8-gentoo-r1 #1
[614835.028913] Hardware name: To be filled by O.E.M. To be filled by
O.E.M./SABERTOOTH 990FX, BIOS 1604 10/16/2012
[614835.029524] task: ffff8807f75c3bf0 ti: ffff8807f30bc000 task.ti:
ffff8807f30bc000
[614835.030154] RIP: 0010:[<ffffffffc0329564>] [<ffffffffc0329564>]
zap_count_write+0x157/0x314 [zfs]
[614835.030814] RSP: 0018:ffff8807f30bfcd8 EFLAGS: 00010286
[614835.031500] RAX: 000000000000001c RBX: ffff880803018000 RCX:
000000000000001e
[614835.032210] RDX: 000000000000001d RSI: 000000000000001c RDI:
ffff8808027cf800
[614835.032982] RBP: ffff8808027cf800 R08: 0000000000000000 R09:
0000000000000002
[614835.033743] R10: ffff8800cbefd2e8 R11: ffffffffc02dd773 R12:
0000000000000002
[614835.034368] R13: 0000000000000000 R14: 000000000000001c R15:
000000000000001d
[614835.035027] FS: 00007f0bb3b3b700(0000) GS:ffff88082fc80000(0000)
knlGS:00000000f7462700
[614835.035648] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[614835.036421] CR2: 0000000000000018 CR3: 00000000cb310000 CR4:
00000000000007e0
[614835.037240] Stack:
[614835.038011] 0000001900000001 ffff88062712d440 ffff8807f30bfd38
ffff880503463cc0
[614835.038672] ffff880803018000 0000000000000000 ffff880503463cc0
0000000000000000
[614835.039465] ffff880803018000 ffff8807f75e4200 ffffffffc0329aa9
ffff880503463cc0
[614835.040246] Call Trace:
[614835.040898] [<ffffffffc0329aa9>] ? spa_feature_decr+0x39/0x9b [zfs]
[614835.041574] [<ffffffffc02ffe43>] ? dsl_scan_sync+0x253/0x88b [zfs]
[614835.042295] [<ffffffffc0255cd4>] ? spl_kmem_cache_free+0x43/0x157 [spl]
[614835.042969] [<ffffffffc030d2b3>] ? spa_sync+0x583/0x921 [zfs]
[614835.043768] [<ffffffff9c07013a>] ? timekeeping_get_ns+0xa/0x2e
[614835.044440] [<ffffffffc031968d>] ? txg_fini+0x455/0x63f [zfs]
[614835.045182] [<ffffffffc0319415>] ? txg_fini+0x1dd/0x63f [zfs]
[614835.045872] [<ffffffffc0319415>] ? txg_fini+0x1dd/0x63f [zfs]
[614835.046497] [<ffffffffc0257447>] ? __thread_exit+0xe/0x83 [spl]
[614835.047169] [<ffffffffc02574b4>] ? __thread_exit+0x7b/0x83 [spl]
[614835.047872] [<ffffffff9c04b52b>] ? kthread+0xa7/0xaf
[614835.048610] [<ffffffff9c040000>] ? ptrace_do_notify+0xd/0x72
[614835.049411] [<ffffffff9c04b484>] ? __kthread_parkme+0x5b/0x5b
[614835.050197] [<ffffffff9c59202c>] ? ret_from_fork+0x7c/0xb0
[614835.050891] [<ffffffff9c04b484>] ? __kthread_parkme+0x5b/0x5b
[614835.051595] Code: db 48 89 45 00 5a 89 d8 5b 5d c3 41 57 48 89 f0 49 89
d7 41 56 49 89 f6 41 55 4d 89 c5 41 54 45 89 cc 55 48 89 fd 53 48 83 ec 20
<41> 83 78 18 00 4c 8d 4c 24 18 48 89 4c 24 08 b9 08 00 00 00 48
[614835.053358] RIP [<ffffffffc0329564>] zap_count_write+0x157/0x314 [zfs]
[614835.054124] RSP <ffff8807f30bfcd8>
[614835.055021] CR2: 0000000000000018
[614835.096283] ---[ end trace 1da9d2f3290bce69 ]---
[614835.097147] ------------[ cut here ]------------
[614835.097972] WARNING: CPU: 1 PID: 609 at kernel/smp.c:292
smp_call_function_single+0x5b/0x7c()
[614835.098732] Modules linked in: r8169 mii zfs(PO) zunicode(PO) zavl(PO)
zcommon(PO) znvpair(PO) spl(O) vboxnetadp(O) vboxnetflt(O) vboxdrv(O)
[614835.099649] CPU: 1 PID: 609 Comm: txg_sync Tainted: P D O
3.17.8-gentoo-r1 #1
[614835.100511] Hardware name: To be filled by O.E.M. To be filled by
O.E.M./SABERTOOTH 990FX, BIOS 1604 10/16/2012
[614835.101372] 0000000000000006 ffffffff9c58c5b8 0000000000000000
ffffffff9c03792e
[614835.102255] ffffffff9c07a7db 0000000000000001 ffffffff9c0995ef
ffff8807f30bf9e8
[614835.103038] 0000000000000001 ffffffff9c07a7db 000000012fc90778
ffff8807f75c3bf0
[614835.103789] Call Trace:
[614835.104574] [<ffffffff9c58c5b8>] ? dump_stack+0x41/0x51
[614835.105512] [<ffffffff9c03792e>] ? warn_slowpath_common+0x6f/0x84
[614835.106394] [<ffffffff9c07a7db>] ? smp_call_function_single+0x5b/0x7c
[614835.107321] [<ffffffff9c0995ef>] ?
perf_swevent_start_hrtimer.part.41+0x51/0x51
[614835.108220] [<ffffffff9c07a7db>] ? smp_call_function_single+0x5b/0x7c
[614835.109132] [<ffffffff9c098a55>] ? task_function_call+0x42/0x4c
[614835.110022] [<ffffffff9c09cdd2>] ? perf_cgroup_switch+0x12a/0x12a
[614835.110852] [<ffffffff9c08649a>] ? cgroup_exit+0xa1/0xc9
[614835.111576] [<ffffffff9c0395fe>] ? do_exit+0x409/0x90b
[614835.112373] [<ffffffff9c58ae3f>] ? printk+0x4a/0x52
[614835.113304] [<ffffffff9c004dd1>] ? oops_end+0x76/0x79
[614835.114149] [<ffffffff9c02cf2f>] ? no_context+0x2ba/0x309
[614835.115027] [<ffffffff9c02d4e4>] ? __do_page_fault+0x1d4/0x37e
[614835.115877] [<ffffffffc02ecc09>] ? dmu_tx_fini+0xe2/0x283 [zfs]
[614835.116648] [<ffffffffc02ecb8e>] ? dmu_tx_fini+0x67/0x283 [zfs]
[614835.117462] [<ffffffffc02ed623>] ? dmu_zfetch+0x6a9/0x13a6 [zfs]
[614835.118148] [<ffffffff9c5936c2>] ? page_fault+0x22/0x30
[614835.118846] [<ffffffffc02dd773>] ? dbuf_hold_impl+0x7c/0x8b [zfs]
[614835.119557] [<ffffffffc0329564>] ? zap_count_write+0x157/0x314 [zfs]
[614835.120354] [<ffffffffc0329aa9>] ? spa_feature_decr+0x39/0x9b [zfs]
[614835.121135] [<ffffffffc02ffe43>] ? dsl_scan_sync+0x253/0x88b [zfs]
[614835.121843] [<ffffffffc0255cd4>] ? spl_kmem_cache_free+0x43/0x157 [spl]
[614835.122492] [<ffffffffc030d2b3>] ? spa_sync+0x583/0x921 [zfs]
[614835.123137] [<ffffffff9c07013a>] ? timekeeping_get_ns+0xa/0x2e
[614835.123905] [<ffffffffc031968d>] ? txg_fini+0x455/0x63f [zfs]
[614835.124604] [<ffffffffc0319415>] ? txg_fini+0x1dd/0x63f [zfs]
[614835.125322] [<ffffffffc0319415>] ? txg_fini+0x1dd/0x63f [zfs]
[614835.125930] [<ffffffffc0257447>] ? __thread_exit+0xe/0x83 [spl]
[614835.126655] [<ffffffffc02574b4>] ? __thread_exit+0x7b/0x83 [spl]
[614835.127276] [<ffffffff9c04b52b>] ? kthread+0xa7/0xaf
[614835.127921] [<ffffffff9c040101>] ? do_signal_stop+0x9c/0x1df
[614835.128514] [<ffffffff9c04b484>] ? __kthread_parkme+0x5b/0x5b
[614835.129207] [<ffffffff9c59202c>] ? ret_from_fork+0x7c/0xb0
[614835.129889] [<ffffffff9c04b484>] ? __kthread_parkme+0x5b/0x5b
[614835.130587] ---[ end trace 1da9d2f3290bce6a ]---

hunley ~ # uname -r
3.17.8-gentoo-r1
hunley ~ # zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
data 5.44T 1.02T 4.41T 18% 1.00x ONLINE -
hunley ~ # zfs list
NAME USED AVAIL REFER MOUNTPOINT
data 1.02T 4.33T 136K none
data/home 548G 4.33T 540G /home
data/media 499G 4.33T 499G /media
data/postgresql 12.3M 4.33T 10.7M /var/lib/postgresql
data/repos 136K 4.33T 136K none
data/root 136K 4.33T 136K /root

oddly, if I open another window, 'zfs list' shows it as gone:
hunley ~ # zfs list
NAME USED AVAIL REFER MOUNTPOINT
data 1.02T 4.33T 136K none
data/home 548G 4.33T 540G /home
data/media 499G 4.33T 499G /media
data/postgresql 12.3M 4.33T 10.7M /var/lib/postgresql
data/root 136K 4.33T 136K /root

but when I reboot to clear things up(which exports and imports the pool) it
comes back.

hunley ~ # eix zfs
[I] sys-fs/zfs
Available versions: (~)0.6.0_rc13-r2^t (~)0.6.0_rc14-r1^t
(~)0.6.1-r4^t (~)0.6.2^t (~)0.6.2-r1^t (~)0.6.2-r2^t (~)0.6.2-r3^t
(~)0.6.2-r4^t (~)0.6.2-r5^t (~)0.6.3^t{tbz2} (~)0.6.3-r2^t{tbz2} **9999^t
{bash-completion custom-cflags debug kernel-builtin +rootfs selinux
static-libs test-suite KERNEL="linux" PYTHON_TARGETS="python2_7 python3_3
python3_4"}
Installed versions: 0.6.3-r2^t{tbz2}(06:01:27 PM
12/03/2014)(bash-completion rootfs -custom-cflags -debug -kernel-builtin
-static-libs -test-suite PYTHON_TARGETS="python2_7 python3_3 -python3_4")
Homepage: http://zfsonlinux.org/
Description: Userland utilities for ZFS Linux kernel module

[I] sys-fs/zfs-kmod
Available versions: (~)0.6.0_rc13-r4^t (~)0.6.0_rc14-r5^t
(~)0.6.1-r2^t (~)0.6.2-r3^t (~)0.6.2-r4^t (~)0.6.2-r5^t (~)0.6.3^st{tbz2}
(~)0.6.3-r1^st{tbz2} **9999^st {custom-cflags debug +rootfs KERNEL="linux"}
Installed versions: 0.6.3-r1^t{tbz2}(12:10:50 PM 02/13/2015)(rootfs
-custom-cflags -debug KERNEL="linux")
Homepage: http://zfsonlinux.org/
Description: Linux ZFS kernel module for sys-fs/zfs

Found 2 matches.

hunley ~ # eix spl$
[I] sys-kernel/spl
Available versions: (~)0.6.0_rc13-r2^t (~)0.6.0_rc14-r3^t
(~)0.6.1-r3^t (~)0.6.2^t (~)0.6.2-r1^t (~)0.6.2-r2^t (~)0.6.2-r3^t
(~)0.6.2-r4^t (~)0.6.2-r5^t (~)0.6.3^st{tbz2} (~)0.6.3-r1^st{tbz2}
**9999^st {custom-cflags debug debug-log KERNEL="linux"}
Installed versions: 0.6.3-r1^t{tbz2}(12:01:46 PM
02/13/2015)(-custom-cflags -debug -debug-log KERNEL="linux")
Homepage: http://zfsonlinux.org/
Description: The Solaris Porting Layer is a Linux kernel
module which provides many of the Solaris kernel APIs

hunley ~ #

hunley ~ # zpool status
pool: data
state: ONLINE
scan: scrub repaired 0 in 1h9m with 0 errors on Tue Feb 24 11:31:38 2015
config:

NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
wwn-0x5000c5005df17d6d ONLINE 0 0 0
wwn-0x5000c5005df13fa3 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
wwn-0x5000c5005dea04a1 ONLINE 0 0 0
wwn-0x5000c5005df28a69 ONLINE 0 0 0
cache
zram0 ONLINE 0 0 0

errors: No known data errors
hunley ~ #

My daily scrub has not found a single issue.

I vaguely recall someone else having something like this before on this
list, but my searching didn't find it.

[AndCycle]

Downgrade to 0.6.3, this is known bug in 0.6.3-r1/r2 package

To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+***@zfsonlinux.org.