Sample Rust Module Kernel Panicking - Near relevant error message (Highlighted)

[Goclipse27]

Description

Architecture(s)

• x86_64

Toolchain versions
All default needed for Rust

• rustc:
• bindgen:
• LLVM/Clang:
• GCC:
• QEMU:

Kernel log

SeaBIOS (version 1.13.0-1ubuntu1.1)

iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+07F8C8B0+07ECC8B0 CA00

Booting from ROM..[ 0.000000] Linux version 5.19.0-159241-g459035ab65c0-dirty (raghaven@raghaven-MacBookPro) (gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #82 SMP2
[ 0.000000] Command line:
[ 0.000000] x86/fpu: x87 FPU will use FXSAVE
[ 0.000000] signal: max sigframe size: 1440
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000007fdffff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000007fe0000-0x0000000007ffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.8 present.
[ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 3096.094 MHz processor
[ 0.007246] last_pfn = 0x7fe0 max_arch_pfn = 0x400000000
[ 0.007885] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.024020] found SMP MP-table at [mem 0x000f5ca0-0x000f5caf]
[ 0.028764] RAMDISK: [mem 0x07d8d000-0x07fd7fff]
[ 0.029163] ACPI: Early table checksum verification disabled
[ 0.029951] ACPI: RSDP 0x00000000000F5AD0 000014 (v00 BOCHS )
[ 0.030175] ACPI: RSDT 0x0000000007FE156F 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.030708] ACPI: FACP 0x0000000007FE144B 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.031248] ACPI: DSDT 0x0000000007FE0040 00140B (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.031317] ACPI: FACS 0x0000000007FE0000 000040
[ 0.031373] ACPI: APIC 0x0000000007FE14BF 000078 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.031398] ACPI: HPET 0x0000000007FE1537 000038 (v01 BOCHS BXPCHPET 00000001 BXPC 00000001)
[ 0.031470] ACPI: Reserving FACP table memory at [mem 0x7fe144b-0x7fe14be]
[ 0.031496] ACPI: Reserving DSDT table memory at [mem 0x7fe0040-0x7fe144a]
[ 0.031504] ACPI: Reserving FACS table memory at [mem 0x7fe0000-0x7fe003f]
[ 0.031510] ACPI: Reserving APIC table memory at [mem 0x7fe14bf-0x7fe1536]
[ 0.031516] ACPI: Reserving HPET table memory at [mem 0x7fe1537-0x7fe156e]
[ 0.033517] No NUMA configuration found
[ 0.033559] Faking a node at [mem 0x0000000000000000-0x0000000007fdffff]
[ 0.034141] NODE_DATA(0) allocated [mem 0x07fdc000-0x07fdffff]
[ 0.036017] Zone ranges:
[ 0.036048] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.036123] DMA32 [mem 0x0000000001000000-0x0000000007fdffff]
[ 0.036135] Normal empty
[ 0.036152] Movable zone start for each node
[ 0.036180] Early memory node ranges
[ 0.036204] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.036333] node 0: [mem 0x0000000000100000-0x0000000007fdffff]
[ 0.036522] Initmem setup node 0 [mem 0x0000000000001000-0x0000000007fdffff]
[ 0.037373] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.037645] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.038745] On node 0, zone DMA32: 32 pages in unavailable ranges
[ 0.039044] ACPI: PM-Timer IO Port: 0x608
[ 0.039445] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.039790] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[ 0.039911] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.040132] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.040165] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.040246] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.040256] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.040416] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.040461] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[ 0.040750] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.041479] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.041521] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[ 0.041540] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000fffff]
[ 0.041638] [mem 0x08000000-0xfffbffff] available for PCI devices
[ 0.041663] Booting paravirtualized kernel on bare hardware
[ 0.041922] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[ 0.053386] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:1 nr_node_ids:1
[ 0.059998] percpu: Embedded 52 pages/cpu s176040 r8192 d28760 u2097152
[ 0.062293] Fallback order for Node 0: 0
[ 0.062522] Built 1 zonelists, mobility grouping on. Total pages: 31968
[ 0.062545] Policy zone: DMA32
[ 0.062717] Kernel command line: console=ttyS0 nokaslr rdinit=/sbin/init
[ 0.064036] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[ 0.064296] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.065588] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.069715] Memory: 89920K/130552K available (16396K kernel code, 2665K rwdata, 5988K rodata, 1552K init, 972K bss, 40376K reserved, 0K cma-reserved)
[ 0.072777] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.082124] Dynamic Preempt: voluntary
[ 0.084838] rcu: Preemptible hierarchical RCU implementation.
[ 0.084854] rcu: RCU event tracing is enabled.
[ 0.084876] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
[ 0.084980] Trampoline variant of Tasks RCU enabled.
[ 0.085075] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[ 0.085104] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.094699] NR_IRQS: 4352, nr_irqs: 256, preallocated irqs: 16
[ 0.099299] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.103505] Console: colour dummy device 80x25
[ 0.129798] printk: console [ttyS0] enabled
[ 0.131295] ACPI: Core revision 20220331
[ 0.136784] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
[ 0.141508] APIC: Switch to symmetric I/O mode setup
[ 0.145220] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.151363] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x2ca0dde3913, max_idle_ns: 440795305876 ns
[ 0.152411] Calibrating delay loop (skipped), value calculated using timer frequency.. 6192.18 BogoMIPS (lpj=3096094)
[ 0.153313] pid_max: default: 32768 minimum: 301
[ 0.154263] LSM: Security Framework initializing
[ 0.155496] SELinux: Initializing.
[ 0.157360] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[ 0.157727] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[ 0.180262] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.180542] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 0.181232] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.181817] Spectre V2 : Mitigation: Retpolines
[ 0.182079] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.182314] Speculative Store Bypass: Vulnerable
[ 0.581371] Freeing SMP alternatives memory: 44K
[ 0.696092] smpboot: CPU0: AMD QEMU Virtual CPU version 2.5+ (family: 0x6, model: 0x6, stepping: 0x3)
[ 0.706553] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.707147] cblist_init_generic: Setting shift to 0 and lim to 1.
[ 0.708268] Performance Events: PMU not available due to virtualization, using software events only.
[ 0.710211] BUG: unable to handle page fault for address: ffff88806f76ec61
[ 0.710211] #PF: supervisor write access in kernel mode
[ 0.710211] #PF: error_code(0x0002) - not-present page
[ 0.710211] PGD 3201067 P4D 3201067 PUD 0
[ 0.710211] Oops: 0002 [#1] PREEMPT SMP NOPTI
[ 0.710211] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.19.0-159241-g459035ab65c0-dirty #82
[ 0.710211] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[ 0.710211] RIP: 0010:init_real_mode+0x8b/0x1ec
[ 0.710211] Code: 00 00 00 80 48 2b 0d f1 d3 74 ff 48 01 ca 8b 3d 6c 2f 0a 00 31 c9 48 89 d6 48 c1 ee 04 48 39 cf 44 8b 04 8d 68 32 05 83 74 0a <66> 42 89 34 00 48 ff c1 eb e9 48 8d 3c bd 6c 32 05 839
[ 0.710211] RSP: 0018:ffffc90000013e88 EFLAGS: 00000206
[ 0.710211] RAX: ffff888000098000 RBX: 0000000000000000 RCX: 0000000000000000
[ 0.710211] RDX: 0000000000098000 RSI: 0000000000009800 RDI: 0000000065722d2d
[ 0.710211] RBP: 0000000000007263 R08: 000000006f6d6c61 R09: ffffc90000013ce8
[ 0.710211] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
[ 0.710211] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 0.710211] FS: 0000000000000000(0000) GS:ffff888007800000(0000) knlGS:0000000000000000
[ 0.710211] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.710211] CR2: ffff88806f76ec61 CR3: 000000000280c000 CR4: 00000000000006f0
[ 0.710211] Call Trace:
[ 0.710211]
[ 0.710211] ? xen_prepare_pvh+0x1c7/0x1c7
[ 0.710211] do_one_initcall+0x3f/0x1f0
[ 0.710211] ? _raw_spin_trylock+0x20/0x50
[ 0.710211] ? try_to_wake_up+0x8f/0x540
[ 0.710211] kernel_init_freeable+0xe9/0x22c
[ 0.710211] ? rest_init+0xc0/0xc0
[ 0.710211] kernel_init+0x11/0x120
[ 0.710211] ret_from_fork+0x22/0x30
[ 0.710211]
[ 0.710211] Modules linked in:
[ 0.710211] CR2: ffff88806f76ec61
[ 0.710211] ---[ end trace 0000000000000000 ]---
[ 0.710211] RIP: 0010:init_real_mode+0x8b/0x1ec
[ 0.710211] Code: 00 00 00 80 48 2b 0d f1 d3 74 ff 48 01 ca 8b 3d 6c 2f 0a 00 31 c9 48 89 d6 48 c1 ee 04 48 39 cf 44 8b 04 8d 68 32 05 83 74 0a <66> 42 89 34 00 48 ff c1 eb e9 48 8d 3c bd 6c 32 05 839
[ 0.710211] RSP: 0018:ffffc90000013e88 EFLAGS: 00000206
[ 0.710211] RAX: ffff888000098000 RBX: 0000000000000000 RCX: 0000000000000000
[ 0.710211] RDX: 0000000000098000 RSI: 0000000000009800 RDI: 0000000065722d2d
[ 0.710211] RBP: 0000000000007263 R08: 000000006f6d6c61 R09: ffffc90000013ce8
[ 0.710211] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
[ 0.710211] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 0.710211] FS: 0000000000000000(0000) GS:ffff888007800000(0000) knlGS:0000000000000000
[ 0.710211] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.710211] CR2: ffff88806f76ec61 CR3: 000000000280c000 CR4: 00000000000006f0
[ 0.711532] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
[ 0.712211] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009 ]---

Kernel config

Here is the repo with the changes: https://github.com/raghvenders/linux/tree/rust-crypto

Kernel Config:
make menuconfig -> Cryptographic API -> Pseudo Random Number Generation for Cryptographic modules.

[bjorn3]

I think the problem is that at https://github.com/raghvenders/linux/blob/95b064142c5a70d9e21d6900451f8b5a0265fdb6/rust/kernel/crypto.rs#L38 you shouldn't dereference the result of buf.as_mut_ptr().

[Goclipse27]

Thanks @bjorn3 . However I am afraid if I could see any other pr_info or crypto pr_info ********* Rust Crypto File *********** any where and moreover even after commenting that line still panics.

Even on some other changes, I encountered these errors. the way I found to move forward is re-creating .config again and re-build entire kernel with enable sample rust modules.

Any better way to find the spot, fix it and re-run?

[ojeda]

As @bjorn3 says, it doesn't look like an issue with code we have in the repo at the moment.

In addition, even before the dereference happens, you are doing uninit().assume_init() which is UB (in general, I suggest trying first with the minimal unsafe code possible, e.g. using an initialized array in this case, when you encounter this kind of issue).

Even on some other changes, I encountered these errors. the way I found to move forward is re-creating .config again and re-build entire kernel with enable sample rust modules.

I am not sure what you mean, but you should not need to recreate the kernel configuration to iterate while developing.

[fbq]

Thanks @bjorn3 . However I am afraid if I could see any other pr_info or crypto pr_info ********* Rust Crypto File *********** any where and moreover even after commenting that line still panics.

Even on some other changes, I encountered these errors. the way I found to move forward is re-creating .config again and re-build entire kernel with enable sample rust modules.

Any better way to find the spot, fix it and re-run?

You may want to see if the exact issue you hit in the log can also happen even without any of your change. Also could you share your qemu command and full .config file?

[Goclipse27]

Thanks @ojeda @fbq. By the way, my understanding is at least to exclude Rust and run in additions to sample rust it is expected to run. I did CONFIG_RUST is not set or N value for rust samples through menuconfig. While simple Make command, I do give CONFIG_RUST as N too.
After I build and run as sudo qemu-system-x86_64 -nographic -kernel vmlinux -initrd initrd.img -nic user,model=rtl8139,hostfwd=tcp::5555-:23, i get the same kernel panicking error.

Here is my config attached , this is the .config file generated by defconfig.(I may have made some modules Y/M for some other testing)
config.txt

1. make defconfig qemu-busybox-min.config rust.config

2. Then I make plain make command, kernel build runs.

---

To make my qemu run again, i might have to do this 1 and 2.

For @ojeda, sure I will try initialized array.

since uninit_array() is nightly I tried below

pub const fn uninit_array<const N: usize>() -> [Self; N] {
        // SAFETY: An uninitialized `[MaybeUninit<_>; LEN]` is valid.
        unsafe { MaybeUninit::<[MaybeUninit<T>; N]>::uninit().assume_init() }
    }

as we have to have *mut u8_ (pointer to an array which hold the data buffer) and array length of 32 is needed. It has to be an uninitialized array of some length. (May be I have to change correct length).

I did the below sample

`#![allow(unused)]
#![allow(invalid_value)]
fn main() {
use std::mem::{self, MaybeUninit};

//let b: bool = unsafe { mem::uninitialized() }; // undefined behavior! ⚠️
// The equivalent code with MaybeUninit<bool>:
let mut buf = unsafe { MaybeUninit::<[u8; 32]>::uninit().assume_init() };

let s = &mut buf.as_mut_ptr();
let l = &mut buf.len();

println!("{:?} {:?}", s, l);
}`

it works.

[ojeda]

my understanding is at least to exclude Rust and run in additions to sample rust it is expected to run.

Yes, the idea is to see whether the problem is reproducible with the "vanilla" repo (i.e. without changes), or whether the issue comes from the changes introduced in your repository.

make defconfig qemu-busybox-min.config rust.config

That will give you a fairly big kernel -- instead of defconfig, consider allnoconfig. I recommend watching Wedson's LF Mentorship series: https://linuxfoundation.org/webinars/writing-linux-kernel-modules-in-rust/

It has to be an uninitialized array of some length.

Are you sure that is the case? If the only reason is performance, I would recommend avoiding optimization-based unsafe blocks until your kernel module works.

let mut buf = unsafe { MaybeUninit::<[u8; 32]>::uninit().assume_init() };

Note that this is still UB, because this is not an array of MaybeUninit<u8>s, but an array of u8s that you are telling the compiler are initialized.

[ojeda]

Closing -- if there is something that needs to still be addressed, please let us know.