The third release in the NetBSD-8 is now available.

This release includes all the security fixes in NetBSD-8 up until this point, and other fixes deemed important for stability.

Some highlights include:

• x86: fixed regression in booting old CPUs
• x86: Hyper-V Gen.2 VM framebuffer support
• httpd(8): fixed various security issues
• ixg(4): various fixes / improvements
• x86 efiboot: add tftp support, fix issues on machines with many memory segments, improve graphics mode logic to work on more machines.
• Various kernel memory info leaks fixes
• Update expat to 2.2.8
• Fix ryzen USB issues and support xHCI version 3.10.
• Accept root device specification as NAME=label.
• Add multiboot 2 support to x86 bootloaders.
• Fix for CVE-2019-9506: 'Key Negotiation of Bluetooth' attack.
• nouveau: limit the supported devices and fix firmware loading.
• radeon: fix loading of the TAHITI VCE firmware.
• named(8): stop using obsolete dnssec-lookaside.

You can download binaries of NetBSD 8.2 from our Fastly-provided CDN.

For more details refer to the CHANGES-8.2 file.

Please note that we are looking for donations again, see Fundraising 2020.

Enjoy!

Maya

Typically, some time after releasing a new NetBSD major version (such as NetBSD 9.0), we will announce the end-of-life of the N-2 branch, in this case NetBSD-7.

We've decided to hold off on doing that to ensure our users don't feel rushed to perform a major version update on any remote machines, possibly needing to reach the machine if anything goes wrong.

Security fixes will still be made to the NetBSD-7 branch.

We hope you're all safe. Stay home.

The NetBSD team of developers maintains two copies of GDB:

• One in the base-system with a stack of local patches.
• One in pkgsrc with mostly build fix patches.

The process of maintaining a modern version (GPLv3) of GDB in basesystem is tainted with a constant extra cost. The NetBSD developers need to rebase the stack of local patches for the newer releases of the debugger and resurrect the support. The GDB project is under an active development and in active refactoring of the code, that was originally written in C, to C++.

Unfortunately we cannot abandon the local basesystem patches and rely on a pristine version as there is lack of feature parity in the pkgsrc version of GDB: no threading support, not operational support for most targets, no fork/vfork/etc events support, no auxv reading support on 64-bit kernels, no proper support of signals, single step etc.

Additionally there are extra GDB patches stored in pkgsrc-wip (created by me last year), that implement the gdbserver support for NetBSD/amd64. gdbserver is a GDB version that makes it possible to remotely debug other programs even across different Operating Systems and CPUs. This code has still not been merged into the mainline base-system version. This month, I have discovered that support needs to be reworked, as the preexisting source code directory hierarchy was rearranged.

Unless otherwise specified all the following changes were upstreamed to the mainstream GDB repository. According to the GDB schedule, the GDB10 branch point is planned on 2020-05-15 with release on 2020-06-05. It's a challenge to see how much the GDB support can be improved by then for NetBSD!

PSIM

The GDB debugger contains PSIM (Model of the PowerPC Architecture) originally developed by Andrew Cagney between 1994 and 1996. This is a simulator that contains, among other things, NetBSD support in the UEA mode. This means that GDB can run static programs prebuilt for NetBSD without execution on a real PowerPC hardware. In order to make it work, there is need to wrap the kernel interfaces such as syscalls, errno values and signals and handle them in the simulator.

I have updated the list of errno names and signal names with NetBSD 9.99.49.

It would be nice to still update the list of syscalls to reflect the current kernels, but I have deferred this into future.

bfd changes

The AArch64 (NetBSD/evbarm) target uses PT_GETREGS and PT_GETFPREGS operation names with the same Machine Dependent values as NetBSD/alpha and NetBSD/sparc. This knowledge is required as these values are used in core(5) files, as emitted by a crashing program. I've added a patch that recognizes these ELF notes in arm64 coredumps appropriately.

I've also added a new define constant NT_NETBSDCORE_AUXV. This allows properly identifying AUXV ELF notes in core files. Meanwhile I have implemented and added detection of LWPSTATUS notes. This note ships with meta information (name, signal context, TLS base, etc) about threads in a process in a core.

The number of ARM and MIPS boards supported by NetBSD is huge and there are multiple variations of them. I have fixed the detection macro in bfd to recognize more arm and mips NetBSD installations.

GDB/NetBSD fixes in CPU specific files

I have reached the state of GDB being more operational for more NetBSD ports out of the box. There were missing features and build issues that has been addressed. I have committed the following changes:

• Define _KERNTYPES in vax-bsd-nat.c
• Define _KERNTYPES in ppc-nbsd-nat.c
• Define _KERNTYPES in mips-nbsd-nat.c
• Inherit vax_bsd_nat_target from nbsd_nat_target
• Add explicit cast to fix build of vax-bsd-nat.c
• Add support for threads in vax_bsd_nat_target
• Add support for NetBSD threads in x86-bsd-nat.c
• Inherit arm_netbsd_nat_target from nbsd_nat_target
• Add support for NetBSD threads in arm-nbsd-nat.c
• Define _KERNTYPES in alpha-bsd-nat.c
• Inherit alpha_netbsd_nat_target from nbsd_nat_target
• Remove unused code from alpha-bsd-nat.c
• Add support for NetBSD threads in alpha-bsd-nat.c
• Define _KERNTYPES in m68k-bsd-nat.c
• Inherit m68k_bsd_nat_target from nbsd_nat_target
• m68k: bsd: Change type from char * to gdb_byte *
• Add support for NetBSD threads in m68k-bsd-nat.c
• Include missing header to get missing declarations
• Inherit sh_nbsd_nat_target from nbsd_nat_target
• Add support for NetBSD threads in sh-nbsd-nat.c
• Add support for NetBSD threads in sparc-nat.c
• Add support for NetBSD threads in amd64-bsd-nat.c
• Add support for NetBSD threads in i386-bsd-nat.c
• Add support for NetBSD threads in ppc-nbsd-nat.c
• Add support for NetBSD threads in hppa-nbsd-nat.c
• Inherit ppc_nbsd_nat_target from nbsd_nat_target
• Update the return type of gdb_ptrace to be more flexible
• Avoid get_ptrace_pid() usage on NetBSD in x86-bsd-nat.c

Now support for NetBSD in various CPU-specific files improved significantly, however there are still missing features, especially KGDB debugging and unwinding the stack over the signal trampoline. There are still smaller or larger changes that might be needed on per-port basis and I will keep working on them. There is need to develop at least proper aarch64 support as it is missing upstream. We might evaluate what to do with at least Itanium and RISCV.

CPU Generic improvements in the GDB codebase

I've switched the nbsd_nat_target::pid_to_exec_file() function from a logic of reading the /proc entries to a sysctl(3) based solution.

As the gdbserver support is around the corner, I have improved small parts of the code base to be compatibile with NetBSD. I've fixed the unconditional inclusion of alloca.h in gdbsupport. Another fix namespaced a local class reg, because it conflicted with the struct reg from the NetBSD headers.

The current logic of get_ptrace_pid function matches the semantics of other kernels suchs as Linux and FreeBSD. With the guidance of upstream developers, I have disabled this function completely for NetBSD instead of patching it for the NetBSD specific behavior of maintaining pairs PID+LWP for each internal ptid_t entry (that reflects the relation of PID, LWP and TID).

Plan for the next milestone

Finish reimplementing operational support of debugging of multi-threaded programs and upstream more patches, especially CPU-independent ones.

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.

In February 2019, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, fixing watchpoint and threading support, porting to i386.

March 2020 was the last month of my contract. During it my primary focus was to prepare integration of LLDB into NetBSD's src tree.

LLDB integration

The last important goal for the contract was to include LLDB in the NetBSD src tree. This mainly involved porting LLDB build into NetBSD src tree Makefiles. The resulting patches were sent to the tech-toolchain mailing list: [PATCH 0/7] LLDB import to src.

My proposed integration is based on LLDB tree from 2019-10-29. This matches the LLVM/Clang version currently imported in NetBSD. Newer version can not be used directly due to API incompatibility between the projects, and it is easier to backport LLDB fixes than to fix LLVM API missync.

The backports applied on top of this commit include all my contracted work, plus Kamil Rytarowski's work on LLDB. This also includes necessary fixes to make LLDB build against current NetBSD ptrace() API. Two source files in liblldbHost are renamed to ensure unique filenames within that library, as necessary to build from NetBSD Makefiles without resorting to ugly hacks.

Upstream uses to build individual LLDB components and plugins into split static libraries, then combine them all into a shared liblldb.so library. Both lldb and lldb-server executables link to it. We currently can not follow this model as LLVM and Clang sources are built without -fPIC and therefore are not suitable for shared libraries.

Therefore, we build everything as static libraries instead. This causes the logic that upstream uses to find lldb-server to fail, as it relies on obtaining the library path from the dynamic loader and finding executables relative to it. I have replaced it with hardcoded path to make LLDB work.

The patches are currently waiting for Joerg Sonnenberger to finish LLVM/Clang update that's in progress already.

Pending tasks

The exact list of pending tasks from my contract follows:

1. Add support to backtrace through signal trampoline and extend the support to libexecinfo, unwind implementations (LLVM, nongnu). Examine adding CFI support to interfaces that need it to provide more stable backtraces (both kernel and userland).

2. Add support for aarch64 target.

3. Stabilize LLDB and address breaking tests from the test suite.

Notes on backtracing through signal trampoline

I have described the problem of backtracing through signal trampoline in February's report. I haven't managed to finish the work on the topic within the contract but I will try to work on it in my free time.

Most likely, the solution would involve modifying the assembly in lib/libc/arch/*/sys/__sigtramp2.S. As suggested by Andrew Cagney, the CFI directives for amd64 would look like:

NENTRY(__sigtramp_siginfo_2)
    .cfi_startproc
    .cfi_signal_frame
    .cfi_def_cfa r15, 0
    /* offsets from mcontext_t */
    .cfi_offset rax, 0x70
    .cfi_offset rbx, 0x68
    .cfi_offset rcx, 0x18
    .cfi_offset rdx, 0x10
    /* ... */
    .cfi_def_cfa rsp, 8
    movq    %r15,%rdi
    movq    $SYS_setcontext, %rax
    syscall
    movq    $-1,%rdi /* if we return here, something is wrong */
    movq    $SYS_exit, %rax
    syscall
    .cfi_endproc
END(__sigtramp_siginfo_2)

Addressing breaking tests

While the most important functions of LLDB work on NetBSD, there are still many test failures. At this moment, there are 80 instances of @expectedFailureNetBSD decorator and 18 cases of @skipIfNetBSD. The former generally indicates that the test reliably fails on NetBSD and needs a fix, the latter is sometimes used to decorate tests specific to other systems but also to indicate that the test crashes, hangs or otherwise can not be reliably run.

Some tests are failing due to the concurrent signal kernel bug explained in the previous post and covered by XFAIL-ing ATF tests.

New regressions both in LLDB and in LLVM in general appear every month. Most of them are fixed by their authors once we report them. I will continue fighting new bugs in my free time and trying to keep the build bot green.

This work is sponsored by The NetBSD Foundation

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can:

https://netbsd.org/donations/#how-to-donate

Back in 2018, Phil Nelson started a long needed WiFi-refresh, basically syncing our src/sys/net80211/ with the version from FreeBSD. He got a few things working, but then ran out of time and was unable to spend enough time on this to complete the task. I am now taking over this work, Phil hopes to join in again later this summer.

The main idea is to get better SMP (locking) support, support for newer (faster) WiFi standards, and support for virtual access points, while also making future updates (and import of drivers) from FreeBSD easier.

I have collected quite a few WiFi cards and USB dongles, but will ask for help with other hardware I don't have handy later.

I hope to have a working setup and a few working drivers by the end of this months.

Thanks to the NetBSD Foundation for funding this work! Please note that we are looking for donations again, see Fundraising 2020.

P.S.: for the curious: the first drivers I hope to have working are the ones we have already drivers for and I have hardware:

• urtwn(4)
• run(4)
• athn(4)
• ath(4)
• wi(4)

followed by things other developers can convert and have hardware (with me assisting where needed), optionally followed by some of the ones where I have hardware but we have no driver yet:

• Realtek (0x2357) 802.11ac WLAN Adapter (0x011e)
• (old) pinebook sdio wifi
• cubietruck a20 wifi (broadcom 40181?)
• GuruPlug wifi: Marvell, 802.11 SDIO (manufacturer 0x2df, product 0x9103)

There's two ways user applications can communicate with the kernel audio layer in NetBSD:

• audio(4) – the native API, based on the Sun API with a number of very useful NetBSD extensions
• ossaudio(3) – a translation layer providing approximate compatibility with OSSv4's ioctls, as also supported in FreeBSD, Solaris, and popular in the past on Linux.

Linux drifted away from OSS and towards ALSA due to licensing disagreements.

Because of this drift, we're seeing increasing problems with OSS adoption today, even if the licensing concerns are no longer relevant, and other implementations of OSS have surpassed the original Linux OSSv3 implementation as far as their feature set and usability are concerned.

So, in NetBSD, it's recommended to use the native API for new code and only rely on the OSS layer for compatibility with existing code.

I spent a while working on third-party software to improve support for native NetBSD audio. These included Firefox, SDL, PortAudio, ffmpeg (working with yhardy@), and more.

However, I've turned my attention to the OSS translation layer. Since a lot of older and less popular software still relies on it, I wanted to go over the OSSv4 specification and iron out surprising differences.

Audacity/PortAudio's OSS usage is strange

I should note that most of these fixes were to enable Audacity to work without patching. Audacity is interesting because it hits a lot of edge cases as far as OSS API usage is concerned. Once I fixed the most notable issues, I made sure Audacity also supported the native API. Writing the necessary PortAudio glue for Sun/NetBSD audio and implementing these fixes took approximately two days.

Incompatibility 1 – SNDCTL_DSP_SPEED

[Out of range sample rates are now handled properly by the OSS layer in NetBSD-current.]

The NetBSD 9 kernel supports sample rates up to 192kHz. Specify anything higher, and NetBSD's audio API returns an error code and keeps the sample rate at its original value, or the legacy default of 8000 Hz (not particularly useful with modern devices).

However, OSS applications expected setting the sample rate to always succeed. The specification states that the actual set sample value may be an approximation and will not always use the exact requested value. So, if the requested value is out of range, NetBSD will now return as if the call succeeded, and set the sample rate to the current configured hardware rate (usually some multiple of 48kHz).

During its startup process, Audacity requested an overly high sample rate of 384kHz. This is well above the maximum supported. I'm still not sure why it does this, because it later configures the audio device to standard CD rate, but it meant that Audacity couldn't properly start without our local patches.

Incompatibility 2 – SNDCTL_DSP_CHANNELS

[Out of range channel numbers are now handled properly by the OSS layer in NetBSD-current.]

This was a very simple fix, similar to that of SNDCTL_DSP_SPEED. The NetBSD kernel supports between 1 and 12 channels for audio playback. Most commonly 1 is mono, 2 is stereo, and higher numbers are used with surround sound systems. The limit of 12 comes from the USB audio device specification.

If an out of range number is specified, libossaudio will now set the channel count to the currently configured number in use at the hardware level.

However, we encounter a more difficult difference between OSS and NetBSD audio when using the audio device in full duplex (recording from and playing back to the same device simultaneously). If your mic is mono and your speakers aren't, how do you set the channel counts to different numbers in OSS? You can't. There is one ioctl for setting both the recording and playback channels. In the native API, this is possible by setting info.record.channels and info.play.channels separately. We should ensure that the recording channels are always duplicated to be same as the number of playback channels.

Incompatibility 3 – SNDCTL_DSP_SETTRIGGER

[NetBSD-current now implements SNDCTL_DSP_SETTRIGGER.]

SNDCTL_DSP_SETTRIGGER is a somewhat more obscure part of the OSS API, in that it's only really useful if you are using poll() or another event notification mechanism on an audio device before performing any I/O, or you're performing I/O via mmap(), neither being particularly common in practice. It has the ability to force initialisation of playback/recording for the device if this isn't already the case.

In terms of the native API, this means that playback/recording becomes unpaused.

Previously in NetBSD, this part of the OSS API wasn't implemented and simply did nothing. However, it became obviously needed due to an incompatible change in NetBSD 9, as discussed on tech-kern.

Basically, we needed recording to be properly triggered without a read() so a few applications using poll() without prior I/O wouldn't block indefinitely.

Incompatibility 4 – SNDCTL_DSP_SETPLAYVOL

OSSv4 has special bits to manipulate the volume of an individual stream in an application while doing all the maths for this inside the kernel.

We don't support this properly yet (but reasonably could)... so code needs to be modified to do the volume manipulation in the application, or the OSSv4 support disabled.

I've only found a couple of applications that try to use this feature (audacious, qmmp). Currently, they're configured to avoid using OSSv4 and layer the audio through SDL or qt-multimedia instead.

I've at least fixed SNDCTL_DSP_GETPLAYVOL to return conforming values. NetBSD audio uses 0-255 for the gain of all channels. OSSv4 uses a range of 0-100 and encodes two channels into an integer, which is very odd in my opinion, and also limits surround sound support.

The future of libossaudio in NetBSD?

Hopefully, after my changes, OSS compatibility is in a much better shape when dealing with unusual parameters and uncommon API usage. The quality of the code also improves – in the process of this work, maxv@ pointed me towards a related information leak in the Linux OSSv3 compatibility layer in the kernel, and I was able to deal with it properly after looking at the OSS specification and Linux headers. All the fixes should be pulled up to 9-stable.

However, I'd personally like to eventually reach a point where we no longer need libossaudio. I've been writing a lot of code towards this goal.

In many cases, the applications relying on it could be easily modified or told to use libao/SDL2/PortAudio/OpenAL/etc instead, which all have native NetBSD audio support.

OSS aside...

We probably need to start thinking about supporting 24-bit PCM in the kernel, since I've found a few audio players that can't handle making the samples 32-bit before writing them to the device. The Sun audio implementation in Solaris has supported this for a long time now.