- One in the base-system with a stack of local patches.
- One in pkgsrc with mostly build fix patches.
The base-system version of GDB (GPLv3) still relies on a set of local patches. I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.
GDB changes
Over the past month I worked on gdbserver for NetBSD/amd64 and finally upstreamed it to the GDB mainline, just in time for GDB 10.
What is gdbserver? Let's quote the official GDB documentation:
gdbserver is a control program for Unix-like systems, which allows you to connect your program with a remote GDB via target remote or target extended-but without linking in the usual debugging stub.
gdbserver is not a complete replacement for the debugging stubs, because it requires essentially the same operating-system facilities that GDB itself does. In fact, a system that can run gdbserver to connect to a remote GDB could also run GDB locally! gdbserver is sometimes useful nevertheless, because it is a much smaller program than GDB itself. It is also easier to port than all of GDB, so you may be able to get started more quickly on a new system by using gdbserver. Finally, if you develop code for real-time systems, you may find that the tradeoffs involved in real-time operation make it more convenient to do as much development work as possible on another system, for example by cross-compiling. You can use gdbserver to make a similar choice for debugging.
GDB and gdbserver communicate via either a serial line or a TCP connection, using the standard GDB remote serial protocol. remote
This illustrated that gdbserver is especially useful for debugging applications on embedded and thin devices, connected to a controlling computer equipped with full distribution sources, toolchain, debugging information etc. Eventually, this approach of gdb and gdbserver can replace the native gdb plugin entirely and spawn all connections debugging sessions using this protocol. This design decision was already introduced into LLDB, where remote process plugin is the only supported program on Linux, NetBSD and highly recommended for other kernels.
I've picked amd64 as the first target as it's the easiest to develop and test.
An example debugging session looks like this:
$ uname -rms NetBSD 9.99.72 amd64 $ LC_ALL=C date Thu Sep 10 22:43:10 CEST 2020 $ ./gdbserver/gdbserver --version GNU gdbserver (GDB) 10.0.50.20200910-git Copyright (C) 2020 Free Software Foundation, Inc. gdbserver is free software, covered by the GNU General Public License. This gdbserver was configured as "x86_64-unknown-netbsd9.99" $ ./gdbserver/gdbserver localhost:1234 /usr/bin/nslookup Process /usr/bin/nslookup created; pid = 26383 Listening on port 1234
Then on the other terminal:
$ ./gdb/gdb
GNU gdb (GDB) 10.0.50.20200910-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-unknown-netbsd9.99".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
--Type for more, q to quit, c to continue without paging--
.
For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
Reading /usr/bin/nslookup from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /usr/bin/nslookup from remote target...
Reading symbols from target:/usr/bin/nslookup...
Reading /usr/bin/nslookup.debug from remote target...
Reading /usr/bin/.debug/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/bin/nslookup.debug...
process 28353 is executing new program: /usr/bin/nslookup
Reading /usr/bin/nslookup from remote target...
Reading /usr/bin/nslookup from remote target...
Reading /usr/bin/nslookup.debug from remote target...
Reading /usr/bin/.debug/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libexec/ld.elf_so from remote target...
Reading /usr/libexec/ld.elf_so from remote target...
Reading /usr/libexec/ld.elf_so.debug from remote target...
Reading /usr/libexec/.debug/ld.elf_so.debug from remote target...
Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote target...
Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote target...
warning: Invalid remote reply: timeout [kamil: repeated multiple times...]
Reading /usr/lib/libbind9.so.15 from remote target...
Reading /usr/lib/libisccfg.so.15 from remote target...
Reading /usr/lib/libdns.so.15 from remote target...
Reading /usr/lib/libns.so.15 from remote target...
Reading /usr/lib/libirs.so.15 from remote target...
Reading /usr/lib/libisccc.so.15 from remote target...
Reading /usr/lib/libisc.so.15 from remote target...
Reading /usr/lib/libkvm.so.6 from remote target...
Reading /usr/lib/libz.so.1 from remote target...
Reading /usr/lib/libblocklist.so.0 from remote target...
Reading /usr/lib/libpthread.so.1 from remote target...
Reading /usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/lib/.debug/libpthread.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/lib/libgssapi.so.11 from remote target...
Reading /usr/lib/libheimntlm.so.5 from remote target...
Reading /usr/lib/libkrb5.so.27 from remote target...
Reading /usr/lib/libcom_err.so.8 from remote target...
Reading /usr/lib/libhx509.so.6 from remote target...
Reading /usr/lib/libcrypto.so.14 from remote target...
Reading /usr/lib/libasn1.so.10 from remote target...
Reading /usr/lib/libwind.so.1 from remote target...
Reading /usr/lib/libheimbase.so.2 from remote target...
Reading /usr/lib/libroken.so.20 from remote target...
Reading /usr/lib/libsqlite3.so.1 from remote target...
Reading /usr/lib/libcrypt.so.1 from remote target...
Reading /usr/lib/libutil.so.7 from remote target...
Reading /usr/lib/libedit.so.3 from remote target...
Reading /usr/lib/libterminfo.so.2 from remote target...
Reading /usr/lib/libc.so.12 from remote target...
Reading /usr/lib/libgcc_s.so.1 from remote target...
Reading symbols from target:/usr/lib/libbind9.so.15...
Reading /usr/lib/libbind9.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libbind9.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libbind9.so.15.0.debug...
Reading symbols from target:/usr/lib/libisccfg.so.15...
Reading /usr/lib/libisccfg.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisccfg.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote target...
--Type for more, q to quit, c to continue without paging--
Reading symbols from target:/usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug...
Reading symbols from target:/usr/lib/libdns.so.15...
Reading /usr/lib/libdns.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libdns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libdns.so.15.0.debug...
Reading symbols from target:/usr/lib/libns.so.15...
Reading /usr/lib/libns.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libns.so.15.0.debug...
Reading symbols from target:/usr/lib/libirs.so.15...
Reading /usr/lib/libirs.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libirs.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libirs.so.15.0.debug...
Reading symbols from target:/usr/lib/libisccc.so.15...
Reading /usr/lib/libisccc.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisccc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libisccc.so.15.0.debug...
Reading symbols from target:/usr/lib/libisc.so.15...
Reading /usr/lib/libisc.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libisc.so.15.0.debug...
Reading symbols from target:/usr/lib/libkvm.so.6...
Reading /usr/lib/libkvm.so.6.0.debug from remote target...
Reading /usr/lib/.debug/libkvm.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libkvm.so.6.0.debug...
Reading symbols from target:/usr/lib/libz.so.1...
Reading /usr/lib/libz.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libz.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libz.so.1.0.debug...
Reading symbols from target:/usr/lib/libblocklist.so.0...
Reading /usr/lib/libblocklist.so.0.0.debug from remote target...
Reading /usr/lib/.debug/libblocklist.so.0.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug...
Reading symbols from target:/usr/lib/libgssapi.so.11...
Reading /usr/lib/libgssapi.so.11.0.debug from remote target...
Reading /usr/lib/.debug/libgssapi.so.11.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug...
Reading symbols from target:/usr/lib/libheimntlm.so.5...
Reading /usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading /usr/lib/.debug/libheimntlm.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug...
Reading symbols from target:/usr/lib/libkrb5.so.27...
Reading /usr/lib/libkrb5.so.27.0.debug from remote target...
Reading /usr/lib/.debug/libkrb5.so.27.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug...
Reading symbols from target:/usr/lib/libcom_err.so.8...
Reading /usr/lib/libcom_err.so.8.0.debug from remote target...
Reading /usr/lib/.debug/libcom_err.so.8.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug...
Reading symbols from target:/usr/lib/libhx509.so.6...
Reading /usr/lib/libhx509.so.6.0.debug from remote target...
Reading /usr/lib/.debug/libhx509.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libhx509.so.6.0.debug...
Reading symbols from target:/usr/lib/libcrypto.so.14...
Reading /usr/lib/libcrypto.so.14.0.debug from remote target...
Reading /usr/lib/.debug/libcrypto.so.14.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug...
Reading symbols from target:/usr/lib/libasn1.so.10...
Reading /usr/lib/libasn1.so.10.0.debug from remote target...
Reading /usr/lib/.debug/libasn1.so.10.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libasn1.so.10.0.debug...
Reading symbols from target:/usr/lib/libwind.so.1...
Reading /usr/lib/libwind.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libwind.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libwind.so.1.0.debug...
Reading symbols from target:/usr/lib/libheimbase.so.2...
Reading /usr/lib/libheimbase.so.2.0.debug from remote target...
Reading /usr/lib/.debug/libheimbase.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug...
Reading symbols from target:/usr/lib/libroken.so.20...
Reading /usr/lib/libroken.so.20.0.debug from remote target...
Reading /usr/lib/.debug/libroken.so.20.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libroken.so.20.0.debug...
Reading symbols from target:/usr/lib/libsqlite3.so.1...
Reading /usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading /usr/lib/.debug/libsqlite3.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug...
Reading symbols from target:/usr/lib/libcrypt.so.1...
Reading /usr/lib/libcrypt.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libcrypt.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug...
Reading symbols from target:/usr/lib/libutil.so.7...
Reading /usr/lib/libutil.so.7.24.debug from remote target...
Reading /usr/lib/.debug/libutil.so.7.24.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libutil.so.7.24.debug...
Reading symbols from target:/usr/lib/libedit.so.3...
Reading /usr/lib/libedit.so.3.1.debug from remote target...
Reading /usr/lib/.debug/libedit.so.3.1.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libedit.so.3.1.debug...
Reading symbols from target:/usr/lib/libterminfo.so.2...
Reading /usr/lib/libterminfo.so.2.0.debug from remote target...
Reading /usr/lib/.debug/libterminfo.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug...
Reading symbols from target:/usr/lib/libc.so.12...
Reading /usr/lib/libc.so.12.217.debug from remote target...
Reading /usr/lib/.debug/libc.so.12.217.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libc.so.12.217.debug...
Reading symbols from target:/usr/lib/libgcc_s.so.1...
Reading /usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libgcc_s.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug...
Reading /usr/libexec/ld.elf_so from remote target...
_rtld_debug_state () at /usr/src/libexec/ld.elf_so/rtld.c:1577
1577 __insn_barrier();
(gdb) b main
Breakpoint 1 at 0x211c00: file /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c, line 990.
(gdb) c
Continuing.
Breakpoint 1, main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
990 main(int argc, char **argv) {
(gdb) bt
#0 main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
(gdb) b pthread_setname_np
Breakpoint 2 at 0x7f7ff4e0c9e4: file /usr/src/lib/libpthread/pthread.c, line 792.
(gdb) c
Continuing.
[New Thread 28353.27773]
Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e41000,
name=name@entry=0x7f7fffffe610 "work-0", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
792 {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 pthread_setname_np (thread=0x7f7ff7e41000,
name=name@entry=0x7f7fffffe610 "work-0", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
2 Thread 28353.27773 0x00007f7ff0aa623a in ___lwp_park60 () from target:/usr/lib/libc.so.12
(gdb) n
796 pthread__error(EINVAL, "Invalid thread",
(gdb) n
799 if (pthread__find(thread) != 0)
(gdb)
802 namelen = snprintf(newname, sizeof(newname), name, arg);
(gdb)
803 if (namelen >= PTHREAD_MAX_NAMELEN_NP)
(gdb)
806 cp = strdup(newname);
(gdb)
807 if (cp == NULL)
(gdb)
810 pthread_mutex_lock(&thread->pt_lock);
(gdb)
811 oldname = thread->pt_name;
(gdb)
812 thread->pt_name = cp;
(gdb)
813 (void)_lwp_setname(thread->pt_lid, cp);
(gdb)
814 pthread_mutex_unlock(&thread->pt_lock);
(gdb) n
816 if (oldname != NULL)
(gdb) n
isc_taskmgr_create (mctx=, workers=workers@entry=1, default_quantum=,
default_quantum@entry=0, nm=nm@entry=0x0, managerp=managerp@entry=0x418638 )
at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1431
1431 for (i = 0; i < workers; i++) {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 isc_taskmgr_create (mctx=, workers=workers@entry=1,
default_quantum=, default_quantum@entry=0, nm=nm@entry=0x0,
managerp=managerp@entry=0x418638 )
at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1431
2 Thread 28353.27773 "work-0" 0x00007f7ff0aa623a in ___lwp_park60 ()
from target:/usr/lib/libc.so.12
(gdb) dis 1
(gdb) b exit
Breakpoint 3 at 0x7f7ff0b530e0: exit. (2 locations)
(gdb) c
Continuing.
Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e42c00,
name=name@entry=0x7f7ff5e6324e "isc-timer", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
792 {
(gdb) dis 2
(gdb) c
Continuing.
Reading /usr/lib/i18n/libUTF8.so.5.0 from remote target...
Reading /usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
Reading /usr/lib/i18n/.debug/libUTF8.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
Then, back to the first terminal:
> netbsd.org Server: 62.179.1.62 Address: 62.179.1.62#53 Non-authoritative answer: Name: netbsd.org Address: 199.233.217.205 Name: netbsd.org Address: 2001:470:a085:999::80 > exit
Thread 1 hit Breakpoint 3, exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
55 {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
(gdb) bt
#0 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
#1 0x0000000000206122 in ___start ()
#2 0x00007f7ff7c0c840 in ?? () from target:/usr/libexec/ld.elf_so
#3 0x0000000000000001 in ?? ()
#4 0x00007f7fffffed20 in ?? ()
#5 0x0000000000000000 in ?? ()
(gdb) kill
Kill the program being debugged? (y or n) y
[Inferior 1 (process 28353) killed]
It worked!
In order to get this functionality operational I had to implement multiple GDB functions, in particular: create_inferior, post_create_inferior, attach, kill, detach, mourn, join, thread_alive, resume, wait, fetch_registers, store_registers, read_memory, write_memory, request_interrupt, supports_read_auxv, read_auxv, supports_hardware_single_step, sw_breakpoint_from_kind, supports_z_point_type, insert_point, remove_point, stopped_by_sw_breakpoint, supports_qxfer_siginfo, qxfer_siginfo, supports_stopped_by_sw_breakpoint, supports_non_stop, supports_multi_process, supports_fork_events, supports_vfork_events, supports_exec_events, supports_disable_randomization, supports_qxfer_libraries_svr4, qxfer_libraries_svr4, supports_pid_to_exec_file, pid_to_exec_file, thread_name, supports_catch_syscall.
NetBSD is the first BSD and actually the first Open Source UNIX-like OS besides Linux to grow support for gdbserver.
Plan for the next milestone
Introduce AArch64 support for GDB/NetBSD.
- One in the base-system with a stack of local patches.
- One in pkgsrc with mostly build fix patches.
The base-system version of GDB (GPLv3) still relies on a set of local patches. I set a goal to reduce the local patches to bare minimum, ideally reaching no local modifications at all.
GDB changes
Over the past month I worked on gdbserver for NetBSD/amd64 and finally upstreamed it to the GDB mainline, just in time for GDB 10.
What is gdbserver? Let's quote the official GDB documentation:
gdbserver is a control program for Unix-like systems, which allows you to connect your program with a remote GDB via target remote or target extended-but without linking in the usual debugging stub.
gdbserver is not a complete replacement for the debugging stubs, because it requires essentially the same operating-system facilities that GDB itself does. In fact, a system that can run gdbserver to connect to a remote GDB could also run GDB locally! gdbserver is sometimes useful nevertheless, because it is a much smaller program than GDB itself. It is also easier to port than all of GDB, so you may be able to get started more quickly on a new system by using gdbserver. Finally, if you develop code for real-time systems, you may find that the tradeoffs involved in real-time operation make it more convenient to do as much development work as possible on another system, for example by cross-compiling. You can use gdbserver to make a similar choice for debugging.
GDB and gdbserver communicate via either a serial line or a TCP connection, using the standard GDB remote serial protocol. remote
This illustrated that gdbserver is especially useful for debugging applications on embedded and thin devices, connected to a controlling computer equipped with full distribution sources, toolchain, debugging information etc. Eventually, this approach of gdb and gdbserver can replace the native gdb plugin entirely and spawn all connections debugging sessions using this protocol. This design decision was already introduced into LLDB, where remote process plugin is the only supported program on Linux, NetBSD and highly recommended for other kernels.
I've picked amd64 as the first target as it's the easiest to develop and test.
An example debugging session looks like this:
$ uname -rms NetBSD 9.99.72 amd64 $ LC_ALL=C date Thu Sep 10 22:43:10 CEST 2020 $ ./gdbserver/gdbserver --version GNU gdbserver (GDB) 10.0.50.20200910-git Copyright (C) 2020 Free Software Foundation, Inc. gdbserver is free software, covered by the GNU General Public License. This gdbserver was configured as "x86_64-unknown-netbsd9.99" $ ./gdbserver/gdbserver localhost:1234 /usr/bin/nslookup Process /usr/bin/nslookup created; pid = 26383 Listening on port 1234
Then on the other terminal:
$ ./gdb/gdb
GNU gdb (GDB) 10.0.50.20200910-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-unknown-netbsd9.99".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
--Type for more, q to quit, c to continue without paging--
.
For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
Reading /usr/bin/nslookup from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /usr/bin/nslookup from remote target...
Reading symbols from target:/usr/bin/nslookup...
Reading /usr/bin/nslookup.debug from remote target...
Reading /usr/bin/.debug/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/bin/nslookup.debug...
process 28353 is executing new program: /usr/bin/nslookup
Reading /usr/bin/nslookup from remote target...
Reading /usr/bin/nslookup from remote target...
Reading /usr/bin/nslookup.debug from remote target...
Reading /usr/bin/.debug/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target...
Reading /usr/libexec/ld.elf_so from remote target...
Reading /usr/libexec/ld.elf_so from remote target...
Reading /usr/libexec/ld.elf_so.debug from remote target...
Reading /usr/libexec/.debug/ld.elf_so.debug from remote target...
Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote target...
Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote target...
warning: Invalid remote reply: timeout [kamil: repeated multiple times...]
Reading /usr/lib/libbind9.so.15 from remote target...
Reading /usr/lib/libisccfg.so.15 from remote target...
Reading /usr/lib/libdns.so.15 from remote target...
Reading /usr/lib/libns.so.15 from remote target...
Reading /usr/lib/libirs.so.15 from remote target...
Reading /usr/lib/libisccc.so.15 from remote target...
Reading /usr/lib/libisc.so.15 from remote target...
Reading /usr/lib/libkvm.so.6 from remote target...
Reading /usr/lib/libz.so.1 from remote target...
Reading /usr/lib/libblocklist.so.0 from remote target...
Reading /usr/lib/libpthread.so.1 from remote target...
Reading /usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/lib/.debug/libpthread.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote target...
Reading /usr/lib/libgssapi.so.11 from remote target...
Reading /usr/lib/libheimntlm.so.5 from remote target...
Reading /usr/lib/libkrb5.so.27 from remote target...
Reading /usr/lib/libcom_err.so.8 from remote target...
Reading /usr/lib/libhx509.so.6 from remote target...
Reading /usr/lib/libcrypto.so.14 from remote target...
Reading /usr/lib/libasn1.so.10 from remote target...
Reading /usr/lib/libwind.so.1 from remote target...
Reading /usr/lib/libheimbase.so.2 from remote target...
Reading /usr/lib/libroken.so.20 from remote target...
Reading /usr/lib/libsqlite3.so.1 from remote target...
Reading /usr/lib/libcrypt.so.1 from remote target...
Reading /usr/lib/libutil.so.7 from remote target...
Reading /usr/lib/libedit.so.3 from remote target...
Reading /usr/lib/libterminfo.so.2 from remote target...
Reading /usr/lib/libc.so.12 from remote target...
Reading /usr/lib/libgcc_s.so.1 from remote target...
Reading symbols from target:/usr/lib/libbind9.so.15...
Reading /usr/lib/libbind9.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libbind9.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libbind9.so.15.0.debug...
Reading symbols from target:/usr/lib/libisccfg.so.15...
Reading /usr/lib/libisccfg.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisccfg.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote target...
--Type for more, q to quit, c to continue without paging--
Reading symbols from target:/usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug...
Reading symbols from target:/usr/lib/libdns.so.15...
Reading /usr/lib/libdns.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libdns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libdns.so.15.0.debug...
Reading symbols from target:/usr/lib/libns.so.15...
Reading /usr/lib/libns.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libns.so.15.0.debug...
Reading symbols from target:/usr/lib/libirs.so.15...
Reading /usr/lib/libirs.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libirs.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libirs.so.15.0.debug...
Reading symbols from target:/usr/lib/libisccc.so.15...
Reading /usr/lib/libisccc.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisccc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libisccc.so.15.0.debug...
Reading symbols from target:/usr/lib/libisc.so.15...
Reading /usr/lib/libisc.so.15.0.debug from remote target...
Reading /usr/lib/.debug/libisc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libisc.so.15.0.debug...
Reading symbols from target:/usr/lib/libkvm.so.6...
Reading /usr/lib/libkvm.so.6.0.debug from remote target...
Reading /usr/lib/.debug/libkvm.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libkvm.so.6.0.debug...
Reading symbols from target:/usr/lib/libz.so.1...
Reading /usr/lib/libz.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libz.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libz.so.1.0.debug...
Reading symbols from target:/usr/lib/libblocklist.so.0...
Reading /usr/lib/libblocklist.so.0.0.debug from remote target...
Reading /usr/lib/.debug/libblocklist.so.0.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug...
Reading symbols from target:/usr/lib/libgssapi.so.11...
Reading /usr/lib/libgssapi.so.11.0.debug from remote target...
Reading /usr/lib/.debug/libgssapi.so.11.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug...
Reading symbols from target:/usr/lib/libheimntlm.so.5...
Reading /usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading /usr/lib/.debug/libheimntlm.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug...
Reading symbols from target:/usr/lib/libkrb5.so.27...
Reading /usr/lib/libkrb5.so.27.0.debug from remote target...
Reading /usr/lib/.debug/libkrb5.so.27.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug...
Reading symbols from target:/usr/lib/libcom_err.so.8...
Reading /usr/lib/libcom_err.so.8.0.debug from remote target...
Reading /usr/lib/.debug/libcom_err.so.8.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug...
Reading symbols from target:/usr/lib/libhx509.so.6...
Reading /usr/lib/libhx509.so.6.0.debug from remote target...
Reading /usr/lib/.debug/libhx509.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libhx509.so.6.0.debug...
Reading symbols from target:/usr/lib/libcrypto.so.14...
Reading /usr/lib/libcrypto.so.14.0.debug from remote target...
Reading /usr/lib/.debug/libcrypto.so.14.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug...
Reading symbols from target:/usr/lib/libasn1.so.10...
Reading /usr/lib/libasn1.so.10.0.debug from remote target...
Reading /usr/lib/.debug/libasn1.so.10.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libasn1.so.10.0.debug...
Reading symbols from target:/usr/lib/libwind.so.1...
Reading /usr/lib/libwind.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libwind.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libwind.so.1.0.debug...
Reading symbols from target:/usr/lib/libheimbase.so.2...
Reading /usr/lib/libheimbase.so.2.0.debug from remote target...
Reading /usr/lib/.debug/libheimbase.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug...
Reading symbols from target:/usr/lib/libroken.so.20...
Reading /usr/lib/libroken.so.20.0.debug from remote target...
Reading /usr/lib/.debug/libroken.so.20.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libroken.so.20.0.debug...
Reading symbols from target:/usr/lib/libsqlite3.so.1...
Reading /usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading /usr/lib/.debug/libsqlite3.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug...
Reading symbols from target:/usr/lib/libcrypt.so.1...
Reading /usr/lib/libcrypt.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libcrypt.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug...
Reading symbols from target:/usr/lib/libutil.so.7...
Reading /usr/lib/libutil.so.7.24.debug from remote target...
Reading /usr/lib/.debug/libutil.so.7.24.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libutil.so.7.24.debug...
Reading symbols from target:/usr/lib/libedit.so.3...
Reading /usr/lib/libedit.so.3.1.debug from remote target...
Reading /usr/lib/.debug/libedit.so.3.1.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libedit.so.3.1.debug...
Reading symbols from target:/usr/lib/libterminfo.so.2...
Reading /usr/lib/libterminfo.so.2.0.debug from remote target...
Reading /usr/lib/.debug/libterminfo.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug...
Reading symbols from target:/usr/lib/libc.so.12...
Reading /usr/lib/libc.so.12.217.debug from remote target...
Reading /usr/lib/.debug/libc.so.12.217.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libc.so.12.217.debug...
Reading symbols from target:/usr/lib/libgcc_s.so.1...
Reading /usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading /usr/lib/.debug/libgcc_s.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote target...
Reading symbols from target:/usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug...
Reading /usr/libexec/ld.elf_so from remote target...
_rtld_debug_state () at /usr/src/libexec/ld.elf_so/rtld.c:1577
1577 __insn_barrier();
(gdb) b main
Breakpoint 1 at 0x211c00: file /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c, line 990.
(gdb) c
Continuing.
Breakpoint 1, main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
990 main(int argc, char **argv) {
(gdb) bt
#0 main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 main (argc=1, argv=0x7f7fffffe768)
at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslookup.c:990
(gdb) b pthread_setname_np
Breakpoint 2 at 0x7f7ff4e0c9e4: file /usr/src/lib/libpthread/pthread.c, line 792.
(gdb) c
Continuing.
[New Thread 28353.27773]
Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e41000,
name=name@entry=0x7f7fffffe610 "work-0", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
792 {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 pthread_setname_np (thread=0x7f7ff7e41000,
name=name@entry=0x7f7fffffe610 "work-0", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
2 Thread 28353.27773 0x00007f7ff0aa623a in ___lwp_park60 () from target:/usr/lib/libc.so.12
(gdb) n
796 pthread__error(EINVAL, "Invalid thread",
(gdb) n
799 if (pthread__find(thread) != 0)
(gdb)
802 namelen = snprintf(newname, sizeof(newname), name, arg);
(gdb)
803 if (namelen >= PTHREAD_MAX_NAMELEN_NP)
(gdb)
806 cp = strdup(newname);
(gdb)
807 if (cp == NULL)
(gdb)
810 pthread_mutex_lock(&thread->pt_lock);
(gdb)
811 oldname = thread->pt_name;
(gdb)
812 thread->pt_name = cp;
(gdb)
813 (void)_lwp_setname(thread->pt_lid, cp);
(gdb)
814 pthread_mutex_unlock(&thread->pt_lock);
(gdb) n
816 if (oldname != NULL)
(gdb) n
isc_taskmgr_create (mctx=, workers=workers@entry=1, default_quantum=,
default_quantum@entry=0, nm=nm@entry=0x0, managerp=managerp@entry=0x418638 )
at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1431
1431 for (i = 0; i < workers; i++) {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 isc_taskmgr_create (mctx=, workers=workers@entry=1,
default_quantum=, default_quantum@entry=0, nm=nm@entry=0x0,
managerp=managerp@entry=0x418638 )
at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1431
2 Thread 28353.27773 "work-0" 0x00007f7ff0aa623a in ___lwp_park60 ()
from target:/usr/lib/libc.so.12
(gdb) dis 1
(gdb) b exit
Breakpoint 3 at 0x7f7ff0b530e0: exit. (2 locations)
(gdb) c
Continuing.
Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e42c00,
name=name@entry=0x7f7ff5e6324e "isc-timer", arg=arg@entry=0x0)
at /usr/src/lib/libpthread/pthread.c:792
792 {
(gdb) dis 2
(gdb) c
Continuing.
Reading /usr/lib/i18n/libUTF8.so.5.0 from remote target...
Reading /usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
Reading /usr/lib/i18n/.debug/libUTF8.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remote target...
Then, back to the first terminal:
> netbsd.org Server: 62.179.1.62 Address: 62.179.1.62#53 Non-authoritative answer: Name: netbsd.org Address: 199.233.217.205 Name: netbsd.org Address: 2001:470:a085:999::80 > exit
Thread 1 hit Breakpoint 3, exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
55 {
(gdb) info threads
Id Target Id Frame
* 1 Thread 28353.28353 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
(gdb) bt
#0 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55
#1 0x0000000000206122 in ___start ()
#2 0x00007f7ff7c0c840 in ?? () from target:/usr/libexec/ld.elf_so
#3 0x0000000000000001 in ?? ()
#4 0x00007f7fffffed20 in ?? ()
#5 0x0000000000000000 in ?? ()
(gdb) kill
Kill the program being debugged? (y or n) y
[Inferior 1 (process 28353) killed]
It worked!
In order to get this functionality operational I had to implement multiple GDB functions, in particular: create_inferior, post_create_inferior, attach, kill, detach, mourn, join, thread_alive, resume, wait, fetch_registers, store_registers, read_memory, write_memory, request_interrupt, supports_read_auxv, read_auxv, supports_hardware_single_step, sw_breakpoint_from_kind, supports_z_point_type, insert_point, remove_point, stopped_by_sw_breakpoint, supports_qxfer_siginfo, qxfer_siginfo, supports_stopped_by_sw_breakpoint, supports_non_stop, supports_multi_process, supports_fork_events, supports_vfork_events, supports_exec_events, supports_disable_randomization, supports_qxfer_libraries_svr4, qxfer_libraries_svr4, supports_pid_to_exec_file, pid_to_exec_file, thread_name, supports_catch_syscall.
NetBSD is the first BSD and actually the first Open Source UNIX-like OS besides Linux to grow support for gdbserver.
Plan for the next milestone
Introduce AArch64 support for GDB/NetBSD.
This report was written by Apurva Nandan as part of Google Summer of Code 2020.
Introduction
This blog post is in continuation of GSoC Reports: Benchmarking NetBSD, first evaluation report and GSoC Reports: Benchmarking NetBSD, second evaluation report blogs, and describes my progress in the final phase of GSoC 2020 under The NetBSD Foundation.
In the third phase, I upgraded to the latest stable version Phoronix Test Suite (PTS) 9.8.0 in pkgsrc-wip, resolved the TODOs and created patches for more test-profiles to fix their installation and runtime errors on NetBSD-current.
Progress in the third phase of GSoC
wip/phoronix-test-suite TODO and update
As a newer stable version of the Phoronix Test Suite was available in upstream, I upgraded the Phoronix Test Suite from version 9.6.1 to 9.8.0 in pkgsrc-wip and is available as wip/phoronix-test-suite. You can have a look at the PTS Changelog to know about the improvements between these two versions.
To get the package ready for merge in pkgsrc upstream, I also resolved the pkgsrc-wip TODOs.
pkgsrc-wip commits:
If any new problems are encountered, please document them in
wip/phoronix-test-suite/TODO file and/or contact me.
Testing of automated benchmarking framework
I had been assigned a remote testing machine having Intel 6138 dual processor, 40 cores, 80 threads, 192GB of RAM. I spent time reproducing my automated framework i.e., Phoromatic-Anita Integration on the machine. I was able to reproduce the integration framework working without networking configuration, but the network bridge needs to be setup on the remote machine and the integration script to be tested with it. I shall continue this task in the post-GSoC period.
Benchmarking Results
I also performed benchmarking of NetBSD-9 amd64 native installation by running 50 test-profiles on a remote machine assigned to me by mentors and uploaded the benchmark results to OpenBenchmarking.org at:
Test-profile debugging
I then continued the task of maintaining/porting test-profiles and fixed the following test-profiles:
Timed FFmpeg Compilation
This test times how long it takes to build FFmpeg.
This test is part of Processor Test category.
Original Test-profile:
https://openbenchmarking.org/test/pts/build-ffmpeg
Patched Test-profile:
https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/build-ffmpeg-1.0.1
Commit:
Compile Bench
Compilebench tries to age a filesystem by simulating some of the disk
IO common in creating, compiling, patching, stating and reading kernel
trees. It indirectly measures how well filesystems can maintain
directory locality as the disk fills up and directories age.
This test is part of Disk Test category.
Original Test-profile:
https://openbenchmarking.org/test/pts/compilebench
Patched Test-profile:
https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/compilebench-1.0.2
Commit:
Timed MAFFT Alignment
This test performs an alignment of 100 pyruvate decarboxylase sequences.
This test is part of Processor Test category.
Original Test-profile:
https://openbenchmarking.org/test/pts/mafft
Patched Test-profile:
https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/mafft-1.5.0
Commits:
- Replaced the make -> gmake for compatibility with NetBSD
- Patch for replacing /bin/bash interpreter with /usr/pkg/bin/bash
Future Plans
This officially summarizes my GSoC project: Benchmark NetBSD, and my end goal of the project that is to integrate Phoronix Test Suite with NetBSD and Anita for automated benchmarking is complete and its deployment on benchmark.NetBSD.org will be continued to be worked on with the coordination of moderators and merging the wip of Phoronix Test Suite 9.8.0 will be done by the pkgsrc maintainers in next days.
I want to thank my mentors and the NetBSD community without whose constant support I wouldn't have achieved the goals.
This report was written by Apurva Nandan as part of Google Summer of Code 2020.
Introduction
This blog post is in continuation of GSoC Reports: Benchmarking NetBSD, first evaluation report and GSoC Reports: Benchmarking NetBSD, second evaluation report blogs, and describes my progress in the final phase of GSoC 2020 under The NetBSD Foundation.
In the third phase, I upgraded to the latest stable version Phoronix Test Suite (PTS) 9.8.0 in pkgsrc-wip, resolved the TODOs and created patches for more test-profiles to fix their installation and runtime errors on NetBSD-current.
Progress in the third phase of GSoC
wip/phoronix-test-suite TODO and update
As a newer stable version of the Phoronix Test Suite was available in upstream, I upgraded the Phoronix Test Suite from version 9.6.1 to 9.8.0 in pkgsrc-wip and is available as wip/phoronix-test-suite. You can have a look at the PTS Changelog to know about the improvements between these two versions.
To get the package ready for merge in pkgsrc upstream, I also resolved the pkgsrc-wip TODOs.
pkgsrc-wip commits:
If any new problems are encountered, please document them in
wip/phoronix-test-suite/TODO file and/or contact me.
Testing of automated benchmarking framework
I had been assigned a remote testing machine having Intel 6138 dual processor, 40 cores, 80 threads, 192GB of RAM. I spent time reproducing my automated framework i.e., Phoromatic-Anita Integration on the machine. I was able to reproduce the integration framework working without networking configuration, but the network bridge needs to be setup on the remote machine and the integration script to be tested with it. I shall continue this task in the post-GSoC period.
Benchmarking Results
I also performed benchmarking of NetBSD-9 amd64 native installation by running 50 test-profiles on a remote machine assigned to me by mentors and uploaded the benchmark results to OpenBenchmarking.org at:
Test-profile debugging
I then continued the task of maintaining/porting test-profiles and fixed the following test-profiles:
Timed FFmpeg Compilation
This test times how long it takes to build FFmpeg.
This test is part of Processor Test category.
Original Test-profile:
https://openbenchmarking.org/test/pts/build-ffmpeg
Patched Test-profile:
https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/build-ffmpeg-1.0.1
Commit:
Compile Bench
Compilebench tries to age a filesystem by simulating some of the disk
IO common in creating, compiling, patching, stating and reading kernel
trees. It indirectly measures how well filesystems can maintain
directory locality as the disk fills up and directories age.
This test is part of Disk Test category.
Original Test-profile:
https://openbenchmarking.org/test/pts/compilebench
Patched Test-profile:
https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/compilebench-1.0.2
Commit:
Timed MAFFT Alignment
This test performs an alignment of 100 pyruvate decarboxylase sequences.
This test is part of Processor Test category.
Original Test-profile:
https://openbenchmarking.org/test/pts/mafft
Patched Test-profile:
https://github.com/apurvanandan1997/pts-test-profiles-dev/tree/master/mafft-1.5.0
Commits:
- Replaced the make -> gmake for compatibility with NetBSD
- Patch for replacing /bin/bash interpreter with /usr/pkg/bin/bash
Future Plans
This officially summarizes my GSoC project: Benchmark NetBSD, and my end goal of the project that is to integrate Phoronix Test Suite with NetBSD and Anita for automated benchmarking is complete and its deployment on benchmark.NetBSD.org will be continued to be worked on with the coordination of moderators and merging the wip of Phoronix Test Suite 9.8.0 will be done by the pkgsrc maintainers in next days.
I want to thank my mentors and the NetBSD community without whose constant support I wouldn't have achieved the goals.
My GSoC project under NetBSD involves the development of the test framework of curses. This is the final blog report in a series of blog reports; you can look at the first report and second report of the series.
The first report gives a brief introduction of the project and some insights into the curses testframe through its architecture and language. To someone who wants to contribute to the test suite, this blog can act as the quick guide of how things work internally. Meanwhile, the second report discusses some of the concepts that were quite challenging for me to understand. I wanted to share them with those who may face such a challenge. Both of these reports also cover the progress made in various phases of the Summer of Code.
This being the final report in the series, I would love to share my experience throughout the project. I would be sharing some of the learning as well as caveats that I faced in the project.
Challenges and Caveats
By the time my application for GSoC was submitted, I had gained some knowledge about the curses library and the testing framework. Combined with compiler design and library testing experience, that knowledge proved useful but not sufficient as I progressed through the project. There were times when, while writing a test case, you have to look into documentation from various sources, be it NetBSD, FreeBSD, Linux, Solaris, etc. One may find questioning his understanding of the framework, documentation, or even curses itself. This leads to the conclusion that for being a tester, one has to become a user first. That made me write minimal programs to understand the behavior. The experience was excellent, and I felt amazed by the capability and complexity of curses.
Learnings
The foremost learning is from the experience of interacting with the open-source community and feeling confident in my abilities to contribute. Understanding the workflows; following the best practices like considering the maintainability, readability, and simplicity of the code were significant learning.
The project-specific learning was not limited to test-framework but a deeper understanding of curses as I have to browse through codes for the functions tested. As this blog says, getting the TTY demystified was a long-time desire, which got fulfilled to some extent.
Some tests from test suite
In this section, I would discuss a couple of tests of the test suite written during the third phase of GSoC. Curses input model provides a variety of ways to obtain input from keyboard. We will consider 2 tests keypad and halfdelay that belong to input processing category but are somewhat unrelated.
Keypad Processing
An application can enable or disable the tarnslation of keypad using keypad() function. When translation is enabled, curses attempts to translate input sequence into a single key code. If disabled, curses passes the input as it is and any interpretation has to be made by application.
include window
call $FALSE is_keypad $win1
input "\eOA"
call 0x1b wgetch $win1
call OK keypad $win1 $TRUE
input "\eOA"
call $KEY_UP wgetch $win1
# disable assembly of KEY_UP
call OK keyok $KEY_UP $FALSE
input "\eOA"
call 0x1b wgetch $win1
As keypad translation is disabled by default, on input of '\eOA', the input sequence is passed as it is and only '\e' (0x1b is hex code) is received on wgetch(). If we enable the translation, then the same input is translated as KEY_UP. In curses, one can disable assembly of specific key symbols using keyok(). See related man page.
Input Mode
Curses lets the application control the effect of input using four input modes; cooked, cbreak, half-delay, raw. They specify the effect of input in terms of echo-ing and delay. We will discuss about the halfdelay mode. The half-delay mode specifies how quickly certain curses function return to application when there is no terminal input waiting since the function is called.
include start
delay 1000
# input delay 1000 equals to 10 tenths of seconds
# getch must fail for halfdelay(5) and pass for halfdelay(15)
input "a"
call OK halfdelay 15
call 0x61 getch
call OK halfdelay 5
input "a"
call -1 getch
We have set the delay for feeding input to terminal with delay of 1s(10 tenths of second). If the application sets the halfdelay to 15, and makes a call to getch() it receives the input. But it fails to get the input with haldelay set to 5. See related man page.
Project Work
The work can be merged into organisation repository https://github.com/NetBSD/src under tests/lib/libcurses.
This project involved:
- Improvement in testframework:
- Automation of the checkfile generation.
- Enhnacement of support for complex character
- Addition of small features and code refactoring
- Testing and bug reports:
- Tests for a family of routines like wide character, complex character, line drawing, box drawing, pad, window operations, cursor manipulations, soft label keys, input-output stream, and the ones involving their interactions.
- Raising a bunch of Problem Report (PR) under
libcategory some of which have been fixed. The list of PRs raised can be found here
Future Work
- The current testframe supports complex character, but the support needs to be extended for its string. This will enable testing of
[mv][w]add_wch[n]str,[mv][w]in_wchstrfamily of routines. - Some of the tests for teminal manipulation routines like
intrflush,def_prog_mode,typeahead,raw, etc. are not there in test suite. - Not specifically related to the framework, but the documentation for wide character as well as complex character routines need to be added.
Acknowledgements
I want to extend my heartfelt gratitude to my mentor Mr. Brett Lymn, who helped me through all the technical difficulties and challenges I faced. I also thank my mentor Martin Huseman for valuable suggestions and guidance at various junctures of the project. A special thanks to Kamil Rytarowski for making my blogs published on the NetBSD site.
My GSoC project under NetBSD involves the development of the test framework of curses. This is the final blog report in a series of blog reports; you can look at the first report and second report of the series.
The first report gives a brief introduction of the project and some insights into the curses testframe through its architecture and language. To someone who wants to contribute to the test suite, this blog can act as the quick guide of how things work internally. Meanwhile, the second report discusses some of the concepts that were quite challenging for me to understand. I wanted to share them with those who may face such a challenge. Both of these reports also cover the progress made in various phases of the Summer of Code.
This being the final report in the series, I would love to share my experience throughout the project. I would be sharing some of the learning as well as caveats that I faced in the project.
Challenges and Caveats
By the time my application for GSoC was submitted, I had gained some knowledge about the curses library and the testing framework. Combined with compiler design and library testing experience, that knowledge proved useful but not sufficient as I progressed through the project. There were times when, while writing a test case, you have to look into documentation from various sources, be it NetBSD, FreeBSD, Linux, Solaris, etc. One may find questioning his understanding of the framework, documentation, or even curses itself. This leads to the conclusion that for being a tester, one has to become a user first. That made me write minimal programs to understand the behavior. The experience was excellent, and I felt amazed by the capability and complexity of curses.
Learnings
The foremost learning is from the experience of interacting with the open-source community and feeling confident in my abilities to contribute. Understanding the workflows; following the best practices like considering the maintainability, readability, and simplicity of the code were significant learning.
The project-specific learning was not limited to test-framework but a deeper understanding of curses as I have to browse through codes for the functions tested. As this blog says, getting the TTY demystified was a long-time desire, which got fulfilled to some extent.
Some tests from test suite
In this section, I would discuss a couple of tests of the test suite written during the third phase of GSoC. Curses input model provides a variety of ways to obtain input from keyboard. We will consider 2 tests keypad and halfdelay that belong to input processing category but are somewhat unrelated.
Keypad Processing
An application can enable or disable the tarnslation of keypad using keypad() function. When translation is enabled, curses attempts to translate input sequence into a single key code. If disabled, curses passes the input as it is and any interpretation has to be made by application.
include window
call $FALSE is_keypad $win1
input "\eOA"
call 0x1b wgetch $win1
call OK keypad $win1 $TRUE
input "\eOA"
call $KEY_UP wgetch $win1
# disable assembly of KEY_UP
call OK keyok $KEY_UP $FALSE
input "\eOA"
call 0x1b wgetch $win1
As keypad translation is disabled by default, on input of '\eOA', the input sequence is passed as it is and only '\e' (0x1b is hex code) is received on wgetch(). If we enable the translation, then the same input is translated as KEY_UP. In curses, one can disable assembly of specific key symbols using keyok(). See related man page.
Input Mode
Curses lets the application control the effect of input using four input modes; cooked, cbreak, half-delay, raw. They specify the effect of input in terms of echo-ing and delay. We will discuss about the halfdelay mode. The half-delay mode specifies how quickly certain curses function return to application when there is no terminal input waiting since the function is called.
include start
delay 1000
# input delay 1000 equals to 10 tenths of seconds
# getch must fail for halfdelay(5) and pass for halfdelay(15)
input "a"
call OK halfdelay 15
call 0x61 getch
call OK halfdelay 5
input "a"
call -1 getch
We have set the delay for feeding input to terminal with delay of 1s(10 tenths of second). If the application sets the halfdelay to 15, and makes a call to getch() it receives the input. But it fails to get the input with haldelay set to 5. See related man page.
Project Work
The work can be merged into organisation repository https://github.com/NetBSD/src under tests/lib/libcurses.
This project involved:
- Improvement in testframework:
- Automation of the checkfile generation.
- Enhnacement of support for complex character
- Addition of small features and code refactoring
- Testing and bug reports:
- Tests for a family of routines like wide character, complex character, line drawing, box drawing, pad, window operations, cursor manipulations, soft label keys, input-output stream, and the ones involving their interactions.
- Raising a bunch of Problem Report (PR) under
libcategory some of which have been fixed. The list of PRs raised can be found here
Future Work
- The current testframe supports complex character, but the support needs to be extended for its string. This will enable testing of
[mv][w]add_wch[n]str,[mv][w]in_wchstrfamily of routines. - Some of the tests for teminal manipulation routines like
intrflush,def_prog_mode,typeahead,raw, etc. are not there in test suite. - Not specifically related to the framework, but the documentation for wide character as well as complex character routines need to be added.
Acknowledgements
I want to extend my heartfelt gratitude to my mentor Mr. Brett Lymn, who helped me through all the technical difficulties and challenges I faced. I also thank my mentor Martin Huseman for valuable suggestions and guidance at various junctures of the project. A special thanks to Kamil Rytarowski for making my blogs published on the NetBSD site.
This post is the third update to the project RumpKernel Syscall Fuzzing.
Part1 - https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1
Part2 - https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls
The first and second coding period was entirely dedicated to fuzzing rumpkernel syscalls using hongfuzz. Initially a dumb fuzzer was developed to start fuzzing but it soon reached its limits.
For the duration of second coding peroid we concentrated on crash reproduction and adding grammar to the fuzzer which yielded in better results as we tested on a bug in ioctl with grammar. Although this works for now crash reproduction needs to be improved to generate a working c reproducer.
For the last coding period I have looked into the internals of syzkaller to understand how it pregenerates input and how it mutates data. I have continued to work on integrating buildrump.sh with build.sh. buildrump eases the task fo building the rumpkernel on any host for any target.
buildrump.sh is like a wrapper around build.sh to build the tools and rumpkernel from the source relevant to rumpkernel. So I worked to get buildrump.sh working with netbsd-src. Building the toolchain was successfull from netbsd-src. So binaries like rumpmake work just fine to continue building the rumpkernel.
But the rumpkernel failed to build due to some warnings and errors similar to the following. It can be due to the fact that buildrump.sh has been dormant recently I faced a lot of build issues.
nbmake[2]: nbmake[2]: don't know how to make /root/buildrump.sh/obj/dest.stage/usr/lib/crti.o. Stop
nbmake[2]: stopped in /root/buildrump.sh/src/lib/librumpuser
>> ERROR:
>> make /root/buildrump.sh/obj/Makefile.first dependall
Few of the similar errors were easily fixed but I couldn't integrate it during the time span of the coding period.
To Do
- Research more on grammar definition and look into the existing grammar fuzzers for a better understanding of generating grammar.
- Integrate syz2sys with the existing fuzzer to include grammar generation for better results.
GSoC with NetBSD has been an amazing journey throughout, in which I had a chance to learn from awesome people and work on amazing projects. I will continue to work on the project to achieve the goal of integrating my fuzzer with OSS Fuzz. I thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their support and Kamil for his continuous guidance.
This post is the third update to the project RumpKernel Syscall Fuzzing.
Part1 - https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls1
Part2 - https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpkernel_syscalls
The first and second coding period was entirely dedicated to fuzzing rumpkernel syscalls using hongfuzz. Initially a dumb fuzzer was developed to start fuzzing but it soon reached its limits.
For the duration of second coding peroid we concentrated on crash reproduction and adding grammar to the fuzzer which yielded in better results as we tested on a bug in ioctl with grammar. Although this works for now crash reproduction needs to be improved to generate a working c reproducer.
For the last coding period I have looked into the internals of syzkaller to understand how it pregenerates input and how it mutates data. I have continued to work on integrating buildrump.sh with build.sh. buildrump eases the task fo building the rumpkernel on any host for any target.
buildrump.sh is like a wrapper around build.sh to build the tools and rumpkernel from the source relevant to rumpkernel. So I worked to get buildrump.sh working with netbsd-src. Building the toolchain was successfull from netbsd-src. So binaries like rumpmake work just fine to continue building the rumpkernel.
But the rumpkernel failed to build due to some warnings and errors similar to the following. It can be due to the fact that buildrump.sh has been dormant recently I faced a lot of build issues.
nbmake[2]: nbmake[2]: don't know how to make /root/buildrump.sh/obj/dest.stage/usr/lib/crti.o. Stop
nbmake[2]: stopped in /root/buildrump.sh/src/lib/librumpuser
>> ERROR:
>> make /root/buildrump.sh/obj/Makefile.first dependall
Few of the similar errors were easily fixed but I couldn't integrate it during the time span of the coding period.
To Do
- Research more on grammar definition and look into the existing grammar fuzzers for a better understanding of generating grammar.
- Integrate syz2sys with the existing fuzzer to include grammar generation for better results.
GSoC with NetBSD has been an amazing journey throughout, in which I had a chance to learn from awesome people and work on amazing projects. I will continue to work on the project to achieve the goal of integrating my fuzzer with OSS Fuzz. I thank my mentors Siddharth Muralee, Maciej Grochowski, Christos Zoulas for their support and Kamil for his continuous guidance.
For more than 20 years, NetBSD has shipped X11 with the "classic" default window manager of twm. However, it's been showing its age for a long time now.
In 2015, ctwm was imported, but after that no progress was made. ctwm is a fork of twm with some extra features - the primary advantages are that it's still incredibly lightweight, but highly configurable, and has support for virtual desktops, as well as a NetBSD-compatible license and ongoing development. Thanks to its configuration options, we can provide a default experience that's much more usable to people experienced with other operating systems.
Recently, I've been installing NetBSD with some people in real life and was inspired by their reactions to the default twm to improve the situation, so I played with ctwm, wrote a config, and used it myself for a week. It's now the default in NetBSD-current.
We gain some nice features like an auto-generated application menu (that will fill up as packages are installed to /usr/pkg), and a range of useful keyboard shortcuts including volume controls - the default config should be fully usable without a mouse. It should also work at a range of screen resolutions. We can add HiDPI support after some larger bitmap fonts are imported - another advantage of ctwm is that we can support very slow and very fast hardware with one config.
If you're curious about ctwm, check out the ctwm website. It's also included in previous NetBSD releases, though not as the default window manager and not with this config.
For more than 20 years, NetBSD has shipped X11 with the "classic" default window manager of twm. However, it's been showing its age for a long time now.
In 2015, ctwm was imported, but after that no progress was made. ctwm is a fork of twm with some extra features - the primary advantages are that it's still incredibly lightweight, but highly configurable, and has support for virtual desktops, as well as a NetBSD-compatible license and ongoing development. Thanks to its configuration options, we can provide a default experience that's much more usable to people experienced with other operating systems.
Recently, I've been installing NetBSD with some people in real life and was inspired by their reactions to the default twm to improve the situation, so I played with ctwm, wrote a config, and used it myself for a week. It's now the default in NetBSD-current.
We gain some nice features like an auto-generated application menu (that will fill up as packages are installed to /usr/pkg), and a range of useful keyboard shortcuts including volume controls - the default config should be fully usable without a mouse. It should also work at a range of screen resolutions. We can add HiDPI support after some larger bitmap fonts are imported - another advantage of ctwm is that we can support very slow and very fast hardware with one config.
If you're curious about ctwm, check out the ctwm website. It's also included in previous NetBSD releases, though not as the default window manager and not with this config.
After I posted about the new default window manager in NetBSD I got a few questions, including "when is NetBSD switching from X11 to Wayland?", Wayland being X11's "new" rival. In this blog post, hopefully I can explain why we aren't yet!
Last year (and early this year) I was responsible for porting the first working Wayland compositor to NetBSD - swc. I chose it because it looked small and hackable. You can try it out by installing the velox window manager from pkgsrc.
Difficulties
In a Wayland system, the "compositor" (display server) is responsible for managing displays, input, and window management. Generally, this means a lot of OS-specific code is contained there.
Wayland does not define protocols for features X11 users expect, like screenshots, screen locking, or window management. Either you implement these inside the compositor (lots of work that has to be redone), or you define your own protocol extension.
The Wayland "reference implementation" is a small set of libraries that can be used to build a compositor or a client application. These libraries currently have hard dependencies on Linux kernel APIs like epoll. In pkgsrc we've patched the libraries to add kqueue(2) support, but the patches haven't been accepted upstream. Wayland is written with the assumption of Linux to the extent that every client application tends to #include <linux/input.h> because Wayland's designers didn't see the need to define a OS-neutral way to get mouse button IDs.
So far, all Wayland compositors but swc have a hard dependency on libinput, which only supports Linux's input API (also cloned in FreeBSD). In NetBSD we have an entirely different input API - wscons(4). wscons is actually fairly simple to write code for, someone just needs to go out there and do it. You can use my code in swc as a reference. 
In general, Wayland is moving away from the modularity, portability, and standardization of the X server.
Is it ready for production?
No, but you can play with it.
- swc has some remaining bugs and instability.
- swc is incompatible with key applications like Firefox, but others like Luakit work, as do most things that use Qt5, GTK3, or SDL2. Not being able to run X11 applications currently is quite limiting.
- Other popular compositors are not yet available. Alternatively, someone could write some new ones.
- You need a supported GPU or SoC with kernel modesetting, since safe software fallbacks don't work here. So far, I've only tested this with Intel GPUs.
Task list
- Adding support for wscons to more Wayland compositors and persuading developers to accept the patches.
- Persuading developers not to add hard dependencies on
epolland instead use an abstraction layer like libevent. - Updating the NetBSD kernel DRM/KMS stack. This is a difficult undertaking that involves porting code from the Linux kernel (a very fast moving target).
- Getting support for newer DRM versions
- Getting support for atomic modesetting
- Getting support for Glamor X servers (for running X11 applications inside wayland, etc)
- Newer AMDGPU drivers, etc
- Adding support for basic (non-DRMKMS) framebuffers to a Wayland compositor. X11 can run from a basic unaccelerated NetBSD framebuffer, but this isn't yet possible in any Wayland compositor.
- Extending swc to add more features and fix bugs.
I've decided to take a break from this, since it's a fairly huge undertaking and uphill battle. Right now, X11 combined with a compositor like picom or xcompmgr is the more mature option.
After I posted about the new default window manager in NetBSD I got a few questions, including "when is NetBSD switching from X11 to Wayland?", Wayland being X11's "new" rival. In this blog post, hopefully I can explain why we aren't yet!
Last year (and early this year) I was responsible for porting the first working Wayland compositor to NetBSD - swc. I chose it because it looked small and hackable. You can try it out by installing the velox window manager from pkgsrc.
Difficulties
In a Wayland system, the "compositor" (display server) is responsible for managing displays, input, and window management. Generally, this means a lot of OS-specific code is contained there.
Wayland does not define protocols for features X11 users expect, like screenshots, screen locking, or window management. Either you implement these inside the compositor (lots of work that has to be redone), or you define your own protocol extension.
The Wayland "reference implementation" is a small set of libraries that can be used to build a compositor or a client application. These libraries currently have hard dependencies on Linux kernel APIs like epoll. In pkgsrc we've patched the libraries to add kqueue(2) support, but the patches haven't been accepted upstream. Wayland is written with the assumption of Linux to the extent that every client application tends to #include <linux/input.h> because Wayland's designers didn't see the need to define a OS-neutral way to get mouse button IDs.
So far, all Wayland compositors but swc have a hard dependency on libinput, which only supports Linux's input API (also cloned in FreeBSD). In NetBSD we have an entirely different input API - wscons(4). wscons is actually fairly simple to write code for, someone just needs to go out there and do it. You can use my code in swc as a reference.
In general, Wayland is moving away from the modularity, portability, and standardization of the X server.
Is it ready for production?
No, but you can play with it.
- swc has some remaining bugs and instability.
- swc is incompatible with key applications like Firefox, but others like Luakit work, as do most things that use Qt5, GTK3, or SDL2. Not being able to run X11 applications currently is quite limiting.
- Other popular compositors are not yet available. Alternatively, someone could write some new ones.
- You need a supported GPU or SoC with kernel modesetting, since safe software fallbacks don't work here. So far, I've only tested this with Intel GPUs.
Task list
- Adding support for wscons to more Wayland compositors and persuading developers to accept the patches.
- Persuading developers not to add hard dependencies on
epolland instead use an abstraction layer like libevent. - Updating the NetBSD kernel DRM/KMS stack. This is a difficult undertaking that involves porting code from the Linux kernel (a very fast moving target).
- Getting support for newer DRM versions
- Getting support for atomic modesetting
- Getting support for Glamor X servers (for running X11 applications inside wayland, etc)
- Newer AMDGPU drivers, etc
- Adding support for basic (non-DRMKMS) framebuffers to a Wayland compositor. X11 can run from a basic unaccelerated NetBSD framebuffer, but this isn't yet possible in any Wayland compositor.
- Extending swc to add more features and fix bugs.
I've decided to take a break from this, since it's a fairly huge undertaking and uphill battle. Right now, X11 combined with a compositor like picom or xcompmgr is the more mature option.