What has been done in NetBSD
I've managed to achieve the following accomplishments:
Introduction of PT_SETSTEP and PT_CLEARSTEP
This allows to:
- singlestep particular threads,
- combine PT_STEP with PT_SYSCALL,
- combine PT_STEP and emission of a signal.
There are equivalent operations in FreeBSD with the same names.
Introduction of helper macro PTRACE_BREAKPOINT_ASM
This code was prepared by Nick Hudson and it was used in ATF tests to verify behavior of software breakpoints.
Addition of new sysctl(2) functions
Add new defines in sysctl(2) on amd64 and i386 ports. These values are defined in <x86/cpu.h>:
- CPU_FPU_SAVE (15)
int: FPU Instructions layout * to use this, CPU_OSFXSR must be true * 0: FSAVE * 1: FXSAVE * 2: XSAVE * 3: XSAVEOPT
- CPU_FPU_SAVE_SIZE (16)
int: FPU Instruction layout size
- CPU_XSAVE_FEATURES (17)
quad: FPU XSAVE features
- Bump CPU_MAXID from 15 to 18.
These values are useful to get FPU (floating point unit) properties in e.g. a debugger. This information is required to properly implement FPR (floating point register) tracer operations on x86 processors.
Corrections in ptrace(2) man-page
Few mistakes were corrected to make the documentation more correct.
ATF tests cleanup in ptrace(2)
There were added new tests for new ptrace(2) operations (PT_SETSTEP and PT_CLEARSTEP).
Also several tests were updated to reflect the current state of "successfully passed" and "expected failure". This is important to mark issues that are already known and quickly catch new regressions in future changes.
F_GETPATH in fcntl(2)
It was decided that NetBSD will not introduce new fcntl(2) function for compatibility with certain other systems. This means that once LLDB will require this feature, we will need to introduce a workaround in the project.
What has been done in LLDB
The NetBSD Process Plugin in LLDB acquired new capabilities. Additionally enhancements in LLDB were developed such as handling threads in core(5) files.
Floating point support
The x86_64 architecture supports in default properties FXSAVE processor instructions. The FXSAVE feature allows to operate over floating point registers. A thread state (context) is composed of (and not restricted to) general and floating point registers.
The NetBSD Process Plugin acquired the functionality to read these registers and optionally set new values for them.
Watchpoint support
A programmer can use hardware assisted watchpoints to stop execution of a tracee whenever a certain variable or instruction was read/written/executed. The support for this feature has been implemented on NetBSD with ptrace(2) operations PT_SETDBREGS and PT_GETDBREGS. These operations are now available in the LLDB Process plugin.
Threads support in core(5) files
I've included support for LWPs in core(5) files. This means that larger programs with threads, like Firefox that emitted coredump for some reason (usually during crash) can be investigated postmortem.
Demo
I've prepared a recording with the script(1) utility from the NetBSD base system. To replay it:
script -p ./firefox-core.typescript
This recording shows a debugging session of a Firefox core(5) file.
(I was kind to prepare a Linux version of the NetBSD script(1) here).
Plan for the next milestone
The plan for the next milestone is continuing development of threads in the NetBSD Process Plugin. I will need to work more on correctness of ptrace(2) calls as new issues were detected in setups with threads that resulted in crashes.
There is also ongoing work on a new build node running NetBSD-current (prerelease of 8) and building LLVM+Clang+LLDB. I'm working on enabling unit tests to catch functional regressions quickly. The original LLDB node cluster was privately funded by myself in the last two years and has been switched to a machine hosted by The NetBSD Foundation.
To keep this machine up and running (8 CPU, 24 GB RAM) community support through donations is required. This is crucial to actively maintain the LLVM toolchain (Clang, LLDB and others) on NetBSD.
This work was sponsored by The NetBSD Foundation.
The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:
What has been done in NetBSD
I've managed to achieve the following accomplishments:
Introduction of PT_SETSTEP and PT_CLEARSTEP
This allows to:
- singlestep particular threads,
- combine PT_STEP with PT_SYSCALL,
- combine PT_STEP and emission of a signal.
There are equivalent operations in FreeBSD with the same names.
Introduction of helper macro PTRACE_BREAKPOINT_ASM
This code was prepared by Nick Hudson and it was used in ATF tests to verify behavior of software breakpoints.
Addition of new sysctl(2) functions
Add new defines in sysctl(2) on amd64 and i386 ports. These values are defined in <x86/cpu.h>:
- CPU_FPU_SAVE (15)
int: FPU Instructions layout * to use this, CPU_OSFXSR must be true * 0: FSAVE * 1: FXSAVE * 2: XSAVE * 3: XSAVEOPT
- CPU_FPU_SAVE_SIZE (16)
int: FPU Instruction layout size
- CPU_XSAVE_FEATURES (17)
quad: FPU XSAVE features
- Bump CPU_MAXID from 15 to 18.
These values are useful to get FPU (floating point unit) properties in e.g. a debugger. This information is required to properly implement FPR (floating point register) tracer operations on x86 processors.
Corrections in ptrace(2) man-page
Few mistakes were corrected to make the documentation more correct.
ATF tests cleanup in ptrace(2)
There were added new tests for new ptrace(2) operations (PT_SETSTEP and PT_CLEARSTEP).
Also several tests were updated to reflect the current state of "successfully passed" and "expected failure". This is important to mark issues that are already known and quickly catch new regressions in future changes.
F_GETPATH in fcntl(2)
It was decided that NetBSD will not introduce new fcntl(2) function for compatibility with certain other systems. This means that once LLDB will require this feature, we will need to introduce a workaround in the project.
What has been done in LLDB
The NetBSD Process Plugin in LLDB acquired new capabilities. Additionally enhancements in LLDB were developed such as handling threads in core(5) files.
Floating point support
The x86_64 architecture supports in default properties FXSAVE processor instructions. The FXSAVE feature allows to operate over floating point registers. A thread state (context) is composed of (and not restricted to) general and floating point registers.
The NetBSD Process Plugin acquired the functionality to read these registers and optionally set new values for them.
Watchpoint support
A programmer can use hardware assisted watchpoints to stop execution of a tracee whenever a certain variable or instruction was read/written/executed. The support for this feature has been implemented on NetBSD with ptrace(2) operations PT_SETDBREGS and PT_GETDBREGS. These operations are now available in the LLDB Process plugin.
Threads support in core(5) files
I've included support for LWPs in core(5) files. This means that larger programs with threads, like Firefox that emitted coredump for some reason (usually during crash) can be investigated postmortem.
Demo
I've prepared a recording with the script(1) utility from the NetBSD base system. To replay it:
script -p ./firefox-core.typescript
This recording shows a debugging session of a Firefox core(5) file.
(I was kind to prepare a Linux version of the NetBSD script(1) here).
Plan for the next milestone
The plan for the next milestone is continuing development of threads in the NetBSD Process Plugin. I will need to work more on correctness of ptrace(2) calls as new issues were detected in setups with threads that resulted in crashes.
There is also ongoing work on a new build node running NetBSD-current (prerelease of 8) and building LLVM+Clang+LLDB. I'm working on enabling unit tests to catch functional regressions quickly. The original LLDB node cluster was privately funded by myself in the last two years and has been switched to a machine hosted by The NetBSD Foundation.
To keep this machine up and running (8 CPU, 24 GB RAM) community support through donations is required. This is crucial to actively maintain the LLVM toolchain (Clang, LLDB and others) on NetBSD.
This work was sponsored by The NetBSD Foundation.
The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:
Coming soon we have a new set of kernel synchronization routines - localcount(9) - which provide a medium-weight reference-counting mechanism. From the manual page, "During normal operations, localcounts do not need the interprocessor synchronization associated with atomic_ops(3) atomic memory operations, and (unlike psref(9)) localcount references can be held across sleeps and can migrate between CPUs. Draining a localcount requires more expensive interprocessor synchronization than atomic_ops(3) (similar to psref(9)). And localcount references require eight bytes of memory per object per-CPU, significantly more than atomic_ops(3) and almost always more than psref(9)."
We'll be adding localcount(9) reference counting to the device driver cdevsw and bdevsw structures, to ensure that a (modular) device driver cannot be removed while it is active. Modular drivers with initializers for these structures need to be modified to initialize their localcount members, using the DEVSW_MODULE_INIT macro (this change is mandatory for all loadable drivers). To take advantage of the reference counting, the drivers also need to replace all calls to bdevsw_lookup() and cdevsw_lookup() with bdevsw_lookup_acquire() and cdevsw_lookup_acquire() respectively, and then release the reference using bdevsw_release() and cdevsw_release().
We'll also be using localcount(9) to provide reference-counting of individual device units, to prevent a unit from being destroyed while it is active. To implement device unit reference-counting, all calls to device_lookup(), device_find_by_driver_unit(), and device_lookup_private() need to be replaced by their corresponding *_acquire() variant; when the caller is finished using the device, it must release the reference using device_release().
More details and examples can be seen by examining the prg-localcount2 branch in cvs including the new localcount(9) manual page!
Coming soon we have a new set of kernel synchronization routines - localcount(9) - which provide a medium-weight reference-counting mechanism. From the manual page, "During normal operations, localcounts do not need the interprocessor synchronization associated with atomic_ops(3) atomic memory operations, and (unlike psref(9)) localcount references can be held across sleeps and can migrate between CPUs. Draining a localcount requires more expensive interprocessor synchronization than atomic_ops(3) (similar to psref(9)). And localcount references require eight bytes of memory per object per-CPU, significantly more than atomic_ops(3) and almost always more than psref(9)."
We'll be adding localcount(9) reference counting to the device driver cdevsw and bdevsw structures, to ensure that a (modular) device driver cannot be removed while it is active. Modular drivers with initializers for these structures need to be modified to initialize their localcount members, using the DEVSW_MODULE_INIT macro (this change is mandatory for all loadable drivers). To take advantage of the reference counting, the drivers also need to replace all calls to bdevsw_lookup() and cdevsw_lookup() with bdevsw_lookup_acquire() and cdevsw_lookup_acquire() respectively, and then release the reference using bdevsw_release() and cdevsw_release().
We'll also be using localcount(9) to provide reference-counting of individual device units, to prevent a unit from being destroyed while it is active. To implement device unit reference-counting, all calls to device_lookup(), device_find_by_driver_unit(), and device_lookup_private() need to be replaced by their corresponding *_acquire() variant; when the caller is finished using the device, it must release the reference using device_release().
More details and examples can be seen by examining the prg-localcount2 branch in cvs including the new localcount(9) manual page!
- Leonardo Taccari will work add multi-packages support to pkgsrc.
- Maya Rashish will work on the LFS cleanup.
- Utkarsh Anand will make Anita support multiple virtual machine systems and more architectures within them to improve testing coverage.
) until August 21st, evaluations, code submission and an announcement of the results on September 6th 2017.
Good luck to all our students and their mentors - we look forward to your work results, and welcome you to The NetBSD Project!
- Leonardo Taccari will work add multi-packages support to pkgsrc.
- Maya Rashish will work on the LFS cleanup.
- Utkarsh Anand will make Anita support multiple virtual machine systems and more architectures within them to improve testing coverage.
) until August 21st, evaluations, code submission and an announcement of the results on September 6th 2017.
Good luck to all our students and their mentors - we look forward to your work results, and welcome you to The NetBSD Project!
This software has been developed for multiplatform environments with support for NetBSD since virtually forever. It's the primary tool used by the NetBSD developers and release engineering team. It is run with continuous integration tests for daily commits and execute regression tests through the Automatic Test Framework (ATF).
Since the projects keep researching and developing support for various modern trends in computing, the gap between the QEMU featureset in NetBSD and Linux diverged due to lack of active NetBSD maintenance resulted in breaking the default build.
The QEMU developers warned the Open Source community - with version 2.9 of the emulator - that they will eventually drop support for suboptimally supported hosts if nobody will step in and take the maintainership to refresh the support. This warning was directed to major BSDs, Solaris, AIX and Haiku.
Thankfully the NetBSD position has been filled - making NetBSD to restore official maintenance.
The current roadmap in QEMU/NetBSD is as follows:
- address all build failures [all patches sent to review, part of them already merged upstream],
- address all build warnings,
- restore the QEMU setup to run regression tests on NetBSD.
This effort is spare time activity - as of now without commercial support - and possible thanks to unloading the developer (myself) from more urgently pending tasks in NetBSD thanks to the contract for enhancing debuggers in business hours.
This software has been developed for multiplatform environments with support for NetBSD since virtually forever. It's the primary tool used by the NetBSD developers and release engineering team. It is run with continuous integration tests for daily commits and execute regression tests through the Automatic Test Framework (ATF).
Since the projects keep researching and developing support for various modern trends in computing, the gap between the QEMU featureset in NetBSD and Linux diverged due to lack of active NetBSD maintenance resulted in breaking the default build.
The QEMU developers warned the Open Source community - with version 2.9 of the emulator - that they will eventually drop support for suboptimally supported hosts if nobody will step in and take the maintainership to refresh the support. This warning was directed to major BSDs, Solaris, AIX and Haiku.
Thankfully the NetBSD position has been filled - making NetBSD to restore official maintenance.
The current roadmap in QEMU/NetBSD is as follows:
- address all build failures [all patches sent to review, part of them already merged upstream],
- address all build warnings,
- restore the QEMU setup to run regression tests on NetBSD.
This effort is spare time activity - as of now without commercial support - and possible thanks to unloading the developer (myself) from more urgently pending tasks in NetBSD thanks to the contract for enhancing debuggers in business hours.