strace–seccomp-bpf: alookunderthehood
35
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion strace –seccomp-bpf: a look under the hood Paul Chaignon February 2, 2020 FOSDEM 2020, January 2, 2019
Transcript of strace–seccomp-bpf: alookunderthehood
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace –seccomp-bpf: a look under the hoodPaul Chaignon
February 2, 2020
FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
Overview of this talk
How strace uses ptrace(2) to stop your process
How strace uses seccomp-bpf to stop only at syscalls of interest
How syscalls are matched in the kernel with 2 cBPF algos
1/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace’s default behavior
2/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace’s default behavior: strace ./tracee
user mode+--------+| strace |+--------+
------------------------------------------kernel mode
+----------+| ./tracee |+----------+
+----->-----PTRACE_SYSCALL---->----+| |
+--------------------+ +-------------------+| syscall-entry-stop | | syscall-exit-stop |+--------------------+ +-------------------+
| |+-----<-----PTRACE_SYSCALL----<----+
3/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace’s default behavior: strace ./tracee
user mode+----------+ +--------+| ./tracee | | strace |+----------+ +--------+
------------------------------------------kernel mode
+----->-----PTRACE_SYSCALL---->----+| |
+--------------------+ +-------------------+| syscall-entry-stop | | syscall-exit-stop |+--------------------+ +-------------------+
| |+-----<-----PTRACE_SYSCALL----<----+
4/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace’s default behavior: strace ./tracee
user mode+--------+| strace |+--------+
------------------------------------------kernel mode
+----------+| ./tracee |+----------+
+----->-----PTRACE_SYSCALL---->----+| |
+--------------------+ +-------------------+| syscall-entry-stop | | syscall-exit-stop |+--------------------+ +-------------------+
| |+-----<-----PTRACE_SYSCALL----<----+
5/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace’s default behavior: strace ./tracee
user mode+--------+| strace |+--------+
------------------------------------------kernel mode
+----------+| ./tracee |+----------+
+----->-----PTRACE_SYSCALL---->----+| |
+--------------------+ +-------------------+| syscall-entry-stop | | syscall-exit-stop |+--------------------+ +-------------------+
| |+-----<-----PTRACE_SYSCALL----<----+
6/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace’s default behavior: strace ./tracee
user mode+--------+| strace |+--------+
------------------------------------------kernel mode
+----------+| ./tracee |+----------+
+----->-----PTRACE_SYSCALL---->----+| |
+--------------------+ +-------------------+| syscall-entry-stop | | syscall-exit-stop |+--------------------+ +-------------------+
| |+-----<-----PTRACE_SYSCALL----<----+
7/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
What’s the issue?
8/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace -e trace=
strace -e trace=seccomp => see seccomp(2) syscalls only
strace -eseccomp => same
strace -e%network => see all network-related syscalls
9/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
Unnecessary overhead
Stops twice per syscall, at all syscalls
Even if we want to see a single syscall!
Stops require 2 context switches
Very expensive!
$ cd linux/$ time make -j$(nproc) > /dev/null[...]real 12m27,010s$ make clean$ time strace -f -econnect make -j$(nproc) > /dev/null# | |# | +----> Display connect(2) syscalls# +----> Trace child processes[...]real 24m54,473s
10/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
We need a way to tell the kernel at which syscalls to stop
11/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
Introducing seccomp-bpf
12/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
Introducing seccomp-bpf
Let’s use seccomp!
Seccomp as a syscall-filtering mechanism
seccomp-bpf to choose syscalls to filter
Used in Chrome’s sandbox
Second user of BPF in Linux after socket filters (e.g., tcpdump)
cBPF, not eBPF!
13/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
seccomp-bpf examples
Allow process to open(2) and openat(2) only
Kill it if it tries anything else
ld [4] /* load seccomp_data->arch */jne #0xc000003e, bad /* is AUDIT_ARCH_X86_64? */ld [0] /* load seccomp_data->nr */jeq #257, good /* is openat(2)? */jeq #2, good /* is open(2)? */bad: ret #0 /* return RET_KILL_THREAD */good: ret #0x7fff0000 /* return RET_ALLOW */
14/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
seccomp-bpf examples
Allow process to open specific files only
Need help from userspace
ld [4] /* load seccomp_data->arch */jne #0xc000003e, bad /* is AUDIT_ARCH_X86_64? */ld [0] /* load seccomp_data->nr */jeq #257, good /* is openat(2)? */jeq #2, good /* is open(2)? */bad: ret #0 /* return RET_KILL_THREAD */good: ret #0x7ff00000 /* return RET_TRACE */
15/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace --seccomp-bpf
16/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
--seccomp-bpf’s behavioruser mode
+--------+| strace |+--------+
------------------------------------------kernel mode
+----------+| ./tracee |+----------+
+------->-------PTRACE_SYSCALL------->------+| |
+--------------+ || seccomp-stop |----<----+ |+--------------+ | |
| |+--------------------+ | +------------------+| syscall-entry-stop | +-<-PTRACE_CONT-<--| syscall-exit-stop |+--------------------+ +------------------+
17/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
--seccomp-bpf’s behavioruser mode
+----------+ +--------+| ./tracee | | strace |+----------+ +--------+
------------------------------------------kernel mode
+------->-------PTRACE_SYSCALL------->------+| |
+--------------+ || seccomp-stop |----<----+ |+--------------+ | |
| |+--------------------+ | +------------------+| syscall-entry-stop | +-<-PTRACE_CONT-<--| syscall-exit-stop |+--------------------+ +------------------+
18/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
--seccomp-bpf’s behavioruser mode
+--------+| strace |+--------+
------------------------------------------kernel mode
+----------+| ./tracee |+----------+
+------->-------PTRACE_SYSCALL------->------+| |
+--------------+ || seccomp-stop |----<----+ |+--------------+ | |
| |+--------------------+ | +------------------+| syscall-entry-stop | +-<-PTRACE_CONT-<--| syscall-exit-stop |+--------------------+ +------------------+
19/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
--seccomp-bpf’s behavioruser mode
+----------+ +--------+| ./tracee | | strace |+----------+ +--------+
------------------------------------------kernel mode
+------->-------PTRACE_SYSCALL------->------+| |
+--------------+ || seccomp-stop |----<----+ |+--------------+ | |
| |+--------------------+ | +------------------+| syscall-entry-stop | +-<-PTRACE_CONT-<--| syscall-exit-stop |+--------------------+ +------------------+
20/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
--seccomp-bpf’s behavioruser mode
+--------+| strace |+--------+
------------------------------------------kernel mode
+----------+| ./tracee |+----------+
+------->-------PTRACE_SYSCALL------->------+| |
+--------------+ || seccomp-stop |----<----+ |+--------------+ | |
| |+--------------------+ | +------------------+| syscall-entry-stop | +-<-PTRACE_CONT-<--| syscall-exit-stop |+--------------------+ +------------------+
21/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
--seccomp-bpf’s behavioruser mode
+--------+| strace |+--------+
------------------------------------------kernel mode
+----------+| ./tracee |+----------+
+------->-------PTRACE_SYSCALL------->------+| |
+--------------+ || seccomp-stop |----<----+ |+--------------+ | |
| |+--------------------+ | +------------------+| syscall-entry-stop | +-<-PTRACE_CONT-<--| syscall-exit-stop |+--------------------+ +------------------+
22/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
--seccomp-bpf’s behavior
Before Linux 4.8, seccomp-stop happens before syscall-entry-stop
We need to restart with PTRACE_SYSCALL twice :(
+------->-------PTRACE_SYSCALL---->----+| |
+--------------------+ || syscall-entry-stop | |+--------------------+ |
| PTRACE_SYSCALL |+--------------+ +------------------+| seccomp-stop |--<--PTRACE_CONT--<--| syscall-exit-stop |+--------------+ +------------------+
23/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
cBPF algorithms: linear
[...]ld [0] /* load seccomp_data->nr */jeq #0, trace /* is read(2)? */jeq #1, trace /* is write(2)? */jeq #2, trace /* is open(2)? */jeq #3, trace /* is close(2)? */jeq #4, trace /* is stat(2)? */jeq #5, trace /* is fstat(2)? */[...]skip: ret #0x7fff0000 /* return RET_ALLOW */trace: ret #0x7ff00000 /* return RET_TRACE */
24/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
cBPF algorithms: linear
[...]ld [0] /* load seccomp_data->nr */jlt #0, skip /* is < read(2)? */jle #5, trace /* is <= fstat(2)? */[...]skip: ret #0x7fff0000 /* return RET_ALLOW */trace: ret #0x7ff00000 /* return RET_TRACE */
25/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
cBPF algorithms: binary match
cBPF has 32-bit bitwise operations
Encode all syscalls of interest as 32-bit bit arrays
select(2) >--------------+ioctl(2) >--------+ |
| |00000000 00000000 10000001 00000000
cBPF doesn’t have indirect jumps => no jump tables
switch over bit arrays implemented as linear search
26/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
cBPF algorithms
none
ptrace
!ptrace
%desc %fi
le%fstat
%ipc
!%ipc
%lstat
custom
100
200
Num
bero
finstructio
ns
linear binary match
custom = %memory,%ipc,%pure,%signal,%network
27/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
Limitations
28/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
--seccomp-bpf implies -f
strace -f to trace children processes
--seccomp-bpf implies -f
In kernel, children inherit seccomp filter chain of parent
Kernel doesn’t copy filters but keeps a reference count
To inherit only some filters in the chain, we need to make copies :(
29/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
strace –seccomp-bpf -p [pid]
strace -p [pid] to trace an existing process
But no way to attach seccomp-bpf filters to existing processes
30/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
Conclusion
31/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
To sum up
strace stops at all syscalls by default (expensive!)
strace --seccomp-bpf -e... to stop only at syscalls of interest
Uses 2 seccomp-bpf algorithms
32/34 FOSDEM 2020, January 2, 2019
strace’s default behavior What’s the issue? Introducing seccomp-bpf strace --seccomp-bpf Limitations Conclusion
Future work
socketcall(2) and ipc(2)’s subcalls not supported!
cBPF program would have to match on first syscall argument
strace -c to print summary of traced syscalls
Perfect use case for eBPF!
Statistics can be aggregated in the kernel and summary only sent tostrace
33/34 FOSDEM 2020, January 2, 2019
Thanks!
![strace: new features · Systemcallspecificationimprovements [3/6] %fstatsyscallclass fstat,fstat64 fstatat64,newfstatat oldfstat strace-y-e%fstatfind/var/empty-mindepth1](https://static.fdocuments.us/doc/165x107/5ad995eb7f8b9a6d7e8b8f13/strace-new-features-cationimprovements-36-fstatsyscallclass-fstatfstat64.jpg)
