[PATCH 8/9] Add thread_set_self_state() trap

[Sergey Bugaev]

This is a new Mach trap that sets the calling thread's state to the
passed value, as if with a call to the thread_set_state() RPC.  If the
flavor of state being set is the one that contains the register used for
syscall return value (i386_THREAD_STATE or i386_REGS_SEGS_STATE on x86,
AARCH64_THREAD_STATE on AArch64), the set register value is *not*
overwritten with KERN_SUCCESS when the state gets set successfully, yet
errors do get reported if the syscall fails.

Although the trap is intended to enable userland to implement sigreturn
functionality in the AArch64 port (more on which below), the trap itself
is architecture-independent, and fully implemented in terms of the
existing kernel routines (thread_setstatus & thread_set_syscall_return).

This trap's functionality is similar to sigreturn() on Unix or
NtContinue() on NT.  The use case for these all is restoring the local
state of an interrupted thread in the following set-up:

1. A thread is running some arbitrary code.
2. An event happens that deserves the thread's immediate attention,
   analogous to a hardware interrupt request.  This might be caused by
   the thread itself (e.g. running into a Mach exception that was
   arranged to be handled by the same thread), or by external events
   (e.g. receiving a Unix SIGCHLD).
3. Another thread (or perhaps the kernel, although this is not the case
   on Mach) suspends the thread, saves its state at the point of
   interruption, alters its state to execute some sort of handler for
   the event, and resumes the thread again, now running the handler.
4. Once the thread is done running the handler, it wants to return to
   what it was doing at the time it was interrupted.  To do this, it
   needs to restore the state as saved at the moment of interruption.

Unlike with setjmp()/longjmp(), we cannot rely on the interrupted logic
collaborating in any way, as it's not aware that it's being interrupted.
This means that we have to fully restore the state, including values of
all the general-purpose registers, as well as the stack pointer, program
counter, and any state flags.

Depending on the instruction set, this may or may not be possible to do
fully in userland, simply by loading all the registers with their saved
values.  It should be more or less easy to load the saved values into
general-purpose registers, but state flags and the program counter can
be more of a challenge.  Loading the program counter value (in other
words, performing an indirect jump to the interrupted instruction) has
to be the very last thing we do, since we don't control the program flow
after that.  The only real place program counter can be loaded from is
popped off the stack, since all general-purpose registers would already
contain their restored values by that point, and using global storage is
incompatible with another interruption of the same kind happening at the
time we were about to return.  For the same reason, the saved program
counter cannot be really stored outside of the "active" stack area (such
as below the stack pointer), since otherwise it can get clobbered by
another interruption.

This means that to support fully-userland returns, the instruction set
must provide a single instruction that loads an address from the stack,
adjusts the stack pointer, and performs an indirect jump to the loaded
address.  The instruction must also either preserve (previously
restored) state flags, or additionally load state flags from the stack
in addition to the jump address.

On x86, 'ret' is such an instruction: it pops an address from the stack,
adjusting the stack pointer without modifying flags, and performs an
indirect jump to the address.  On x86_64, where the ABI mandates a red
zone, one can use the 'ret imm16' variant to additionally adjust the
stack pointer by the size of the red zone, atomically restoring the
value of the stack pointer at the time of the interruption while loading
the return address from outside the red zone.  This is how sigreturn is
implemented in glibc for the Hurd on x86.

On ARM AArch32, 'pop {pc}' (alternatively written 'ldr pc, [sp], #4') is
such an instruction: since SP and PC are just general-purpose, directly
accessible registers (r13 and r15), it is possible to perform a load
from the address pointed to by SP into PC, with a post-increment of SP.
It is, in fact, possible to restore all the other general-purpose
registers too in a single instruction this way: 'pop {r0-r12, r14, r15}'
will do that; here r13, the stack pointer, gets incremented after all
the other registers get loaded from the stack.  This also preserves the
CPSR flags, which would need to be restored just prior to the 'pop'.

On ARM AArch64 however, PC is no longer a directly accessible general-
purpose register (and SP is only accessible that way by some of the
instructions); so it is no longer possible to load PC from memory in a
single instruction.  The only way to perform an indirect jump is by
using one of the dedicated branching instructions ('br', 'blr', or
'ret').  All of them accept the address to branch to in a general-
purpose register, which is incompatible with our use case.

Moreover, with the BTI extension, there is a BTYPE field in PSTATE that
tracks which type (if any) of an indirect branch was the last executed
instruction; this is then used to raise an exception if the instruction
the indirect branch lands on was not intended to be a target of an
indirect branch (of a matching type).  It is important to restore the
BTYPE (among the other state) when returning to an interrupted context;
failing to do that will either cause an unexpected BTI failure exception
(if the last executed instruction before the interruption was not an
indirect branch, but the last instruction of the restoration logic is),
or open up a window for exploitation (if the last executed instruction
before the interruption was an indirect branch, but the last instruction
of the restoration logic is not -- note that 'ret' is not considered an
indirect branch for the purposes of BTI).

So, it is not possible to fully restore the state of an interrupted
context in userland on AArch64.  The kernel can do that however (and is
in fact doing just that every time it handles a fault or an IRQ): the
'eret' instruction for returning from an exception is accessible to EL1
(the kernel), but not EL0 (the user).  'eret' atomically restores PC
from the ELR_EL1 system register, and PSTATE from the SPSR_EL1 system
register (and does other things); both of these system registers are
inaccessible from userland, and so couldn't have been used by the
interrupted context for any purpose, meaning their values doesn't need
to be restored.  (They can be used by the kernel code, which presents an
additional complication when it's the kernel context that gets
interrupted and has to be returned to.  To make this work, the kernel
masks interrupt requests and avoids doing anything that could cause a
fault when using those registers.)

The above justifies the need for a kernel API to atomically restore
saved userland state on AArch64 (and possibly other platforms that
aren't x86).  Mach already has an API to set state of a thread, namely
the thread_set_state() RPC; however, a thread calling thread_set_state()
on itself is explicitly disallowed.  We have previously relaxed this
restriction to allow setting i386_DEBUG_STATE and i386_FSGS_BASE_STATE
on the current thread, so one way to address the need for such an API on
AArch64 would be to also allow setting AARCH64_THREAD_STATE on the
current thread.  That is what I have originally proposed and
implemented.  Like the thread_set_self_state() trap implemented by this
patch, the implementation of setting AARCH64_THREAD_STATE on the current
thread needs to ensure that the set value of the x0 register does not
get immediately overwritten with the return value of the mach_msg()
trap.

However, it's not only the return value of the mach_msg() trap that is
important, but also the RPC reply message.  The thread_set_state() RPC
should not generate a reply message when used for returning to an
interrupted context, since there'd be nobody expecting the message.
This could be achieved by special-casing that in the kernel as well, or
(simpler) by userland not passing a valid reply port in the first place.
Note that the implementation of sigreturn in glibc already uses the
strategy of passing an invalid reply port for the last RPC is does
before returning to the interrupted context (which is deallocating the
reply port used by the signal handler).

Not passing a valid reply port and consequently not blocking on awaiting
the reply message works, since the way Mach is implemented, kernel RPCs
are always executed synchronously when userland sends the request
message (unless the routine implementation includes explicit asynchrony,
as device RPCs do, and gsync_wait() should do, but currently doesn't),
meaning the RPC caller never has to *wait* for the reply message, as one
is produced immediately.  In other words, the mere act of invoking a
kernel RPC (that does not involve explicit asynchrony) is enough to
ensure it completes when mach_msg() returns, even if a reply message is
not received (whether because an invalid reply port has been specified,
or because MACH_RCV_MSG wasn't passed to mach_msg(), or because a
message other than the kernel RPC's reply was received by the call).

However, the same is not true when interposing is involved, and the
thread's self port does not in fact point directly to the kernel, but to
a userspace proxy of some sort.  The two primary examples of this are
Hurd's rpctrace tool, which interposes all the task's ports and proxies
all RPCs after tracing them, and Mach's old netmsg/netname server, which
proxies ports and messages over network.  In this case, the actual
implementation only runs once the request reaches the actual kernel, and
not once the request message has been sent by the original caller, so it
*is* necessary for the caller to await the reply message if it wants to
make sure that the requested action has been completed.  This does not
cause much issues for deallocation of a reply port on the sigreturn code
path in glibc, since that only delays when the port is deallocated, but
does not otherwise change the program behavior.  With
thread_set_state(mach_thread_self()), however, this would be quite
catastrophic, since the message-send would return back to the caller
without changing its state, and the actual change of state would only
happen at some later point.

This issue is avoided nicely by turning the functionality into an
explicit Mach trap rather than an RPC.  As it's not an RPC, it doesn't
involve messaging, and doesn't need a reply port or a reply message.  It
is always a direct call to the kernel (and not to any interposer), and
it's always guaranteed to have completed synchronously once the trap
returns.  That also means that the thread_set_self_state() call won't be
visible to rpctrace or forwarded over network for netmsg, but this is
fine, since all it does is sets thread state (i.e. register values); the
thread could do the same on its own by issuing relevant machine
instruction without involving any Mach abstractions (traps or RPCs) at
all if it weren't for the need of atomicity.

Finally, this new trap is unfortunately somewhat of a security concern
(as any sigreturn-like functionality is in general), since it would
potentially allow an attacker who already has a way to invoke a function
with 3 controlled argument values to set the values of all registers to
any desired values (sigreturn-oriented programming).  There is currently
no mitigation for this other than the generic ones such as PAC and stack
check guards.

The limit of 150 used in the implementation has been chosen to be large
enough to fit the largest thread state flavor so far, namely
AARCH64_FLOAT_STATE, but small enough to not overflow the 4K stack.  If
a new thread state flavor is added that is too big to fit on the stack,
the implementation should be switched to use kalloc instead of on-stack
storage.
---
This is the detailed explanation I have previously promised to write :)

 include/mach/syscall_sw.h |  2 ++
 kern/ipc_mig.c            | 41 +++++++++++++++++++++++++++++++++++++++
 kern/ipc_mig.h            |  5 +++++
 kern/syscall_sw.c         |  2 +-
 tests/include/syscalls.h  |  1 +
 5 files changed, 50 insertions(+), 1 deletion(-)

diff --git a/include/mach/syscall_sw.h b/include/mach/syscall_sw.h
index 89597e99..94ca6678 100644
--- a/include/mach/syscall_sw.h
+++ b/include/mach/syscall_sw.h
@@ -95,6 +95,8 @@ kernel_trap(syscall_mach_port_insert_right,-74,4)
 kernel_trap(syscall_mach_port_allocate_name,-75,3)
 kernel_trap(syscall_thread_depress_abort,-76,1)
 
+kernel_trap(thread_set_self_state,-77,3)
+
 /* These are screwing up glibc somehow.  */
 /*kernel_trap(syscall_device_writev_request,-39,6)*/
 /*kernel_trap(syscall_device_write_request,-40,6)*/
diff --git a/kern/ipc_mig.c b/kern/ipc_mig.c
index b753a25f..18ac88ef 100644
--- a/kern/ipc_mig.c
+++ b/kern/ipc_mig.c
@@ -55,6 +55,7 @@
 #include <device/dev_hdr.h>
 #include <device/device_types.h>
 #include <device/ds_routines.h>
+#include <machine/pcb.h>
 
 /*
  *     Routine:        mach_msg_send_from_kernel
@@ -878,6 +879,46 @@ kern_return_t 
syscall_thread_depress_abort(mach_port_name_t thread)
        return result;
 }
 
+/*
+ *     Routine:        thread_set_self_state [mach trap]
+ *     Purpose:
+ *             Set current thread's state, as if with
+ *             thread_set_state() RPC.
+ *
+ *             When setting the generic state (one that contains
+ *             the register used for syscall return value) succeeds,
+ *             the successful return does not overwire the just-set
+ *             register value.
+ *     Conditions:
+ *             Nothing locked.
+ *     Returns:
+ *             Nothing at all          Successfully set generic state.
+ *             KERN_SUCCESS            Successfully set other state.
+ *             KERN_INVALID_ARGUMENT   Invalid flavor or state.
+ */
+kern_return_t thread_set_self_state(
+       int             flavor,
+       thread_state_t  new_state,
+       natural_t       new_state_count)
+{
+       thread_t        t = current_thread();
+       kern_return_t   kr;
+       natural_t       new_state_copy[150];
+
+       if (new_state_count <= 0 || new_state_count > 150)
+               return KERN_INVALID_ARGUMENT;
+
+       if (copyin(new_state, new_state_copy, new_state_count * 
sizeof(natural_t)))
+               return KERN_INVALID_ARGUMENT;
+
+       thread_set_syscall_return(t, KERN_SUCCESS);
+       kr = thread_setstatus(t, flavor, new_state_copy, new_state_count);
+       if (kr == KERN_SUCCESS)
+               thread_exception_return();
+
+       return kr;
+}
+
 /*
  * Device traps -- these are way experimental.
  */
diff --git a/kern/ipc_mig.h b/kern/ipc_mig.h
index 422e8d84..f5d8b07a 100644
--- a/kern/ipc_mig.h
+++ b/kern/ipc_mig.h
@@ -124,6 +124,11 @@ extern kern_return_t syscall_mach_port_allocate_name(
 
 extern kern_return_t syscall_thread_depress_abort(mach_port_name_t thread);
 
+extern kern_return_t thread_set_self_state(
+       int             flavor,
+       thread_state_t  new_state,
+       natural_t       new_state_count);
+
 extern io_return_t syscall_device_write_request(
                        mach_port_name_t        device_name,
                        mach_port_name_t        reply_name,
diff --git a/kern/syscall_sw.c b/kern/syscall_sw.c
index 4249b711..15270b1a 100644
--- a/kern/syscall_sw.c
+++ b/kern/syscall_sw.c
@@ -160,7 +160,7 @@ mach_trap_t mach_trap_table[] = {
        MACH_TRAP(syscall_mach_port_insert_right, 4),   /* 74 */
        MACH_TRAP(syscall_mach_port_allocate_name, 3),  /* 75 */
        MACH_TRAP(syscall_thread_depress_abort, 1),     /* 76 */
-       MACH_TRAP(kern_invalid, 0),             /* 77 */
+       MACH_TRAP(thread_set_self_state, 3),            /* 77 */
        MACH_TRAP(kern_invalid, 0),             /* 78 */
        MACH_TRAP(kern_invalid, 0),             /* 79 */
 
diff --git a/tests/include/syscalls.h b/tests/include/syscalls.h
index f958154c..33c37a1a 100644
--- a/tests/include/syscalls.h
+++ b/tests/include/syscalls.h
@@ -70,6 +70,7 @@ MACH_SYSCALL4(65, kern_return_t, syscall_vm_allocate, 
mach_port_t, vm_offset_t*,
 MACH_SYSCALL3(66, kern_return_t, syscall_vm_deallocate, mach_port_t, 
vm_offset_t, vm_size_t)
 MACH_SYSCALL3(72, kern_return_t, syscall_mach_port_allocate, mach_port_t, 
mach_port_right_t, mach_port_t*)
 MACH_SYSCALL2(73, kern_return_t, syscall_mach_port_deallocate, mach_port_t, 
mach_port_t)
+MACH_SYSCALL3(77, kern_return_t, thread_set_self_state, int, natural_t *, 
natural_t)
 
 /*
   todo: swtch_pri swtch ...
-- 
2.44.0