I've been migrating from my old Intel Mac to a new M3 Mac, which uses ARM64 instead of x64 architecture. I ran into a strange error message when I tried to run a Packer build that used VirtualBox as its provider:

==> Starting the virtual machine...
==> Waiting 10s for boot...
==> Typing the boot command...
==> Error running boot command: VBoxManage error: VBoxManage: error: Failed to send a scancode.

Manually trying to type in the virtual machine window triggered an error notification from VirtualBox:

Keyboard failure

Failed to change keyboard parameter.
The console is not powered up (putScancodes).

Result Code:  E_ACCESSDENIED (0x80070005)
Component:    KeyboardWrap
Interface:    IKeyboard {755e6bdf-1640-41f9-bd74-3ef5fd653250}

I couldn't find any relevant posts online about these errors, but given this build worked with an identical configuration on my x64 Mac and Windows hosts, I assumed the problem was related to the architecture change.

Finding the culprit

The "access denied" error code immediately made me suspicious that macOS's privacy settings were interfering with VirtualBox's ability to read keyboard input. However, the system hadn't prompted me about any attempted access, and I had given VirtualBox fairly permissive access already.

I started troubleshooting by manually creating a test VM in VirtualBox with all of the default hardware settings. I booted into the installation ISO I was using for the build, and keyboard input worked right away. This hinted that some hardware setting in my build configuration differed from VirtualBox's defaults, and that setting was likely responsible for the problem.

When you create a new VM within VirtualBox, it defaults to using a virtual USB 3.0 (xHCI) controller. Conversely, Packer's VirtualBox plug-in natively configures a pretty sparse machine that doesn't even enable USB by default. To mimic the build configuration, I disabled USB on the test VM, and switched the emulated pointing device from its default "USB Tablet" setting to "PS/2 Mouse" — and on my next boot, I was greeted by the same "Keyboard failure" error as earlier. Interesting!

The fix

For VirtualBox running on macOS with Apple Silicon (ARM64) hardware, "Keyboard failure" errors can potentially be fixed by enabling USB 3.0 on the VM. It seems to specifically require USB 3.0, as my VM still wasn't registering keyboard input when I tried with USB 1.1 and USB 2.0.

Additionally, if the mouse isn't working (such as in the Ubuntu graphical installer), the pointing device may need to be set to "USB Tablet". I noticed that if the pointing device was set to "PS/2 Mouse" (which the Packer plugin uses by default), mouse input wouldn't work in the guest host unless the VirtualBox guest additions were installed.

I was able to fix my Packer build with the following additions to my virtualbox-iso source block. Note, the iso_interface option isn't related to this issue, but is necessary for booting ISOs on ARM64.

source "virtualbox-iso" "base" {
  # Required for booting ISOs on EFI / ARM64
  iso_interface = "sata"

  # Use USB 3.0 and 'USB Tablet' pointing device
  vboxmanage = [
    [ "modifyvm", "{{ .Name }}", "--usb-xhci=on"],
    [ "modifyvm", "{{ .Name }}", "--mouse", "usbtablet" ]
  ]
}

I spend a lot of time in the terminal these days, and I've started to accumulate a pretty nice set of tricks for making that time less tedious.

This is the first post of my Terminal Velocity series, where I'll be discussing several interesting features and capabilities of modern terminal emulators that can help you operate more swiftly while using the command line.

What are ANSI escape sequences?

I'll start with a brief primer on ANSI escape sequences, as a lot of what I'll be discussing uses them.

ANSI escape sequences are special sequences of control characters that allow a system to send signals to a terminal. When a terminal receives an ANSI escape sequence, it is interpreted, not displayed. Common usages include setting text color and positioning the cursor (through CSI sequences).

Of particular interest are Operating System Command (OSC) sequences, which allow a system to interact with features of a terminal emulator. OSC sequences enable useful functionality like setting the window title, letting a remote system copy text to the local clipboard, and semantically marking up command output so it's easier to parse.

The format of OSC sequences

All ANSI escape sequences start with \e, followed by a control code. The control code for OSC is ]. The OSC sequence is then comprised of an OSC code, which specifies the command to perform, followed by any arguments for the command separated by semicolons. The sequence is finally terminated with \a¹.

For example, you can use OSC 0 to change the window title to meow by running: printf "\e]0;meow\a"

There are many OSC codes that are widely-supported, but a number of modern terminal emulators have extended the OSC command set with proprietary commands as well. For example, iTerm2 even supports inline file transfers through its own OSC 1337.

Shell integration & command marks with OSC 133

One of the most tedious things while working in a terminal can be dealing with commands that repeatedly produce a lot of output. It can very quickly become a chore to navigate through the results of each execution, let alone do things like quickly copy the output.

In 2013, the Final Term emulator pioneered a clever solution for this. By embedding special OSC sequences within shell prompt variables, it was able to differentiate between an entered command and the command's subsequent output. And although Final Term is now defunct, modern terminal emulators like iTerm2 and Windows Terminal now use this same approach to support features like navigating through previous commands, automatic output selection, and showing markers with exit results in the scroll bar.

This feature is now often referred to as shell integration, command marks, or semantic prompts.

How it works

Bash has several special prompt variables that are used in different stages of input and output, including:

• PS1, the primary prompt string, printed before reading the command
• PS0, printed after reading the command, but before it starts
• PROMPT_COMMAND, executed after the command finishes, but before PS1 is displayed

Shell integration works by embedding 4 OSC sequences into these variables (see below for their values). These sequences are often referred to by names that pay tribute to their origin in Final Term:

When integrated into the prompt variables, their position during the shell's read–eval–print loop looks like:

[FTCS_PROMPT]raccoon@gibson$ [FTCS_COMMAND_START]whoami
[FTCS_COMMAND_EXECUTED]raccoon
[FTCS_COMMAND_FINISHED]

The terminal is then able to use the positioning of the OSC sequences to distinguish between an entered command and its subsequent output.

FTCS_COMMAND_FINISHED is able to take the command's exit code as an argument, further allowing the terminal to interpret if a command succeeded or not.

Integrating the sequences

ℹ️ Note

When using ANSI escape sequences in Bash prompt variables, they must be wrapped in \[ and \]

The following Bash snippet demonstrates the process of wrapping existing prompt variable values with the OSC sequences to enable shell integration features:

FTCS_PROMPT="\e]133;A\a"        # OSC 133;A (start of the prompt)
FTCS_CMD_START="\e]133;B\a"     # OSC 133;B (start of entered command / end of the prompt)
FTCS_CMD_EXECUTED="\e]133;C\a"  # OSC 133;C (start of the command's output)

# PS1: Mark the start of the prompt and the start of the entered command
PS1="\[${FTCS_PROMPT}\]${PS1}\[${FTCS_CMD_START}\]"

# PS0: Mark the start of command output
PS0="${PS0}\[${FTCS_CMD_EXECUTED}\]"

# PROMPT_COMMAND: Mark the end of command output, with exit code as an argument
# Exit code *must* be retrieved first, otherwise it may be overwritten.
print_cmd_finished() {
  local FTCS_CMD_FINISHED="\e]133;D;${?}\a" # OSC 133;D (end of the command's output)
  printf $FTCS_CMD_FINISHED
}
PROMPT_COMMAND="print_cmd_finished; $PROMPT_COMMAND"

The positioning of the sequences within the prompt variables is an important detail. Terminal emulators effectively use these sequences as delimiters, so any extraneous data encountered between marks can result in output not being interpreted correctly.

FTCS_PROMPT must be at the start of PS1, with FTCS_CMD_START at the end, so the position of the prompt and the command can correctly be identified. Similarly, FTCS_CMD_EXECUTED must be at the end of PS0 to correctly mark the start of command output.

The call to print FTCS_CMD_FINISHED must be at the start of PROMPT_COMMAND, both to correctly mark the end of output and to retrieve the command's exit code via $?. If any other commands run before that point, the entered command's exit code will be overwritten.

The above code is written in a generic way that wraps existing prompt variables, so it may be appended to the very end of an existing .bashrc file.

Choice of prompt variables

There are multiple ways to go about printing the sequences at the correct time. For example, Microsoft recently published a related tutorial that dynamically updates PS1 with the FTCS_COMMAND_FINISHED sequence. While this does work, I find it to be unnecessarily complicated, as the same thing can be achieved through simply printing the sequence via PROMPT_COMMAND.

Compatibility

This works nicely in iTerm2 and Windows Terminal, but regardless of terminal support, it won't work with programs like tmux or screen (at least not out of the box). Shell integrations are not currently supported by terminal emulators like xfce4-terminal, PuTTY and mintty – but at least according to my testing, the sequences are merely ignored, so they shouldn't interfere with anything.

Trying to use this with cmd.exe (and by extension ConEmu) may result in some funny characters. :)