Last updated: 8 Mar 2021

• Overview
• Commands
• Manual Interaction Commands

Overview

If there are no remote desktop or ssh services available, the only way to interact with your VM is to simulate interaction via taking periodic snapshots of the VM and sending keystrokes to the VM. This is a tedious method, but it can get you to the point of enabling remote desktop or ssh services, or simply issuing the right commands to the VM for whatever purpose its guest operating system serves.

The Virtual Computing Lab (VCL) was created to be centered around commands, because commands can be scripted and the results of the commands parsed for information. ssh is used to control and interact with the VMs via the host, which passes the commands to the VM or the guest operating system within it.

Commands

A command is either a custom command allowed by and implemented in the ssh_vbm script, or it is a subcommand of the VirtualBox VBoxManage command. Often, ssh_vbm "commands" use VBoxManage subcommands to implement the desired functionality.

All of the documented ssh_vbm commands can be found by issuing the help command. The manage_vc app issues commands either by:

• selecting a predefined command from the drop-down box to the right of the "Do cmd:" button, or
• typing in a command into the drop-down box to the right of the "Do cmd:" button

and then clicking the "Do cmd:" button to execute the command, and the "OK" button to confirm the execution.

Without manage_vc, Windows, Mac and Linux users can issue commands directly to the host. You must know the host IP address, the user or team account name, and the user or team account password. For example, if user janedoe wants to ask for the commands she can enter on host 140.142.71.5:

ssh janedoe@140.142.71.5 help

and enter the password (from the .vcl file) when prompted. Currently, this would be the result:

------------ Control VM state ------------
vcl_status vmname -- state of the VM;
                  -- guest OS's suspend/hibernation state will show "running"

start vmname -- start the VM
             -- if vcl_status is powered off, aborted, saved, or suspended
poweroff vmname -- poweroff ("pull the plug") the VM
shutdown vmname -- attempt to gracefully shutdown the guest OS in the VM

reset vmname -- reset the VM, like a computer's reset button
wakeup vmname -- tell the VM's guest OS to exit suspend/hibernation state

--------------- Show VM's desktop/console via web ---------------
screenshot vmname help -- see help about taking screenshots and viewing them

--------------- Send keys to VM ---------------
sendkeys vmname help -- see help about sending keyboard keys to VM
send_cad vmname -- send Ctrl-Alt-Del to the VM

------------ Show and configure network ------------
show_nic vmname -- show all NICs associated with VM

show_vm_ip vmname -- show IPv4 address used by guest OS
show_ip -- show the host's IP information

start_vde args -- start a VDE switch
stop_vde args -- stop a VDE switch
show_vde args -- information about VDE switches
config_vde vmname args -- configure a VM to use VDE

config_vboxnet0 -- configure host-only networking to 192.168.254.1/16

config_nat_port vmname nic_num/guest_port/tcp_or_udp
                -- configure NAT networking for port forwarding
                -- e.g., to port forward win811's guest port 3389 (RDP)
                   on NIC2 using tcp:
                config_port win811 2/3389/tcp

-------------- Manage CDs/DVDs --------------
eject_cd vmname -- eject any found CD (.iso file) from VM
add_cd vmname cd_file -- add CD (.iso file) to VM if controller port is empty
show_cd vmname -- show CDs (.iso files)attached to VM

-------------- Kill a hung VM (last resort) --------------
killvm vmname -- kill all VirtualBox processes with vmname in them

-------------- Issue other commands --------------
any other command and args -- sent to VBoxManage as is

Note that this is the not the correct way of sending any ssh command to a remote server. Complications arise when the command includes special characters such as the single quote and other shell special characters. Examples of shell special characters are using semicolons (;) to separate multiple commands, ampersands (&) to combine commands conditionally, and or bars (|) to pipe command output. To avoid confusion, the correct way is to surround the command with single quotes, as in:

ssh janedoe@140.142.71.5 'help'

That way, if your command includes special characters (except another single quote), they will be passed as is to the remote host. For more explanation, please see a sendkeys case that highlights the importance of using quotes.

Manual Interaction Commands

However, for this topic, we only are concerned about these ssh_vbm commands:

• vcl_status
• start
• screenshot
• sendkeys

The idea is that you make sure that the VM is running by using vcl_status, start the VM if it is not running, look at the desktop or console using screenshot, and send keys to the desktop/console to specify what to do.

The documentation below is for users using ssh commands to control their VMs; however, if you simply omit ssh janedoe@140.142.71.5 from the examples below, you would have the command to type into manage_vc's "Do cmd:" drop-down box.

vcl_status

vcl_status will show the state of the VM, which is usually "running" or "powered off". For screenshot or sendkeys to work, the VM must be running.

For example, for the VM "ubuntu" (the name of the VM running the Ubuntu OS as a guest), and the same information as above:

ssh janedoe@140.142.71.5 vcl_status ubuntu

and enter the password (from the .vcl file) when prompted, will show its state.

start

If the VM is not "running", it must be to use the screenshot or sendkeys commands.

For example, for the VM "win10" (the name of the VM running the Windows 10 OS as a guest):

ssh janedoe@140.142.71.5 start win10

and enter the password (from the .vcl file) when prompted, will start the win10 VM.

screenshot

Once a VM is started, we would like to make sure of its functional state by looking at its virtual monitor, which we can see if we can take a screenshot of the monitor. That is, was there a problem with booting? Is it still booting up? Is it waiting for a login?

While one could take just one screenshot, it is more useful to periodically take a screenshot, so that changes made by you or the OS are reflected in the desktop or console output. Although you could take a screenshot every second, that would use up a lot of resources and slow down your VM as well as any others on the host. We suggest every five seconds, which can be accomplished by this command (again, for VM win10):

ssh janedoe@140.142.71.5 screenshot win10 every 5

This does two things:

1. starts a per-user/team web server on the host at a particular port (if there is no prior web server started), and
2. starts taking screenshots of the VM every 5 seconds, updating a web page with the new screenshot.

For example, the preceding screenshot command would generate output similar to this:

Browse to http://140.142.71.5:54250/win10.html

And if one would copy and paste that URL into a web browser, one should see the screenshot for win10. If that person would wait at least five seconds, and refresh the win10.html page, she could see a different screenshot, depending on the activity of the VM's OS or boot process. Of course, your screenshot experience will be different.

There are web browser add-ons or extensions that may be able to refresh the page automatically for you, but such extensions are beyond the scope of this description, and left as an exercise for the user.

Each VM you have uses the same web server; just the web page name changes. Each screenshot has a date and time stamp displayed, which you should compare to the current time to see if you are getting the most recent screenshot or if something else happened (like the VM went down).

This is just the tip of the iceberg for screenshot features. Since you started a periodic screenshot, you must be able to stop it; e.g.:

ssh janedoe@140.142.71.5 screenshot win10 stop

or it will keep attempting to take screenshots until the host restarts.

You may need to stop the periodic screenshots when you restart your VM, using another "screenshot win10 every 5" command after stopping the old one.

For a list of all of the screenshot subcommands:

ssh janedoe@140.142.71.5 screenshot win10 help

sendkeys

Although an OS can and does do a lot behind the scenes to provide a usable system, it is often most useful to you when you get to control what it does, by sending it commands to execute. Note that there currently (4 Aug 2020) is no ability to send mouse movements and clicks to the OS, so all interaction is via the keyboard. It is very useful to know keyboard shortcuts (see cross-platform keyboard shortcuts, Windows keyboard shortcuts or GNOME keyboard shortcuts) and use the alt-x technique to get to different control elements in a window, like buttons and menus, but this also is left as an exercise for the user.

Most computer users are very comfortable with a keyboard, especially for typing text. However, there are a lot of complexities in using keys to control a computer. Perhaps the fastest way to show that is by invoking the help:

ssh janedoe@140.142.71.5 'sendkeys win10 help'

which currently will show this:

sendkeys sends keyboard scan codes to the VM as if you pressed and released
the keyboard keys yourself.

sendkeys takes a VM name followed by a list of named keys, key sequences,
or quoted strings. They are separated by a percent sign "%". Named keys
such as 'alt', 'ctrl' and 'lshift' reflect the keys on a keyboard ('lshift'
is the left shift key), but represent just the key press/down event, NOT
the key release/up event. This is so that other keys can be combined
with them, e.g., for capitalizing a string. For example:

sendkeys win10 lshift-d%'abc'%lshift-u

will provide this string:

ABC

to the win10 VM.

Some named keys represent common key sequences, such as 'ctrl-alt-del', which
would be fairly long to enter correctly every time. Key sequences often have
hyphens in them. The 'enter' named key sends an enter key, but sometimes the
enter key from the numeric keypad is desired or works better, and that is
'kpenter'.

Individual letters, numbers and symbols should be specified between
single-quotes. Quotes make it clear that a string and not a named key is
being requested to be sent, and it is sent using a more efficient method.

Here are all of the names keys, key sequences and individual characters
that can be sent:


Modifier Keys (e.g., lalt-d means press/hold left alt; lalt-u means release)
  lalt-d lalt-u ralt-d ralt-u lctrl-d lctrl-u rctrl-d rctrl-u lwin-d lwin-u
  rwin-d rwin-u lshift-d lshift-u

Named Key Sequences
  alt-home alt-home-r shift-end ctrl-alt-del ctrl-alt-end ctrl-c ctrl-esc
  shift-home lwin-r shift-f10

Special Keys
  menu esc kpenter enter tab bksp home end up down left right pgdn pgup ins
  del

Function Keys
  f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12

Special Named Single Keys, key with name under it, in order:
  \   '  %   `  ;  &   |
  bsl sq per bt sc amp or

Letter Keys
  a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L
  M N O P Q R S T U V W X Y Z

Numbers and Special Keys
  1 2 3 4 5 6 7 8 9 0   , . / < > ? ! @ # $ % ^ & * ( ) - _ = + ` ~ [ { ] } \
  | ; : ' "

Window Control Navigation Keys
  alt-a alt-b alt-c alt-d alt-e alt-f alt-g alt-h alt-i alt-j alt-k alt-l
  alt-m alt-n alt-o alt-p alt-q alt-r alt-s alt-t alt-u alt-v alt-w alt-x
  alt-y alt-z alt-1 alt-2 alt-3 alt-4 alt-5 alt-6 alt-7 alt-8 alt-9 alt-0
  alt-sp alt-, alt-. alt-/ alt-< alt-> alt-? alt-@ alt-# alt-$ alt-per alt-^
  alt-& alt-* alt-( alt-) alt-- alt-+ alt-= alt-+ alt-bt alt-~ alt-[ alt-{
  alt-] alt-} alt-bsl alt-or alt-sc alt-: alt-sq alt-dq alt-f1 alt-f2 alt-f3
  alt-f4 alt-f5 alt-f6 alt-f7 alt-f8 alt-f9 alt-f10 alt-f11 alt-f12

Let's say you have that win10 VM, which you can tell (from screenshot) is at the default Windows screen. It is likely instructing you to enter Ctrl-Alt-Del, which is a sequence of keys (actually, key presses/down events and key release/up events) in a certain order; namely, hold down Ctrl, hold down Alt, press and release Del, release Alt, then release Ctrl. Since it is a common sequence, we provide the named key sequence ctrl-alt-del for you to use:

ssh janedoe@140.142.71.5 'sendkeys win10 ctrl-alt-del'

and if you refresh the screen shot web page, you should see a login prompt. Note that we did NOT use:

ssh janedoe@140.142.71.5 'sendkeys win10 '\''ctrl-alt-del'\'''

because that would send the string "ctrl-alt-del", which is all of the key presses/releases for each character in the string, to the VM -- which is not what we wanted.

However, timing is everything, because Windows will not keep the login prompt showing for a long time. So you should be sending the password for the default login user soon after ctrl-alt-del. Let's say the password is "my-secret":

ssh janedoe@140.142.71.5 'sendkeys win10 '\''my-secret'\''%enter'

Before we get into this much, let's digress and look at what would happen if we had entered the above command the wrong way:

ssh janedoe@140.142.71.5 sendkeys win10 'my-secret'%enter

That looks like the right way to do it, but what is received by the remote host is:

sendkeys win10 my-secret%enter

when what you wanted was to send the string 'my-secret' to the remote host, so that sendkeys understands you meant a string, not a named key sequence called my-secret. You must surround the entire sendkeys command with single quotes, but that's not enough -- you have to also "escape" the quotes that surround my-secret, which is done by replacing a single quote by '\'', resulting in the awkward-looking but functionally correct:

ssh janedoe@140.142.71.5 'sendkeys win10 '\''my-secret'\''%enter'

To continue past that digression: what is %enter doing there? It is making sure the enter key is pressed after the password is entered, so you don't have to use another "sendkeys" command with the named key enter. Sometimes, the keypad enter (kpenter) works more reliably.

You can send as many strings and keys at once (separated by %) as you dare, within command line limits, but sometimes one or two of the keys you send are not passed to the VM. This is frustrating, and you must decide what you will do to try to fix it. In a command prompt, if you didn't send the enter key, you could send the esc (escape) key to clear the input. But that doesn't always work. This is part of what is meant by sendkeys being a "tedious" method of interaction.

If you look at the screenshot and find you successfully logged on, then you have to decide what you are going to do next. Normally, since commands are all you have, you want to use a command shell so you can enter more complicated commands or even start some windows/applications:

ssh janedoe@140.142.71.5 'sendkeys win10 lwin-r%'\''cmd'%enter'\'''

That keyboard sequence will hold down the left windows key and press/release the "r" key, then release the left windows key, which tells Windows to open the Start Menu's run prompt. It waits for the user to enter a command to run, and since you might want to run several, telling it to start the Windows command prompt via 'cmd' will do so -- as long as you press the enter key after it. If successful, your screenshot should show a command prompt window.

If you are working with a graphical window (with buttons and menus and the like), you will either use the "Window Control Navigation Keys" to click on a button (where "x" in alt-x is typically an underlined character in the window) or keep sending tab characters and checking the screenshot to determine on which window control element the focus is (look for extra highlighting, like a dotted box around the control). If the focus is on the control element you desire, sending an enter key will be the same as clicking on that button, activating the tabbed panel or following that link.

Using the menu key can open up a menu like the right mouse-click would, then you can use the arrow keys (up, down, left, and right) to navigate on the menu and expand/collapse submenu items.

You can do quite a bit using sendkeys once you are comfortable with how Windows and sometimes other windowing systems like GNOME work, but perhaps the best thing you can do is enable the remote desktop or secure shell if it is not enabled, so it will allow you to be much more productive. sendkeys/screenshot is a last resort for OSes with better interaction capabilities built-in, but may be the only way other OSes or BIOSes can be interacted with.

Change Log