Proposal for Remote Server Notification and Control

Intent:

To allow Institute lab staff to respond quickly to critical server problems in off hours

Applies to:

Any course dependent on lab's computing infrastructure

Author:

Stephen Rondeau

Date:

3 Jun 2003

Context:

Institute lab staff are on campus from 7:30 am to 6:30 pm on weekdays. At all other times, including holidays, there is no on-site presence to respond to and correct a server should it become unavailable to use. This means that it could take up to three days (including long weekends) to resolve a problem that might impact a significant number of students in the labs.

There are several issues:

1. Lab staff desires and lack of funds prevent the ability to have staff on call for 24 hours a day, seven days a week, each day of the year (24x7x365).

2. There is no way to make the call.

   Staff don't want everyone to know their home phone numbers, and students would be the most likely people who would take advantage of an on-call number. This could result in a lot of calls about the same issue.

3. Remote diagnosis of a problem requires at least a computer and a connection to the server rooms.

   The distance the on-call person is from such a system affects the ability to respond to the problem. For some staff, it may be impossible to be travel from their residence to the campus during the early morning hours (e.g., no ferry service).

4. To remotely and completely control a computer, extra hardware is needed.

   Some newer servers have this ability built-in. Most don't, and staff-built servers definitely do not. The main operations you want to be able to do remotely are:

   • turn on/off the power to the server
   • view the server's display
   • send keystrokes to the server

   Once notified of a critical problem, one needs to be able to diagnose the problem. Sometimes the system simply hangs, and it needs to be restarted; this is difficult to diagnose remotely. To fix it requires someone to physically restart the computer. Other problems require watching the system as it restarts, and/or controlling the restart process.

The Proposal:

There are three options for the proposal:

1. provide hardware and software for immediate notification of server problems

2. provide hardware for remote control of servers

3. both of the above

There would be two phases to this proposal:

• evaluation phase

  This is where a smaller amount of money is allocated to purchase enough equipment to evaluate the approach. Notification system components have fixed and recurring costs. Remote control components have fixed costs, but such costs are per server location and per server.

  Basically, the full notification equipment is required to test notification. For remote control, only the power controller and the equipment that sends the keystrokes to the server are required; enough spare equipment exists to evaluate.

  The combined solution may not be completely compatible. The remote display of the monitor may not work on the notification and control device (see specifications below). However, if after notification, one is close to a computer with a broadband network connection, the remote display would be valuable.

• implementation phase

  This phase puts into production the components used in the evaluation phase, and extends it to all servers to be controlled. This means more money needs to be allocated to provide the added control.

  There are three locations that have a collection of servers which are conceptually all on a rack per location. Not all racks are fully populated to the eight mentioned below, and not all servers need to be monitored. Consequently, the totals in the implementation fixed cost table is a maximum; actual will probably be several hundred dollars less -- not a lot since it is the per-location costs that are the most.

Fixed Costs:

Costs are approximate, but should be within 10% of the real cost per item, excluding shipping and handling and tax.

When?	Quantity	Item	Description	Unit Price
eval	1	Blackberry 6510 w/ Nextel	Wireless Notification/Control Device	$535
eval	1	NeedTEXT Shell	Server software for notification/control device	$700
eval	1	Blackberry Enterprise Server (5-client)	Server software for notification/control device	$999
eval	1	Pager	Wireless Notification Device	$40
eval	1	Cell Phone	Connectable to Sharp PDA	$300
eval	1	Cell Phone-PDA Connection	SDG	$90
eval	1	USB power controller	turn on/off computer via USB	$40
eval	1 per 8-server rack	KeyAT	serial-port controlled keystroke inserter	$100
Pager/PDA/Cell eval	-----	Total		$570
Blackberry eval	-----	Total		$2374
implem	1 per 8-server rack	Veo Observer	Network camera	$200
implem	1 per 8-server rack	USB PCI cards (2) and 8 cables	communicate with power controllers	$100
implem	1 per 8-server rack	KVM switch	Keyboard and monitor switch	$200
implem	8 per 8-server rack	KVM cables	attach to each server	$20
implem	1 per 8-server rack	networkable computer	two NICs	$0
implem	8 per 8-server rack	USB power controller	turn on/off computer via USB	$40
implem	-----	SubTotal	for one full server rack	$980
implem	-----	Total	for all full server racks (3)	$2940

Recurring Costs:

Costs are approximate, but should be within 10% of the real cost per item, excluding shipping and handling and tax.

When?	Frequency/year	Item	Description	Unit Price
eval	12	One-way Pager Service	monthly service	$9/mo, $108/yr
eval	12	Cell Phone Service	Sprint PCS Vision Unlimited Data	$80/mo, $960/yr
eval	12	Blackberry/Nextel wireless service with internet access	connect device to campus network	$70/mo, $840/yr
Pager/Cell/PDA eval	-----	Total	1-yr contract	$1068/yr
Blackberry eval	-----	Total	1-yr contract	$840/yr
implem	1	NeedTEXT Maintenance	keep up with releases	$140/yr
Pager/Cell/PDA implem	-----	Total	annually	$1068/yr
Blackberry implem	-----	Total	annually	$980/yr