Problem Solving and Tricks

Problem: Arcon appears hung, or freezes during reboot

Occasionally, the Arcon controllers get into a weird state.

SOLUTION: There are roughly 4 levels of severity with which the user can respond:

• Restart Arcon: Use the pull-down menu in the ctioa0 VNC window and select End Arcon Session, wait a few seconds, then select (Re)start Arcon. This will take a little while. It may take several (3-5) Restarts to succeed. Always try at least 3 Restarts before proceeding to the next level of severity listed below.
   
• Restart Arcon AND cycle the Trambox power: Use the pull-down menu in the ctioa0) VNC window and select End Arcon Session. Go through the side door in the 4- console to the computer room and find the Mosaic Trambox (on the same rack as the ctioa0 pizza box). Turn off the power, wait a few seconds, then turn on again. Go back to the console room, and select (Re)start Arcon from the pull-down menu in the ctioa0 VNC window.
   
• Exit Arcon, turn off the Trambox, AND reboot ctioa0:
  First, exit Arcon, kill the ctioa0 VNC window, and turn off the Trambox. Now find the ctioa0 keyboard and monitor in the side computer room. Then follow these steps:

1. Press the "L1" key in the upper left corner of the keyboard; it may be labelled something else, like "Stop".
2. While holding down the "L1" key, press "A". You should see "ok".
3. Type "boot".
4. Wait for the reboot to complete. Turn the Trambox back on.
5. On ctio4m, type "view_a0" at the Unix prompt. This should start the VNC server and bring up the ctioa0 window. (See Using VNC with MOSAIC for details.
6. Restart Arcon from the menu in the ctioa0 VNC window.

• This is the most serious mode and it should be done with the cognizance of NOAO personnel. This will be a total restart of the system. Follow the process in item (3) above for the data-taking computer, but only through step (c). Now:

1. Follow the procedure used to reboot ctioa0 to reboot ctio4m: log out of OpenWindows completelt, then type "L1 A" and then "boot" on the ctio4m keyboard.
2. Go to the grey panels in the side computer room and power off the Arcon. Wait 10 seconds, then power them back on.
3. Power the Trambox on.
4. Log back into ctio4m. Restart both ctioa0 and ctio4m VNC servers and clients by typing "view_a0view_4m" in a Unix window.
5. In the ctioa0 VNC window, Restart Arcon from the pull-down menu. In thectio4m VNC window, Restart DCA from the pull-down menu.
6. If this didn't help, notify your staff contact on the cell phone!

Problem: The MSE box on the MCCD Configuration comes up red with an X.

The most common explanation for this is that msmid is not running on the data acquisition machine.

SOLUTION: start up msmid on the data acquisition machine. This is described in step 2 of the detailed instruction list for changing data acquisition computers.

Problem: Only the upper GUI (MCCD Configuration) comes up.

This usually happens because there is another copy of the GUI running on the data acquisition machine. The title of the GUI usually says "MCCD Configuration - Monitor mode".

SOLUTION: kill off all other 'arInt' processes. First do a ps aux | grep Int. Look for any 'arInt' processes. Kill these with kill -9 (process id number). Then you can EITHER restart Arcon (~3 minutes), OR, if you're in a hurry, you can go to the blue IRAF (Data Acquisition) window and type gwcclose, return, thengwcopen, and return. This should bring up both GUI windows.

Problem: Checking communication between data acquisition machine and Arcon boxes

try b1 vtt telemetry. The 'b1' can be replaced by b2, b3, and b4 to check communication and voltages on each box.

Problem: triggering not working?

Try turning off triggering. To do this, in the blue IRAF (Data Acquisition) window, type engpars and change the trigger parameter to 'no'. Then run setdet to download this change to the controller. Each box will now run on its own, not waiting for the master for trigger signals.

Problem: Debugging MSE communication

To start direct communication with the MSE system (which is connected via the serial port on the data acquisition machine), do the following...

• kill msmid
  • ps -awx | grep msmid
  • kill -9 #### (the pid corresponding to /usr/local/bin/msmid)
• tip ttya (communicate through serial port)
• After hitting return, you should get the "MSE>" prompt
• diag to bring up diagnostic menus
• quit out of diag menus
• hit return, then type "~" followed by a ctrl-d

Problem: the PVM on one or both machines is not recognizing the other machine

Occasionally, the one or both PVMs get into a weird state. This can be seen by typing "pvm", and then "conf" on either ctioa0 or ctioa8. Either machine should list both ctioa0 and ctioa8 as connected.

SOLUTION: Restart the PVMs from zero, following the instructions on the PVM restart page.