David's Astronomy Pages
Notes - Session 999 Attempt (2022-04-09)

 
Bullet Session Aims & Highlights
 - Observing Result
 - Night Summary Plot
 - Session Event Log
Bullet Operational Issues
  - Critical Issues (0),  Major Issues (1),  Minor Issues (0),  Small Defects (2),  Continuous Improvement (3)
 
2022-04-10
Bullet Investigation - Commands to open Observatory shutter failed (00:07-00:30)

Session Aims & Highlights (2022-04-09)

Main aims

  1. Targets.  Acquire images of a selection of variable stars, nearby stars, comets & deep sky targets as allowed by time & conditions.

Equipment & Software

Highlights

Notes:

  Summary Plots & Logs

Night Sky Summary Plot
Top axis: Sky Brightness at Zenith (in ADU/s)
Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)

Image   
  
 

Back to Top


Operational Issues (2022-04-09, S999A)

[ Prev | Next ]

Critical Issues

Major Issues

Minor Issues

Small Defects

Continuous Improvement

[ Prev | Next ]

Back to Top


2022-04-10


Investigation -  Commands to open Observatory shutter failed (00:07 to 00:30)

Issue 
Session failed as repeated commands to open the Obseratory shutter all failed (00:07 to 00:30)

Description
Session started normally by Autostart at 00:07 but during the Session Equilibration stage the first attempt and numerous follow-up attempts to open the dome (using ASCOM.Dome.OpenShutter method) failed.  AstroMain produced multiple messages  '"Dome opening hasn't begun within 20s (ShutterState is 'closed'). There may be a mechannical, electrical or driver issue".

Remote monitoring at 00:30 concluded that there was some show-stopping issue that was preventing the operation to open the dome and a user intervention was made to place the observatory back into Manual Mode.  After manually initiated attempts to open the dome also failed it was decided to cancel the session attempt, and review the situation the following day.

The Aux Light operated using Hue Remote Power Plug was used to demonstrate that the observatory was physically closed (It was deemed to late to visit the Observatory to try rebooting the dome control unit).

Investigation/Resolution
Diagnostics on the following day showed that attempting to open the dome from the Pulsar Dome Control Unit (using Open Shutter option) failed, but an open shutter attempt using the Red Button on the Shutter Unit did work, showing that Shutter was working and that there must be some failure in the Dome Control Unit Software logic or in the wireless connection to/from the Shutter Unit.   

 Issue resolved by rebooting the Dome Control Unit.  

Analysis
Reviewing the Pulsar ASCOM Log from the session shows the Driver receiving 'OpenShutter' commands every 20s.   Following each command there was a temporary change in ShutterStatus to 'Opening'  but Volatile records show ShutterStatus firmly set to 1 (Closed) and the ShutterStatus quickly reverts to 'Closed'.  

 It was initially thought that the issue might relate to a failure of the dome/dome driver system at the end of previous S998 session (2022-04-02).  This might be true but graphs show that Battery Health data only began flat-lining at 2022-04-05 14:00 (approx) suggesting the two issues were not  connected. 

Instead the logfile shows an anomaly at 14:45:25 with the ObsMonitor apparently frozen for 16s in the section 'Monitor TheSky6 Scope'  and 'Get Telescope.IsConnected' was logged as taking 15718 ms.  Scope wasn't connected so is this a DeviceHub or Windows OS problem.

The primary issue could have been pre-empted at anytime prior to the start of session by examining and noting that data on the Battery Status Charts had been flat-lining for the past 5 days.   Separate tickets have been raised to i) stop the Obs.Manager becoming 'stuck' indefinately in a state where it repeatedly attempts to open the shutter and ii) raise an alert if the Battery Status data appears to have flat-lined and to iii) prevent Autostart from starting if Battery Status data appears to have flat-lined.

Conclusion
Problem would seem to be due to failure in the Dome System whereby the Dome Control Unit Software froze or otherwise couldn't recieve fresh data from the Shutter Unit starting at around 2022-04-05 14:00 (approx) .   This may or may be related to a computer anomaly at 14:45.

It is noted that the issue could have been pre-empted at anytime prior to the start of session by examining and noting that data on the Battery Status Charts had been flat-lining for a period of several days.

The following defect is identified

The following gaps are identified 

Actions

Following actions identified and raised as ticket items (see Operational Issues section above for details)

Back to Top