David's Astronomy Pages
Notes - Session 869 (2021-03-13)

 
Bullet Session Aims & Highlights
 - Observing Result
 - Night Summary Plot
 - Session Event Log
 
Bullet Operational Issues
  - Critical Issues (0),  Major Issues (0),  Minor Issues (6),  Continuous Improvement (6)
Bullet Images from 2021-03-13 >>     [ Local Files >> ]
Bullet Investigation - Multiple Dome Closure Commands

Session Aims & Highlights (2021-03-13)

Main aims

  1. AstroMain.  Continue to check AstroMain 3.38/3.38.2 for stability and presence/absence of UI lockups following major refactoring
  2. Targets. Acquire images of a selection of variable stars, nearby stars, comets & deep sky targets as allowed by conditions

Equipment & Software

Highlights

Lowlights

Summary Plots & Logs

Observing Plan
Image
  
Observing Result
Image
   
Dome & Scope Slewing Performance
Image
  
Slew/Centering Performance
Image
  
Guiding Performance
Image
Image
  
Sky Conditions (Locate Frames)
Image
  
Night Sky Summary Plot
Top axis: Sky Brightness at Zenith (in ADU/s)
Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)
Image   
  
Pre-Session Weather Forecast (API data from darksky.net)
Image
Image   
  
Session Event Log
Time     Event Detail
19:46:01 Session Monitoring AutoStart monitoring for Live Session opportunity starting between 19:45 & 04:32
19:46:03 Session AutoStarting Session autostarting (19:46)
19:46:36 Session Created Live Session Created (2021-03-13 S00869, ImageSaveNum: 869001)
19:46:39   Scope Switched On Telescope Power has been switched on via UPB Powerbox.
19:48:37   Services Started Observatory Services started
19:48:45 Observatory (Auto) Observatory placed in Fully-Automated Mode
19:48:48 Session Pending Session pending (2021-03-13)
19:48:50 Session Initiating Session initiating (2021-03-13)
19:48:52   Plan Requested Observing Plan requested from AstroPlan (1.21.3)
19:49:31   Plan Loaded Observing Plan loaded to queue (, Plan ID: 607)
19:49:45   Camera1 Connected SBIG Camera Connected (set point -20°C)
19:49:51   Telescope Connected Telescope Connected (TheSky6)
19:50:02 Session Equilibration Session ready for dome & camera equilibration
19:50:48   Dome Opened Dome opened (opening time 46s)
19:51:48 Session Running Session running
19:51:49   Queue Started Observing Queue started (29 targets selected)
19:53:03       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
19:55:54       Focusing Completed Foc1 AutoFocus Completed (Profile No 1)
19:57:45       Focusing Completed Foc1 AutoFocus Completed (Profile No 1)
19:57:49       Focusing Started-Foc2 Foc2 Focusing Started (Secondary Scope, using ShCap)
20:00:10       Focusing Completed Foc2 AutoFocus Completed (Profile No 2, wide)
20:01:59       Focusing Completed Foc2 AutoFocus Completed (Profile No 2)
20:02:15       Focusing Checked Foc2 Focusing Check Succeeded
20:03:48     Target Started (1/29) Target started (1/29, NGC 304 w/SN2021dnn)
20:07:30       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
20:24:34     Target Completed Target completed (1/29, NGC 304 w/SN2021dnn)
20:29:19     Target Started (2/29) Target started (2/29, M31 w/Nova2021fbz)
20:33:00       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
20:50:11     Target Completed Target completed (2/29, M31 w/Nova2021fbz)
20:50:13     Target Started (3/29) Target started (3/29, UGC 1436 w/SN2021nv)
20:56:20       Focusing Skipped Foc1 focusing skipped - star is lost (TCF-S)
21:13:29     Target Completed Target completed (3/29, UGC 1436 w/SN2021nv)
21:13:31     Target Started (4/29) Target started (4/29, Nova Per 2020 (CV))
21:16:11       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
21:18:12       Focusing Completed Foc1 AutoFocus Completed (Profile No 3)
21:26:23     Target Completed Target completed (4/29, Nova Per 2020 (CV))
21:26:25     Target Started (5/29) Target started (5/29, AT2021ckv)
21:30:06       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
21:32:16       Focusing Completed Foc1 AutoFocus Completed (Profile No 4)
21:48:04     Target Completed Target completed (5/29, AT2021ckv)
21:48:06     Target Started (6/29) Target started (6/29, GCVS SY Cnc)
21:51:04       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
21:55:42     Target Completed Target completed (6/29, GCVS SY Cnc)
21:56:17     Target Started (7/29) Target started (7/29, AT2021bxp (CV))
21:59:44       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
22:07:34     Target Completed Target completed (7/29, AT2021bxp (CV))
22:08:09     Target Started (8/29) Target started (8/29, NGC 4227 w/SN2021biz)
22:12:23       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
22:14:29       Focusing Completed Foc1 AutoFocus Completed (Profile No 5)
22:30:17     Target Completed Target completed (8/29, NGC 4227 w/SN2021biz)
22:30:19     Target Started (9/29) Target started (9/29, GCVS RZ LMi)
22:34:04       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
22:35:56       Focusing Completed Foc1 AutoFocus Completed (Profile No 6)
22:40:50     Target Completed Target completed (9/29, GCVS RZ LMi)
22:41:49     Target Started (10/29) Target started (10/29, UGC 5829 w/LBV2021blu)
22:45:18       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
22:47:34       Focusing Completed Foc1 AutoFocus Completed (Profile No 7)
23:03:15     Target Completed Target completed (10/29, UGC 5829 w/LBV2021blu)
23:03:41     Target Started (11/29) Target started (11/29, NGC 4631 w/AT2021biy)
23:06:42       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
23:21:09     Target Completed Target completed (11/29, NGC 4631 w/AT2021biy)
23:25:33     Target Started (12/29) Target started (12/29, SN 2021xn)
23:30:04       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
23:47:06     Target Completed Target completed (12/29, SN 2021xn)
23:47:55     Target Started (13/29) Target started (13/29, AT2021dbp (CV))
23:52:07       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
23:54:31       Focusing Completed Foc1 AutoFocus Completed (Profile No 8)
00:00:07     Target Completed Target completed (13/29, AT2021dbp (CV))
00:00:09     Target Started (14/29) Target started (14/29, UGC 3502 w/AT2021cwc)
00:02:52       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
00:19:44     Target Completed Target completed (14/29, UGC 3502 w/AT2021cwc)
00:21:39     Target Started (15/29) Target started (15/29, NGC 3619)
00:25:38       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
00:28:20       Focusing Completed Foc1 AutoFocus Completed (Profile No 9)
00:44:08     Target Completed Target completed (15/29, NGC 3619)
00:44:10     Target Started (16/29) Target started (16/29, UGC 6930 w/SN2020rcq)
00:47:40       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
00:49:58       Focusing Completed Foc1 AutoFocus Completed (Profile No 10)
01:00:42     Target Completed Target completed (16/29, UGC 6930 w/SN2020rcq)
01:01:23     Target Started (17/29) Target started (17/29, NGC 4041 w/AT2021cek)
01:04:43       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
01:06:44       Focusing Completed Foc1 AutoFocus Completed (Profile No 11)
01:22:25     Target Completed Target completed (17/29, NGC 4041 w/AT2021cek)
01:23:16     Target Started (18/29) Target started (18/29, M58)
01:28:32       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
01:30:21       Focusing Completed Foc1 AutoFocus Completed (Profile No 12)
01:46:22     Target Completed Target completed (18/29, M58)
01:46:24     Target Started (19/29) Target started (19/29, NGC 4941)
01:50:05       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
01:52:05       Focusing Completed Foc1 AutoFocus Completed (Profile No 13)
02:07:40     Target Completed Target completed (19/29, NGC 4941)
02:07:59     Target Started (20/29) Target started (20/29, AT2021dtd)
02:11:48       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
02:13:37       Focusing Completed Foc1 AutoFocus Completed (Profile No 14)
02:29:20     Target Completed Target completed (20/29, AT2021dtd)
02:29:51     Target Started (21/29) Target started (21/29, GCVS BL Lac)
02:32:29       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
02:34:31       Focusing Completed Foc1 AutoFocus Completed (Profile No 15)
02:38:04     Target Completed Target completed (21/29, GCVS BL Lac)
02:40:09     Target Started (22/29) Target started (22/29, UGCA 305)
02:44:30       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
02:46:30       Focusing Completed Foc1 AutoFocus Completed (Profile No 16)
03:02:24     Target Completed Target completed (22/29, UGCA 305)
03:02:32     Target Started (23/29) Target started (23/29, GCVS R CrB)
03:05:24       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
03:07:02       Focusing Failed Foc1 focusing failed - failed QC check (Ambiguous)
03:08:50   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
03:09:00   SoftSuspend Cancelled Soft Suspend is cancelled due to Improved Conditions
03:09:59     Target Completed Target completed (23/29, GCVS R CrB)
03:11:12     Target Started (24/29) Target started (24/29, UGC 10310)
03:13:58       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
03:16:06       Focusing Completed Foc1 AutoFocus Completed (Profile No 18)
03:21:30   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
03:22:35   SoftSuspend Cancelled Soft Suspend is cancelled due to Improved Conditions
03:33:30 Critical Wind Alert Critical Wind Alert (Obs.Manager will close the Shutter)
03:33:32   HardSuspend Called Hard Suspend is called due to Critical Conditions (wind)
03:33:38     Target Aborted (24/29) Target aborted (24/29, UGC 10310) due to wind
03:33:40   Queue Paused Queue paused for Session Suspension
03:33:43 Session Suspended Session suspended
03:34:30   Dome Closed Dome closed (closing time 45s)
03:36:38     Target Cancelled Target cancelled (24/29, UGC 10310) due to wind
03:57:50     Target Cancelled Target cancelled (25/29, C/2020 T2 (Palomar)) due to wind
03:59:00 Session Resuming Session resuming
03:59:45   Dome Opened Dome opened (opening time 45s)
03:59:47 Session Running Session running
03:59:49     Target Started (26/29) Target started (26/29, IC 992 w/SN2021dwg)
03:59:51   Queue Resumed Observing Queue resumed
04:03:04       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
04:12:56 Critical Wind Alert Critical Wind Alert (Obs.Manager will close the Shutter)
04:12:58   HardSuspend Called Hard Suspend is called due to Critical Conditions (wind)
04:13:01     Target Aborted (26/29) Target aborted (26/29, IC 992 w/SN2021dwg) due to wind
04:13:03   Queue Paused Queue paused for Session Suspension
04:13:06 Session Suspended Session suspended
04:13:56   Dome Closed Dome closed (closing time 50s)
04:17:27     Target Cancelled Target cancelled (26/29, IC 992 w/SN2021dwg) due to wind
04:30:17 Session Resuming Session resuming
04:31:06   Dome Opened Dome opened (opening time 49s)
04:31:08 Session Running Session running
04:31:10     Target Started (27/29) Target started (27/29, 7P/Pons-Winnecke)
04:31:12   Queue Resumed Observing Queue resumed
04:44:19     Target Completed Target partially completed (27/29, 7P/Pons-Winnecke)
04:44:21     Target Started (28/29) Target started (28/29, HIP 87937)
04:47:45       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
04:49:35       Focusing Completed Foc1 AutoFocus Completed (Profile No 19)
04:51:44     Target Completed Target completed (28/29, HIP 87937)
04:51:46     Target Started (29/29) Target started (29/29, GCVS CY Lyr)
04:54:48       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
04:56:38       Focusing Completed Foc1 AutoFocus Completed (Profile No 20)
05:00:09     Target Completed Target completed (29/29, GCVS CY Lyr)
05:00:12   Queue Completed Job Queue completed
05:00:15 Session Closing Session closing
05:01:08   Dome Closed Dome closed (closing time 50s)
05:01:34   Dome Parked Dome parked (parking time 18s), Az: 90.0 deg
05:02:15   Telescope Parked Telescope parked (parking time 34s)
05:02:45   Telescope Switched Off Telescope Power has been switched off via UPB Switch.
05:03:34   Services Stopped Night Services stopped
05:03:36 Session Housekeeping Housekeeping Started (Cleanup FITS, Create Fits Summary, Transfer Files)
05:03:43 Session Finished Session Finished
 
Session Alerts
Time     Alert Detail
00:10:23 Dome Dome has been stationary for more than 10 mins
00:14:53 Dome Dome Movements have begun again (outage 14.5 mins)
00:23:09 Dome Discrepency Difference between Telescope Az and Dome Az is > 8 deg (NGC 3619)
01:03:10 Dome Discrepency Difference between Telescope Az and Dome Az is > 8 deg (NGC 4041)

Back to Top


Operational Issues (2021-03-13 S869)

[ Prev | Next ]

Critical Issues

Major Issues

Minor Issues

Continuous Improvement

[ Prev | Next ]

Back to Top


Investigation - Multiple Dome Closure Commands

Issue:   End of Session reporting shows more close dome requests are being made than are necessary

Description.  End of session reporting show there were 6 close dome requests (3 Completed). This may or may not be a reoccurance of the issue where Pulsar Dome doesn't respond to a Close Dome request even though it is received by Pulsar Driver (as shown by Pulsar ASCOM Trace Log). Alternatively it may be an accounting issue only.   S866 has 3 close requests (1 completed) , S867 session had 4 close requests (4 completed).  S868 session had 2 close requests (2 completed),  S870 session had 9 close requests (5 completed).

Analysis.  'Close Dome Requests' is an end of session reporting of the variable Dome.nCloseShutterCommands, whilst ('Completed') is an end of session  reporting of the variable Dome.nShutterCloseCompleted.  

nCloseShutterCommands is incremented by 1 in the Dome.CloseShutter() routine after its starts the CloseShutterTask thread (started by calling the CloseShutterByThread() routine). 

Straightaway it would seem that the Dome.CloseShutter is being called more than once each time the Shutter requires closing.. 

Separate to nCloseShutterCommands counterd there is a CloseShutterAttempts counter for recording the number of repeat close shutter attempts for a given Dome Closure.  CloseShutterTask (thread) makes up to 3 attempts to initiate shutter closing with each attempt separated by 6 secs (Note: bCloseShutterTaskRunning = True whilst CloseShutterTask is active)

End of Session reporting shows more

For dome closure at 03:33 it seems that CloseShutter() routine is called 3 times at 03:33:41.09 (T0) ,  03:33:41.13  (T+0.04s) & 03:33:45.89 (T+4.8s),   with Closing Dome message coming at 03:33:50.86 (T+9.77s)


First call to Dome.CloseShutter is clearly made from Obs.Manager's  SetState_Suspended() routine.

03:33:41.07 | Observatory Manager      Change State    Info | Session Suspended
03:33:41.09 | Debug Trace              Suspending      Info | Calling Dome.CloseShutter
03:33:41.09 | Observatory Manager      Close Dome      Info | Sending CloseShutter command to Dome (Attempt 1)
03:33:41.10 | Debug Trace              Suspending      Info | Back from Dome.CloseShutter
03:33:41.10 | Debug Trace              Shutter Closing Info | Calling objDome.CloseShutter (attempt 1)
03:33:41.13 | Observatory Manager      Close Dome      Info | Sending CloseShutter command to Dome (Attempt 2)
03:33:41.14 | Debug Trace              Shutter Closing Info | Calling objDome.CloseShutter (attempt 1)
03:33:41.52 | Debug Trace              Shutter Closing Info | Back from objDome.CloseShutter (attempt 1)
03:33:41.55 | Debug Trace              Shutter Closing Info | Back from objDome.CloseShutter (attempt 1)
03:33:45.89 | Observatory Manager      Close Dome      Info | Sending CloseShutter command to Dome (Attempt 2)
03:33:45.90 | Debug Trace              Shutter Closing Info | Calling objDome.CloseShutter (attempt 1)
03:33:46.31 | Debug Trace              Shutter Closing Info | Back from objDome.CloseShutter (attempt 1)
03:33:50.86 | Observatory Manager      Close Dome      Info | Closing Dome
03:34:30.86 | Observatory Manager      Close Dome      Info | Dome is Closed (45s)

An issue is that the CloseShutterAttempts counter is used by both Obs.Manager and the CloseShutterTask and usage of the counter is getting mixed up.
Proposal is to separate the counter inside CloseShutterTask () from that used by Obs.Manager.  
- CloseShutterAttempts is set to  0 in SetState_Suspended()
- CloseShutterAttempts is incremented by 1 in CloseShutter()
- CloseShutterAttempts is set to 0 in  CloseShutterByThread
- CloseShutterAttempts is set to 0 in CloseShutterTask
- CloseShutterAttempts  is incremented by 1 in CloseShutterTask (directly after setting to 0, so equivalent of setting it to 1)
- CloseShutterAttempts  is incremented by 1 in CloseShutterTask by each further attempt if required (up to 3 attempts)

CloseShutterByThread () doesn't check that CloseShutterTaskThread is already executing for launching it (therefore more than one thread could be running at same time !)  if CloseShutter()( routine is being called twice within 0.04s.

For Dome Closure at 04:13:06 the CloseShutter() routine is called 2 times at 04:13:06.60 (T0) ,  04:13:06.64  (T+0.04s).   Closing Dome message comes at 04:13:11.38 (T+4.78s)

 

Actions

Back to Top