David's Astronomy Pages Notes - Session 1298 Attempt (2025-01-20)

Notes (S1297)

Notes Main

Home Page

Notes (S1298)

	Session Aims & Highlights
	- Observing Result  - Night Summary Plot  - Session Event Log
	Operational Issues
	- Critical Issues (0),  Major Issues (1),  Minor Issues (4),  Small Defects (0),  Continuous Improvement (0)
	Images from 2025-01-20 >>
	Current/Latest Session >>

Session Aims & Highlights (2025-01-20)

Main aims

1. Targets.  Acquire images of a selection of variable stars, nearby stars, comets & deep sky targets as allowed by sky conditions.

 Equipment & Software

• 12" LX200 and ST-10XME
• 80mm APO and ZWO ASI 178MC
• TheSky6, CCDSoft5, DeviceHub 7.0.3 RC3.4694, MeadeGeneric 1.3.12.522 , UPB 1.5,  Pulsar Dome Firmware 1.51 & Driver 6.3.10
• Pegasus UPB & Motor Focus, TCF-S 6.0.6
• SharpCap 3.2, PHD2  2.6.13, Siril 1.0.6, deCONZ 1.18.2, ASCOM 7.0.3 RC3
• [ DomeLink 1.0 / 1.1 - not used ]
   
• AstroMain 3.73.11,  AstroPlan 1.42.6,  AstroShCap 1.3.15, AstroGuard 1.10.6   (any new items or versions in bold)
  AstroVOE 1.15.1,  AstroSN 1.14,  AstroProtect 1.10.6,  AstroMove 1.14   
  AstroWeather 1.27.3, AstroAllSky 3.22.2,  AstroObsCam 1.11.4 (not 1.11.5), AstroColorCam 1.2.2, AstroLaunch 1.11.14

Highlights

• Targets. 0/28 planned targets observations were completed. Other targets failed due to i) Camera issues that caused targets to overrun scheduled time slots and not acquire any main frames, and ii) later cloud. Due to expected continuing cloud, the session was ended early.
• CCDCamera. Problems with CCDCamera (not yet understood)
•   Dome. There was no issues opening/closing the shutter during session. Dome closed and parked at end of session Ok

Notes

Summary Plots & Logs

Observing Plan

Observing Result

Dome & Scope Slewing Performance

Slew/Centering Performance

Guiding Performance

Sky Conditions (Locate Frames)

Night Sky Summary Plot Top axis: Sky Brightness at Zenith (in ADU/s) Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)

Actual Weather vs Pre-Session Weather Forecast

Session Event Log

Time     Event Detail 17:43:07 Session Monitoring AutoStart monitoring for Live Session opportunity between 17:43 & 06:14 17:43:11 Session AutoStarting Session autostarting (17:43) 17:43:32   CCDSoft Restarting CCDSoft being restarted (to set AutoSave No.) 17:43:59   Camera1 Connected SBIG Camera connected (Set point -20°C) 17:44:01 Session Created Session Created (Live, 2025-01-20 S01298, ImageSaveNum: 1298001) 17:44:06   Scope Switched On Telescope Power has been switched on via UPB Powerbox. 17:44:09   CcdCamera Frozen CcdCamera appears to be frozen at around 17:44 in section 'GetCameraProperties / FilterIndexZeroBased' 17:45:52   Services Started Observatory Services started 17:45:58   CCDSoft Restarting CCDSoft being restarted (attempt to fix frozen CcdCamera) 17:46:05   SBIG Camera Reset SBIG Camera being power-cycled 17:46:59   Camera1 Connected SBIG Camera connected (Set point -20°C) 17:47:01 Observatory (Auto) Observatory placed in Fully-Automated Mode 17:47:04 Session Pending Session pending (2025-01-20) 17:47:06 Session Initiating Session initiating (2025-01-20) 17:47:13   Camera1 Connected SBIG Camera connected (Set point -25°C) 17:47:20   Plan Requested Observing Plan requested from AstroPlan (1.42.6) 17:48:08   Plan Loaded Observing Plan loaded to queue (Plan ID: 1081) 17:48:47   Telescope Connected Telescope connected (TheSky6) 17:49:11 Session Equilibration Session ready to Open Dome 17:49:57   Dome Opened Dome opened (Opening time 46s, Zigbee 40s) 17:50:21 Session Running Session running 17:50:24   Queue Started Observing Queue started (28 targets selected) 17:50:28     Target Started (NrZen) Target started (Focus Field 2, HIP 9508) 17:50:42       Dome Unparked Dome unparked 17:58:58       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 18:02:03       Focusing Completed Foc1 AutoFocus Completed (Profile No 1, wide) 18:04:26       Focusing Completed Foc1 AutoFocus Completed (Profile No 1) 18:04:28       Focusing Started-Foc2 Foc2 Focusing Started (Secondary Scope, using ShCap) 18:06:40       Focusing Completed Foc2 AutoFocus Completed (Profile No 2, wide) 18:08:12       Focusing Failed Foc2 focusing failed - star lost 18:08:16     Target Completed (NrZen) Target completed (Focus Field 2, HIP 9508) 18:08:21     Target Started (1/28) Target started (1/28, AT2025la) 18:14:34       SoftSuspend Called Soft Suspend called due to Deteriorating Conditions (too few stars) 18:14:54   Critical Cloud Alert Critical Cloud Alert (Obs.Manager will close the Dome) 18:14:56   HardSuspend Called Hard Suspend called due to Critical Conditions (cloud) 18:14:58     Target Aborted (1/28) Target aborted (1/28, AT2025la) due to cloud 18:15:00   Queue Paused Queue paused for Session Suspension 18:15:02 Session Suspending Session suspending 18:15:50   Dome Closed Dome closed (Closing time 50s, Zigbee 43s) 18:15:52 Session Suspended Session suspended 18:23:04     Target Cancelled Target cancelled (1/28, AT2025la) due to cloud 18:24:31 Session Resuming Session resuming 18:25:15     Target Started (NrZen) Target started (Focus Field 2, HIP 9508) 18:25:18   Dome Opened Dome opened (Opening time 45s, Zigbee 40s) 18:25:20 Session Running Session running 18:25:22   Queue Resumed Observing Queue resumed 18:32:32       Focusing Started-Foc2 Foc2 Focusing Started (Secondary Scope, using ShCap) 18:34:44       Focusing Completed Foc2 AutoFocus Completed (Profile No 3, wide) 18:36:36       Focusing Completed Foc2 AutoFocus Completed (Profile No 3) 18:36:55     Target Completed (NrZen) Target completed (Focus Field 2, HIP 9508) 18:36:59     Target Started (2/28) Target started (2/28, AT2025oj (CV)) 18:46:46       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 18:49:18       Focusing Completed Foc1 AutoFocus Completed (Profile No 4) 18:54:49     Target Failed (2/28) Target failed due to (2/28, AT2025oj (CV)) 18:54:51     Target Missed (3/28) Target's time slot was missed (3/28, Gliese 65) 18:54:53     Target Started (4/28) Target started (4/28, 49P/Arend-Rigaux) 19:12:51     Target Failed (4/28) Target failed due to (4/28, 49P/Arend-Rigaux) 19:12:55     Target Started (5/28) Target started (5/28, AT2025md (CV)) 19:28:31     Target Failed (5/28) Target failed due to (5/28, AT2025md (CV)) 19:28:35     Target Started (6/28) Target started (6/28, AT2025lc (CV)) 19:42:32     Target Failed (6/28) Target failed due to (6/28, AT2025lc (CV)) 19:42:36     Target Started (7/28) Target started (7/28, AT2025il (CV)) 20:00:57     Target Failed (7/28) Target failed due to (7/28, AT2025il (CV)) 20:00:59     Target Missed (8/28) Target's time slot was missed (8/28, GCVS HH And) 20:01:01     Target Started (9/28) Target started (9/28, GCVS BL Lac) 20:14:13     Target Failed (9/28) Target failed due to (9/28, GCVS BL Lac) 20:14:15     Target Missed (10/28) Target's time slot was missed (10/28, WDS GRB 34) 20:14:17     Target Started (11/28) Target started (11/28, UGC 1325 w/SN2024aedt) 20:23:08       SoftSuspend Called Soft Suspend called due to Deteriorating Conditions (too few stars) 20:26:24   Critical Cloud Alert Critical Cloud Alert (Obs.Manager will close the Dome) 20:26:26   HardSuspend Called Hard Suspend called due to Critical Conditions (cloud) 20:26:28     Target Aborted (11/28) Target aborted (11/28, UGC 1325 w/SN2024aedt) due to cloud 20:26:30   Queue Paused Queue paused for Session Suspension 20:26:32 Session Suspending Session suspending 20:27:19   Dome Closed Dome closed (Closing time 49s, Zigbee 43s) 20:27:21 Session Suspended Session suspended 20:31:44     Target Cancelled Target cancelled (11/28, UGC 1325 w/SN2024aedt) due to cloud 20:54:04     Target Cancelled Target cancelled (12/28, UGCA 77) due to cloud 21:16:24     Target Cancelled Target cancelled (13/28, UGCA 87) due to cloud 21:31:23     Target Cancelled Target cancelled (14/28, GCVS RR Tau) due to cloud 21:51:28     Target Cancelled Target cancelled (15/28, AT2022udd) due to cloud 21:58:55   Camera1 Connected SBIG Camera connected (Set point -25°C) 22:09:48     Target Cancelled Target cancelled (16/28, NGC 2170) due to cloud 22:16:05 User Intervention User request to close and finish session 22:16:08   Queue Aborted Job Queue aborted 22:16:10 Session Closing Session closing 22:16:15   Dome Closed Dome already closed (closed at 20:27, closing time 49s, zigbee 43s) 22:17:31   Dome Parked Dome parked (parking time 67s), Az: 90.0 deg 22:18:16   Telescope Parked Telescope parked (parking time 37s total) 22:18:46   Telescope State Handbox reads 'Scope parked Turn scope off.' 22:19:08   Telescope Switched Off Telescope Power has been switched off via UPB Switch. 22:20:33   Services Stopped Night Services stopped 22:20:36 Session Finishing Session Finishing started (Create Fits Summary, Transfer Files) 22:20:43     Dome (Find Park) Find Park started (Search Az 88.0 to 92.0°, Step 0.2°, Narrow) 22:21:31     Dome Find Park found (Best Park Az 90.4°) 22:21:37     Dome Dome has been synced at park position (Az 90.0°, Adjustment: -0.4°) 22:21:39   Dome Parked Dome parked 22:21:56   Dome Charging Dome is parked and re-charging Ok (15mA) 22:21:58 Session Finished Session Finished

Session Alerts & Alarms

Time     Type       Name Detail 17:45:56 Red Alert CcdCamera CcdCamera appears to be frozen (last cycle at 17:44:09) in section 'GetCameraProperties / FilterIndexZeroBased' 21:36:51 Red Alert Dome Alert Shutter Data isn't being refreshed 21:36:53 Yellow Alert Dome Shutter Pulsar Shutter appears to be asleep (last fresh data was at 21:16:49) 21:39:32 Green Clear Dome Shutter Pulsar Shutter has woken up (fresh data at 21:39:20, outage 22.7 min) 21:39:36 Green Clear Dome Alert Shutter Data has resumed updating 22:21:56 Green Clear Dome Charging Dome is parked and re-charging Ok (15mA)

Back to Top

Operational Issues (2025-01-20 S1298 Attempt)

• Critical Issues (0)
• Major Issues (1)
• Minor Issues (4)
• Small Defects (0)
• Continuous Improvement (0)
  

[ Prev | Next ]

Critical Issues

• None

Major Issues

• Multiple targets encountered timeouts during 'Wait for CCD Cooler' tasks, leading to delays of up to 10 mins, causing main images to be cancelled t. Targets 2/28, 4/28, 5/28, 6/28, 7/28 & 9/28 failed due to all planned main images being cancelled to prevent target overrunning its scheduled time slot.  Other targets failed as there time slots was missed.  Investigating showed that Wait On Cooler routine was executed before centering and again before main imaging. In each case the process that waited for CCD Temperature to stablise timed-out after 5 minutes, causing a 10 minute delay for each target. The CCD Temperture chart was missing CCD Temperature/Cooler lines.   The Session Monitor file was missing CCD Temperature/Cooler Data. Problem began earlier.  During NearZenith Focusing, the WaitOnCooler operation was run at 17:51 with the message "Waiting for CCD Temperature (-26.3) to stabilise at new set point (-25.0°C)", yet timed out after 5 minutes (17:56).  There is no indication that CCDCamera is frozen or that CCD Temperature can't be read. CCD Temperature values stored in Locate Frames show that CCD Temperature stabilised at -25.0°C (-24.9 to 25.4). It is hypothesized that the thread/process that monitor's CCD Temperature wasn't running or had stalled or was otherwise not functioning correctly.  ObsMonitor.RunTempMonitor  was reported to be 'Already Running' at  17:43, 17:44, 17:46.  The Wait On Cooler task calls CheckForStableCCDTemperature() routine, and if CcdMonitor.CcdTempIsAcceptable <> true it enters a wait loop and waits for CcdTempIsAcceptable = True or times-out after 5 minutes.  CcdTempIsAcceptable is set by CcdMonitor.MainLoop().  Provided that bObservatoryMonitoringOn is True , the ObsOverseer checks the status of the CcdMonitor.MainLoop and raise alert if it is deemed to be pending or frozen. On this occasion it raised no such alert.     Problems with WaitOnCooler have occasionally been seen before, but the cause and required fix couldn't be found.  The CcdTempIsAcceptable check is only performed by CCDMonitor.MainLoop when  bRestartingCCDSoft = False AndAlso IsNothing(Camera) = False  and gCameraConnected = True and And Camera1_IsAccessible().   It is likely that either bRestartingCCDSoft hadn't been set to false after earlier RestartCCDSoft operation or gCameraConnected hadn't been set to True after earlier RestartCCDSoft operation, or had otherwise turned False, as these are the same situations in which CCDTemp & Cooler Power aren't being written to MonitorFile or drawn on the CCDTemp Chart.  Issue is likely a one-off glitch.  The reason for underlying issue that was causing CCDCooling to not be monitored isn't clear, however a partial workaround has been added so the CCDMonitor.MainLoop sets a bMonitoringCCDCooling flag according to whether it is or is not monitoring CCD Temp/Cooling and this flag is used by CheckForStableCCDTemperature() to see whether it is appropropriate to enter and stay-in the wait loop or to bypass the wait loop. Diagnostics are dumped to log file in the event that bMonitoringCCDCooling is unexpectedly false.
  Partial fix implemented 2025-01-21 (AstroMain 3.73.12).

Minor Issues

• CCDCamera red alert was raised with message "CcdCamera appears to be frozen" (17:45).  A CCD Camera' red alert was raised at 17:45:56 by the ObsOverseer with the message "CcdCamera appears to be frozen (last cycle at 17:44:09) in section 'GetCameraProperties / FilterIndexZeroBased' ".  This occurred during or shortly after Observatory Services were started to connect Night Services (Telescope, PHD2, SharpCap), and just after 'CCDTempChart' was started . ObsOverseer recognised the problem and a RestartCCDSoft operation was run (including restarting SBIG Camera).  Camera reconnection appears to have been successful.  CCD Set Point Temperture was initially set to -20°C but changed shortly after to -25°C .    Session Log shows "Camera has been connected " at 17:44:06.81, followed by "Temperature Regulation is ON" and  "CCD Temperature Set Point has changed to -20.0 degC" (17:44:06.98), and was then followed by a series of messages like "CcdCamera is Pending (7s) '  in section 'GetCameraProperties / FilterIndexZeroBased' .  These occured whilst Observatory Services was being started and specifically whilst waiting 60s after Start Telescope Power to connect Telescope.  At 17:45:56 the logs shows CcdCamera Alert with message "CcdCamera is Frozen (107s) in section 'GetCameraProperties / FilterIndexZeroBased' ", and is followed by RestartCCDSoft() operation. 
  Resolved in-session 2025-01-20 (AstroMain ObsOverSeer)
   
• Task in SlewToTargetRaDec/PostSlewCalcs reported to be taking a long time (18:09). ObsOverseer generated a log line at 18:09:49 during AT2025la, with the message "JobQueue  Info | Task in SlewToTargetRaDec/PostSlewCalcs is taking a long time?" . Similar messages logged at 18:38:22.25 (during AT2025oj),  18:56:14.09 (during 49P/Arend-Rigaux),   19:14:31.12 (during AT2025md), 19:29:42.74 (during AT2025lc) and during other targets.   Unclear if this is due to a bug introduced in AstroMain 3.73.11 or a consequence of another issue such as the CCDTemp Monitor.  It is noteable that all the messages occurred whilst JobQueue was waiting in vain for 5 minutes (ie to a 5 min timeout) whilst CCD Temperature stablised.  It is probable that the execution had finished the SlewToTargetRaDec() routine on the 'Last Alive' message referring to SlewToTargetRaDec/PostSlewCalcs is a red-herring.   Fixed in AstroMain 3.73.12,  by adding a  IamAlive(ThisRoutine(), "Continue", "", 0)  statement in GotoTarget() routine after finishing the call to SlewToTargetRaDec(), and by adding  IamAlive(ThisRoutine(), "WaitLoop", "", 5 * 60) before entering wait loop in CheckForStableCCDTemperature(T,Mode) routine.
  Fixed 2025-01-21 (AstroMain 3.73.12)
   
• Multiple targets failed without a stated reason (18:54 - 20:14). Targets 2/28, 4/28, 5/28, 6/28, 7/28 & 9/28 failed without giving a stated reason (reason was 'blank').  In most cases this seems to be due to a CCDCamera/Camera Monitor issue whereby CCD Temperature couldn't be correctly obtained, and there were 5 minutes delays whilst trying to start imaging, causing Main Images to be cancelled to prevent target from overrunning its time slot.  This issue is focused on the target observations having a blank / unstated reason.  Separate ticket(s) address the problem that held up each target.  In each case the Failure Reason should be 'NoImages'. Fixed in AstroMain AstroMain 3.73.12, 'NoImages' (36) is given as the stated reason at end of ExecuteTarget if gNumCompletedImages = 0.   ExceededTimeSlot = 37 is given as stated reason if end of time slot has been past before entering TargetImage routine.
  Fixed 2025-01-21 (AstroMain 3.73.12)
   
• Monitor File is missing CCD Temperature / Cooler Power data throughout session. CCD Temperature / Cooler Power data is similarly missing from the Session's tab-delineated Monitor File, the respective column fields are blank.  It is conjectured that issue is result of same problem that caused  'WaitOnCooler' routine to continuously TimeOut.  Cooler appears to be running ok. Camera's CCD Temperature appears to be available ok at 17:43 (8.4°C), 17.46 (-6.9°C), 17:47 (-10.6°C ), 17:50 (-25.4°C, 94% cooling power).  Records are written to the Monitor File by the Services.RunTempMonitor() routine. This routine was obviously running ok as Monitor records are being written every 2 minutes and other column fields have values as expected. Blank Cam1.CCDtemp & Cam1.CoolerPower fields are written when 'CCDtemp' and 'CoolerPower' (local variables) have value vcNull (-99).  These would have had null values if the following condition gCameraConnected  = False or bRestartingCCDSoft  = True.  This might would indicate that either i) gCameraConnected wasn't never set to True or had been set to False when RestartCCDsoft operation was run and not set to True afterwards, and/or ii) bRestartingCCDsoft flag was set back to false following the completion of the RestartCCDsoft operation.
  Issue understood 2025-01-21 (AstroMain 3.73.12)
   
• CCD Temperature chart is missing CCD Temperature / Cooler Power data. CCD Temperture, Set Point Temperature and Cooling lines are missing from Session's CCD Temperature Graph / Chart (CcdTemp.20250120.png). CCD Temperature / Cooler Power data is similarly missing from the Session's Monitor File. Shortly after 'CCDTempChart' was started the ObsOverseer raised a CCDCamera red alert was raised with message "CcdCamera appears to be frozen" at 17:45, which might be relevant.  Cooler appears to be running ok. Camera's CCD Temperature appears to be available ok at 17:43 (8.4°C), 17.46 (-6.9°C), 17:47 (-10.6°C ), 17:50 (-25.4°C, 94% cooling power).  It is conjectured that issue is result of same underlying problem that caused  'WaitOnCooler' routine to continuously TimeOut, and caused the output of blank CCDTemp & Cooler Power columns to the Monitor File. Lines are drawn by CCDTempChart.AddData() routine which is called from Services.RunTempMonitor() which is the same routine that writes data to the Monitor File, and therefore this demonstrates a direct linkage between the two issues.  Ticket closed 2025-01-21, leaving open the ticket for the MonitorFile issue.
  Issue understood 2025-01-21 (AstroMain 3.73.12)
   

Small Defects

• None

Continuous Improvement

• None

[ Prev | Next ]

[ Prev | Next ]

Back to Top