David's Astronomy Pages
Notes - Session 1037 (2022-09-14)

 
Bullet Session Aims & Highlights
 - Observing Result
 - Night Summary Plot
 - Session Event Log
Bullet Operational Issues
  - Critical Issues (0),  Major Issues (1),  Minor Issues (3),  Small Defects (2),  Continuous Improvement (1)
 
Bullet Images from 2022-09-14 >>        [ Local Files >> ]    
   
Bullet PHD2 Update 2.6.11dev2
Bullet Investigation - PHD2 AutoFind Anomalies
   

Session Aims & Highlights (2022-09-14)

Main aims

  1. Targets.  Acquire images of a selection of variable stars, nearby stars, comets & deep sky targets as allowed by time & twilight conditions.

 Equipment & Software

Highlights

Notes:

Summary Plots & Logs

Observing Plan
Image
  
Observing Result
Image
   
Dome & Scope Slewing Performance
Image
  
Slew/Centering Performance
Image
  
Guiding Performance
Image
Image
  
Sky Conditions (Locate Frames)
Image
  
Night Sky Summary Plot
Top axis: Sky Brightness at Zenith (in ADU/s)
Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)

Image   
  
Actual Weather vs Pre-Session Weather Forecast
Image
Image   
  
Session Event Log
Time     Event Detail
23:50:16 Session Monitoring AutoStart monitoring for Live Session opportunity between 23:50 & 04:35
23:50:19 Session AutoStarting Session autostarting (23:50)
23:50:51 Session Created Session Created (Live, 2022-09-14 S01037, ImageSaveNum: 1037001)
23:50:54   Scope Switched On Telescope Power has been switched on via UPB Powerbox.
23:52:40   Services Started Observatory Services started
23:52:47 Observatory (Auto) Observatory placed in Fully-Automated Mode
23:52:49 Session Pending Session pending (2022-09-14)
23:52:51 Session Initiating Session initiating (2022-09-14)
23:52:53   Plan Requested Observing Plan requested from AstroPlan (1.32.5)
23:53:32   Plan Loaded Observing Plan loaded to queue (Plan ID: 799)
23:53:54   Camera1 Connected SBIG Camera connected (set point -15°C)
23:54:01   Telescope Connected Telescope connected (TheSky6)
23:54:25 Session Equilibration Session ready to Open Dome
23:55:10   Dome Opened Dome opened (opening time 45s)
23:55:56 Session Running Session running
23:55:58   Queue Started Observing Queue started (16 targets selected)
23:56:01     Target Started (NrZen) Target started (Focus Field 23, HIP 113729)
23:56:15   Dome Unparked Dome unparked
23:59:51   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
00:00:23 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 00:00 in section Update Foc1 Fields
00:00:51   Critical RainThreat Alert Critical Rain Threat Alert (Obs.Manager will close the Dome)
00:00:53   HardSuspend Called Hard Suspend is called due to Critical Conditions (threat of rain)
00:00:57 Job Queue Frozen Job Queue appears to have frozen at around 00:00 - 00:00
00:02:52 Session Suspended Session suspended
00:03:36   Dome Closed Dome closed (closing time 45s)
00:09:01   CCDSoft Restarting CCDSoft being killed and restarted in attempt to fix locked threads
00:24:07 Session Resumed Session Resumed (Live, 2022-09-14 S01037, ImageSaveNum: 1037002)
00:24:36   Obs.Manager Started Obs.Manager started
00:24:38   Obs.Overseer Started Obs.Overseer started
00:24:42   Services Started Observatory Services started
00:25:29   Services Started Observatory Services started
00:26:47   Services Started Observatory Services started
00:27:38 Session Monitoring AutoStart monitoring for Live Session opportunity between 00:27 & 04:35
00:27:40 Observatory (AutoStart) Observatory placed in Auto-Start Mode
00:27:42 Session AutoStarting Session autostarting (00:27)
00:28:21   Services Started Observatory Services started
00:28:28 Observatory (Auto) Observatory placed in Fully-Automated Mode
00:28:32 Session Pending Session pending (2022-09-14)
00:28:34 Session Initiating Session initiating (2022-09-14)
00:28:36   Plan Loaded Observing Plan loaded to queue (Plan ID: 799)
00:28:48   Camera1 Connected SBIG Camera connected (set point -15°C)
00:28:54   Telescope Connected Telescope connected (TheSky6)
00:29:13 Session Equilibration Session ready to Open Dome
00:29:58   Dome Opened Dome opened (opening time 45s)
00:30:00 Session Running Session running
00:30:02   Queue Started Observing Queue started (15 targets selected)
00:30:05     Target Started (NrZen) Target started (Focus Field 23, HIP 113729)
00:36:50       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
00:39:39 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 00:39 in section Monitor Camera1
00:41:20 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 00:41 in section Monitor Camera1
00:42:00 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 00:42 in section Monitor Camera1
00:42:40 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 00:42 in section Monitor Camera1
00:44:14   CCDSoft Restarting CCDSoft being killed and restarted in attempt to fix locked threads
00:44:28       Focusing Aborted Foc1 Focusing Aborted
00:44:30     Target Aborted (NrZen) Target aborted (Focus Field 23, HIP 113729) due to Unknown Reason
00:44:32   Queue Aborted Job Queue aborted
00:44:44   Camera1 Connected SBIG Camera connected (set point -10.0212248083951°C)
00:47:50   Camera1 Connected SBIG Camera connected (set point -15°C)
00:48:22 Session Monitoring AutoStart monitoring for Live Session opportunity between 00:48 & 04:35
00:48:24 Observatory (AutoStart) Observatory placed in Auto-Start Mode
00:48:27 Session AutoStarting Session autostarting (00:48)
00:48:48   Services Started Observatory Services started
00:48:53 Observatory (Auto) Observatory placed in Fully-Automated Mode
00:48:57 Session Pending Session pending (2022-09-14)
00:48:59 Session Initiating Session initiating (2022-09-14)
00:49:01   Plan Loaded Observing Plan loaded to queue (Plan ID: 799)
00:49:13   Camera1 Connected SBIG Camera connected (set point -15°C)
00:49:19   Telescope Connected Telescope connected (TheSky6)
00:49:38 Session Running Session running
00:49:40   Queue Started Observing Queue started (14 targets selected)
00:49:42     Target Started (NrZen) Target started (Focus Field 23, HIP 113729)
00:56:34       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
00:59:20       Focusing Completed Foc1 AutoFocus Completed (Profile No 1, wide)
01:01:17       Focusing Completed Foc1 AutoFocus Completed (Profile No 1)
01:01:19       Focusing Started-Foc2 Foc2 Focusing Started (Secondary Scope, using ShCap)
01:03:28       Focusing Completed Foc2 AutoFocus Completed (Profile No 2, wide)
01:05:11       Focusing Completed Foc2 AutoFocus Completed (Profile No 2)
01:05:28     Target Completed (NrZen) Target completed (Focus Field 23, HIP 113729)
01:05:30     Target Missed (3/16) Target's time slot was missed (3/16, GCVS BL Lac)
01:05:32     Target Started (4/16) Target started (4/16, AT2022kl (CV))
01:08:40       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
01:10:39       Focusing Completed Foc1 AutoFocus Completed (Profile No 3)
01:15:10   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
01:16:15   SoftSuspend Cancelled Soft Suspend is cancelled due to improved conditions
01:20:01     Target Completed Target partially completed (4/16, AT2022kl (CV))
01:20:05     Target Started (5/16) Target started (5/16, GCVS HH And)
01:21:29     Dome Slew Dome slew held up at Az. 222.4° (Delay 52s)
01:25:21     Target Completed Target completed (5/16, GCVS HH And)
01:25:25     Target Started (6/16) Target started (6/16, WDS GRB 34)
01:28:06       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
01:30:05       Focusing Completed Foc1 AutoFocus Completed (Profile No 4)
01:32:07     Target Completed Target completed (6/16, WDS GRB 34)
01:32:11     Target Started (7/16) Target started (7/16, UGC 12005 w/SN2022ovb)
01:36:13       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
01:38:25       Focusing Completed Foc1 AutoFocus Completed (Profile No 5)
01:54:23     Target Completed Target completed (7/16, UGC 12005 w/SN2022ovb)
01:54:27     Target Started (8/16) Target started (8/16, AT2022tru (CV))
01:55:46     Dome Slew Dome slew held up at Az. 222.5° (Delay 60s)
01:58:12       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
02:15:21     Target Completed Target completed (8/16, AT2022tru (CV))
02:15:25     Target Started (9/16) Target started (9/16, NGC 7642 w/AT2022df)
02:16:45     Dome Slew Dome slew held up at Az. 223.1, 222.6° (Total delay 56s)
02:19:17       Focusing Skipped Foc1 focusing skipped - unable to find a star (TCF-S)
02:33:35   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
02:35:15     Target Completed Target completed (9/16, NGC 7642 w/AT2022df)
02:35:17   Queue Paused Queue paused for Session Suspension
02:35:20 Session Suspended Session suspended
02:36:10   Dome Closed Dome closed (closing time 50s)
02:48:01 Session Resuming Session resuming
02:48:46   Dome Opened Dome opened (opening time 45s)
02:48:49 Session Running Session running
02:48:51   Queue Resumed Observing Queue resumed
02:48:52     Target Started (10/16) Target started (10/16, NGC 157 w/SN2022jli)
02:54:07   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
03:01:03   SoftSuspend Cancelled Soft Suspend is cancelled due to improved conditions
03:02:14     Target Completed Target partially completed (10/16, NGC 157 w/SN2022jli)
03:02:18     Target Started (11/16) Target started (11/16, LEDA 1352843 w/AT2021acbu)
03:05:32       Focusing Skipped Foc1 focusing skipped - star is too dim (TCF-S)
03:19:55   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
03:21:45     Target Completed Target completed (11/16, LEDA 1352843 w/AT2021acbu)
03:21:47   Queue Paused Queue paused for Session Suspension
03:21:49 Session Suspended Session suspended
03:22:31   Dome Closed Dome closed (closing time 45s)
03:30:17 Session Resuming Session resuming
03:31:02   Dome Opened Dome opened (opening time 45s)
03:31:04 Session Running Session running
03:31:06   Queue Resumed Observing Queue resumed
03:31:08     Target Started (12/16) Target started (12/16, AT2022oar)
03:46:44   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
03:47:19     Target Completed Target partially completed (12/16, AT2022oar)
03:47:21   Queue Paused Queue paused for Session Suspension
03:47:24 Session Suspended Session suspended
03:48:14   Dome Closed Dome closed (closing time 50s)
03:51:34 Session Resuming Session resuming
03:52:19   Dome Opened Dome opened (opening time 45s)
03:52:21 Session Running Session running
03:52:23   Queue Resumed Observing Queue resumed
03:52:26     Target Started (13/16) Target started (13/16, Nova Per 2020)
04:01:19     Target Completed Target partially completed (13/16, Nova Per 2020)
04:01:23     Target Started (14/16) Target started (14/16, AT2022tqu)
04:03:52       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
04:05:26       Focusing Failed Foc1 focusing failed - star lost
04:08:56   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars)
04:22:10     Target Completed Target completed (14/16, AT2022tqu)
04:22:12   Queue Paused Queue paused for Session Suspension
04:22:13 Session Suspended Session suspended
04:22:58   Dome Closed Dome closed (closing time 45s)
04:28:34 Session Resuming Session resuming
04:29:19   Dome Opened Dome opened (opening time 45s)
04:29:21 Session Running Session running
04:29:23   Queue Resumed Observing Queue resumed
04:29:26     Target Started (15/16) Target started (15/16, UGC 3923 w/SN2022rqc)
04:46:06     Target Completed Target partially completed (15/16, UGC 3923 w/SN2022rqc)
04:46:10     Target Started (16/16) Target started (16/16, AT2022tmq)
05:05:46     Target Completed Target completed (16/16, AT2022tmq)
05:05:49   Queue Completed Job Queue completed
05:05:51 Session Closing Session closing
05:06:39   Dome Closed Dome closed (closing time 45s)
05:06:58   Dome Parked Dome parked (parking time 12s), Az: 90.0 deg
05:08:16   Telescope Parked Telescope parked (parking time 76s)
05:08:31   Telescope State Scope parked Turn scope off. (Handbox)
05:08:53   Telescope Switched Off Telescope Power has been switched off via UPB Switch.
05:09:46   Services Stopped Night Services stopped
05:09:48 Session Housekeeping Session Housekeeping started (Create Fits Summary, Transfer Files)
05:09:59 Session Finished Session Finished
 
Session Alerts
Time     Alert Detail
00:00:55 Obs.Monitor Obs.Monitor appears to be frozen in section Update Foc1 Fields (last cycle at 00:00:23)
00:11:01 Job Queue Job Queue appears to be frozen (last IamAlive message at 00:00:27)
00:40:13 Obs.Monitor Obs.Monitor appears to be frozen in section Monitor Camera1 (last cycle at 00:39:39)
00:40:23 Obs.Monitor Obs.Monitor has resumed again after 0.7 mins
00:41:53 Obs.Monitor Obs.Monitor appears to be frozen in section Monitor Camera1 (last cycle at 00:41:20)
00:42:03 Obs.Monitor Obs.Monitor has resumed again after 0.7 mins
00:42:33 Obs.Monitor Obs.Monitor appears to be frozen in section Monitor Camera1 (last cycle at 00:42:00)
00:42:43 Obs.Monitor Obs.Monitor has resumed again after 0.7 mins
00:43:14 Obs.Monitor Obs.Monitor appears to be frozen in section Monitor Camera1 (last cycle at 00:42:40)
00:43:34 Obs.Monitor Obs.Monitor has resumed again after 0.8 mins
 

Back to Top


Operational Issues (2022-09-14, S1037)

[ Prev | Next ]

Critical Issues

Major Issues

Minor Issues

Small Defects

Continuous Improvement

[ Prev | Next ]

Back to Top


PHD2 Update 2.6.11dev2

PHD2 application was updated on Observatory and Development Computers today (2022-09-14) with the new 2.6.11dev2 development version (dated 30 Aug 2022).  This was after temporarily dropping back from 2.6.11 to 2.6.10dev3 on 2022-09-01 as part of trying to understand/fix an issue with PHD2's Auto-Find star.  This new version 'reduces CPU utilization when rendering the camera image on the screen'.  The new version was used for the first time during the S1037 Session (2022-09-14).

PHD2 Development Snapshots - https://openphdguiding.org/development-snapshots/

Back to Top


Investigation - PHD2 AutoFind Anomalies

Issue
Poor guiding on some targets due to the selected Guide Star being either too dim (SNR only 15) or being probably just a random patch of warm pixels (SNR only 6 or 7), leading to the guide star being lost and causing trailed star images.  The guide star is selected even though there are clearly better (higher SNR) candiate stars in the image (and within the specified ROI).

Description
This particular issue seemed to appear a few weeks ago, and is seen in the 3 different versions of PHD2 tried in this period :  2.6.10dev3, 2.6.11 & 2.6.11dev2.

Session is run by automated program control. For each target guiding is initially via PHD2's OpenGuiding Server using the "guide" method. A ROI region is specified (but this has only relatively recently added)

  {"method": "guide", "params": [{"pixels": 2.0, "time": 10, "timeout": 30}, false, [341, 229, 2415, 1622] ], "id": 151}

The method starts looping exposures, performs 'autofind' to select a guide star, and begins guiding. The method returns a 'SettleDone' event either when guiding has settled or the when the method timesout without guiding being settled.

PHD2 Debug Log shows that the

02:14:53.962 00.004 11704 Star::AutoFind called with edgeAllowance = 0 searchRegion = 30 roi = 2415x1622@341,229
02:14:53.963 00.001 11704 AutoFind: using ROI 2415x1622@341,229

Event Monitoring wiki page on PHD2's GitHub site gives the following the following example Guide Method call:

{"method": "guide", "params": {"settle": {"pixels": 1.5, "time": 8, "timeout": 40}, "recalibrate": false, "roi": [20, -50, 1280, 960]}, "id": 42}

I had problems with the method being accepted if I include "roi":  so I exclude the roi keyword from the call. The ROI specification is nevertheless accepted by PHD2 as can be seen by PHD2 Debug Log which shows :

02:14:53.962 00.004 11704 Star::AutoFind called with edgeAllowance = 0 searchRegion = 30 roi = 2415x1622@341,229
02:14:53.963 00.001 11704 AutoFind: using ROI 2415x1622@341,229

If I use the "settle", "recalibrate" and "roi" keywords like the wiki example

{"method": "guide", "params": ["settle": {"pixels": 2.0, "time": 10, "timeout": 30}, "recalibrate": false, "roi": [341, 229, 2415, 1622] ], "id": 43}

PHD2 returns an error

"error":{"code":-32700,"message":"invalid JSON request: Unexpected character on line 1 at \": {\"pixels\":...\""}

But PHD2 does accept the following ok "[  ]" brackets for params keyword replaced by "{ }" bracket.

{"method": "guide", "params": {"settle": {"pixels": 2.0, "time": 10, "timeout": 30}, "recalibrate": false, "roi": [341, 229, 2415, 1622] }, "id": 87}

When the issue with 'selected guide stars having only SNR 6 or 7' was first noticed, the Minimum SNR was set to higher and higher values in an attempt to 'discourage PHD2 from choosing a guide star that is known to be too faint for sustained guiding'.  Value was eventually raised to 50. This appeared to work for some targets but not others. Hypothesis was that if the exposure contained a star with SNR > 50 it was picked ok, but if it didn't then PHD2 would typically ignore stars with SNR 20-49 and just choose one with a SNR of only 6 or 7 instead.  Based on this hypothesis Minimum SNR was then reduced down to a value of 13 for sessions S1036 and S1037.   However this didn't stop the problem from sometime occuring, with either a star with SNR  only 6 or 7 being picked (clearly lower than 13) or one being picked with SNR of 16 that was higher than the Minimum SNR of 13 but not higher enough that the star was readily lost after 2 or 3 minutes.   In all these case better candidate (higher SNR, non-saturated) guide stars are clearly present in the exposures. 

Whilst changing the Minimum SNR to a new value of 20 was considered it is noticed that the position of guide star auto-selected is nearly always situated on or very close to the border of specified ROI and is rarely chosen within the central part of the ROI where the better candidate stars are often present

Looking at the position of the guide star that PHD2 has auto-selected (green box) for each of the guide runs during the last observing session it can be seen that the guide star is located on or very close to the ROI border (white rectangle) in 12 out of the 14 guide runs.  A more centrally positioned guide star has been selected in just 2 guide runs (For Info, The yellow rectangle shows the FOV of the Main Imaging Camera, the Blue Circle is the target object)


Image Guide Star SNR 56

Image Guide Star SNR 633

Image Guide Star SNR 442

Image Guide Star SNR 6

Image Guide Star SNR 17

Image Guide Star SNR 35

Image Guide Star SNR 33 (guide star of SNR 6 initially selected)

Image Guide Star SNR 13

Image Guide Star SNR 148

Image Guide Star SNR 18

Image Guide Star SNR 197

Image Guide Star SNR 6

Back to Top