David's Astronomy Pages
Notes - Session 940 (2021-12-10)

 
Bullet Session Aims & Highlights
 - Observing Result
 - Night Summary Plot
 - Session Event Log
 
Bullet Operational Issues
  - Critical Issues (1),  Major Issues (5),  Minor Issues (11),  Small Defects (1),  Continuous Improvement (3)
 
Bullet Images from 2021-12-10 >>         [ Local Files >> ]
Bullet First Session with new Observatory Computer
Bullet Investigation - Guiding Runaway (SideOfPier Issue)
2021-12-12
Bullet Investigation - LX200 Tracking Rate Discrepency

Session Aims & Highlights (2021-12-10)

Main aims

  1. Observatory Computer.  Test out new observatory computer with freshly installed software and settings
    Collate any teething issues or problems.
  2. Targets.  Acquire images of a selection of variable stars, nearby stars, comets & deep sky targets as allowed by time & conditions.
  3. Wind Exclusion Zone.  Continue to test new procedures that create and use a Azimuthal Exclusion Zone to avoid facing telescope into moderate/strong wind.
  4. Shutter Operations. Test corrected code for managing shutter operations (AstroMain 3.48.2)
  5. AstroMain 3.48.2 Test new feature and bug fixes in AstroMain 3.48.2

Equipment & Software

Highlights

Notes:

  Summary Plots & Logs

Observing Plan
Image
  
Observing Result
Image
   
  
Dome & Scope Slewing Performance
Image
  
Slew/Centering Performance
Poor centering performance in Dec. One rogue jog.
Image
  
Guiding Performance
Reasons for poor guiding :
Black points (3 & 4)   RA Guiding Runaway due to calibration flip associated with inappropriate
setting of SideOfPier by telescope drivers

Image
Image
  
Sky Conditions (Locate Frames)
Sky Brightness measurements are invalid due to a SBIG camera bias level issue (eventually fixed 2022-01-02)
Image
 
Night Sky Summary Plot
Top axis: Sky Brightness at Zenith (in ADU/s)
Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)

Image   
  
Actual Weather vs Pre-Session Weather Forecast
Image
Image   
  
Session Event Log
Time     Event Detail
18:15:01   Camera1 Connected SBIG Camera Connected (set point -15°C)
18:15:03 Session Created Live Session Created (2021-12-10 S00940, ImageSaveNum: 940001)
18:15:17   Scope Switched On Telescope Power has been switched on via UPB Powerbox.
18:16:54   Services Started Observatory Services started
18:17:15   Telescope Connected Telescope Connected (TheSky6)
18:19:39   Dome Opened Dome opened (opening time 46s)
18:52:59 Observatory (Auto) Observatory placed in Fully-Automated Mode
18:53:03 Session Pending Session pending (2021-12-10)
18:53:05 Session Initiating Session initiating (2021-12-10)
18:53:07   Plan Requested Observing Plan requested from AstroPlan (1.29)
18:53:39   Plan Loaded Observing Plan loaded to queue (, Plan ID: 693)
18:53:54   Camera1 Connected SBIG Camera Connected (set point -25°C)
18:53:59   Telescope Connected Telescope Connected (TheSky6)
18:54:18 Session Equilibration Session ready for dome & camera equilibration
18:55:08 Session Running Session running
18:55:11   Queue Started Observing Queue started (40 targets selected)
18:55:14     Target Started (NrZen) Target started (Focus Field 0, HIP 106)
18:56:49 Job Queue Frozen Job Queue appears to have frozen at around 18:56 - 18:56
19:06:28 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 19:06 in section Refresh Dome Picture
19:06:55 Session Suspended Session suspended
19:07:45   Dome Closed Dome closed (closing time 50s)
19:37:54 Session Resumed Live Session Resumed (2021-12-10 S00940, ImageSaveNum: 940001)
19:38:22   Obs.Manager Started Obs.Manager started
19:38:24   Obs.Overseer Started Obs.Overseer started
19:38:28   Services Started Observatory Services started
19:45:32   Services Stopped Observatory Services stopped
19:45:37 Program Closed Program closed by User
20:01:44 Session Resumed Live Session Resumed (2021-12-10 S00940, ImageSaveNum: 940001)
20:07:40   Obs.Manager Started Obs.Manager started
20:07:42   Obs.Overseer Started Obs.Overseer started
20:07:45   Services Started Observatory Services started
20:08:06   Camera1 Connected SBIG Camera Connected (set point -25°C)
20:10:04   Scope Switched On Telescope Power has been switched on via UPB Powerbox.
20:11:44   Services Started Observatory Services started
20:12:03   Telescope Connected Telescope Connected (TheSky6)
20:15:11   Dome Opened Dome opened (opening time 45s)
20:15:31 Observatory (Auto) Observatory placed in Fully-Automated Mode
20:15:33 Session Pending Session pending (2021-12-10)
20:15:35 Session Initiating Session initiating (2021-12-10)
20:15:39   Plan Loaded Observing Plan loaded to queue (, Plan ID: 693)
20:15:52   Camera1 Connected SBIG Camera Connected (set point -25°C)
20:15:57   Telescope Connected Telescope Connected (TheSky6)
20:16:22 Session Equilibration Session ready for dome & camera equilibration
20:16:24 Session Running Session running
20:16:26   Queue Started Observing Queue started (36 targets selected)
20:16:29     Target Started (NrZen) Target started (Focus Field 1, HIP 4911)
20:17:38       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
20:19:50       Focusing Completed Foc1 AutoFocus Completed (Profile No 1, wide)
20:21:57       Focusing Completed Foc1 AutoFocus Completed (Profile No 1)
20:21:59       Focusing Started-Foc2 Foc2 Focusing Started (Secondary Scope, using ShCap)
20:24:09       Focusing Completed Foc2 AutoFocus Completed (Profile No 2, wide)
20:25:55       Focusing Completed Foc2 AutoFocus Completed (Profile No 2)
20:26:13     Target Completed Target completed (Focus Field 1, HIP 4911)
20:26:15     Target Missed (5/40) Target's time slot was missed (5/40, GCVS X TRI)
20:26:17     Target Started (6/40) Target started (6/40, AT2021abed)
20:31:38       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
20:33:25       Focusing Completed Foc1 AutoFocus Completed (Profile No 3)
20:47:39     Target Completed Target partially completed (6/40, AT2021abed)
20:47:43     Target Started (7/40) Target started (7/40, 10P/Tempel)
20:52:43       Focusing Skipped Foc1 focusing skipped - star is lost (TCF-S)
21:02:34     Target Completed Target completed (7/40, 10P/Tempel)
21:02:38     Target Started (8/40) Target started (8/40, AT2021zkf)
21:06:13       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
21:08:53       Focusing Completed Foc1 AutoFocus Completed (Profile No 4)
21:26:23     Target Completed Target completed (8/40, AT2021zkf)
21:26:27     Target Started (9/40) Target started (9/40, NGC 7446 w/SN2021agdd)
21:29:22       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
21:31:12       Focusing Failed Foc1 focusing failed - star lost
21:47:43     Target Completed Target completed (9/40, NGC 7446 w/SN2021agdd)
21:47:47     Target Started (10/40) Target started (10/40, M31 w/AT2021aaxp)
21:50:21       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S)
21:50:53       Focusing Aborted Foc1 Focusing Aborted
21:50:55     Target Failed Target failed due to aborted (10/40, M31 w/AT2021aaxp)
21:50:57   Queue Aborted Job Queue aborted
23:56:49   Dome Closed Dome closed (closing time 47s)
02:02:42 User Intervention User request to close and finish session
02:02:44 Session Closing Session closing
02:02:46   Dome Closed Dome already closed (closed at 23:56, closing time 47s)
02:04:43   Dome Parked Dome parked (parking time 113s), Az: 90.0 deg
02:05:59   Telescope Parked Telescope parked (parking time 70s)
02:06:13   Telescope State Scope parked Turn scope off. (Handbox)
02:06:34   Telescope Switched Off Telescope Power has been switched off via UPB Switch.
02:07:29   Services Stopped Night Services stopped
02:07:31 Session Housekeeping Housekeeping Started (Cleanup FITS, Create Fits Summary, Transfer Files)
02:07:32 Session Finished Session Finished
 
Session Alerts
Time     Alert Detail
19:06:54 Job Queue Job Queue appears to be frozen (last IamAlive message at 18:56:16)
19:06:59 Obs.Monitor Obs.Monitor appears to be frozen in section Refresh Dome Picture (last cycle at 19:06:28)
20:06:49 UPB.Switch Restarting UPB Powerbox Software
20:06:56 UPB.Switch UPB.Switch service hasn't been restored

Back to Top


Operational Issues (2021-12-10, S940)

[ Prev | Next ]

Critical Issues

Major Issues

Minor Issues

Small Defects

Continuous Improvement

[ Prev | Next ]

Back to Top


First Session with new Observatory Computer

Following a total failure of old Observatory Computer (2021-12-05) a replacement computer has been acquired (2021-12-09) and software including multiple pieces of observatory software installed (2021-12-09 to 10). This included ASCOM 6.5 SP1,Software Bisque software (TheSky6, CCDSoft5 and TPoint),  Pegasus Power Box,  and ASCOM drivers (UPB, PulsarDome, TCF-S focuser, MeadeGeneric, SBIG, ZWO ASI) and AstroSuite/Observatory Control software (AstroMain, AstroPlan, AstroGuard, AstroLaunch, AstroShCap)  and support programs (PHD2, SharpCap 3.2). Various settings were taken from setting recovered from old observatory computer's SSD drive (using an mSata to USB adapter) or recreated where necessary.  

- Issues discovered during installation and resolved are described here.

Pre-Session Issues (Daytime Testing)

The following issues were found during the daytime testing in the observatory and resolved prior to the nightime session.

(see Operational Issues for more details and current status)

Session Issues (Night-time Testing)

The following issues arose setting up and conducting a nightime session

Com Ports

Device   Primary Com Port   Secondary Com Port
Pulsar Dome   COM3   unknown (n/a)
UPB Powerbox   COM4   unknown (n/a)
TCF-S Focuser   COM5   unknown (n/a)
LX200 Telescope   COM6   unknown (n/a)

Whilst primary COM Port numbers are identified for the key observatory equipment that use Serial ports, and recorded in the necessary program settings (AstroMain and later in AstroLaunch) it hasn't been possible to determine secondary ports.  These are port numbers that device can potentially jump to if inserting USB cables into different ports on the USB hub/computer or for some other hardware, software or Windows OS glitch.  This was seen on the old observatory computer (Windows 7) but it is not yet known if this behaviour will be seen on the new observatory computer (Windows 10).    Rebooting the computer always results in devices returning to their respective primary COM Port.    

AstroMain has routines that automatically update ASCOM profiles to use the primary/secondary COM port dependant on which Port is active, and monitor sessions in progress to see if Com Port has switched and automatically restart the connection to the respective device on the new port.    Until secondary ports are identified these routines won't be able to perform the task of automatically resetting the driver COM port setting if the device jumps to a secondary port.

Update 2022-06-21:  During the first 6 months / 80 sessions since installation of a new observatory computer on 2021-12-10 the attached devices have continuously remained locked-on their original COM Port settings..  This would seem to be associated with either new computer hardware / USB controller or with the use of Windows 10 operating system (instead of Windows 7).

Back to Top


Investigation - Guiding Runaway (SideOfPier Issue)

Issue :  Guiding Runaway occuring in RA after performing PHD2 Calibration on new observatory computer

Description
After the failure of the previous Observatory Computer after session S939 (2021-12-05) a new Calibration had to be performed on the new replacement computer.  This was performed at the start of session S940 (2021-12-10) using a starfield close to Dec 0 and just east of the celestrial meridian.    Calibration data was checked and satisfied with its quality, the observing session was begun

Guiding performance on the first two targets was good. Both target lays east of Meridian, i.e on the same side as where the Calibration was performed.

Guiding performance on the next 3 targets (AT2021zkf, NGC_7446 & M31) was terrible with immediate runaway in RA guiding.  It appeared that each RA guide pulses was pushing the star further and further away from star lock position.   Quickly the RA guide pulses reached the Maximum RA guide pulse setting of 2500 ms.  Over the course of a 180s exposure telescope was being shifted some 9 arc minutes in cases where the guide star was followed and not lost.   Often the guide star would eventually became lost.

Image
180s exposure, C Filter, #940017

Image

The 3 targets with terrible guiding all lay on the west side of the Meridian, i.e. on the opposite to where the calibration was performed.

Whilst guiding on the observatory telescope is generally poorer on the west side of the Meridian (due usually to Dec guiding / possibly balance issue),  the  guiding character is nothing like these new cases which are clearly suffering from RA guding runaway.

Investigation
The observing objectives for the session was placed on hold, whilst investigation was carried out.   This began with guiding test on starfields just west of and just of the Meridian.   The starfield just west of the Meridian suffered from the same terrible guiding whilst the starfield just east of the Meridian had normal guiding.
The tests were repeated with same result.  A test was performed where guiding started east of the Meridian and kept running whilst the starfield crossed the Meridian,  this has normal guiding throughout.  The pointing position (the SideOfPier) when the guiding started was a key factor, not the pointing position (SideOfPier) later on.

This indicated a 'SideOfPier' related issue, and seemed to try up somehow with the appearence in July 2021 onwards, with warning messages from PHD2 saying that 'SideOfPier had changed but the Calibration doesn't have SideOfPier information'.  The Calibration that was being used on the old Observatory Computer was made prior to June 2021.  With the making of a new Calibration (for the new Computer) it suddenly had SideOfPier information and it was evident that PHD2 was performing a Calibration flip when starting to guide on target starfields lying on the opposite of the Meridian.

Analysis
A calibration flip is the appropriate action for a GEM Mounted scope when guiding on targets lying on the opposite side of the Meridian to where the Calibration was performed.   RA pulses (and for some designs Dec pulses) have to be inverted to produce the neccessary guiding effect.

However for my Fork & Wedge Mounted LX200 Scope, RA and Dec Pulses shoudn't be inverted when crossing the Meridian (It doesn't really have a SideOfPier so to speak).   There is a way to turn off Calibration Flipping in PHD2. All that can be done is to toggle a setting so that a new calibration is automatically performed on each and every target, which would interfer with observing and could possibly give rise to a hold-up to automated operations if the calibration fails and user involvement is required.

The problem is due to ASCOM telescope sending SideOfPier (pierEast or pierWest) through to PHD2.
     SideOfPier = pierEast, or
     SideOfPier = pierWest

Whilst the issue only became exposed in my system just now when it became necessary to do a new Calibration in PHD2 (because of a new computer)
looking back at earlier logs on my system it can see that the issue inherently dated back to the June 2021 version of MeadeGeneric telescope driver (v1.2.0) when it  implemented the SideOfPier property and on my system it began returning.
    SideOfPier = pierUnknown

This would probably have been ok for PHD2 except that a previous change in DeviceHub made earlier in Oct 2020 (v 6.5.0.1) caused DeviceHub to step in upon seeing 'pierUnknown' from the telescope driver and calculated a SideOfPier value ( 'pierEast' or 'pierWest') based on the telescope position.
    SideOfPier = pierEast, or
    SideOfPier = pierWest

The pierWest/pierEast became used by PHD2 and resulted in the showstopping RA guiding runaway issue on my system.
It also now explains the frequent PHD2 warnings that began to appear in July 2021, saying

Prior to June 2021 MeadeGeneric was throwing an exception on my system
     SideOfPier = Not implemented

Fix (2021-12-11)
Colin Dawson and Rick Burke (authors of MeadeGeneric Driver and of DeviceHub respectively) were contacted regarding the issue with result that a fix was made in MeadeGeneric development version 1.3.3.370.  This was tested (2021-12-11) and for my system the original behaviour of throwing a 'Not Implemented' exception is restored :

   SideOfPier = Not implemented

DeviceHub see this exception and reports SideOfPier as 'Unknown' on its Telescope Control tab and doesn't attempt to set a pierEast or pierWest but simply passes on the exception.    PHD2 reports the SideOfPier as 'N/A'.

Testing (2021-12-13)
Testing confirm that the fix in MeadeGenric 1.3.3.370 has resolved the earlier Guiding Runaway/SideOfPier issue
(The fix will no doubt be ruled up into a official MeadeGeneric release in due course).

Update (2021-12-14)
A new version 6.5.1.9 version of DeviceHub was released today (2021-12-14) with a fix to 'Ensure that a telescope side-of-pier is only calculated for a German Equatorial mount'.

Actions
Image  SideOfPier : Update 'TheSky6 connection' routine in AstroMain to report the scope's SideOfPier to check that pierEast/pierWest doesn't start to appear again following future releases of DeviceHub/MeadeGeneric.

Back to Top


2021-12-12


Investigation - LX200 Tracking Rate Discrepency (2021-12-12)

Issue :  Telescope Tracking isn't at Sidereal Rate (contrary to the running assumption that is was on sidereal rate)

Description
Whilst looking at the telescope logs in connection with a separate ( SideOf Pier) issue it was noticed that there were TrackingRate messages like

  13:40:56.199 TrackingRate Get 60.0 driveLunar

That was considered odd, because :

  1) A rate of 60.0 Hz would be Solar Tracking rate (driveSolar),  Lunar Tracking Rate is 57.9 Hz
  2) I thought I was using a Sidereal Tracking rate  (60.1 Hz) driveSidereal

What’s going on, I expected to see "60.1 driveSidereal'  being reported !

Image

Confirmation of Discrepency
Scope was powered up, connected and the the Tracking Rate setting in AutoStar II examined using Virtual Handbox . It shows that the scope is set to "Sidereal" tracking

Image

Setting was confirmed and the MeadeGeneric log reexamined but is still shows "60.0 driveLunar"

  13:44:12.360 TrackingRate Get 60.0 driveLunar

Analysis (Problem 1)
Problem 1 was determined to be a bug/limitation of the MeadeGeneric code which only considers Sidereal and Lunar options

In the Get Tracking Rate function it can be see that the following code line leads to the function returning driveLunar whenever the scope tracking rate is not 60.1, regardless of what the rate actually is.
   DriveRates result = rate == "60.1" ? DriveRates.driveSidereal : DriveRates.driveLunar.

Bug has been reported to author of MeadeGeneric driver for his examination/fix.

Analysis (Problem 2)
Problem 2 was harder to explain:   Why was the value returned by a GT (Get Tracking) command 60.0 , rather than the expected sidereal rate (60.1) which should be associated with the scope’s current Tracking Rate setting (“Sidereal”).   

Eventually this was determined to be a misunderstanding / forgotten awareness of exactly how the observatory scope is configured after its firmware was upgraded from 4.2g to a 4.2G patch back in 2013, and also an apparently undocumented feature of the Custom Tracking Rate in AutoStar (or at least its behaviour in 4.2G Patch)

In 2013 I updated the scope’s firmware to 4.2G in order to access two (at the time critical) enhancements.

This firmware behavior might explain why the GT command is returning 60.0 since if I might still have ‘60.0’ in the custom setting in the scope from when the custom setting was last used for Solar Observing more than 12 months ago  and my 4.G patched scope might use the custom value even though the Autostar II implies that I’m using Sidereal.

Custom Rate is specified by entering a signed value that adjusts the  'standard' tracking rate.  In 4.2g the adjustment is 'signed value * 0.1%'.   In 4.2G the adjustment is 'signed value * 0.01%'

It was thought that the standard tracking rate reference would be sidereal rate (60.1), but it now seems that the standard tracking rate reference is in fact 60.0

The LX200GPS/R manual states:

Custom Tracking Rate allows you to enter values from -999 (stands for -99.9%) to 999 (stands for +99.9%). The lower the number, the slower the rate. If you enter -999 the telescope will move so slow as to appear to be stopped. If you enter 999, the telescope will be moving at approximately twice the tracking rate.

 but doesn't explicity say what the reference tracking rate (what does 0 exactly stand for ?)

The Sun tracks at 99.7270% of sidereal rate. I original though that this would be entered in Autostar II's Custom Tracking as -3, representing 0.3% slower than sidereal  (-999 would stand for 99.9% slower). However for my LX200 which has firmware updated to 4.2G v19 and uses 10x finer tracking rate I originally thought the Custom Tracking should be set as -27, representing 0.27% slower than sidereal,  -999 would stand for 9.99% slower) .
( I believe that I have subsequently show that this is probably wrong and Solar Rate should be entered as +0 on my scope (i.e. 60 Hz + 0.00%) )

However the discrepancy doesn’t end there :

- I’ve gone into Scope Tracking Rate setting and went to ‘Custom’ expecting to see –27 (for Solar Rate equivalent) but it doesn’t , instead it has the value 0 (I suppose it’s possible that it was –27 but going into custom from a previous position of Sidereal being previously selected might set the displayed custom value to 0 upon entry)

- If I enter a custom value of 0, and Save it , I find ‘Sidereal’ is still the selected Tracking Rate after going up to Telescope Setup and then going back down into Tracking rate.

- I then entered a custom rate of +1 (which I originally understood to be Sidereal Rate + 0.01% on my scope, ie 60.10601 Hz
But after saving this the drivers log file still reports  “60.0 driveLunar”

- I then restarted the scope thinking that it needed to go through a reboot stage to pick up the new value. The TrackingRate in the scope is still showing as “Custom” with a value of +1 which would be equal 60.10601 Hz but the driver log file still reports “60.0 driveLunar”

Eventually I concluded that the % adjustment in ‘Custom’ isn’t a +/- % adjustment to sidereal rate (60.1) but a +/- % adjustment to a standard 60.0 Hz rate .

- Putting in a custom rate of +17 (from  (1 - (60.1/60.0) *100 *100), leads to the driver log reporting “TrackingRate Get 60.1 driveSidereal”

So after a lot of investigation I think that’s the second half of the discrepancy is explained.

Scope will be left with this custom rate of +17 and the tracking/guiding behaviour of the scope monitored during the next live session.

Actions

Image Communicate driveLunar Tracking Rate bug to Colin Dawson (author of MeadeGeneric driver)

Image
Change Custom Tracking Rate in scope to +17  (ie 60.1 Hz equivalent)

Image Monitor the tracking/guiding behaviour of the scope during the next live session (S941 session)

Image Update TheSky6 connection routine in AstroMain to report scope's current Tracking Rate

Image Add routines in AstroMain and accompanying user button options for setting the Tracking Rate on the observatory's main telescope (LX200GPS/R)

Update 2021-12-14
Using :STtt.t# method of setting tracking rate on LX200 (via btnSetTrackingRate.Click) shows some anomalies

:ST60.2# puts LX200 Tracking Rate to "Custom" , +16  (4.2G)    [ ~ +2  (4.2g)  ]
 but GT command returns +60.1 (ie Sidereal Tracking Rate)

 :ST60.1#  (intended Sidereal Tracking Rate)  puts LX200 Tracking Rate to "Sidereal"
 but GT command returns +60.0  (ie Solar Tracking Rate)

:ST57.9# (intended Lunar Tracking) puts Tracking Rate to Custom, -365 (4.2G)   [ ~ 37 (4.2g) ]
  but GT command returns 57.8

:ST58.0# puts Tracking Rate to Custom
  but GT command returned 57.9 (i.e. Lunar Tracking Rate)

 Send GT command... Info Tracking Rate is unknown
GT Command : Fail Timed out waiting for received data

Send GT command
17:01:07.395 CommandString raw: False command GT
17:01:07.426 CommandString Completed: +60.1

Back to Top