Fig 1.   Dome Slewing Performance Chart for Session S1172.
 

Issue
a) Holdups occurred slewing to last target and to Pre-Park  (possible holdup at Az 80 or so ?)
b) Last two targets were devoid of stars (sky view occluded by observatory roof)
c) Dome left at physical azimuth 52° at end of session (38° away from Park Postion)

Description

a) For slew to last target (12P/Pons-Brooks) and later on for the slew to Pre-Park position the dome held-up at Az 80°& Az 76° or otherwise stopped moving, for 22s, 99s.  

 b)The last two targets in the session (C/2019 T4 (Atlas) & 12P/Pons-Brooks) had ocluded sky views which seemed to be due to the dome roof and their images had to be rejected.

c) Dome was left at around physical Az 52° at end of session instead of it's expected Park Position at Az 90° (an error of 38°), even though Dome reported that it had reached and parked at Az 90.1.

Dome Slewing Performance Chart for Session S1172.
showing issues slewing to 12P/Pons Brooks (orange arc) and to Pre-Park (red arc)

Issue is serious because images of 12P/Pons-Brooks (a main target for the session) was unsuccessful.
Issue could have been worse if it had begun earlier in the session.

The Observatory was visited later in the morning but was unable to establish exactly caused of the problem.  There didn't seem to any significant hold-up when the dome was rotated clockwise/anticlockwise.  The ends of the rubber trim didn't appear to be catching on anything. The rubber trim is well clear of the wall flanges (which were filed back after the S1169 Session). If there was any catching of the trim it would have been on the SE roof clamp (as this is position of trim ends when Dome is at Az 80) but there was no strong indication of this - the gap seemed sufficient. Dome was rotated to it's physical park position, (where the induction receiver lies next to the induction charger),  and then resynced to Az 90 (Park).

Analysis 

The questions that need to be answered are :

• Was there a physical hold-up at Az 80°/Az 76° ?
   
• How did dome roof come to occlude the sky view for `C/2019 T4 (Atlas) when neither its slew to target or preceeding slews indicate any hold-ups or delays ?
   
• Why was dome left at physical Az 52° when the dome controller reported that it had finished the session at Az 90.1°. ?

The possible hold up point (at Az 80°/Az76°) is around 90° from a 168° hold-up point that was a major issue in S1169 session ( see Investigation - Pulsar Dome holding up at Az 168°, 2023-11-30° ). This might point to a similar underlying cause ?.   When the Observatory was visited later there didn't seem to any sign of any significant hold-up.  perhaps a wet/slippery roof flange was involved ?.

It can be ruled out, but overall the data doesn't point to there necessarily being any physical hold-up that prevented the dome from rotating l(which would have involved the dome drive wheels spinning on the roof flange for periods of time).  

The data would seem to be most consistent with a hypothesis that the encoder wheel lost or partially lost tension against the roof flange and wasn't fully rotating for a period or periods of time near the end of the session, even though the dome itself was rotating.    Can we prove this or at least demonstrate that it was possible or probably responsible for the issue.

Looking at the session's last 5 slew operations:
 
Slewing Details                          Dome Slew                                       Telescope Slew               
Target             Time     Az1     Az2  Slew  Angle  Speed  Holdup    Az1    Az2  Alt1  Alt2  Slew  Angle  Speed  Slew   
                  hh:mm     deg     deg  time    deg  deg/s   time    deg    deg   deg   deg  time    deg  deg/s  time
NGC 2848          05:33   186.8   195.3    7s    8.5   1.28          187.4  195.8  50.1  15.5   21s   44.3   2.08   21s   
NGC 5395          05:55   199.9   105.1   59s   94.8   1.60          201.1  101.8  14.7  52.8   29s  122.6   4.17   59s
C/2019 T4 (ATLAS) 06:17   110.8    89.0   16s   21.8   1.34          107.3   87.1  55.5  17.3   27s   61.4   2.26   27s
12P/Pons-Brooks   06:32    91.2    55.8   43s   35.4   0.83 D* 22s   90.2    54.2  19.3  23.0   24s   51.3   2.15   43s
Pre-Park          06:55    58.1    90.1  121s   32.0   0.27 D* 99s   57.2   180.6  24.9   5.2   36s  170.1   4.69   37s


- Target NGC 5395 had a seemingly normal 94.8° anticlockwise slew from Az 199.9° to Az 105.1° (consistent with slaved coords based on Telescope Slew). Images show that telescope had a clear view of the sky (i.e. the target was not occluded by roof) and we can assume that Dome's physical Az matched its reported Az.  By end of this target the dome had moved on to Az 110.8° whilst slaved to Telescope moving at Sidereal speed.

- Target  C/2019 T4 (Atlas) involved a planned anticlockwise slew from Az 110.8° to 89° and we know that the roof ended up occluding the telescope view of the sky.  It is conjectured that the encoder wheel was slipping during the slew (causing to to begin underestimate the ture position of the dome), such that the dome rotation overran the planned physical Az .  As slew was only 16s in length, it is unlikely that the dome over rotated by far (it couldn't have reached Az 52° - the final physical Az of dome). At a reasonable maximum speed of 2.0°/s  the dome could have ended up at Az 79 (110.8 - 16 x 2.0). At a speed of 1.6°/s (speed of preceeding target) the dome would have ended up at Az 85°.   As the roof fully occluded the telescope view, it is assumed that that Dome must have ended up at a physical Az between 79° and 85° (at a speed of 1.34°/s the dome would have ended up at Az 89° and the roof wouldn't have occluded the sky view). Let assume that Dome reached Az 82°. The dome seemed to have moved clockwise by 2.2° during the course of the C/2019 T4 (Atlas) ( consistent with the movement of the scope at sidereal speed_so we will assume that by the end of target C/2019 T4 (Atlas) the dome lay at Physical Az 80° (from 82 - 2).

- Target 12P/Pons-Brooks was a planned 35.4° anticlockwise slew from Az 91.2°  to 55.8° and took 43s with two hold-ups totalling 22s.   Again the telescope had an occuled view of the sky. We don't know if the 22s was due to dome drive wheels slipping or due to the encoder wheel not moving.   Assuming i) the dome wheels were slipping would imply that Dome ended the slew at physical Az 45 (80-35).  If dome had moved 35.4° in 21s (from 43-22) would imply a dome speed when moving of 1.68°/s  (from 35.5/21) which is quite feasible. Assuming ii) that the encoder wheel wasn't moving or was slipping, would imply again that the Dome over-rotated.   Assuming dome speeds of 1.6°/s to 2.0°/s would imply a physical slew of 68.8° to 86°, and that by the end of slew the Dome was at physical Az 11° (from 80-68.8)  to 354°  (from 80-86 + 360).   During the course of the target the dome seemed to have moved clockwise by 2.3° (consistent with the movement of the scope at sidereal speed).   Dome could therefore been a physical Az anywhere between 45° and 354° .     Any of these would explain why the roof occluding the telescope's view of the sky/target.

- Pre-Park was a planned 32° clockwise slew from Az 58.1 to Az 90.0 (90.1) and took 121s including a 99s hold-up.   Again we don't know if the 99s was due to the dome drive wheels slipping or due to the encoder wheel not moving. Assuming i) the dome wheels were slipping would suggest dome move 32° in 22s which is a 1.45°/s which is quite feasible. As the dome ended at physical Az 52° it might imply that 12P/Pons-Brooks had ended at physical Az  20°, which is in the range estimated.    Assuming ii) that the encoder wheel wasn't moving or was slipping would imply that dome had over-rotated. Assuming dome speeds of 1.6°/s to 2.0°/s would imply a physical slew of 193.6° to 242°. As the dome ended at Physical Az 52° it would have required 12P/Pons-Brooks to have ended at physical Az between 170°  (52-242+360) and 218°  (52-194+360) . This is outside the Dome's estimated physical Dome Az at end of target 12P/Pons-Brooks.

From this it is concluded that it is likely that both of the slews to target C/2019 T4 (Atlas) and target 12P/Pons-Brooks involved slippage or some not rotation of the encoder wheel.   The slew to Pre-Park may have involved some slippage of the encoder wheel, but there was almost certainly a physical hold-up in dome rotation (with slippage of dome drive wheels).

Conclusion
In regard to questions

• Was there a physical hold-up at Az 80°/Az 76° ?
  
  - There was very likely a physical hold-up during slew to Pre-Park but it was likely to have been somewhere between physical Az 20° and 52° rather than at the reported Az (76.1).
  
  - based on reported hold-up at Az 76 (for reported slew from Az 58.1 to 90.1) it is estimated that the hold-up corresponding to physical Az 38° (from 52 - (90-76) )
   
• How did dome roof come to occlude the sky view for `C/2019 T4 (Atlas) when neither its slew to target or preceeding slews indicate any hold-ups or delays ?
  
  - The tension in encoder arm was significantly weakened during the course of the slew to C/2019 T4 (Atlas) causing the encoder wheel to slip
  - Tension in encoder arm was probably lost for at least 22s during slew to 12P/Pons-Brooks again causing the encoder wheel to slip.
  - The total slippage in encoder wheel was likely equivalent to 38°
  
  - Precise reason of slippage of encoder wheel is not known.  The elastic ribbon that provides tension to the encoder arm was intact when it was checked post-session
   
• Why was dome left at physical Az 52° when the dome controller reported that it had finished the session at Az 90.1°. ?
  
  - This was almost certainly due to slippage of encoder wheel, but for reasons unknown.
  
  - Slippage of Drive Wheels alone can not produce the observed 38° error in Dome Az. Whilst drive wheel are slipping there is no rotation or the dome and and consequently no movement of the encoder wheels so they can't be accumulated Az error during this time.
  
  - a sudden lost in calibration is unlikely, dome was calibrated before the session and dome slewing wasn't in error during slews to first 37 targets in the session.

Actions

• Look at ways to tighten the tension that holds the encoder arm/wheel against the roof flange.
  Done 2023-12-15
  
  
• Look at how a physical hold-up of dome could have occured with the dome at physical Az 38°
  The reason for hold-up continues to be a mystery

• 1) About DomeLink
• 2) Creating DomeLink Project
• 3) Developing the Code
• 4) Driver Installation
• 5) Driver Set-Up & Use
• 6) Progress

1) About DomeLink
DomeLink is my second attempt at building an ASCOM Driver (and my first attempt as building a Dome Driver)

It is intended to be an ASCOM dome 'driver' that doesn't actually drive dome hardware itself but is designed instead to sit between Pulsar Dome and DeviceHub.Dome. Except for forcing the pass back of Slewing=True during the first 1s after sending on a SlewToAzimuth to the Dome it will do nothing more that pass communications to and fro between the Pulsar Dome and DeviceHub.Dome.

DomeLink is intended to deliver a workaround for a non-ASCOM compliant behaviour in Pulsar Dome Observatories driver whereby the Slewing property doesn't immediately become True after receiving a SlewToAzimuth request, but is delayed by up to 200ms or so.  This doesn't interfere with ASCOM Conform's Asynchoronous SlewToAzimuth test, probably because of its polling frequency or methodology, but it does affect DeviceHub's ability to perform fast-polling during the initial 5s of a dome slew.  After sending a SlewToAzimuth request, DeviceHub looks at the response to Slewing property and seeing it have the value False considers the Slew as finished and changes polling rate to a normal 5s rate, then later finding that slewing=True it then goes to fast polling.  The initial slow polling interfers with AstroMain's monitoring of dome during slewing, and limits the ability to measure the precise slew time for very short slews (5-6s). Unable to convince either Pulsar Observatories or Device Hub to make a change it is proposed to create this intermediary Dome Driver called 'DomeLink'. It would operate in a manner than is rather similar to how TeleLink works to workaround a bug in TheSky6/TPoint (TeleLink 1.0 - Telescope Driver, 2022-11-14 ).

2) Creating the DomeLink Project

Firstly the "ASCOM Driver Project Templates" needs to be downloaded / added as a Visual Studio Extension if not already available. This is done using Visual Studio's "Extensions/Manage Extensions" menu option


Next a new project solution is created from Visual Studio's "Getting Started - Create a new project" option, and then filtering on "Visual Basic / Windows / ASCOM " templates, selecting "ASCOM Device Driver (VB)" 



The Next button is then clicked and the "Configure your new project" form filled in



Clicking 'Create'  leads to the "ASCOM Device Driver Project Wizard" where the driver's Device Class and Device Name are completed.



Clicking 'Create'  then creates and opens the project solution in Visual Studio.

3) Developing the Code

The driver's Driver.vb file (DomeDriver.vb in case of the DomeLink driver) is then edited & extended to include the required functionality.

The 'SetupDialogForm.vb' form this edited as required.  In the case of the DomeLink driver this meant the removal of the COM Port (not required as DomeLink is not an  'End-Driver', and the addition of a Dome ID field (for defining the Dome that DomeLink is connected to)

The principal coding features in DomeLink's  DomeDriver.vb file are as follows:

Customised variables:

' Dome ID
Friend Shared domeIdProfileName As String = "Dome ID" 'More Constants used for Profile persistence
Friend Shared domeIdDefault As String = "ASCOM.Simulator.Dome"
Friend Shared domeID As String = "ASCOM.Simulator.Dome"

Private objScope As ASCOM.DriverAccess.Dome

Private SlewToAzimuthTime As Date

 A problem initially occured as ASCOM.DriverAccess.Dome wasn't recognised.  This was solved by adding a new Reference to project, using browse to add in
   C:\Program Files (x86)\ASCOM\Platform 6 Developer Components\Components\Platform6\ASCOM.DriverAccess.dll

ASCOM.DriverAccess.dll Customised Connected Property

 
 Public Property Connected() As Boolean Implements IDomeV2.Connected
' =====================
  Get
    Dim bIsConnected

    bIsConnected = IsConnected

    ' TL.LogMessage("Connected Get", bIsConnected.ToString())
    Return bIsConnected
  End Get

  Set(value As Boolean)

    TL.LogMessage("Set Connected", value.ToString())

    ' Return if scope already connected / already disconnected
    ' --------------------------------------------------------
    If value = IsConnected Then
      Return
    End If

    If value Then

      ' check DeviceHub is running
      '....

      ' Create Dome
      ' ------------
      TL.LogMessage("Creating objDome", "Setting objDome = New ASCOM.DriverAccess.Telescope(" + domeID + ")")
      objDome = New ASCOM.DriverAccess.Dome(domeID)

      ' Connect Dome
      ' -------------
      TL.LogMessage("Connect Dome", "Setting objDome.Connected " + value.ToString())

      objDome.Connected = True

      TL.LogMessage("Connect Dome", "Setting connectedState = True")
      connectedState = True

    Else

      ' Disconnect Dome
      ' ----------------
      TL.LogMessage("Disconnect Dome", "Disconnecting from " + domeID)
      objDome.Connected = False

      TL.LogMessage("Disconnect Dome", "Should now be disconnected from " + domeID)

      ' TODO disconnect from the device

      If IsNothing(objDome) = False Then
        TL.LogMessage("Disconnect Dome", "Disposing objDome")
        objDome.Dispose()
        objDome = Nothing
      End If

      TL.LogMessage("Disconnect Dome", "Setting connectedState = False")
      connectedState = False

    End If
  End Set
End Property

Customised properties and methods which simply pass-on requests and return results, e.g.

Public ReadOnly Property ShutterStatus() As ShutterState Implements IDomeV2.ShutterStatus
'======================================
  Get
    Return objDome.ShutterStatus
  End Get
End Property

Public Property Slaved() As Boolean Implements IDomeV2.Slaved
'======================
  Get
    Return objDome.Slaved
  End Get

  Set(value As Boolean)
    objDome.Slaved = value
  End Set
End Property

Public Sub SyncToAzimuth(Azimuth As Double) Implements IDomeV2.SyncToAzimuth
'=======================
  objDome.SyncToAzimuth(Azimuth)
End Sub



Key Customised Methods which fix issue with Slewing Value

 Public Sub SlewToAzimuth(Azimuth As Double) Implements IDomeV2.SlewToAzimuth
'=======================
  objDome.SlewToAzimuth(Azimuth)
  SlewToAzimuthTime = Now()
End Sub 

 Public ReadOnly Property Slewing() As Boolean Implements IDomeV2.Slewing
'=================================
  Get
    ' Special Handling
    ' ----------------
    If Now().Subtract(SlewToAzimuthTime).TotalSeconds < 1.0 Then
       Return True   ' return True to workaround small bug in Pulsar Dome driver that can
                     ' initially return False following call to SlewToAzmiuth
    Else
       Return objDome.Slewing
    End If
  End Get
End Property

4) Driver Installation

To install the new driver a setup file needs to be created using the 'Inno Setup' Program (accessed from "Start/ASCOM Platform 6/Developer Tools/Inno Installer Web Site" or direct from https://jrsoftware.org/isinfo.php to install latest stable version 6.2.1 as at 2022-11-12)  and a script file that is built by running  "Start/ASCOM Platform 6/Developer Tools/Driver Install Script Generator" .

Picture below is for TeleLink - picture needs to be replaced by one specific to DomeLink.

After filling in the relevant details and clicking Save a script file ("DomeLink Dome Setup.iss) is then generated.

Inno Setup is then run and the script file selected, or the .iss file can be simply clicked upon.  This is then checked and Built / Run to generate a setup file (DomeLink Dome Setup.exe) which can be either immediately lauched,  or launched later, in order to register the driver in ASCOM and in Windows.

The Setup.exe file can also be copied to the Observatory Computer, where it can launched in order to install the driver on the Observatory Computer.

When the Setup.exe is run is copies the driver DLL file to "C:\Program Files (x86)Common Files\ASCOM\Dome\" folder and an entry is created in  ASCOM Profile Root\Dome Drivers (viewable via Profile Explorer).

5) Driver Set-Up & Use

From the DeviceHub Dome Setup'  chose  'DomeLink Dome' as Dome and click on Properties.
(Note Dome Geometry settings will be preserved)



Then click on Settings to get to 'ASCOM Dome Chooser' form and select "DomeLink Dome". 

The DomeLink settings must now be setup. A dialog appears to ensure that the DomeLink configuration is checked/set before first time use.

Click on Properties  to get to the DomeLink Setup' form. Choose  'ASCOM.Pulsar_Observatories_Dome.Dome'  and optionally turn on Trace :

  Note : On Development Computer (or if running Sim Dome' on Observatory Computer) choose ASCOM.Simulator.Dome.

Clicking on OK returns to Device Hub Dome Setup :

Ok is then clicked on the various forms to save the setup.

6) Progress

The first working version (my first ever ASCOM driver) was developed in a matter of just a few hours.  DomeLink 1.0 was released on 2023-12-07 and installed on Observatory Computer ready for its first live session test. 

Tests show that it is allowing DeviceHub to operate on Fast-Polling for the entire slew, showing that DomeLink successfully works around the issue that Pulsar Dome causes by carrying Slewing=False for the first 200ms or so following reciept of a SlewToAzimuth request. 

Looking at DeviceHub's activity log for a slew from Az 90° to Az 110° we can see in the existing environment involving a direct connection to 'Pulsar Observatories Dome' (test a), DeviceHub.Dome sees Slewing = False at 21:56:01.323 and switches to polling every 5000 ms, before it switches to 1000 ms polling (this is not logged) when it sees Slewing=True at 21:56:06.  The causes an absence of any fresh Azimuth information arriving back at the end client between Az 90° and Az 100.3°.   The entire 20° slew is defined by only 8 points.

Test a) DeviceHub Activity Log for Slew from 90 to 110 with direct connection to Pulsar Observatories Dome

21:56:01.320: Dome - Commands: SlewToAzimuth (110.00000°): (slew started)
21:56:01.320: Dome - Statuses: Get Azimuth: 90
21:56:01.320: Dome - Statuses: Get Slewing: False
21:56:01.322: Dome - Commands: Started fast polling every 1000 ms.
21:56:01.323: Dome - Statuses: Get Azimuth: 90
21:56:01.323: Dome - Statuses: Get Slewing: False
21:56:01.323: Dome - Commands: Returning to normal polling every 5000 ms.
21:56:06.329: Dome - Statuses: Get Azimuth: 100.3
21:56:06.329: Dome - Statuses: Get Slewing: True
21:56:07.328: Dome - Statuses: Get Azimuth: 102.4
21:56:07.328: Dome - Statuses: Get Slewing: True
21:56:08.328: Dome - Statuses: Get Azimuth: 104.6
21:56:08.328: Dome - Statuses: Get Slewing: True
21:56:09.328: Dome - Statuses: Get Azimuth: 106.9
21:56:09.328: Dome - Statuses: Get Slewing: True
21:56:10.329: Dome - Statuses: Get Azimuth: 109.2
21:56:10.329: Dome - Statuses: Get Slewing: True
21:56:11.328: Dome - Statuses: Get Azimuth: 109.8
21:56:11.328: Dome - Statuses: Get Slewing: True
21:56:12.345: Dome - Statuses: Get Azimuth: 110
21:56:12.345: Dome - Statuses: Get Slewing: False

21:56:16.344: Dome - Statuses: Get Azimuth: 110
21:56:16.344: Dome - Statuses: Get Slewing: False
21:56:16.344: Dome - Commands: Returning to normal polling every 5000 ms.

If we compare this with the new enviroment involving an indirect connection to ''Pulsar Observatories Dome' using DomeLink as an intermediary driver (test b) DeviceHub.Dome sees Slewing = True  and remains on fast polling every 1000ms for the entire slew, this means fresh azimuth data can arrrive with the end client (AstroMain) for every second of the slew (including the first 5s).  The entire 20° slew is now defined by 12 point. The new environment means that the actual time to make short slews (less than 5s) can now be recorded by AstroMain.

Test b) DeviceHub Activity Log for Slew from 90 to 110 with indirect connection to Pulsar Observatories Dome via new DomeLink driver.

22:05:32.623: Dome - Commands: SlewToAzimuth (110.00000°): (slew started)
22:05:32.623: Dome - Statuses: Get Azimuth: 90
22:05:32.623: Dome - Statuses: Get Slewing: True
22:05:32.623: Dome - Commands: Started fast polling every 1000 ms.
22:05:32.623: Dome - Statuses: Get Azimuth: 90
22:05:32.623: Dome - Statuses: Get Slewing: True
22:05:32.624: Dome - Statuses: Get Azimuth: 90
22:05:32.624: Dome - Statuses: Get Slewing: True
22:05:33.635: Dome - Statuses: Get Azimuth: 90.7
22:05:33.635: Dome - Statuses: Get Slewing: True
22:05:34.639: Dome - Statuses: Get Azimuth: 92.9
22:05:34.639: Dome - Statuses: Get Slewing: True
22:05:35.636: Dome - Statuses: Get Azimuth: 95.5
22:05:35.636: Dome - Statuses: Get Slewing: True
22:05:36.643: Dome - Statuses: Get Azimuth: 97.4
22:05:36.644: Dome - Statuses: Get Slewing: True
22:05:37.650: Dome - Statuses: Get Azimuth: 99.8
22:05:37.650: Dome - Statuses: Get Slewing: True
22:05:38.635: Dome - Statuses: Get Azimuth: 102.1
22:05:38.635: Dome - Statuses: Get Slewing: True
22:05:39.639: Dome - Statuses: Get Azimuth: 104.7
22:05:39.640: Dome - Statuses: Get Slewing: True
22:05:40.635: Dome - Statuses: Get Azimuth: 106.8
22:05:40.635: Dome - Statuses: Get Slewing: True
22:05:41.634: Dome - Statuses: Get Azimuth: 109.1
22:05:41.635: Dome - Statuses: Get Slewing: True
22:05:42.634: Dome - Statuses: Get Azimuth: 109.8
22:05:42.634: Dome - Statuses: Get Slewing: True
22:05:43.633: Dome - Statuses: Get Azimuth: 110
22:05:43.634: Dome - Statuses: Get Slewing: False
22:05:47.648: Dome - Commands: Returning to normal polling every 5000 ms.

DomeLink produces the following  log file for the same slew (file was made on a subsequent day.

01:17:15.691 Creating objDome Setting objDome = New ASCOM.DriverAccess.Telescope(ASCOM.Pulsar_Observatories_Dome.Dome)
01:17:15.726 Connect Dome Setting objDome.Connected True
01:17:15.873 Connect Dome Setting connectedState = True
01:18:21.437 SlewToAzimuth SlewToAzimuth(110.0)
01:18:21.452 Azimuth Get Azimuth = 90.0
01:18:21.452 Slewing Get True (Slewing forced to True for 500ms after calling SlewToAzimuth()
01:18:21.452 Azimuth Get Azimuth = 90.0
01:18:21.452 Slewing Get True (Slewing forced to True for 500ms after calling SlewToAzimuth()
01:18:21.453 Azimuth Get Azimuth = 90.0
01:18:21.453 Slewing Get True (Slewing forced to True for 500ms after calling SlewToAzimuth()
01:18:22.460 Azimuth Get Azimuth = 90.0
01:18:22.460 Slewing Get True
01:18:23.467 Azimuth Get Azimuth = 92.0
01:18:23.467 Slewing Get True
01:18:24.477 Azimuth Get Azimuth = 94.3
01:18:24.478 Slewing Get True
01:18:25.462 Azimuth Get Azimuth = 96.5
01:18:25.462 Slewing Get True
01:18:26.460 Azimuth Get Azimuth = 99.2
01:18:26.460 Slewing Get True
01:18:27.461 Azimuth Get Azimuth = 101.3
01:18:27.461 Slewing Get True
01:18:28.475 Azimuth Get Azimuth = 103.6
01:18:28.475 Slewing Get True
01:18:29.476 Azimuth Get Azimuth = 105.9
01:18:29.476 Slewing Get True
01:18:30.479 Azimuth Get Azimuth = 108.3
01:18:30.479 Slewing Get True
01:18:31.476 Azimuth Get Azimuth = 109.7
01:18:31.476 Slewing Get True
01:18:32.476 Azimuth Get Azimuth = 110.0
01:18:32.476 Slewing Get False


Update 2023-12-17

The DomeLink driver was successfully used during the S1173 live session on  2023-12-16, and also on the 5 following session between 2023-12-17 & 2023-12-29 (S1174 -1178).

Update 2024-01-05

A new ASCOM Driver (6.4, dated 2024-01-04) for the Pulsar Dome was installed on the Observatory Computer on 2024-01-05, and was confirmed to have addressed the issue with earlier 6.3 versions of the driver. The new driver correctly returns Slewing=True immediately after making a SlewToAzimuth() request.  This makes the Pulsar System ASCOM compliant or at least behave in the manner expected by an ASCOM Dome Client.
 (See Pulsar Dome - Software Update (Firmware 1.52 & ASCOM Driver 6.4), 2024-01-05 )

The Observatory's Pulsar Dome will sometimes fail to park the dome at the end of a session at the precise point for Induction Charger to recharge the Shutter battery. This can happen even after a recalibration of the dome. When this means that Observatory has to be visited the following morning and the dome nudged under manual control so that the two halves of the induction charger at visually aligned.  Whether the Shutter is charging or not following a Park operation can be gleaned from the Pulsar Dome's ASCOM driver log and historically this is protrayed on Observatory Status Webpage (see Shutter Battery Graphs).  Code has recently been added to AstroMain to raise alerts when recharging does commence after parking the dome. This is useful but doesn't preclude the need to visit the Observatory to nudge the Dome.

A new routine 'SearchForBestParkPosition()' has been been developed that searches across a range in azimuth (90° +/- 5°, step 0.5°) to find positions where charging occurs and then uses the Azimuth with highest charge rate to slew the dome to that position.  This routine takes around 45 minutes to run due to the need to wait up to 140s at each Azimuth point to get fresh Shutter Battery information (the Dome Controller seems to poll the Shutter every 135s).   The routine is also only efffective when the Shutter Battery is slightly depleted (which is normal after a session), it may not work when the Shutter is fully charged and charging pulse only happens every 15 mins of so.

In theory the Dome can be asked to Find Home (180°) and then move to Park Position (90°), in practice this can't be used as the dome's Home Magnet is not be left permanently in place because it quickly corrodes and looses magnetic strength, and therefore taken out when not running a Dome Calibration.

It was decided to add a 'At Park' Sensor that could independantly used to find the optimal Park Position, with the idea that it could be run via a Remote VNC Connection (avoiding the need to visit the Observatory) or could be run automatically at the end of the session (avoiding the need for any user intervention at all). The idea was that the Open/Close State of the sensor could be used as a proxy for AtPark (Parked) State with  Open=Unparked, Closed=Parked. It was believed that the Sensor Element could be attached to the top of one of the dome's roof clamps (stationary) with the magnet element attached to the roof (ie it rotates with the dome), and that this could be done without interferring with the rotation and use of the dome.

A new Zigbee Sensor of the same type as previously used for Observatory Door and Shutter Top/Bottom and Greenhouse Vent (Aqara MCCGQ11LM Door & Window Sensor, https://www.amazon.co.uk/Aqara-MCCGQ11LM-Door-Window-Sensor/dp/B07D37VDM3) was ordered and it arrived the following day (2023-12-12).

Sensor Installation, Zigbee (2023-12-12)
The new sensor was installed on the observatory's zigbee network (AstroGW) on 2023-12-12, using Phoscon from the deCONZ Application. 

The sensor properties record type=ZHAOpenClose, manufacturer=LUMI, modelid=lumi.sensor_magnet.aq, swversion 20161128, and took sensor id = 21. A Zigbee Sensor of the same type for Greenhouse Door, sensor id = 20 was installed at the same time.   A Network Diagram showing Observatory's Zigbee Network after the addition of the two new sensors.

AstroMain 3.65.5 was updated to access and utilise the state of the new 'AtPark' Sensor and successfully tested.
 

Sensor Installation, Observatory (2023-12-13)
The Sensor was installed in the Observatory the following day (2023-12-13).  With the Dome at its correct physical Park Location (Az 90), the sensor element was attached to the SW Roof Clamp and the associated magnet element attached to the top surface of the roof flange in a position directly opposite the sensor.  The magent element was raised up on a small piece of wood so that it was at the same height as the sensor element.  The Dome was then rotated to check for any interference from the new sensor or other issue

'AtPark' Sensor Sensor element attached to Roof Clamp (SW) Magnet element attached to roof flange (raised up on small piece of wood)		'AtPark' Sensor - Top View Magnet Element positioned so that it lies opposite the Sensor element when Dome is at its physical Park position (note that the separation between the two elements was widened the following day after more detailed testing of FindPark routines)
Dome Check 1 - Fail Rotating the dome by 90 deg quickly uncovered a major issue with the rotation being held up when the sensor element hit the flanges and a nut between adjacent roof quadrants. Initially there was grinding noise from the dome drive and then the sensor was knocked off. (Photo taken with the sensor reattached)		Dome Check 1 - Fail View from side illustrating the issue (nut has been removed by this stage) Note: The issue effects two of the four quadant junctions  (N & S). The other two junctions (ones where the shutter is (E) and the one opposite (W) ) weren't affected as their flanges are set further back
Dome Check 1 - Remedial Fix Section of Flange cutout and nut relocated to a higher position		Dome Check 1 - Pass (A) After remedial fix the dome sensor passes the S quadant junction ok and doesn't affect sensor state
Dome Check 1 - Pass (B) After remedial fix the dome sensor passes the N quadant junction ok  and doesn't affect sensor state		Dome Check 2 - Pass Magnet Element clears all 4 roof clamps (Note: the offset noted to vary from clamp to clamp)
Dome Check 3 - Pass Sensor Element clears the Induction Receiver ok and doesn't affect sensor state		Dome Check 4 - Pass Sensor Element clears the Shutter Unit and doesn't affect sensor state

Find Park / SearchForBestParkPosition Routine
The SearchForBestParkPosition() routine was modified to use the 'AtPark' (Parked) state of the new AtPark Sensor.  As the process is a lost faster than earlier method looking at the amperage in mA going to the Shutter Battery, a step size of 0.2° or 0.1° can easily be used.

The routine searches for the first and last azimuth with 'AtPark' = True and then take the midpoint of this 'AtPark Range' as the Best Park Azimuth.  The routine was successful in finding correct Park Position
taking 2.3 minutes. The detailed results were surprising however. Instead of a single and hopefully fairly narrow 'At Park' zone, the routine detected three 'AtPark' zones with 'UnParked zones in between, such that the overall 'At Park Zone' was 3.8° wide.   It is supposed that the Sensor which is designed to sense the magnet coming directly toward the sensor (a normal Open-Closed situation) isn't designed or intended for a situation where the magnet slides past the sensor. 

 Search for Optimal Park Position      2023-12-13 19:24  (Local, GMT)
  Search Angle                Info      Search 90.0° +/- 4°, Step 0.2°

  Jog Dome (85.0°)            Ok        Az: 85.0°
  Jog Dome (85.2°)            Ok        Az: 85.2°
  Jog Dome (85.4°)            Ok        Az: 85.4°
  Jog Dome (85.6°)            Ok        Az: 85.6°
  Jog Dome (85.8°)            Ok        Az: 85.8°
  Jog Dome (86.0°)            Ok        Az: 86.0°
  Jog Dome (86.2°)            Ok        Az: 86.2°
  Jog Dome (86.4°)            Ok        Az: 86.4°
  Jog Dome (86.6°)            Ok        Az: 86.6°
  Jog Dome (86.8°)            Ok        Az: 86.8°
  Jog Dome (87.0°)            Ok        Az: 87.0°
  Jog Dome (87.2°)            Ok        Az: 87.2°
  Jog Dome (87.4°)            Ok        Az: 87.4°
  Jog Dome (87.6°)            Ok        Az: 87.6°
  Jog Dome (87.8°)            Ok        Az: 87.8°
  Jog Dome (88.0°)            Ok        Az: 88.0°
  Jog Dome (88.2°)            Ok        Az: 88.2°
  Jog Dome (88.4°)            Ok        Az: 88.4°, AtPark
  Jog Dome (88.6°)            Ok        Az: 88.6°, AtPark
  Jog Dome (88.8°)            Ok        Az: 88.8°, AtPark
  Jog Dome (89.0°)            Ok        Az: 89.0°, AtPark
  Jog Dome (89.2°)            Ok        Az: 89.2°, AtPark
  Jog Dome (89.4°)            Ok        Az: 89.4°
  Jog Dome (89.6°)            Ok        Az: 89.6°
  Jog Dome (89.8°)            Ok        Az: 89.8°, AtPark
  Jog Dome (90.0°)            Ok        Az: 90.0°, AtPark
  Jog Dome (90.2°)            Ok        Az: 90.2°, AtPark
  Jog Dome (90.4°)            Ok        Az: 90.4°, AtPark
  Jog Dome (90.6°)            Ok        Az: 90.6°, AtPark
  Jog Dome (90.8°)            Ok        Az: 90.8°, AtPark
  Jog Dome (91.0°)            Ok        Az: 91.0°
  Jog Dome (91.2°)            Ok        Az: 91.2°
  Jog Dome (91.4°)            Ok        Az: 91.4°, AtPark
  Jog Dome (91.6°)            Ok        Az: 91.6°, AtPark
  Jog Dome (91.8°)            Ok        Az: 91.8°, AtPark
  Jog Dome (92.0°)            Ok        Az: 92.0°, AtPark
  Jog Dome (92.2°)            Ok        Az: 92.2°, AtPark
  Jog Dome (92.4°)            Ok        Az: 92.4°
  Jog Dome (92.6°)            Ok        Az: 92.6°
  Jog Dome (92.8°)            Ok        Az: 92.8°
  Jog Dome (93.0°)            Ok        Az: 93.0°
  Jog Dome (93.2°)            Ok        Az: 93.2°
  Jog Dome (93.4°)            Ok        Az: 93.4°
  Jog Dome (93.6°)            Ok        Az: 93.6°
  Jog Dome (93.8°)            Ok        Az: 93.8°
  Jog Dome (94.0°)            Ok        Az: 94.0°

  Best Park Position          Ok        Best Park at Az 90.3° (range 88.4 - 92.20°, park width 3.8°
  Slew To Best Park Az..      Ok        Az: 90.3°

Whilst the routine finds Best Park Positon ok, the wide 'At Park' Zone (3.8°)  with two intervening 'Unparked' zones, means that the sensor would act as a good independant indicator of the park status of the dome.   With Dome positioned +1.9° or -1.9° away from Park would show 'AtPark' but the Induction Charger alignement wouldn't  allowing charging of the Shutter battery.
 
It is proposed to trial the Magnet Element so that it is orientated perpendicular to the Sensor Element instead of parallel.

Update 2023-12-14
Trial conducted with long axis of Magnet Element oriented perpendicular to the Sensor Element, with a Step Size of 0.2°.  This showed a smaller 'At Park' Zone of 3.0° instead of 3.8°, but again had a strange profile.   In detail there were two AtPark zones separated by a narrow 'Unparked' zone.  

  Search for Optimal Park Position      2023-12-14 13:37  (Local, GMT)
  Search Angle                Info      Search 90.0° +/- 3°, Step 0.2°

  Jog Dome (87.0°)            Ok        Az: 87.0°
  Jog Dome (87.2°)            Ok        Az: 87.2°
  Jog Dome (87.4°)            Ok        Az: 87.4°
  Jog Dome (87.6°)            Ok        Az: 87.6°
  Jog Dome (87.8°)            Ok        Az: 87.8°
  Jog Dome (88.0°)            Ok        Az: 88.0°
  Jog Dome (88.2°)            Ok        Az: 88.2°
  Jog Dome (88.4°)            Ok        Az: 88.4°
  Jog Dome (88.6°)            Ok        Az: 88.6°, AtPark
  Jog Dome (88.8°)            Ok        Az: 88.8°, AtPark
  Jog Dome (89.0°)            Ok        Az: 89.0°, AtPark
  Jog Dome (89.2°)            Ok        Az: 89.2°, AtPark
  Jog Dome (89.4°)            Ok        Az: 89.4°, AtPark
  Jog Dome (89.6°)            Ok        Az: 89.6°, AtPark
  Jog Dome (89.8°)            Ok        Az: 89.8°, AtPark
  Jog Dome (90.0°)            Ok        Az: 90.0°, AtPark
  Jog Dome (90.2°)            Ok        Az: 90.2°
  Jog Dome (90.4°)            Ok        Az: 90.4°, AtPark
  Jog Dome (90.6°)            Ok        Az: 90.6°, AtPark
  Jog Dome (90.8°)            Ok        Az: 90.8°, AtPark
  Jog Dome (91.0°)            Ok        Az: 91.0°, AtPark
  Jog Dome (91.2°)            Ok        Az: 91.2°, AtPark
  Jog Dome (91.4°)            Ok        Az: 91.4°, AtPark
  Jog Dome (91.6°)            Ok        Az: 91.6°, AtPark
  Jog Dome (91.8°)            Ok        Az: 91.8°
  Jog Dome (92.0°)            Ok        Az: 92.0°
  Jog Dome (92.2°)            Ok        Az: 92.2°
  Jog Dome (92.4°)            Ok        Az: 92.4°
  Jog Dome (92.6°)            Ok        Az: 92.6°
  Jog Dome (92.8°)            Ok        Az: 92.8°
  Jog Dome (93.0°)            Ok        Az: 93.0°

  Best Park Position                    2023-12-14 13:39  (Local, GMT)
  Best Park Position          Ok        Best Park at Az 90.1° (range 88.6 - 91.60°, park width 3.0°
  Slew To Best Park Az..      Ok        Az: 90.1°

The Search run was repeated using a smaller step size of 0.1° but showed a similar result with a narrow unparked zone.  Although the Best Park Position is suitable for finding the best Park Position for the Dome, this configuration has the issue that with the dome at the determined best park position the AtPark Sensor shows the dome as being Unparked, which would limit the use of the sensor to finding AtPark . It can't be used as an independant indicator of the Park Status of the dome.  Increasing the separation between the Magnetic and Sensor Elements produced no change to this two zone pattern or was too far away to see any At Park Zone.

  Jog Dome (88.0°)            Ok        Az: 88.0°
  Jog Dome (88.2°)            Ok        Az: 88.2°
  Jog Dome (88.4°)            Ok        Az: 88.4°
  Jog Dome (88.6°)            Ok        Az: 88.6°, AtPark
  Jog Dome (88.8°)            Ok        Az: 88.8°, AtPark
  Jog Dome (89.0°)            Ok        Az: 89.0°, AtPark
  Jog Dome (89.2°)            Ok        Az: 89.2°, AtPark
  Jog Dome (89.4°)            Ok        Az: 89.4°, AtPark
  Jog Dome (89.6°)            Ok        Az: 89.6°, AtPark
  Jog Dome (89.8°)            Ok        Az: 89.8°, AtPark
  Jog Dome (90.0°)            Ok        Az: 90.0°
  Jog Dome (90.2°)            Ok        Az: 90.2°
  Jog Dome (90.4°)            Ok        Az: 90.4°, AtPark
  Jog Dome (90.6°)            Ok        Az: 90.6°, AtPark
  Jog Dome (90.8°)            Ok        Az: 90.8°, AtPark
  Jog Dome (91.0°)            Ok        Az: 91.0°, AtPark
  Jog Dome (91.2°)            Ok        Az: 91.2°, AtPark
  Jog Dome (91.4°)            Ok        Az: 91.4°, AtPark
  Jog Dome (91.6°)            Ok        Az: 91.6°
  Jog Dome (91.8°)            Ok        Az: 91.8°
  Jog Dome (92.0°)            Ok        Az: 92.0°

  Best Park Position                    2023-12-14 13:47  (Local, GMT)
  Best Park Position          Ok        Best Park at Az 90.0° (range 88.6 - 91.40°, park width 2.8°
  Slew To Best Park Az..      Ok        Az: 90.0°

The position of the Magnetic Element was changed back to being parallel to the Sensor Element, and a series of experienents conducted changing the separation distance.  Increasing the separation it was found that the original triple 'AtPark' zone profile with intervening unparked zones was replaced by a single 'AtPark' zone profile, and at the same time the width of overall 'AtPark' zone reduced from 3.8° to 1.2°.  

 Search for Optimal Park Position      2023-12-14 14:06  (Local, GMT)
  Search Angle                Info      Search 90.0° +/- 3°, Step 0.2°

  Jog Dome (87.0°)            Ok        Az: 87.0°
  Jog Dome (87.2°)            Ok        Az: 87.2°
  Jog Dome (87.4°)            Ok        Az: 87.4°
  Jog Dome (87.6°)            Ok        Az: 87.6°
  Jog Dome (87.8°)            Ok        Az: 87.8°
  Jog Dome (88.0°)            Ok        Az: 88.0°
  Jog Dome (88.2°)            Ok        Az: 88.2°
  Jog Dome (88.4°)            Ok        Az: 88.4°
  Jog Dome (88.6°)            Ok        Az: 88.6°
  Jog Dome (88.8°)            Ok        Az: 88.8°
  Jog Dome (89.0°)            Ok        Az: 89.0°
  Jog Dome (89.2°)            Ok        Az: 89.2°
  Jog Dome (89.4°)            Ok        Az: 89.4°
  Jog Dome (89.6°)            Ok        Az: 89.6°, AtPark
  Jog Dome (89.8°)            Ok        Az: 89.8°, AtPark
  Jog Dome (90.0°)            Ok        Az: 90.0°, AtPark
  Jog Dome (90.2°)            Ok        Az: 90.2°, AtPark
  Jog Dome (90.4°)            Ok        Az: 90.4°, AtPark
  Jog Dome (90.6°)            Ok        Az: 90.6°, AtPark
  Jog Dome (90.8°)            Ok        Az: 90.8°
  Jog Dome (91.0°)            Ok        Az: 91.0°
  Jog Dome (91.2°)            Ok        Az: 91.2°
  Jog Dome (91.4°)            Ok        Az: 91.4°
  Jog Dome (91.6°)            Ok        Az: 91.6°
  Jog Dome (91.8°)            Ok        Az: 91.8°
  Jog Dome (92.0°)            Ok        Az: 92.0°
  Jog Dome (92.2°)            Ok        Az: 92.2°
  Jog Dome (92.4°)            Ok        Az: 92.4°
  Jog Dome (92.6°)            Ok        Az: 92.6°
  Jog Dome (92.8°)            Ok        Az: 92.8°
  Jog Dome (93.0°)            Ok        Az: 93.0°

  Best Park Position                    2023-12-14 14:08  (Local, GMT)
  Best Park Position          Ok        Best Park at Az 90.1° (range 89.6 - 90.60°, park width 1.0°
  Slew To Best Park Az..      Ok        Az: 90.1°

Whilst this produced better results,  the slew to best park position didn't take align the Induction Charger perfectly - it was out by 0.2° of so.   This was resolved by adjusting the position of the Magnetic Element slightly to the right of centre. The final configuration of the Sensor parts is shown below (right).

'AtPark' Sensor - Original Configuration With Dome at its physical Park position the Magnet Element is positioned so that it lies close to and directly opposite the Sensor Element		'AtPark' Sensor - Final  Configuration With Dome at its physical Park position the Magnet Element is positioned so that it lies slightly further away from the Sensor Element and lies slightly to the right of centre.

With the speed that the Zigbee Events are generated and picked up by the AstroMain, it might theoretically be possible to slew directly across the search range and detect 'AtPark' True/False changes on the fly. This might run into an issue if the thread doesn't recieve sufficient CPU time to constantly interrogate the 'AtPark Sensor' state. But a greater problem is that with the dome slewing at up to 2°/s , a 0.2° change in Az would therefore take place in just 0.1s, but DeviceHub.Dome which is used in the communication with the Dome (via Pulsar Dome Driver) can only poll the dome every 1 to 1.3s (shortest possible time when on fast polling), so measuring an accurate 'AtPark' profile isn't feasible.  The best that could be done is to measure the fairly infrequent Azimith values and then perform a linear interpolation to derive inferred azimuth values at the times when AtPark state changes occur. This could only be done on completion of the whole slew.   Despite these limitations the approach might be suitable for a wide search range to get an approximate position for AtPark azimuth, and then follow it up with a step based search on a much narrower range (say +/-4°).

A new Zigbee Sensor of the same type as previously used for Greenhouse Roof Vent  (Aqara MCCGQ11LM Door & Window Sensor, https://www.amazon.co.uk/Aqara-MCCGQ11LM-Door-Window-Sensor/dp/B07D37VDM3) was ordered and it arrived the following day (2023-12-12).  It will be used for monitoring the Open/Closed status of the Greenhouse Door.

Sensor Installation, Zigbee (2023-12-12)
The new sensor was installed on the observatory's zigbee network (AstroGW) on 2023-12-12, using Phoscon from the deCONZ Application. 

The sensor properties record type=ZHAOpenClose, manufacturer=LUMI, modelid=lumi.sensor_magnet.aq, swversion 20161128, and the sensor took sensor id = 20. A Zigbee Sensor of the same type was installed a the same time to act as an Observatory 'AtPark' Indicator. 

AstroMain 3.65.5 was updated to access and utilise the state of the new 'Greenhouse Door' Sensor and successfully tested.

Sensor Installation, Greenhouse (2023-12-13)
The Sensor was installed on the inside of the Greenhouse Door the following day (2023-12-13). 

The state of the Observatory Door is shown on the AstroMain's  More/Greenhouse tab and on the Greenhouse Virtual Picture that is updated every 30s and posted to the Observatory Website.

After a fairly continuous spell of wet weather water it was noted on around 2023-12-10 or 11 that the observatory floor mat was wet  and there were signs of water coming up between the joins on some of the interlocking rubber floor tiles.

The floor mat and interlocking tiles were lifted a couple of days later during a break in the weather (2023-12-13).  Picture below show the distributuon of the water ingress. The floor is wet in E (bottom) SE, E (left) and  SW directions (top-left). It's hard to tell whether the water has come in from one or two places and then spread out or it  has come in along wider sections of the wall.  The N (right) and NW (top-right direction are mostly dry and the area around the pier is notably dry.



With the tiles lifted, the floor proceeded to dry out over the next 18 hours of at least by the time the Observatory was checked the following day (2023-12-14). 

The graphs below show the extent of wet weather over the last week. Every day has had rain, with the heaviest and most prolonged rain being on Dec 7th-8th.

This problem of water ingress has been seen before after certain spells of wet weather and is due to seepage of water from below the observatory wall and/or coming up the bolt holes associated with bolts that hold down the observatory.

With the outside circumference of the Observatory's Concrete pad being almost continously wet/damp in winter and with water/damp still present below the Observatory walls a remedial fix isn't possible at this time.  Replacemen of the silicone seal around the walls will have to wait till until next spring/summer next year during a suitably dry spell.

Issue

1.  LAN Connection to Observatory went down, stopping Uploads to Observatory Computers and stopping remote access & window explorer access  (2023-12-09)
2. LAN connection to Observatory went down, stopping uploads to Observatory Website  (2023-12-10 22:43)
3.  LAN Connection to Observatory went down (again), stopping upload of information to website and stopping remote access & window explorer access (2023-12-13 22:29)
4. LAN Connection to Observatory went down (again), stopping upload of information to website (2023-12-15 20:00).

Description

1) LAN Connection to Observatory went down, stopping Uploads to Observatory Computers and stopping remote access & window explorer access (2023-12-09). Uploading of fresh data to Observatory Website from both the Observatory Computer and the AllSky/Weather Computer stopped at around 16:45 on 2023-12-09. At the same time and for the remainder of the evening  the Development Computer lost LAN Network access to both computers, and it wasn't possible to make a VNC Connection to either computer. Observatory was visited and a power reset of the Ethernet Over Powerline plug was made which resolved both issues. LAN connection wouldn't come back by itself.  Issue couldn't be resolved  by restarting the Ethernet Over Powerline plug at the house/router end.  Issue resolved by visiting the Observatory to restart the Observatories Ethernet Over Powerline plug.   Whilst the LAN was down Observatory Computer and AllSky/Weather Computer seemed to continue TCP/IP communication as they use a command Network Switch unit.  Upload of fresh data to Website from both computers didn't automatically restart and AstroPlan & AstroAllSky (containing the web uploaders for the two computers) had to be manually restarted.  A LAN Connection issue also happened the following evening (2023-12-10) however it restored itself in 10 minutes.  It is possible power reset of the 'Ethernet Over Powerline' plug near the router, then forced a situation that then required the later power reset on the Observatory's 'Ethernet Over Powerline' plug, meaning the Observatory's LAN connection  couldn't then resolve itself, when prehaps the rest of the network did resolve itself ?  A separate Continuous Improvement ticket has been raised to consider adding a Smart Plug to the Zigbee Network to allow automated resetting of Observatory's LAN connection.

2)  LAN Connection to Observatory went down, stopping uploads to Observatory Website  (2023-12-10 22:43). The Observatory's LAN connection was lost for 10 minutes at 22:43 - 22:53 based on LAN flag on ObsPic pictures. Uploading of fresh data to Observatory Website from both the Observatory Computer (AstroPlan program) and the AllSky/Weather Computer (AstroAllSky program) stopped and didn't resume even though the LAN connection restored itself.   (Unlike the LAN Connection issue on the previous day (2023-12-09) the Ethernet Over Powerline reboots were required). Windows/Remote Access to observatory computers came back at the end of the LAN outage.  Upload of files to Observatory Website was manually restored by restarting AstroPlan and AstroAllSky programs over a VNC remote connection, which reenabled their respective file uploaders. Next time this happens did to see if upload to website is possible from Development Computer to see if issue is an issue of internet connection via the ISP service, or a problem on the home network. This might not isolate the problem as the Observatory is using LAN connection to the router, whilst the Development Computer is using a Wifi connection.  Error message logged by AstroPlan at the time of the outage were
  2023-12-10 22:42:06.55 | RunFtpQueue has finished uploading a batch of 3 files
  2023-12-10 22:43:35.28 | FTP Overseer >> FTP Queue Loop stalled at 2023-12-10 22:42:34
  2023-12-10 22:43:35.28 | FTP Overseer >> Stall point is inside UploadFileFTP2
  2023-12-10 22:45:22.37 | Exception in 'UploadFileFTP2' during PutFiles() for Session.Header.dat
  2023-12-10 22:45:22.37 | Error: Network error: Software caused connection abort.
  2023-12-10 22:45:22.37 | Copying files to remote side failed.
  2023-12-10 22:45:22.37 | Exception in 'UploadFileFTP2' during PutFiles() for Session.Header.dat
  2023-12-10 22:45:22.37 | Error: Session has unexpectedly closed
  2023-12-10 22:45:22.37 | Exception in 'UploadFileFTP2' during PutFiles() for GreenhousePic.gif
  2023-12-10 22:45:22.37 | Error: Element session@0 already read to the end
  2023-12-10 22:45:22.37 | Exception in 'UploadFileFTP2' during PutFiles() for Session.ObsManagerTab.png
  2023-12-10 22:45:22.37 | Error: Element session@0 already read to the end
  2023-12-10 22:45:22.37 | Exception in 'UploadFileFTP2' during PutFiles() for Session.ServicesTab.png
  2023-12-10 22:45:22.37 | Error: Element session@0 already read to the end
  2023-12-10 22:45:22.37 | FTP Queue >> FTP stopped at 2023-12-10 22:42:06
  2023-12-10 22:45:22.37 | Exception in 'UploadFileFTP2' during PutFiles() for Session.Header.js
  2023-12-10 22:45:22.37 | Error: Element session@0 already read to the end
The 'Element session@0 already read to the end' errors then continued for each subsequent attempt to upload a file.  A separate Continuous Improvement ticket has been created to add a button to each of these two programs that can restart each program's file uploader using a fresh WinSCP Connection.

3) LAN Connection to Observatory went down (again), stopping upload of information to website and stopping remote access & window explorer access (2023-12-13 22:29). VNC remote access to Observatory Computer and AllSky/Weather Computer went down at 22:29.  Checks showed that the Internet was accessible from Development Computer (over Wifi) and that LAN access to Internet and 'Ethernet Over Powerline' was working ok for the house's SkyHD Box. After waiting more than hour to see if VNC Remote Access to observatory computers would come back the Observatory was visited that confirmed that it had no access to the internet and that there were previously seen exceptions in AstroPlan and AstroAllSky (Error: Network error: Software caused connection abort) from WinSCP/SecureTransfer object.  Issue resolved at 23:58 by restarting the Observatories Ethernet Over Powerline plug (proving that there was no need to restart the equivalent plug at the router end, and strengthening the proposal to get a Zigbee Controlled Smart Plug for Powerline Adapter ).  The situation provided opportunity to use the 'Restart Loader' buttons in AstroPlan and AstroAllSky to restart their SecureTransfer connections and resume uploading files to Observatory Website, without having to completely restart both programs.  This was the third time in five days that the LAN Communication to Observatory has gone down.  [ Note:  Router admin in 192.168.1.254  not 192.168.1.1 ]

4) LAN Connection to Observatory went down (again), stopping upload of information to website (2023-12-15 20:00). Checking on Observatory Website at 00:55 showed that uploads from Observatory Computer and AllSky/Weather Computer had seemingly stopped with the last AllSky, WeatherGraph and Obsrevatory pictures having timestamps around 20:00 .  Investigating logs confirmed that exceptions from SecureTransfer connection began at 20:02 and were still be produced at 00:55. A user intervention was made to restart SecureTransfer/Web Upload using AstroPlan and AstroAllSky's  'Restart Loader' buttons . Checking ObsPics showed that the LAN went down just before 20:00 but came back at 20:07.  Whilst the LAN outage was only 7 minutes long the outage in web uploads was 5 hours (and would have lasted to the morning if the issue hadn't been recognised and an intervention made. A continuous improvement ticket has already been raised to make AstroPlan and AstroAllSky programs automatically restart the web loader if/when the LAN connection is restored (this has now been completed 2023-12-16)

Analysis 
xx

Conclusion

xx

Actions

• Look ...