Somewhen between 23:15 and 23:20 it was noticed that there was an outage in scorecard / weather data on the Obs.Manager and AstroGuard screens. There was also issue with communications with AllSky computer and Observatory Computer to a lesser degere. With live session underway and an absence of scorecard & weather services a late night visit out to the observatory was made.   Eventually services were restored at 23:33-23:35, but seemingly only after restarting the AllSky Computer and suffering an unexpected closue of the dome at 23:33

Obs Monitor Log (on Laptop9)
Weather data dropped out at ~ 23:15 coming back at ~ 23:20.   (5 minute outage)
Data dropped out again at ~23:27 coming back at ~ 23:34 (7 minute outage)

Obs. Env Log (on Laptop7)
Obs/Env data dropped out at 23:15 to 23:20 (5 minute outage )
Data dropped out again at 23:27 coming back at 23:34 (7 minute outage)

AllSky Stars Log
Log shows continuous data through 23:14 to 23:20
but shows a data gap between 23:27 and 23:32 (with reduced data till 23:34)

AllSky Log
Log shows continuous operations/data through 23:14 to 23:20
Program aborted around 23:27 and restarted at 23:32

AstroGuard log
Scorecard reported as being silent for 5 mins at 23:19.   Scorecard reported as resumed at 23:20.
This is consisent with Network comms being out for 5 minutes between 23:14 and 23:19

A Telescope Safety Alert was given at 23:33 'Light Conditions are a warning'
following by  message announching the action 'Closing Shutter due to Light Conditions'

CloudSensor data file
File shows continuous data through 23:14 to 23:20
but shows a data gap between 23:27 and 23:35

The first instance was seen by AstroGuard as a outage, the second instance was presumably short enough not to raise a flag. 

During first outage AstroMain' Job Executor was busy collecting a set of 5x180s frames, and by the time the job queue examined the Current Conditions at 23:20 the weather data stream was back and running.

Conclusion:  
The LAN seems to have gone down at 23:15 for some unknown reason.

This was observed as an outage in scorecard / weather data in Obs.Manager and AstroGuard screens.   Although the outage lasted only 5 minutes (and came back by itself), problems trying to raise the AllSky Computer during this time using VNC/Network created enough concern to visit the Observatory.   User overreaction to the problem then contributed to a decision to directly access and ultimately restart the AllSky Computer  causing a second data outage.

After restarting the AllSky Computer at 23:32 the observatory shutter unexpectedly closed at 23:33 and the session suspended itself until 23:44.

The job executor was busy collecting a set of 5x180s frames but was interrupted by a Dome Closure alert ("Dome is unexpectedly closing (Poor Conditions), 2019-10-01 23:33".  The current target was aborted "due to other conditions" and the next target was delayed by suspension  'due to non-defined reason (0)'      The Alert didn't follow the normal Critical Conditions Alert message style.  

Analysis

Examining the code indiates this alert  is produced in the situation where the Shutter is detected as Closing but the Closure hasn't been commanded by the ObsManager /AstroMain.  

Examing the AstroGuard log showed that at 23:33 it gave a telescope safety alert and actioned closure of the shutter due to 'Light Conditions'

Closing the shutter due to 'Light Conditions' at this time when the Sun was well below the horizon (SunAlt -33.5deg) is clearly not expected.  
There was no indication of any cloud at this time that could have triggered the light conditions. or anomalous local light source
It seems more than just coincidence that this Light Condition alert came at about the same that scorecard services came back on line. 
 
When the Scorecard service came back online at 23:33 it did so before the Cloud Sensor was fully online (which wasn't until 23:35). For these two minutes the Scorecard/Synopsis gave a 'Light Warning', which only dropped when data from light sensor (0 light) came through.

Examing the AstroGuard code shows that it commands a Dome Closure when Shutter is open and there is a Light Warning

Conclusion:  
The AstroGuard design would appear to be overreactive.   It should have
  a) given time for AstroMain to react first  and
  b) give the time for the warning to go away by itself. 

Scorecard is probably correct in giving a warning, but should indicat it someway that it has only just started up and data values may not have settled down.

 

have gone down at 23:15 which was observed at an outage in scorecard / weather data in Obs.Manager and AstroGuard screens.

Although the outage lasted only 5 minutes (and came back by itself), problems trying to raise the AllSky Computer using VNC/Network created enough concern to visit the Observatory.   Overreaction to the problem contributed to a decision to directly access and ultimately restart the AllSky Computer.   causing a second data outage.

The first instance was seen by AstroGuard as a outage, the second instance was presumably short enough not to raise a flag.

During first outage AstroMain' Job Executor was busy collecting a set of 5x180s frames, and by time the job queue examined at the Current Conditions at 23:20 the weather data stream was back and running.

During the second outage AstroMain'

Synopsis.CriticalLight = ThreatLevel.Warning