Data showing additional information about the status of the Observatory and about the current or latest observing session are now being feed to a new web page.  

The primary purpose of this is to allow the early morning observatory check to be faster, and specifically to check that the observatory has been shutdown properly (ie shutter fully closed, dome parked, telescope parked).   Its secondary purpose is to be an aid to quickly monitoring a live observing session monitoring by showing latest information about progress vs plan,  the targets completed (including any ToO targets), latest session events, latest focus profiles and latest slew/centering data.

Data is uploaded to website by 'AstroPlan' which is always running during a live session, when specific data files in the 'Live' folder change (eg 'Session.FocusProfile1.gif' or when requested by 'AstroMain' after inserting a new event or session observation into the database or modifiying the status of a planned target observation.   

Screen capture from new Observatory Session web page (2019-09-04 am)

Inclusion of html data (Session Events, Slew/Centering data) is done by the method described on the
 W3 Schools 'How TO - Include HTML' page  (https://www.w3schools.com/howto/howto_html_include.asp)

Attempts to include htm using <Object> or <iFrame> had issues with IPad (IOS) scrolling and inability to set the length show to that of the document
(resulting in either too much white space if included document was short or overflow if the document was longer than the div, iframe or object height)

Display of Slew/Centering data (non-html ascii data with tabs & line breaks) is enabled by creating a special 'Session.Slew.htm' file for upload in which the original data is surrounded by <pre> </pre> tags, which allows the data to be displayed in a fxied-wdith font and presevres both spaces and line breaks  (W3 School HTML <pre> Tag)

Whilst starting the 4th Mosaic Point for Target 21/25 (NGC 1817) thirteen critical cloud alert messages were reported by the AdvMain program within a 60s period at 03:55, interrupting the flow of the Report and Log File reporting of the session. Whilst the messages didn't interfere with the automated task of recognisng a critical condition, closing the observatory shutter and suspending the session, they are nevertheless annoying, and can complicate the task of analysing the performance of the Observatory Manager & its coopeartion with the Job Queue executor and make it more difficult to diagnosing and fixing any problems that occur.

Report File Messages 03:51 to 03:57

  Mosaic Point 3/12
  Checking Altitude ...       Ok        Altitude          32.7 degs
  Checking Azimuth ...        Ok        Azimuth           112.2 degs
  Slewing ...                 Ok        Mosaic A3           
  Checking Dome Position ...  Ok
  Guidescope Guiding          Ok        Using PHD2 (null), Guide Exposure null
  Send Guide Request
    Guiding ...               Ok        Guide Request Returned
    Lock Position             OK        X: 775.4, Y: 115.4
    Star Selected             OK        X: 775.4, Y: 115.4
    Settling...               Fail      Error: timed-out waiting for guider to settle
    Review Guiding ...        Fail      Guiding failed to start within Settle Period

  Taking 45s image (C)...     Ok        [00689005] 02:53:10 (UT)  Full 3x3 45s C
  Taking 45s image (C)...     Ok        [00689006] 02:54:07 (UT)  Full 3x3 45s C

  Mosaic Point 4/12
  Checking Altitude ...       Ok        Altitude          33.0 degs
  Checking Azimuth ...        Ok        Azimuth           112.4 degs
  Slewing ...                 Ok        Mosaic A4           
  Checking Dome Position ...  Ok
  Guidescope Guiding          Ok        Using PHD2 (null), Guide Exposure null
  Send Guide Request
    Lock ...                  Fail      Lock Position Lost
    Guiding ...               Ok        Guide Request Returned
    Lock Position             OK        X: 2906.3, Y: 444.6
    Star Selected             OK        X: 2906.3, Y: 444.6
    Lock ...                  Fail      Lock Position Lost
    Star ...                  Fail      Star lost - low HFD

---

Observatory Alert                       2019-09-04 03:55  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)
    Lock Position             OK        X: 2633.1, Y: 658.8
    Star Selected             OK        X: 2633.1, Y: 658.8
    Lock ...                  Fail      Lock Position Lost
    Star ...                  Fail      Star lost - low SNR

---

Observatory Alert                       2019-09-04 03:55  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)
    Lock Position             OK        X: 2633.7, Y: 659.0
    Star Selected             OK        X: 2633.7, Y: 659.0

---

Observatory Alert                       2019-09-04 03:55  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)
    Settling Begun            Ok        

---

Observatory Alert                       2019-09-04 03:55  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)

---

Observatory Alert                       2019-09-04 03:55  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)
    Lock Position             OK        X: 2656.8, Y: 692.4
    Guiding ...               Ok        Guiding Started

---

Observatory Alert                       2019-09-04 03:55  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)
    Star ...                  Fail      Star lost - low HFD

---

Observatory Alert                       2019-09-04 03:56  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)

---

Observatory Alert                       2019-09-04 03:56  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)
    Star ...                  Fail      Star lost - low HFD
    Settling...               Fail      Error: timed-out waiting for guider to settle
    Review Guiding ...        Fail      Guiding failed to start within Settle Period

  Taking 45s image (C)...     

---

Observatory Alert                       2019-09-04 03:56  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)

---

Observatory Alert                       2019-09-04 03:56  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)
Failed    ()
  Taking 45s image (C)...     

---

Observatory Alert                       2019-09-04 03:56  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)

---

Observatory Alert                       2019-09-04 03:56  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)
Failed    ()

  Target Aborting ...         OK        Target aborting due to cloud

---

Session Suspended                       2019-09-04 03:56  (Local)
  Session: 2019-09-03         Info      Session suspended

---

Observatory Alert                       2019-09-04 03:56  (Local)
Critical Cloud Conditions:    Alert     Observatory Manager will close the Dome Shutter
Reason To Suspend             Info      Cloud (State.11)

---

GCVS RR Tau  (22/25)                    2019-09-04 03:56  (Local)
  Target:                     Ok        GCVS RR Tau       (ID: 224)
  Type:                       Ok        Variable
  Program:                    Ok        Main Variables    (ID: 12)

  Planned Start..             Ok        2019-09-04 04:10  (Scheduled)
  Planned End..               Ok        2019-09-04 04:20  (Scheduled) Duration : 10.6 mins
  Session Status:             Hold      Session is suspended due to cloud
  Waiting on weather...       Hold      Wait for conditions to improve
  Close Shutter...            Closing   
  Close Shutter...            Ok        Shutter is Closed (48s)

The report file messages show Cloud Alert messages every 5s (which is the cycle time of the main Observatory Manager loop) interspersed with messages from the Job Queue executor doing the actual target positioning, autoguide initiating & imaging (Log File messages tell a similar story).  It can be seen that execution has moved to the next target (GCVS RR Tau) after the Session has entered State.Suspended but before the shutter has finished closing.  Once the session is suspended the Alert messages stop and Job Queue continues normally with tracking through the remaining targets. Since the cloud conditions continue, each of the remaining target eventually time out. Finally the time slot for target 25/25 expires, the end of the queue is reached and the following message is output :

Script Aborting ...           Aborted   (Script aborted by user)

Now the underlying design (at least the intended design) is that upon detecting a Critical Condition, the Obs. Manager announces it and sets flags that lead to the current target being aborted, and, in event of just cloud (rather than actual rain),  gives time for the Job Executor (TakeFrameJobsQSO,  ImageTargetT_QSO & ImageMosaicT_QSO) to finish aborting the target, place execution into a holding loop and flag this fact to the Obs.Manager. When the return flag is recieved the Obs.Manager then closes the Shutter and places the session into State.Suspended.  Once suspended the Job Execution can break out of its holding loop and proceed to the next target (which will then wait under conditions improve & the session resumes, or its time slot expires and things move to the next target after that).  Unless the user has aborted the operation the script should finish as  'Completed'.

Main issues to be addressed are:

• 1) The second and subsequent Alert Messages are unnecessary and not required.
  >> Obs.Manager code to be modfied to only report the Alert on the first pasage through the loop.
  FIXED
   
• 2) The aborting of the active mosaic target was too slow and could have happened more quickly 
  >> Job Executor code for a mosaic target to be modified to make aborting happen more quickly.
  FIXED
   
• 3) Execution continues to the next target before the Shutter has finished closing
  >> Job Executor should remain in a holding loop until the Shutter has finished closing.
  FIXED
   
• 4) Execution of queue finishes it with it reported as having been aborted by the user, when it hasn't
  >> Job Executor should finish the Job Queue with a 'Completed' Message.
  FIXED

All four problems were fixed in AdvMain 3.10.8 (2019-9-06),  but this new version needs to be tested in a live session in case aby bugs or side issues have been introduced, by the fixes added or by the heavy refactoring of the TakeFrameJobsQSO routine.

Notes

3)  Within Obs.Manager's main loop there is the following section of code

 ' Handle Hard Case Suspension (Aborts)
 ' -----------------------------------
If bHardSuspend Then
    If bHardSuspendOk Or MinutesSince(HardSuspendStartTime) > 2 Or
       ((Synopsis.CriticalRain = ThreatLevel.Critical Or CloudSensorData.Rain > 0.3) And
        MinutesSince(HardSuspendStartTime) > 0.15) Then
            SetState_Suspended()

            bHardSuspendOk = False
            bHardSuspend = False
    End If
End If

This releases Job Executor from its holding loop at the same time as setting state to suspended.   This can clearly allow Job Executor to move on the next target before the Shutter finishes closing.

Proposal is to modify above code to

If bHardSuspend Then
    If MinutesSince(HardSuspendStartTime) > 2 Or
       ((Synopsis.CriticalRain = ThreatLevel.Critical Or CloudSensorData.Rain > 0.3) And
        MinutesSince(HardSuspendStartTime) > 0.15) Then
            SetState_Suspended()
    End If
End If

and then place this new section of code within the "If SessionState = SessionState.Suspended then ... End If " block

If bHardSuspend and bHardSuspendOk and ShutterIsClosed() Then      
       bHardSuspendOk = False
       bHardSuspend = False   ' Release Job Executor from holding loop
End If

 

 

4) The message  (Script aborted by user) is generated in PostJobTasks when AbortScript() = True, i.e. when jobAbort = True

 	If AbortScript() = True Then
		ReportComment(LAlign2("Aborted", 10) + "(Script aborted by user)")
		ClearAbortLock()
		UpdateJobStatus("Aborted")
 	...

jobAbort can be set to true in a number of ways but in the context of automated operations is most likely to have set to true by a call to objconsole.AbortJobExplicit() and probably within SetState_Closing when  RunningFrameJobs is True.

Job Executor puts out a message saying Job Queue has completed message at  04:37:35.95
(this is done very close to end of TakeFrameJobs routine, after this a Event "Queue Completed" is sent to the database, followed by a 1s sleep.
Control then drops back to RunTakeFramesJobQSO which does some post run housekeeping including calling 'StopPHD2Guiding()' at 04:37:39.29. It then calls PostJobTasks (which outputs the 'Script aborted by user' message) and finally RunningFrameJobs is set to False & the thread closes. But the time of this uncertain.   Obs.Manager moves Session State to  'Closing'  at 04:37:42.80

Within Obs.Manager's main loop there is the following section of code

 ' Completed FrameJobs
' -------------------
If (gObservatoryMode = omSemiAutomated Or gObservatoryMode = omFullyAutomated) _
    And (SessionState = SessionState.Running Or SessionState = SessionState.Suspended) _
    And CompletedFrameJobs = True Then

    ObservingDB.UpdatePlanDescription(QsoPlanID, "EXECUTED PLAN " + "S" + CStr(Session.ID))
    SetFlagChanged("Plan", 0) ' Signal to AstroPlan to upload revised ObservingPlan Chart

    If objConsole.cboxDropOutToManual.Checked Then
        objConsole.SwitchToManualMode()
    ElseIf objConsole.cboxContinueWithNewPlan.Checked Then
        objConsole.SetSkipEquilibration()
        SetState_ReEnterInitiating()
    Else
        SetState_Closing()
    End If
End If

 Since CompletedFrameJobs is set to True at end of TakeFrameJobs and before passing control back to RunTakeFramesJobQSO where PostJobTasks is called and jobAbort is tested, it is clear that in some circumstances the SetState_Closing will be called and jobAbort set to true before Job Executors execution reaches the very end of RunTakeFrameJobsQSO where RunningFrameJobs is finally set to false.

So proposed solution is that above code includes an additional test that 'runningframejobs' is False before SetState_Closing can be called

If (gObservatoryMode = omSemiAutomated Or gObservatoryMode = omFullyAutomated) _
    And (SessionState = SessionState.Running Or SessionState = SessionState.Suspended) _
    And CompletedFrameJobs = True and RunningFrameJobs = False Then

Opportunity taken today to some maintainance around the observatory.  

•  Exterior of Dome Observatory cleaned with warm soapy water.
  Using a ladder and the Dome's controls to slew the dome round in 90 deg portions the majority of the roof could be reached and cleaned
  (inspiration and encouragment for this task came from recent Sky At Night article  "How to clean your astronomy observatory" by Steve Richards (no relation), who looks at the cleaning of his Pulsar Dome observatory)
• Northern hedge line and a garden tree were trimmed
• A temporary light shield was installed to block bright light getting to AllSky Camera from a bright security light at nearby building site.

 

Observatory Exterior getting a good cleaning with warm soapy water & sponge		Northern hedge line trimmed so that it doesn't obscure the northern sky from aurora monitoring with 180° AllSky Camera
Workaround installed to block nuisance lighting caused by a security light 'guarding' equipment at nearby building site		Garden Tree trimmed so that it doesn't obscure southern sky from low Declination targets

A housing development around 100m from the observatory is using a very bright light in the area of the contractors containers and equipment. This is supposedly to put of people who might snoop around / steal things etc.  The light stays on to 00:20 in summer varying to around 20:30 in mid winter. In winter the light comes back on at around  05:00.   Apart from adding to overall effect on light pollution, the lights glares in the northern view of the Observatory's 180° AllSky Camera makes it impossible to record any aurora during the time that the light is on and prevents the visual viewing of aurora from my property.

  The following pair of pictures taken earlier this year illustrate the impact of this nusiance lighting.

Northern View with example of Aurora : 2019-01-31, 22:21 (more) Security Light is OFF

Northern View taken 1.5 hours earlier : 2019-01-31, 20:54 (more) Security Light is ON

North Sky Summary Plot for  2019-01-31 Times on plot are local GMT times (= UT time) Whiteouts in the image are due the security light at the nearby building site. Aurora is visible on plot between 21:23 and 23:35

There lights won't be there for ever but they are expected to continue for another 9-12 months or so.  The foreman at the site has been approached and I've tried to engage with him regarding the effect the lights were on me whilst he presented the 'security advice' he's recieved about having the light on . He was unwilling to move/redirect lights since the light is in the ideal position for the 'security' task that is required.  At present I'm try to agree a reduction in the time period that the lights are on, so that that they go off by the time that nautical or astronomical twilight ends (1 to 2 hours after sunset) but he can only relay on my concerns request to those responsible for  'Security' .

In the meantime I've had a go at installing a workaround to block the glare from shinning on the AllSky Camera lens.  This has been done with a piece of wood positioned about 35-40cm away from the camera, in away that doesn't interfere too much with the northern view.   Adjustment was performed at night whilst looking at live shots viewed on IPad. The results is pretty good.

Workaround to block the glare from nuisance lighting			North Sky Summary Plots for 2019-09-02 (top) & 2019-09-04 (bottom) These are plots from the night before and the night after the  light baffle was installed

 

Northern View  : 2019-09-03, 22:27 Security Light is ON Glare completely obscures the view of the northern horizon

Northern View : 2019-01-31, 22:36 Security Light is ON. Initial positioning of the wooden post blocks the light glare but obscures too much sky

Northern View : 2019-09-03, 22:51 Security Light is ON. Repositioned wooden post blocks the light glare but doesn't obscure too much sky

Northern View : 2019-09-04, 00:18 Security Light is ON. Repositioned wooden post blocks the light glare

The above view of the sky taken with light on, but blocked with wooden post compare pretty well with the following image taken one minute later and after the security light had gone off.

Northern View : 2019-09-04, 00:19 Security Light is OFF Wooden baffle doesn't obscure too much sky.