David's Astronomy Pages Notes - Session 683 (2019-08-22)

Notes (S682)

Notes Main

Home Page

Notes (S684)

	Session Aims & Highlights
	- Observing Result  - Night Summary Plot  - Session Event Log
	Operational Issues
	- Critical Issues (0),  Major Issues (0),  Minor Issues (10),  Continuous Improvement (4)
	Investigation -  Session Suspension Issues
	Images from 2019-08-22  >>

Session Aims & Highlights (2019-08-22)

Main aims

1. Mixed Targets - Image selection of targets including Variable Stars, Comets and Deep Sky Targets
2. Confidence in Automated Operations - Check for correct and safe Automated Operations through an entire night of variable cloud conditions.. Confirm correct Observatory Shutdown from Suspended Mode
3. Software & Systems  - Check stability of software and presence of any side effects following ongoing enhancements to AstroMain and associated support programs.

 
Equipment & Software

• 12" LX200 and ST-10XME
•  80mm APO and ZWO ASI 178MC
•  CCDApp2, TheSky6, CCDSoft5, PHD2,  POTH Hub
•  Pegasus UPB & Motor Focus, TCF-S
   
•  AstroMain 3.10.2,  Astroguard 1.2.3,   AstroPlan 1.9.2, AstroVOE 1.3.1   (AstroWeather 1.7)

Highlights

• Automated Plan Building started at new Sun Alt trigger height of -11°
• Observatory Manager performed flawlessly through entire night, closing and opening shutter at appropriate moments relative to the prevailing cloud conditions
• Observatory Manager successfully shutdown the Observatory (checked shutter, parked dome, parked scope) from Suspended Mode state

Lowlights

• Cloud rolled in earlier than forecast.

Summary Plots & Logs

Observing Plan

Observing Result

Night Sky Summary Plot Top axis: Sky Brightness at Zenith (in ADU/s) Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)

Pre-Session Weather Forecast (AstroWeather 1.7.1) (cloud forecast from ~ 01:00 onwards with 43% initial cover building to 100% by 04:00 However in the event cloud arrived at 23:00 with 90% cover building to 100% cover at 01:30, before reducing to 30% cover from 02:30 onwards, but still with poor clarity)

Session Event Log

Time   Event Detail 21:35 Session Created Live Session Created (2019-08-22 S00683, ImageSaveNum: 683001) 21:37 Obs.Manager Started Obs.Manager started 21:37 Services Started Observatory Services started 21:38 Observatory (Auto) Observatory placed in Fully-Automated Mode 21:38 Session Pending Session pending (2019-08-22) 21:39 Session Initiating Session initiating (2019-08-22) 21:39   Plan Requested Observing Plan requested from AstroPlan () 21:42   Plan Loaded Observing Plan loaded to queue (2019-08-22 V1 BEST PLAN, Plan ID: 321) 21:43   Camera1 Connected SBIG Camera Connected (set point -10°C) 21:43   Telescope Connected Telescope Connected (TheSky6) 21:54 Shutter Opened Shutter opened (opening time 45s) 21:54 Equilibration Started Dome Equilibration started 22:01 Equilibration Ended Dome Equilibration ended (7.3 mins total) 22:01 Session Running Session running 22:01   Queue Started Observing Queue started (28 targets selected) 22:01     Target Started (1/28) Target started (1/28, Kn 61) 22:04   Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 22:06   Focusing Completed Foc1 AutoFocus Completed (Profile No 1) 22:22     Target Completed Target completed (1/28, Kn 61) 22:22     Target Started (2/28) Target started (2/28, GCVS FG Sge) 22:23   Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 22:25   Focusing Completed Foc1 AutoFocus Completed (Profile No 2) 22:28     Target Completed Target completed (2/28, GCVS FG Sge) 22:28     Target Started (3/28) Target started (3/28, Pal 13) 22:30   Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 22:32   Focusing Completed Foc1 AutoFocus Completed (Profile No 3) 22:39     Target Completed Target completed (3/28, Pal 13) 22:39     Target Started (4/28) Target started (4/28, NGC 6756) 22:42   Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 22:47     Target Completed Target completed (4/28, NGC 6756) 22:48     Target Started (5/28) Target started (5/28, Abell 55) 22:51   Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 22:52 Critical Cloud Alert Critical Cloud Alert (Obs.Manager will close the Shutter 22:52     Target Aborted (5/28) Target aborted (5/28, Abell 55) due to Unknown Reason 22:52 Session Suspended Session suspended 22:52 Shutter Closed Shutter closed (closing time 48s) 02:31 Session Resuming Session resuming 02:32 Shutter Opened Shutter opened (opening time 45s) 02:32 Queue Resumed Observing Queue resumed 02:35 Critical Cloud Alert Critical Cloud Alert (Obs.Manager will close the Shutter 02:35 Session Suspended Session suspended 02:36 Shutter Closed Shutter closed (closing time 48s) 02:51 Session Resuming Session resuming 02:51 Shutter Opened Shutter opened (opening time 45s) 02:51 Queue Resumed Observing Queue resumed 02:51     Target Started (22/28) Target started (22/28, NGC 6885) 02:54   Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 02:56   Focusing Completed Foc1 AutoFocus Completed (Profile No 4) 03:01     Target Completed Target completed (22/28, NGC 6885) 03:01     Target Started (23/28) Target started (23/28, GCVS EV Lac) 03:01 Session Suspended Session suspended 03:02 Shutter Closed Shutter closed (closing time 48s) 03:07   Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 03:09     Target Completed Target completed (23/28, GCVS EV Lac) 03:13 Session Resuming Session resuming 03:14 Shutter Opened Shutter opened (opening time 45s) 03:14 Queue Resumed Observing Queue resumed 03:16     Target Started (25/28) Target started (25/28, GCVS U And) 03:18   Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 03:20   Focusing Completed Foc1 AutoFocus Completed (Profile No 5) 03:26 Critical Cloud Alert Critical Cloud Alert (Obs.Manager will close the Shutter 03:26 Critical Cloud Alert Critical Cloud Alert (Obs.Manager will close the Shutter 03:26     Target Aborted (25/28) Target aborted (25/28, GCVS U And) due to Unknown Reason 03:26 Session Suspended Session suspended 03:26 Critical Cloud Alert Critical Cloud Alert (Obs.Manager will close the Shutter 03:27 Shutter Closed Shutter closed (closing time 48s) 03:56 Queue Completed Job Queue completed 03:56 Session Closing Session closing 03:57 Dome Parked Dome parked (parking time 33s) 03:58 Telescope Parked Telescope parked (parking time 42s) 03:59 Session Finished Session Finished

Back to Top

Operational Issues (2019-08-22, S683)

• Critical Issues (0)
  
• Major Issues (0)
• Minor Issues (10)
• Continuous Improvement (4)

[ Prev | Next ]

Critical Issues

• None

Major Issues

• None

Minor Issues

• User had to manually chick refresh button in order to see latest session listed in AstroPlan (Laptop10) even though code had been updated earlier in the day.
  Investigating showed that Laptop10 was still running AstroPlan 1.9.1, rather than the new AstroPlan 1.9.2
    Resolved 2019-08-23.  (AstroPlan 1.9.2 executable installed on Laptop10)
   
• Older Sessions show invalid target information for  ImageSets (AstroMain/Analysis), e.g session 2018-12-28
  Information fields show target data from the last Image Set having Target reference information.  This is probably due to images from older sessions not having Target UID numbers written to their FITs headers.  Target fields need to be set to blank where FITs files in the ImageSet don't have a Target UID or where the UID can not be found in the database.
    Fixed 2019-08-24 (AstroMain 3.10.3)
   
• Missing closing bracket in Event description 'Critical Cloud Alert (Obs.Manager will close the Shutter' . Event line is missing a closing bracket.
    Fixed 2019-08-25 (AstroMain 3.10.3)
   
• Target Reporting uses an unnessary extra blank line between 'Planned End' and 'Details'
    Fixed 2019-08-24 (AstroMain 3.10.3)   >> Check in next session
   
• Confusing 'Planned End' time for final target.  Planned End for WDS GRB 34 (28/28) reported as 04:14, yet 03:56 (start time) + 7.4 mins (duration) is 04:03
    Planned Start..             Ok        2019-08-23 03:56  (Scheduled)
    Planned End..               Ok        2019-08-23 04:14  (Scheduled) Duration : 7.4 mins
   
    Issue closed (2019-08-25) as issue is now understood (10 minutes are automatically added to the usual End Time for the final target).
  
  - an earlier session (S677) had a final target RX And (21/21) under Session Suspended state and it also had an anomalous Planned End time.
   (Plan S677 had a End Plan date of 03:05). This could be a feature of final targets ?
    Planned Start..             Ok        2019-05-10 02:36  (Scheduled)
    Planned End..               Ok        2019-05-10 02:53  (Scheduled) Duration : 7.5 mins
  
   - the Planned End.. code outputs the variable "T.EndTimeSlot".
   - when the plan is loaded  the final target's EndTimeSlot is set slightly differently to preceeding targets.
      .EndTimeSlot= .StartTime.AddSeconds(.Duration).AddMinutes(10)
   - the otherwise confusing Planned End times are consistent with adding 10 minutes to normal End Time
   - it is believed there was some good reason why the 10 min addition was deemed appropriate/necessary.
   
• After a Critical Cloud Alert code the report fIle showed FailureState.0    (Blank / Undefined), rather than FailureState.11 (Cloud)
    Reason To Suspend             Info      FailureState.0
  Investigation showed this was due to an incorrect threshold for Cloud  in UpdateReasonToSuspend()' routine
    Fixed 2019-08-25 (AstroMain 3.10.3). Code corrected from Clarity threshold from 20 to 24
   
• Event line "Target aborted (25/28, GCVS U And) due to Unknown Reason" should be more specific.  
    Fixed 2019-08-26 (AstroMain 3.10.4). Obs.Manager code modified to include the line "ReasonToAbort = AbortReason.CriticalConditions"
         This produces the message "Target aborted (...) due to Critical Conditions"
  
  - U And had been shooting images when the critical cloud conditions alert arrived (two alerts at 03:26:45.46 and 03:26:50.49)
  - Job Executor produced log message "Target (GCVS U And) has been aborted due to Unknown Reason" at 03:26:50.81
  - Report file shows message "Target aborting due to Unknown Reason"
  - Event tables shows an event with description "Target aborted (25/28, GCVS U And) due to Unknown Reason"
  
  - Report file message gets it's reason from  GetAbortReason()
     [code :   ReportComment2B(Tab2 + "Target Aborting ...", "OK", "Target aborting due to " + GetAbortReason()) ]
  - GetAbortReason gets it abort reason from global integer variable called "ReasonToAbort"
    the "Unknown reason" message is output when ReasonToAbort = AbortReason.Undefined  (= 0)
   
• Three Critical Cloud Alert events sent to Database within 11s Check if there were actually 3 events around 03:26 (hence 3 event lines) or there was just a single event (which should have just one event line). 
    >> Believed Fixed 2019-09-06 (AstroMain 3.10.8) 
   
  Log file shows threes Critical Cloud Conditions.Alert lines at 03:26:45.46 and 03:26:50.49 t3 messages separated by 5 sec, i.e. the Obs.Manager cycle time)
  Report file shows 3 Critical Cloud Conditions Alert sections at 03:26, 03:26 & 03:27
  Event tables shows two Critical Cloud Alerts at 03:26, 03:26 & 03:27
  
  Alert triggered U And to abort.
  Job executor moved to next target SS Cyg, before Suspension happened at 03:26:57.70
  Further Crtical Cloud Condition Alert at 03:26:57.75
   
• Anomalous Script Aborting Message after reaching end of Job Queue.   At 03:56 the job executor output the message "Script Aborting ... Aborted (Script aborted by user)" after reaching the final target in the plan (WDS GRB 34) and it failing due to Observatory being suspended due to Cloud.  The job should finish with a "Job Completed" message. Even if there was a reason for aborting the 'script' it should state 'Script aborted by Observatory Manager'  not by the user.
  The script 'aborted' at 03:56, yet planned end time for final target was 04:14 (based on report file) or 04:03 (based on Plan Duration). The plan had an end time of 04:19.
  The log file records a normal 'Job Queue has completed' line which adds to the confusion.
    >> Believed Fixed 2019-09-06 (AstroMain 3.10.8) 
   
• Final target in Observing Plan stops at around Sun Alt -12.5° or -13° rather than -11°. Need to understand why the final target doesn't end with Sun Alt closer to -11°. May need to check that Schedule Builder wasn't 'Target Limited'.  May need to look at adding code that fills the Best Plan with targets out to the Sun Alt limit (-11) , or add a tolerance to the time slot for the final target, in order to stop this problem. Testing confirms that plans aren't arbitarily ending at/before -12°.
    Fixed 2019-09-25 (AstroPlan 1.11.1) 
   
• NGC 5602 report message "Session is suspended due to non-defined reason" isn't sufficiently informative.
  Database record failure mode as 'Cloud'.  Message for next target ok (Session is suspended due to Cloud)
    Believed Fixed in Aug/Sep 2019

Continuous Improvement

• Targets that are passed over due to session suspension should be recorded as 'Targets Cancelled' Events which are saved to the database.
    Done 2019-08-25 (AstroMain 3.10.3)
   
• Tweaks to new Weather Forecast graph, including adding Day/Night bar, Sunset & Sunrise times, Pressure, and cosmetic tweaks
    Done 2019-08-24 (AstroWeather 1.7.2)
   
• Tweaks to Graphs display OTA Temperature Difference from DewPoint, including modifying vwsGraph37 (with differences relative to Outside DewPoint)
  and adding new graph  vwsGraph49 (with differences relative to UPB Env DewPoint)
    Done 2019-08-24 (AstroWeather 1.7.2)
   
• Add Laptop7 Ping Flag to AstroMain's Backup tab
  Done 2019-08-27 (AstroMove 1.3)

[ Prev | Next ]

Back to Top

Investigation - Session Suspension Issues

Back to Top