David's Astronomy Pages
Notes - Session 613  (2018-06-11)

   
Bullet Session Aims & Highlights
- Night Summary Plot
Bullet Operational Issues
  - Critical Issues (1),  Major Issues (2),  Minor Issues (2),  Small Defects (0),  Continuous Improvement (0)
   
Bullet Observing Plan
Bullet Investigation - Telescope/TheSky6 - Error, poor communication, link automatically teminated (Error code=213)
   
Bullet Images from 2018-06-11  >>
   
2018-06-11 to 2018-07-16 (AllSky Laptop)
Bullet AllSky Laptop - Failure & Replacement
Bullet AllSky Laptop - Windows 10 Installation
Bullet AllSky Laptop - CCDSoft, TheSky & CCDApp2_AllSky Installation
Bullet AllSky Laptop - Eurotech Cloud Sensor & PL2303 USB-Serial Adapter
Bullet AllSky Laptop - Weather Server & CCDApp3_AllSky
Bullet AllSky Laptop - Ethernet Connection
 

Session Aims & Highlights (2018-06-11)

Main Aims

Main aims of this nighttime session was to continue commissioning of Clair3 Observatory including

  1. Automated Operations. Test automated acquisition of images of a sequence of targets (Plan) that have been scheduled by a newly developed genetic algorithum optimisation program working with target data held in SQLite.
  2. Imaging. Acquire some images
  3. System Stability. Continue checking stability of POTH.Hub (Telescope + Dome) & Observatory Control Software following major issues during the last nighttime session on 2018-06-06.

Equipment & Software

Highlights

Notes

Summary Plots & Logs

Night Sky Summary Plot - 2018-06-11
Top axis: Sky Brightness at Zenith (in ADU/s)
Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)
 
Not available
 


Back to Top


Operational Issues (2018-06-11, S613)

[ Prev | Next ]

Critical Issues

Major Issues

Minor Issues

Small Defects

Continuous Improvement

[ Prev | Next ]

Back to Top


Observing Plan  (2018-06-11)

One of aims of this session was to test the automated acquisition of a sequence of targets (Observing Plan) that have been scheduled by a newly developed genetic algorithum optimisation program working with target data held in SQLite.  The test was moderately  successful after allowing for the delay in the start time caused by initial instability of communication between TheSky6 and the LX200GPS Scope (using Telescope API & POTH. Hub) .  Pictures of the executed observing plan are shown below.

- 7 targets (U Her to NGC 5023) were missed due to the delayed execution start.

- 5 targets were acquired as schedulebut there were issues with accuracy and time taken to peform high precision centering
 (scope jogs were noticed to be not particularly effective in nudging the scope onto desired target coordinates, resulting in extra locate images/ extra plate solution overheads (pointing error would typically reduce to ~1.8 arc mins after the first 'Image/Plate Solve/Jog' cycle. would find it difficult to make jogs that would bring the error down to < 1.0. ( Is there a telescope balancing issue?))

- 2 targets (Hickson 71 & Vega) were missed as delays in centering the preceeding target caused them to overrun their 'timeslot' and result in the two targets missing their scheduled start times.

- 1 target (IC 4997) was manually aborted as the frames for this final target were becoming saturated by the brightening sky.
  (Normally this target would be acquired during longer nights with greater solar depression, but solar depression limits were relaxed
  to enable testing during June (summer))

 

Observing Plan executed during session 2018-06-11 (Chart)
 
Image
 
Observing Plan being executed (screenshot)
Image
 

Back to Top


Investigation - TheSky6/Telescope Error, poor communication, link automatically teminated (Error code=213) 
(Part 1, 2018-06-11)

In the past two sessions (2018-06-06 and 2018-06-11) observatory operations have been affected by a suite of problems which appears to all originate from a primary issue whereby communications with the LX200GPS are intermittently lost with the TheSky6 error message "Error, poor communication, link automatically teminated. Error code=213(0xd5)" and the LST field in the POTH.Hub window changes from showing the LST time to showing  "--:--:--".   See screenshot below

Automatic Disconnection of Telescope upon poor communications
Error Message 213 in TheSky6
and LST "--:--:--" in POTH.Hub
Image

Some Internet Group Message/Pages discussing  TheSky's Error Code 213 are :

Communication problem with Sky6 (Software Bisque Support Forum)
- The error message means that TheSky6 is not receiving a (correct) reponse from the control system after repeated tries.
- Check the computer to control system cabling (and connections) to make sure they're in tact.
-
You might try this: Reset TheSky6's "Number of Retries before failure count" in the Registry and see if that makes a difference.
- Changing registry setting to 80 and the connection error does not show but the link is lost without the computer losing indication of loss. The telescope still tracks but you cannot go to a new object. When trying to slew to a new object get message "No Response from device: Error code 203 (0xcb)"
- If you are actually getting 80 errors, something is very wrong, likely with hardware or connection (noisy serial line).
Needs to do a thorough isolation of every hardware component connecting computer and telescope.

TheSky automatically disconnects the telescope (Software Bisque Knowledge Base Article)
- Upon poor communications TheSky automatically disconnects the telescope.

- This is by design. To make TheSky not automatically disconnect the telescope upon poor communications do the following:
1. Go to Start, Run and type "Regedit" and press OK.
2. Navigate to the HKEY_CURRENT_USER\Software\SoftwareBisque\TheSky\TELESCOPE.
3. Select, Edit, New, DWORD value, for the Name type "AutoTerminateErrCnt" and for the value put in something larger than 3 which is the default.

My new MX+ isnt working (Software Bisque Support Forum)
- What you describe means TheSkyX cannot reliably communicate with the mount.

USB Device Communication Failures (Software Bisque Knowledge Base Article)
- Microsoft Windows power management plan can suspend power to one or more USB devices connected to the computer (called USB Selective Suspend). Once power to the USB port is suspended, TheSky will report communication errors, for any mount, camera, hub connected to this USB port.

Back to Top


TheSky6/Telescope - Error, poor communication, link automatically teminated. Error code=213(0xd5) 
(Part 2, 2018-06-13)

Thinking back to what has changed between 2018-05-17 / 2018-05-22 (the last nighttime and last daytime sessions with stable POTH/Telescope Comms) and 2018-06-06 (when Error Code=213 problems were noticed) it is possible that cabling routing modifications on 2018-05-23 might have pinched the RS-232 cable to the LX200 Mount or have brought the cable into very close proximity with certain other cables creating noisey communications, and causing TheSky6 to intermittently disconnect the telescope.

Way forward:
- Create Registry DWORD for AutoTerminateErrCnt and set to a number larger than the default value of 3
- Double-check USB cable, USB Serial Adapter and RS-232 serial cable joining computer and telescope
- Loosen the cable protector that was introduced in the observatory on 2018-05-23
- Carry out daytime stress testing of the Observatory System (Hardware & Software)

HKEY_CURRENT_USER\Software\SoftwareBisque\TheSky\TELESCOPE  : AutoTerminateErrCnt
DWORD AutoTerminateErrCnt added and set to value of 8
Image
 
RS-232 Cable Run
USB Hub, USB Cable, USB-Serial Connector
 and RS-232 Serial Cable & Connection to
Scope checked
  Cable Protector/Guide  
Recently added Cable Protector loosened
to relieve pressure on RS-232 Cable
Image   Image
 

Back to Top


2018-06-11 to 2018-07-16 (AllSky Laptop)


AllSky Laptop - Failure & Replacement

After providing around 4 years of nearly 24x7 service my already old AllSky laptop (it was previously my eldest son's "first laptop") stopped working on 2018-06-11 due to a hard disk failure (C Drive). Data had been achived to an external hard drive (G Drive) up to and including Night Date 2018-06-10. A new internal hard disk was considered, but eliminated on the basis that my AllSky plus recently introduced Weather Monitoring requirements are close to the top end of what its CPU can handle - around 10 of more images can't be processed in the time gap between frames, a situation which will worsen when nights begin to lengthen towards winter. 

It was decided to put into service another old laptop available in the household. This was my eldest son's "second laptop" - an Acer Inspire (CPU i5-2430M 2.4 GHz). Advantages over previous laptop - faster processor (64 bit vs 32 bit), larger RAM ( 8 GB vs 3 GB),  larger hard disk (500 GB vs 100 GB). Athough it has a failing keyboard this issue is easily remedied by using a USB keyboard.    

It was decided to do a fresh operating install using Windows 10 Pro instead of using its current Windows 7 Home as part of a longterm plan to gradually  move all household computers to Windows 10.  An HP Desktop Computer used for remote monitoring/operation of the Observatory was previously upgraded to Windows 10 (Nov 2017 notes) and a fresh Windows 10 Install was placed on my youngest son's desktop computer, but this was the first time to use Windows 10 for operating some of the observatory equipment (Oculus AllSky Camera, Weather Station USB Link and Aurora Cloud Sensor) with the potential risk of issues relating to compatability of equipment & drivers.  A previous attempt to upgrade my observatory laptop to Windows 10 in 2016 was quickly undone and returned to Windows 7 after it it failed to access my SBIG CCD Camera using CCDSoft5 complaining of multiple missing drivers, plus discomfort with the new operating system interface at that time.

Althought not checked first, the Acer laptop was assumed to have a Video Card that that would be compatible with Windows 10.   (my older HP desktop computer had required a new video card to work with Windows 10 at the full resolution of its monitor)

Installating CCDSoft5 and TheSky6 and bringing them up to the latest version is regularly prone to issues with getting their ObjectModels fully installed to the Windows Registry and available to external programs like CCDApp2 and CCDApp2-AllSky, and several installatiion attempts are usually needed.   These installations would need to be regardless of computer / operating system, but there was an additional risk of issues getting ObjectModels installed/working with Windows 10.   

Back to Top


AllSky Laptop - Windows 10 Installation

Preparation
- Windows 10 Software (64 bit) was put onto a bootable USB stick using Windows Media Creation Tool
- Windows 10 Pro Licence Key was bought online from SamSoft (from Amazon Marketplace) for £9.99
 (Windows 10 licence keys have been bought using this method on two previous occasions and have installed and worked without problems)
- CD's with critical Software (CCDSoft6, TheSky6, PaintShopPro5) were prepared , as was other software which available as electronic installation files  from my main observatory laptop (TextPad4, Drivers, Virtual Weather Station, UltraVNC)
- Key/useful files and folders were taken from copies on my main observatory laptop
  (files not available from old AllSky laptop due to its failed hard drive)

Windows 10 Installation
- Installation performed as a fresh OS install
  (the bootable USB stick with Windows 10 software was placed in a USB slot and the computer turned on.
  The computer bios was accessed (F2 key) and temporarily updated to set the USB drive as the top priority device to boot from.
 
- Installation went reasonably smoothly,  Windows 10 activated ok with the purchased licence key.
 
  The sole issue during the Windows 10 installation was that after leaving the laptop unattended whilst it finished off the installation I came back to find it had done an automatic restart at the end of the process and reaccessed the bootable USB with the prompt to start a Windows 10  installation, unaware that it was already just installed. The trick was to remove the USB stick and restart the laptop so that it could pick up the operating system installed on the hard drive.  The software activated as Windows 10 Pro as anticipated

Software

Key software was then installed:
- Kaspersky Internet Security (antivirus/firewall software)
- CCDSoft5 (for accessing Oculus AllSky Camera)
- CCDApp2_AllSky (for operating AllSky Imaging system including Weather Scorecard)
- TheSky6 (for supporting the AllSky Imaging system)
- Virtual Weather Station (for accessing weather data from Oregon Weather Station)
- Weather Server (for fowarding current weather data to CCDApp2_AllSky and to the Main Observatory computer)
- Aurora Cloud Sensor 3 (for accessing data on Eurotech Cloud Detector)
- Textpad 4 (for text file reading & editing )
- PaintShopPro 5 (for image capture and editing)
- Ultravnc (for remote monitoring and control of AllSky Laptop)
- ACDSee32 (for viewing jpg and gif images)
- Visual Studio 2017 with VB.Net, Community Edition (for debugging CCDApp3_AllSky)

Problems were encountered with the installation of CCDSoft5, TheSky6 and CCDApp2_AllSky (see notes below).
Problems were also encountered with using existing USB-Serial Adapter (PL2303 Prolific) for the Cloud Sensor. (see notes below)

A week later I updated Windows 10 with 'April 2018' (1803) update.   This installed ok, but it temporarily broke the CCDSoft5 Installation with multiple message complaining of missing DLL files, till those DLL's files were manually replaced. (see notes below)

Back to Top


AllSky Laptop - CCDSoft , TheSky & CCDApp2_AllSky Installation

After installing CCDSoft5 and TheSky6 applications and bringing them up to the latest versions,  the CCDApp2_AllSky program for controlling the AllSky/Weather Services was started up but immediately crashed before bringing up the first window and without any error messages.  It turned out that this was due a failure of CCDSoft5 and TheSky6 to correctly register their Object Model in the Windows Registry with result that Camera and TheSky objects couldn't be created within 'CCDApp2_AllSky' program and a subsequent uncaught exception brought the program down when attempting to use one of the expected objects.

CCDSoft5 and TheSky6 were both uninstalled and then reinstalled and again brought up to their latest versions ('Regedit' was useful for showing whether objects such as CCDSoft.Camera, CCDSoft.Folder, TheSky.RASCOMTheSky had been successfully installed in the registry). The trick appears to ensure that the two applications are both initially 'run as Administrator' to ensure that object references are created in the registry.

To understand why 'CCDApp2_AllSky' was crashing, it was necessary to run the program in debugging mode.   The program is written in VS 2010 on my main Observatory Laptop, and VS2010 is now longer available to download from MicroSoft to allow debugging. Rather than a long trial and error method of finding the source line causing the crash, by outputting 'got to here' lines to the logfile the program was ported to VS2017 as CCDApp3_AllSky.  This was achieved with just a few relatively minor issues. One legacy form was producing a error related to duplicate entries in resources and was disabled and Visual Basic Powerpacks had to downloaded from Microsoft website and included as a  Visual Studio 2017 reference.   Finally Visual Studio 2017 was installed on the AllSky laptop and used to identify the cause of the crash (discussed above).

Whilst Third Party Camera Plugin "Starlight Xpress SXV Plug In.dll" was copied to "C:\Program Files (x86)\Software Bisque\CCDSoft Version 5\Camera Plug Ins"  the Oculus AllSky Camera couldn't be successfully connected until latest SX Drivers ("Signed drivers for 32 and 64 bit Windows 7, 8 and 10 (version 1.2.2.0)" had been downloaded and installed from the Starlight Xpress website (https://www.sxccd.com/drivers-downloads)

Later on CCDSoft5 stopped working after installing the 'April 2018 update' to Windows 10. Multiple error messages apeared whilst attempting to start the program complaining that certain .DLL files couldn't be found
  (APOGEE32.dll, CFWAPI.dll, DSS_DLL.dll, ipl.dll, TiffPli.dll,  RelayAPI.dll,  TTY32.dll, ASTROM.dll,
  (EIGENVAL2.dll, SE.dll, PVAPI.DLL),
and an error message complaining that "No IPLib DLL was found in the Waterfall procedure"  (.iplPX.dll  & IP.dll)
Files are missing from 'C:\Program Files (x86)\Common Files\System'
Copying DLL files from Common\System folder on Main Observatory Computer into Common\System folder on AllSky Computer fixed the problem.
(A similar problem with CCDSoft & DLL files was noted a couple of years ago when an attempt to update the Window 7 operating system on my main Observatory laptop to Windows 10)

Back to Top


AllSky Laptop - Eurotech Cloud Sensor & PL2303 USB-Serial Adapter

Problems were encountered with getting the Eurotech Cloud Sensor online.

Initial connection of Prolific USB-Serial Adapter (PL2303 chip) produced yellow triangular warning flag in Device Manager and a 'The device cannot start (Code 10)' error in the status window.  Internet search showed that this was a common problem with Proflic PL2303 devices when used with Windows 10 with latest 64 bit drivers.

Problem is essentially that a lot of USB-Serial adapter contain a counterfeit PL2303 chip and Prolific modified their newest drivers to render the counterfeit adapters unusable. Unfortunately this decision also renders all earlier adapters with authentic chips inoperative as well.    The Prolific 64-bit drivers that Microsoft supplies via Windows Update, Versions 3.4.67.325, 3.4.25.218, 2.1.51.238 and 3.4.62.293 - will not work with most adapters and therefore shows the generic "Code 10" error.

There are lots of proposed fixes out there which claim to fix the problem but don't necessarily do so.  The generally agreed method is to use an older 64 bit driver such as 3.3.2.10 (09/29/2008),  instead of more recent 2015 or 2017 drivers. The trick is ensuring that Windows doesn't use the newest non-working driver when it next re-boots or updates.  

After trying various things I eventually got a 'This device is working properly' message in Device Manager without the yellow triangle ,  and I was able to then connect to COM port with the Cloud Sensor 3 software,  but it (and the earlier Cloud Sensor 2 version) would only display values of 0 for Sky Temperature, Clarity, Light & Rain (instead of expected non-zero real values), indicating that something was still wrong.

Finally I elected to purchase a new USB to Serial Adapter that uses the Prolific chipset version that is compatible with Windows 10
Plugable USB to RS-232 DB9 Serial Adapter (Prolific PL2303HX Rev D Chipset) from Amazon UK
New adapter was plugged in and began working immediately (using Prolific Driver 3.8.18.0,  dated 2017-10-17). 
Cloud Sensor3 began using real (non-zero) values for Sky Temperature, Clarity, Light & Rain.

 

Yellow Triangular Flag against Prolific UBB to Serial Comm Port (Device Manager)
 
Image
 
Device Cannot Start (Code 10) Error (Device Properties)
 
Image
 
Prolific USB to Serial Device Driver (latest 3.8 driver, 64 bit)
 
Image
 
Selecting older 3.3 driver to use instead
 
Image
 
Prolific USB to Serial Device Driver (older 3.3 driver)
 
Image
Prolific UBS to Serial Comm Port appears to now be working
(Earlier Yellow Triangular Flag is no longer present)
 
Image
 
Aurora Cloud Sensor Connected to the Comm Port
but only displaying zero values, ie not real data.
 
Image
 
Running CheckChipVersion on Main Observatory Laptop (Windows 7)
reveals that the USB to Serial Adapter uses a PL-2303 XA / HXA chip.
(this is the older chip and is not directly compatible with Windows 10)
 
Image
 
New USB to Serial Adapter Purchased
(contains Prolific PL2303HX Rev D Chipset, which works with Windows 10)
 
Image
 
Prolific UBS to Serial Comm Port working after inserting new Adapter
(contains the Prolific PL2303HX Rev D Chipset)
 
Image
 
Prolific USB to Serial Device Driver (latest 3.8 driver, 64 bit)
 
Image
 
Aurora Cloud Sensor Connected to the Comm Port
and finallu displaying real (non-zero) values
 
Image
 

Back to Top


AllSky Laptop - Weather Server & CCDApp3_AllSky

Several teething issues associated with getting AllSky Laptop operations fully working. In particular the service provided by WeatherServer & CCDApp3_AllSky programs.

Back to Top


AllSky Laptop - Ethernet Connection

A reliable Ethernet Connection is extremely important/critical for the operation of the Observatory and especially for its planned future operation in 'fully-automated mode'. Specific tasks that the AllSky Laptop needs to perform using the ethernet connection are :
 - automated upload of AllSky Images & Graphs to the 'astro-richweb' website,
 - automated download of Aurora related information
 - automated download of digital Weather Forecast
 - automated sending of Weather, AllSky and Scorecard Synopsis information to Main Observatory Computer
  (critical to safe operation of the Dome)
- automated receipt of current telescope position and dome shutter status
- remote access to the AllSky Laptop using UltraVNC for monitoring and remote operation control
- remote access to various files and data on AllSky Laptop

During the first 2 weeks of operation using the 'new' AllSky Laptop, it was found that it's ethernet connection would drop out every 2 or 3 days or so.   The connection would go down and stay down until noticed (usually several hours later) and then fixed by physically visiting the observatory and running Window Network TroubleShooter on the Laptop.  Windows Network Diagnostics would identifiy the problem as 'the default gateway is not available' and the Troubleshooter would fix it by Resetting the 'Ethernet' Adapter. Other computers in the house were not afflicted by the problem.  Without ethernet connection, it not possible to make a VNC (remote) connection in order to start up Network Troubleshooter.

Without any real clue to why the ethernet connection is dropping out, a proposed workaround is to have a routine running on the AllSky Laptop (probably a Thread within the "CCDApp3_AllSky" program that will
a) detect that the Ethernet Connection has dropped out (and stayed down for at least 10 minutes or so)
b) reset the Ethernet Adapter (by disabling and then re-enabling the adapter)

It is proposed to reset the adapter by calling a BAT script :
    netsh interface set interface "Ethernet" DISABLED
    netsh interface set interface "Ethernet" ENABLED

The first opportunity to test this script for real came on morning of 2018-07-31 when it was noticed that communications with the AllSky Laptop had stopped.   Visiting the observatory and running the script duly brought the Ethernet Adapter online and allowed restored commications between the Laptop and other computers on the LAN and with the Internet.

With this workaround proven, the next step was to embed checking of the Network Connection and the calling of the Reset Ethernet Script (upon Network Connection failur)  into my Weather Server software that operates on the Laptop 24x7.  This coding upgrade was made on 2018-08-01, and should allow the Laptop to henceforth automatically fix Network Connection Failures associated with the Ethernet Adpater within 5 minutes of a failure occuring. This interval will be adjusted if necessary.


No Internet or LAN Access (screenshot)
 
Image
 
Windows Network Troubleshooter (screenshot)
 
Image
 
Windows Network Diagnositics
Troubleshooting report - Detailed Information
'the default gateway is not available'
 
Image
 
Windows Network Interfaces
 
Image

Back to Top