Datalab, the La Silla Paranal Observatory Platform for data analysis, is being migrated from Docker Swarm to Kubernetes to align with the integrated operations program's goals: Remote, Lean, Sustainable, and High performance. The migration implied to move from an on-premises to a cloud-native infrastructure replicated locally into a Cloud-Edge, providing hybrid cloud containerized applications support, implementing DevOps practices and automation. Using infrastructure as code and configuration management tools like Terraform and Ansible. Building CI/CD pipelines in Gitlab to automatically deploy the proper infrastructure into to the hybrid cloud to hold Kubernetes clusters (Azure Kubernetes Cluster and Vanilla Kubernetes). This approach allows the Observatory to enhance efficiency, reducing power consumption and improving scalability. Using Datalab as proof of concept but setting up the foundation to standardize these technologies in the organization. This paper outlines the provisioning and deployment of the new hybrid cloud infrastructure, providing a concise overview of its architecture, operational impact, and benefits for the observatory.
KEYWORDS: Data analysis, Observatories, Visualization, Error analysis, Data modeling, Telescopes, Education and training, Machine learning, Analytical research
The VLT at Paranal Observatory has been in operation for over two decades, and soon, the ELT will be managed by the same operational team. Maintaining operational efficiency and minimizing downtime with limited resources will be crucial. Previous research has shown that software logs effectively capture the telescopes' behavior, providing valuable operational insights. We've integrated various log analysis techniques from academic literature and industry best practices. These techniques allow engineers to monitor system health, analyze error sequences, detect anomalies, and reconstruct processes which improve maintenance and extract new insights. Additionally, we've utilized generative artificial intelligence and NLP transformer-based models, to infer observation behavior and predict execution failures. We have taken advantage of both the Paranal Datalab on-premises facility and Azure Cloud. In this work, we provide technical details and outline the key challenges and opportunities in adopting this technique within an astronomy facility.
The Very Large Telescope Interferometer (VLTI) must control its Optical Path Differences (OPD) to extremely high precision in order to achieve its characteristic and desired high performance. This proves a challenge when using Very Large Telescope’s (VLT) 8 meter Unit Telescopes (UT) given they are not fully dedicated to interferometry and can be equipped with up to three different instruments each. Among the several important control systems that allow the VLTI to achieve the necessary precision for this task is Manhattan II (MNII), which measures vibrations along the Optical Path (mirrors M1 to M7) and sends Optical Path Length (OPL) corrections to the Delay Lines (DL). In the context of GRAVITY+ upgrade, MNII is being extended to cover a larger portion of the light path (previously M1 to M3) and expanded with Phase-locked Loop (PLL) to improve OPD control by targeting specific frequencies. Alongside, several options are being explored to further improve the capabilities of the system. Active compensation is improved by the upgrade of MNII’s PLL. In addition, better troubleshooting tools and automatic Anomaly Detection (AD) systems are needed to constantly monitor and react to the changing vibration signature of the UTs. Furthermore, similar AD systems will be fundamental in the future for the operation of the upcoming Extremely Large Telescope (ELT). This work is about the ongoing efforts to develop an automatic AD system using Machine Learning on MNII’s vibration data. We focus on the different methods and models used in the proof of concept which include Auto-encoders, clustering and classical statistical methods as well, the infrastructure required to have a working end-to-end prototype, the data pipeline, preprocessing and the future envisioned production system.
The efficiency of science observation Short-Term Scheduling (STS) can be defined as being a function of how many highly ranked observations are completed per unit time. Current STS at ESO’s Paranal observatory is achieved through filtering and ranking observations via well-defined algorithms, leading to a proposed observation at time t. This Paranal STS model has been successfully employed for more than a decade. Here, we summarize the current VLT(I) STS model and outline ongoing efforts of optimizing the scientific return of both the VLT(I) and future ELT. We describe the STS simulator we have built that enables us to evaluate how changes in model assumptions affect STS effectiveness. Such changes include: using short-term predictions of atmospheric parameters instead of assuming their constant time evolution; assessing how the ranking weights on different observation parameters can be changed to optimize the scheduling; changing STS to be more ‘dynamic’ to consider medium-term scheduling constraints. We present specific results comparing how machine learning predictions of the seeing can improve STS efficiency when compared to the current model of using the last 10 min median of the measured seeing for observation selection.
SPHERE is the VLT exo-planet imager and is based on XAO and coronagraphy. Malfunctioning DM actuators can have a severe impact on the instrument contrast. 18 dead and 8 sluggish actuators were identified during commissioning, but the actuator's behavior needs to be monitored during the whole instrument lifetime. Daily, the temporal responses of SPHERE's 1377 actuators are measured at 1380Hz. The method to automatically identify the status of the actuators is based on machine learning. We used the SciKit toolbox (INRIA, France) and implemented a Support Vector Machine algorithm. The model was trained on data acquired on 167 daily measurements of dead actuators, 73 daily measurements of sluggish actuators and 334 daily measurements of good actuators. The model was then validated on 73 daily measurements of dead actuators, 26 daily measurements of sluggish actuators and 147 daily measurements of good actuators.
The method accurately identified malfunctioning actuators with an extremely low number of false positives (1). The method is easy to implement, fast (30ms) and easily scalable to systems with more degrees of liberty such as MOEMS DMs and the future ELT DMs.
Precise control of the optical path differences (OPD) in the Very Large Telescope Interferometer (VLTI) was critical for the characterization of the black hole at the center of our Galaxy - leading to the 2020 Nobel prize in physics. There is now significant effort to push these OPD limits even further, in-particular achieving 100nm OPD RMS on the 8m unit telescopes (UT’s) to allow higher contrast and sensitivity at the VLTI. This work calculated the theoretical atmospheric OPD limit of the VLTI as 5nm and 15nm RMS, with current levels around 200nm and 100nm RMS for the UT and 1.8m auxiliary telescopes (AT’s) respectively, when using bright targets in good atmospheric conditions. We find experimental evidence for the f−17/3 power law theoretically predicted from the effect of telescope filtering in the case of the ATs which is not currently observed for the UT’s. Fitting a series of vibrating mirrors modelled as dampened harmonic oscillators, we were able to model the UT OPD PSD of the gravity fringe tracker to <1nm/ √Hz RMSE up to 100Hz, which could adequately explain a hidden f−17/3 power law on the UTs. Vibration frequencies in the range of 60-90Hz and also 40-50Hz were found to generally dominate the closed loop OPD residuals of Gravity. Cross correlating accelerometer with Gravity data, it was found that strong contributions in the 40-50Hz range are coming from the M1-M3 mirrors, while a significant portion of power from the 60-100Hz contributions are likely coming from between the M4-M10. From the vibrating mirror model it was shown that achieving sub 100nm OPD RMS for particular baselines (that have OPD∼200nm RMS) required removing nearly all vibration sources below 100Hz.
There exists a fundamental link between data quality, system performance and environmental conditions. An end-to-end data driven monitoring approach allows for better system understanding and opens the door for more efficient root cause investigations when anomalies occur. To prepare for future operations, Paranal is establishing a dedicated data & system analysis framework, to investigate different operational scenarios, and test new techniques and technologies. With industry partners we are exploring the best data infrastructure suited for multi-site, multi-instrument operations to achieve reliable and robust data access for operations. By increasing system understanding we are paving the way to fully integrated operations.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.