The Universe of Data.
In Mexico, there are three major observatories: the San Pedro Mártir National Astronomical Observatory, the Large Millimeter Telescope Alfonso Serrano (GTM), and the HAWC project. These three major observatories are responsible for covering three large and important research areas: visible infrared, millimeter wavelengths, and gamma rays, respectively.
Despite not having the best infrastructure or a vast number of observatories in the national territory, one thing is quite clear: although I couldn't find data specific to Mexican observatories, according to an article published by the NRAO (National Radio Astronomy Observatory) in 2011, the Atacama Large Millimeter/submillimeter Array (ALMA) telescope located in Chile, which bears a strong resemblance to the GTM in Mexico, is capable of generating 1GB/s or 43 TB/day.
In Mexico, this data is primarily processed by the National Autonomous University of Mexico (UNAM), which has a high-performance data and computing center located at the Institute of Nuclear Sciences (ICN).
It is composed of 45 processing nodes and 52 storage servers that offer an incredible five petabytes. Let's view this data center as the brain of Mexican astronomy; each processing node is a neuron that processes fragments of the universe, and together they convert millions of data points into knowledge.
But… that's not all. UNAM not only works with national observatories; according to a recent article from UNAM's General Directorate of Social Communication, work is also underway with a data center that will help process two thousand Terabytes of data out of the 500 thousand that will be produced over 10 years of information by the Vera C. Rubin Observatory located in Chile.
Lots of data information, but now let's get to what really matters: how is this data processed?. The fundamental steps to calibrate the data and convert it into coherent and observable images for humans are divided into two essential parts
Basic processing, which calibrates and corrects data based on individual CCDs. We can think of CCDs like old photographic film rolls; images come out with imperfections, different light levels, and spots. Calibration and basic processing algorithms are responsible for correcting these imperfections by adjusting brightness, contrast, color, and removing spots.
The other essential part of processing is post-processing, which performs the final image construction. Between these two processes, there's something called pixel tables, which are nothing more than small images like a puzzle. These images have already been cleaned and reviewed but are not yet placed to fit with the others, which is what post-processing does, assembling hundreds or thousands of these pieces into a single coherent astronomical image.
Currently, in addition to all the work already done with data received from observatories, artificial intelligence also plays a fundamental role. Technologies like machine learning, which is a branch of AI, analyze images and can differentiate galaxies, stars, and other celestial bodies, making the prediction of cosmic events much easier and accelerating the pace of astronomical research.
All this leads us to believe that data science, and generally careers like computer systems engineering, are fundamental for the proper development of Mexican astronomy. We can analyze how this field depends almost entirely on data analysis to process information received from space.
It's incredible to imagine all the scientific advancements that led us to develop such advanced algorithms, to manipulate data in a way that brings us closer to understanding the vast universe. If we achieved this in recent years, I can't even imagine what we will accomplish in the near future.
