NASA’s Hubble telescope has a mystery computer problem. Here’s how they are trying to fix it | ZDNet

NASA’s engineers are currently trying to diagnose and fix a computer failure; but this is not an easy thing to do when the problem is on a bus-sized telescope currently orbiting in space, 547 kilometers above our heads, at a speed of about 27,000 kilometers per hour. 

A couple of weeks ago, NASA’s beloved Hubble Space Telescope, which over the course of its 31-year career has provided us with many enchanting pictures of distant stars and galaxies, started showing some worrying symptoms.  

The telescope’s payload computer, one of the central systems that controls and coordinates the science instruments onboard the spacecraft, and transmits science and engineering data to the ground, unexpectedly came to a halt.  

This was immediately picked up by NASA’s team because the payload computer and the main computer on the telescope are normally constantly sending each other a “keep-alive” signal – known by NASA engineers as a “handshake” – to indicate that all is well. 

On 13th June, the handshake was lost, causing the main computer to automatically place all of the telescope’s science instruments in safe mode. This means that, although the telescope continues to point to its scheduled target to make sure that the sun keeps hitting its solar panels, all of the instruments on board are on standby. 

At first sight, the glitch did not seem particularly worrying, explains Paul Hertz, NASA’s astrophysics division director.  

“In general, for all our space telescopes, anomalies that interfere with normal operations happen once or twice a year, so it’s not like this has never happened before,” Hertz tells ZDNet. “Something doesn’t behave as the computer expected it, therefore we pause so the human can get in the loop, figure out what’s wrong and tell the computer what to do next.” 

Since Hubble launched in 1990, as a space observatory tasked with watching the most distant stars and galaxies yet seen, the telescope has been monitored by a team back on Earth, which remotely controls the system and ensures its health and safety.  

And in a similar fashion to Earth-based IT resolutions, the team started by switching the payload computer on and off – but soon hit the same problem.  

The engineers then hypothesized that the issue came from a degrading memory module, and switched on one of the three back-up modules available. There again, the problem persisted, with the command to initiate the backup module failing to complete. This is when NASA decided to ramp things up and switch on the backup payload computer.  

Both the payload computer that has been in use so far and the backup system were installed by astronauts during Hubble’s last servicing mission in 2009. In other words, NASA’s team placed its bets on a backup system that has been switched off for 12 years. To put things into perspective, that’s like trying to bring back to life a Blackberry Storm. 

But space is not as concerned with such earthly timeframes. “Hubble is 31 years old, so many of the systems on the telescope are 31 years old too” says Hertz. “The computers are actually the newest thing on Hubble since they were installed in the last servicing mission.” 

It remains that swapping payload computers was a delicate operation, says Hertz, requiring carefully drafted procedures and several levels of review before getting the green light.  

Whether or not the backup payload computer had amassed too much space dust over the years ended up being the least of NASA’s concerns. Once the switch was successfully carried out, the team was faced with exactly the symptoms as before – commands to write into or read from memory are still not going through. 

That means, the issue is somewhere else in the telescope. NASA has started the hunt, says Hertz, with a list of components that might be the culprits. They have even brought back on the team some of the engineers that worked on designing Hubble over three decades ago.  

“It’s very much like detective work,” says Hertz. “You have a limited number of clues and you run through all of your hypotheses to see which ones match the clues. Then once you have a match, you have to figure out what questions to ask the system, what changes you can make to see how it behaves that could help distinguish between the hypotheses, and of course without putting the observatory at any risk.” 

Among the possible contenders, NASA’s engineers are particularly suspicious of Hubble’s command unit, which sends commands and data to the science instruments, and of its science data formatter, which formats science data from the instruments and transmits them to the ground. 

But the problem could also come from the power regulator, which supplies voltages to different hardware.

It’s looking like the next few days, and potentially weeks, will be all about trial and error for NASA’s engineers – with the additional challenge, of course, of having to perform operations on a system that cannot be physically touched. 

“If Hubble were sitting in a lab, we’d just go in, check out this component and that component, and find immediately what’s wrong,” says Hertz. “But we can only talk to Hubble through its radio, we can only ask it to do the things it was programmed to do, we can only ask it to send the data it was programmed to send down.” 

Hubble was never programmed to send down information about whether voltage is well distributed among all of the systems, for example. Testing, therefore, will take longer and require a dose of inventiveness. 

Some might speculate, rather poetically, that the problem is elsewhere entirely. In the next few months, Hubble will be joined in space by a new observatory, the James Webb Space Telescope, due to launch in October 2021 to study the evolution of our own solar system. Has the telescope sensed that after 31 years, it was time to gracefully retire to make space for the next generation of space-bound observatories? 

Absolutely not, according to Hertz. Today, Hubble can boast a total of 1.4 billion observations that have contributed to 18,000 peer-reviewed science papers; the telescope has travelled for than six billion kilometers and transmits about 150 gigabits of raw science data every week.  

For Hertz, therefore, it remains one of the most productive astronomical telescope in the world, and is nowhere near ready for early retirement. 

“We will continue to operate Hubble for as long as it remains scientifically productive,” says Hertz. “There is a very long list of problems that we can fix, and I’m very confident that somewhere in that list is the thing that’s actually wrong. We’ll figure it out, make the correction needed and bring Hubble back into service.” 

Working in tandem with James Webb, Hubble is expected to keep up its contribution to the scientific community. Assuming that NASA can find and fix the current problem, hopefully it will be a while yet before the old telescope decides that it is time to bow out.