At the same time, Microsoft is publicly sharing its “optimization solver” algorithm and the “digital twin” it developed so that researchers from other organizations can investigate this new computing paradigm and propose new problems to solve and new ways to solve them.
Francesca Parmigiani, a Microsoft principal research manager who leads the team developing the AOC, explained that the digital twin is a computer-based model that mimics how the real AOC behaves; it simulates the same inputs, processes and outputs, but in a digital environment – like a software version of the hardware.
This allowed the Microsoft researchers and collaborators to solve optimization problems at a scale that would be useful in real situations. This digital twin will also allow other users to experiment with how problems, either in optimization or in AI, would be mapped and run on the AOC hardware.
“To have the kind of success we are dreaming about, we need other researchers to be experimenting and thinking about how this hardware can be used,” Parmigiani said.
Hitesh Ballani, who directs research on future AI infrastructure at the Microsoft Research lab in Cambridge, U.K. said he believes the AOC could be a game changer.
“We have actually delivered on the hard promise that it can make a big difference in two real-world problems in two domains, banking and healthcare,” he said. Further, “we opened up a whole new application domain by showing that exactly the same hardware could serve AI models, too.”
In the healthcare example described in the Nature paper, the researchers used the digital twin to reconstruct MRI scans with a good degree of accuracy. The research indicates that the device could theoretically cut the time it takes to do those scans from 30 minutes to five. In the banking example, the AOC succeeded in resolving a complex optimization test case with a high degree of accuracy.
Applying the AOC for practical solutions
The modern concept of analog optical computing dates to the 1960s, and the technology used to create this AOC is not new either. For nearly 50 years, fine glass threads, which make up fiber optic cables, have been used to transmit data.
Photons are the fundamental particles of light, and they do not interact with each other. But when they pass through an intermediary, like the sensor in a digital camera, they can be used in computations. The Microsoft researchers used projectors with optical lenses, digital sensors and micro-LEDs – which are many times finer than a human hair – to build the AOC.
As the light passes through the sensor at different intensities, the AOC can add and multiply numbers – this is the basis for solving optimization problems. This was the first class of problems that the researchers were able to address using the AOC.
Optimization problems, simply defined, have the goal of finding the best solution from among nearly endless possibilities. The classic example is the “traveling salesman problem”: If a traveling salesperson tried to find the most efficient route for visiting five cities just once before returning home, there are 12 possible routes. But if there are 61 cities, the number of potential routes surpasses billions.
For the research that led to the Nature paper, the team built an AOC with 256 weights, or parameters. The previous generation of the AOC had only 64.
More weights mean the capacity to solve more complex problems. As researchers refine the AOC, adding more and more micro-LEDs, it could eventually have millions or even more than a billion weights. At the same time, it should get smaller and smaller as parts are miniaturized, researchers say.
Parmigiani said that the AOC is “not a general purpose computer, but what we believe is that we can find a wide range of applications and real-world problems where the computer can be extremely successful.”
Making the right choices in transactions
One such practical problem resides in the world of finance. The Nature paper details a multi-year research project with Barclays Bank PLC to try to solve the type of optimization problem that is used every day at the clearinghouses that serve as intermediaries between banks and other financial institutions.
The delivery-versus-payment (DvP) securities problem aims to find the most efficient way to settle financial obligations between multiple parties in compliance with regulations while minimizing costs or risks within the constraints of time and the balances available.
The team building the AOC consists of experts from several different disciplines, including Kiril Kalinin, a mathematics-focused senior researcher with expertise in optimization and machine learning who worked with Barclays’ research team to create a sample transaction settlement problem and solve it.
The problem Barclays and Microsoft Research created involved up to 1,800 hypothetical parties and 28,000 transactions.
That represents only one batch of transactions among the hundreds of thousands that are settled daily in a large clearinghouse. Solving a representative smaller version of the problem on the actual hardware and large ones on the digital twin showed that it could be done at a much larger scale with future generations of the AOC, which the Microsoft Research team envisions creating every two years.
“It is an absolute giant problem with massive real-world finance impact,” said Ballani, noting that the value of the research transcends the interests of one bank. “It’s already a problem where banks need to collaborate, and better algorithms help everyone.”
Shrirang Khedekar is a senior software engineer with the Advanced Technologies department at Barclays. He worked with the Microsoft Research team to create the dataset and parameters used in the research, and he is a co-author on the Nature paper about the AOC. He said he and the Cambridge U.K. Microsoft Research team constructed a version of the transaction settlement problem. The results showed the potential of the technology, he said, and Barclays is interested in continuing to solve optimization problems as the capacity of future generations of the AOC grows.
“We believe there is a significant potential to explore,” Khedekar said. “We have other optimization problems as well in the financial industry, and we believe that AOC technology could potentially play a role in solving these.”
A future with shorter scans?
Another promising area for analog optical computers is in MRI scans.
Microsoft researchers crafted an algorithm for the AOC that could solve an optimization problem that would reduce the amount of data needed to produce an accurate result. The Nature paper describes how this use of the AOC could potentially allow a much quicker scan, which would make it possible to do more scans with one MRI machine each day.
Michael Hansen is senior director of biomedical signal processing at Microsoft Health Futures. He worked with the Cambridge-based researchers on the AOC project and is also a co-author of the Nature paper.
“To be transparent, it’s not something we can go and use clinically right now,” he said. “Because it’s just this little small problem that we ran, but it gives you that little spark that says, ‘Oh boy! If this instrument was actually in full scale’ …”
He said that the digital twin of the AOC was key in proving the viability of future versions of the machine in this use case. “The digital twin is where we can work on larger problems than the instrument itself can tackle right now,” he said. “And in that we can actually get good image quality.”
The research is based on the processing of mathematical equations, the researchers say. It is not at a point of being used in a clinical setting.
Hansen said he and the Cambridge team are thinking about a future where the data from MRI machines could be streamed to an AOC in Azure, and the results streamed back to the clinic or hospital. “We have to find ways to take the raw data and stream it to where the computers are,” he said.
A future with AI capabilities
From the beginning of the AOC project, the team hoped to be able to use it to run AI workloads. At first, they didn’t see a clear path forward.
That changed with a serendipitous moment during a group lunch at the Microsoft lab in Cambridge. Jannes Gladrow, a principal researcher whose specialty is AI and machine learning, was in the audience, Ballani recalled.
“He started asking very detailed questions, and I think we ended up talking for about three hours,” he said. In hearing about the unique qualities of the AOC, Gladrow saw potential ways to capitalize on them.
Gladrow and Jiaqi Chu from the AOC research team worked together to map an algorithm to the AOC that would allow it to carry out simple machine learning tasks. The team’s success in carrying out these tasks is detailed in the Nature paper and points toward a future where it could run large language models.
“I think what’s important to understand is the machine is small,” Gladrow said. “It can only run a small number of weights at the moment because it’s a prototype.”
But he said that because of the way the AOC operates, computing a problem again and again in search of a “fixed point,” it has the potential to do a kind of energy-demanding reasoning that current LLMs running on GPUs struggle with – state tracking – at a much lower cost in energy.
State tracking can be compared with playing chess. You have to be aware of the rules of the game, the moves and strategies being made in the present moment and then anticipate and strategize to achieve checkmate. An LLM running on a future version of the AOC could in theory execute complex reasoning tasks with a fraction of the energy.
“The most important aspect the AOC delivers is that we estimate around a hundred times improvement in energy efficiency,” Gladrow said. “And so that alone is unheard of in hardware.”
In Ballani’s view, the research team has reached an important milestone, but it’s really just the beginning of a steep climb toward a commercially viable analog optical computer.
“We’ve been able to convince ourselves and hopefully a broader segment of the world that, well, actually, you know what? There are real applications for the AOC,” Ballani said.
“Our goal, our long-term vision is this being a significant part of the future of computing, with Microsoft and the industry continuing this compute-based transformation of society in a sustainable fashion.”
Top photo: A detail of the analog optical computer at the Microsoft Research lab in Cambridge, U.K. It uses different intensities of light passing through a digital sensor to make its computations. Photo by Chris Welsch for Microsoft.
Related links:
Learn more: Nature publishes peer-reviewed paper describing the AOC project and its use cases
Read more: Building a computer that solves practical problems at the speed of light
Learn more: The basics of the AOC project
Access the algorithm used in the optimization use cases: The AOC optimizer QUMO abstraction
Test the digital twin: https://github.com/microsoft/aoc