Artificial Intelligence is when intelligence is demonstrated by machines, as opposed to natural intelligence displayed by animals including humans.
AI is shown when a computer can be made smart by learning from patterns to understand something that it wasn't programmed on.
While AI shows a lot of promises, the process to train AIs is expensive.
With increasing amount of data and the needed hardware to train AIs, only a few big companies can really afford the resources.
Microsoft is one of the most prominent in the AI industry, and this time, the company introduces 'Singularity'.
In a research paper codenamed "Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads that is co-authored by 26 Microsoft employees, it is detailed that:
"Singularity is built with one key goal: driving down the cost of AI by maximizing the aggregate useful throughput on a given fixed pool of capacity of accelerators at planet scale, while providing stringent SLAs for multiple pricing tiers."
The project aims to reduce the cost of AI training, and eliminate the wasted efforts when computing at a global scale.
Microsoft mentions a test run on Nvidia DGX-2 servers using a Xeon Platinum 8168 with two sockets of 20 cores each, 8 V100 Model GPUs per server, 692GB of RAM, and networked over InfiniBand. With hundreds of thousands of GPUs under Singularity's disposal, it also uses FPGAs and other accelerators.
According to Microsoft, Singularity is a global infrastructure service that treats all devices within the infrastructure as a single cluster, which helps ensure that the devices are used to their full potential.
This in turn should reduce the resources needed, and should then reduce the cost needed to train AIs.
"While opportunistically using spare capacity, Singularity simultaneously provides isolation by respecting job-level SLAs," said Microsoft.
"For example, Singularity adapts to increasing load on an inference job, freeing up capacity by elastically scaling down or preempting training jobs."
Another way Singularity can reduce resources, is through its ability to resume where it left when it failed.
In contrast to some other systems that require restarting from scratch when failure happens, Singularity can jump back to where the job was cut off.
This can significantly reduce wasted efforts and resources during training jobs. In certain cases, this can translate to saving weeks of work.
In short, Singularity achieves what it must do using a "novel workload-aware scheduler that can transparently preempt and elastically scale deep learning workloads to drive high utilization without impacting their correctness or performance, across a global fleet of AI accelerators (e.g., GPUs, FPGAs)."
Singularity is also able to convert niche features such as elasticity into mainstream.
"Singularity achieves all of this with a remarkably simple user experience: the user focuses only on the ML task and does not need to think about checkpointing or elasticity; these mechanisms are infrastructure optimizations that are completely transparent to the user," the researchers concluded on the paper.
Microsoft is among the biggest players in the tech industry, which also focuses on AI development.
Previously, Microsoft has invested heavily in AI, including a $1 billion investment in OpenAI in 2019.
At this time, Microsoft Azure system is already among the most powerful supercomputers in the world, capable of large-scale computing and machine learning.
