The gap between designers and developers has just gotten smaller.
Previously, it has been the task of front-end developers to transform the work of designers from graphical user interface (GUI) mockups to actual source code.
This process can be and time-consuming, preventing developers from dedicating the majority of their time implementing with actual features and logic of the software they're building.
What's worse, the computer languages used to implement the GUIs are specific to each target platform. Since it can be a repetitive task for developers, the work can be tedious, putting a wide margin for errors.
But with Artificial Intelligence (AI), this is changing.
Copenhagen-based startup Ulzard Technologies has developed a novel approach based on Convolutional and Recurrent Neural Networks that allows the generation of computer code from a single GUI screenshot as input.
The software is called 'Pix2Code'. What it needs, is only one image of a design for its graphic user interface (GUI) to work. Once its AI recognizes the image, it will start creating the corresponding code.
The AI is trained but still in prototype. The model can generate source code targeting three different platforms (i.e. iOS, Android and multi-platform web-based technologies which use HTML/CSS interfaces) from a single input image with over 77 percent of accuracy.
Tony Beltramelli, the founder of Ulzard, said that he believes Pix2Code has the potential to "end the need for manually-programmed GUIs".
For the AI to work, the team at Ulzard Technologies needed to overcome three main problems:
The first is computer vision. Computers won't be able to understand a context without identifying the objects that are present, as well as their characteristics (labels, tags, buttons, etc.).
Second is the language problem. They need to teach the network to understand text so it could create the correct samples.
And the third is deep learning. The network had to be trained to understand the connections between code, text and corresponding images. The AI needs to be able to connect the three while generating codes.
According to Beltramelli, Pix2Code can use Generative Adversarial Networks (GANs) to improve its ability. GANs have shown promising accuracy when generating sequences and images. However, since it is a relatively unexplored area of research, it may take a considerable amount of time to train the neural network. But with the internet, the time frame can be reduced.
"Considering a large number of websites are already available online and the fact that new websites are created every day, the web could theoretically supply an unlimited amount of training data," said Beltramelli.
Beltramelli has documented Pix2Code in an academic paper. Ulzard Technologies is also giving the AI's source code to the public.