AI-Powered Image Generator From OpenAI Can Go 3D With 'Point·E' Model Generator

OpenAI Point·E

AI is about intelligence demonstrated by machines, as opposed to natural intelligence displayed by living things.

OpenAI is among the pioneers in the AI field, and having created the 'GPT' for images, the company also has a few other AI products, each with remarkable features.

And this time, it's going a step further.

Using what it calls the 'Point·E', the company debuts an AI that takes an approach of a 3D model generators.

Unlike DALL·E 2' which takes image-generation AI to an eerie level, Point·E uses AI to create 3D objects.

Whereas DALL·E products create 'raster' images of anything imaginable, Point·E creates 'vector' images of anything imaginable.

OpenAI Point·E

What this means, Point·E doesn’t create 3D objects in the traditional sense.

Instead, it generates the 3D objects using points, which are discrete sets of data points in space to represent a 3D shape it wants to create.

The thing is, point clouds it uses cannot capture an object's fine-grained shapes and textures, resulting in blocky or distorted results.

But rather than giving in to the fact that point clouds cannot create life-like results like DALL·E, OpenAI uses machine-learning technology to covert the point clouds into meshes.

Meshes, which are collections of vertices, edges and faces that define an object, are commonly used in 3D modeling and design.

Outside of the mesh-generating model, which stands alone, Point·E consists of two models that work together.

The first, is a text-to-image model, a generative art systems like DALL-E and Stable Diffusion, which was trained on labeled images to understand the associations between words and visual concepts.

After the first model finishes its task, the second model goes to work. The second is an image-to-3D model, which was trained with a set of images paired with 3D objects.

OpenAI trained the AI using a dataset of "several million" 3D objects and their associated metadata.

According to OpenAI, Point·E could produce colored point clouds that frequently matched text prompts.

OpenAI Point·E

While the approach makes results little less convincing than DALL·E in a real lifelike sense, and that the images it generates may not be perfect, Point·E's 3D objects are a lot faster to create.

In fact, the letter "E" in Point·E stands for "efficiency," and this is possible because generating 3D objects using points is a lot less resource consuming.

"While our method performs worse on this evaluation than state-of-the-art techniques, it produces samples in a small fraction of the time," the researchers wrote in their research paper (PDF). "This could make it more practical for certain applications, or could allow for the discovery of higher-quality 3D object.

At this time, all eyes are on 2D art generators, as software products enhanced with model-synthesizing AI are trying to disrupt existing industries.

3D object generators are less likely to reach end users, since they're more widely used in films and TV, interior design, architecture and various science fields.

This is why this kind of AI can be used for things, like fabricating real-world objects, where it enhances 3D printing using its mesh-converting model.

And if the technology behind it is perfected, Point·E's AI could also be used into the game industry, in which it could help with the generation of artifacts and visual animations.

OpenAI has released Point·E's codebase to GitHub.

Published: 
22/12/2022