Point-E: OpenAI's Open-Source Text-to-3D Model, Explained

Point-E is an open-source AI model from OpenAI, released in December 2022, that generates a 3D point cloud (a cloud of colored dots in the shape of an object) from a short text prompt or a single image, plus a second small model that turns that cloud into a mesh you can open in other 3D software. It's free under the MIT license, and it's a Python project you run on your own computer, on an Nvidia graphics card if you have one (the example notebooks fall back to the processor if you don't). Two things up front: the code hasn't been touched since December 2022, and OpenAI's follow-up, Shap-E, arrived in 2023. On paper, it's a great way to see how text-to-3D started, and not where I'd start a 3D project today.

What Point-E actually makes

A quick bit of jargon, sorry: a point cloud isn't a solid 3D model. It's a set of points floating in space, each with a color, that together draw the shape of an object. A mesh is the surface made of tiny triangles that games, 3D printers and modeling programs actually use.

The paper describes the full system in two steps. First, a text-to-image model (a version of OpenAI's GLIDE, fine-tuned on 3D renders) draws a single synthetic view of the object. Then a second diffusion model looks at that picture and generates a colored point cloud, which an upsampler grows from 1,024 to 4,096 points. If you want a mesh, a separate SDF model (it estimates where the object's surface is) builds one from the cloud, and the example notebook saves it as a PLY file "to import into some other program."

Its selling point in 2022 was speed. The text-to-3D methods of the time needed hours of GPU time per object; the abstract says Point-E produces a 3D model in 1-2 minutes on a single GPU. OpenAI was upfront about the trade-off: it falls short of the best methods in quality, but it's one to two orders of magnitude faster.

The catch: the text-to-3D in the repo isn't the one in the paper

This is the part that's easy to miss. The official release includes the point cloud models (three image-to-3D sizes: 40 million, 300 million and 1 billion parameters), the upsampler and the mesh model. The fine-tuned GLIDE model that turns text into the starting image isn't among the downloads, and a user has asked for those weights in the project's issues.

So, from the repo, you have two routes. The text route uses a small model that reads your prompt directly, and the README is refreshingly honest about it: "small, worse quality," with "limited" capabilities, though "it does understand some simple categories and colors." The notebook's sample prompt is "a red motorcycle." The README doesn't add that warning to the image route, and its example notebook notes that the bigger models give "better results." That route expects a single rendered view of an object, like the stack of cubes in the example notebook.

What you need to run it

Point-E is a Python package built on PyTorch. The README's whole install guide is one line, pip install -e ., run inside the downloaded repository, and the package also pulls in OpenAI's CLIP straight from GitHub. The README doesn't list hardware requirements. What the code tells you is that the notebooks use an Nvidia card through CUDA when there's one and the processor when there isn't. The paper's 1-2 minutes were measured on an Nvidia V100, a data-center GPU, and the paper gives no figures for running it on a processor.

The model weights download automatically from OpenAI's servers the first time a notebook needs them, and those links still work today. If "notebook," "CUDA" and "pip" all sound like a foreign language, the two videos on this page are both about getting Point-E running.

Is Point-E still maintained?

No. The official repository has a handful of commits, all from December 2022, no releases, and open issues and unmerged pull requests from users ever since. It hasn't been archived, and it's still public, with the code there to read and fork, but nobody at OpenAI is updating it.

The same goes for the online demo. OpenAI published a Point-E Space on Hugging Face in December 2022, and today it shows a runtime error instead of the app.

Point-E vs Shap-E

Shap-E is OpenAI's follow-up 3D model, published in May 2023 by two of Point-E's authors, also under the MIT license. Instead of points, it generates the parameters of an implicit function, a mathematical description of the shape that can be turned into a textured mesh or a NeRF (a neural radiance field, a way of rendering a 3D scene from learned data). Its paper compares it directly with Point-E and says Shap-E learns faster and reaches comparable or better quality. Like Point-E, it comes with example notebooks for text-to-3D and image-to-3D.

One honest caveat: Shap-E isn't fresh either, as its last commit is from November 2023. Still, if you're going to install one of these two, the authors' own comparison points to Shap-E. Point-E is the one to read if you want to understand the idea in its simplest form.

When to use something else

Point-E makes rough shapes from simple prompts, so for many jobs another tool on this site fits better:

• You want a 3D person. LHM rebuilds an animatable 3D human from a single image.
• You want a real object or place in 3D. Luma captures objects and scenes with your phone and exports them to game engines.
• You actually need a flat graphic. StarVector is an open-source model that turns images and text into SVG vectors.

And if what you want is to peek inside an early text-to-3D model anyone can still download, Point-E is still right there on GitHub, MIT license and all. Not bad for a project that hasn't changed in years. 🧊