Hunyuan3D on Apple Silicon
A state-of-the-art image-to-3D model, ported natively to Apple Silicon and numerically verified against the original. Plus a custom web tool that turns the pipeline into a usable product
June 2026
Project
Hunyuan3D on Apple Silicon
Technologies Used
Challenge
The story starts with a birthday present: a 3D printer. What I quickly ran short of was new models to print. A 3D scanner was out of budget, and services like Meshy.ai that generate 3D models from photos end up costing more in subscriptions than the printer itself. The developer answer was obvious: build it myself.
The foundation exists as open source: Tencent's Hunyuan3D-2.1 turns a single photo into a complete 3D mesh, but like practically every image-to-3D model it is nailed to CUDA and therefore to NVIDIA hardware. If you develop on a Mac, you're left watching from the sidelines.
The goal: get the complete shape pipeline (diffusion transformer, VAE decoder, vision conditioner) running natively on my own MacBook (M5 Max, 64 GB). With Apple's MLX framework directly on the Metal GPU, no CUDA and no PyTorch at runtime.
The real difficulty isn't translating code, it's proving correctness: a diffusion model keeps producing pretty results out of even the most subtle porting bug, they're just wrong. “Looks good” is not a test. Three requirements were therefore non-negotiable:
- Numerical parity: every ported component must demonstrably compute the same thing as the PyTorch reference, not just look similar
- Fully local: from image upload to finished GLB mesh, everything runs on my own machine
- Usable: not just a CLI script, but a real web interface with a 3D viewer, progress display and download
Solution
The result is a monorepo with two halves: the MLX port as the compute engine (Python) and a custom web tool as the product on top (Next.js with react-three-fiber), connected via an asynchronous API server. The engine first.
The port: diffusion transformer through volume decode
The entire chain was ported: the MMDiT diffusion transformer including mixture-of-experts routing, the flow-matching scheduler with classifier-free guidance, the DINOv2 vision conditioner and the ShapeVAE decoder with FlashVDM, a hierarchical volume decode that samples the 3D field coarsely first and only refines where the surface actually runs. On top of that, a weight converter that transfers the original checkpoints into MLX safetensors. Alongside the single-view pipeline, the multiview variant also runs, combining four views (front, left, back, right) into a more consistent mesh.
Parity as methodology, not spot checks
Every component is tested against the original PyTorch implementation: identical weights, identical input, then bit-level comparison of the outputs (tolerance ~1e-5 on CPU). The lesson of the project: random-weight tests aren't enough. A bug in the mixture-of-experts gate (the expert selection picked more than k experts when scores were nearly tied) stayed invisible with random weights and only surfaced with real model weights. Data-dependent paths like top-k routing need tests with real data.
End-to-end proof instead of gut feeling
The final test runs real sampling steps with identical noise and conditioning on both implementations and compares the latents: relative deviation ~1.3e-3, pure float16/GPU rounding. Since VAE decode and marching cubes are deterministic and individually verified, latent equality implies mesh equality. That makes the port not a “reimplementation by eye”, but a provably equivalent implementation.
The web tool: turning the pipeline into a product
To make the engine actually usable, I built a custom web tool that guides the whole flow: upload an image via drag and drop (a single image, or four views for the multiview variant), set quality and mesh resolution with sliders, generate, inspect, download. Behind it, a five-state machine drives the flow from empty through ready and generating to success or error. Because a generation takes 20 to 120 seconds, the API server deliberately holds no HTTP request open (a dev proxy would cut the socket): the work runs in a background thread, the frontend polls for progress and receives the finished GLB at the end.
Details that tell the story of the wait
The polish is in the choreography. While the model is computing, a GLSL particle preloader shows a billowing point cloud that crossfades into the real mesh when it's done, rendered as a clay material with orbit controls and robust against WebGL context loss. And because the whole thing is meant for 3D printing, there's a live preview of triangle reduction: a slider decimates the mesh in real time right in the viewer before you export it, for smaller files with no nasty surprise in the slicer.
Code Examples
Results
A state-of-the-art image-to-3D model runs entirely locally on the MacBook, from photo to downloadable 3D mesh, without a single byte leaving the machine:
- Complete shape pipeline natively on the Metal GPU: no CUDA, no PyTorch at runtime
- Numerically verified: denoiser rel. error 5.6e-6, end-to-end latents ~1.3e-3 (pure fp16 rounding) against the PyTorch reference
- Two pipeline variants: single-view and multiview (four views for more consistent geometry)
- Custom web tool on top of the engine: drag and drop, 3D viewer with particle preloader, live preview of triangle reduction and GLB export
- Three ways into the pipeline: CLI, asynchronous JSON API and the web app
- Test suite with a parity harness and mutation tests as a permanent safety net
An honest comparison is part of the story: the port doesn't match the resolution of Meshy.ai, where data-center GPUs do the computing while here it's a laptop. In return, everything runs locally, there's no subscription, and the 3D printer stays fed. Beyond the original motivation, a transferable methodology emerged: verifying ML ports so that “runs on my hardware” also means “computes the same thing”. The model weights are subject to Tencent's non-commercial license and the port is purely a research and learning project, which is why the code isn't published yet: I'm currently clarifying the EU licensing situation directly with Tencent.
Insights
A PNG logo becomes a 3D mesh: single-image upload, quality controls, 345,000 faces in the react-three-fiber viewer. Export as GLB, STL, OBJ or PLY, for 3D printing and more.