PaperclipOrg
Claude Agent Skill · by Aradotso

Modly Image To 3d

Install Modly Image To 3d skill for Claude Code from aradotso/trending-skills.

Works with Paperclip

How Modly Image To 3d fits into a Paperclip company.

Modly Image To 3d drops into any Paperclip agent that handles this kind of work. Assign it to a specialist inside a pre-configured PaperclipOrg company and the skill becomes available on every heartbeat — no prompt engineering, no tool wiring.

S
SaaS FactoryPaired

Pre-configured AI company — 18 agents, 18 skills, one-time purchase.

$27$59
Explore pack
Source file
SKILL.md448 lines
Expand
---name: modly-image-to-3ddescription: Desktop app that generates 3D models from images using local AI running entirely on your GPUtriggers:  - generate 3D model from image  - image to 3D mesh locally  - run Modly AI model  - install Modly extension  - set up local 3D generation  - Modly GPU image to 3D  - open source image to mesh desktop app  - Hunyuan3D local generation--- # Modly Image-to-3D Skill > Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection. Modly is a local, open-source desktop application (Windows/Linux) that converts photos into 3D mesh models using AI models running entirely on your GPU — no cloud, no API keys required. --- ## Architecture Overview ```modly/├── src/                    # Electron + TypeScript frontend│   ├── main/               # Electron main process│   ├── renderer/           # React UI (renderer process)│   └── preload/            # IPC bridge├── api/                    # Python FastAPI backend│   ├── generator.py        # Core generation logic│   └── requirements.txt├── resources/│   └── icons/├── launcher.bat            # Windows quick-start├── launcher.sh             # Linux quick-start└── package.json``` The app runs as an Electron shell over a local Python FastAPI server. Extensions are GitHub repos with a `manifest.json` + `generator.py` that plug into the extension system. --- ## Installation ### Quick start (no build required) ```bash# Windowslauncher.bat # Linuxchmod +x launcher.sh./launcher.sh``` ### Development setup ```bash# 1. Clonegit clone https://github.com/lightningpixel/modlycd modly # 2. Install JS dependenciesnpm install # 3. Set up Python backendcd apipython -m venv .venv # Activate (Windows).venv\Scripts\activate # Activate (Linux/macOS)source .venv/bin/activate pip install -r requirements.txtcd .. # 4. Run dev mode (starts Electron + Python backend)npm run dev``` ### Production build ```bash# Build installers for current platformnpm run build # Output goes to dist/``` --- ## Key npm Scripts ```bashnpm run dev        # Start app in development mode (hot reload)npm run build      # Package app for distributionnpm run lint       # Run ESLintnpm run typecheck  # TypeScript type checking``` --- ## Extension System Extensions are GitHub repositories containing:- `manifest.json` — metadata and model variants- `generator.py` — generation logic implementing the Modly extension interface ### manifest.json structure ```json{  "name": "My 3D Extension",  "id": "my-extension-id",  "description": "Generates 3D models using XYZ model",  "version": "1.0.0",  "author": "Your Name",  "repository": "https://github.com/yourname/my-modly-extension",  "variants": [    {      "id": "model-small",      "name": "Small (faster)",      "description": "Lighter variant for faster generation",      "size_gb": 4.2,      "vram_gb": 6,      "files": [        {          "url": "https://huggingface.co/yourorg/yourmodel/resolve/main/weights.safetensors",          "filename": "weights.safetensors",          "sha256": "abc123..."        }      ]    }  ]}``` ### generator.py interface ```python# api/extensions/<extension-id>/generator.py# Required interface every extension must implement import sysimport jsonfrom pathlib import Path def generate(    image_path: str,    output_path: str,    variant_id: str,    models_dir: str,    **kwargs) -> dict:    """    Required entry point for all Modly extensions.        Args:        image_path:  Path to input image file        output_path: Path where output .glb/.obj should be saved        variant_id:  Which model variant to use        models_dir:  Directory where downloaded model weights live        Returns:        dict with keys:            success (bool)            output_file (str) — path to generated mesh            error (str, optional)    """    try:        # Load your model weights        weights = Path(models_dir) / variant_id / "weights.safetensors"                # Run your inference        mesh = run_inference(str(weights), image_path)                # Save output        mesh.export(output_path)                return {            "success": True,            "output_file": output_path        }    except Exception as e:        return {            "success": False,            "error": str(e)        }``` ### Installing an extension (UI flow) 1. Open Modly → go to **Models** page2. Click **Install from GitHub**3. Paste the HTTPS URL, e.g. `https://github.com/lightningpixel/modly-hunyuan3d-mini-extension`4. After install, click **Download** on the desired model variant5. Select the installed model and upload an image to generate ### Official Extensions | Extension | Model ||-----------|-------|| [modly-hunyuan3d-mini-extension](https://github.com/lightningpixel/modly-hunyuan3d-mini-extension) | Hunyuan3D 2 Mini | --- ## Python Backend API (FastAPI) The backend runs locally. Key endpoints used by the Electron frontend: ```python# Typical backend route patterns (api/main.py or similar) # GET /extensions         — list installed extensions# GET /extensions/{id}    — get extension details + variants# POST /extensions/install — install extension from GitHub URL# POST /generate          — trigger 3D generation# GET /generate/status    — poll generation progress# GET /models             — list downloaded model variants# POST /models/download   — download a model variant``` ### Calling the backend from Electron (IPC pattern) ```typescript// src/preload/index.ts — exposing backend calls to rendererimport { contextBridge, ipcRenderer } from 'electron' contextBridge.exposeInMainWorld('modly', {  generate: (imagePath: string, extensionId: string, variantId: string) =>    ipcRenderer.invoke('generate', { imagePath, extensionId, variantId }),   installExtension: (repoUrl: string) =>    ipcRenderer.invoke('install-extension', { repoUrl }),   listExtensions: () =>    ipcRenderer.invoke('list-extensions'),})``` ```typescript// src/main/ipc-handlers.ts — main process handlingimport { ipcMain } from 'electron' ipcMain.handle('generate', async (_event, { imagePath, extensionId, variantId }) => {  const response = await fetch('http://localhost:PORT/generate', {    method: 'POST',    headers: { 'Content-Type': 'application/json' },    body: JSON.stringify({ image_path: imagePath, extension_id: extensionId, variant_id: variantId }),  })  return response.json()})``` ```typescript// src/renderer/components/GenerateButton.tsx — UI usagedeclare global {  interface Window {    modly: {      generate: (imagePath: string, extensionId: string, variantId: string) => Promise<{ success: boolean; output_file?: string; error?: string }>      installExtension: (repoUrl: string) => Promise<{ success: boolean }>      listExtensions: () => Promise<Extension[]>    }  }} async function handleGenerate(imagePath: string) {  const result = await window.modly.generate(    imagePath,    'modly-hunyuan3d-mini-extension',    'hunyuan3d-mini-turbo'  )   if (result.success) {    console.log('Mesh saved to:', result.output_file)  } else {    console.error('Generation failed:', result.error)  }}``` --- ## Writing a Custom Extension ### Minimal extension repository structure ```my-modly-extension/├── manifest.json└── generator.py``` ### Example: wrapping a HuggingFace diffusion model ```python# generator.pyimport torchfrom PIL import Imagefrom pathlib import Path def generate(image_path, output_path, variant_id, models_dir, **kwargs):    device = "cuda" if torch.cuda.is_available() else "cpu"    weights_dir = Path(models_dir) / variant_id     try:        # Load model (example pattern)        from your_model_lib import ImageTo3DPipeline                pipe = ImageTo3DPipeline.from_pretrained(            str(weights_dir),            torch_dtype=torch.float16        ).to(device)         image = Image.open(image_path).convert("RGB")                with torch.no_grad():            mesh = pipe(image).mesh         mesh.export(output_path)         return {"success": True, "output_file": output_path}     except Exception as e:        return {"success": False, "error": str(e)}``` --- ## Configuration & Environment Modly runs fully locally — no environment variables or API keys needed. GPU/CUDA is auto-detected by PyTorch in extensions. Relevant configuration lives in: ```package.json          # Electron app metadata, build targetsapi/requirements.txt  # Python dependencies for backend``` If you need to configure the backend port or extension directory, check the Electron main process config (typically `src/main/index.ts`) for constants like `API_PORT` or `EXTENSIONS_DIR`. --- ## Common Patterns ### Check if CUDA is available in an extension ```pythonimport torch def get_device():    if torch.cuda.is_available():        print(f"Using GPU: {torch.cuda.get_device_name(0)}")        return "cuda"    print("No GPU found, falling back to CPU (slow)")    return "cpu"``` ### Progress reporting from generator.py ```pythonimport sysimport json def report_progress(percent: int, message: str):    """Write progress to stdout so Modly can display it."""    print(json.dumps({"progress": percent, "message": message}), flush=True) def generate(image_path, output_path, variant_id, models_dir, **kwargs):    report_progress(0, "Loading model...")    # ... load model ...    report_progress(30, "Processing image...")    # ... inference ...    report_progress(90, "Exporting mesh...")    # ... export ...    report_progress(100, "Done")    return {"success": True, "output_file": output_path}``` ### Adding a new page in the renderer (React) ```typescript// src/renderer/pages/MyPage.tsximport React, { useEffect, useState } from 'react' interface Extension {  id: string  name: string  description: string} export default function MyPage() {  const [extensions, setExtensions] = useState<Extension[]>([])   useEffect(() => {    window.modly.listExtensions().then(setExtensions)  }, [])   return (    <div>      <h1>Installed Extensions</h1>      {extensions.map(ext => (        <div key={ext.id}>          <h2>{ext.name}</h2>          <p>{ext.description}</p>        </div>      ))}    </div>  )}``` --- ## Troubleshooting | Problem | Fix ||---------|-----|| `npm run dev` — Python backend not starting | Ensure venv is set up: `cd api && python -m venv .venv && pip install -r requirements.txt` || CUDA out of memory | Use a smaller model variant or close other GPU processes || Extension install fails | Verify the GitHub URL is HTTPS and the repo contains `manifest.json` at root || Generation hangs | Check that your GPU drivers and CUDA toolkit match the PyTorch version in `requirements.txt` || App won't launch on Linux | Make `launcher.sh` executable: `chmod +x launcher.sh` || Model download stalls | Check disk space; large models (4–10 GB) need adequate free space || `torch` not found in extension | Ensure PyTorch is in `api/requirements.txt`, not just the extension's own deps | ### Verifying GPU is detected ```bashcd apisource .venv/bin/activate   # or .venv\Scripts\activate on Windowspython -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'no GPU')"``` --- ## Resources - **Homepage**: https://modly3d.app- **Releases**: https://github.com/lightningpixel/modly/releases/latest- **Official extension**: https://github.com/lightningpixel/modly-hunyuan3d-mini-extension- **Discord**: https://discord.gg/FjzjRgweVk- **License**: MIT (attribution required — credit Modly + Lightning Pixel in forks)