Vision & Multimodal

Image/video understanding and vision-language models.

Vision-language models

QwenLM/Qwen3-VL - Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.
haotian-liu/LLaVA - Large Language-and-Vision Assistant; open GPT-4V-style model.
OpenGVLab/InternVL - Open-source multimodal model family rivaling proprietary vision-language models.
LLaVA-VL/LLaVA-NeXT - Open-source large multimodal models for image and video understanding.
OpenBMB/MiniCPM-V - Pocket-sized multimodal LLM for efficient image and video understanding on phones.
NVlabs/Eagle - Frontier vision-language models trained with data-centric strategies.
NVlabs/VILA - Vision-language models for multimodal AI across edge, data center, and cloud.
OpenSenseNova/SenseNova-U1 - Native unified multimodal model for understanding, reasoning, and generation.
m87-labs/moondream - Tiny vision-language model for fast image understanding on any device.
ATH-MaaS/Ovis - Multimodal LLM architecture structurally aligning visual and textual embeddings.
om-ai-lab/VLM-R1 - Reinforcement learning framework for visual understanding with VLMs.
eulogik/TinyDoc-VLM - 256M-parameter document VLM that runs on CPU with ONNX export.
andreagemelli/baguettotron-vlm - Fully reproducible sub-1B vision-language model with open training.
microsoft/Magma - Foundation model for multimodal AI agents acting in digital worlds.
apple-aiml-research/ml-ferret - Apple's region-aware VLM for referring expression grounding.
jingyaogong/minimind-v - Train a small vision-language model from scratch in a couple of hours.

Detection, segmentation & tracking

roboflow/supervision - We write your reusable computer vision tools.
facebookresearch/segment-anything - Promptable image segmentation from Meta.
ultralytics/ultralytics - YOLO models for detection, segmentation and tracking.
open-mmlab/mmdetection - OpenMMLab detection toolbox.
facebookresearch/sam2 - Meta's promptable segmentation for images and video; a vision pipeline staple.
facebookresearch/detectron2 - Platform for object detection, segmentation, and visual recognition tasks.
ChaoningZhang/MobileSAM - Lightweight Segment Anything variant for mobile and real-time use.
facebookresearch/sam3 - Segment Anything Model 3: promptable segmentation for images and video.
yujunwei04/UnSAMv2 - Self-supervised segmentation at any granularity without manual labels.
IDEA-Research/Grounded-SAM-2 - Ground and track anything in videos using text prompts.
roboflow/rf-detr - Real-time detection transformer, SOTA on COCO and built for fine-tuning.
lyuwenyu/RT-DETR - Real-time detection transformer beating YOLOs on the speed-accuracy curve.
Peterande/D-FINE - Fine-grained distribution refinement for DETR-style object detectors.
obss/sahi - Sliced inference framework for small-object detection on large images.
PaddlePaddle/PaddleDetection - PaddlePaddle toolkit for detection, segmentation, and tracking.
qubvel-org/segmentation_models.pytorch - Segmentation models with 500+ pretrained convolutional and transformer backbones.
mikel-brostrom/boxmot - Pluggable state-of-the-art multi-object tracking modules for any detector.
blakeblackshear/frigate - NVR with real-time local object detection for IP cameras.
roboflow/inference - Deploy computer vision models to any computer or edge device.
google-deepmind/tapnet - DeepMind's Tracking Any Point models and benchmark.

Vision libraries & backbones

openai/CLIP - Contrastive image-text model; the backbone of most multimodal retrieval.
facebookresearch/dinov2 - Meta's self-supervised vision foundation model for visual perception backbones.
mlfoundations/open_clip - Open-source implementation of CLIP with many pretrained weights.
facebookresearch/dinov3 - Self-supervised vision transformer for dense visual features.
rom1504/clip-retrieval - Compute CLIP embeddings and build billion-scale image search.
illuin-tech/colpali - Train and run ColPali/ColQwen visual document retrieval models.
facebookresearch/ImageBind - One embedding space binding images, text, audio, and more.
pytorch/vision - Datasets, transforms, and models for computer vision.
huggingface/pytorch-image-models - Largest collection of PyTorch image backbones with pretrained weights.
open-mmlab/mmcv - OpenMMLab computer vision foundation library.
kornia/kornia - Differentiable geometric computer vision library.
rerun-io/rerun - Visualize, query, and stream multimodal robotics and vision data.

Face analysis

deepinsight/insightface - State-of-the-art 2D and 3D face analysis toolkit.
serengil/deepface - Lightweight face recognition and facial attribute analysis for Python.
ageitgey/face_recognition - The simplest facial recognition API for Python and the command line.

Video understanding & agents

OpenGVLab/InternVideo - Video foundation models and data for multimodal understanding.
mbzuai-oryx/Video-ChatGPT - Video conversation model generating dialogue about video content.
OpenGVLab/Ask-Anything - VideoChat: conversational video understanding with chat models.
Breakthrough/PySceneDetect - Scene cut and transition detection for video processing pipelines.
PyAV-Org/PyAV - Pythonic bindings for FFmpeg libraries in video ML pipelines.
NVIDIA-AI-Blueprints/video-search-and-summarization - GPU-accelerated blueprint for video search and summarization agents.
video-db/Director - Framework for building AI video agents and video workflows.
HKUDS/VideoAgent - Agentic framework for video understanding, editing and creation.

Depth & 3D reconstruction

DepthAnything/Depth-Anything-V2 - Capable foundation model for monocular depth estimation.
ByteDance-Seed/Depth-Anything-3 - Depth and geometry from any views with a single plain transformer.
DepthAnything/Video-Depth-Anything - Consistent depth estimation for super-long videos.
localai-org/depth-anything.cpp - From-scratch C++17/ggml port of Depth Anything 2 and 3.
apple-aiml-research/ml-depth-pro - Sharp monocular metric depth in under a second.
prs-eth/Marigold - Diffusion-based monocular depth estimation repurposing image generators.
microsoft/MoGe - Accurate monocular geometry estimation for open-domain images.
facebookresearch/vggt - Transformer inferring cameras, depth, and 3D tracks from any views.
HengyiWang/amb3r - Feed-forward metric-scale 3D reconstruction with backend optimization.
colmap/colmap - Structure-from-motion and multi-view stereo reconstruction pipeline.
isl-org/Open3D - Modern library for 3D data processing and visualization.
facebookresearch/pytorch3d - Reusable components for deep learning with 3D data.
nerfstudio-project/nerfstudio - Collaboration-friendly studio for neural radiance fields.
graphdeco-inria/gaussian-splatting - Original 3D Gaussian Splatting reference implementation.
nerfstudio-project/gsplat - CUDA-accelerated Gaussian splatting rasterization library.
VAST-AI-Research/TripoSR - Fast 3D object reconstruction from a single image.
harry7557558/spirula-studio - Cross-vendor 3D Gaussian Splatting trainer from video to mesh.