The posting
The Multimedia Lab builds highly efficient, scalable video codec hardware solutions (FPGA and ASIC) from the ground up, serving billions of users. We bring together algorithm, architecture, software, firmware, and hardware design and verification expertise, designing high-performance video silicon IP across ASIC and SoC digital design, verification, and next-generation video codec hardware. Alongside the silicon work, we advance research in video understanding, perceptual quality assessment, image and video enhancement, and intelligent coding, including using multimodal models to improve coding decisions.
Quality, efficiency, and speed rarely improve together. Sharper images and better perceived quality usually cost bitrate, compute, or latency, and a technique that works in research does not always survive production at global scale.
You will take ideas from research through to silicon and production systems, where research decisions show up in the cost and quality of video across ByteDance's products.
Explore how multimodal large language models (MLLMs) can understand video content, reason over events across time, and assess perceptual quality. Research may cover long-video understanding, temporal and spatial grounding, interpretable quality assessment, user-aligned video enhancement, and content-aware processing and compression.
We are looking for talented individuals to join us for an internship. PhD internships at Our Company provide students with the opportunity to actively contribute to our products and research, as well as to the organization's future plans and emerging technologies. Our dynamic internship experience blends hands-on learning, enriching community-building and professional development events, and collaboration with industry experts. Applications will be reviewed on a rolling basis, so we encourage you to apply early. Please clearly state your availability in your resume (Start date, End date).
Responsibilities - Research and prototype multimodal models for video understanding, including temporal reasoning, event localization, temporal grounding, region-of-interest detection, and salient object detection. - Develop video quality assessment methods that identify and explain spatial and temporal degradations, and investigate how semantic context and user preferences affect perceived quality. - Contribute to dataset and benchmark development, including data curation, annotation design, evaluation protocols, and analysis of model accuracy, robustness, and generalization. - Explore efficient adaptation and inference of multimodal foundation models for video tasks, such as supervised fine-tuning, parameter-efficient adaptation, frame selection, and visual token compression. - Work with researchers and engineers to connect video understanding and quality assessment with enhancement and encoding strategies; evaluate prototypes through offline experiments and, where applicable, production validation and A/B testing.
Minimum Qualifications - Currently pursuing a PhD in Computer Science, Computer Engineering, Electrical Engineering, or a related technical discipline. - Strong foundations in machine learning and computer vision, with research or project experience in at least one of the following: multimodal learning, video understanding, image/video quality assessment, or video processing. - Proficiency in Python and hands-on experience with a deep learning framework such as PyTorch or TensorFlow. - Ability to design experiments, implement research ideas, analyze results, and communicate technical findings clearly. - Familiarity with Transformer architectures, vision-language models, or MLLMs; hands-on experience with model training, fine-tuning, inference, or evaluation.
Preferred Qualifications - Experience in long-video understanding, temporal grounding, spatiotemporal reasoning, video question answering, or cross-modal alignment. - Experience in subjective/objective quality assessment, perceptual modeling, or explainable analysis of visual and temporal degradations. - Familiarity with supervised fine-tuning, parameter-efficient fine-tuning (e.g., LoRA), preference alignment, or data-efficient learning methods. - Familiarity with video enhancement and compression, including super-resolution, denoising, diffusion models, or content-aware encoding. - Experience with Linux, OpenCV, FFmpeg, distributed data processing, or GPU-based model training; proficiency in C/C++ is a plus. - Evidence of research capability through publications, open-source contributions, or substantive research projects, and an interest in translating research into practical multimedia applications.
