Tsung-Yi Lin

I am a principal research scientist at NVIDIA Research. I was previously at Google Research, Brain Team.

I work on computer vision and machine learning. I did my PhD at Cornell University and Cornell Tech, where I was advised by Serge Belongie. I did my masters at University California, San Diego and my bachelors at National Taiwan University. I received the Best Student Paper Award for Focal Loss at ICCV 2017. I led the creation of the COCO dataset which received the PAMI Mark Everingham Prize at ICCV 2023 and Koenderink Prize at ECCV 2024.

Email  /  CV  /  Google Scholar  /  Twitter

I work on computer vision, machine learning, and generative AI. Particularly, I am recently interested in generative AI in 3D. Below are recent and selected publications.

Revisit ResNets with modern scaling and training strategies, showing ResNets are still competitive against modern model architectures.

Apply pseudo labeling to Harness knowledge in multiple datasets/tasks to train one general vision model, achieving competitive results to SoTA on PASCAL, ADE20K, and NYUv2.

Learning a patch-based image-CAD embedding space for retrieval based 3D reconstruction, improving upon our prior work Mask2CAD.

Given an image of an object and a NeRF of that object, you can estimate that object's pose.

Explore a hybrid architecture of CNN and transformer by simply replacing spatial convolutions with self-attention in the final three bottleneck blocks.

Study copy-paste augmentation for instance segmentation and demonstrating SoTA performance on COCO and LVIS datasets.

Compare self-training and pre-training and observe self-training can still improve when pre-training hurts in a region with more labeled data .

Leverage visual pre-training from passive observations to aid fast trail-and-error robot learning. Can learn to pick up new objects in ~10 mins.

Given a single-view image, predict object's 3D shape based on retrieval of CAD models and object pose estimation.

Propose a benchmark and a simple yet effective class-balanced loss for long-tailed image classification.

Drop intermediate features randomly during training to regularize learning, working for image classification, object detection, and semantic segmentation.

Propose Focal Loss to address fg/bg imbalanced issue in dense object detection. Focal Loss has been adopted beyond object detection since its invention.

Implement an efficient deep network to bring back the idea of pyramidal representations for object detection.

Collecting instance segmentation masks of 80 common objects for training object detection models. The dataset was then extended for panoptic segmentation, multi-modal image-text learning, and beyond.

I am a happy user of this template!