TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Models

Aditya Chinchure*, Pushkar Shukla*, Gaurav Bhatt, Kiri Salij, Kartik Hosanagar, Leonid Sigal, Matthew Turk (*equal contribution)

We propose a general approach to study and quantify a broad spectrum of biases, for any TTI model and for any prompt, using counterfactual reasoning. Unlike other works that evaluate generated images on a predefined set of bias axes, our approach automatically identifies potential biases that might be relevant to the given prompt, and measures those biases.

From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models

Mehar Bhatia, Sahithya Ravi*, Aditya Chinchure*, Eunjeong Hwang, Vered Shwartz (*equal contribution)

We introduce the GlobalRG benchmark, comprising two challenging tasks, retrieval across universals and cultural visual grounding, to test VL models' cultural inclusivity.

VLC-BERT: Visual Question Answering with Contextualized Commonsense Knowledge

Sahithya Ravi*, Aditya Chinchure*, Leonid Sigal, Renjie Liao, Vered Shwartz (*equal contribution)

We present a new Vision-Language-Commonsense transformer model, VLC-BERT, that incorporates contextualized knowledge using Commonsense Transformer (COMET) to solve Visual Question Answering (VQA) tasks that require commonsense reasoning.

DE-TensoRF: Data-efficient and fast NeRFs

Developed DE-TensoRF, a model that can render 3D objects with as few as 3 images, and in under 15 min on a single GPU. We achieved the highest grade in our class, and led to collaboration efforts with Dr. Helge Rhodin’s research group.

VisualCOMET+: Visual Commonsense Generation & its incorporation into a Multimodal Topic Modeling algorithm

Developed an extension to VisualCOMET to generate general-purpose commonsense knowledge from images. Showed improvements on coherence and diversity scores of a novel topic modelling algorithm using the generated knowledge

VL-BERT-Graph: Graph-enhanced Transformers for Referring Expressions Comprehension

Incorporated Graph Neural Networks in a visual-linguistic Transformer

Investigating extensions to VLC-BERT and comparing it with GPT-3

This project extends VLC-BERT with pointer generator networks and object detection models. Furthermore, we compare the performance of VLC-BERT with GPT-3 on the OK-VQA dataset.

Learning faster Genetic Algorithms with dynamic mutation power

This project introduces a modification to the GA algorithm to introduce dynamic mutation power, to solve the Lunar Lander evironment on OpenAI Gym in 30 generations.

A Summary of Recent Text Summarization Techniques

In this project paper, we surveyed text summarization models by evaluating existing extractive and abstractive models. We studied the metrics and datasets used to evaluate the latest models and evaluated upcoming abstractive techniques. Finally, we highlighted future pathways for text summarization and suggested areas for improvement

Universal Machine Learning API

A powerful Python API template, built on Flask, for plug-and-play use with machine learning models.