AI Bias Detection Tool

Identifying dataset bias using fine-tuned language models

About the Project

Building tools to detect and prevent bias in AI training data

The Problem
LLMs learn from biased data sources like social media and Reddit, leading to stereotypical associations (e.g., "Doctor" → male, "Nurse" → female).
Our Solution
A fine-tuned BERT model trained on 28,000 Reddit comments to detect bias across gender, race, orientation, and religion categories.
The Impact
Help teams identify biased data early in the development process, creating fairer and more inclusive AI systems.

Model Performance

83%
F1 Score
80%
Accuracy
88%
Recall
28K
Training Samples
Dataset Distribution

After data cleaning and preprocessing:

6,573
Biased Samples
4,899
Non-Biased Samples
Data Sources
RedditBias dataset covering gender, race, orientation, and religious bias categories
Methodology
Fine-tuned BERT with hyperparameter optimization, early stopping, and regularization techniques
Improvement
Improved F1 score from 0.74 (baseline) to 0.83 through systematic fine-tuning

Meet Our Team

AI Studio Final Project - Meta 1B

JC
Jay Chan
jayc10@uci.edu
RL
Rianna Lei
rxlei@calpoly.edu
WF
Wen Fan
wqfan05@gmail.com
HK
Hala Khattab
hala.khattaab@gmail.com
MC
Mia Carter
Mia.L.Carter04@gmail.com
LS
Leo Siu
leonardo.siu.dev@gmail.com
RD
Rhythm Dawar
rythamdawar30@gmail.com

Special thanks to our advisors: Candace Ross, Megan Ung, Rajshri Jain, and Break Through Tech