All work
Data engineering
Facebook Marketplace Data Pipeline
A Python pipeline for collecting, cleaning, validating, and structuring large-scale marketplace data for discrimination research.
Data pipelinesResearch infrastructurePlatform research
ETL pipeline
ingestvalidatenormalisededupeexport
"Rs. 15,000 approx"15000
quality score98%
1.5M+ listings · validated & deduplicated
The problem
Discrimination research on digital marketplaces needs structured, analysis-ready data from unstable public interfaces.
What I built
- Collection and validation scripts for marketplace listings.
- Cleaning steps for structured research tables.
- Quality checks for deduplication and schema consistency.
My role
Research engineering support across scraping, cleaning, validation, and data quality workflows.
Methods
Web automationETL designData validationSchema design
Why it matters
Shows how social research questions can become careful data infrastructure problems.
What it demonstrates
- Data cleaning
- Scraping judgement
- Research workflow design
Outcome
The workflow supported large-scale marketplace research while keeping screenshots and examples anonymised for public presentation.