Self-Supervised & Semi-Supervised Contrastive Learning for Visual Representations
A research-grade implementation of SimCLR and SupCon on CIFAR-10, featuring a 42-experiment augmentation ablation suite, shortcut-learning analysis, linear evaluation protocol, and a live real-time visual similarity search engine.

Try the visual search engine — no installation required:

https://huggingface.co/spaces/mahmoudalyosify/SimCLR-Visual-Search-Engine

Upload any image or pick a random CIFAR-10 test image and retrieve the 5 most semantically similar images in under 5 ms using the ONNX-exported ResNet-50 encoder + FAISS index.

This repository implements SimCLR (Chen et al., ICML 2020) and Supervised Contrastive Learning (SupCon) (Khosla et al., NeurIPS 2020) for self-supervised and semi-supervised visual representation learning on CIFAR-10.

Key Contributions:

• 42-Experiment Augmentation Ablation Suite — Systematic sweep over spatial, photometric, and structural transforms, identifying Exp 41 (Crop + Flip + GaussianBlur + ColorJitter) as the optimal configuration (NT-Xent = 4.9906, linear probe 84.30%).
• Color-Jitter Shortcut Mitigation Study — 5 paired 200-epoch ResNet-50 experiments demonstrating that NT-Xent loss is an unreliable proxy for representation quality. Color Jitter alone yields a mean +18.10 pp accuracy gain with zero architectural changes.
• Architecture — ResNet-50 modified with a custom small-image stem (Conv2d 3×3, stride 1, no max-pooling) following Chen et al. Appendix B.9, preserving spatial resolution for 32×32 inputs.
• Contrastive Frameworks:
  • Unsupervised (SimCLR): Two-layer MLP projection head + NT-Xent loss (τ = 0.5).
  • Semi-supervised (SupCon): Supervised Contrastive Loss (τ = 0.1) on a stratified 10% subset (5,000 labeled samples). Six iterative debugging runs required to stabilize training — see SupCon Diagnostics below.
• Real-Time Visual Search Engine — ResNet-50 encoder exported to ONNX (max deviation 3.81×10⁻⁶ from PyTorch reference) paired with FAISS IndexFlatIP for exact sub-5 ms similarity retrieval over 10,000 CIFAR-10 test embeddings.
• Interactive Web GUI — Streamlit application with real-time visual similarity search and ablation dashboard.

Our SimCLR encoder, trained exclusively on 50,000 unlabeled 32×32 CIFAR-10 images, reaches 95% of CLIP ViT-B/32 zero-shot accuracy while using 8,000× less training data.

The 8-configuration ablation reveals a clear three-tier loss hierarchy at 20 epochs (ResNet-18, BS=128):

Higher absolute NT-Xent loss reflects a harder contrastive task — not worse learning. Exp 8 (Full SimCLR pipeline) achieves the largest loss reduction (ΔL = 0.916) and the most semantically structured t-SNE embeddings.

Five structural pipelines were trained twice at full scale (ResNet-50, 200 epochs, BS=512) — once without and once with RandomApply([ColorJitter(0.8, 0.8, 0.8, 0.2)], p=0.8):

Critical finding: All five base pipelines converge to nearly identical NT-Xent losses (≈ 4.95–4.96) yet span a 31.87 pp accuracy range (34.40%–66.27%). Without Color Jitter, encoders exploit color-histogram shortcuts rather than learning semantic structure. Color Jitter eliminates this shortcut, raising initial losses by ~1 nat and forcing structural reasoning.

SupCon-10% (75.20%) falls below SimCLR (84.30%) despite having label information. The cause is reduced semantic diversity: restricting pretraining to 5,000 images limits intra-class variation, making it harder to learn class-consistent invariances. The batch composition (class positives per batch) remains approximately unchanged due to the fixed batch size and balanced CIFAR-10 distribution — the loss floor rises because the representation manifold is harder to align, not because of fewer positives.

ResNet-50 with CIFAR-10 stem (Chen et al. Appendix B.9):

Input (32×32×3)
    │
    ├── [Augmentation t  ~ T] → x_i ─┐
    └── [Augmentation t' ~ T] → x_j ─┤
                                     │
                        Encoder f(·) — ResNet-50
                        ┌──────────────────────────┐
                        │ Conv2d(3→64, 3×3, s=1)   │  ← replaces 7×7 stride-2 stem
                        │ BatchNorm → ReLU         │
                        │ Identity (no MaxPool)    │  ← removed per Appendix B.9
                        │ Layer1 → 2 → 3 → 4       │
                        │ AvgPool                  │
                        └──────────────────────────┘
                                      │  h ∈ ℝ²⁰⁴⁸
                                      │
                        Projection Head g(·) — MLP
                        ┌──────────────────────────┐
                        │ Linear(2048 → 2048)      │
                        │ BatchNorm → ReLU         │  ← BN in hidden layer ONLY
                        │ Linear(2048 → 128)       │  ← no final BN (causes collapse)
                        └──────────────────────────┘
                                      │  z ∈ ℝ¹²⁸ → ℓ₂-norm
                                      │
                    NT-Xent Loss (τ=0.5, 1,023 in-batch negatives)

    ── After pretraining: discard g(·), freeze f(·) ──
                                      │  h ∈ ℝ²⁰⁴⁸
                                      │
                        nn.Linear(2048 → 10) → 84.30% Top-1

Critical implementation detail: Applying BatchNorm to the projection output and subsequently applying ℓ₂-normalization constitutes double normalization that collapses all embeddings to a single point (confirmed empirically in SupCon runs 4–5). BatchNorm must be placed in the hidden layer only.

Six iterative runs were required to stabilize SupCon training. Each failure mode independently prevented any learning.

Root causes:

• Run 1 (NaN): exp(1.0/0.1) = e¹⁰ ≈ 22,026; FP16 overflows at e^11.09 ≈ 65,504. Solution: disable AMP entirely (FP32 throughout).
• Runs 2–3 (L = 2.31 = log(10)): AdamW lr=0.5 is ~167× the typical maximum; gradient updates destroy all learned structure from step 1, leaving the model at the uniform-class fixed point.
• Runs 4–5 (L = 6.93 = log(1023)): Final BatchNorm + subsequent ℓ₂-normalization = double normalization, over-constraining the representation manifold.
• Run 6: Remove final BN + τ=0.1 + per-step SGD warmup from lr=0.01 → resolves all failure modes simultaneously.

pip install streamlit onnxruntime faiss-cpu numpy pillow torch torchvision matplotlib scikit-learn
streamlit run app.py

python src/train_master.py \
  --epochs 200 \
  --batch_size 512 \
  --backbone resnet50 \
  --exp_id 41

python train_supcon.py \
  --epochs 100 \
  --batch_size 512 \
  --fraction 0.1 \
  --learning_rate 0.05

Note — SupCon uses FP32: with τ=0.1, exp(z·z/τ) can reach e¹⁰ ≈ 22,026, which approaches the FP16 overflow threshold of 65,504.

SimCLR-Vision-SSL/
├── deployment/                     # Production assets for Streamlit GUI & inference
│   ├── app.py                      # Streamlit visual search engine & ablation explorer
│   ├── build_faiss.py              # Extracts 2048-d features and builds FAISS index
│   ├── export_onnx.py              # Exports Exp 41 ResNet-50 weights to ONNX format
│   ├── simclr_encoder_exp41.onnx   # ONNX encoder (~89.6 MB, max deviation: 3.81e-6)
│   ├── cifar10_index.faiss         # FAISS IndexFlatIP (10,000 vectors)
│   ├── metadata.json               # Vector ID → class name / image path mapping
│   ├── test_images/                # 10,000 reference CIFAR-10 PNG images
│   └── simclr_encoder_exp41.onnx.data
│
├── src/Final Work/                 # Primary codebase (Experiments 1–42)
│   ├── augmentations.py            # 42 augmentation pipelines (Exps 1–42)
│   ├── dataset.py                  # DataLoader builders for training & evaluation
│   ├── loss.py                     # NT-Xent loss implementation
│   ├── model.py                    # ResNet-50/18 encoder with custom CIFAR-10 stem
│   ├── train_master.py             # Master contrastive training loop (AMP, W&B logging)
│   ├── run_beast_experiments.py    # BEAST hyperparameter sweep orchestrator
│   ├── run_top_5_production.py     # Runs top 5 configurations for production
│   ├── run_final_ablations_18_may.py # Final ablation experiment suite
│   ├── plot_ablation.py            # Visualization & plotting for ablations
│   ├── run_config.json             # Configuration template for runs
│   ├── configs/                    # Augmentation YAML configurations
│   ├── supcon/                     # Supervised Contrastive pretraining modules
│   │   ├── train_supcon.py         # SupCon Stage 1 pretraining orchestrator
│   │   ├── loss_supcon.py          # Supervised Contrastive Loss implementation
│   │   └── dataset_subset.py       # Stratified data sampler (10% / 100% labels)
│   └── Evaluation/                 # Post-training evaluation & analysis
│       ├── evaluate_all_probes.py  # Linear probe evaluation across all experiments
│       ├── evaluate_and_plot_beast.py # BEAST sweep evaluation & visualization
│       ├── evaluate_clip_zeroshot.py  # Zero-shot CLIP baseline comparison
│       ├── confusion_matrix/       # Confusion matrix outputs
│       └── figures/                # Generated evaluation figures
│
├── outputs/                        # Training checkpoints & logs (Experiments 1–8)
│   ├── Experiment 1–8/
│   │   ├── logs/
│   │   │   └── training_log.csv    # Loss / accuracy per epoch
│   │   └── plots/                  # Per-experiment visualizations
│
├── notebooks/                      # Interactive analysis & visualization
│   ├── visualize-pairs.ipynb       # Augmentation pair visualization
│   ├── Linear evaluation notebook/
│   │   ├── supervised-baseline-and-linear-probe-evaluation.ipynb
│   │   └── logs/                   # Baseline & probe training histories
│
├── configs/                        # Top-level configuration files
│   └── augmentation.yaml           # Global augmentation configuration
│
├── figures/                        # Publication-ready figures
│
├── requirements.txt                # Full Python dependency list
├── run_ablations.py                # Runs all ablation configs sequentially
├── README.md                       # Project overview & usage guide
├── LOG.md                          # Development log with AI usage disclosure
├── LICENSE                         # Project license
└── .gitignore                      # Git ignore rules

All 42 experiments tracked on Weights & Biases with per-epoch NT-Xent loss, learning rate schedule, GPU utilization, wall-clock time, and t-SNE projections at checkpoints. Fixed seed 42 across Python, NumPy, and PyTorch ensures bitwise-reproducible results.

View W&B Dashboard

%%{init: {'theme': 'default', 'themeVariables': { 'background': '#ffffff', 'primaryColor': '#ebf5fb', 'primaryBorderColor': '#2980b9', 'textColor': '#000000', 'lineColor': '#7f8c8d'}}}%%
gantt
    title SimCLR & SupCon Project Structured Timeline
    dateFormat  YYYY-MM-DD
    axisFormat  %d/%m
    tickInterval 7day
    todayMarker off

    section Week 1 (Setup)
    Repo Init & CIFAR-10 Loader (N.N & M.I)    :w1_1, 2026-05-07, 3d
    Supervised ResNet-50 Baseline (N.N & M.I)  :w1_2, 2026-05-10, 4d
    ResNet-18/50 Stem Modifications (M.A)      :crit, w1_3, 2026-05-07, 7d

    section Week 2 (Core Head)
    Stochastic Augmentation Module (N.N & M.A) :crit, w2_1, 2026-05-14, 4d
    MLP Head & NT-Xent Loss (N.N & M.A)        :w2_2, 2026-05-14, 4d
    20-epoch Ablations & Validation (M.I & M.A):w2_3, 2026-05-18, 3d
    MIDTERM SUBMISSION                         :milestone, m1, 2026-05-21, 0d

    section Week 3 (SimCLR)
    200-epoch SimCLR Final Run (M.A)           :crit, active, w3_1, 2026-05-21, 7d
    Augmentation Difficulty Analysis (N.N)     :w3_2, 2026-05-21, 4d
    SimCLR Linear Probe Eval (M.I)             :w3_3, 2026-05-25, 3d

    section Week 4 (SupCon)
    SupCon Adaptation & Fixes (M.A)            :w4_1, 2026-05-28, 7d
    Shortcut Visualization Fig (N.N)           :w4_2, 2026-05-28, 4d
    SupCon Linear Probe Eval (M.I)             :w4_3, 2026-06-01, 3d

    section Week 5 (Deployment)
    ONNX Export & Streamlit (M.A)              :w5_1, 2026-06-04, 7d
    Class Diagnostics & Conf. Matrix (M.I)     :w5_2, 2026-06-04, 4d
    Presentation Recording (All)               :w5_3, 2026-06-08, 3d
    IEEE Manuscript & LOG.md (All)             :crit, active, w5_4, 2026-06-08, 3d
    FINAL PROJECT SUBMISSION                   :milestone, m2, 2026-06-11, 0d

Course: CISC 867 Deep Learning, Queen's University, Spring 2026

Hardware: NVIDIA RTX 5000 Ada Generation (34.4 GB VRAM)

@inproceedings{chen2020simple,
  title     = {A Simple Framework for Contrastive Learning of Visual Representations},
  author    = {Chen, Ting and Kornblith, Simon and Norouzi, Mohammad and Hinton, Geoffrey},
  booktitle = {ICML},
  year      = {2020}
}

@inproceedings{khosla2020supervised,
  title     = {Supervised Contrastive Learning},
  author    = {Khosla, Priyank and Teterwak, Piotr and Wang, Chen and others},
  booktitle = {NeurIPS},
  volume    = {33},
  year      = {2020}
}

Licensed under the MIT License