Part of my university seminar on AI and extended reality, and my hands-on way into robotics: I collected demonstrations and trained an Action Chunking with Transformers (ACT) policy to pick up a block and place it in a box.

61 episodes · 17,324 frames · front and wrist cameras.
Published the cleaned dataset and a policy trained for 50,000 steps.

LeRobotACTImitation learning

Project & recording Policy Dataset

VOICE × MEMORY × ACTIONLANGGRAPH

VOID—AI

AI AGENTS 2025

VOID-AI: an assistant that remembers.

A voice-controlled assistant with persistent context and agents that turn conversations into actions across everyday tools.

Episodic, semantic, and procedural memory layered over an LLM core.
Whisper STT + Kokoro TTS; orchestration across six external tools.

Hierarchical memoryVoice AIAgent orchestration

Architecture & project Source code

REINFORCEMENT LEARNING OCT–DEC 2025

Minesweeper: PPO vs. PPO+RND.

Compared standard PPO with curiosity-driven exploration in a pixel-based 6×6 Minesweeper environment. Implemented invalid-action masking and shaped rewards, and documented learning curves and failure modes.

PPORandom Network DistillationPyTorch

Results & experiment Study notes

BRAIN-COMPUTER INTERFACE JAN–MAR 2024

SyncTew: brain signals to game control.

Built an EEG-to-control pipeline: BioAmp acquisition, preprocessing, SVM classification of attention and relaxation, and live game input.

EEGSVMSignal processing

Pipeline & project Watch demo Source code

DEEP REINFORCEMENT LEARNING · HUGGING FACE COURSE 2024–2025

15 agents, from Q-tables to self-play.

Worked through the Hugging Face Deep RL course: implemented Q-learning, REINFORCE, and PPO myself, and trained agents with Stable-Baselines3, ML-Agents, and Sample Factory. Every agent is published with its evaluation.

PPODQNREINFORCEA2CMulti-agent RL

Selected published agents and their reported evaluation results
Agent	Environment	Mean reward
PPO, my implementation	LunarLander	288.9 ± 19.0
REINFORCE, my implementation	CartPole	500 ± 0
DQN	Atari Space Invaders	676.5 ± 194.5
PPO + RND curiosity	Pyramids, sparse reward	—
MA-POCA self-play	SoccerTwos, 2 vs. 2	—
A2C	Panda robot-arm reach	−0.96 ± 1.65

All agents on Hugging Face Study notes

DATA & MACHINE LEARNING 2023

Customer Segmentation: from receipts to personas.

Turning retail data into customer personas with RFM analysis, K-Means clustering, and an interactive Plotly dashboard.

Scikit-learnPlotlyClustering

Live demo Source code

INFORMATION SYNTHESIS 2024

InfoSynth: web research inside your spreadsheet.

Bringing web research into your workflow. Extract and synthesize information directly from CSV files and Google Sheets.

GroqSerpAPIStreamlit

Watch demo Source code

VISION & ROBOTICS 2023

AI-Powered Gimbal: a camera that follows.

A self-balancing camera system combining object detection, Arduino-based PID control, and kinematics to track moving subjects.

Computer visionC++Arduino

Watch demo Source code

CONVERSATIONAL AI 2023

HealthBot: health answers on WhatsApp.

A health information chatbot that brings language models and web information retrieval to a familiar WhatsApp interface.

LangChainTwilioLLMs

Live demo Source code

More experiments live on GitHub

02 / RESEARCH & EXPERIMENTS

What I’m working on.

Current projects, published experiments,
and the notes behind them.

ROBOT LEARNING

Ongoing

SO-101, from data to policy

Part of my seminar topic, Teaching Embodied AI through XR: Immersive Robot Training and Teleoperation, where I’ll bring a VR headset into collecting training data. The demonstration dataset and an ACT policy are published. Next: a structured real-robot evaluation, then a loop where the robot learns from my corrections instead of only from fresh demonstrations.

The setup & where it’s going

EXPERIMENT REPORT

Can curiosity make a better agent?

Benchmarked PPO against PPO+RND on pixel-based 6×6 Minesweeper, with invalid-action masking and shaped rewards.

8× fewer steps to beat plain PPO’s best win rate

PPO+RND reached a 5.0% win rate after 100k steps; plain PPO reached 3.2% after 800k (6×6 board, 6 mines, 2 evaluation seeds). Setup, limitations, and learning curves are in the report.

Read the experiment

STUDY NOTES

Reinforcement learning, from the ground up

Working through Sutton & Barto with practical experiments.

DEEP DIVE

Understanding Proximal Policy Optimization

Policy updates, clipping, and learning to act.

FOUNDATIONS

Thinking in Bellman equations

Connecting immediate rewards to future value.

03 / A LITTLE ABOUT ME

Curiosity is the
common thread.

Umang BansalPaderborn, Germany

I’m an MSc Computer Science student at Paderborn University. I’m interested in the space between understanding the world and acting in it.

That curiosity takes me from multimodal RAG and AI agents to reinforcement learning, neural signals, and robotics. I like turning a theoretical idea into something I can test, use, and improve.

The full story, in my résumé

EXPERIENCE

FEB 2025 · REMOTE

GenAI Intern

Stealth SaaS Startup

Built multimodal RAG with LLMs and vision encoders for Q&A across slides, charts, tables, and images. Aligned live transcripts with meeting agendas and built pre-meeting agents with LangChain.

EDUCATION

2026 — PRESENT

MSc Computer Science

Paderborn University · Germany

Focus: intelligent systems, machine learning, and embodied AI.

JUL 2026

Foundation Models Meet Robotics

IEEE CIS & COLOURS Summer School · Paderborn

2021 — 2025

BTech Computer Science

Vellore Institute of Technology, Bhopal

CGPA 8.82 / 10

MY TOOLKIT

Tools to think with.
Tools to build with.

Intelligence

PyTorch · TensorFlow · Scikit-learn · LangChain · NLP · Computer vision

Engineering

Python · Java · SQL · Git · Docker · HTML & CSS

Physical systems

Arduino · Raspberry Pi · EEG · Robotics

Human languages

English — Fluent · Hindi — Native
German — Learning (A1)

CONTINUING TO LEARN

Deep Reinforcement LearningHugging Face · 2025

Cloud Digital LeaderGoogle Cloud · 2024

Data Science ExternshipSmartInternz · 2023

Deep Learning SpecializationDeepLearning.AI · 2023

OPEN TO WHAT’S NEXT

Good things start
with a conversation.

Have an interesting AI problem, a research idea, or a role in mind?
I’m open to Werkstudent, internship, and thesis opportunities.

umangbansal32@gmail.com

Perception.
Reasoning.
Action.
And a little curiosity.

Umang Bansal.

Built, trained, and tested.