01Overview
Role Overview
We are looking for an experienced LLM Scientist to research, design, and prototype advanced Generative AI systems. This role is experimentation- and coding-heavy, with a strong focus on Agentic AI design, RAG architecture, LLM fine-tuning, and rapid Python-based prototyping of new GenAI techniques ahead of production handoff.
Key Responsibilities
Agentic AI & GenAI Prototyping
- Design and prototype Agentic AI / Multi-Agent systems using LangGraph, LangChain, and MCP.
- Research and experiment with emerging LLM and Agentic AI techniques.
- Rapidly prototype and validate current GenAI approaches using Python.
- LLM Fine-Tuning & Experimentation
- Fine-tune LLMs using LoRA/QLoRA.
- Conduct experiments comparing models, prompting strategies, and architectures.
- Run controlled experiments, including A/B tests and ablation studies.
- Apply statistical rigor when designing experiments and interpreting results.
- RAG Architecture
- Architect and iterate on RAG pipelines.
- Experiment with chunking strategies, embeddings, hybrid search, and reranking.
- Evaluate retrieval quality and optimize retrieval strategies.
1. Evaluation & Benchmarking
- Build evaluation and benchmarking frameworks for GenAI systems.
- Evaluate retrieval quality, groundedness, hallucination, and relevance.
- Implement observability and tracing using tools such as LangFuse.
- Develop robust evaluation methodologies and avoid metric gaming.
Machine Learning & Statistical Analysis
- Apply statistical concepts including hypothesis testing, confidence intervals, significance testing, and sample-size considerations.
- Design and evaluate supervised, unsupervised, and semi-supervised learning approaches.
- Analyze bias-variance tradeoff, regularization, and overfitting/underfitting.
- Apply classical ML algorithms and evaluation metrics to support GenAI system development.
- Collaboration & Research
- Track emerging LLM and Agentic AI research and assess applicability to business problems.
- .
