01Overview
At Lilly, we unite caring with discovery to make life better for people around the world. We are a global healthcare leader headquartered in Indianapolis, Indiana. Our employees around the world work to discover and bring life-changing medicines to those who need them, improve the understanding and management of disease, and give back to our communities through philanthropy and volunteerism. We give our best effort to our work, and we put people first. Were looking for people who are determined to make life better for people around the world.
We are a global healthcare leader headquartered in Indianapolis, Indiana. Our 42,000+ employees around the world work to discover and bring life-changing medicines to those who need them, improve the understanding and management of disease, and give back to our communities through philanthropy and volunteerism. We give our best effort to our work, and we put people first. We're looking for people who are determined to make life better for people around the world. Come build advanced software capabilities to accelerate our digital transformation and support Lilly's evolution to be the leader in Pharma-tech!
THE ROLE
The Software Product Engineering (SPE) organization is actively looking for a motivated Principal Engineer QE (R3) who is deeply technical and passionate about engineering quality at scale. This is a core individual contributor role for a hands-on quality engineer with deep expertise in building evaluation frameworks for AI platforms including chat and voice bot applications alongside robust test automation frameworks for API, web, and mobile testing. Are you a quality-engineering craftsperson who thrives on solving complex technical problems Do you have a passion for AI quality validation, framework architecture, and continuous innovation Do you want to shape Lilly's engineering direction as a leader in Pharma-tech If so, please apply. WHAT YOU'LL BE DOING
In this role, you will be at the technical forefront of quality engineering designing and building evaluation frameworks for AI-based platforms including LLM-powered chat and voice bots, and developing scalable automation frameworks for API, web, and mobile applications. You will work hands-on with Python, Java, and JavaScript/TypeScript to build production-grade test infrastructure, integrate with CI/CD pipelines, and apply advanced AI testing methodologies. You will also contribute to performance testing, cloud-based test execution, and the continuous improvement of quality practices across a global cross-functional team. KEY RESPONSIBILITIES
AI Platform & Evaluation Framework Engineering
Designs and builds end-to-end evaluation frameworks for AI platforms including LLM-based chat bots and voice bots , covering accuracy, relevance, hallucination detection, latency, and response quality. Implements AI model evaluation pipelines using RAGAS, DeepEval, and LangChain to benchmark and validate LLM outputs against ground truth. Develops test strategies for conversational AI validating intent recognition, slot filling, dialogue flow, fallback handling, and multi-turn context retention. Builds voice bot quality validation covering speech recognition accuracy, TTS quality, call flow logic, and DTMF handling. Defines quality metrics for AI systems including precision, recall, F1-score, BLEU/ROUGE scores, and semantic similarity thresholds. Collaborates with AI/ML engineers and data scientists to integrate evaluation frameworks into model training and deployment pipelines. Test Automation Framework Development
Architects and builds robust, scalable, and maintainable test automation frameworks for API (REST/SOAP), web, and mobile applications using pytest, Selenium, WebdriverIO, and Appium. Designs framework components including page object models, data-driven layers, reporting modules, parallel execution engines, and retry mechanisms. Develops API automation suites covering contract testing, schema validation, authentication flows, and end-to-end service integration testing. Implements mobile test automation for both iOS and Android platforms, covering native, hybrid, and responsive web applications. Builds reusable test libraries and utility modules in Python, Java, and JavaScript/TypeScript to accelerate framework adoption across teams. Integrates test frameworks with CI/CD pipelines using GitHub Actions, Jenkins, or Bamboo for continuous test execution and reporting. Technology Leadership & Engineering Excellence
Serves as a key technical contributor and subject matter expert in quality engineering technologies, automation architectures, and testing methodologies. Demonstrates deep understanding of software design patterns, SOLID principles, and architectural best practices applied to test automation. Proactively adopts and evaluates new programming languages and tools including Go, Rust, JavaScript, Python, and TypeScript to improve .
