01Responsibilities
Translate business needs into clear functional requirements and prompt design documentation, acceptance criteria, and test cases
Interact with business stakeholders to understand automation uses cases and triage based on fit for purpose AI low code/no code as well as enterprise AI implementations
Write, version, and optimize prompts (system, user, tool-calls) to achieve required functionality across use cases
Conduct development in Microsoft Power Automate, Power Apps, Copilot Studio, Copilot Cowork, Azure AI Foundry, Azure Content Understanding, Azure App Service, Github Copilot, Python, SQL, Java, RAG pipelines, SODA framework
Design robust prompt patterns and guardrails for reliability, consistency, and compliance (e.g., role prompting, chain-of-thought redaction, constrained generation)
Diagnose error modes (hallucination, drift, formatting issues); implement mitigation strategies and continuous improvement cycles.
Maintain strong relationships and provide regular status updates; set expectations on AI capabilities, constraints, and risk
Create and maintain clear functional specifications, prompt playbooks, technical documentation, and runbooks
Prepare stakeholder-friendly summaries of findings, performance results, risks, and recommendations.
Contribute to standards for prompt governance, versioning, and reuse across the enterprise.
Adhere to data privacy, confidentiality and regulatory requirements relevant to insurance and risk management.
Apply responsible AI principles (bias awareness, explainability, auditability) and participate in model risk assessments.
Measure and improve quality using metrics such as precision/recall, accuracy, consistency, latency, and cost.
Develop evaluation plans and golden datasets; test AI functionality and validate outputs against requirements and source documents.
Ability to read/modify Python scripts, debug minor issues, and write clear, maintainable code. Familiarity with version control and basic CI/CD or repeatable execution (e.g., scripts, notebooks, pipelines).
Competencies:
Prompt Engineering & Generative AI: Expertise in Generative AI capabilities and tools; ability to create and test AI prompts to produce output based on requirements, implement LLM workflows (prompt engineering, RAG, agents, tools) using frameworks such as Semantic Kernel or LangChain, design and maintain vector stores, embeddings pipelines, and content moderation/guardrails to protect sensitive data (PII/PHI), evaluate models (quality, bias, latency, cost) and establish prompt/version governance and A/B testing.
Data and MLOps: Build and operate data/feature pipelines and model services using Azure Databricks/ML, MLflow, and CI/CD (GitHub Actions/Azure DevOps), containerize and deploy on AKS or serverless where appropriate; implement monitoring for drift, accuracy, latency, and spend, collaborate with Data Engineering on data quality, lineage, cataloging, and secure access controls consistent with enterprise policy.
Business Acumen: Ability to understand Gallagher business objectives, processes, and systems; translate .
