Senior Fullstack Data & AI Search Engineer 100% Remote

We are looking for a hands-on Data & AI Search Engineer to design and deliver a production-grade, AI-augmented enterprise search capability for a large international organisation. The engagement covers the full pipeline from raw data ingestion through to AI-generated, grounded answers surfaced via a conversational or search interface. The right candidate combines deep Elasticsearch engineering with practical experience building Retrieval-Augmented Generation (RAG) pipelines and agentic AI workflows. This is an individual contributor role with direct impact on a critical knowledge management platform. Key Responsibilities 1. Data Engineering and Ingestion Design and build scalable ingestion pipelines and connectors from enterprise sources including SharePoint, Liferay, web crawls, Data Lakes, and corporate systems into Elasticsearch or equivalent search indexes.Support batch, incremental, and near-real-time indexing; implement change tracking, version management, source provenance, access permission mapping, and deletion event handling to keep the index accurate.Build document conversion pipelines for PDF, Word, Excel, PowerPoint, HTML, email, and scanned content; convert to structured Markdown and vector embeddings using tools such as Marker, Docling, or equivalent frameworks.Design semantic chunking strategies (chunk size, overlap, section-aware splitting, heading preservation, table handling) and implement metadata extraction, enrichment, and deduplication during ingestion. 2. Retrieval and Search Develop hybrid search capabilities combining BM25 keyword search, semantic vector search, metadata filtering, and contextual retrieval.Build re-ranking pipelines using embedding models, cross-encoders, or custom ranking logic to improve result relevance.Implement advanced retrieval techniques: query rewriting, query expansion, multi-query retrieval, parent-child retrieval, contextual document embeddings, and contextual compression.Enforce security controls so users retrieve only content they are authorised to access. 3. RAG Pipeline and Agentic Workflows Design and build the end-to-end RAG pipeline connecting enterprise search to large language models for grounded answer generation.Implement agentic workflows where the AI can invoke tools, call enterprise APIs, perform multi-step reasoning, and refine searches iteratively to answer complex queries.Engineer prompt orchestration patterns: system prompts, retrieval prompts, guardrails, context assembly, response formatting, and fallback strategies for low-confidence or ambiguous queries. Technical Requirements Core Search Engineering Deep, hands-on Elasticsearch experience: query DSL, BM25 tuning, function_score, boosting and decay functions, multi-field matching.Index and data modelling: field type selection, custom analyzers and tokenizers per content type (code, prose, structured records, multimedia).Cluster operations: shard strategy, index sizing, reindexing, query latency tuning, and cluster health management.Search evaluation and relevance testing: building ground-truth benchmarks, measuring precision/recall, NDCG, and iterating against them.Experience with Elasticsearch, OpenSearch, Azure AI Search, or equivalent enterprise search platforms. Data and Ingestion Engineering Proven experience building or configuring connectors for SharePoint, Liferay, databases, and Azure Data Lake including incremental sync, CDC, rate limiting, and API edge-case handling.Proficiency in Python; experience with data processing frameworks and document conversion libraries. AI and RAG Engineering Hands-on experience with embedding models, re-ranking models, cross-encoders, prompt engineering, and response grounding techniques.Experience with LLM orchestration frameworks: LangChain, LlamaIndex, Haystack, or equivalent.Practical experience with tool calling, agentic workflows, function calling, and multi-step retrieval.Experience integrating with commercial or open-source LLMs: Azure OpenAI, OpenAI, Anthropic, Google Gemini, Meta Llama, Mistral, or similar. Frontend Working knowledge of React or equivalent front-end technologies to support search UI integration (desirable, not mandatory). Qualifications and Experience First-level university degree in Computer Science, Computer Engineering, Information Systems, or a related discipline.8 years of professional experience in software or data engineering.Minimum 6 years of hands-on experience building enterprise search, AI-powered search .