Senior AI Engineer

About the role

Building a Data Warehouse that consolidates [property/appraisal data — comps, valuations, market data, inspection records, etc.]. We're hiring an AI Engineer to build the intelligent interface on top of it: a system that lets our team and product query that data in natural language and get answers they can trust.

We currently use Claude for appraiser-facing questions, and while it's strong at general reasoning, it hallucinates on domain-specific appraisal questions because it isn't grounded in our data. Your core mission is to fix that — to make our AI answer appraisal questions accurately, with answers traceable back to real data in our warehouse.

This is a hands-on, applied role for someone who understands modern LLM engineering (retrieval, grounding, evaluation) and knows how to connect a model to a live data source without letting it make things up.

What you'll do

•Build and own the interface between our LLM and the Data Warehouse — enabling natural-language and structured querying of appraisal data.

•Design and implement retrieval / RAG pipelines that ground model responses in real warehouse data, so answers are accurate and citable rather than invented.

•Diagnose and reduce hallucination on appraisal-specific questions through better retrieval, prompt engineering, guardrails, and output validation.

•Build evaluation harnesses and test sets (ideally with input from our appraisers) that measure answer accuracy and catch regressions before they reach users.

•Develop the query layer that translates user questions into safe, correct queries against the warehouse (e.g., natural language → SQL, with validation).

•Collaborate with appraisers and domain experts to encode appraisal logic, terminology, and standards into prompts, retrieval, and evaluation criteria.

•Monitor model performance in production and iterate on quality, latency, and cost.

What we're looking for

•5 years of software or ML engineering experience, with recent hands-on work on LLM-powered applications.

•Practical experience building RAG / retrieval-augmented systems — embeddings, vector search, chunking, and grounding strategies.

•Strong understanding of why LLMs hallucinate and proven techniques to reduce it (grounding, prompt design, guardrails, structured output, evaluation).

•Solid ML fundamentals — you understand how models are trained and evaluated, even if this role is more applied than research.

•Experience working with a data warehouse in Azure (Microsoft Fabric, Azure Synapse, Azure SQL, or Databricks on Azure) and writing efficient SQL.

•Hands-on experience with the Azure AI / data stack — e.g. Azure OpenAI Service, Azure AI Search (vector search), and deploying services on Azure.

•Proficiency in Python and/or C#/.NET, and comfort building APIs / backend services.

•Experience integrating LLM APIs into production systems — we run Claude (Anthropic) and Azure OpenAI, so familiarity with either (or open-source models) is valuable.

•Ability to build evals and think rigorously about correctness — not just "does it sound right," but "is it right."

Nice to have

•Experience in a regulated or domain-heavy field where accuracy matters (real estate, finance, insurance, legal, healthcare).

•Familiarity with real estate appraisal or valuation concepts (comps, adjustments, USPAP, market analysis).

•Experience fine-tuning open-source models, or knowing when fine-tuning is (and isn't) the right tool.

•Familiarity with natural-language-to-SQL techniques and their failure modes.

•Experience with orchestration/tooling frameworks (LangChain, LlamaIndex, DSPy, or equivalent) — and the judgment to know when to skip them.

•MLOps / observability experience for LLM systems (tracing, monitoring, cost control).

Our environment

•Cloud: Microsoft Azure (we're a Microsoft shop end to end)

•LLM: Claude (Anthropic) and Azure OpenAI — you'll pick the right model for each job

•Data: [Microsoft Fabric / Azure Synapse / Azure SQL] warehouse currently being built

•AI tooling: [Azure AI Search, Azure AI Foundry, etc.]

•Stack: [Python, C#/.NET, your API framework]