Building retrieval and reasoning for flavour and fragrance data

The problem

Flavourists and perfumers work with messy, specialist knowledge: molecules, sensory descriptors, safety data, usage limits, source materials and supplier information.

Generic AI knows some of this, but not enough to be trusted on its own. The system needed to retrieve relevant domain data, reason over it, and avoid pretending it knew more than it did.

The constraints

The system needed to work with structured and semi-structured chemistry data, stay cost-aware, and support specialist questions without turning into a slow or expensive research project.

The data came from multiple sources, with different levels of quality and consistency.

What I built

I built a retrieval layer using Postgres and pgvector, enriched with public chemistry and flavour sources.

The pipeline expands the user query, retrieves relevant compounds and descriptors, reranks the results, then has the model answer from retrieved context, falling back on its own domain knowledge only where needed.

What broke

Embedding model choice mattered. Chunk shape mattered. Safety data was inconsistent. Some sources looked useful until they created noisy retrieval.

HNSW memory tuning also mattered during migration. None of that shows up in a clean demo, but it matters if the system has to be useful.

What changed

The result was a working retrieval and reasoning system for a specialist domain, built around engineering trade-offs rather than prompt magic.

Specialists can ask questions in their own language — about molecules, descriptors, materials, safety — and get answers grounded in retrieved data rather than generic guesses. When retrieval is thin, the system recognises the gap rather than fabricating.

The work that made this useful was not the model choice. It was the chunking, the embedding evaluation, the reranking, and the data cleanup. The model is small. The pipeline did the work.

Stack

• Postgres
• pgvector
• Python/Node
• AWS Aurora
• Azure Postgres
• Bedrock
• Claude Haiku
• OpenAI embeddings
• Cohere reranking