Open Source Summit Korea 2026

Agentic search lets you ask in Natural Language and have OpenSearch plan and execute retrieval. The Query Planner re-writes Natural Language to OpenSearch DSL using SOTA LLMs. This works amazingly well but not without important limitations:

1/ The biggest factor is added latency for remote inference. eCommerce Search demands sub-100ms response times. Even the best results when slow lead to search abandonment.
2/ We'll demo overcoming latency by hosting SLMs locally within OpenSearch nodes using vLLM/Ollama. This is faster & cheaper, but SLMs suffer quality decline in query re-writes. We'll explore fine-tuning with domain-specific data—proving fine-tuned SLMs beat SOTA LLMs.
3/ Relevance remains generic even with Agentic Search. We'll show how to improve it using user and business contexts with hybrid search and reranking.

This hands-on talk covers:
1/ Search Latency with Remote Inference
2/ Locally hosting SLMs using vLLM/Ollama
3/ Agentic search with SLM Local Inference
4/ Improving relevance with user and business contexts
5/ Fine-tuning SLMs for domain-specific query re-writing via Axolotl/LLaMA-Factory