GoYaana Travel Chatbot

The problem

GoYaana wanted a conversational front door for trip planning. Users type something vague like "I want a vacation" or specific like "Plan a 3 day trip to Goa on December 15th" and the system either asks the right follow-up questions or generates a complete itinerary with flights, hotels, activities, pricing, and a downloadable PDF. The challenge was making this feel like a single coherent conversation across 118+ destinations while staying inside Lambda and API Gateway constraints.

Architecture

The backend is a single Java 21 Lambda behind API Gateway, deployed with AWS SAM. The frontend is a Python Gradio app running in Docker, sending the full conversation history with every request (no server-side sessions, no DynamoDB). When a request lands, a ConversationRouter prompts Llama 3.3 70B with the conversation so far and asks it to either generate a follow-up question or extract a complete trip spec as JSON. If the spec is complete, the handler hits the GoYaana getPlan API for flights, hotels, activities, and restaurants, then makes a single batched call to Llama 3.1 8B to generate descriptions for all activities at once. The PDF is generated client-side in the Gradio app, not in Lambda, which keeps the response payload small.

Hardest technical decision

Splitting the LLM work across two models. The 70B model is reliable for structured JSON extraction and conversation routing, but slower and rate-limited. The 8B model is fast and cheap, but less consistent on structured outputs. Generating a description per activity individually would blow past the 29-second API Gateway timeout. The decision was to use 70B only for the routing and extraction step (one call per turn), then make a single batched 8B call that generates all activity descriptions for an entire itinerary in one shot. This kept the cold path under budget and the warm path well under it. Stateless backend with conversation history sent on every request was the other call worth flagging. It made the Lambda trivial to scale and deploy, at the cost of slightly larger payloads, which was the right trade for the project scope.

What I'd do differently

Cache the destination list at module scope rather than re-fetching on cold start. The current implementation loads the destination cache during the first invocation after a cold start, which adds 1 to 2 seconds. Loading it as a static field on the handler class would amortise the cost across all warm invocations on a given container. I would also add proper structured logging with request IDs threaded through the conversation, which would have made the production debugging cycle on flaky Groq responses noticeably faster.