Hi, my name is Sinan Tang and I'm a Senior Software Engineer on the Zalando Assistant team. Our team is creating a knowledgeable fashion advisor to guide you through your shopping experience, much like an expert in a retail store. With Zalando Assistant, customers can navigate through Zalando's range using their own words or fashion terms, making the process more intuitive and natural. For example, if a customer asks, "What should I wear to a wedding in Santorini in July?" Zalando Assistant is able to understand that this is a formal event, what the weather is like in Santorini in July, and therefore provide an explanation with clothing recommendations based on this input. As a Senior Software Engineer on the Zalando Assistant team, I work at the intersection of backend development and machine learning technologies, which is why I was invited to speak about my work and the hot topics of AI engineering and LLMs at the recent Women+ in Data and AI Festival.
My role in the Zalando Assistant team
Every day, I work closely with colleagues from different functions - as a team we tackle complex customer challenges. It's rewarding work that combines technical problem solving with creative teamwork, and we have a lot of fun doing it! We recently launched the Zalando Assistant for all Zalando customers across Europe. I led much of the backend and offline evaluation work that enabled its multilingual capabilities, allowing the Assistant to support 20 languages and serve customers in different markets. It's fascinating to see how users from different countries and regions interact with the assistant in unique ways.
Presenting at the Women+ in Data and AI Festival
My talk at the Women+ in Data and AI Festival, titled 'Building Applications with LLMs', focused on taking AI applications from prototype to production. I explored what the buzzword 'AI engineering' really means, how it differs from traditional machine learning engineering, and the unique benefits and challenges it brings. I also highlighted two essential skills for AI engineers: timely engineering and robust evaluation techniques. I used Zalando Assistant as a case study to show how these techniques enabled us to deliver the product to our first customers within weeks and scale it across all markets in which Zalando operates.
An exit strategy for LLM hallucination
Productionisation is a well-known challenge in AI engineering. It involves several layers of complexity. For example, LLMs can generate responses to almost any input, even if it contains incorrect information or contradicts prompt instructions - a problem known as 'LLM hallucination'. In the past, we have seen LLMs generate non-existent product SKUs, leading to confusing errors. One way to mitigate hallucination is to structure the prompt to allow the model an "out" so that it doesn't force a response if there are no products that match the query.
Collaborating with the machine
Don't underestimate the power of a human-in-the-loop approach! AI is a tool, and it's up to us to use it responsibly. Our team holds regular annotation sessions, where team members annotate real customer conversations together. This helps us to identify bugs and issues in production, and to cross-check the quality of LLM scores.
We also don't shy away from using traditional engineering techniques when they suit your needs. We still rely on non-AI classifiers in the product because they're fast and effective. Ultimately, our goal is to deliver real customer value, not just to use AI for its own sake.
