What I've been learning about LLMs and Generative AI

Hey there! I’m Raika, a Senior UX and Conversation Designer at Amazon. If you’re new here, welcome! You can subscribe to my Secrets to Great UX Design newsletter for weekly insights. I share actionable ways to create great experiences, grow your career and more… for designers and non-designers.

---

I'm back this week and eager to share some of my recent learnings about Large Language Models (LLMs) and generative AI with you.

First, tons of AI news this week, here’s a recap:

• Sam Altman was fired as OpenAI CEO just days ago but is now returning as CEO with mostly new board members

• ChatGPT launched its voice feature to all free users

• The generative AI startup AI21 Labs, known for the generative AI writing and reading assistant Wordtune, added $53M to the earlier $155M funding round

• Microsoft renamed Bing chat to Copilot, expanding Copilot branding and generative AI features

• You.com launched search APIs giving generative AI chatbots real-time web access

• Amazon shut down hundreds of Alexa positions last week

• Amazon unveiled PartyRock an AI app creator that counters OpenAI’s GPT Builder

So many things we could discuss but today I’ll focus on my first impressions of ChatGPT Voice, tips for fine-tuning LLMs, and design considerations for generative AI experiences.

ChatGPT Voice

I. First Impressions of ChatGPT Voice

Simply put, I’m impressed. The voice feature is seamless, the tone makes the longer more conversational responses enjoyable, and details like the response “it seems like your question got cut off…” make it feel very human-like.

1. The voices offered (Juniper, Sky, Cove, etc.) appear to have unique personalities and communication styles. Their short intros about “teaming up”, “working together”, and “hitting the ground running” bring a positive energy and new promise to being an assistant.

2. The ability to keep the app running in the background makes ChatGPT a strong competitor to Siri on iPhones. The microphone is still open even when you’re looking at other apps or your phone is locked.

3. The experience is hands-free, however, you must “tap to interrupt”. No “wake word” keeps you tied to your phone for now. Because interrupting is important given the long-form responses when multi-tasking, I’m curious to see how they’ll address this given they don’t own the device.

All in all, I’m excited to keep using the voice feature. I encourage you to give it a try — it totally changes the ChatGPT experience.

🔥 I would love to hear your thoughts, reply to this email or leave a comment!

II. Tips for Fine-tuning (training) LLMs

LLMs like ChatGPT and Bard have shown immense promise for natural language generation. As we’ve seen, with good data and proper training, these models can produce remarkably human-like text.

For a conversation designer like me, that means my role is changing. Instead of writing out all variations of utterances (how the user phrases their request, now referred to as prompts) and responses, I help train and fine-tune the model. After all, LLMs still have limitations and will generate text that is inconsistent or not tailored to the specific needs.

Fine-tuning is the process of taking a pre-trained LLM and steering the LLM towards producing the kind of output you want. When fine-tuning:

1. Keep the prompts simple and focused, avoid compound sentences

   1. Good prompt ✔️ : “Suggest a recipe that’s quick to make”

   2. Bad prompt ❌: “It’s getting late, what’s a quick food option?”

2. Correctly note punctuation

   1. Punctuation is generally important when fine-tuning large language models (LLMs)

   2. When you fine-tune a language model for a specific task, the model learns from the input data, including the patterns and relationships between words and punctuation marks

3. Be mindful of bias and experiment with diverse training scenarios

4. Iterate for optimal performance by testing with real customers to ensure the models align with actual expectations and needs

III. Design Considerations for Generative AI

Most generative AI experiences are wrapped in a user interface (UI). One of the biggest design challenges we see in this space is that customers “didn’t think to try that” or “don’t know where to start”. So, here are some things to keep in mind when designing:

1. Design for continuous education and discovery of how to engage with the AI

2. Show don’t just tell, use examples to illustrate what the AI can do

3. Manage customer expectations by communicating limitations

4. When teaching customers how to write or speak prompts, the more detailed a prompt the better the output will be

5. Good responses and outputs consider context, offering concise options for the customers

6. Add mechanisms to collect feedback from customers to improve the model

That’s all for now. More to come on generative AI and LLMs! Please send questions or comments, I read each one.

---

Favorite Quote and Photo of the Week

"Curiosity is the one thing invincible in Nature.” — Freya Stark, British explorer and travel writer

Early November in Ruidoso, New Mexico

---

Support the newsletter

If you enjoy my content, here’s how you can help support me:

• Like or comment on this post 💛

• Reply with a question or topic you’d like covered

• Forward it to a friend and recommend that they subscribe

• Share it with your network

---

That’s it for today. Thanks for reading!

Until next week,
raika