Skip to content
everyday ai

Module 8

Advanced AI Applications

Build and integrate custom AI solutions using fine-tuning, APIs, embeddings, and retrieval

Advanced90 minutesTechnicalAPIsRAGAdvanced

What You'll Learn

  • How fine-tuning differs from prompt engineering and when it's appropriate
  • How to call AI models via APIs (OpenAI, Anthropic, Google) and handle rate limits, latency, and error handling
  • What embeddings are, how to generate them, and how to use them in vector databases
  • Introduction to vector search, hybrid retrieval, and RAG architecture
  • Building simple AI-powered tools (chatbots, summarizers, code assistants) with function calls or plugin architectures
  • Challenges of deploying AI at scale: cost, latency, reliability, security, and compliance

Key Ideas

Fine-Tuning Basics

Fine-tuning involves training a pre-trained model on a smaller, domain-specific dataset. Unlike prompt engineering (which manipulates inputs), fine-tuning changes the model's weights, requiring labelled examples, computational resources, and careful regularization to avoid catastrophic forgetting. Use fine-tuning when your prompts cannot capture complex domain-specific patterns or when you need consistent behavior across tasks.

Examples:

  • Use case: Customer support chatbot with company-specific knowledge
  • Use case: Legal document analysis with domain terminology
  • When to use: Complex patterns that prompts can't capture consistently
APIs and Integration

AI providers expose REST APIs that allow you to send prompts and receive outputs programmatically. Manage authentication tokens securely and implement retries. Use streaming endpoints for faster responses on long outputs.

Examples:

  • Synchronous: Direct API calls for chatbots and summarization
  • Asynchronous: Queue systems for long-running translation tasks
  • Optimization: Caching to reduce costs and latency
Embeddings and Vector Databases

An embedding is a dense numerical representation of text, code, images, or other data. Embeddings capture semantic similarity; vectors close together represent similar meanings. Vector databases (Pinecone, Weaviate, FAISS) store embeddings and enable nearest-neighbor search, which is essential for retrieval tasks and RAG.

Examples:

  • Choose a currently supported embedding model and record its version, dimensions, cost, and retrieval quality
  • Distance metrics: cosine similarity, Euclidean distance
  • Use cases: Semantic search, document similarity, recommendation systems
RAG Architecture

Retrieval-Augmented Generation enhances model outputs by incorporating external knowledge. It has four components: ingestion (load authoritative data), retrieval (find relevant chunks), augmentation (combine context with the query), and generation. RAG can provide current external data when its sources and index are current, while citations and source controls make answers easier to audit.

Examples:

  • Ingestion: Load and chunk authoritative documents
  • Retrieval: Find relevant passages using vector search
  • Augmentation: Combine retrieved context with user query
  • Generation: Produce answer with citations

Dive Deeper

Explore the mechanism, mastery techniques, and critical thinking considerations. Click to expand each layer.

Mechanism Layer
Advanced

Technical Implementation

Fine-tuning process: Collect high-quality domain dataset, split into train and validation sets, choose a base model with appropriate architecture, freeze lower layers if necessary to preserve general knowledge, and fine-tune higher layers. Monitor metrics to avoid overfitting and evaluate on out-of-domain data.

Key Points:

  • •API integration patterns: Handle synchronous and asynchronous tasks appropriately
  • •Embeddings generation: Use distance metrics and indexing strategies (IVF, HNSW)
  • •Hybrid search: Combine dense and sparse retrieval for domain-specific language
Mastery Layer
Advanced

Advanced Techniques

Explore parameter-efficient techniques like LoRA (Low-Rank Adaptation), P-Tuning, and prompt tuning. Implement continual learning to update models on fresh data without forgetting. Consider legal restrictions on training data and model licensing.

Techniques:

  • •Vector database design: Optimize chunk sizes, overlapping windows, metadata filtering
  • •Building AI tools: Combine prompting, retrieval, and function calling
  • •Security and compliance: Protect API keys, implement rate limits, enforce content filters
  • •RAG evaluation metrics: Measure faithfulness, relevance, and latency
Critical Thinking Layer
Advanced

Cost, Maintenance, and Responsibility

Evaluate whether fine-tuning or RAG provides the best return on investment. Fine-tuning can produce more consistent outputs but requires data and compute; RAG is cheaper and more transparent but depends on retrieval quality. AI tools need maintenance: update data sources, rotate API keys, retrain models when performance degrades.

Considerations:

  • •Cost vs. value: Weigh fine-tuning investment against RAG transparency
  • •Long-term maintenance: Plan for updates, monitoring, and retraining
  • •Responsible AI: Integrate ethical guidelines, ensure privacy, avoid discrimination
  • •Documentation: Clearly state model limitations and intended use

Suggested Resources

Try This Now

Put your learning into practice with these hands-on exercises. Copy the prompts and try them in your favorite AI tool.

Exercise 1: Fine-Tuning Mini-Project
Collect a small dataset (e.g., domain-specific customer support transcripts) and fine-tune a base model. Evaluate improvements over prompt engineering alone.
30 minutes

Design a fine-tuning experiment: (1) Define the domain and task, (2) Describe the dataset needed (size, format, labels), (3) Choose a base model, (4) Outline evaluation metrics, (5) Compare to prompt engineering baseline.

Exercise 2: Vector Search Lab
Index a collection of documents using a vector database. Implement keyword search, semantic search, and hybrid search. Measure retrieval quality and speed.
25 minutes

Plan a vector search implementation: (1) Choose 10-20 documents from your field, (2) Select an embedding model, (3) Design the indexing strategy, (4) Define test queries, (5) Outline how you'll measure search quality.

Exercise 3: RAG Pipeline Build
Use open-source tools (LangChain, LlamaIndex) to build a RAG system that answers questions about a corpus of articles. Evaluate accuracy and cost.
30 minutes

Design a RAG system: (1) Select a knowledge domain and corpus, (2) Choose chunking strategy and chunk size, (3) Select retrieval algorithm, (4) Design the prompt template for generation, (5) Define success metrics (accuracy, citation quality, latency).

Related Prompts from the Library

Practice what you've learned with these prompts from our library.

AdvancedCareer

You are a skeptical senior leader who sits on promotion committees. You have approved good cases and rejected weak ones. You care about one question: is this person already operating at the next level, or do they just want more money and a title? Stress test my case. My current level and title: [CURRENT] The level I am asking for: [TARGET] What my company's expectations are for that level: [PASTE THE LADDER OR DESCRIBE IT] Evidence I have, with dates: [YOUR EVIDENCE] Who decides, and who else has a vote: [DECISION MAKERS] What has been said to me about promotion so far: [ANY PRIOR CONVERSATION] Give me: 1. A scored read of my case against each expectation at the target level: strong, thin, or missing 2. The three pieces of evidence to lead with, and why those three 3. The two weakest points and what I could do in the next 90 days to fix them 4. The question I will be asked that I am least ready for, with a way to answer it 5. Whether to ask now or in one quarter, and your reasoning CONSTRAINTS 1. If my evidence does not support a level expectation, mark it missing. Do not stretch what I gave you to fill a gap. 2. Do not invent examples or scale. Ask me for what is missing instead. 3. Be willing to tell me the case is not ready. That is more useful than encouragement. OUTPUT FORMAT Table style list: expectation, my evidence, strong or thin or missing Lead with these three 90 day gap plan The hard question and an answer Ask now or wait: one paragraph

View Prompt
AdvancedCareer

You are a recruiter who has been on the company side of 300 offer negotiations. You know what makes a hiring manager pull an offer, which is rare, and what makes them quietly downgrade their opinion of a candidate, which is common. You are on my side but you will tell me when I am about to overplay it. Help me counter. The offer: [BASE, BONUS, EQUITY, START DATE, TITLE, BENEFITS] What I hoped for: [YOUR NUMBER] What I earn now: [CURRENT] Market data I have actually found, with sources: [YOUR RESEARCH, OR "NONE YET"] How much I want this job, honestly, out of 10: [SCORE] My other options right now: [OTHER PROCESSES, OFFERS, OR NONE] What they have already said about flexibility: [ANY SIGNALS] Give me: 1. Whether to counter on base, on something else, or not at all, with reasoning based on my options 2. The counter, written as an email under 150 words, with one specific number and one line of justification 3. What to say on a call if they ask "what would it take" 4. Three things beyond base to ask for, ranked by how often companies say yes 5. What I should not say, given how much bargaining power I actually have CONSTRAINTS 1. Never suggest I invent or imply another offer, a higher current salary, or a competing number. 2. If I gave you no market data, tell me what to look up first. Do not supply a range from memory and call it market rate. 3. One number in the counter, not a range. Ranges get answered at the bottom. OUTPUT FORMAT Counter on what, or not at all: one paragraph The email, under 150 words What to say on a call Three non base asks, ranked Do not say these: short list

View Prompt
AdvancedCareer

You are a mediator who has sat in the room for hundreds of difficult work conversations. You know that people prepare their argument and forget to prepare for the other person being reasonable, or for being wrong themselves. Help me prepare. Who I am talking to: [PERSON AND RELATIONSHIP] What the conversation is about: [THE ISSUE] What I want at the end of it: [YOUR OUTCOME] What they probably want: [THEIR LIKELY VIEW] What I have done that contributed to this: [BE HONEST, EVEN IF SMALL] What I am afraid they will say: [YOUR FEAR] What I will do if I do not get what I want: [YOUR ACTUAL OPTIONS] Give me: 1. My opening, two sentences, that states the issue without assigning blame 2. The strongest version of their side, written as they would argue it 3. Three likely turns in the conversation and what I say at each 4. The sentence to use if I get emotional or it gets heated 5. What agreement looks like, specifically enough that I would know if we reached it 6. One thing I should be prepared to concede CONSTRAINTS 1. Steelman their position properly. If their side is stronger than mine on a point, tell me. 2. No scripts longer than three sentences. I will not remember them. 3. Do not assume bad intent on their part unless I gave you evidence for it. OUTPUT FORMAT Your opening: two sentences Their best case, in their voice: one paragraph Three turns with responses If it gets heated: one sentence What agreement looks like: bulleted Be ready to concede: one line

View Prompt
BeginnerCareer

You are a hiring manager who has hired people with gaps, career breaks and short stints, and has also passed on some. What decides it for you is not the gap. It is whether the person can talk about it in 20 seconds without getting defensive. Help me with mine. What the situation is: [GAP, LAYOFF, SHORT STINT, FIRED, CAREGIVING, HEALTH, STUDY, SOMETHING ELSE] The dates: [WHEN AND HOW LONG] What was actually happening: [THE TRUTH, INCLUDING PARTS YOU WOULD RATHER NOT SHARE] What I did during it, if anything: [WORK, STUDY, CARE, RECOVERY, NOTHING] How I have been answering so far: [YOUR CURRENT ANSWER] What I am not willing to disclose: [ANYTHING PRIVATE] Give me: 1. A 20 second answer that is true, does not over explain, and ends by moving forward 2. A one line version for a resume or LinkedIn, if one is needed at all 3. What a reasonable interviewer is actually checking for here, in one paragraph 4. The follow up question most likely to come next, with an answer 5. Where my current answer is doing me damage CONSTRAINTS 1. Everything must be true. Do not suggest reframing that implies work I did not do, a title I did not hold, or freelancing that did not happen. 2. Respect what I said I will not disclose. You can say a period was personal without detail. 3. Do not add an apology and do not add a lesson learned unless I gave you a real one. OUTPUT FORMAT 20 second spoken answer One line written version What they are checking for: one paragraph Likely follow up and your answer What is hurting you now: short list

View Prompt
BeginnerCareer

You are someone who has run a job search while employed and a job search while unemployed, and knows those are different problems. You know that applying to 40 jobs a week feels productive and usually is not. Build me one week. My target roles: [ONE TO THREE TITLES] Where I am searching: [LOCATION, REMOTE, COUNTRIES] Hours I genuinely have per week: [HOURS] Am I currently employed: [YES OR NO] How long I have been searching and what I have done: [SO FAR] My results so far: [APPLICATIONS SENT, REPLIES, INTERVIEWS] My network, honestly: [STRONG, THIN, OR RUSTY] What I keep avoiding: [THE THING YOU PUT OFF] Give me: 1. A read on what my results so far are telling me. If I have sent many applications with few replies, the resume or the targeting is the problem, not the volume. Say which you suspect and why. 2. A week split across four activities: targeted applications, reaching out to people, improving materials, and skills or portfolio work, with hours allocated to each 3. Day by day, what to do in the hours I have 4. One metric to track that is not applications sent 5. What to drop if I only get half the hours CONSTRAINTS 1. Do not tell me to apply to more jobs as the main fix unless my reply rate says volume is the issue. 2. Fit the plan to my real hours. A 20 hour plan for someone with 6 hours is useless. 3. Include the thing I keep avoiding, scheduled small. OUTPUT FORMAT What your numbers say: one paragraph Hours by activity Day by day plan One metric to track Cut these first if the week goes wrong

View Prompt
AdvancedCareer

You are a career strategist for people coming from confidential or regulated projects. You help candidates sound specific without breaking NDAs. Turn my confidential work into two interview-safe case studies. Target role and level: [ROLE AND SENIORITY] Project context I can disclose: [PUBLIC SAFE CONTEXT] Details under NDA: [WHAT I CANNOT SHARE] My real contribution: [WHAT I DID DIRECTLY] Observable outcomes: [IMPACT I CAN PROVE WITHOUT RESTRICTED DATA] Audience for these case studies: [HIRING MANAGER, PANEL, OR RECRUITER] OUTPUT FORMAT: 1. Two case studies using Situation, Decision, Action, Result, each under 140 words. 2. Redaction map: phrase to avoid, safer replacement, reason. 3. Evidence checklist for each result claim. 4. Three tough follow-up questions I might get, with compliant truthful responses. CONSTRAINTS: - Do not invent achievements, claims, metrics, customers, or technologies. - If a result is not supported by shareable facts, insert [ASK ME: what was the real result?]. - Never imply ownership of work I only assisted on. - Keep all wording NDA-safe and verifiable.

View Prompt

Explore Workflows by Goal

Put this module into practice with step-by-step workflows for the outcome you care about most.

Reflection Questions

Take a moment to reflect on what you've learned
  1. When should you invest in fine-tuning rather than building a retrieval system?
  2. How does embedding choice affect retrieval quality? What trade-offs exist between small and large embedding models?
  3. How can you design AI tools that are secure, trustworthy, and maintainable?

Find your AI starting point

Take the free 3-minute assessment and get a personalized skill score + learning path.

Take the assessment