Under the Hood
Peek inside how AI actually works (optional technical module)
What You'll Learn
- ✓The high-level flow: Data → Training → Model → Application
- ✓What tokens are and why they matter for cost and context
- ✓The concept of embeddings (turning words into numbers)
- ✓What parameters mean and why GPT-4 is bigger than GPT-3
- ✓How transformers and attention mechanisms work at a conceptual level
Key Ideas
AI follows a flow: (1) Collect massive amounts of text/image data, (2) Train a model by learning patterns, (3) Fine-tune for specific tasks, (4) Deploy as an application (like ChatGPT). Understanding this helps you grasp AI's strengths and limitations.
Examples:
- • Data: Books, websites, conversations
- • Training: Learning word relationships and patterns
- • Fine-tuning: Making it helpful, harmless, and honest
- • Application: The chatbot you interact with
AI doesn't see words; it sees tokens (word chunks). 'Hello' might be one token, 'ChatGPT' might be two. This matters for costs (priced per token) and limits (max tokens per conversation).
Examples:
- • Roughly 1 token ≈ 4 characters or 0.75 words
- • A typical conversation uses thousands of tokens
- • Longer prompts = higher costs and slower responses
Parameters are like the 'brain cells' of an AI model. More parameters generally mean better capability but also more compute power needed. GPT-4 has hundreds of billions of parameters.
Dive Deeper
Explore the mechanism, mastery techniques, and critical thinking considerations. Click to expand each layer.
How Transformers Work
The transformer architecture revolutionized AI through the attention mechanism, which allows models to weigh the importance of different words in context. Unlike older sequential models, transformers process entire sequences in parallel, making them much faster to train. The key innovation is self-attention: the model learns which words are most relevant to each other, even if they're far apart in the text.
Training Dynamics and Emergent Capabilities
As models scale up in parameters and data, they exhibit emergent capabilities—abilities not explicitly programmed that appear at certain thresholds. Training involves forward passes (making predictions), calculating loss (how wrong the predictions are), and backward passes (adjusting weights via gradient descent). The process requires massive computational resources: training GPT-3 took thousands of GPU-days.
Technical Limitations and Research Frontiers
Despite impressive capabilities, transformers have fundamental limitations. Context windows restrict how much information they can process at once. They struggle with arithmetic and logical reasoning because they learn patterns rather than true understanding. They can't update their knowledge without retraining. Current research focuses on extending context windows, improving reasoning through techniques like chain-of-thought, reducing hallucinations through retrieval-augmented generation, and making models more efficient.
Suggested Resources
Coursera
arXiv
Try This Now
Put your learning into practice with these hands-on exercises. Copy the prompts and try them in your favorite AI tool.
Visit OpenAI's GPT-4 technical report or Anthropic's Claude documentation. Read the introduction and summarize 3 key capabilities or limitations in plain English.
Use OpenAI's tokenizer tool to see how different texts are broken into tokens. Compare: 'Hello' vs 'Supercalifragilisticexpialidocious' vs 'ChatGPT'.
Find a model card for GPT-4, Claude, or Gemini. What does it say about the model's training, intended use, and limitations?
Find an attention visualization tool (like BertViz or the Transformer Explainer). Input a sentence and observe which words the model pays attention to. What patterns do you notice?
Related Prompts from the Library
Practice what you've learned with these prompts from our library.
ROLE: You are a assistant. GOAL: Develop a concept for a subscription-based model for [service/product] to enhance customer engagement. CONTEXT: Input details: service/product. Ask clarifying questions if any input details are missing. TASK: Develop a concept for a subscription-based model for [service/product] to enhance customer engagement. OUTPUT FORMAT: Script CONSTRAINTS:
View Prompt →ROLE: You are a assistant. GOAL: Design a questionnaire to investigate the effectiveness of [recent advertising campaign]. CONTEXT: Input details: industry, recent advertising campaign, specific demographic, time frame, your product line. Ask clarifying questions if any input details are missing. TASK: Design a questionnaire to investigate the effectiveness of [recent advertising campaign]. Build a model for predicting customer needs in [industry] over the next [time frame]. Outline a strategy for a longitudinal study on brand loyalty among [specific demographic]. Formulate a plan to research customer response to price changes in [your product line]. OUTPUT FORMAT: Text CONSTRAINTS:
View Prompt →ROLE: You are a assistant. GOAL: Design an Employee Training. CONTEXT: Ask clarifying questions if any input details are missing. TASK: Design an Employee Training OUTPUT FORMAT: Text CONSTRAINTS:
View Prompt →ROLE: You are a expert consultant in digital marketing. GOAL: help me create a detailed business model canvas for a [business details] company. CONTEXT: Input details: business details. Observe constraints: Tabular format. Ask clarifying questions if any input details are missing. TASK: help me create a detailed business model canvas for a [business details] company. Organize your answers in a table that reproduces the original format used in consulting. I want you to write detailed answers that are focused on adding value and act as an expert consultant in digital marketing. Business details: Insert business details] OUTPUT FORMAT: Table CONSTRAINTS: Tabular format
View Prompt →ROLE: You are a assistant. GOAL: Identify potential crisis scenarios for [company/industry]~Create a crisis management team structure~Develop ke Assess current environmental impact of [company/industry]~Set sustainability goals and objectives~Develop stra Assess current digital capabilities of [company]~Identify areas for digital innovation and improvement~Outline req eir marketing and branding approaches~Assess their strengths and weaknesses~Identify potential opportu ycle~Develop sales tactics and techniques~Create a sales forecast~Suggest tools and resources for sales fits and value proposition~Present a pricing structure or cost analysis~Include a project timeline and miles eate an outline of training modules and content~Suggest delivery methods (e. CONTEXT: Input details: company, company/industry. Observe constraints: Follow specified tone. Ask clarifying questions if any input details are missing. TASK: Identify potential crisis scenarios for [company/industry]~Create a crisis management team structure~Develop ke Assess current environmental impact of [company/industry]~Set sustainability goals and objectives~Develop stra Assess current digital capabilities of [company]~Identify areas for digital innovation and improvement~Outline req eir marketing and branding approaches~Assess their strengths and weaknesses~Identify potential opportu ycle~Develop sales tactics and techniques~Create a sales forecast~Suggest tools and resources for sales fits and value proposition~Present a pricing structure or cost analysis~Include a project timeline and miles eate an outline of training modules and content~Suggest delivery methods (e.g., workshops, e-learning)~D ategies to improve customer satisfaction~Create a loyalty program outline~Design a communication plan fo ected profit margins~Develop cash flow projections~Identify potential financial risks~Suggest strategies for e for the launch process~Design a pricing strategy~Plan the actual launch event or campaign~Develop a s for cost reduction~Recommend strategies for better inventory management~Propose methods to enhanc ey messaging templates~Outline communication channels and protocols~Create a step-by-step response ategies for reducing carbon footprint~Create initiatives for waste reduction and resource conservation~Des quired technological investments~Develop a plan for data management and analytics~Create a timeline fo nities for differentiation~Summarize findings and strategic recommendations s team tones~Close with a compelling call to action OUTPUT FORMAT: Text CONSTRAINTS: Follow specified tone
View Prompt →ROLE: You are a public speaker. GOAL: act as a public speaker. CONTEXT: Ask clarifying questions if any input details are missing. TASK: act as a public speaker. You will deliver a motivational speech to a <target audience> audience. Y With your expertise in Simulate Expert, what strategies would you recommend to <small business/entrepreneur As a seasoned Content Promotion and Distribution expert who has worked for <big company> for <number> ye As a Content Promotion and Distribution expert with <number> years of experience, what advice would you giv As an expert in SEO copywriting who graduated from <university>, and now works at <company>, what advice In your opinion, what is the most effective way to conduct keyword research for a website in <industry>? Share You are a seasoned SEO copywriter who's worked with both <B2B/B2C> clients, graduating from <School>. W You are an experienced SEO copywriter who graduated from <University>, and you are working with <company As a top talented SEO copywriter who finished <university> and works at <big company>, how can I effectively Share your opinion on the importance of meta descriptions in SEO copywriting for <industry>. How can we effe As a skilled <field> expert with a degree from <University> and employment at <company>, how would you sug You are an accomplished SEO copywriter who has worked with <client type> businesses, and graduated from As a knowledgeable <field> specialist who graduated from <School>, and is currently employed by <company> As an experienced SEO copywriter, how can we ensure that our website content is optimized for voice search w As a respected expert in <field>, how do you balance the need for original content with the value of sharing exis You are an expert in the field of Buyer Persona Development and have graduated from <university/school>. You As a skilled curator in <field>, how do you decide which <type of content> to feature on your platform? Are ther As a content performance analysis expert with a degree from <university>, can you provide advice on how to im As a simulated expert in content creation and curation, how would you advise someone who graduated from <u As an expert in <social media marketing> who studied at <renowned university> and works at <well-known com You are an expert in Marketing Storytelling and you graduated from Harvard University and you are working on You are an expert in <branding>, having attended <New York University> and currently working at <Apple>. Ca You are a Marketing Storytelling expert who earned their degree from the London School of Economics and you If you're an expert in content creation, having graduated from <university> and working at <company>, how wo You are a top-notch <brand manager> with a degree from <esteemed institution>, and you are a vital part of <m As someone experienced in buyer persona development, what advice would you give to a company struggling You have established yourself as a successful content creator after graduating from <school/university>. What As a top-performing content analyst with a degree from <school>, what advice would you give to a company st hould you track to measure engagement? t are some strategies for maintaining a consistent and engaging content pipeline? ess? lude identifying influencers, compensation models, promotional materials, and success metrics. lick-through rates, and conversion rates. Include A/B testing, improved targeting, and personalized mess s. d working in <YouTube studio>, what are some things I can do to improve video content performance an anking with respect to our <target audience> and increase our organic traffic. egies can I use to increase engagement and attract new followers? e strategies for optimizing my website's user experience and increasing conversions? strategies used to analyze data. What have you found to be the most effective method for measuring eng How do you conduct an audience analysis to ensure your content is meeting their needs and preference ons, and how can you optimize them? ws from <1,000> to <100,000> within one year. nto the preferences and behaviors of <specific demographic>. The infographic should be easy to read an ntent agency>, can you share with me your advice on how to engage readers with my articles and improv ct/service>, with a strong call to action for followers to visit our website or make a purchase. Your speech should include personal anecdotes, inspiring quotes, and real-life examples to motivate and r> in order to optimize their content and boost engagement? ears, what advice would you offer to <social media marketer/PR strategist> on how to create more buzz f ve to <newly-launched blog/website> on how to increase their traffic? e do you have for improving website traffic through <keyword> optimization? e your process and recommended tools. OUTPUT FORMAT: Script CONSTRAINTS:
View Prompt →Reflection Questions
- 1.How does understanding the technical side change how you use AI tools?
- 2.What technical concepts do you want to learn more about?
- 3.How might this knowledge help you explain AI to others?
- 4.What technical limitations should you keep in mind when using AI for important tasks?