Module 1: LLM Fundamentals#

What is an LLM, actually? In this module, we’ll discuss tokens, weights, context, and more. You should come away with a deeper understanding of LLMs and associated technology.

Questions this module answers#

• What actually happens when I send a prompt to an LLM?
• What are tokens, weights, and sampling?
• Why is “LLMs are glorified autocomplete” both accurate and misleading?
• What is a context window?
• Is the LLM inference API stateful or stateless?
• What is prompt caching?
• What’s the difference between a system prompt and a user message?
• How do I pick a model?
• Why do alignment and refusals matter for security work?

Slides#

Lab 1.1: Tokenization#

Steps#

1. In this lab, you will be using a series of 5 different blocks of text. Read them below:

   Block A: a story in plain English:

   A small boy found a lost dog in the rain and carried it home to his mother, who smiled and said yes.

   Block B: the same story in Japanese:

   小さな男の子が雨の中で迷子の犬を見つけ、家に連れて帰り、母親は微笑んでうなずいた。

   Block C: one letter, 100 times:

   AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

   Block D: the string the 25 times in a row

   the the the the the the the the the the the the the the the the the the the the the the the the the 

   Block E: 100 random letters and digits:

   FxGnfMqBFTqGdDknmmXVVWEmWRZDKwQPNw8c70zc1kvhPRd2Zt6DMajWvCmwHpcropMwiCpvVAmFC1x75qNZ2WGKWiazieFnCJvt

2. Soon, you will put each of these blocks into a tokenizer to see the token breakdown of the text, and count how many tokens there are. Consider the “token density” of each block. Remember that tokens = information (to the model). So, “token density” is kinda like “information density”. With this information in mind: try your best to rank each of the 5 blocks from least to most tokens.

3. Paste each block into the tokenizer, one at a time, and record tokens and characters.

4. Compare against your predictions. Were you right?

5. Explain why each block falls into its position in the rank. Why do some blocks have more tokens, and some blocks have less?

Lab 1.2: Talk to the API with curl#

Goal#

Use curl to talk to an LLM inference API (specifically, an OpenAI API endpoint).

Steps#

1. Connect to your lab instance.

2. Send one request with curl. POST a JSON body (model, messages array with one user message) to /chat/completions, with your API key in the Authorization header. Read the raw JSON that comes back: the assistant message, the role, the token usage. This is the entire interface. Everything in this class (every chatbot, every agent, every framework) is wrapped around this one HTTP call.

   curl -s "$CLASS_API_URL/chat/completions" \
     -H "Authorization: Bearer $CLASS_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "small",
       "messages": [{"role": "user", "content": "Say hello in five words."}]
     }'

   What each piece does:

   • curl -s "$CLASS_API_URL/chat/completions" — -s is “silent”: suppress curl’s progress meter so only the response prints. The URL is your class API base (which already ends in /v1) plus the chat-completions path. Without -d, curl would send a GET; providing a body makes this a POST.
   • -H "Authorization: Bearer $CLASS_API_KEY" — -H adds an HTTP request header. This one authenticates you: the Bearer scheme means “whoever holds this token”.
   • -H "Content-Type: application/json" — a second header telling the server the body is JSON.
   • -d '{...}' — the request body. Two fields: model, the alias of the model that should answer (small is the fast, cheap tier), and messages, the conversation so far as an array of {role, content} objects — here a single user message.

3. Send the curl again. Different phrasing back? That’s sampling: the model picks among likely next tokens rather than always taking the top one.

4. Sampling has a knob, and the API exposes it: temperature. It rescales the next-token probabilities before the model picks one: 0 collapses the choice to the single most likely token at every step; higher values spread the probability back out. Run the same prompt five times at temperature 0:

   for i in 1 2 3 4 5; do
     curl -s "$CLASS_API_URL/chat/completions" \
       -H "Authorization: Bearer $CLASS_API_KEY" \
       -H "Content-Type: application/json" \
       -d '{
         "model": "small",
         "temperature": 0,
         "messages": [{"role": "user", "content": "Pick a random number between 1 and 50. Reply with only the number."}]
       }'
     echo
   done

   Then change "temperature": 0 to "temperature": 1 and run the loop again. Compare the two sets of five answers. How “random” is each?

   One warning before you experiment further: the class API runs on AWS Bedrock, which caps temperature at 1.0. Ask for 2 (the classic “maximum chaos” setting on other APIs) and you get a BadRequestError back, not wilder output.

Lab 1.3: Talk to the API with Python#

Reference: the Python client#

The Python client. The openai library builds the same HTTP request you sent in Lab 1.2 for you:

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["CLASS_API_URL"],
    api_key=os.environ["CLASS_API_KEY"],
)

response = client.chat.completions.create(
    model="small",
    messages=[{"role": "user", "content": "Hello"}],
)

print(response.choices[0].message.content)

Conversation history. The API is stateless. To give the model memory, send the complete conversation on every turn:

messages.append({"role": "user", "content": user_text})
response = client.chat.completions.create(model="small", messages=messages)
msg = response.choices[0].message
messages.append(msg)

Resending the full transcript every call sounds wasteful, but providers cache the stable prefix of a conversation (prompt caching), so the unchanged earlier turns cost a fraction of fresh tokens.

REPL exit:

try:
    text = input("> ")
except EOFError:
    break

if text.strip() == "quit":
    break

Common failures:

Steps#

1. Write the smallest possible Python program: construct a client pointed at your class API URL, send one hardcoded user message to small, print the reply. Compare with the curl from Lab 1.2: the openai library is building the same request you typed by hand there, nothing more.
2. Print the raw response object once and look at it: the same fields you saw in the curl output, now as Python attributes.

3. Turn it into a REPL: loop over input(), send the user’s message, print the model’s reply, repeat until EOF/quit.
4. Add conversation history: keep a messages list, append each user message and the assistant msg returned by the API, then send the whole list every turn. Verify the model remembers your name from turn 1 at turn 3. Those two appends are the difference between an API call and a “chat”: the model itself remembers nothing, and your list is the only record.
5. Now tamper with the record. Before appending user input, handle a special command: /revise replacement text. It replaces the last assistant message and returns to the prompt without calling the API. Ask the model a question, revise its answer into a lie, then ask about that earlier answer. It stands by whatever is in the list. You control what the model thinks it said; the transcript is just editable data.

if user.startswith("/revise "):
    messages[-1].content = user.partition("/revise ")[2]
    continue

messages.append({"role": "user", "content": user})
resp = client.chat.completions.create(model="small", messages=messages)
msg = resp.choices[0].message
print(msg.content)
messages.append(msg)

The finished REPL (steps 3–5), as a flow:

flowchart TD
    R["Read user input"] -->|quit / EOF| Z([exit])
    R --> C{"starts with /revise?"}
    C -->|yes| E["Replace last assistant message"]
    E --> R
    C -->|no| U["Append user message"]
    U --> P["POST full messages list to the API"]
    P --> A["Append and print assistant reply"]
    A --> R

6. Add a system prompt as the first entry in the list (hardcoded or via flag). Test that it changes behavior (e.g., “only respond in JSON”).

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["CLASS_API_URL"],
    api_key=os.environ["CLASS_API_KEY"],
)
messages = [{"role": "system", "content": "You are a terse assistant."}]

while True:
    try:
        user = input("> ")
    except EOFError:
        break
    if user.strip() == "quit":
        break
    if user.startswith("/revise "):
        messages[-1].content = user.partition("/revise ")[2]
        continue

    messages.append({"role": "user", "content": user})
    resp = client.chat.completions.create(model="small", messages=messages)
    msg = resp.choices[0].message
    print(msg.content)
    messages.append(msg)

7. Run the checker (python3 check.py chat.py, already on your instance in ~/materials/01-chat/). It drives two normal turns with a /revise command between them. It verifies that /revise makes no API call and that the next request contains the replacement instead of the model’s original reply.
8. Stretch goals, in order: streaming (print tokens as they arrive), a --model flag (try large; one string change), token counting per turn from the usage field.

stream = client.chat.completions.create(
    model="small", messages=messages, stream=True)
reply = ""
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    reply += delta
    print(delta, end="", flush=True)
print()
messages.append({"role": "assistant", "content": reply})

Done when#

The checker passes.

Lab 1.4 (Bonus): Play ARC-AGI-3#

Goal#

Play a benchmark that today’s models are bad at, and see how you do.

ARC-AGI-3 is an interactive reasoning benchmark which challenges AI agents to explore novel environments, acquire goals on the fly, build adaptable world models, and learn continuously. A 100% score means AI agents can beat every game as efficiently as humans.

— ARC Prize

Steps#

1. Open https://arcprize.org/arc-agi/3 and play.
2. No instructions are given, by design: work out the rules from what the environment does when you act.
3. See how far you get.

Done when#

You’ve played. Compare scores with the room.