WEEK 1

CHAPTER 1

Agents & Environments

Perceive → decide → act. Repeat. That loop is every AI system you will ever meet.

Manga page: You have SENSORS and ACTUATORS. Everything outside you is the ENVIRONMENT. / BEEP! / So I'm a loop? / I took the best road I knew... was I stupid? / No! Rational doesn't mean all-knowing.

The agent–environment loop

An agent is anything that perceives its environment through sensors and acts on it through actuators. What the sensors deliver at one moment is a percept. The full history of percepts is the percept sequence. The agent's behaviour is described by its agent function, which maps percept sequences to actions. A program running on real hardware implements that function.

Key subtlety: sensors give evidence, not the whole truth. A camera doesn't show what is behind a truck. A thermometer says "38 °C", not "the motor is about to fail". Much of this course is about acting well despite that gap.

PEAS: describe the job before the algorithm

Before choosing any algorithm, engineers write down the task environment with PEAS: Performance measure, Environment, Actuators, Sensors.

Example: Bota as a delivery robot on Astana streets

PParcels delivered on time, no collisions, low battery use, polite to pedestrians

ESidewalks, snow and ice, traffic lights, pedestrians, dogs, curbs

AWheels (speed, steering), brakes, lid lock, speaker

SCameras, lidar, GPS, wheel encoders, thermometer, battery sensor

Agent	P	E	A	S
Self-driving car	Safety, travel time, comfort	Roads, traffic, weather, pedestrians	Steering, throttle, brakes	Cameras, radar, lidar, GPS
Pac-Man	Score, survival, completion	Maze, ghosts, pellets, power-ups	Move up/down/left/right	Grid position, visible maze, events
Chess player	Win, avoid loss, position	Board, opponent, clock	Choose a legal move	Board, move history, clock

IN PLAIN WORDS

PEAS is a "job description" for the AI. If you can't say how success is measured (P), you can't know whether your AI is good.

Rationality

A rational agent chooses, for every possible percept sequence, the action that is expected to maximise its performance measure, given the evidence so far and whatever built-in knowledge it has. Rationality depends on four ingredients:

Performance: what counts as success?
Beliefs / prior knowledge: what does the agent expect about the world?
Actions: what can it actually do?
Evidence: what has it observed so far?

We judge a decision by the information available before acting, not with hindsight. An omniscient agent would know the actual outcome, and that's impossible in real tasks. A rational agent should still gather information (look before crossing the road) and learn from experience.

Classifying environments

The properties of the environment decide which algorithms work. The six main dimensions:

Dimension	One end	Other end
Observability	Fully observable: sensors show everything relevant	Partially observable: sensors show only a slice (or nothing: unobservable)
Agents	Single-agent	Multi-agent: competitive (chess) or cooperative (robot team)
Uncertainty	Deterministic: same state + action → same next state	Stochastic: one action can have several outcomes. With opponents, outcomes are also strategic.
Time structure	Episodic: each decision is independent	Sequential: current choices change future options
Change	Static: the world waits while you think	Dynamic: the world changes while you decide
Actions/states	Discrete: chess moves, grid steps	Continuous: steering angle, torque

Task	Observable	Agents	Deterministic?	Episodic?	Static?	Discrete?
Crossword puzzle	Fully	Single	Deterministic	Sequential	Static	Discrete
Chess with a clock	Fully	Multi	Deterministic (strategic)	Sequential	Semi-dynamic	Discrete
Taxi driving	Partially	Multi	Stochastic	Sequential	Dynamic	Continuous
Medical diagnosis	Partially	Single	Stochastic	Sequential	Dynamic	Continuous
Defect detection on photos	Fully	Single	Deterministic	Episodic	Static	Continuous

The hardest case is partially observable, multi-agent, stochastic, sequential, dynamic and continuous, which describes driving on Almaty roads in winter.

CLASSIFY IT!

Poker: is it fully or partially observable?

You can't see the other players' hands, so you must reason about beliefs. It's also stochastic (shuffled deck) and multi-agent.

A spam filter judging one email at a time: episodic or sequential?

Each email is an independent episode. A robot exploring a maze, by contrast, is sequential: every step changes where it can go next.

A robot vacuum on a carpet where wheels sometimes slip: deterministic or stochastic?

The same "move forward" action can lead to different positions. Chapter 5 (MDPs) models exactly this.

Four agent architectures (and one bonus)

From simplest to richest. More structure means smarter behaviour but more computation and design effort.

1 · Simple reflex

Acts on the current percept with IF–THEN rules. Fast and interpretable, but brittle: it forgets everything it saw before.

percept → rules → action

2 · Model-based

Keeps an internal state to remember what sensors can't show now ("the car behind me is still there").

percept → update state → rules → action

3 · Goal-based

Imagines future states and plans a sequence that reaches a goal. This is where search (Ch. 2) comes in.

state + goal → search/plan → action

4 · Utility-based

Goals aren't enough when outcomes differ in quality or are uncertain. A utility scale lets the agent trade off speed vs safety (Ch. 5, 8).

options → predict → utility → best expected

★ Learning agent

Any of the above plus a learning element that improves performance from feedback (Ch. 5, 8, 9).

critic → learning element → better rules

CASE STUDY · PAC-MAN

A pure reflex rule "run away from the visible ghost" can trap Pac-Man in a dead end. Good moves depend on what happens after the move: escape routes, power-up timing. So Pac-Man needs goal-based or utility-based reasoning.

Try it: a simple reflex agent

The lecture's smart-office agent in Python:

def office_agent(temperature, co2, people):
    if people == 0:      return "Energy-saving mode"
    elif temperature < 20: return "Heating ON"
    elif temperature > 26: return "Cooling ON"
    elif co2 > 1000:       return "Fresh-air fan ON"
    else:                 return "No action"

TRY IT! Reflex agent playground: the first matching rule wins

Temperature 24 °C CO₂ 900 ppm People 5

Action → No action

Try temperature 18 and CO₂ 1200. Two rules match, but only the first one fires. Now swap the order. Rule order is part of the agent's design.

Bridge: from reflexes to search

When rules aren't enough, a goal-based agent describes the problem formally as a search problem:

Initial state: where the problem starts (Astana)
Actions: legal moves (drive along a road)
Transition model: the result of an action (now in Karaganda)
Goal test: have we arrived? (Almaty?)
Path cost: how expensive the solution is (sum of road costs)

The design workflow from the lecture: PEAS → environment taxonomy → representation → architecture → evaluation.

DID YOU KNOW?

A thermostat is a genuine (very simple) reflex agent: one sensor, one rule, one actuator. Many "smart" home devices are still reflex agents with more rules.

MINI-QUIZ

1. Bota remembers that a road was closed yesterday, even though it can't see it now. Which architecture needs that?

Internal state separates "what I see now" from "what I know happened".

2. Which PEAS element is "number of parcels delivered on time"?

It measures success, so it belongs to P.

3. A rational agent…

Rationality is about expected value under uncertainty, not about guaranteed success.

Manga page: I have a goal now: ALMATY! / But which road should I take?

1