Loading…
Glossary term
The trade-off between trying new actions to learn and using what is already known to score well.
An agent must balance trying unfamiliar actions to learn more (exploration) with using the best action it already knows (exploitation). Too little exploration can trap it in a mediocre strategy; too much wastes effort.
A common rule, epsilon-greedy, explores at random a small fraction of the time.
Epsilon-greedy action choice
import random
def choose(q_values, epsilon=0.1):
if random.random() < epsilon:
return random.randrange(len(q_values)) # explore
return max(range(len(q_values)), key=lambda i: q_values[i]) # exploitThis trade-off appears throughout learning, from game-playing agents to recommendations.