Loading…
Glossary term
A metric that scores machine translation by its overlap with reference translations.
BLEU compares a machine translation with one or more human reference translations by counting overlapping word sequences, with a penalty for being too short. Scores run from 0 to 1, or 0 to 100.
It is quick and automatic, but rewards surface similarity and can miss good paraphrases.
BLEU with NLTK
from nltk.translate.bleu_score import sentence_bleu
reference = ["the cat sits on the mat".split()]
candidate = "the cat is on the mat".split()
print(sentence_bleu(reference, candidate))It was the standard translation metric for years and still appears in research, though newer metrics correlate better with human judgment.