Solve an English cloze test in python with FitBERT

The following is a cloze test solved using FitBERT, and provided with the top 10k English words as options. The AI answers are marked bold, followed by the correct options in brackets.

There were very few (few) people at the party when we arrived but half an hour later, it was crowded.
How was I to know that she would have an allergic reaction to (to) the nuts in the cake? She should have said something!
Arthritis is a very painful disease (-condition-) that affects not only the old but also many younger people making many everyday activities difficult.
He said he couldn't come to the meeting because of a previous engagement but I think he is just making (making) excuses.
You were driving at over fifty miles an (per/an) hour and the limit here is only forty.
I have (have/need/ought) to go to school now, otherwise I will be late.
I can eat almost any (any) type of fish or seafood except for octopus which I can't stand.
The city is pretty safe although you may have some problems if you go into certain neighborhoods at (at) night.

That’s 7 out of 8 correct answers. And if you also accept disease as a valid alternative to condition (which I do), then we get 100% accuracy. Although our small sample is not representative, these are pretty impressive results for an off-the-shelf product.

Python code:

from fitbert import FitBert

def replace_mask(s):
    mask = '***mask***'
    placeholder = '___'
    return s.replace(placeholder, mask)


sentence_file = open("examples/cloze_esl_lounge.txt", "r")
sentences = map(replace_mask, sentence_file.readlines())
print(sentences)

word_file = open("words-10k.txt", "r")
words = word_file.readlines()

fb = FitBert()

for masked_sentence in sentences:
    filled_in = fb.fitb(masked_sentence, options=words)
    print(filled_in)

Resources:

1. FitBERT python library: https://github.com/Qordobacode/fitbert
2. Cloze exam taken from: https://www.esl-lounge.com/student/first-certificate/fce-046-open-cloze-exercise.php
3. Top 10k English wordlist: http://www.mit.edu/~ecprice/wordlist.10000