Labs

(it is okay to make a tiny change after the deadline if we found a bug)

the tests should not crash, they may not pass

• we can switch the domain later
• but we should imagine two domains
• at least one of them should be our preferred
• we want a task-oriented system
  • anything that operates on top of a database or API
• we don't need to focus on parsing errors (or misunderstanding) in the flowchart
• we can use anything as the backend – even a local file

• look at the data in data/hw2 (only the train data?)
• write a script that separates the user and the system turns
• some search calls (after silence) will be parts of the text
  • we ignore only the lines without tabs
• don't forget to comment the results
• we should run the tests locally

• NLU, just understanding, no reply
• we will install dialmonkey using pip (see readme)
  • it's an empty shell
• there is a config file
  • there are some components
• jupyter notebook with demo snippets
• pick one domain
• implement ruse-based NLU
  • there are restaurant examples we can look at
  • we need to update our repo from the upstream
• we should create a config file – we can start with the sample config and replace the dummy NLU with our own implementation
• we should test it (write 15 test utterances)
  • input on the left, dialogue act on the right
  • we should separate it by a real tab character
  • there can be multiple intents on the right (with & separating them)

• we won't work with our domain, we will use DSTC2 restaurant data
• for each sentence, there is a DA annotation (sentence-level, not token-level)
• our goal is to do sentence-level classification (no need for sequence analysis)
• idea: for each intent-slot pair we should train a classifier (~ 50 classifiers total)
  • the goal is to make it work, we don't have to follow that strictly
• we should put the classification results back together into DAs
• there's an evaluation script
• then we should set it up so that we can use it to chat

• belief tracker
• so far we've been filling dial.nlu
• now, we need to fill dial.state dictionary … key = slot, value = dict (value → probability)
  • dial.state = {'price': {'cheap': 0.2, 'moderate': 0.5, 'expensive': 0.1, None: 0.2}, 'area': {'north': 0.5, 'east': 0.1, None: 0.4}}
  • after each turn we update the values with probabilities from dial.nlu
  • (initially: dial.state = {'price': {None: 1.0}, 'area': {None: 1.0}})
• the probabilities should sum up to one! (it's a job of NLU to assure that)
• in hw4 we don't need to set the probabilities (but we can do that)
• in hw5 the NLU needs to set the probabilities (in sklearn, predict_proba)

• implement the rule-based policy
• we should take the instructions about the policy with a grain of salt
• we can merge hw3, we should not delete the branch
• the database can be CSV table, SQLite database, …

how to count the bigrams?

• we don't need to optimize the number of used templates (in the third requirement) – if we do not find the match, we can go one by one
• load the upstream changes into the repository, there is a reference implementation we can use

• find the top 10 corresponding keys (utterances) using TF-IDF cosine similarity
• select one utterance randomly
• return the response to that utterance