SEGMENTATION

• Segmentation
• Choices
• Count in 1M Tokens (WSJ)
• Algorithmic Ideas
• Simple Path Example
• Problems with Path Approach
• Probabilistic Language Models
• Unigram Algorithms
• Probability of a Segmentation
• Maximum Probability Segmentation
• Algorithm
• Discussion
• Correctness
• Algorithmic Design