This program tokenizes a string and can be used as a reference for Rutgers students creating the tokenizer assignment. I have removed the while loop in the main and it can be easily placed back inside with a little analysis.
shereif/Tokenizer-
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|