deftokenize(expression):
tokens = []
i = 0while i < len(expression):
ch = expression[i]
if ch.isspace():
i += 1elif ch.isdigit():
num = ""while i < len(expression) and expression[i].isdigit():
num += expression[i]
i += 1
tokens.append(("NUM", int(num)))
elif ch in"+-*/":
tokens.append(("OP", ch))
i += 1else:
i += 1return tokens
result = tokenize("12 + 34 * 5")
for token in result:
print(token)
Output
Click "Run" to execute your code
while loops are natural for parsers and state machines where the number of iterations depends on the input content, not just its length.
Challenge
Try modifying the code above to explore different behaviors. Can you extend the example to handle a new use case?