tokenize
Stdlib — IntrospectionPython 2.0+Advanced
Tokenize Python source code into individual tokens
Quick Info
- Documentation
- Official Docs
- Python Version
- 2.0+
- Dependencies
- None — Python Standard Library
- Install
Included with Python
Learn by Difficulty
Quick Example
python
import tokenize print(f"Module: tokenize") print(f"Contents: {dir(tokenize)[:10]}")
The tokenize module is part of Python's standard library. Tokenize Python source code into individual tokens.
Try in PlaygroundTags
stdlibintrospectionmetaprogramming