TOKEN SEPARATION USING PYTHON
TOKEN SEPARATION USING PYTHON Token separation can also be called as Lexical Analyzer. Lexical Analyzer reads a input and separated it as identifiers , operators etc. The code for token separation is given below. lex.py import re file = open("read.py") operators = {'=' : 'Assignment op','+' : 'Addition op','-' : 'Subtraction op','/' : 'Division op','*' : 'Multiplication op','<' : 'Lessthan op','>' : 'Greaterthan op' } operators_key = operators.keys() data_type = {'int' : 'integer type', 'float': 'Floating point' , 'char' : 'Character type', 'long' : 'long int' } data_type_key = data_type.keys() punctuation_symbol = { ':' : 'colon', ';' : 'semi-colon', '.' : 'dot' , ',' : 'comma' } punctuation_symbol_key = punctuat...