..

2023-08-09

Lexical Analysis

  • It is a aggregation and classification process
  • Micro syntax is the lexical structure of a language

Token vs Lexeme vs Pattern

  • Tokens can be thought of as predefined categories of a grammar. Tokens can’t be broken down further
  • A lexeme is a part of the program that was matched
  • A pattern is the RE of a valid token

Example

  • main is a lexeme
  • KEYWORD is its Token
  • main is its pattern