A compiler is usually divided into different phases. The input to the compiler is the source program and the output is a target program.
Lexical analyzer is the first phase of a compiler which gets source program as input. It scans the source program from left to right and produces tokens as output. A token can be seen as a sequence of characters having a collective meaning.
Lexical analyzer also called by names like scanner, linear analyzer etc.
Chat with our AI personalities
Simplicity-Techniques for lexical analysis are less complex than those required for syntax analysis, so the lexical-analysis process can be sim- pler if it is separate. Also, removing the low-level details of lexical analy- sis from the syntax analyzer makes the syntax analyzer both smaller and less complex.Efficiency-Although it pays to optimize the lexical analyzer, because lexical analysis requires a significant portion of total compilation time, it is not fruitful to optimize the syntax analyzer. Separation facilitates this selective optimization.Portability-Because the lexical analyzer reads input program files and often includes buffering of that input, it is somewhat platform dependent. However, the syntax analyzer can be platform independent. It is always good to isolate machine-dependent parts of any software system.
lexical analysis involves scanning the program to be compiled and recognizing the tokens that make up the source statements .Scanners are usually designed to recognize keywords ,operators,and identifiers as well as integers,floating point numbers,character strings etc.....so they are known as scanners
Lexical analyzer generators translate regular expressions (the lexical analyzer definition) into finite automata (the lexical analyzer). For example, a lexical analyzer definition may specify a number of regular expressions describing different lexical forms (integer, string, identifier, comment, etc.). The lexical analyzer generator would then translate that definition into a program module that can use the deterministic finite automata to analyze text and split it into lexemes (tokens).
Lexical refers to the "lexicon" or the kinds of words specific to a certain specialty or field. Think of it as slang or jargon, if you have a lexical inconsistency, the term you use in one specialty doesn't translate to other disciplines.
lex for example.