What Are Tokens in C/C++?

 

What Are Tokens?

 In simple terms, tokens represent the smallest meaningful components of any program that the compiler easily comprehends. They can be classified into six sub-categories:

1. Keywords
2. Constants
3. Strings
4. Identifiers
5. Operators
6. Special Symbols

Now, let's explore each category:

Keywords:

Keywords are reserved terms in programming languages that provide specific functionalities to the program. They cannot be used as variable names. C/C++ pre-processor directives, such as header files, can modify keywords before compilation.

 

Constants:

Constants, like variables, are unchanging values. The key distinction is that the program cannot modify the value of a constant after it has been defined. Constants can belong to various data types, including integer, floating-point, octal, hexadecimal, character, and string constants.

 

Strings:

Strings are arrays of characters ending with a null character "\0". They are enclosed in double quotes (" ") and differ from characters within single quotes (' ').

 

Identifiers:

Identifiers are user-defined names for variables, arrays, and functions. They must follow specific rules, such as starting with a letter or underscore, and cannot be keywords.

 

Operators:

Operators are symbols that trigger actions when applied to variables or objects. Unary operators work on a single operand, while binary operators require two operands. Ternary operators involve three operands and include the conditional operator (?:).

 Special Symbols:

These symbols serve specific functions in C/C++:

- Brackets [] for array element reference.

- Braces {} mark the start and end of a code block.

- Comma (,) separates statements, e.g., in for loops.

- Semicolon (;) terminates statements.

- Parenthesis () indicate function parameters and calls.

- Asterisk (*) creates a pointer variable.

- Assignment Operator (=) assigns values.

- Preprocessor (#) is used with header files for program transformation before compilation.