Tokens in C

What are Tokens in C?

Tokens in C language are the minor elements or the building blocks used to construct or develop together a C program. These tokens in C are meaningful to the compiler. The online C compiler breaks a program into the possible minor units known as tokens and proceeds further to the various stages of the compilation.

Example:

Every meaningful character, word, or symbol in this C program is a C token. Compiler groups together these characters of the program into tokens.

The compilation process:
C Program ---> Group characters into C tokens ---> Translate tokens into target code.

Uses of Tokens in C

The following are the uses of tokens:

• Tokens in C are building blocks which means a program can’t be created without tokens.
• Tokens are classified into various subcategories, which are very important like identifiers are used to give identity to a variable, keywords can be used to predefine things, operators can be used to perform operations and so on.
• Tokens like strings and special symbols play a major role in dealing with problems as strings are required to define something, and special symbols are used as a part of the syntax and many more.

Types of Tokens in C language

Tokens in C language can be classified as:

• Keywords
• Identifiers
• Constants
• Special Characters
• Strings
• Operators

As we move through the course of the article, we will study different types of tokens in C and their examples and syntax.

Keywords

Keywords in C language are the collection of pre-defined or reserved words. These are case-sensitive and written in lower cases. Their meaning and functionality are already known to the compiler.

We can't use these keywords as variable names or function names because by doing so, we are trying to assign a new meaning to the keyword, which is not allowed in C language. There are a total of 32 keywords supported by the C language:

Example:
Here, we are using int, char and auto keywords. We can simply use auto keyword to deduce the data type of any variable.

In this example, instead of using int and character array, we can just use auto keyword which will automatically identify the data type for storage purposes.

Identifiers

Identifiers in C are short and informative names that uniquely identify variables or function names.

These are user-defined words used for naming of functions, variables, structures, unions, arrays etc. These can be composed of lowercase letters, uppercase letters, underscore or digits, but the first character should be either an underscore or an alphabet.

There are some defined rules in C language for declaring identifiers:

• Identifiers shouldn't begin with any numerical digit and hence, the first character must be either an underscore or an alphabet.
• Identifiers are case-sensitive and hence, both lowercase and uppercase letters are distinct.
• The length of identifiers shouldn't be more than 31 characters.
• Commas and blank spaces are not allowed while declaring an identifier.
• Also, the most important rule is that we can't use keywords as identifiers because keywords in C language are reserved words for special purposes only.

Some Valid Identifiers:

The above examples are following all the essential rules for defining identifiers i.e, they are not started with a numerical digit, not a keyword and also, there is no blank space or any special operator.

Some invalid identifiers:

The above examples are not following all the essential rules for defining identifiers and hence, are invalid identifiers.

Constants

Constants are the variables whose values are fixed and can not be modified during the execution of a program once they are defined. They are also known as literals.

The constant variables in C can be initialized only once and their default value is zero. If we try to re-initialize or re-define any constant variable, then we will get a compilation error.

We can declare constants in C language using:

• const keyword Here, we are using the const keyword to declare a variable and assigning a value to it that can not be modified later.

• #define pre-processor Here, we are using #define pre-processor and constant ll will be an alias-name for long keyword.

Types of Constants in C Language

Constants in C language can be classified into two categories:

Primary Constants

Numeric -> Numeric constant is a negative or a positive numeric value that is either an integer constant or a fractional constant with an optional decimal point. Also, no space or any other special symbol is allowed.

Numeric Constants are sub-divided into two types:

• Integer
• Float

Example: 1.0, 5, 6, -2, -3.6

Character -> Character constants are formed when we represent a single character within single quotation marks ('').

They are further sub-divided into three types:

• Single Character
• String
• BackSlash

Example:

Logical -> Logical Constants in C consist of logical operators and can take either of the two values: true or false.

They are generally of two types:

• logical connectives
• quantifiers.

The equality predicate ('=) is also treated as a logical constant. Some symbols that are treated as logical constants are:

Secondary Constants

1. Arrays -> Arrays in C language are collection of elements of similar data type with unique index numbers for their accessing. The elements of an array are stored at contiguous memory locations.

Example: int A[10];
The name of the array A is a constant pointer to the first element of the array and hence, it can be considered as a const int*.

Array Declaration

Array Initialization

2. Pointer -> Pointer is a variable in C and its value denotes the address of a memory location. Pointers make it possible to return more than one value from the function.

Example of Pointer Declaration in C

3. Structure -> Structure is a user-defined data type in C which is used to store a collection of different kinds of data. We use the struct keyword to declare the structure in C. Variables inside the structure are called members of the structure.

Example of defining structure in C

4. Union -> Union is a user-defined data type in C which is used to store a collection of different kinds of data. We use union keyword to declare the union in C. Variables inside the union are called members of the union.

How Union is different from Structure? Union is similar to structure but it uses less memory in comparison to structure in C. ALso, with unions we can only store information in one field at a time.

Example of defining union in C

5. Enum -> Enumeration (or enum) is a user-defined data type in C which is mainly used to assign names to integral constants.

Example of declaring and initializing enum in C

Here, days is a variable of enum keyword and parameters in curly braces are called enumerators with different state values (0,1,2).

Special Characters in C

Special characters as the name suggests, are symbols in C language that have special meaning and can not be used for any other purpose. Let us now see their significance and purpose of use in C language.

Types of Special Characters in C

Square Brackets [ ]
The opening and closing square brackets represent single and multi-dimensional subscripts and they are used as array element reference for accessing array elements.
Example:

Simple Brackets ( )
The opening and closing circular brackets are used for function calling and function declaration.
Example:

Curly Braces { }
In C language, the curly braces are used to mark the start and end of a block of code containing executable logical statements.
Example:

Comma (,)
Commas are used to separate variables or more than one statement just like separating function parameters in a function call.
Example:

Pre-Processor / Hash (#)
It is a macro-processor that is automatically used by the compiler and denotes that we are using a header file.
Example:

Asterisk (*)
Asterisk symbols can be used for multiplication of variables and also for creating pointer variables. Example:

Tilde (~)
It is used as a destructor to free some space from the memory.
Example:

Period (.)
It is used to access members of a structure or a union.
Example:

Colon (:)
It is used as a part of conditional operator ( ? : ) in C language.
Example:

Semicolon (;)
It is known as a statement terminator and thus, each logical statement of C language must be ended with a semi-colon.
Example:

Assignment Operator (=)
It is used to assign values to a variable and is sometimes used for logical operation validation.
Example:

Strings in C

Strings in C are represented as an array of characters having null character '\0' at the end. This null character '\0' denotes the end of the string. Also, these strings are always enclosed within double quotes. The size of the string is basically the number of characters it contains and also, 1 byte memory space is always reserved for null character value.

Examples of Describing Strings in C

• Here using string[10] denotes that 10 bytes of memory space is allocated for holding the string value.
• When we declare char as “string []”, memory space will be allocated dynamically as per the requirements during the execution of the program.

Operators in C

Operators in C are special symbols used to perform specific functions, and data items on which they are applied upon are known as operands.
Depending upon the no. of operands, operators in C are classified as:

• Unary Operators: The operators who require only a single operand to act upon are known as unary operators. For example: increment operator (++), decrement operator (--), sizeof etc.

C Program to Illustrate the Use of Unary Operators:

Output:

• Binary Operators: The operators who require two operands to act upon are known as binary operators.

Binary operators are classified into:

1. Arithmetic Operators

2. Relational Operators

3. Logical Operators

4. Increment and Decrement Operators

5. Bitwise Operators

6. Assignment Operators

7. Special Miscellaneous Operators

C Program to illustrate the Use of Binary Operators:

Output:

• Ternary Operators: The operators which require three operands to act upon are known as binary operators. Conditional Operator (?) is known as ternary operator.

Syntax:

Here, if Expression1 evaluates to true, then expression2 will be assigned to the variable else expression3 will be considered upon false result.

Example:

Examples to Implement C Tokens

Now, we will look at some of the examples to understand the use and importance of tokens while programming in C language.

Example Keywords

In this C program, we will illustrate the use of different types of keywords available in C i.e, int, float, char, const, short, return. We will declare and initialize variables of different kinds with some values and will print them at the end.

Output:

Example 2. Switch

Switch is a reserved keyword in C language and here, in this C program we will illustrate the use of switch statements and will print the resultant value according to the user's input depending upon the defined cases.

Output:

Example Functions

In this C program, we will illustrate the use of different C tokens used alongside defining functions. Here, we have made two functions to get the area of square and area of circle respectively upon function calling with input as parameter value of the function.

Here, we have used circular brackets, # (pre-processor), keywords and some valid identifiers.

Output:

Example Typedef

Typedef in C is a reserved keyword and is used to provide an additional or an alias name for a particular data type. Likewise in this C program, we will illustrate the concept of Typedef.
Here, we have used typedef keyword along with structure definition and hence, we are using book as an alias name of Book1 to access and then print the initialized values of members of structure in C.

Output: