Python Tokens Explained

In my previous article, we covered how to declare variables in Python Editor Jupyter. Now, we will continue with more features and learn the concept of object-oriented programming in Python. In this article, we learn about the Python Tokens: Keywords, Identifiers, Literals and Operators.

Introduction 
 
We previously discussed variable declaration. Now we will see how to assign the variables and show exact implementation with different tokens. 
 

How to assign a variable in Python?

 
In variables, values are assigned by two methods,
  • Single assigned values
  • Multiple assigned values 
Example
 
Python🐍 Tokens 
 
In the example above:
 
a = 10
b = 20
c = 30
 
These are single values for the different variables, whereas
 
a,b,c = 10,20,30
 
This is a multiple value declaration in a variable.
 

Python Tokens

 
Tokens a very basic component of the source code. Characters are classified by four major categories:
  1. Keyword
  2. Identifier
  3. Literal
  4. Operator

Keyword

 
Keywords were discussed in the previous article.
 

Identifiers

 
Identifiers are names that you give to a variable, Class, or Function. There are certain rules for naming identifiers similar to the variable declaration rules, such as:
  • No special character except _  (underscore is used as an identifier)

    Example

    Python🐍 Tokens

  • Keywords are not used as identifiers.

    Example

    Python🐍 Tokens
  • The first character of an identifier should be _ underscored or a character, but a number is not valid for identifiers.

    Example
    Python🐍 Tokens

  • As we discussed, python is case sensitive. Therefore, if we declared Python and python, both are different identifier names.

    Example

    Python🐍 Tokens 

Literals

 
Literals are the data given in a variable or a constant.
 
There are four basic types of literals available in Python:
 
String Literals
 
These literals are formed by enclosing text in a quotation. Both single and double quotes are used.
 
Example
 
Python🐍 Tokens 
 
In the first line, p1 and p2 are declared in quotation marks with both single and double quotes respectively.
 
On the other hand, if we want to print something in pre-text format, then use triple quotes:’’’. Multiline identifiers print the text written between triple quotations.
 
Numeric Literals
 
Int, Long, Float, and Complex are different ways of storing numbers in Python. There is no special arrangement for storing large numbers in Python. There is no restricted bit for storing Integer number value; it can be stored up to the memory limit in Python.
 
Boolean Literals
 
Boolean literals have just two values: True or False. Remember that you cannot use them as identifiers.
 
Special Literals
 
Python has just a single special literal which is NONE, which specifies that a field is not created.
 
Note
 
Check out the attachment, PythonTokens.ipynb
 

Operators

 
These are specific characters which need to perform some specific task according to the functions. There are 7 types of operators available for Python.
 
Arithmetic Operator
 
There are two operands to perform the arithmetic operation. For example, 1+2, 1 and 2 are operands and + is an arithmetic operator.
 
There are different arithmetic operators used, such as:
+, -, /,*,%
 
Example
 
Python🐍 Tokens 
 
Assignment Operator
 
This is used to assign a value to a variable. The different assignment operators are, =,+=,-=,*=
 
Example
 
Python🐍 Tokens
 
Here, asop is a variable assigned by 10. Then we write asop+=10; which means asop=asop+
10; which is 10+10=20. So, we get an output of 20. The same for the next one, asop=asop*2;282=4. So, the output is 4 while we use *= this assignment operator.
 
Comparison Operator
 
The comparison operator is used to compare two values and return an output in the form of TRUE or FALSE. There are different comparison operators, such as, <,>,<=,>=,!=
 
Example
 
Python🐍 Tokens 
 
Logical Operator
 
Logical operators perform logical operations and return the values TRUE or FALSE. Logical operators include these three keywords: and, or, not
 
Example
 
Python🐍 Tokens
 
Bitwise Operator
 
These are the bitwise operators :&,|,<<,>>,~.
 
Example
 
Python🐍 Tokens 
 
Here, all the conversions are done on the bases of Binary to Decimal bit conversions. You can check the answers through the scientific calculators.
 
Identity Operator
 
It is used to share an identity to test two operands. There are two operators used, is, is not.
 
Example
 
Python🐍 Tokens 
 
It’s just used to check if operands are working properly or not.
 
Membership Operator
 
This operator is used to check if the member of the value is a sequence or not. It means sequence is list, almost the same as an array, tuple, or a string. These operators include: in, in not
 
Example
Python🐍 Tokens 
Note
 
Check out the attachment PythonOpr.ipynb
 
Tip
 
There is a keyboard shortcut for running code,
  • Ctrl-Enter runs the current cell and enters command mode.

Summary

 
In this article we have learned about the Python Tokens in detail with appropriate examples. These are the basic concepts that will be used in the detailed programming of  Python Applications. You can find the file attachments of the Python Tokens and Python Operators in .zip file formats.