Python Tokens Explained


We previously discussed the variable declaration. Now we will see how to assign the variables and show exact implementation with different tokens. 

How to assign a variable in Python?

In variables, values are assigned by two methods,
  • Single assigned values
  • Multiple assigned values 
Python🐍 Tokens
In the example above
  1. a = 10  
  2. b = 20  
  3. c = 30 
These are single values for the different variables, whereas
  1. a,b,c = 10,20,30 
This is a multiple value declaration in a variable.

Python Tokens

Tokens a very basic component of the source code. Characters are classified by four major categories:
  1. Keyword
  2. Identifier
  3. Literal
  4. Operator


Keywords were discussed in the previous article.


Identifiers are names that you give to a variable, Class, or Function. There are certain rules for naming identifiers similar to the variable declaration rules, such as:
  • No special character except _  (underscore is used as an identifier)
    Python🐍 Tokens
  • Keywords are not used as identifiers.
    Python🐍 Tokens
  • The first character of an identifier should be _ underscored or a character, but a number is not valid for identifiers.
    Python🐍 Tokens
  • As we discussed, python is case sensitive. Therefore, if we declared Python and python, both are different identifier names.
    Python🐍 Tokens


Literals are the data given in a variable or a constant.
There are four basic types of literals available in Python:

String Literals

These literals are formed by enclosing text in a quotation. Both single and double quotes are used.
Python🐍 Tokens
In the first line, p1 and p2 are declared in quotation marks with both single and double quotes respectively.
On the other hand, if we want to print something in pre-text format, then use triple quotes:’’’. Multiline identifiers print the text written between triple quotations.

Numeric Literals

Int, Long, Float, and Complex are different ways of storing numbers in Python. There is no special arrangement for storing large numbers in Python. There is no restricted bit for storing Integer number value; it can be stored up to the memory limit in Python.

Boolean Literals

Boolean literals have just two values: True or False. Remember that you cannot use them as identifiers.

Special Literals

Python has just a single special literal which is NONE, which specifies that a field is not created.
Check out the attachment, PythonTokens.ipynb


These are specific characters which need to perform some specific task according to the functions. There are 7 types of operators available for Python.

Arithmetic Operator

There are two operands to perform the arithmetic operation. For example, 1+2, 1 and 2 are operands and + is an arithmetic operator.
There are different arithmetic operators used, such as
+, -, /,*,%
Python🐍 Tokens

Assignment Operator

This is used to assign a value to a variable. The different assignment operators are, =,+=,-=,*=
Python🐍 Tokens
Here, asop is a variable assigned by 10. Then we write asop+=10; which means asop=asop+10; which is 10+10=20. So, we get an output of 20. The same for the next one, asop=asop*2;282=4. So, the output is 4 while we use *= this assignment operator.

Comparison Operator

The comparison operator is used to compare two values and return output in the form of TRUE or FALSE. There are different comparison operators, such as, <,>,<=,>=,!=
Python🐍 Tokens 

Logical Operator

Logical operators perform logical operations and return the values TRUE or FALSE. Logical operators include these three keywords: and, or, not
Python🐍 Tokens

Bitwise Operator

These are the bitwise operators :&,|,<<,>>,~.
Python🐍 Tokens 
Here, all the conversions are done on the bases of Binary to Decimal bit conversions. You can check the answers through the scientific calculators.

Identity Operator

It is used to share identity to test two operands. There are two operators used, is, is not.
Python🐍 Tokens 
It’s just used to check if operands are working properly or not.

Membership Operator

This operator is used to check if the member of the value is a sequence or not. It means sequence is list, almost the same as an array, tuple, or a string. These operators include: in, in not
Python🐍 Tokens 
Check out the attachment PythonOpr.ipynb
There is a keyboard shortcut for running code,
  • Ctrl-Enter runs the current cell and enters command mode.


In this article, we have learned about the Python Tokens in detail with appropriate examples. These are the basic concepts that will be used in the detailed programming of  Python Applications. You can find the file attachments of the Python Tokens and Python Operators in .zip file formats.