Learn BigData With Me - Linux Basics

I am writing a series of articles on Big Data. The industry uses Linux or Unix mostly, for deploying their Big Data solutions. So, here I am covering some basics of Linux.

In Linux, most of the work is done with command prompt, and for this, we use the terminal. Other work is done with editors that are same as notepad in Windows.

Please find some useful commands and their usage.

Sno.	Command Prompt	Command	Use
1	user@machine:~$	pwd	Print working directory
2	user@machine:~$	ls	List
3	user@machine:~$	ls -a	All listing
4	user@machine:~$	ls -al	All listing having permission to user
5	user@machine:~$	ls -alh	Human readable a long listing like 4 KB size
6	user@machine:~$	cd <directory>	Change directory
7	user@machine:~$	mkdir <dirname>	Make directory
8	user@machine:~$	mkdir -p <coolfolder/child/nested>	To create complete path
9	user@machine:~$	ls Desktop/Coolfolder<tab>	To get data
10	user@machine:~$	rm -r coolfolder	Remove recursive
11	user@machine:~$	rm -I coolfile.txt	To notify for remove
12	user@machine:~$	man <command>	Help or knowledge
13	user@machine:~$	touch <filename>	To create a file
14	user@machine:~$	mv tet.txt somefolder/	To move file
15	user@machine:~$	cp somefolder/text.txt /testcopy.txt	To copy
16	user@machine:~$	mv test.txt newName.txt	To rename
17	user@machine:~$	cd ..	Move up a dir
18	user@machine:~$	cd ../..	Move two up
19	user@machine:~$	cd <home/username>
20	user@machine:~$	cd ~	To home dir
21	user@machine:~$	cat <filename>	To read file
22	user@machine:~$	cat <filename1> <filename2>	To concat two files output

Editors

nano <filename>
1. use ^X (ctrl+X) to exit, it will prompt to save the file.
2. ^W to search some data.
vi <filename>
1. for creating files.
2. for modifying & appending some text on already existing file.

For displaying the content of file

use :wq to save and exit.
:q! to exit without save.

Note

vi editor works on three modes,
a. Esc mode (Press Esc)
b. Insert mode (Press i)
c. Colon mode (press shift + ;)

emacs <filename>
1. Fast & mostly used by programmers
vim <filename>
1. Secure & mostly used by sysadmins

Cat command

This is one of the most important commands. It is mostly used for displaying the file contents but it has so many other uses too.

cat> for creating files
cat>> creating files if it is not exit or appending extra text if it is already exist

user@machine:~$ cat > file.txt

Write your content here.

Press ctrl+d to save and exit
Press ctrl+c exiting file without saving
But you can’t change the content in previous line

Most of us are familiar with Windows OS. Linux is also a good platform. If you start working on it, you will realize it.

Note

Most of the issues in Big Data come from the Linux security section. I will try to cover this too. Today, we will learn new topics under Linux basics. Be careful, Linux is case sensitive.

echo command

This is used to print some statements.

Eg. user@machine:~$echo “This is sample statement by vimal”

We have few concepts for defining and saving standard input, output, and errors.

0 – for standard input
1 – for standard output
2 – for standard error

Please look at the below image to understand it.

Mail command

To send mail, we will use mail command.

user@machine:~$ mail –s”This is subject” user<messagebody.txt

grep command

grep is a search, finding, and filtering tool. It can do things like pattern matching & mapping.

user@machine:~$ grep sometext file.txt
user@machine:~$ grep sometext ./*
user@machine:~$ grep sometext ./* | unique | cut –d: -f1

Pipes & Directional operators

user@machine:~$Program1 | Program2
- This means output of program1 works as input for program2
- user@machine:~$ ps aux (to check processes)
- user@machine:~$ ps aux | less | uniq | sort
user@machine:~$Program1 && Program2
- This means when program1 true then program2 execute
- user@machine:~$ ls filenotexist.txt && echo “Unsuccessful”

Package management with apt-get

apt-get mainly used for installation of softwares, packages,

user@machine:~$ apt-get update
- It may prompt permission denied. To avoid this we will use-
user@machine:~$ sudo apt-get update
user@machine:~$ sudo apt-get upgrade
user@machine:~$ sudo apt-get install <applicationname>
- To install any software or application
user@machine:~$ apt-cache search editor
- To find editor
user@machine:~$ sudo apt-get remove <applicationname>
- To remove software or application