Open up a terminal and then walk through the commands below, this will get you familiarised with the command line and how to navigate it.
make a new directory:
mkdir testfolder
change directory:
cd testfolder
Create an empty file:
touch test.txt
command line editors:
nano, vi, emacs
count number of words in a file:
wc filename
get manual on a command:
man wc
go to home folder:
cd ~
show current folder:
pwd
go to previous folder:
cd –
TAB is autocomplete
go to the start of a line:
ctrl+a
go to the end of a line:
ctrl+e
the up arrow shows your previous command.
go up one folder:
cd ..
delete directory (it must be empty):
rmdir testfolder
delete contents of a folder:
rm testfolder/*.*
move a file:
mv filename testfolder
move and rename:
mv filename testfolder/newfilename
copy a file:
cp folder1/filename folder2/
copy a contents of folder
cp -r folder1/* folder2/
copy a whole folder:
cp -r folder1 folder2/
recursively delete folder AND contents:
rm -rf testfolder
show all files:
ls -a
show details, human readable:
ls -lh
get fifth column from results using awk:
ls -lh | awk ‘{print $5}’
show in order of creation:
ls -ltr
show command history:
history
remove the line numbers:
history | cut -c 8-
Creating an alias:
edit the .bashrc file in your home folder
add the command with no spaces around the equals sign:
alias cmd=”nano ~/commands.txt”
alias rm=’rm -v -i’
alias open=’gvfs-open’
Search a directory for a file:
find . -name *.py
weird thing about find, you must use “” if there is something with that name in the folder.
find a word in a file:
grep ‘def’ *.*
the filesystem structure
/ The slash / character alone denotes the root of the filesystem tree.
/bin Stands for binaries and contains certain fundamental utilities, such as ls or cp
/boot Contains all the files needed for successful booting process.
/dev Stands for devices. Contains file representations of peripheral devices and pseudo-devices.
/etc Contains system-wide configuration files and system databases
/home Contains user home directories on Linux and some other systems. In the original version of Unix, home directories were in /usr instead.[15] Some systems use or have used different locations still: OS X has home directories in /Users, older versions of BSD put them in /u, FreeBSD has /usr/home.
/lib Originally essential libraries: C libraries, but not Fortran ones. On modern systems, it contains the shared libraries needed by programs in /bin, and possibly loadable kernel module or device drivers. Linux distributions may have variants /lib32 and /lib64 for multi-architecture support.
/media Default mount point for removable devices, such as USB sticks, media players, etc.
/mnt Stands for mount. Empty directory commonly used by system administrators as a temporary mount point.
/opt Contains locally installed software.
/proc procfs virtual filesystem showing information about processes as files.
/root The home directory for the superuser root – that is, the system administrator. This account’s home directory is usually on the initial filesystem, and hence not in /home (which may be a mount point for another filesystem) in case specific maintenance needs to be performed, during which other filesystems are not available. Such a case could occur, for example, if a hard disk drive suffers physical failures and cannot be properly mounted.
/sbin Stands for “system (or superuser) binaries” and contains fundamental utilities, such as init, usually needed to start, maintain and recover the system.
/srv Server data (data for services provided by system).
/sys In some Linux distributions, contains a sysfs virtual filesystem, containing information related to hardware and the operating system.
/tmp A place for temporary files not expected to survive a reboot. Many systems clear this directory upon startup or use tmpfs to implement it.
/usr The “user file system”: originally the directory holding user home directories,[15] but already by the Third Edition of Research Unix, ca. 1973, reused to split the operating system’s programs over two disks (one of them a 256K fixed-head drive) so that basic commands would either appear in /bin or /usr/bin.[17] It now holds executables, libraries, and shared resources that are not system critical, like the X Window System, KDE, Perl, etc. In older Unix systems, user home directories might still appear in /usr alongside directories containing programs, although by 1984 this depended on local customs.[13]
/include Stores the development headers used throughout the system. Header files are mostly used by the #include directive in C language, which historically is how the name of this directory was chosen.
/lib Stores the needed libraries and data files for programs stored within /usr or elsewhere.
/libexec Holds programs meant to be executed by other programs rather than by users directly. E.g., the Sendmail executable may be found in this directory.[18] Not present in the FHS until 2011;[19] Linux distributions have traditionally moved the contents of this directory into /usr/lib, where they also resided in 4.3BSD.
/local Resembles /usr in structure, but its subdirectories are used for additions not part of the operating system distribution, such as custom programs or files from a BSD Ports collection. Usually has subdirectories such as /usr/local/lib or /usr/local/bin.
/share Architecture-independent program data. On Linux and modern BSD derivatives, this directory has subdirectories such as man for manpages, that used to appear directly under /usr in older versions.
/var Stands for variable. A place for files that may change often – especially in size, for example e-mail sent to users on the system, or process-ID lock files.
/log Contains system log files.
/mail The place where all incoming mails are stored. Users (other than root) can access their own mail only. Often, this directory is a symbolic link to /var/spool/mail.
/spool Spool directory. Contains print jobs, mail spools and other queued tasks.
/tmp The /var/tmp directory is a place for temporary files which should be preserved between system reboots.
cat /proc/meminfo
cat /proc/cpuinfo
cat /proc/version
devices:
cat /dev/input/mouseX
cat /dev/input/byid/keyboard
run previous command as sudo:
sudo !!
The standard streams (stdout, stderr, stdin):
stdin(0): keyboard
stdout(1): normal output to screen
stderr(2): error output to screen
output something to a file:
ls > out.txt (this is the same as 1>)
append to a file:
ls >> out.txt
redirect errors to a file:
ls 2> err.txt
search the whole filesystem and redirect errors to nowhere
find / -name “*.py” 2> /dev/null
redirect stderr to stdout:
ls > out.txt 2>&1
2>&1 redirects the standard error to the standard output so they appear i
together and can be jointly redirected to a file. (Writing just 2>1 would
write it out to a file called 1
pipes: | take the output of one command and pipe it to the input of another
translate byte values into ascii
cat keyboard.txt | tr -cd ‘a-f0-9’
use tee to redirect to a file AND stdout:
ls | tee all.txt
get a random number from /dev:
cat /dev/urandom | tr -cd ‘a-f0-9’ | head -c 32
writing a script:
#!/bin/bash
var=”5″
echo “hello, $1, the value of variable is $var”
if [ $var -gt 8 ]; then
echo “var is greater than 8”
fi
another sample script:
#!/bin/bash
echo “converting $1 to $2”
for file in *.$1; do
filename=`echo $file | sed s/$1/$2/`
echo “converting $file to $filename”
#convert $file $filename
done
chmod +x script.sh
./script.sh
The & makes the command run in the background.
make command run in the background after executing:
ctrl+z
bg
creating a symlink
ln -s ~/filename
debugging stuff:
logs are located in /var/log/
show all driver messages:
dmesg
use less when opening a big file ( it prints to the screen as it reads):
less filename
use tail to read the end of a logfile (-n 100 reads the last 100 lines):
tail -n 100 filename
read a file and keep reading it (follow):
tail -n 100 -f filename
ssh, rsync and scp:
if you set up ssh server on your machine then install fail2ban
debugging it:
ssh [email protected] -v
tcptraceroute github.com 22
scp filename [email protected]:
if it fails to copy then you can resume it, the -u only copies new files:
rsync -u –partial –progress –rsh=ssh user@host:remote_file local_file
screen, a must for dodgy ssh connections:
screen – start a new screen session
Ctrl a c – Creates a new screen session so that you can use more than one screen session at once.
Ctrl a n – Switches to the next screen session (if you use more than one).
Ctrl a p – Switches to the previous screen session (if you use more than one).
Ctrl a d – Detaches a screen session (without killing the processes in it – they continue).
list all screens:
screen -ls
reattach to a screen:
screen -d
Edit your crontab file:
crontab -e
minute hour day-of-month month day-of-week command
every minute:
* * * * *
every day at 7pm:
0 19 * * *
run something at reboot:
@reboot root /var/kiosk/btest.sh
the cron jobs are logged to /var/log/syslog:
grep CRON /var/log/syslog
—————————-cool commands—————————-
show all the environment variables.
printenv | sort -h
show all processes:
top
show it with nice colors:
htop
list all processes:
ps -ef
kill maximum killage: kill all processes of this type: get exif data from images: combine mp3s together: AWK and SED! convert 4k video to 1080p: tar and untar: tar -xzvf file.tar.gz make a gif: ping a website: get network details: download a website: pull all links out of a webpage: echo “This is the body.”| mailx -s “mailx Test1” [email protected] :w !sudo tee % <- save a write protected file in VIM
show disk usage for a folder and subfolders, sorted by size:
du -h --max-depth=1 | sort -h
find all executable files:
find . -type f -executable -print
Add a prefix to all files in a folder:
rename 's/^/wide/' *
see where package contents are installed to:
dpkg -L remove all trailing white space trim the first 50 lines off a file: grep and replace using sed: Automatically pep8 all your code: change ownership of a folder, its contents and all subfolders: encrypt a file check which process is hogging the ram: Profile a piece of python code: No password ssh (from a to b):
kill -9
killall
exiftool imagename
or
exiv2 imagename
cat song1.mp3 song2.mp3 > song3.mp3
ls -l | awk ‘{n+=$5}END{print n}’ <-add up all the file sizes
cat *.dat | awk '{print$2}' <-concatenate files and read the second column
svn status | awk '{print $2}' | xargs svn add
sed s/day/night/ < moo.txt > new.txt <- replace day with night, write to file
sed -e 's/day/night/g' moo.txt > blah.txt <- s(ubsitiute) g(lobal) e(add script)
sed -i 's/foo/bar/' * <- i(use same file)
sed -i 's/foo/bar/g' <- global, all matches in a line replaced
show differences between two files:
diff file1 file2
or
meld file1 file2
create a video from images:
ls *.jpg > frames.txt
mencoder mf://@frames.txt -mf w=1920:h=1080:fps=20:type=jpg -ovc x264 -x264encopts subq=6:partitions=all:8x8dct:me=umh:frameref=5:bframes=3:b_pyramid=normal:weight_b -o a.avi
mencoder
tar -cvpf file.tar archive folder
c-reate v-erbose p-reservepermission f-ile
x-tract z-ip v-erbose f-ile take out the z for .tar
convert *.jpg pictures.gif
ping reddit.com
ifconfig (ipconfig in windows)
wget https://reddit.com
grep -Po ‘(?<=href=")[^"]*' index.html
wget -q -O- reddit.com/r/thedonald | grep -o -i 'Trump' | wc -w
get word count from pdf
pdftotext filename.pdf - | tr -d '.' | wc -w
restart from the commandline:
sudo shutdown -r now
backing up a database:
/usr/bin/mysqldump --opt -u
sed -i ‘s/[ \t]*$//’ “$1”
sed -i ‘1,50d’ filename
truncate last 50 lines:
head -n -50 myfile.txt
grep -l add_gene *.py | xargs sed -i ‘s/add_gene/add_extra/g’
find . -iname “*.py” | xargs -I filename autopep8 –in-place –aggressive –aggressive filename
also use yapf -i
user group
sudo chown -R byrnj:byrnj public_html/
gpg -c moo.txt
gpg moo.gpg
ps -e -o pid,vsz,comm= | sort -n -k 2
sort by second column:
sort -k 2
python -m cProfile -o profile.pyprof -s ‘time’ myScript.py
pyprof2calltree -i profile.pyprof -k
if you have not generated a key then (ONLY IF NO KEY):
Be careful of DSA vs RSA
ssh-keygen -t rsa <- generate key for autologin on machine a
otherwise just do this
ssh-copy-id -i ~/.ssh/id_rsa.pub name@server <- copy file across from machine a
cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys’ <- alternative way
if it doesnt work immediately then type:
ssh-add on machine a to register the keys
And thats it, you are now an expert 🙂