Simple Text Processing

Simple Text Processing

Count Number of Files in Each Subdirectory

START=$HOME   
# change your directory to command line if passed 
# otherwise use home directory  
[ $# -eq 1 ] && START=$1 || :   if [ ! -d $START ] then
    echo "$START not a directory!"
    exit 1
fi
 
# use find command to get all subdirs name in DIRS variable
DIRS=$(find "$START" -type d)
 

 
[toc hidden:1]


Count Number of Files in Each Subdirectory

START=$HOME   
# change your directory to command line if passed 
# otherwise use home directory  
[ $# -eq 1 ] && START=$1 || :   if [ ! -d $START ] then
    echo "$START not a directory!"
    exit 1
fi
 
# use find command to get all subdirs name in DIRS variable
DIRS=$(find "$START" -type d)
 
# loop thought each dir to get the number of files in each of subdir
for d in $DIRS
do
   [ "$d" != "." -a "$d" != ".." ] &&  echo "$d dirctory has $(ls -l $d | wc -l) files" || :
done

Count the number of lines in a file or a set of files

Count lines in one file ignoring blank lines:

$ cat myfile.txt | sed '/^\s*$/d' | wc -l

Count lines in a set of files (in a given directory) ignoring blank lines:
$ cat mydir/* | sed '/^\s*$/d' | wc -l

 

Count number of words in a file

Count list of open files per user

/usr/sbin/lsof | grep 'user' | awk '{print $NF}' | sort | wc -l

Convert a text file to all uppercase/lowercase

Compress entire directory with files

compress

$ tar -zcvf archive-name.tar.gz directory-name

uncompress

$ tar -zxvf archive-name.tar.gz

-z: Compress archive using gzip program 
-c: Create archive
 -v: Verbose i.e display progress while creating archive
 -f: Archive File name

 

Delete line feed and Return in a text/html file

Remove all html tags from file

Export a mysql dump

mysqldump -u username -ppassword database_name > dump.sql

Remove empty lines in a file

Select specific lines from file

 

Sort contents of textfile

Generate counts of words in descending order of term frequency

 

Move files from one extension to another

Import a MySQL dump

 

SSH Tunnel to connect to a remote MySQL Server

Simply open a bash prompt on Cygwin and type:

Replace your-server with the machine name of the MySQL server. Now, it's possible to connect to the MySQL database using 127.0.0.1:5000 using a mysql client.

 

References

CS 410 Text Information Systems