Tips and Tricks

“grep” is a useful Linux tool allowing the user to print lines matching a specific search pattern. Most popular uses is by “piping” the output of one command to “grep” (such as cat /path/to/file | grep findme), or by directly looking for a pattern inside one or more file (e.g. grep findme /path/to/*.txt).

This would print lines that match the search string / expression. However, how to colorize the matching patterns? Simple! Use the --color=always flag, i.e.:

grep --color=always -i findme /path/to/*.txt

Next, if you’re doing lots of “grep” work and don’t want to keep specifying the color flag, you can opt for the following:

Add it to the GREP_OPTIONS environment variable

export GREP_OPTIONS=’--color=always’

Or, if you’re using bash as your shell, add it to /etc/bashrc. Simply edit that file using your preferred editor (e.g. vim, nano, etc.), and add the above export command at the very end of the file. When done, save and exite the editor, then issue the following command, so that the changes take effect: source /etc/bashrc

SSH is sometimes the only mode of access you have to a server. So how do you instantly determine the distro in use?

The following would work for most popular distributions (RedHat, CentOS, Ubuntu, etc.):

cat /etc/*-release

If the above returns “No such file or directory”, try the following which would work on Debian-based distros:

lsb_release -a

Finally, for kernel-related data, use the following command:

uname -a

You’ll be able to determine the kernel version and release date, whether it’s a 32 or 64 bit hardware and the current machine hostname.

You might have come across a difficulty in deleting a file in Linux whose name starts with a dash/hyphen.

Adding an escape character (e.g. rm -rf \-filename) or quoting the file name (e.g. rm -rf “-filename”) will not do the job.

The solution is simple: explicitly provide the full or relative path to the file. e.g.:

rm -rf /path/to/-file

or the following – if for instance your current working directory is the one containing the file in question

rm -rf ./-file

The ‘find’ utility is quite handy when used with the proper parameters. Below is a few ways to find files/folders based on the time they were accessed or modified.

Example 1:

find /path/to/folder -type f -name “*.txt” -mtime -5

This will find ‘files’ (-type f) within the folder /path/to/folder (or one of its sub directories), whose name ends with ‘.txt’, and which were modified (mtime) less than 5 days ago

Example 2:

find /path/to/folder -type d -name “stat*” -mtime +10

This would find ‘directories’ (-type d) within the folder /path/to/folder (or one of its subdirectories), whose name starts with ‘stat’, and which were modified more than 10 days ago

Example 3:

find . -iname “*.pdf” -mtime +2 -mtime -7

This would help you find files or folders in the ‘current directory’ (.), whose name ends with ‘.pdf’ regardeless of the case of that suffix (e.g. example.PDf would be matched), which were modified more than 2 days and less than 7 days ago (i.e. between 3 to 6 days ago).

Example 4:

find / -type f -atime -5 -size +100k

This example makes use of atime, i.e. time of access (as opposed to mtime, which is the time the file was modified). The above command would help you find files in your entire system (/ being the system root, unless your shell is jailed), which were accessed less than 5 days ago, and which are at least 100 kbytes in size.
PS: if you do not specify the ‘b’, ‘k’, ‘m’, etc., the default would be blocks of 512 bytes. For instance, using +100 instead of +100k, means you’re looking for files sized at least 100*512=51200 bytes (50 kbytes).

More examples to be added…

IPTables allows you to easily setup rules for packet filtering/forwarding.

So, to keep it short and simple: assume you’d like to forward any traffic coming to your machine (192.168.0.1) on port 80 to machine2 (192.168.0.2) on port 8080 then:

- Enable port forwarding:

echo 1 > /proc/sys/net/ipv4/ip_forward

- Now add the rules:

iptables -t nat -A PREROUTING -p tcp -d 192.168.0.1 –dport 80 -j DNAT –to 192.168.0.2:8080

iptables -t nat -A POSTROUTING -d 192.168.0.2 -j MASQUERADE

The -p tcp flag specifies that the protocol is TCP (as opposed to UDP, ICMP for example).

The -d 192.168.0.1 flag specifies that the packets was destined to IP 192.168.0.1

The –dport 80 means that the packet had port 80 as destination

-j is the target of the rule being enforced, NAT (network address translation). In more advanced rules, one can specify a predefined iptables chain of rules.

–to will specify to where this packet should be forwarded, and on which port

As for the second rule, it says that outgoing packets to the second machine should be masqueraded. In other terms, the user trying to access the original machine on port 80 will not have a feel that his request was forwarded. The packet will show up as if it came from the masquerading host, while in practice, the request was forwarded to machine2, and the reply came back from that same machine.

On a side note, make sure to add these iptables rules to a file, give it an execution permission (chmod +x /path/to/file), and add it to /etc/rc.local so that it is executed on every system boot. Otherwise, the above rules will be fliushed every time your system is restarted.

Whenever you upgrade your MySQL installation, make sure to upgrade any client that uses it.

In some cases, clients that use a version prior to 4.1 will have a problem authenticating against the MySQL database if the latter has a post 4.1 version.

The trick is that after 4.1 (i.e. 4.11 and up), MySQL changed the way it stores the passwords in the user table inside the mysql system database.
Password hashes are now 41 bytes long instead of the old 16 bytes.

So for example, if your MySQL server is 5.0, while your php-mysql library is 4.1, your web applications will fail to connect to the database. As such, it is recommended that you upgrade the client.

In any case, MySQL offers a way to change the hash back into the old format. For the sake of argument, assume the user in question is john, and you want to be able to connect using password dummy. In this case, connect to your MySQL server from the prompt (SSH and use ‘mysql -u root -p mysql’ on linux, or go to your mysql/bin windows directory and execute the same query), then issue the following queries:

update user set Password=OLD_PASSWORD(‘dummy’) where User=’john’;
flush privileges;

the OLD_PASSWORD() function will generate the old 16 bytes hash. The first query will eventually update the user password to use this hash. The second query is necessary in order for the MySQL service to re-read the new user privileges.

PS: if your root password is not working, refer as well to the guide on resetting it.

Alternatively, if your database has many users and you didn’t keep track of them, you can use the following query and it will return usernames that are using the new hash

select distinct(User) from User where LENGTH(Password)!=41;

If you’re a regular vi user, you may have noticed that some files, when being edited in vi, contain ^M characters at line ends.

This usually happens when you edit a file using certain windows-editors, then transfer it to your *nix machine.
Luckily, it is easily to get rid of this control character. While in vi, execute the following command:

:1,$s/^M//g

Important note: do not manually type a caret then the capital M character. Actually, in order to type ^M, press CTRL+V followed by CTRL+M.

A quick note:  the above command will look for the ^M character starting on line 1, substituting it ($s) with nothing (thus having the two consecutive forward slashes / with nothing in between). And this substitution is performed globally (g).

Sometime you may want to replace occurences of a string across multiple files.

There is an easy way to do so with the help of the vi editor.

This example will illustrate the power of vi:

Suppose you have 100 .html files, and you want to replace the occurence of the string ’2007′ with ’2008′.

As such, execute the following at your prompt:

vi *.html

This will open all the files ending with ‘.html’ in your current working directory. Then issue the following command:

:argdo %s/2007/2008/g | wq

That’s it! the above command will loop over each file, replacing (%) the word 2007 with the word 2008 globally (g) then it will save each file (w) and exit from it (q).

Note that the string to find and the string to use for replacement can be replaced with regular expressions. For instance, a caret (^) refers to the start of a line, the dollar sign ($) refers to its end, etc. So if we were to replace any line starting with ‘sample’ and ending with ‘test’, one could use:

:argdo %s/^sample.*test$/g | wc

where .* matches anything in between the two string.

Note too that you may do the find and replace action without respect to the case (i.e. a search for ‘word’ will match ‘WoRd’). For that matter, simply replace ‘/g’ with ‘/gi’

Some people may not be very knowledgeable about *nix file permissions, and what they mean, so this is intended to be some sort of a guide.

Any file or folder has a permission associated to it. Look at it as: ‘who has the power over this file/folder, and what kind of power does he have’.

There are 3 types of permissions:

• read
• write
• execute

These permissions apply to 3 groups of people:

• The file/folder owner
• The group to which the owner of this file/folder belongs
• The rest of the crowd, called world.

So suppose you have a file ‘x’, then this file x will certainly have:

• a set of permissions defining what its owner can do with it
• a set of permissions defining what users that belong to the same group as the owner can do
• what other people that do not fit int he first 2 categories can do

You can assign a single permission (e.g. read) or a set of permissions (e.g. read, execute) to this file/folder.

As an instance, you can allow the owner to do whatever he wants with the file (read,write,execute), allow the group to which the owner belongs to simply read the file, and disallow everyone else on the system from doing anything.

Now, to make it easier for you, and to not get into how these numbers are calculated, just memorize the following:

• read equals to 4
• write equals to 2
• execute equals to 1

Let’s get back to the example listed above.

• We wanted the owner to have full permissions, this means the permission for the owner should be: 4 (read) + 2 (write) + 1 (execute) = 7
• We wanted to give the group read and write permissions, meaning 4+2=6
• We wanted to deny complete access to the world, so that’s a 0

So, the permission for all the 3 types of users we mentioned earlier sums up to: 750

Keep in mind, you use the digit corresponding to the owner’s permissions first (7) , then the group (5) then to world.

Pretty simple, eh?

How do you set such a permission for a certain file ‘x’? Simply execute at your shell prompt chmod 755 x

Let’s do another exercise but backward this time. If i were to tell you that by default, a new created file would have a permission of 644, what would the permission of each of the 3 types of users be?

6 is basically 4+2 (you can’t get it any other way, remember you only have 3 numbers: 4,2 and 1 which would potentially take part of making up this number). This means read and write.
4 is well, 4 … that’s a simple read

So a 644 permission is a read and write permission for this file/folder’s owner, and a read permission for the owner’s group users and for the rest of the world.

One question arises though: what is the difference between the permissions we set to a file and those we set to a folder?

Basically:

• A read permission on a file means we can see its content, while a read permission on a folder means we can list its content (get the list of files and folders it contains).
• A write permission on a file means we can alter its content, while a write permission on a folder means we can create, and modify files and folders under it
• An execute permission on a file means we can literally execute it (e.g. shell script), while an execute permission on a folder means we can browse into it.

A final point:

If you get a long listing of the files/folders on a *nix system, you may as an example notice the following:

-rwxr-xr-x

This is the textual representation of the numbered permissions I explained above. The dash – means ‘no’. ‘r’ means read, ‘w’ means write, ‘x’ means execute. The very first bit would be an indication whether this is a file (-) or a folder (d).

Again, we go by the owner, group, world sequence. So the above example indicates that this is a file, owner has read, write, execute permission, group has read, execute permission, world has read,execute permission.

Hope this guide was simple yet beneficial to the readers.

Many users run small networks, be it at home or work. One of the widely used techniques for sharing the internet connection over a small local area network is Microsoft’s ICS (Internet Connection Sharing).

What if the main server is not running windows? Some prefer to opt for linux, and run several services that benefit client computers on this network (such as dns, dhcp, samba, NIS, etc…)

The good news for linux users is that they can benefit from the same functionality of ICS, but using linux iptables.

The following is a sample rule:

/sbin/iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

Note that you will need to have ip forwarding enabled.

RedHat/Fedora users can execute the following:

echo 1 > /proc/sys/net/ipv4/ip_forward

Now, these rules can be saved in a .sh file, make that shell script executable (chmod +x filename.sh)

Then, make this file execute whenever your system boots. An example would be editing /etc/rc.local and setting the path to that file in there.

This simple iptables rule can be expanded in order to allow certain ports, block others, check protocols for incoming packets on such ports (tcp, udp) , etc.

Advanced rules to come later hopefully.