Log mails send via php mail function

Here is a simple script which can log necessary information regarding mails send via php mail() function.

Create a file /usr/local/bin/sendmail-logger with the following code inside and correct the file ownership/permission to root/755

 
logger -p mail.info sendmail-logger: site=${HTTP_HOST}, 
client=${REMOTE_ADDR}, script=${SCRIPT_NAME}, pwd=${PWD}, uid=${UID}, 
user=$(whoami) 
/usr/sbin/sendmail -t -i $*

Create a file /usr/local/bin/php_set_envs.php with the following code inside

<? 
putenv("HTTP_HOST=".@$_SERVER["HTTP_HOST"]); 
putenv("SCRIPT_NAME=".@$_SERVER["SCRIPT_NAME"]); 
putenv("SCRIPT_FILENAME=".@$_SERVER["SCRIPT_FILENAME"]); 
putenv("DOCUMENT_ROOT=".@$_SERVER["DOCUMENT_ROOT"]); 
putenv("REMOTE_ADDR=".@$_SERVER["REMOTE_ADDR"]); 
?>

Edit the following values in php.ini

sendmail_path = /usr/local/bin/sendmail-logger 
auto_prepend_file = /usr/local/bin/php_set_envs.php

The log information will be written on /var/log/maillog

root@ [~]# grep sendmail-logger /var/log/maillog 
Nov 26 04:47:03 cpanel1 logger: sendmail-logger: site=11.22.33.44, 
client=11.11.11.11, script=/testmail.php, pwd=/usr/local/apache/htdocs, 
uid=99, user=nobody
Advertisements

Dmidecode

dmidecode is a tool for dumping a computers DMI (some say SMBIOS) table contents in a human-readable format. This table contains a description of the systems hardware components, as well as other useful pieces of information such as serial numbers and BIOS revision. You can retrieve this information without having to probe for the actual hardware.

Options are:

-d, --dev-mem FILE     Read memory from device FILE (default: /dev/mem)
-h, --help             Display this help text and exit
-q, --quiet            Less verbose output
-s, --string KEYWORD   Only display the value of the given DMI string
-t, --type TYPE        Only display the entries of given type
-u, --dump             Do not decode the entries
    --dump-bin FILE    Dump the DMI data to a binary file
    --from-dump FILE   Read the DMI data from a binary file
-V, --version          Display the version and exit

In the options we need to learn about –type

-t, --type TYPE --> Only display the entries of type TYPE. TYPE can be either a DMI type number, or a comma-separated list of type numbers, or a
keyword  from  the following list: bios, system, baseboard, chassis, processor, memory, cache, connector, slot

The complete list is pasted below.

     Type   Information
      ----------------------------------------
         0   BIOS
         1   System
         2   , type the following command 
         3   Chassis
         4   Processor
         5   Memory Controller
         6   Memory Module
         7   Cache
         8   Port Connector
         9   System Slots
        10   On Board Devices
        11   OEM Strings
        12   System Configuration Options
        13   BIOS Language
        14   Group Associations
        15   System Event Log
        16   Physical Memory Array
        17   Memory Device
        18   32-bit Memory Error
        19   Memory Array Mapped Address
        20   Memory Device Mapped Address
        21   Built-in Pointing Device
        22   Portable Battery
        23   System Reset
        24   Hardware Security
        25   System Power Controls
        26   Voltage Probe
        27   Cooling Device
        28   Temperature Probe
        29   Electrical Current Probe
        30   Out-of-band Remote Access
        31   Boot Integrity Services
        32   System Boot
        33   64-bit Memory Error
        34   Management Device
        35   Management Device Component
        36   Management Device Threshold Data
        37   Memory Channel
        38   IPMI Device
        39   Power Supply
        40   Additional Information
        41   Onboard Device

EG USAGE


If you want to get information about Base Board, type the following command

dmidecode -t 2
SMBIOS 2.3 present.
Handle 0x0002, DMI type 2, 8 bytes
Base Board Information
Manufacturer: Supermicro
Product Name: X5DPA-TGM+
Version: A1
Serial Number: 00000000

If you want get the full details information regarding your processor type the command “dmidecode -t 4” . You will get the information about all the processors. A sample output with one processor detail is pasted below.

dmidecode -t 4
 

Using dmidecode to find out what memory chips you have
 =========================================
dmidecode -t 16
#

This confirmed the ECC type is not defined and Maximum Capacity is 4GB. Here the number of device is 4 which means we have 4 slots to place the ram sticks.You will get the details of each memory device using the command dmidecode -t 17. Since you have 4 devices, you will get the details of each of the 4 devices.

A sample output is pasted below.

Memory Device
Array Handle: 0x001F
Error Information Handle: 0x001E
Total Width: 64 bits
Data Width: 64 bits
Size: 128 MB
Form Factor: DIMM
Set: None
Locator: DIMM4
Bank Locator: BANK1
Type: SDRAM
Type Detail: Synchronous
Speed: Unknown
Manufacturer: Manufacturer2
Serial Number: SerNum2
Asset Tag: AssetTagNum2
Part Number: PartNum2

Here you will see the details of a RAM chip like “type” which is SDRAM and size which is 128MB

In the same way, you will get information regarding bios, chasis, processor….

 

Errors

 =====

If you get the following error while running the command dmidecode

dmidecode

  1. dmidecode 2.9

/dev/mem: No such file or director

Do the following

mknod -m 660 /dev/mem c 1 1

Also check the permission of the file “/dev/mem”. It should be like the following

chown root:kmem /dev/mem

How to create SWAP space in Linux Servers

You need to use the dd command to create swap file. The mkswap command is used to set up a Linux swap area on a device or in a file.

Login as root user, and use the following command to create a swap file.

The following dd command example creates a swap file with the name “tmpDSK” under /backup directory with a size of 2000MB (2GB)

# dd if=/dev/zero of=/backup/tmpDSK bs=1M count=2000

Setup correct file permission for security reasons, enter:

# chown root:root  /backup/tmpDSK
# chmod 600 /backup/tmpDSK

Make this file as a swap file using mkswap command.

# mkswap /backup/tmpDSK
Setting up swapspace version 1, size = 2097147 kB

To make this swap file available as a swap area even after the reboot, add the following line to the /etc/fstab file.

/backup/tmpDSK       swap                     swap     defaults        0 0

Enable the newly created swapfile.

# swapon /backup/tmpDSK

# free -m
             total       used       free     shared    buffers     cached
Mem:          4050       3222        828          0        210       2589
-/+ buffers/cache:        422       3628
Swap:         1999          0       1999

DDOS attack measures

How do we confirm that the server is under DDOS attack?

We can confirm it by checking the result of netstat command:

netstat -an|awk '/tcp/ {print $6}'|sort|uniq -c

This will show the states and number of connections at that time. The different states that are visible mostly in servers are:

1. ESTABLISHED - This will be legitimate connections established to the server
2. SYN_SENT - The client will be actively attempting to establish a connection.
3. SYN_RECV - A connection request has been received from the network.
4. FIN_WAIT - The socket is closed, and the connection is shutting down.
5. TIME_WAIT - The socket is waiting after close to handle packets still in the network.
6. LISTEN - The socket is listening for incoming connections.
7. LAST_ACK - The remote end has shut down, and the socket is closed. Waiting for acknowledgement.

If the number of connections in SYN_SENT, SYN_RECV, TIME_WAIT, FIN_WAIT are very large in the rate of 1000s then the server is surely under attack.

As a first step we can tweak the values set for SYN_SENT, SYN_RECV, TIME_WAIT, FIN_WAIT in the file /etc/sysctl.conf. Reduce the value of net.ipv4.tcp_fin_timeout to 3 or 5. Normally it will be set to 120 as default. Make the following changes in /etc/sysctl.conf

# Enable TCP SYN cookie protection
net.ipv4.tcp_syncookies = 1
# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 3
# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0
# Turn off the tcp_sack
net.ipv4.tcp_sack = 0

Then execute the command :

sysctl -p

Then we will have to find out how the attack is being performed, is it from any particular IP or from large number of IP addresses to the server. If it is from any particular IP to the server, then we can fix it by blocking the IP in the firewall. If it is from a large number of IP with one or 2 connections then we will have to find more details to stop it. But will will not be able to completely stop the DDOS attack, we will have to tweak some settings in the server so that the number of connections can be reduced.

Once we reach the result that the server is under attack by checking the number of connections in different state, we need to find to which port the attack is being done. Suppose the number of connections in state SYN_RECV is large. Then we can get the details using the following command:

netstat -lpan | grep SYN_RECV | awk '{print $4}' | cut -d: -f2 | sort | uniq -c | sort -nk 1

The result will be the number of connections and the port open in the server. If the second field is 80 then the attack is to apache port.

In addition to the netstat command, you can use tcpdump command to find out if there is dos attack to a particular port.

tcpdump -nn -tttt -i any port 80

Similarly you can give different ports to find out to which port attack is being done. For example, port 53, 25 etc.

Once you understand the port you need to figure out is the attack done on a particular domain or IP. Suppose the attack is done on port 80, then we can tweak the apache settings as follows:

1. Increase the MaxClients so that we can prevent the condition of apache reaching its limit, since apache could not serve new requests. MaxClients can be set to a max value of the limit set in ServerLimit
2. Set KeepAlive on to set the KeepAliveTimeout
3. KeepAliveTimeout value to be reduced to 3 or 5

So the settings will be as follows:

MaxClients 500
KeepAlive On
KeepAliveTimeout 3
/etc/init.d/httpd restart

In order to narrow down the issue, we need to find out if the attack is on any particular IP in the server. This can be found using the following command:

netstat -lpan | grep SYN_RECV | awk '{print $4}' | cut -d: -f1 | sort | uniq -c | sort -nk 1

After confirming the attack to the IP, we need to find out if the attack is made to a particular domain in that IP or to the IP as a whole. For that, you can check the apache error logs or top command. If in the apache error logs, you are finding the errors for a particular domain, then you will have to perform steps to prevent attack to the domain. For that we can perform the following steps:

1. We can block the connections to the domain using modsecurity. CSF is connected to modsecurity so that if we write rule to block a domain, the IP from whcih connections to the domain are made will be blocked. Since it is DDOS attack, there will be many IPs connecting to the server and blocking high number of IP addresses can cause load in the server and thus server can go down. In order to prevent that, you will have to first block the checking of modsecurity in lfd.

In /etc/csf/csf.conf, set the following:

LF_MODSEC = "0"
csf -r

Then, in the modsecurity configuration file, you can add the following:

SecRule REQUEST_FILENAME|ARGS|ARGS_NAMES|REQUEST_HEADERS|!REQUEST_HEADERS:Referer "domain\.com"

2. You can block the acesses to port 80 of the domain in the firewall using the following command:

iptables -I INPUT -p tcp --dport 80 -m string --string "domain.com" --algo bm -j DROP

3. If the connections are still not getting reduced, then you can limit the number of connections to the domain using bandwidth module as follows:

/scripts/setbwlimit --domain=domain.com --limit=256000

By executing the above command, a file named /usr/local/apache/conf/userdata/std/2/account/domain.com/cp_bw_all_limit.conf will be created. The content of the file will be :

<IfModule mod_bw.c>
 ForceBandWidthModule On
 BandWidthModule On
 BandWidth all 256000
</IfModule>
<IfModule mod_bandwidth.c>
 ForceBandWidthModule On
 BandWidthModule On
 BandWidth all 256000
</IfModule>

Add a line “MaxConnection all 1” such that the number of connections will be limited to 1. So the contents will be as follows:

<IfModule mod_bw.c>
 MaxConnection all 1
 ForceBandWidthModule On
 BandWidthModule On
 BandWidth all 256000
</IfModule>
<IfModule mod_bandwidth.c>
 MaxConnection all 1
 ForceBandWidthModule On
 BandWidthModule On
 BandWidth all 256000
</IfModule>

4. If nothing helped, you can nullroute the IP using the command:

iptables -I INPUT -d XX.XX.XX.XX -j DROP

If the domain is having dedicated IP, then there is no need of above steps, you can directly make the IP down, by deleting the IP from the /etc/ips and restarting ipaliases. But in case of main shared IP, this cannot be done. We will have to reduce the TTL of the domains and change all the domains except the domain to which attack is being made to a free IP after 4 hours and then make the IP down after that so that the attack will be there for only 4 hours. But in such cases there will be issue with cpanel license etc. We will also have to make sure of the name server setting of the domain to which attack is being made. If the domain is using remote name servers, then we cant change any DNS setting of the domain in the server.

In order to prevent this in future, you can add the following commands:

iptables -A INPUT -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
iptables -A INPUT -p tcp --tcp-flags SYN,RST SYN,RST -j DROP
iptables -A INPUT -p tcp --tcp-flags FIN,RST FIN,RST -j DROP
iptables -A INPUT -p tcp --tcp-flags ACK,FIN FIN -j DROP

cgi script not working for all the users

Error: script not found or unable to stat: /usr/local/apache/cgi-bin/test.cgi

Solution: I added in httpd.conf, the following domain settings:

<IfModule alias_module>
ScriptAlias /cgi-bin/ /home/user/www/cgi-bin/
</IfModule>

It works!

Remove “eval(base64_decode” using linux commands from all php files across multiple WordPress

Assuming you have logged into a Linux Shell and already have BACKUP of all files (including infected files) lets move ahead!

Command to list all infected files:

grep -lr --include=*.php "eval(base64_decode" /path/to/webroot

This is not necessary but its better to check some files manually to confirm if they have malicious code we are looking for. Also we can use this command after running cleanup command to crosscheck if cleanup is really successful.

Command to remove malicious code:

If above command gives you correct output, execute following command to perform actual cleaning:

grep -lr --include=*.php "eval(base64_decode" /path/to/webroot | xargs sed -i.bak 's/<?php eval(base64_decode[^;]*;/<?php\n/g'

Executing above will remove eval(*) codes.  Above command will also generate a backup version of files it will modify. For example, if it removes code from index.php, you will find a new file index.php.bak in same directory with original content of index.php

Now after running above command, you still find some more infected files, then you need to adjust search and replace parameters in for “sed” part.    You may also use following command for a “liberal” cleaning at the risk of breaking something. (in case you really break something, like I did, you can jump to “Troubleshooting” section below!)

grep -lr --include=*.php "eval(base64_decode" /path/to/webroot | xargs sed -i.bak '/eval(base64_decode*/d'

Trying to avoid re-appearance of this code injection

Its really though to cover every possible way to protect yourself from such attach in this post.

If you remember, WordPress community faced this kind of issue because of WP-PhpMyAdmin plugin sometime back. In our case, we found some old WordPress demo sites were having that plugin installed.

To remove WP-PhpMyAdmin plugin form all WordPress sites on your server, execute following command:

find /path/to/webroot -name "wp-phpmyadmin" -type d | xargs rm -rf

Above is all we did to get rid of eval(base64_decode(*)) codes from all files on our test server. If this happens again on our server, I will update this post with added info.

Troubleshooting:

Just in case you end up in a mess, below are some useful commands.

Missing <?php tag in the beginning:

To add “<?php: tag in the beginning of index.php files, in case if you remove it accidentally use following command:

find /var/www/ -name "index.php" | grep "/htdocs/index.php" | xargs grep -L "<?php" | xargs sed -i "1s/^/<?php \n/"

Don’t worry. If you already have a “<?php ” tag in the beginning, it won’t be added again.

Extra Newlines at the top!

If you find after cleanup, extra newlines at the top of your code, then use following command to remove trailing newlines. Extra newlines creates problem for blog feeds.

find . -name '*.php' -exec sed -i -e :a -e '/^\n*$/{$d;N;ba' -e '}' '{}' \;

CentOS OpenVZ – how to secure tmp directory

First thing that we need to do is open up the fstab file for editing, we are going to use nano for this, however any editor will do the job.

nano -w /etc/fstab

Now we need to create a new line, so navigate to the bottom of the file using your arrow key’s and append the following line, I recommend copying & pasting to ensure you don’t get it wrong.

none /tmp tmpfs nodev,nosuid,noexec 0 0
mount -o remount /tmp

There is also another temp directory which is wise to secure (/var/tmp dir)
So make a backup (don’t skip this step, you need the files in a bit)

mv /var/tmp /var/tmpfiles

We can now make a link to map /tmp to /var/tmp

ln -s /tmp /var/tmp

Restore the files from the backup you made before

cp /var/tmpfiles/* /tmp/

Restore the files from the backup you made before, and make sure that the files in tmpfiles are now in tmp.

ls /var/tmpfiles
ls /var/tmp

If it looks ok, you can remove the tmpfiles directory.

rm -rf /var/tmpfiles

That’s it! You should now be a bit more secure!