Ensim add-ons and utilities by Max @

Ensim add-ons and utilities by Max @ . While a few of these are purely examples of how to use the Ensim API, most provide additional command-line functionality that we at find very useful for day-to-day management and activity analysis on Ensim-based web hosts (some work on non-Ensim hosts as well). All are based on what I have learned from using Ensim. Feel free to mail any questions, comments, and/or suggestions to us at scripts@wwd-hosting.net.



Do you like these scripts? Have they saved you time the way they save us time? We don't require anyone to pay for any of these, but if you feel they have helped you, how about donating a few dollars towards future enhancements and new scripts? We always welcome your suggestions/feedback! Any amount, regardless of how small, is welcome! No obligation.

Thank you if you do decide to contribute!!

Please view the contributor list!


NOTE


If you are using the now-deprecated Ensim 3.1 you will need to change the first line of any of these Ensim API/python scripts (this includes the quota checking script) from

#!/usr/bin/python2.2

to

#!/usr/bin/python2.1

As Ensim 3.5 and later uses a newer version of python than 3.1.x did.

If this is the case, you will see an error message like this when you try to run one of these scripts:

bad interpreter: No such file or directory

If any scripts you have been using on 3.1.x break on 3.5.x or 4.x for reasons other than the path issue above, please let me know! You can contact me via the EV1 forums; my username on the forums is perldork.

List Of Scripts

Daily Admin Report Quota Checker And Alerter Virtual Host Command Runner
Virtual Host Command Utility HTTP Access Log Searcher Sendmail Log Search Utility
Sendmail Transaction Search And Display Utility Sendmail Mail Queue Info Utility Sendmail Mail Queue Cleaner Utility
Virtual Host Customer Mailer Virtual Host SSH Login Viewer Virtual Host Sendmail Log Appender
Enhanced Virtual Cron Control Script HTTPS Backup Trigger Utility Virtual Host Process Viewer
View Site Bandwidth Usage Utility Virtual Host Bandwidth Summary Utility Virtual Host Quota Viewer
Virtual Host User Quota Viewer HTTP Site Hit Summary Utility Last Login Viewer
Snort Summary Report API Example: Reseller Control API Example: Get Site Information
API Example: Get Sites For a Reseller API Example: Get Reseller By Domain API Example: Concise Bandwidth Report
Web Hosting Account Partitioning Utility    

Downloads


Quota Checker And Alerter

Download: ww_quota_check.tar.gz
Ensim only?Yes

Quota checker! This will check all quotas on your host based on thresholds you define:

  1. Checks user quotas against your thresholds; notifies user if they have exceeded a user threshold.
  2. Checks site quotas against your thresholds; notifies site admin if they have exceeded a site threshold.
  3. Notifies reseller with list of sites that have exceeded thresholds.
  4. Notifies site admins with list of users that have exceeded thresholds.
  5. Includes support for Unicode character sets in templates!

Emails are only sent to admins if user/site thresholds were exceeded. All emails are XML template based.

To install (as root):

# cd /usr/local/src/
# wget http:///ww_quota_check.tar.gz
# tar zxvf ./ww_quota_check.tar.gz
# cd ./ww_quota_check-0.93
# less README
# ./install.sh

Please please read the README file before installing :).


Virtual Host Command Runner

Download: vrun
Ensim only?Yes

Run a command over one or more virtual hosts, as root or as the site admin for each host. The script chroots you to the virtual domain before running the command; by default it runs the command line you specify on every host on your machine. Use the '--on' option to specify a single host.

Example:  see who has queued mail on every host you host

# vrun --verbose ls /var/mail/mqueue/

Example 2:  Run the pine command line as the admin for
            site foobar.com.

#  vrun --as-admin --on foobar.com pine

Example 3:  Do a less on the access log for site bar.net

# vrun --on bar.com less /var/log/httpd/access_log


Virtual Host Command Utility

Download: vdo
Ensim only?Yes

Run a command over all virtual hosts. Slightly different than vrun as this does NOT chroot to the site root first. Useful for doing things like copying files from your main site to all user sites. vdo substitutes the following printf-like codes into virtual domain significant variables.

  1. %d == domain name for current domain
  2. %D == Full path to virtual domain using /home/virtual/domain-name
  3. %s == site handle (e.g. site2)
  4. %S == Full path to virtual domain root using site handle .. e.g. /home/virtual/site2/fst
Example:  Copy /etc/hosts to all virtual hosts

# vdo cp /etc/hosts %D/etc/hosts

Now change  to the name of the local domain
for each of the new copies

# vdo perl -p -i -e 's//%d/g' %D/etc/hosts

Example 2:  See all last logins for each virtual host

# vdo last -f %D/var/log/wtmp


HTTP Access Log Searcher

Download: vsearch_access_logs
Ensim only?No

Search httpd access logs across all virtual hosts or a specific one for URI patterns; search by day, date pattern, HTTP status code, action (GET/POST), and optionally have the script resolve IP addresses to host names.

Output shows a count of hosts that accessed the file, along with several other fields that can be turned on/off (date, status, action, matched URI) .. entries can be sorted by several fields and are always grouped by line so that repeated lines show only once along with a count of their occurrences.

Usage and examples (as root):

vsearch_access_logs [--on-host host] [--resolve] 
                    [--status code] [--hide-status] \
                    [--match-user regex] [--user-length N] [--all-users] [--show-user] \
                    [--action name] [--hide-action] \
                    [--show-referrer] [--date-pat pat] \
                    [--on-day pat] [--hide-date] \
                    [--sort-by [host|date|action|match|status|referrer]] \
                    [--hide-match] [--match-length N] \
                    [--output-separator] \
                    [--hide-host] pat1..patn
vsearch_access_logs --help (this screen)

Example 1:  Search all host access logs for cmd.exe and 
            default.ida attempts.  Just show IP addresses
            so we don't spend all resolving IPs! ;)

vsearch_access_logs --on-day today index.html cmd.exe default.ida

Example 2:  How many times have visitors attempted formmail (we
            want to see host names if we can!)

vsearch_access_logs --resolve formmail.pl

Example 3:  How many times has package.tar.gz been downloaded
            from host foo.com?  Again, lets see the hosts if
            we can get em

vsearch_access_logs --resolve --on-host foo.com package.tar.gz

Example 4:  How many 404s for today?

vsearch_access_logs --resolve --on-host foo.com --status 404 --on-day today

Example 5:  Lets see March log entries, sorted by URI, for requests
            that resulted in a 404.

vsearch_access_logs --sort-by match --date-pat Mar --status 404

Example 6:  Show all users [HTTP AUTH] that have downloaded .zip files today

vsearch_access_logs --all-users --hide-host --hide-status --match-length 100 --hide-date --hide-action .zip

[root@ensim sbin]# ./vsearch_access_logs --on-host  \
                   --on-day 'last wednesday' 'cmd.exe|default.ida'
Query:   Pattern "cmd.exe|default.ida" on , sorted by # unique matches
Filters: (any action) 02/Apr/2003 (any status)

1   211.158.13.68   GET  /scripts/..%c1%1c../winnt/system32/cmd.exe   404  04/02
1   211.158.13.68   GET  /scripts/..%c0%2f../winnt/system32/cmd.exe   404  04/02
1   211.158.13.68   GET  /scripts/..%c1%af../winnt/system32/cmd.exe   404  04/02
1   211.158.13.68   GET  /scripts/..%c0%af../winnt/system32/cmd.exe   404  04/02
1   211.158.13.68   GET  /scripts/..%f8%80%80%80%af../winnt/system32/ 404  04/02
1   211.158.13.68   GET  /scripts/..%c1%9c../winnt/system32/cmd.exe   404  04/02
1   211.158.13.68   GET  /scripts/..%f0%80%80%af../winnt/system32/cmd 404  04/02
1   207.192.227.245 GET  default.ida?XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 404  04/02
1   211.158.13.68   GET  /scripts/..%fc%80%80%80%80%af../winnt/system 404  04/02
1   211.158.13.68   GET  /scripts/..%e0%80%af../winnt/system32/cmd.ex 404  04/02
1   211.158.13.68   GET  /scripts/..%c1%9f../winnt/system32/cmd.exe   404  04/02
1   211.158.13.68   GET  /scripts/..%c0%9f../winnt/system32/cmd.exe   404  04/02
----------------
2 unique hosts
12 total matches

====================================================

[root@ensim admin]# vsearch_access_logs  '/ensim/\\w' --hide-status \
                    --status 200 --hide-action  --on-host   \
                    --hide-host --hide-date   
Query:   Pattern "/ensim/\w" on , sorted by # unique matches
Filters: (any action) (any date) (200)

57   /ensim/mails_per_day                        
50   /ensim/ww_quota_check.tar.gz                
40   /ensim/vrun                                 
33   /ensim/vlast                                
20   /ensim/site_bw_usage.py                     
18   /ensim/showbw                               
15   /ensim/vusers.tar                           
15   /ensim/virtual_sendmail_logs.tar            
13   /ensim/get_reseller_sites.py                
12   /ensim/https_backup.tar                     
11   /ensim/reseller_control.py                  
10   /ensim/get_reseller_by_domain.py            
9    /ensim/vsearch_access_logs                  
----------------
13 unique entries
303 total matches

====================================================


HTTP Site Hit Summary Utility

Download: dwhcount
Ensim only?Yes

This script uses vsearch_access_logs to produce a report of web hit count totals for one or more sites on your server (all sites by default).

Why use this? Well, it is easy to use, you can filter by HTTP status code and domain, and it is relatively quick. Good for getting a sense of general error states too .. if you see a lot of 404s / 500s on a given domain, you might get browny points :P for letting them know something is going on!

Usage and examples (as root):

[root@host host]# ./dwhcount -h
Invalid time specification `-h'!
Usage:   ./dwhcount day_spec [-n N] [-e domain1,domain2,...,domainN] \
                     [-s NNN] [-m EXPR] [domain.one ... domain.nnn]
Where:
    day_spec: Date::Manip compatible date expression
              (Examples: 'today', 'yesterday', '12/15/2003')
          -n: Limit number of sites output to N
          -h: This screen
          -e: Exclude comma-separated list of domain names
              from output.
          -m: Limit count to URLs that match EXPR
          -s: Limit count to URLs that generated a status NNN

#  Top 10 hit site hit counts for yesterday
./dwhcount yesterday -n 10

#  Show #s of 404 counts for all sites except foo.com and bar.org
./dwhcount today -s 404 -e foo.com,bar.org

Sample output -- get 2xx status totals for all sites for today:


# dwhcount today -s '2[0-9][0-9]'
===============================================================
                   REPORT FOR DATE: 01/20/04
DOMAIN                                            HITS    PCT %
===============================================================
yyyy.nnn                                           984    39.33
hhhhh.ooo                                          435    17.39
yyyyyyyyyy.aaa                                     241     9.63
fff.ooo                                            226     9.03
ggggg.nnn                                          224     8.95
aaa.ss                                             102     4.08
vvvvvvvvvvvv.ooo                                    90     3.60
mmmmmmmmmmmm.ll                                     71     2.84
aaaaaa.dd                                           42     1.68
ooooooooooooo.dd                                    22     0.88
mmmmmmmmmmmmmmm.aa                                  22     0.88
xxxxxxxxxxx.vv                                      13     0.52
jjjjjjj.kkk                                         10     0.40
oooooooooooooo.iii                                   8     0.32
llllllllllll.aaaa                                    7     0.28
fffff.ii                                             3     0.12
hhhhhhhhhhhhhhh.ooo                                  2     0.08
===============================================================
TOTAL HITS/TOTAL %                                2502   100.00
===============================================================


Sendmail Log Search Utility

Download: vmail_search
Ensim only?No

(formally mails_per_day): Show various email stats (incoming emails, outgoing, deferred, rejects, blocked) emails on a server for a given day or timeframe (in minutes|hours|days). By default the script will show emails for the current day (if you don't pass in a date spec). The script can search as far back as your /var/log/maillog* files go.

Usage (as root):

USAGE: /usr/local/sbin/vmail_search command \
       [ --minutes n | --hours n | --days n | date_specifier]

Where command is one of:
    --in:        Names/count of virtual hosts receiving mail
    --out:       Names/count of virtual hosts sending mail (e.g. via squirrelmail)
    --relay-out: To addresses of mail sent from virtual host via .forward, catchall, etc.
    --relay-in:  To addresses of mail sent TO this host
    --stat:      List of states mail sent by this machine elsewhere
    --estat:     List of to address, relay, and state tuples for mail relayed offsite
    --estat-l:   List of to address, relay, and state tuples for mail relayed to virtual
                 hosts (local relays)
    --deferred:  List of deferred outbound emails, with recipient name and 
                 reason for defer.
    --blackhole: Host names/ips of sending hosts blocked by sendmail (status 553)
    --rejected:  Count of number of mails rejected, ordered by status
    --rejected-relays:  Names/IPs of sending relays blocked by sendmail
    --pop3-logins: Client email addresses/relays of POP3 logins
    --imap-logins: Client email addresses/relays of IMAP logins

Example usage:

/usr/local/sbin/vmail_search --in
#  Names/count of hosts rejected from sending mail to us in last 30 minutes
/usr/local/sbin/vmail_search --blackhole --minutes 30
#  To addresses for mail sent from our host via a .forward, catchall (outbound relay)
#  in last hour
/usr/local/sbin/vmail_search --relay-out --minutes 60
#  Names/count of rejected relays for yesterday
/usr/local/sbin/vmail_search --rejected-relays yesterday
#  Addresses/counts/reasons of deferred outbound emails over last 5 days
/usr/local/sbin/vmail_search --deferred --days 5
#  Outbound emails that were not succesfully sent offiste
/usr/local/sbin/vmail_search --estat | fgrep -v sent


Sendmail Transaction Search And Display Utility

Download: smhunt
Ensim only?No

Unlike vmail_search, which shows various sendmail statistics based on line-by-line searches through system sendmail logs, this script searches and displays sendmail transactions. What do I call a transaction? Example: you@yourdomain.com emails me@mydomain.com

  1. Mail is relayed in from you@yourdomain.com
  2. Mail is relayed to virtualhost
  3. Mail is relayed locally to local user at the virtual host

The difficulty with reading / searching sendmail logs is that the above transaction (you@yourdomain.com to me@mydomain.com) could span 4-6 different entries in your system's sendmail log file(s). Each entry is associated with the next by a queue id. Even this id changes as the relay in -> your machine can be done with one queue id, then the relay from the local machine -> a virtual domain can be done with another queue id. Correlating these by hand can be tedious, so I have attempted to automate the process with this script. With no options, this script will show you all sendmail transactions for the current day. The script can search as far back as your /var/log/maillog* files go.

Example output, narrowed down to a single transaction as I define the word transaction in this context:

  • you@foobar.com: External From address
  • virtuser@localdom.com: Ensim virtual host
  • local_user: User local to domain localdom.com

Search for sendmail transactions involving user that occurred in the last hour:

# smhunt -r 1h local_user
Transaction hBTKb3T30560:
   1: Dec 29 15:37:03 sendmail   RELAY IN FROM REMOTE MTA [1344 bytes] (* hBTKb3f30555)
         FROM: <you@foobar.com>
        RELAY: externel.bar.org [192.168.1.1]
   2: Dec 29 15:37:03 sendmail   DELIVER LOCAL VIRTUAL HOST (* hBTKb3f30555)
           TO: <virtuser@localdom.com>
       MAILER: virthostmail
        RELAY: foobar.com
         STAT: Sent (hBTKb3T30560 Message accepted for delivery)
   3: Dec 29 15:37:03 sendmail   RELAY IN FROM LOCAL USER root MTA [1566 bytes]
         FROM: <you@foobar.com>
        RELAY: root@localhost
   4: Dec 29 15:37:04 sendmail   RELAY OUT (SUCCESS)
           TO: external_user@external.domain.com
       MAILER: esmtp
        RELAY: external.mail.relay.org. [192.168.32.44]
         STAT: Sent (ok dirdel)
   5: Dec 29 15:37:04 sendmail   DELIVER LOCAL USER
           TO: local_user
       MAILER: local
        RELAY: 
         STAT: Sent

As you can see, there were FIVE actions involved in this transaction and 2 separate queue IDs involved, just to deliver a single email!!

How is this script useful? For troubleshooting, I am finding it to be a very valuable tool. Not only does it help me hunt down the source of mail-related problems (hence smhunt -- sendmail hunt), it also helps me view the larger picture of what is going on with sendmail when something mail-related goes amiss.

smhunt allows for regexp-searching. Big deal, right? The big deal is that the searching is also transacation based. If you search for "me@mydomain.com," you won't just get single lines back, you will get the sendmail transaction(s) that match back! Much more useful for in-depth debugging.

I have found sendmail logs quite difficult to parse and sendmail transactions very challenging to assemble; this first release is far from perfect but does work for basic transaction viewing and searching. Just keep in mind this is more or less a beta-quality release .. i.e. BUGS EXIST! :)

Usage screen:
# smhunt --help | less
Usage: /usr/local/sbin/smhunt [--debug|-d] [[--range|-r] N] [[--from|-f] N] \
          [[--long-output|-lo] [[--nolink|-nl] \
          [ SEARCH_TERM_1 ... SEARCH_TERM_N ]
Where:
    --debug: Turns on debug mode (very verbose!)
    --range: Set how many minutes/hours/days back in your mail logs to
             search.  Use suffixs "h" for hours, "m" for minutes, or
             "d" for days.  

             Example: Search back 3 days "--range 3d".

             Defaults to current day.

     --from: Latest date to search.  This is very useful if you want
             to see transactions that happened for say, last Friday
             only, not Friday to now. 

             Example: "--from '12/14/2003 23:59:59'"

             Defaults to current time/date.
             
     --long-output: 

             Show EVERY FIELD and value in each action for each
             transaction.  Good for hunting details of a problem.
             Default is to summarize fields and value for each action 
             within a transaction to make viewing easier.

     --no-link:

             Do not link multiple, related transactions together for
             viewing.  By default this script WILL link together related
             transactions for ease of viewing sendmail action histories
             (this part of the code needs more work).  Default is TO
             LINK together transactions.  Useful if you are searching over
             a large period of time (> 3 days).

    SEARCH_TERM:  Can be a simple string or a perl regular expression to
                  match.  Note that if a single ACTION in a transaction
                  matches, the whole transaction will be captured.
    
    Example usage:

    #  Search for all email transactions over last day.
    /usr/local/sbin/smhunt

    #  Find all deferred transactions over last 2 days
    /usr/local/sbin/smhunt -r 2d eferred

    #  Find all transactions over last 4 days, without linking
    #  related transactions together, that involved you@yourdomain.
    /usr/local/sbin/smhunt -r 4d -nl you@yourdomain.com

    #  Find all transactions that had DSN errors on 12/13 and 12/14.
    /usr/local/sbin/smhunt -r 1d -f '12/14/2003 23:59:59' DSN

Sendmail Mail Queue Info Utility

Download: vmqinfo
Ensim only?Yes

This script show various levels of detail about one or more virtual host mail queues on your system.

In its' default mode (no options, it will show you the following information for each virtual host on your system that has queued mail:

  • Domain name of host
  • Number of queued messages (not total # files, which can be two or three times the number of messages)
  • Total size for all FILES stored in the mail queue

Sample output:

[root@GGGGG uuuuu]# vmqinfo
DOMAIN                         MSGS      QUEUE SIZE
==============================================================================
yyyyyyyyyyyy.yyy                 10      56.41 KB
NNNNNNNNN.NNN                     9      70.81 MB *WARNING*
XXXXXXXXXXXXXX.YYY                5      25.88 KB
BBBBBBBBBBB.OOO                   5      19.19 KB
==============================================================================

If you pass a single "-v" to the script, it will show brief summary information for each file in each queue in addition to the domain name, # messages, and queue dir size:

  • Name of queue file
  • Number of times sendmail has attempted to deliver message
  • Date current attempt was started
  • How long message has been in queue since latest delivery attempt
  • How OLD the original message is (when was delivery FIRST attempted for it)

Sample output:

[root@GGGGG uuuuu]# vmqinfo -v AAAAAAAAAAA.FFF
DOMAIN                         MSGS      QUEUE SIZE
==============================================================================
AAAAAAAAAAA.FFF                   5      19.19 KB
------------------------------------------------------------------------------
  QUEUE FILE    [TRY]  THIS ATTEMPT STARTED AT  (QUEUE AGE)      MSG AGE
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  qfi011JeH27479[120]: Mon Jan  5 18:43:02 2004 (38:39 m:s)  -  4+  days
  qfi040Tjx27211[049]: Mon Jan  5 18:43:02 2004 (38:39 m:s)  -  1+  days
  qfi04BcCL27314[037]: Mon Jan  5 18:42:02 2004 (39:39 m:s)  -  1+  days
  qfi04DIKZ08233[036]: Mon Jan  5 18:41:02 2004 (40:39 m:s)  -  1+  days
  qfi05GG4C01529[009]: Mon Jan  5 18:41:01 2004 (40:40 m:s)  -  8+ hours
------------------------------------------------------------------------------

Pass "-v -v" and you get MORE information about each queued message. This includes all of the above plus:

  • From address on message
  • To address on message
  • Subject of message
  • Reason for deferral (if present)
  • Date of initial delivery attempt + time to the second since initial delivery was attempted

Too verbose to include sample output, try it yourself! :)

If you pass any domain names on the command line to the script, the output will be limited to those domains, e.g.

# vmqinfo -v foobar.com barfoo.net

This script requires perl 5.6.x or newer, Sys::Hostname, and Date::Manip. Date::Manip and Sys::Hostname come with the versions of perl that come with Ensim. For other OSes, you may need to download and install them. Both modules can be found at CPAN.

Installation instructions (as root):

# cd /usr/local/sbin/
# wget http:///vmqinfo
# chown root vmqinfo
# chmod 755 vmqinfo

Usage summary:

# vmqinfo -h

Usage: /usr/local/sbin/vmqinfo [-v] [-v] [domain1 ... domainN] [qid1 .. qidN]
       -h: This screen
       -v: Be verbose, give queue id, create time, queue date, msg age
    -v -v: Very verbose, give much information on each file in queue
           (name, create time, from, to, subject, reason for delay, age)
       -m: Machine readable (greppable) format.  This outputs all the
           same fields, but delimited by ';'.  Field output is as
           follows (Numbers of fields indicate position in output line!)
           1) Normal mode, no "-v" switches:
              0) domain name
              1) Number of messages in queue
              2) Size of queue in bytes
           2) Verbose "-v" mode.  After each domain line (item 1) above
              is output, lines are output describing the files belonging
              to the domain listed in item 1:
              0) The word "file" to indicate this is a file belonging
                 to the previously output domain.
              1) Queue file name
              2) The number of this delivery attempt since delivery was
                 first attempted.
              3) Unix time stamp date indicating when message was last
                 queued for delivery
              4) Number of seconds message has been in the queue this
                 go around.
              5) Number of seconds before now that message delivery was first
                 attempted (time the messge first entered your host).
           3) Super verbose mode "-v -v" includes all of the items in
              #2 plus:
              6) Unix time stamp indiciating when message first entered
                 the sendmail mail queue (first delivery attempt). 
              7) From address of message
              8) To address of message
              9) Subject of message
             10) Reason for deferral (may be blank)
   
    If you specify "-v" or "-v -v" you may also pass in a list of queue 
    IDs if you wish to limit the verbose output to just one or more
    queue IDs.
 
Examples:

#  Show details of queue id for a deferred message
#  (you can see queue ids with a single "-v" or you can prepend
#  "qf" to a message id you find in your sendmail mail log.

/usr/local/sbin/vmqinfo -v -v qfi053QTp24589

#  Show verbose queue summary for hosts foo.com and bar.com
/usr/local/sbin/vmqinfo -v foo.com bar.com

#  Show very verbose output for all hosts
/usr/local/sbin/vmqinfo -v -v

#  Output verbose summary in machine-readable moed
/usr/local/sbin/vmqinfo -v -m

#  Show short summary for all hosts
/usr/local/sbin/vmqinfo


Sendmail Mail Queue Cleaner Utility

Download: cmq
Ensim only?Yes

This script will help you clean both the global mail queue and virtual host mail queues on your Ensim-based host. Why would you need to do that?

  1. When sendmail is unable to deliver a message, it leaves the message in the sendmail queue (/var/spool/mqueue).
  2. High system loads or transmission errors can cause sendmail to leave a 0-byte mail spool file sitting in the queue.

Over time, the accumulation of these 'stale' messages can result in significant additional disk use as well as slower response times for sendmail (I have heard of systems with more than 300,000 invalid messages in their queues).

Installation instructions (as root):

# cd /usr/local/sbin/
# wget http:///cmq
# chown root cmq
# chmod 755 cmq

Usage summary:

USAGE: cmq [-d time_spec] [-e] [-o] [-n] [-z] [-r number] [-H] [-h] \
          [-D domain1,domain2,...,domainN]

-d time_spec: Delete messages that have been around 
              since this date or earlier.  Date
              spec must be one that Date::Manip
              recognizes.

          -e: Delete messages in 'ERR' state.

          -n: Don't actually delete messages, just
              show what would be deleted.

          -o: Delete 'orphaned' transcript or data
              files (no queue file present).

          -z: Delete zero-byte queue messages or 
              zero-byte transcript/data files that have
              no queue file associated with them.

   -r number: Delete messages that have failed to
              be delivered in 'number' or more
              retries.

          -H: Hot clean queues (do not shut down sendmail while
              cleaning): USE WITH CAUTION
          -D: Instead of operating across all domains, just clean the queues
              for the specified domain (separate multiple domains using
              comma)
          -h: This screen.

Examples:

#  Delete messages that have been in the queue for 5 or more days
cmq -d '5 days ago'

#  Delete all messages in error state or that failed delivery 15 or more times
cmq -e -r 15

#  Delete zero-byte queue messages and messages in error state
cmq -z -e

#  Hot clean example.com and my.example.com of messages more than 3 days old
cmq -H -d '4 days ago' -D example.com,my.example.com

# Delete orphaned transcript / data files
cmq -o

Virtual Host Process Viewer

Download: vps
Ensim only?Yes

Show processes that are running on virtual hosts. This script shows a "ps" like output of some or all processes that were started from a virtual host (chrooted). This is a VERY useful command if you do admin from the shell as it not only shows you JUST processes that are running under one or more virtual hosts, it also TRANSLATES usernames so they appear as they exist on the virtual host!

By default vps shows proceses on all domains; if you wish to limit the display to one or more domains just type them in as arguments, separated by spaces. Example:

vps  webscorpion.com domain.com

You can also specify match or skip filters to limit the display based on process type, owner, etc. Example:

#  Show all ssh/bash sessions
vps --match 'bash|ssh'
#  Don't show crond processes running under virtual hosts
vps --skip crond
Sample output (filtered to not show real domains other than my own):
[root@ensim ensim]# vps  xxxxxxxxx.xxx
USER     PID    DOMAIN                           START  %CPU %MEM COMMAND
root     6332                     14:54  0.0  0.1  /usr/sbin/crond
root     6134   www.xxxxxxxxx.xxx                14:53  0.0  0.1  /usr/sbin/crond
uuuuuuu  4559   www.xxxxxxxxx.xxx                18:47  0.0  0.3  proftpd: ...

[root@ensim ensim]# vps --help
/usr/local/sbin/vps [--match expr] [--skip expr] [domain_name1 ... domain_nameN]
    --match: limit output to lines that match egrep regular expression 
    --skip:  limit output to lines that DO NOT match egrep regular expression 

Examples:

#  Show all non-root commands running on virtual hosts
/usr/local/sbin/vps --skip root
#  Show all current virtual host ssh logins
/usr/local/sbin/vps --match 'bash|ssh'
#  Show all processes for 3 domains
/usr/local/sbin/vps  domain.com webscorpion.com

This is one of my favorite scripts in terms of how useful it can be; much easier than doing a "ps auxwww | fgrep blah | less" and THEN trying to figure out what is what.


Last Login Viewer

Download: vlast
Ensim only?Yes

Show who has logged into every virtual host for a given day


Virtual Host Customer Mailer

Download: email_all_customers.sh
Ensim only?Yes

Send an email to every site administrator, customer, and/or virtual site user on your host. This script will prompt you for a subject for your email, then start a pico session to let you enter your email body. After that it will send your email. It will use the appliance admin email address as the From address for the email and gives you a choice of ways to send the email. Run it as root.


Virtual Host SSH Login Viewer

Download: vusers.tar
Ensim only?Yes

Show a list of current virtual ssh logins (tar file has install script)


Virtual Host Sendmail Log Appender

Download: virtual_sendmail_logs.tar
Ensim only?Yes

Give every virtual user their sendmail entries every hour (become root, cd to /, then untar the tar file to install).


Enhanced Virtual Cron Control Script

Download: virtualcron
Ensim only?Yes

My prettied up version of the virtualcron /etc/rc.d/init.d/virtualcron script. Includes working start, stop, status, domain-start, and domain-status targets with Redhat init-type color coding ... oooooh, try it, it's pretty! Many thanks to the original author who wrote the virtualcron utlity .. it is great!

To install, as root:
1) Download the file to /etc/rc.d/init.d/virtualcron
2) chmod 755 /etc/rc.d/init.d/virtualcron
3) chown root /etc/rc.d/init.d/virtualcron

HTTPS Backup Trigger Utility

Download: https_backup.tar
Ensim only?Yes

Backup all virtual hosts on a server remotely using python to send the HTTPS request that would normally be sent via the web interface (much faster than actually going through the interface and you can script your backup so you don't have to re-enter information every time!) .. requires python 2.0


API Example: Reseller Control

Download: reseller_control.py
Ensim only?Yes

Disable/Enable all sites for a reseller from the command line. To use, just pass the script the username of the reseller and either the switch '--enable' or '--disable.' Doesn't disable the reseller (yet); to do that you will still have to use the GUI (for now).

Usage (as root):

# ./reseller_control.py username --disable

or

# ./reseller_control.py username --enable


View Site Bandwidth Usage Utility

Download: site_bw_usage.py
Ensim only?Yes

View a relatively nicely formatted bandwidth report for a given site from the command line. This script can use some serious cleaning up as it arose purely out of my interest in trying to figure out how to get bandwdith information via the Ensim API.

Usage (as root):

# ./site_bw_usage.py sitename.com | less


API Example: Concise Bandwidth Report

Download: showbw
Ensim only?Yes

View a more concise summary of bandwidth usage for a site.

Usage (as root):

# ./showbw sitename.com 


Virtual Host Bandwidth Summary Utility

Download: bwsummary
Ensim only?Yes

View a summary of current bandwidth usage for all sites on your host.

Usage (as root):

# bwsummary


Virtual Host Quota Viewer

Download: qsummary
Ensim only?Yes

View a summary of quota allocation and usage for all sites on your host.

Usage (as root):

# qsummary


Virtual Host User Quota Viewer

Download: user_quota.py
Ensim only?Yes

Show all disk quotas for USERS on a given site.

Usage (as root):

# user_quota.py hostname.com


Daily Admin Report

Download: Daily_Admin_Report
Ensim only?Yes

This script produces a simple report using six of the utilities listed on this page. Install it in /etc/cron.daily to get an email every morning that contains the following information:

  • Bandwidth usage summary
  • Quota summary
  • Email status summary: # sent, # deferred, etc
  • Top 10 domains receiving mail
  • Top 10 domains sending mail from the server itself (via Squirrelmail or other localhost based mail client)
  • Top 10 recipients (email addresses) of mail relayed out from the server (via mail forwarding, programs that relay through localhost, etc)
  • Top 10 email addresses (SMTP FROM) for mail relayed into the server
  • Top 10 email addresses (SMTP FROM) of deferred email delivery attempts from your server to other servers
  • Top 10 clients retrieving email via POP3 from your server
  • Top 10 clients retrieving email via IMAP from your server
  • SSH/FTP logins
  • Number of messages and size of sendmail mail queues for every virtual host on your system that has one or more messages queued.
  • List of top 10 sites for HTTP accesses, all statuses
  • List of top 10 sites for HTTP accesses, 403 status
  • List of top 10 sites for HTTP accesses, 404 status
  • List of top 10 sites for HTTP accesses, 500 status

To use this script, first install these utilities in /usr/local/sbin (all are available on this page):


To install the script, do this (as root):
# cd /etc/cron.daily
# wget http:///Daily_Admin_Report
# chmod 700 ./Daily_Admin_Report

You will start receiving the summary email every morning after you place the script in the cron.daily directory.


Snort Summary Report

Download: snort-summary
Ensim only?No

Receive an emailed hourly and daily summary of snort activity on your host.

This script produces a relatively easy-to-read (to me at least) summary of snort activity either for the last hour or for the last day.

Installation (as root):

# cd /usr/local/sbin/
# wget http:///snort-summary
# chmod 755 snort-summary

*  Edit the snort-summary script and set the location of the snort
   log file (SNORT_LOG variable).
*  Set the address of the person to be emailed the report 
   (MAILTO variable).  This should preferably be an OFFSITE
   address.
*  Save the file and exit the editor.

# echo "/usr/local/sbin/snort-summary daily" > /etc/cron.daily/snort-summary
# chmod 755 /etc/cron.daily/snort-summary
# echo "/usr/local/sbin/snort-summary" > /etc/cron.hourly/snort-summary
# chmod 755 /etc/cron.hourly/snort-summary

You will start getting snort reports hourly and daily. You will only receive a report if events occurred.

Sample output (for a single signature, my IP addresses masked):

#############################################################################
                                FTP Bad login                                
               6 times, 4 unique sources, 3 unique destinations               
#############################################################################

Top 5 talkers:
    =========================================================================
    Time span                           #   Source           Destination
    =========================================================================
    Jul 22 23:16:15 - Jul 22 23:16:15   1   XXX.XX.XXX.XX    63.207.128.14    
    Jul 22 18:08:06 - Jul 22 18:08:06   1   XXX.XX.XXX.XX    217.122.250.71   
    Jul 22 10:01:38 - Jul 22 10:01:38   1   YYY.YY.YYY.YY    80.235.60.223    
    Jul 22 10:01:24 - Jul 22 10:01:24   1   XXX.XX.XXX.XX    80.235.60.223    
    Jul 22 10:01:35 - Jul 22 10:01:35   1   YYY.YY.YYY.YY    80.235.60.223    

All Connections by IP:
    =========================================================================
    Time span                           #   Source           Destination
    =========================================================================
    Jul 22 10:01:26 - Jul 22 10:01:26   1   ZZZ.ZZ.ZZZ.ZZZ   80.235.60.223    
    Jul 22 10:01:24 - Jul 22 10:01:24   1   XXX.XX.XXX.XX    80.235.60.223    
    Jul 22 18:08:06 - Jul 22 18:08:06   1   XXX.XX.XXX.XX    217.122.250.71   
    Jul 22 23:16:15 - Jul 22 23:16:15   1   XXX.XX.XXX.XX    63.207.128.14    
    Jul 22 10:01:35 - Jul 22 10:01:35   1   YYY.YY.YYY.YY    80.235.60.223    
    Jul 22 10:01:38 - Jul 22 10:01:38   1   YYY.YY.YYY.YY    80.235.60.223    

Abbreviated log:
    =========================================================================
    Time stamp                    Source                Destination
    =========================================================================
    Jul 22 10:01:24               XXX.XX.XXX.XX:21      80.235.60.223:4227
    Jul 22 10:01:26               ZZZ.ZZ.ZZZ.ZZZ:21     80.235.60.223:4368
    Jul 22 10:01:35               YYY.YY.YYY.YY:21      80.235.60.223:4413
    Jul 22 10:01:38               YYY.YY.YYY.YY:21      80.235.60.223:4450
    Jul 22 18:08:06               XXX.XX.XXX.XX:21      217.122.250.71:4999
    Jul 22 23:16:15               XXX.XX.XXX.XX:21      63.207.128.14:4316

#############################################################################


API Example: Get Sites For a Reseller

Download: get_reseller_sites.py
Ensim only?Yes

Get information on all sites for a reseller by reseller user name

Usage (as root):

# ./get_reseller_sites.py reseller_user_name


API Example: Get Reseller By Domain

Download: get_reseller_by_domain.py
Ensim only?Yes

Get reseller information for a given domain

Usage (as root):

# ./get_reseller_by_domain.py domain_name


API Example: Get Site Information

Download: get_site_info.py
Ensim only?Yes

View all configuration information (or just one type) available for a domain.

Usage (as root):

# ./get_site_info.py domain.com
# ./get_site_info.py domain.com bandwidth


Web Hosting Account Partitioning Utility

Download: whapu.phps
Ensim only?No

Get a general idea of how much theoretical profit can be made from your server(s). This web-based utility can help you set initial price points for accounts on your server.

Try the script out here

Installation (as root):

# cd /var/www/html
# wget http:///whapy.phps
# mv whapu.phps whapu.php
# chown 644 whapu.php
Usage: http://www.example.com/whapu.php


Do you like these scripts? Have they saved you time the way they save us time? We don't require anyone to pay for any of these, but if you feel they have helped you, how about donating a few dollars towards future enhancements and new scripts? We always welcome your suggestions/feedback! Any amount, regardless of how small, is welcome! No obligation.

Thank you if you do decide to contribute!!

Contributors!

Companies and individuals who have contributed to the ongoing development of the scripts listed on this page. Special thanks to all of you! Any site or person who contributes can have a hyperlink to their company/website, with the caveat that we will not post links to adult-content sites (sorry!). Links are posted in no special order. The scripts on this page are downloaded 800-1200 times a month, with 20-40 views of this page a day (some downloads come from forum links or links from other sites).

Web hosting provided by Web Wizard Design