Dec 14

I downloaded a bunch of images using wget.

The original file names were the same 6 names from 40 different pages. wget names those like this:
01_resize.jpg
01_resize.jpg.1
01_resize.jpg.2

etc.

The other files were:
02_resize.jpg
03_resize.jpg

etc, up to 6.

Here is what I came up with that gives each file a unique name and a valid extension ( 01_resize.jpg.02 becomes 02_01_resize.jpg ) (you can yell at me for parsing the output of ‘ls’ later)

for i in $(ls); do
seq=$(echo $i | cut -d. -f 3)
fname=$(echo $i | cut -d. -f 1-2)
mv $i $seq\_$fname
done

Hopefully it’s of some use to someone! I’m basically grabbing the different parts of the filename using cut with a delimiter of ‘.’ ; that’s where the magic is. There’s probably a more efficient way of doing this, but it’s what made sense to my brain after being up for 24 hours.

Sep 19

UPDATED: now works with most apache installs and is more portable / useful:

http://ip2k.com/sitelist.sh

This bash script combines many useful little tricks that you may be interested in even if you don’t need the whole thing. Some include:

- Print a list of IPs on the linux system
- Print a list of websites hosted on the server that match a certain pattern (app = Magento in this case)
- Looping


trap sorry INT
#Two levels of debugging, 1 and 2. 1 shows just pings, 2 shows everything.
dbg=0

function prnt {
echo -e "\e[1;36m[ $1 ]\e[00m"
}

function prntok {
echo -e "\e[1;32m[ $1 ]\e[00m"
}

sorry ()
{
prnt "Caught SIGINT, cleaning up..."
if [ -f tempfile ]; then
rm -rf tempfile
prntok "Found and removed tempfile, exiting now"
else
prntok "tempfile not found, exiting..."
fi
exit 1
}

function getip {
ping -c1 -W1 -q $1 | grep PING | awk '{print $3}' |tr -d \(\)
}

iplist="$(ifconfig | grep 'inet addr' | awk '{print $2}' | tr -d [a-z,:] | grep -v '127.0.0.1')"
echo > tempfile
for ip in $iplist; do
echo $ip >> tempfile
done

sites="$(awk '$1 == "ServerName" { print $2 }' /etc/httpd/conf.d/vhost_* | uniq)"

for site in $sites; do
if [ $dbg -gt 0 ]; then
prnt "DEBUG: Pinging $site..."
fi

siteip=$(getip $site)
ison=$(grep -c $siteip tempfile)

if [ $dbg -gt 1 ]; then
prntok "DEBUG: siteip = $siteip"
fi

if [ $ison = 1 ]; then
if [ $dbg -gt 1 ]; then
prntok "DEBUG: $site found!"
fi

echo "http://$site"
else
if [ $dbg -gt 1 ]; then
prnt "DEBUG: $site not found"
fi

fi

done

rm -rf tempfile
exit 0

Aug 23

I oversee a few hundred Linux-based systems, and >99% of them run the typical QMail / ClamAV / SpamAssassin stack. I personally think QMail sucks, but I have written a script that usually fixes most issues (stuck queue, delayed mail, other strange issues). You could easily edit this to be compatible with cPanel-based servers, but as it stands it’s written for InterWorx-CP and servers without a control-panel system. Note that this needs BASH to run and won’t work properly due to the way the counter is incremented in regular POSIX ‘sh’.

Link: http://seanp2k.com/m.sh

It is below:

#!/bin/bash
# ----- BEING DECLARE FUNCTIONS -----

# prnt - formats and echos input
function prnt {
echo -e "\e[1;32m[ $1 ]\e[00m"
}

# killproc - uses pidof to find process name then tries to kill COUNTER number of times.
# Accepts $1 as process name and $2 as $
PROCNAME=""

function killproc {
procname=$1
counter=0
pids="$(pidof "$procname")"
while [ "$pids" ] && ((counter++ < $2)); do
if [ $counter -eq $2 ]; then
echo -e "\e[1;31m[ Couldn't kill $procname, tried $2 times. PID is $pids ]\e[00m"
else
# ----- DEBUGGING - uncomment next line -----
# ps -elf | grep -i $procname
for pid in $pids; do
echo -e "\e[1;36m[ $procname killed, PID $pid, try $counter/$2 ]\e[00m"
pkill -9 $pid
done
sleep 1
pids="$(pidof "$procname")"
echo
fi
done
}

# ----- END DECLARE FUNCTIONS -----
# ----- BEING MAIN SUB -----

prnt "Reset local and remote delivery concurrency limits"
echo 200 > /var/qmail/control/concurrencylocal
echo 200 > /var/qmail/control/concurrencyincoming
echo 200 > /var/qmail/control/concurrencyremote

prnt "Restart InterWorx-CP"
/etc/init.d/iworx restart

prnt "Restart ClamAV & SpamAssassin"
/etc/init.d/clamd stop
/etc/init.d/spamassassin stop

killproc spamd 5
killproc clamd 5

/etc/init.d/clamd start
/etc/init.d/spamassassin start
prnt "Restart POP3"
/etc/init.d/pop3 restart

prnt "Restart POP3-SSL"
/etc/init.d/pop3-ssl restart

prnt "Restart IMAP4"
/etc/init.d/imap4 restart

prnt "Restart IMAP4-SSL"
/etc/init.d/imap4-ssl restart

prnt "Restart SMTP"

svc -d /service/send /service/smtp /service/smtp2
/etc/init.d/smtp stop
/etc/init.d/smtp status

if [ "$(pidof qmail-send)" ]; then
counter=11
echo -en "\e[1;32m[ Waiting for SMTP to stop: "
while [ "$(pidof qmail-send)" ] && ((counter-- > 0)); do
echo -n "$counter.."
sleep 1
done
echo -e " ]\e[00m"
fi

/etc/init.d/smtp status
svc -k /service/send
killproc qmail-send 5
prnt "Force-send qmail message queue"
svc -u /service/send
svc -a /service/send
svc -u /service/smtp /service/smtp2
/etc/init.d/smtp start
prnt "Please ensure that SMTP and send are up"
/etc/init.d/smtp status
/var/qmail/bin/qmqtool -s
# ----- END MAIN SUB -----
exit 0;

Mar 5

Auto-detects databases, has the temp file and backup file locations separated, purple colored real-time output, only logs important output, md5sums into a text file for databases, every filename uniquely hashed with month, day, year, hour, minute, and second, timestamps everything, uses bzip2 compression for smallest file sizes.

#!/bin/bash

# vars – no trailing slash please
THEDATE=$(date +%m%d%y%H%M%S)
BACKUPDIR=’/backups/mysql’
TEMPDIR=’/backups/mysql/temp’
LOGFILE=”$TEMPDIR/info-$THEDATE.txt”
USER=root
PASSWORD=password
date > $LOGFILE

date

echo -e “\e[1;35m[ " $(date +%H:%M:%S) "-- Starting MySQL Backup Script v0.3 by Seanp2k ]\e[00m"

# magic
DATABASES=$(mysql --user=$USER --password=$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database)

echo -e "\e[1;35m[ " $(date +%H:%M:%S) "-- Found" $(echo $DATABASES | wc -w) "databases:" $DATABASES "]\e[00m"

# loop -- dump and hash
for DB in $DATABASES; do

echo -en "\e[1;35m[ " $(date +%H:%M:%S)
echo -n " -- Processing " $DB ".."
mysqldump --user=$USER --password=$PASSWORD --databases $DB > $TEMPDIR/$DB-$THEDATE.sql
echo -n "..md5sum..."
md5sum $TEMPDIR/$DB-$THEDATE.sql >> $LOGFILE
echo -e "Done ]\e[00m"
done

# compress and clean
echo -e "\e[1;35m[ " $(date +%H:%M:%S) "-- Creating Archive " $BACKUPDIR/mysql-THEDATE.tar.bz2 "]\e[00m"
time tar -cvvjf $BACKUPDIR/mysql-$THEDATE.tar.bz2 $TEMPDIR/*.sql $TEMPDIR/info-$THEDATE.txt
rm -rf $TEMPDIR/*.sql $TEMPDIR/info-$THEDATE.txt
date
echo -e "\e[1;35m[ " $(date +%H:%M:%S) "-- Finished ]\e[00m”

Jan 23

I had a problem: I wanted my radio station to automagically play my newer music and update the stuff that it plays every day, yet keep enough older stuff that there is still some variety, especially if no new music gets discovered for a while. I’d need to find some way to keep the most recent 200 or so tracks playing on my station.

Let’s start by looking at what we have and what we have to do:
Files on a Linux fileserver playing on a Windows server (via samba) in Winamp using EdCast to stream to sc_serv (Shoutcast) on a different Linux server in the DMZ. Currently, I have the X: drive on the Windows server mapped to the Linux Fileserver. Winamp re-scans the fileserver every 3 days for new music and adds it to the library, but I still have to manually go in and add new tracks to the playlist.

Some initial ideas and problems:
-Use XML-RPC / SOAP from another music cataloging program that I currently use
+I just really feel like there was an easier way to do what I wanted without going through all the trouble.

-Telling Winamp to [re]load the playlist: the Winamp API Uses C++…yuck…I don’t know or want to learn C++ just for this project. Plus, DRY applies, someone HAS to have thought of this before.
+Solution: httpQ, a simple http listener plugin for Winamp that uses a simple and easy to implement control scheme: simply call the right URL and Winamp does what you want. Awesome. ( http://httpq.sourceforge.net )

-Finding files that have been modified recently, and if none have been modified in the time specified, keep looking until you find enough files
Read the rest of this entry »

Aug 13

MySQL has become an important part of most Web servers, especially over the last few years. With all those databases, the risk of corruption, intrustion, data loss, hardware failure, and lots of other factors necessitate automated backups.

Luckily, backing up MySQL isn’t nearly as hard as it might sound. Actually, it’s only one command. But, not many people have only one MySQL DB, and dumping all of them to a single file can be hard to parse should you only need to import one. This is easily remedied by making a script run on cron to dump them all individually.

I set this script to run once a week, monday morning at 5am. It generates a logfile as well, info.txt.sl, which contains the date, a list of files, and their MD5Sum. Finally, it uses bzip2 to compress the files efficiently. MD5Sums are critical in determining the integrity of the archive and to ensure the database has not been corrupted or altered since the snapshot was taken.
Read the rest of this entry »