Disk space - df command

df command:
Hit a quota the other day, disk was at 81% - issue was my inodes were all used up.. Can find like this:
Find inodes used - in either bare metal or openvz:

df -i

Solaris:

df -o i

ISO files - of drives and directories

Making a backup of files..
To make an ISO of a directory

mkisofs -o ./vps-20090517.iso backups/

To make ISO's from devices:

dd if=/dev/cdrom of=/var/tmp/disk.iso

Found this simply explained here - thanks!
here

Mondo - Achive Examples

Mondo archive examples - just got intro'd to this program at work about a year ago, works very well - creates ISO images of a running system that can be restored very simply later on by installing like an OS - from CD...

This basically what is in man mondoarchive - /home and other places are backed up in another fashion, so don't need to have these to restore my base desktop..

mondoarchive -Oi -7 -d /backups/mondo/ -E "/backups /home /media /opt /srv /vz" -N -p `hostname`-`date +%Y-%m-%d` -s 680m

• -O - backup (that's an uppercase "O")
• -i - backup to ISO files
• -d - backup destination
• -7 - compression level (0-9)
• -E - exclude list
• -N - Don't include network mounted filesystems
• -p - filename prefix - 'stead of mondorescue-1.iso, etc.
• -s - specify max ISO file sizes m or g

SSH Tips and Tricks

Notes for SSH - various access methods and use for ssh

Local Computer

Remote Computer

port 9999

Set up a local proxy that would forward all traffic (encrypted) through the remote server - in this example from an internet cafe, one would connect to remote server, set the browser to use a socks proxy of "localhost:9999" and now all internet traffic is encrypted to/from the remote server. ssh -D 9999 user@remote .. or if you run your SSH on the server on a different port, say 4000 - ssh -D 9999 -p 4000 user@remote Set your Socks proxy to localhost:9999 - all webtraffic encrypted through the tunnel to the remote server. Add more tunnels for FTP, POP3 and with some proxy config will be secure.

port 4000

port 1234

Tunneling: Local forwarding tunnel: set up a forwarding tunnel so when access localhost:1234 its forwarded to remote:23 ssh -L 1234:localhost:23 user@remote or if server uses port 4000 for ssh - ssh -L 1234:localhost:23 -p 4000 user@remote - here we are setting up secure telnet tunnel. Users access the remote server just by telnet to localhost:1234.

port 23 ssh runs on port 22 & 4000

port 23

Tunneling: Remote forwarding Tunnel: set up a remote tunnel so when access remote:1234 its forwarded to localhost:23 ssh -R 1234:localhost:23 user@remote - note the only difference is -L and -R. Here we are setting up secure telnet tunnel. Users on the remote server access the local server just by telnet to localhost:1234.

port 1234

X forwarding: Start X applications on remote server, send to localhost: ssh -X user@remote or for different ssh port: ssh -X -p 4000 user@remote The -X to forward X - if that is enabled on the remote server in /etc/sdh/sshd_config: X11Forward yes

port 22

Firefox forwarding, multiple instances: Start X applications on remote server, send to localhost: On localhost:xhost +; then login with the -X optoin to the server, then firefox -no-remote will force firefox to start on the server and that instance will be forwarded to localhost. To run another instance - since only one profile can run at once, create a profile with firefox -no-remote -ProfileManager&. From Ubuntu forums and Lifehacker

port 22

crontab

CRON FILES
/var/spool/cron/crontabs/username
/etc/cron.d/cron.deny, /etc/cron.d/cron.allow Set who to deny and allow

Numbering format is:
n Matches if field value is n
n, p, q matches of field value is n, p,q 10,40 (matches 10 & 40)
n-p Matches field has values between n&p inclusive
* always matches.

Use crontab -e to edit a file. Some systems by default set pico or nano
as the default editor - I prefer vi - to force that, add the following to your .bash_profile if you use bash.

export EDITOR=vi
export VISUAL=vi

Source it:

 % source ~/.bash_profile

Edit with crontab -e

Example
minute          0-59
hour            0-23
dayof month     1-31
month           1-12
dayofweek       0-6 0=sunday

0 0 1,15 * *  /some/path/to/a/command    # whatever on 1st, 15th of the month
10 3 * * 0 /usr/lib/newsyslog            # daily at 3:10am
15 17 * * 5 /usr/bin/banner "Time to go" > /dev/console    # send out the geeks at 5:15pm on Friday

View crontab file with crontab -l

find

find . -type f -mtime +90 -maxdepth 1 -exec rm -v {} \;

Note the space in between {} and \; is crucial...

Else will get:

#  find . -type f -mtime +90 -maxdepth 1 -exec rm -v {}\;
find: warning: you have specified the -maxdepth option after a non-option 
argument -type, but options are not positional (-maxdepth affects tests 
specified before it as well as those specified after it).  
Please specify options before other arguments.

find: missing argument to `-exec'

finger

finger - good to see who is logged in, or idle (have not typed). Yes this can be used to see if someone is slacking off

user@example.com [~/notes]% finger
Login Name Tty Idle Login Time Office Office Phone
gregg ttyp0 1d Jan 5 20:04 (ppp-65-139-123-221.dsl.mabell.net)
gregg ttyp1 Jan 9 17:15 (ppp-65-139-123-221.dsl.mabell.net)
gregg ttyp2 15:08 Jan 9 23:44 (ppp-65-139-123-221.dsl.mabell.net)

grep & egrep

grep and egrep (extended grep) are fast useful utilities.

System V and GNU grep are different, but 90% the same. The one thing I notice lacking in SystemV is the -R flag that recursively digs down to find what I am looking for...

Find the word "style" in files in a directory

user@example:% grep style *

Find the work "style" recursively in a directory - won't work on Solaris.. yet..

user@example:% grep -R style *

List files except that contain "access"

user@example:% ls | grep -v access

List files except that contain "access -or- error"

user@example:% ls | egrep -v 'access|error'

List files except that contain start with "access" - use -e then a regex

user@example:% ls | egrep -e  '^access'

Grep a file for the word "access"

user@example:% grep access /var/log/http/error.log

ip, ifup, ifdown in CentOS/Redhat/Fedora

Redhat/Fedora /etc/sysconfig/network-scripts/ifcfg-ethN files and the ifup/ifdown scripts - these use the HWADDR variable and then set the name with the ip(8) command.

Rename eth1 to eth0:

# ip link set down eth1
# ip link set eth1 name eth0

Or if like me you simply cat the ifcfg-eth1 file to ifcfg-eth0 and its still
not working after a reboot - would not take an IP address via ifconfig, this fixed it:

# ip link set down eth1
# ip link set eth1 name eth1

Given the IP/NetmaskGateway are set in the files..

Much of this article is from here - thanks!

kill

kill -9 PID Kill the process mercilessly.
kill -15 PID Ask the process to nicely stop running
kill -HUP PID Refresh a running process after config file changes - I have used this one a lot after making changes to /etc/ssh/sshd_config
kill -HUP -1 (kill all of my processes)

===========================================

A nasty solution:

stick in the following line:

echo $$ > ~/PID

in the user's .login file. and issue the following
commands to logout (actually 'slaughter') the user:

kill -9 `cat ~/PID`

kind regards,
Jos aka jos@and.nl

ln

ln - link a files and directories

Create symbolic link "B" that points to real file "A" Be careful.

ln -s   a       b
        ^       ^
       must    destroyed
       exist   if exists

user@example.com [~/test]% ln -s ln.txt ln-notes.txt
user@example.com [~/test]% ln ln.txt ln-notes2.txt
user@example.com [~/test]% mkdir DIR 
user@example.com [~/test]% ln DIR symlinkdir    <-- Can make hard links for files, but not  directories
ln: `DIR': hard link not allowed for directory
user@example.com [~/test]% ln -s DIR symlinkdir
user@example.com [~/test]% ls -al
total 20
drwxr-xr-x  3 gregg gregg 4096 Jan 11 00:13 ./
drwx--x--x 27 gregg gregg 4096 Jan 11 00:12 ../
drwxr-xr-x  2 gregg gregg 4096 Jan 11 00:13 DIR/
lrwxrwxrwx  1 gregg gregg    6 Jan 11 00:13 ln-notes.txt -> ln.txt*
-rwxr-xr-x  1 gregg gregg  146 Jan 11 00:12 ln.txt*
lrwxrwxrwx  1 gregg gregg    3 Jan 11 00:13 symlinkdir -> DIR/

login and profiles

/etc/.login /etc/profiles

add commands to be executed at login to these, files, see login manpage for more information.

mdadm - random notes

These are random notes - I'm working through them to ensure they are correct..

Random mdadm notes

Create RAID Arrays

mdadm --create /dev/md2 --level 1 -n 2 /dev/sda3 /dev/sdb3

Create RAID arrays w/missing drives - like a 1 drive RAID1 array, 2 drive RAID5 array

mdadm --create /dev/md1 --level 1 -n 2 /dev/sdc2 missing

mdadm --create /dev/md2 --level 5 -n 3 /dev/sda4 /dev/sdb4 missing

Restore previous group of disks into an array

mdadm --assemble /dev/md2 /dev/sda3 /dev/sdb3

Remove an array - mark drives failed, then stop, then remove

# mdadm /dev/md2 --fail /dev/sda3
mdadm: set /dev/sda3 faulty in /dev/md2
# mdadm /dev/md2 --fail /dev/sdb3
mdadm: set /dev/sdb3 faulty in /dev/md2
# mdadm --stop /dev/md2
# mdadm --remove /dev/md2

Recreate that array

mdadm --force --create /dev/md2 --level 1 -n 2 /dev/sda3 /dev/sdb3

The force is needed to rewrite the raid information on the drive.

Query the RAID detail for a drive

mdadm --query --detail /dev/md0

RAID Status

cat /proc/mdstat

Save RAID information for boot time

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Remove a drive from a RAID1 array

root@gregg-desktop:~# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid10]
md53 : active raid5 sdb3[2] sdc3[1] sda3[0]
      450703360 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md54 : active raid5 sdb4[2] sda4[0] sdc4[1]
      3453824 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md52 : active raid5 sdc2[1] sda2[0] sdb2[2]
      35664128 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md51 : active raid1 sdb1[2] sda1[0] sdc1[1]
      200704 blocks [3/3] [UUU]

unused devices: 
root@gregg-desktop:~# mdadm --manage /dev/md52 --fail /dev/sdc2
mdadm: set /dev/sdc2 faulty in /dev/md52
root@gregg-desktop:~# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid10]
md53 : active raid5 sdb3[2] sdc3[1] sda3[0]
      450703360 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md54 : active raid5 sdb4[2] sda4[0] sdc4[1]
      3453824 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md52 : active raid5 sdc2[3](F) sda2[0] sdb2[2]
      35664128 blocks level 5, 64k chunk, algorithm 2 [3/2] [U_U]

md51 : active raid1 sdb1[2] sda1[0] sdc1[1]
      200704 blocks [3/3] [UUU]

unused devices: 
root@gregg-desktop:~# mdadm --manage /dev/md52 --remove /dev/sdc2
mdadm: hot removed /dev/sdc2
root@gregg-desktop:~# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid10]
md53 : active raid5 sdb3[2] sdc3[1] sda3[0]
      450703360 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md54 : active raid5 sdb4[2] sda4[0] sdc4[1]
      3453824 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md52 : active raid5 sda2[0] sdb2[2]
      35664128 blocks level 5, 64k chunk, algorithm 2 [3/2] [U_U]

md51 : active raid1 sdb1[2] sda1[0] sdc1[1]
      200704 blocks [3/3] [UUU]

unused devices: 
root@gregg-desktop:~#

Commands and notes also relevant for RAID conversions, etc.

Get the UUID of /dev/md0 - this is useful for /etc/fstab mounts

blkid | grep md0

List out disks or partitions

fdisk -l
fdisk -l /dev/sda

Make a partition swap

mkswap /dev/sda3 -L SWAP

Create a 1 disk RAID, then add another drive later on - these drives have 3 partitions already
Create the RAID partitions: level = RAID1, n=2 drives

mdadm --create /dev/md0 --level 1 -n 2 /dev/sdc1 missing
mdadm --create /dev/md1 --level 1 -n 2 /dev/sdc2 missing
mdadm --create /dev/md2 --level 1 -n 2 /dev/sda3 missing

Save the partitioning information

sfdisk -d /dev/sda > raidinfo-partitions.sda
 sfdisk -d /dev/sdb > raidinfo-partitions.sdb
 sfdisk -d /dev/sdb < raidinfo-partitions.sda

Format /dev/sdb - this drive had old RAID info on it, thus the "--force"

sfdisk --force /dev/sdb < raidinfo-partitions.sda

Add /dev/sdb into the RAID arrays

mdadm /dev/md0 -a /dev/sdb1
 mdadm /dev/md1 -a /dev/sdb2

Fail, remove, re-add a disk to an array

[root@gregg-desktop root]# mdadm /dev/md51 --fail /dev/sdc1
mdadm: set /dev/sdc1 faulty in /dev/md51
[root@gregg-desktop root]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] 
md53 : active raid5 sda3[0] sdb3[2] sdc3[1]
      450703360 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
      
md12 : active raid1 sdc2[0]
      17832064 blocks [2/1] [U_]
      
md54 : active raid5 sda4[0] sdb4[2] sdc4[1]
      3453824 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]
      
md51 : active raid1 sda1[0] sdb1[2] sdc1[3](F)
      200704 blocks [3/2] [U_U]
      
unused devices: 

[root@gregg-desktop root]# mdadm /dev/md51 --remove /dev/sdc1
mdadm: hot removed /dev/sdc1
[root@gregg-desktop root]# mdadm /dev/md51 --re-add /dev/sdc1
mdadm: re-added /dev/sdc1

mdadm

I have a friend that had his P3 windows box crash - it had 3 40GB drives.

I asked him if what he used the machine for - internet mostly - and pictures from the
digicam..
"Mind being a guinea-pig? I want to install Linux - I'll support and help you
when stuck - plus you'll never really get virus's.."

He said OK!!

So, I installed Ubuntu 8.04, got flash working, sound - had to do some mod work to get the nvidiafb module to install at boot time - else machine was dead slow..
I also asked (yes - do ask them, don't just add and creep folks out later) if I could install a front/backdoor - for support.. I set a cron to wget an image from my server every 1/2 hour, changed the SSH port, installed VNC.

Wow - past 2 years - when he added a printer, need something tweaked, I remote in - fire up VNC, do the work and bam.. No funny Windows-Tight VNC console 0 junk. It just works. Virus - what virus? Sure there are ways in Linux but its so much more work - why when there are millions of easier targets running Redmond OS???

Running Raid1 + a spare - nice for me - no visits yet for crashed drives but he needed more space for video's - so broke a 40GB drive off the mirror for data for a while...

Here are the notes..

See mirror components (this part got scrolled away in screen, I think this is what it looked like before the work)

cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sdb1[1] sdc[1]S sda1[0]
      240832 blocks [2/2] [UU]
      
md1 : active raid1 sdb2[1] sdc[2]S sda2[0]
      39961600 blocks [2/2] [UU]

# marked as failed

mdadm --fail /dev/md0 /dev/sdc1
mdadm --fail /dev/md1 /dev/sdc2

# verify its its failed

cat /proc/mdstat

# remove from array

mdadm --remove /dev/md0 /dev/sdc1
mdadm --remove /dev/md1 /dev/sdc2

# add new FS to part 2

mkfs.ext3 /dev/sdc2

# edit fstab

vi /etc/fstab
/dev/sdc2 /home/user/data ext3 defaults 0 2

# make mount point

mkdir /home/user/data; chown user:user /home/user/data

# mount it

mount -a

# verify its there

df -k

# don't check it that often

tune2fs -c 100 /dev/sdc2

rsync

rsync - lots of uses - for pulling files off a server -and- not saturating the DSL line:

rsync -avz --port=4321 -e "ssh -p 4321" --bwlimit=120 --progress user@example.com:/home/cpmove-user.tar.gz .

-avz a - archive mode (preserve links/perms/modes) v - verbose, z - compression
--port=4321 -e "ssh -p 4321" Use a different ssh port if needed...
--bwlimit=120 Expressed in kBps - my 3Mb/s AT&T DSL line tops out at about 278kB/s - I set this to ~40% of that - the wife appreciates it :)
--progress See the progress as rsync is working..

swaks - smtp testing

A great writeup is here

Here is an example for my own purposes:

user@ ~ $ swaks --to user@mochabomb.com \
> --from=thing@mochabomb.spiffy 
> --auth \
> --auth-user=testuser@mochabomb.com \
> --auth-password=mypassword \
> --server mochabomb.com:587  

=== Trying mochabomb.com:587:25...
=== Connected to mochabomb.com:587.
<-  220-vps2.mochabomb.info ESMTP ChickenLips Byte #Transporter version 12.0.314.159.26.53.59-2 rev 17 build 5682.126-05a Tue, 01 Sep 2009 23:05:59 -0700 
<-  220-   No one is authorized to use this fine system to transport unsolicited, 
<-  220    and/or bulk e-mail and other cruft.
 -> EHLO mydesktopbox.gotdns.com
<-  250-vps2.mochabomb.info Hello adsl-##-###-##-147.dsl.pltn13.sbcglobal.net [##.##.##.147]
<-  250-SIZE 52428800
<-  250-PIPELINING
<-  250-AUTH PLAIN LOGIN
<-  250-STARTTLS
<-  250 HELP
 -> AUTH LOGIN
<-  334 VXNlcyahooWU6
 -> YWeWdogpileLm1vYexcite@homejb20=
<-  334 UGFzyeehawQ6
 -> TDgoogleyM=
<-  235 Authentication succeeded
 -> MAIL FROM:
<-  250 OK
 -> RCPT TO:
<-  250 Accepted
 -> DATA
<-  354 Enter message, ending with "." on a line by itself
 -> Date: Tue, 01 Sep 2009 23:05:57 -0700
 -> To: user@mochabomb.com
 -> From: thing@mochabomb.spiffy
 -> Subject: test Tue, 01 Sep 2009 23:05:57 -0700
 -> X-Mailer: swaks v20061116.0 jetmore.org/john/code/#swaks
 -> 
 -> This is a test mailing
 -> 
 -> .
<-  250 OK id=1Miiym-0001Mr-8O
 -> QUIT
<-  221 vps2.mochabomb.info closing connection
=== Connection closed with remote host.

Pretty simple package - Debian the package is "swaks".

tar archive

tar -cvf - `find . -print` > ../whatever.tar

truss

babysit processes
like snoop

truss -f -a -e -o /tmp/truss.out batchtest_4.2

wget - copy a website

Clone a website - see the wget manpage for info..

$ wget -r -l 5 -k -o ./log.txt -v http://www.example.com/site1/index.php --progress=dot --limit-rate=100k --wait 1 --random-wait -x