~sagar

This is “NOT” a HowTo for setting up a ROCKS Cluster, but I tried to show ya off some of my try outs and some aftermath.

If ya are new to ROCKS; Please refer the well equipped ROCKS User’s Guide or ya might be lost.

I used VERSION:-4.1 [Rocks v4.2 Beta is released for i386 and x86_64 CPU architectures are avail now]
and my cluster details are registered here

Frontend a.k.a Head Node installation is just a breeze, only if ya refer the manual.

To saY a word about frontend installation, what is your requirement so what rolls ya need to select.

BASE DISK
0.Area51 Roll :- For added security features like Tripwire and chkrootkit. Opt-out, if you really not othered about high-funda security.
1.Viz Roll :- Visualization, you don’t required unless you have a big and tiled monitor.
2.hpc :- Yes, I am into HPC lane
3.Ganglia :- To show off my cluster set-up and obviously for cluster’s health monitoring.
4.Web-server :- Yes.
5.Kernel Roll :- Yes.

OS DISK

Disk-1 and Disk-2 is sufficient, disks-3 and 4 are optinal

..and next, I did bind to our local ntp server.

DISK PARTITIONING -> Disk-druid for my 147 GB SCSI

/boot : 128 MB
/ : 15 GB
/usr/local : 20 GB ( For mannual installation og Globus and Torque scheduler)
/var : 25 GB (I expect a little more log)
swap : 2 GB
/myspace : 10 GB (For the non-cluster/local users home directory)
/export : Fill Available space

Now the installation has got over; system booted-up and no color (GUI) :-)

# system-config-display

To say, I had an issue and I dont want to see the smoke behind my flat BenQ. What I did was just copied
the /etc/Xll/xorg.conf file from another system with “same” hardware loadead with RedHat-AS-4.
I repeat… Linux, its a large file ! ;-)

Oh..yeah monitor, it’s single BenQ flat, shared over the systems with ATEN KVM switch.

#startx
…hoo·ray ! I got the color ( when ya logged in, the only difference I felt, there wasn’t any red HAT logo but centOS and the grub was different…. so Luke… its our shadow-man ! )
…then I stopped smartd service.

Compue Node Installation

I want the control over the compute node installation, atleast partitioning.

# cd /home/install/site-profiles/4.1/nodes/
Copy the skeleton.xml to extend-auto-partition.xml and edit extend-auto-partition.xml
++ refer the manual ^

I tried editing the manual option on the XML, showed strange and weired so I went with exyend-a-p.

# cd /home/install; rocks-dist dist [to apply this configuration to the distribution]
# insert-ethers
If your your frontend and compute nodes are connected via a managed ethernet switch, you’ll want to select ‘Ethernet Switches’ from the list above. This is because the default behavior of many managed ethernet switches is to issue DHCP requests in order to receive an IP address that clients can use to configure and monitor the switch.

Now, restart insert-ethers and continue reading the user guide for a procedure on how to configure your compute nodes.

# insert-ethers
and choose compute then wait [ Really, I felt I need patience, through out the set-up ] after putting the base cd to your compute node, restart and boot from the CD.
That’s it ( do remember ya have gotta PXE boot option, if you got CD-Drive outage :) )

Its fast..pretty fast and I finished my 2 compute nodes instllation in 3 minutes simultaniously.

You can monitor the installation of compute nodes by using ssh with p0rt 2200.

# ssh compute-0-0 -p 2200

Once the installation got over,
login: root
password: {frontend ’s root password }

# df -h; free
Good all the partitions and swap space are correct.

NO..ITS NOT CORRECT
…reallY… go to front end
0. check the XML file ( my problem was I put forward slash instead of / before part), what’s yours…?
1. # cd /home/install; rocks-dist dist [to apply this configuration to the distribution]
2. # rocks-partition –list –delete –nodename {compue node’s hostname}
3. Use the nukeit.sh script for removing .rocks-release from the first partition of each disk on the computenodes.
[ for nukeit.sh ]
4. # ssh {compue node’s hostname} ’sh /home/install/sbin/nukeit.sh’
5. # ssh {compue node’s hostname} ‘/boot/kickstart/cluster-kickstart’

Compute node restarted; check the default grub option; re-install, go ahead by ENTER.

Hic-cup Session
0. How do I run my Linpack HPL.dat?
Luke…refer the manual
1.How do I change frontend’s Public IP Address?

Don’t use {}

# echo ‘ update app_globals set value=”{newip}” where value=”{oldip}”‘ | mysql -u apache custer
# echo ‘ update networks set IP=”{newIP}” where IP=”{oldIP}”‘ | mysql -u apache cluster
# insert-ethers –update
2. My Ganglia status shows all/some of my compute nodes are dead but actuallY its running.
If ya tried the following…

[root@rocks mongoose]# cluster-fork /bin/date ; date
compute-0-0:
Sat Jul 8 04:30:39 IST 2006
compute-0-1:
Sat Jul 8 04:30:39 IST 2006

Sat Jul 8 04:30:39 IST 2006

[root@rocks mongoose]# cluster-fork service gmond restart
compute-0-0:
Shutting down GANGLIA gmond: [ OK ]
Starting GANGLIA gmond: [ OK ]
compute-0-1:
Shutting down GANGLIA gmond: [ OK ]
Starting GANGLIA gmond: [ OK ]

[root@rocks mongoose]# service gmond restart
Shutting down GANGLIA gmond: [ OK ]
Starting GANGLIA gmond: [ OK ]

root@rocks mongoose]# service gmetad restart
Shutting down GANGLIA gmetad: [ OK ]
Starting GANGLIA gmetad: [ OK ]

I refreshed the ganglia webpage
…then it showed Hosts Up = 1 (frontend) in while 1 changed to 2….. after sometime
it showed me
Hosts Up: 2 and hosts down=1
and now the case is back to Hosts Up=1 and Hosts Down=2.
Check multicas is enabled on your switch, blocking this on the networking device may cause the problem.

3.How do I manually broadcast 411 update instaead of hourly update.

# make -C /var/411 force
[You may have to use this just after creating a cluster-user on Frontend and to get updated across the nodes]

Disclaimer

All the above said materials are tested in a real time environment though Your Miles May Vary (YMMV)

System Installation Checklist for Server mongoose Dateded: 19-June-2006
=============++++++++++++++++++=============

This System Installation Check-list particularly designed for the the server mongoose.

0.System Information

Hostname : mongoose
Domain Name : mongoose.animals.org
IP Address : 192.168.63.82 (may change)
Serial No : B2-xxx-A05060-558
Platform : Intel Dual Xeon (2×3.6 Ghz), 1MB cache
OS Version : RedHat Advanced Server-4 (Kernel-2.6.9-5.ELsmp)
Disk Devices : 2×146GB
Raid Level : 0 [mirrorred]
Disk Storage : 146 GB
RAID Driver Disk : Adaptec Ultra SCSI [a320]

1.Drive Configurations

Filesystem Size Used Avail Use% Mounted on

/dev/sda7 4.9G 632M 4.0G 14% /
/dev/sda1 122M 12M 104M 10% /boot
none 1013M 0 1013M 0% /dev/shm
/dev/sda2 58G 107M 55G 1% /home
/dev/sda9 11G 485M 9.2G 5% /home/admin
/dev/sda6 15G 69M 14G 1% /opt
/dev/sda3 25G 1.8G 22G 8% /usr
/dev/sda5 20G 139M 19G 1% /var

2.Security Settings

a. Enabled SELinux Policy.
b. IP-Tables Firewall enabled except the services ssh, http, ftp, sendmail.

NOTE:
a. The home directory for local Administrator has assaigned as /home/admin
b. RedHat Network registration information.

Done by: Scooby Doo
Verified by: Shrek

Linux Security Checklist

Hey people stop reading…if the box ya want to make secure is not getting powered ON… ya got it…!!

Introduction

I gotta an assaignment to prepare a securitY check-list and here I make it general for anybodY who wanna have a look…I spent quite some time over the jungle…..
This crap maY provide ya some of the keY concepts that can go a long way in keeping a Linux system in secure[/insecure :-P].

General
0.Hardware
1.OS Distribution
2. File System Allocation( Disk Partitions)
3.OS Installation / Package Selection
4.Physical Security
5.Back-Ups
6.Expired Systems
7.Make a Boot and Rescue Media
8.Remove Unnecessary Software Package
9.Keep the System Patched and Up-to-Date
10.Set Off the Unnecessary Services
11.Disable the Unused Ports
12.Cross Check for Xinetd Services
13.Check Security on Key Files
14.User Account Management
15.Remove Unwanted/Zombie Files
16.Customized Banners
17.Harden the Services/Applications which are Required
0.nfs
1.ssh
2.ftp
3.xinetd
4.sendmail
5.apache (httpd)

18.Kernel Tunable Security Parameters
19.iptables
20.TCP Wrappers
21.Pluggable Authentication Module (PAM)
22.Proper System Logging
23.SELinux
24.Tripwire

General
To say ideally, the check list start right from the Hardware, OS Distribution, File System Allocation( Disk Partitions), OS Installation, Physical Security, Back-Ups and finally dump the system by ensuring that data can not be recovered from the Hard disk(s).

Hardware
Is that an OS distro certified hardware vendor?
Choose the hardware vendor who are good at customer support.
Choose the hardware, which meet our requirements (do we need a dual CPU, what is going to be its future role)
Have a plan for Annual Maintenance Contract (AMC) and how long we need it.

OS Distribution
This is all about our choice but must consider the facts, getting security updates, bug-fixes, enhancements and patch management within a short time-frame and in priority wise is an important step to be pro-actively secure the Linux System.

File System Allocation (Disk Partitions)
The system should have separate partitions to avoid “panics�?. This is just a DIVIDE & RULE Policy for better management and for recovery when we had troubles. Make separate partitions and allocate required space for /boot, /, /usr, /home, /var, /tmp and /opt for your optional and third party applications. This step is very important for both Production Servers, Workstations and Desktops (I mean to say, when you do a Linux installation)

OS Installation (Package Selection)
Do you need an Office Suite or xpdf to run your Database Server ? NO. So smart package selection avoid unwanted services and reduce the Risk Factor. May be the vulnerability is more for a package that you really never use.

Physical Security
The only truly secure system is one that is powered off, cast in a block of concrete and sealed in a lead-lined room with armed guards (Gene Spafford)
The systems should be in locked Server-Rack and locked room/datacenter. Physical access to the systems are restricted to authorized users. Set BIOS and Grub password (These days KVM switches can handle from BIOS level to avoid remote reboot chaos).
I am not saying anything hereabout Disaster Recovery Management and room Air Conditioning.

Back-Ups
Data are important for any level of organizations, so the back-up.
Simple back-up utilities are tar, gzip, bzip2, dump – for multiple level of back-up for the entire file-system, rsync – for transfer data between servers and keep in sync, amanda – for a client-server environment.

Expired Systems
Make sure the data can not be recovered from the hard-disks of the systems which is expired and not in use anymore. Disksanitizer is a tool to remove from all traces of data from the storage media according to the U.S. DoD standards.

Make a Boot and Rescue Media
…I just gotta finger pain…but to be continued…. (…where is the vicks bottle…hmm..)

:) I am not a MySQL expert so far…but here was my one day with the MySQL =>

I gotta requirement for MySQL Server version 5.x.x but my distro RH-AS-4 Update-1 (kernel-2.6.9-5) has MySQL-4.1.7. So I erased/un-installed all the MySQL RPMS [ rpm -e mysql* - -nodeps ]
and I choose the source bundle mysql-5.0.21, configured for a separate database on a different partition named /database and made install. Things were fine but some integration issue with Perl and PHP. Both are not able to connect MySQL (were I got screwed up).. I am sure, its not because of the php-mysql and perl-mysql packages Yeah… the default database comes under /var/lib/mysql now its /database/mysql/

I couldn’t see mysql module in php -m. Whats the solution “google” I gotta hell lot of out put that everybody saying I do have the same issue, some stamped this a as bug.

Well… now I am ready to go back to the packages ;
the RPM’s coming wih the distro [ rpm -ivh mysql*4.17* - -force :) ]

Here my DIVIDE & RULE Policy got worked. I umount the /database partition and mount to /var/lib

Issues: fstab is not ready to take the new partition
Solution: edit /etc/rc.local [ mount /dev/cciss/c0d0p11 /var/lib ]

Any luck…? the screw is still getting tight for me…

I gotta the following errors when I started using my middle finger to set this up….at different stages…

1. /usr/libexec/mysqld: Can’t change dir to ‘/var/lib/mysql/’
2. mysql error Errcode: 13
Error code 13: Permission denied [ you can try bash-3.00$ perror 13 ]
3. ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock
4. mysqld dead but subsys locked
5. /usr/libexec/mysqld: Can’t change dir to ‘/var/lib/mysql/’ (Errcode: 13)
6. Timeout error occurred trying to start MySQL Daemon.
7. ‘Problems running mysql_install_db’
8. Installation of system tables failed!

Woops..!! Dido is Stoned after The Sand In My Shoes …
Yokay..
I un-mount the /var/lib for my old /var/lib. Confused..well

/dev/cciss/c0d0p7 4.9G 155M 4.5G 4% /var [ Created at installation ]
/dev/cciss/c0d0p11 51G 144M 48G 1% /var/lib [ Newly mounted ]

:) because I have to back up all the files under “4% /var/lib “ to “1% /var/lib” with out loosing the permission settings.

cd /var/lib
find . -print -depth | cpio -pvdum ~admin/bkup_lib

Mounted back /var/lib to /dev/cciss/c0d0p11
Remove all files under /var/lib/ and once again use the find-cpio combination to place all the files back to ” 1% /var/lib/ “

Now the time to rpm -ivh mysql* [if you are “ivh” ing WITH OUT placing the files under /var/lib …heY..it’s gonna be a PITA with depedency and if those files are not with proper permission settings…..well… whats your numer from 1 to 8 ]

If all went fine so far; one request, dont use the mysql_install_db script now, but you can have a try and collect your number :)

Good…try this

[root@python ~] /usr/bin/mysqld_safe –user=mysql –skip-grant-tables &
[root@python ~] /usr/bin/mysql mysql

Yes..! it’s running; atleast for me but with one issue that I have to manually stop/kill the mysqld daemon.
There is NO service mysqld stop/restart …
I tried all the day to get it up and once it got up…hmmm… very funny…
Do ya have any hack around… :)

Dear All,

I am writing this mail as an information when we do installation of Linux Operating System by considering the manual partitioning (Diskdruid or fdisk).

People get annoyed after using Linux machine for a while by saying ” hey.. I got a Kernel Panic. I am worried about my data” or ” I don’t know where my GUI has gone”

One of the reason for this panic is, your ” / ” file system got squeezed ! ( say used space for / partition is 98% )

Please do comment, if you have got any suggestion.

I suggest; its a good practice to take care of this issue from the scratch; at the time of OS installation.

Here I am considering a 40GB HDD [** workstation specific NOT Server**]

By considering the fact; most of our machines are in dual boot so I don’t wanna deal with 15GB for Windows (15GB is fair enough for viruses to play and flood around :) )

well…the rest 25 GB.

This is just a DIVIDE & RULE Policy for better management and for recovery when we had troubles.

/boot

= 100MB [Make this as the first choice when you do partition, because older BIOS were not able to detect the second part of boot loader beyond 1024 cylinders of the HDD]

/usr

= 8 GB [ Happily we can deal this for a workstations, mine is 6 GB and 73% so far ]

swap

= Rule of Thumb; 2xRAM Size, but not always true [Try to have it on the middle part of the HDD, because its fast to access the middle portion of HDD]

/home

= 12 GB [ If you don’t have a dual boot, add much more or go for a separate user(you) defined partition to keep your data like documents, pdfs, mp3s and other stuffs ]

/

= 2 GB is more than enough. Yes I said 2048 MB *

/var

= 500 MB [ This separate partition avoid the electronic jamming of / by logfiles, mails and other junks..]
If you are using any RedHat distro and trying to configure MySQL, please consider much more space for the growing database which comes under /var/lib/mysql

/opt

= Are ya trying to install any applications like Oracle db &| its client ? Do you have any “optional” application which you don’t have to mess around ?. If the answer is “yes” go ahead and allocate desired space. I feel 3-4GB is okay. The best part is, you can remove the installed package under /opt as such, because all the files will come under that particular directory (directory=pkg-name) even the “bin” files. If you are not sure about this, add up this amount of space to your /usr or /home filesystem.

/tmp

= Normally, this never go beyond 100MB

NOTE:

1. Never log into your system as root. Log into as a normal user and configure your mail, desktop, browser and all other part which makes you comfortable. This way all your mails and other heavy stuff only fall under /home/[normalusr].

2. Do sudo or su - option when needed.

3. If you are in dual boot and have more space; its a good option to create FAT-32 partition by naming /winshare or something. So that we could access the data [pdfs, mp3s and other stuffs] from both OS.

Please do revert for any suggestion which you feel much practical or logical.

Thank You

~vipin

Contrary to popular belief, Unix is user friendly.
It just happens to be selective about who it makes friends with. — Dave Parnas

One of the hot stuff over the surf ajaxWrite (Asynchronous JavaScript and XML). I say this gonna be a killer application….
It silently says ” Killing is my bussiness and the bussiness is good!”

…and the foxY offered a good company for the Ace-Jack.

* Global access, all you need is an internet connection.
* Platform independent, you can use it with any operating system.
* Automatic updates and upgrades, no more computer restarts or missed patches/updates.
* Server side management, all the busywork is done for you.

http://www.ajaxlaunch.com/ajaxwrite/internals/ajaxwrite-noffox.html

Tail: It doesn’t work with internet explorer.
Did we mention it’s free? That’s right…

Microsoft Office Professional 2007TM - $499

ajaxWrite - $0

Hey..if you are a bailiwick and trying to fortify your security chklist, make sure be updated with latest security news and watch weekly advisories. Here I found some updates in one place.

Debian, Fedora, FreeBSD, Mandriva, Red Hat, and Ubuntu released security advisories this week. Affected packages include BMV, GPdf, Xpdf, pdftohtml, tar, Heimdal, PostgreSQL, and irssi-text. Fedora distributed a kernel update addressing several security vulnerabilities in the kernel. No security advisories were issued by Gentoo.
Got interest? go ahead…

When The Open Source rules the world…there will not be any traditional nomenclature for the kingdom other than Hacker, Geek, Guru, Nerd… and its different status….

So do ya wanna be a hacker…. or wanna know how to become a hacker…? or ya gotta a Q..?

Q: How can I get the password for someone else’s account?
A: This is cracking. Go away, idiot.

esr [ Eric Steven Raymond ]
….if you had watch the movie, “Revolution OS” probably no chance to forget his facial expression when he narrates about the incident happened inside a lift
(mnky..mnky)…I’ll be your worst nightmare…

That was my first time I came across to know about him in a Linux meet where they played the movie and later on I have read some of his essays from The C&B [ The Cathedral and the Bazaar ] and its worth.

This was another workaround I have done to set-up a gateway between the internal and external network to access the internal resource from outside.

Yes, there is no change, it’s a dedicated Linux machine for the Gateway installed with RedHat Advanced Server-4.

Setting Up The Gateway

I will just brief the set-up before I start into the configuration part.
Well… The machine has two NIC and configured accordingly:

eth0: 192.168.1.1/24 connecting to internal network.
Gateway: 192.168.1.254

eth1: 10.1.0.1/28 Connected to a Layer-3 switch(WAN Connection) and then it terminate at the users end whom gonna access the internal resource.
Gateway: 10.1.0.3.
The resource is in 192.168.2.0/24 network. Note not on the same network.

Yes.. thats a Q..! Dude why don’t you set it up on the Layer-3 Switch ?

hmmm… I don’t have the access to the CPE and that is dealt by the Service Provider. (We should always keep an alternate way to run out of the heck….) and moreover its a temporary set-up for a short period of time.

Next step enable the IP_forwarding between the NIC.

camel # vi /etc/sysctl.conf
net.ipv4.ip_forward = 0 # change this 0 to 1

You have done with the IP_forwarding and now the beauty; restart the system to apply the change…

â€?hey…I don’t wanna restart my system…â€?

Oho..really, then please talk to the kernel.

To pass the information to the kernel on the fly without restarting
camel # echo “1″ > /proc/sys/net/ipv4/ip_forward
or more easily by updating the sysctl by
camel # /sbin/sysctl -p

And now keep talking to the kernel about the routing table.

camel # netstat -nr shows the current kernel routing table.

I configured in the following

camel # route del default
camel # route add -net 192.168.2.0/24 gw 192.168.1.254 #now kernel knows all the request to the destination192.168.2.0 network should go via192.168.1.254 and the rest will taken care by internal routing table.
camel # route add default gw 10.1.0.3 #anythin else should go via 10.1.0.3

Now the external people has the access only to the 192.168.2.0 network after they logged into the gateway and get authenticated, thats the Security Hardening part. Yes.. the rest of the hardening part is coming right away…

After setting up the Gateway, the next Q was how could I H4RD3N this machine ?

Hardening A Linux Machine…huhh… You can write a book for that. Securing a Linux system called hardening can be done using both manual methods and open source security solutions. So I ask some Q to
myself:

0. What am I supposed to do with the system.
It should act as a gateway to access my resources from outside (dedicated) networks.

1. Does it has any wild connection to meet up any unknown people.
No; so far it is not connected to the Internet

2. How about users.
I am going to provide a common usrname and passwd, yeah…but I don’t know them personaly.

3. Should I allow the users to play with the system and keep their files.
NO, not even the execute permission. Please, no junk/bulky files.

4. After all, how do I monitor this box.
iptraf [its just a breeze]
…so my hardening process focuses on the operating system than any extra powerful tools.

STEPS TAKEN TO HARDENED THE LINUX BOX (GATEWAY)

0. The message before get authenticated.
Access to this computer system is restricted to personnel of the
[your wish is my command]. All connections are logged.
By attempting connection without permission, you are in violation of law and ethics.

1.a Edited /etc/motd
ACCESS RESTRICTED TO AUTHORIZED USERS ONLY

1. No: of users to access the system
a) root
b) admin (enable sudo)
c) Let there be users :)
2. Iptables and TCP wrappers enabled ( Allow access to SSH and HTTPD)

/etc/hosts.deny
sshd:ALL EXCEPT 10. 192.168.
httpd:ALL EXCEPT 10. 192.168.

3. Disable remote ssh as root; only console access.
4. ssh service enabled only for usr xxx and local networks.

Changes done on /etc/ssh/sshd_config
Protocol 2 restriction [Protocol 2]
PermitRootLogin no
Banner /etc/warn.txt [The file which contains the message,
that displays before get authenticated]
RhostsAuthentication no
IgnoreRhosts yes
RhostsRSAAuthentication no

5. Disable creating an executables, a device or a set-uid executables in /home directory

Changes done on /etc/fstab
LABEL=/home /home ext3 noexec,nodev,nosuid,usrquota 1 2
6. Set-up quota for usr xxx.
Soft limit 200MB and can be used 250MB maximum (grace period for 50 MB is 7 days)

7. Disable GCC for normal users; including Admin :) chmod 750

7-5-0 2 root root 94800 Feb 30 2004 /usr/bin/gcc

8. Disable all un-wanted service.

camel # chkconfig –list | grep on
camel # chkconfig –list | awk ‘/xinetd based services/,/”"/’
9. enable syslog service and configure iptraf
10. Keep updated with latest security news and watch weekly advisories were issued by vendors.
11. Rule of Thumb: click OO here
12. Reference - 0

UNIX System Hardening Checklist

SELinux
Unix Articles
Reference +1
Linux System Security: The Administrator’s Guide to Open Source Security Tools [ I own this book]