在SuSE Linux Enterprise Server 8上安装Oracle 9iR2 RAC

How to install Oracle 9iR2 RAC on SuSE Linux Enterprise Server 8
================================================================
    --- Using the Oracle Cluster File System (OCFS) ---
(Warning: OCFS if for the database files *only*, it is not a general
 purpose filesystem and should not be used for regular files, e.g.
 for ORACLE_HOME)
Requires at least:
 * United Linux 1.0 (SuSE Linux Enterprise Server 8 is "Powered by UL 1.0")
 * UL Kernel update for Oracle, at least these version numbers:
   k_smp-2.4.19-196.i586.rpm     - SMP kernel, for almost all Oracle users
   k_deflt-2.4.19-207.i586.rpm   - Single CPU
   k_athlon-2.4.19-200.i586.rpm  - Optimized for AMD Athlon
   k_debug-2.4.19-164.i586.rpm   - Debug kernel
   k_psmp-2.4.19-201.i586.rpm    - Support for *very* old Pentium CPUs
   kernel-source-2.4.19.SuSE-152.i586.rpm
 * orarun.rpm: Version 1.8 or greater
Tip: There is an easy way to work on all nodes simultaneously! Simply open
     a KDE "Konsole" and in it open one terminal for each node. Now log
     into each node on each of the terminals. After that, under "View"
     in the "KDE Konsole" menu enable "Send Input to All Sessions" for
     one of the terminals. Now, whatever you type in this session is also
     sent to the other sessions, so that you work on all nodes 
     simultaneously! This greatly reduces the amount of typing you have
     to do! If you do that, remember a few things: The node names, IPs etc.
     will be different on each node. The shell history may be different on
     each node. "vi" remembers where in a file you left off - so if you
     edit a file on all nodes simultaneously first check that the cursor
     is in the same position in the file on all terminals. And so on -
     check what's going on on the other terminals often (SHIFT left/right
     arrow makes this a very quick and painless exercise)!!!  
Tip: Use "sux" instead of "su" and X-server permissions and setting DISPLAY
     happens automatically! 
Tip: If you work in a noisy server room: Get a Bose noise canceling headset
     which is sold for frequent flyers. We found it very valuable in server
     rooms, too! 
ALL NODES
---------
- Install SLES-8 and the latest Service Pack, esp. orarun (at least version
  1.8) and kernel updates.  Make sure to satisfy all dependencies of
  package "orarun"!
  SP1: United Linux Service Pack #1. Alternatively, get at least the orarun
  package from:
  ftp://ftp.suse.com/pub/suse/i386/supplementary/commercial/Oracle/sles-8/
  Kernel- and other updates are available on the SuSE Maintenance web and
  can be installed conveniently via YOU (Yast Online Update).
- You may have to add "acpi=off" to the boot options if the system hangs
  during boot! Selecting "Safe Settings" as boot option includes that.
- set the password for user oracle: as root do "passwd oracle"
- optional: create /home/oracle:
    cp -a /etc/skel /home/oracle
    chown oracle:oinstall /home/oracle
    usermod -d /home/oracle oracle
- QLogic FC cards: If you use QLogic cards, be prepared for 1-2 minutes of
  "thinking" of your system when the qla... driver is inserted! 
  Also be prepared to edit /etc/sysconfig/kernel to add the qla... driver
  manually, because some cards are not detected automatically. There're
  the two driver modules "qla2200" and "qla2300", which card you have you
  can see in /proc/pci. After editing that file run (as root) "mk_initrd",
  and if your boot loader is LILO also run "lilo" (default bootloader
  in UL is Grub).
- remove gcc 3.2 ("rpm -e gcc --nodeps") to be sure it's not used - we prefer
  an error message during installation over inadvertently using gcc 3.2
  If you choose not to remove it you have to edit $ORACLE_HOME/bin/genclntsh
  as well as $ORACLE_HOME/bin/genagtsh
  and add "/opt/gcc295/bin" *in front* of the PATH variable set in those
  scripts! Then do "relink all".
- in file /etc/sysconfig/suseconfig set
    CHECK_ETC_HOSTS="no"
    BEAUTIFY_ETC_HOSTS="no"
- Tip: if you use NFS at all (we have a separate document based on this 
  one explaining how to setup RAC on a shared NFS mount), either mount
  with option "nolock", or also start the locking daemon (indeed needed
  on the nfs CLIENTS too!):
   chkconfig nfslock on; chkconfig nfs on; rcnfslock start; rcnfs start
- setup the network interfaces (internal/external)
- set up /etc/hosts and /etc/hosts.equiv (for rsh)
- Install ocfs from www.ocfs.org/ocfs/
- Install ocfs tar archive (the tools) from same URL, by unpacking in /
  and then remove /ocfs.o
- rm /etc/ocfs*
- run ocfstool, press CTRL+G, ENTER, CTRL-Q (creates /etc/ocfs.conf)
- edit /etc/ocfs.conf to add node number (obvious, already there but empty)
  Note: You HAVE TO start at "0" for the first node!!!
- rcoracle start 
  This sets the kernel parameters *before* we even start the
  installation! 
- edit /etc/inetd.conf (or use yast2 module for inetd), remove the "#" 
  in front of "shell..." and "login...", one service is needed for "rsh",
  the other one for "rcp"
  (there are two lines for each, the one with the additional "-a" option
   does hostname verification via reverse lookup before accepting a
   connection)
- as root, do "chkconfig inetd on" and "rcinetd on" (for immediate start)
- Check if you can "rsh" and "rcp" - as user oracle - from any node to 
  any other node in the cluster
- install and configure xntpd to synchronize the time/date on all nodes
  example: my ntp.conf
- edit /etc/profile.d/oracle[c]sh and set ORACLE_SID to some SID[#nodenumber]
  for each node
- add to fstab: 
  /dev/system/oracle  /var/opt/oracle  ocfs  defaults  0 0
NODE #1 (installation node)
---------------------------
- Optional: Set up LVM if you want to. You can use OCFS on partitions or on
  logical volumes, it's the same instructions. Below we use a logical volume
  "oracle" on volume group "system". 
- /sbin/load_ocfs
- on one node only:
  mkfs.ocfs -F -b 128 -L oracle -m /var/opt/oracle/ -u 59 -g 55 -p 0775 /dev/system/oracle
- mount -t ocfs /dev/system/oracle /var/opt/oracle
  umount /var/opt/oracle
  Why: The FIRST time you mount this it must be done on one node
  ONLY, then the others. This is because the filesystem is initialized when
  mounted for the first time. Afterwards you can mount it simultaneously
  on all nodes.
- as oracle ./runInstaller - Install cluster manager
  For quorum file enter this: /var/opt/oracle/oracm
- as oracle: ./runInstaller - change source to 9202 patchset and install
  9202 patch for "Cluster Manager"
- edit /etc/sysconfig/oracle to enable start of OCFS, OCM and GSD
  (GSD will work only later after the full software is installed)
    START_ORACLE_DB_OCFS="yes"
    START_ORACLE_DB_OCM="yes"
    START_ORACLE_DB_GSD="yes"
- Start/stop the oracm on one node - the very first time it starts it
  will create the shared quorum file. If started simultaneously on other
  nodes when this file doesn't exist yet creates conflicts and oracm starts
  only on some nodes, a question of random timing! Once this file exists there
  is no problem any more! 
    rcoracle start
    rcoracle stop
ALL NODES
---------
- rcoracle start
  Starts OCFS, OCM, and hangcheck-timer (called "iofence-timer" by oracm)
  If you didn't install our Oracle update kernel you will get an error
  about a missing module "iofence-timer"!
- On each node: Check processes and $ORACLE_HOME/oracm.log/cm.log if oracm is up.
  Check /var/log/messages and cm.log if there are problems.
  
  The end of cm.log should look like this (here: 4 nodes):
  ....
  HandleUpdate(): SYNC(2) from node(0) completed {Thu Feb 13 18:20:19 2003 }
  HandleUpdate(): NODE(0) IS ACTIVE MEMBER OF CLUSTER {Thu Feb 13 18:20:19 2003 }
  HandleUpdate(): NODE(1) IS ACTIVE MEMBER OF CLUSTER {Thu Feb 13 18:20:19 2003 }
  HandleUpdate(): NODE(2) IS ACTIVE MEMBER OF CLUSTER {Thu Feb 13 18:20:19 2003 }
  HandleUpdate(): NODE(3) IS ACTIVE MEMBER OF CLUSTER {Thu Feb 13 18:20:19 2003 }
  NMEVENT_RECONFIG [00][00][00][00][00][00][00][0f] {Thu Feb 13 18:20:20 2003 }
  Successful reconfiguration,  4 active node(s) node 0 is the master, my node num is 0 (reconfig 3) {Thu Feb 13 18:20:20 2003 }
NODE #1 (installation node)
---------------------------
- as oracle: export SRVM_SHARED_CONFIG=/var/opt/oracle/SharedConfig
- as oracle: ./runInstaller
  Select "software only", i.e. no database creation (we want to upgrade to
  patchset 9202 first)
  Exit the installer. 
- as oracle: in $ORACLE_BASE/oui/bin/linux/, do:
  ln -s libclntsh.so.9.0 libclntsh.so
- as oracle: runInstaller
  As source select the 920x patchset directory (./stage/products.jar)
  Install 920x patchset (we already patched the Cluster Manager earlier)
- copy the installation node file /etc/oratab to the same location on all
  other nodes, making sure the owner (oracle:oinstall) remains the same!
ALL NODES
---------
- rcoracle stop
- rcoracle start
  So that this time GSD is started too, which we only just installed 
  GSD is needed by OEM and by dbca
- Go to $ORACLE_BASE and create a link to the shared NFS mounted directory
    cd $ORACLE_BASE
    ln -s /var/opt/oracle oradata
  (assuming you mounted the OCFS directory under /var/opt/oracle)
- Go to $ORACLE_HOME and create a link to the shared NFS mounted directory
    cd $ORACLE_HOME
    rm -rf dbs
    ln -s /var/opt/oracle dbs
  (assuming you mounted the OCFS directory under /var/opt/oracle)
- The installer "forgot" all log directories on the other nodes
  when copying the software from the installation node:
    mkdir $ORACLE_HOME/rdbms/log
    mkdir $ORACLE_HOME/rdbms/audit
    mkdir $ORACLE_HOME/network/log
    mkdir $ORACLE_HOME/network/agent/log
    mkdir $ORACLE_HOME/ctx/log
    mkdir $ORACLE_HOME/hs/log
    mkdir $ORACLE_HOME/mgw/log
    mkdir $ORACLE_HOME/srvm/log
    mkdir $ORACLE_HOME/sqlplus/log
    mkdir $ORACLE_HOME/sysman/log
FINISHED
--------
Now the software is installed and ready, and the cluster manager
and the GSD are up and running, we are ready to create a database! 
NODE #1 (installation node)
---------------------------
- (as of orarun-1.8-8 this is done by "rcoracle start" when oracm is started)
  as root: 
    touch /etc/rac_on
  A dirty Oracle hack, see last 5 lines of $ORACLE_HOME/bin/dbca for what it does
  if you are interested. Or: edit "dbca" and you don't need this "touch" command.
- Tip: (as of orarun-1.8-8 this is done by /etc/profile.d/oracle.sh - for user
   oracle only)
  to run gsdctl or gsd or srvconfig you have to do this in the same shell:
    unset JAVA_BINDIR JAVA_HOME 
- as oracle: run "netca" to create an Oracle network configuration
  The Network Configuration Assistant should detect the running cluster
  manager and offer a cluster configuration option! You must at least 
  configure a listener. You can accept all the defaults, i.e. simply
  press NEXT until the listener configuration is done. 
- run "lsnrctl start" on ALL NODES. 
- as oracle: dbca -datafileDestination $ORACLE_HOME/dbs
  Set up a database. Without the -datafileDestination parameter dbca 
  assumes (and checks for!) raw devices which we don't use here! 
  If there's an error right at the start, try restarting the cluster 
  manager and GSD via "rcoracle stop; rcoracle start". 
- edit /etc/sysconfig/oracle to start additional services, e.g. the 
  Oracle listener. If you set START_ORACLE_DB="yes" you have to edit
  /etc/oratab (on ALL NODES) and change the last letter in the line for
  your database (usually just one line, at the bottom) to "Y" or no
  database will be started.
URL: http://www.suse.com/oracle/
Contact: oracle@suse.com