Raw devices for Oracle on RedHat (RHEL) 5

There is a major confusion among DBAs regarding how to setup raw devices for Oracle RAC or Oracle Clusterware. This confusion is caused by the turn RedHat took in how to define raw devices.

Raw devices are actually a manifestation of character devices pointing to block devices. Character devices are non-buffered, so they act as FIFO, and have no OS cache, which is why Oracle likes them so much for Clusterware CRS and voting.

On other Unix types, commonly there are two invocations for each disk device – a block device (i.e /dev/dsk/c0d0t0s1) and a character device (i.e. /dev/rdsk/c0d0t0s1). This is not the case for Linux, and thus, a special “raw”, aka character, device is to be defined for each partition we want to participate in the cluster, either as CRS or voting disk.

On RHEL4, raw devices were setup easily using the simple and coherent file /etc/sysconfig/rawdevices, which included an internal example. On RHEL5 this is not the case, and customizing in a rather less documented method the udev subsystem is required.

Check out the source of this information, at this entry about raw devices. I will add it here, anyhow, with a slight explanation:

1. Add to /etc/udev/rules.d/60-raw.rules:

ACTION==”add”, KERNEL==”sdb1″, RUN+=”/bin/raw /dev/raw/raw1 %N”

2. To set permission (optional, but required for Oracle RAC!), create a new /etc/udev/rules.d/99-raw-perms.rules containing lines such as:

KERNEL==”raw[1-2]“, MODE=”0640″, GROUP=”oinstall”, OWNER=”oracle”

Notice this:

  1. The raw-perms.rules file name has to begin with the number 99, which defines its order during rules apply, so that it will be used after all other rules take place. Using lower numbers might cause permissions to be incorrect.
  2. The following permissions have to apply:
  • OCR Device(s): root:oinstall , mode 0640
  • Voting device(s): oracle:oinstall, mode 0666
  • You don’t have to use raw devices for ASM volumes on Linux, as the ASMLib library is very effective and easier to manage.

    Tags: , , , , , , , ,

    5 Responses to “Raw devices for Oracle on RedHat (RHEL) 5”

    1. Persistent raw devices for Oracle RAC with iSCSI | Running Systems Says:

      […] a system with persistent disk naming, follow this process, however, on a system with changing disk names (every reboot names are different), the process can […]

    2. jay Says:

      why redhat makes easy stuff more complicated, i cannot understand this move!!! frustrating…

    3. ez-aton Says:

      This seems to be more complicated. It consolidates several different hardware marking methods into one. This seems more complicated, but still – allows for more flexibility (see my post about mapping changing LUNs (and sd device names…) here: http://run.tournament.org.il/persistent-raw-devices-for-oracle-rac-with-iscsi ) and can manage changing and larger volumes of devices. The raw abilities is only a minor part of it. We should learn and understand this mechanism, as it is rather useful.
      Ez

    4. Don V Says:

      More complicated? Hardly – it’s a couple text files, no more or less difficult than before.

      Red Hat making it difficult? Again, not hardly, RAW is a stupid way to implement – use ASM.

      See also:
      http://kbase.redhat.com/faq/docs/DOC-10164

      RAW will be gone soon enough – get used to it.

    5. ez-aton Says:

      True. But you cannot place CRS and voting files on ASM. Which means you either need OCFS2, which introduces another clustering mechanism (DLM with o2cb) or you use RAW disks. There are no other options currently.

    Leave a Reply