Building an Exadata Compute Node USB imaging drive

I’ve been having a lot of fun recently with doing a bare metal restore of an Exadata compute node. This process is reasonably well documented in MOS: 1084360.1. However one area where it falls down on, is how to actually image the node.

So here is my guide to building an Exadata Compute Node USB imaging drive. This will use Virtualbox on OS X. Yes, you could build it on an existing Exadata node.

Obtaining Exadata Software

First thing you need to do is download the correct version you are after. Yes, you can grab it from Oracle edelivery, as long as the version you are after is not too old. There are choices for the various versions and separate downloads for compute nodes and storage nodes. You will download a zip file. When unzipped, assuming it is a compute node you are building you will find a computeImageMake_VERSION.tar.zip. You need to uncompress and then untar this as root

You will now have a dl360 directory. Just happens that dl360 is a model of HP server. If you are building a storage node, it will be dl180, also a type of HP Proliant server.

Descending to this newly created directory you will see a readme: README_FOR_FACTORY.txt, which explains how to build the image. You will also see makeImageMedia.sh which actually does the building of the image.

Building the Image

Using a Linux VM and VirtualBox. present the USB stick to the VM. You need at least 2GB in size.

[root@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       14G   11G  2.3G  83% /
/dev/sda1              99M   13M   82M  14% /boot
tmpfs                 502M     0  502M   0% /dev/shm
Downloads             297G  210G   88G  71% /media/sf_Downloads
/dev/hdc               43M   43M     0 100% /media/VBOXADDITIONS_4.1.6_74713
/dev/sdb1             3.8G  888K  3.8G   1% /media/UNTITLED

Unmount the device:

[root@localhost ~]# umount /dev/sdb1

Use fdisk to remove the FAT32 formatting and create a partition:


[root@localhost ~]# fdisk /dev/sdb

Command (m for help): p

Disk /dev/sdb: 4041 MB, 4041211904 bytes
255 heads, 63 sectors/track, 491 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         492     3946495    b  W95 FAT32
Partition 1 has different physical/logical beginnings (non-Linux?):
     phys=(1023, 254, 63) logical=(0, 0, 3)
Partition 1 has different physical/logical endings:
     phys=(1023, 254, 63) logical=(491, 80, 37)

Command (m for help): d
Selected partition 1

Command (m for help): p

Disk /dev/sdb: 4041 MB, 4041211904 bytes
255 heads, 63 sectors/track, 491 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-491, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-491, default 491):
Using default value 491

Command (m for help): p

Disk /dev/sdb: 4041 MB, 4041211904 bytes
255 heads, 63 sectors/track, 491 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         491     3943926   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

This is the crucial part, the build will fail if it is formatted with FAT when you try and create the image.

Now you are ready to run the makeImageMedia.sh script:

[root@localhost dl360]# ./makeImageMedia.sh

Please wait. Calculating md5 checksums for cellbits ...
Please wait. Making initrd ...
180027 blocks
Please wait. Calculating md5 checksums for boot ...

Choose listed USB devices to set up the Oracle CELL installer

sdb   Approximate capacity 3946 MB
Enter the comma separated (no spaces) list of devices or word 'ALL' for to select all: ALL
sdb will be used as the Oracle CELL installer

All data on sdb will be erased. Proceed [y/n]? y

Command (m for help): Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): Command action
   e   extended
   p   primary partition (1-4)
Partition number (1-4): First cylinder (1-491, default 1): Last cylinder or +size or +sizeM or +sizeK (1-491, default 491):
Command (m for help): The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
umount2: Invalid argument
umount: /dev/sdb1: not mounted
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
492032 inodes, 983973 blocks
49198 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1010827264
31 block groups
32768 blocks per group, 32768 fragments per group
15872 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736

Writing inode tables: done                           
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
Copying files... will take several minutes


    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

 [ Minimal BASH-like line editing is supported.  For the first word, TAB
   lists possible command completions.  Anywhere else TAB lists the possible
   completions of a device/filename.]
grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"...  15 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded
Done.
grub> Done creation of installation USB for DL360

This will now have built your image on the USB drive. It is now a good idea to test that it really has worked and contains the software you require.

Testing your Image

There are a couple of ways of testing you have the image you want. Thanks to my e-dba colleague, Matthew Walden, for pointing these out.

First off, present again the USB drive to your VM.

On OS X when inserting an Exadata USB image, it is unrecognised by the OS and you see this dialogue box:

You need to choose eject. Then you can attach the USB drive to the VirtualBox VM. Once this is done, the following folder will appear:

You can look inside the image.id file to check what the image was built with:

You can clearly see this is a Compute node and is built with the 11.2.1.2.3 software version.

You can also test that your image will boot using VirtualBox and the plop bootloader. This allows you to select a USB device to boot from in VirtualBox. It downloads to an iso file which you boot a VM from. You then see the following menu:

After presenting your USB drive containing the Exadata image to the VM, you can select the USB device to boot from. Eventually you will see the following splash screen:

The above steps should give you some confidence that the USB image you have created will actually boot and Exadata node.

Note the above is only intended to for bare metal restore or reimaging of an Exadata node. In later software versions mounting an iso image via virtual cdrom on the iLOM works fine and would be my preferred solution.

Advertisements

10 thoughts on “Building an Exadata Compute Node USB imaging drive

  1. Hi,
    Created a USB image & tried booting the ‘cell node’ (actually a VM session) with the generated image and got this:

    [ERROR] get_node_type: Not a supported hardware model Virtualbox. dmidecode -s system-product-name returns invalid value

    Ever came across this?

    Thanks

  2. I tried creating an ISO image instead USB and it went fine then i mounted the ISO image but got the bellow error :

    [ERROR] get_node_type: Not a supported hardware model Virtualbox. dmidecode -s system-product-name returns invalid value
    followed by :
    login id prompt and password.

    I tried using root and welcome1 but no luck.

    • I am only suggesting you use VirtualBox to test that you built a bootable image. Clearly there is a boot check in there to determine that it is running on the supported hardware. the dmidecode is the standard practice for determining what hardware you are running on X4170 etc.

      I too have seen this, but when you run the image or indeed ISO on the correct hardware it works. I am only suggesting you run this on Oracle Exadata machines.

      jason.

  3. Last week I wanted to re-image the Compute and Cell Nodes on Exadata V1 HP machine. I used the latest Exadata images, 11.2.3.1.1. Everything was good on installing Cell Servers. But when I tried to re-image Compute nodes, I got the serious problem. Upon its bootup, the compute node did not detect the USB stick as a bootable media so that it booted up with the pre-existing kernel from HDD. Although I made the imaging disk several times again and again, all were same.
    So I made the ISO file using the files on the USB and virtually mount it under ILOM. Surprisingly, it could boot up with it. So I found out that the USB stick has right image for Exadata Compute node because there’s no problem for booting up.

    Can you suggest why the system does not detect the USB stick as a bootable media upon starting up?

    Thank you so much.
    Han

    • Hello Han,

      One possibility of course, could be you have not changed the boot order of the compute node? Is the USB an option to boot from on the compute node?

      I also wonder if it could be something to do with the firmware version you are at? Have you tried upgrading the iLOM firmware independently, then trying the USB again?

      These are just some possibilities.

      jason.

    • Change the Boot Device priority first order to USB
      i) Reboot the cell node
      ii) Press F2 to enter BIOS screen
      iii) Set USB as first Boot Device priority
      iv) Save and Exit.
      now check CELL_USB_BOOT_CELLBOOT_usb_in_rescue_mode list

    • Hi,

      This is available from oracle e-delivery when you download the exadata software. I believe you need to own an exadata to download this software.

      jason.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s