I’ve been having a lot of fun recently with doing a bare metal restore of an Exadata compute node. This process is reasonably well documented in MOS: 1084360.1. However one area where it falls down on, is how to actually image the node.
So here is my guide to building an Exadata Compute Node USB imaging drive. This will use Virtualbox on OS X. Yes, you could build it on an existing Exadata node.
Obtaining Exadata Software
First thing you need to do is download the correct version you are after. Yes, you can grab it from Oracle edelivery, as long as the version you are after is not too old. There are choices for the various versions and separate downloads for compute nodes and storage nodes. You will download a zip file. When unzipped, assuming it is a compute node you are building you will find a computeImageMake_VERSION.tar.zip. You need to uncompress and then untar this as root
You will now have a dl360 directory. Just happens that dl360 is a model of HP server. If you are building a storage node, it will be dl180, also a type of HP Proliant server.
Descending to this newly created directory you will see a readme: README_FOR_FACTORY.txt, which explains how to build the image. You will also see makeImageMedia.sh which actually does the building of the image.
Building the Image
Using a Linux VM and VirtualBox. present the USB stick to the VM. You need at least 2GB in size.
[root@localhost ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup00-LogVol00 14G 11G 2.3G 83% / /dev/sda1 99M 13M 82M 14% /boot tmpfs 502M 0 502M 0% /dev/shm Downloads 297G 210G 88G 71% /media/sf_Downloads /dev/hdc 43M 43M 0 100% /media/VBOXADDITIONS_4.1.6_74713 /dev/sdb1 3.8G 888K 3.8G 1% /media/UNTITLED
Unmount the device:
[root@localhost ~]# umount /dev/sdb1
Use fdisk to remove the FAT32 formatting and create a partition:
[root@localhost ~]# fdisk /dev/sdb Command (m for help): p Disk /dev/sdb: 4041 MB, 4041211904 bytes 255 heads, 63 sectors/track, 491 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 492 3946495 b W95 FAT32 Partition 1 has different physical/logical beginnings (non-Linux?): phys=(1023, 254, 63) logical=(0, 0, 3) Partition 1 has different physical/logical endings: phys=(1023, 254, 63) logical=(491, 80, 37) Command (m for help): d Selected partition 1 Command (m for help): p Disk /dev/sdb: 4041 MB, 4041211904 bytes 255 heads, 63 sectors/track, 491 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-491, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-491, default 491): Using default value 491 Command (m for help): p Disk /dev/sdb: 4041 MB, 4041211904 bytes 255 heads, 63 sectors/track, 491 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 491 3943926 83 Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
This is the crucial part, the build will fail if it is formatted with FAT when you try and create the image.
Now you are ready to run the makeImageMedia.sh script:
[root@localhost dl360]# ./makeImageMedia.sh Please wait. Calculating md5 checksums for cellbits ... Please wait. Making initrd ... 180027 blocks Please wait. Calculating md5 checksums for boot ... Choose listed USB devices to set up the Oracle CELL installer sdb Approximate capacity 3946 MB Enter the comma separated (no spaces) list of devices or word 'ALL' for to select all: ALL sdb will be used as the Oracle CELL installer All data on sdb will be erased. Proceed [y/n]? y Command (m for help): Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): Command action e extended p primary partition (1-4) Partition number (1-4): First cylinder (1-491, default 1): Last cylinder or +size or +sizeM or +sizeK (1-491, default 491): Command (m for help): The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. umount2: Invalid argument umount: /dev/sdb1: not mounted mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 492032 inodes, 983973 blocks 49198 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1010827264 31 block groups 32768 blocks per group, 32768 fragments per group 15872 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 25 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. Copying files... will take several minutes GNU GRUB version 0.97 (640K lower / 3072K upper memory) [ Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists the possible completions of a device/filename.] grub> root (hd0,0) Filesystem type is ext2fs, partition type 0x83 grub> setup (hd0) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded. succeeded Running "install /grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded Done. grub> Done creation of installation USB for DL360
This will now have built your image on the USB drive. It is now a good idea to test that it really has worked and contains the software you require.
Testing your Image
There are a couple of ways of testing you have the image you want. Thanks to my e-dba colleague, Matthew Walden, for pointing these out.
First off, present again the USB drive to your VM.
On OS X when inserting an Exadata USB image, it is unrecognised by the OS and you see this dialogue box:
You need to choose eject. Then you can attach the USB drive to the VirtualBox VM. Once this is done, the following folder will appear:
You can look inside the image.id file to check what the image was built with:
You can clearly see this is a Compute node and is built with the 220.127.116.11.3 software version.
You can also test that your image will boot using VirtualBox and the plop bootloader. This allows you to select a USB device to boot from in VirtualBox. It downloads to an iso file which you boot a VM from. You then see the following menu:
After presenting your USB drive containing the Exadata image to the VM, you can select the USB device to boot from. Eventually you will see the following splash screen:
The above steps should give you some confidence that the USB image you have created will actually boot and Exadata node.
Note the above is only intended to for bare metal restore or reimaging of an Exadata node. In later software versions mounting an iso image via virtual cdrom on the iLOM works fine and would be my preferred solution.