Wednesday, May 9, 2012

Check Point UTM-1 270 Upgrade Via CLI: R75.40

I always prefer to upgrade Check Point via CLI.  So, here is what I went through today on an upgrade from R75.30 to R75.40.  It actually failed, and Ill explain at the end of this explanation.  But for now, lets go through the upgrade process via CLI.

First, you must FTP the .TGZ image to the UTM, preferrably to a partition with enough disk space other than the system partition.  I sent it to /var/log.  Then, I ran the following command to strat the upgrade process:
[Expert@CP]# tar -zxvf Check_Point_Upgrade_for_R75.40.Splat.tgz

After it extracts the needed files, I get this error at the end:
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

However, I kept on going, as I saw this the last time when I upgraded from R75.20 to R75.30 and the install worked fine afterwards.
Run the following command to start the upgrade from CLI:
[Expert@CP]# ./

Once the upgrade completes, reboot the system.  Make sure it comes up.  Download the SmartConsole for the version you are going to, login, and push policy.  All should be good afterwards.

HOWEVER, Today was not like this.  I had a UTM-1 270 that had 79% of its root partition filled up on the hard disk.  This comes with 7.9 Gig of free space for the root partition from scratch.  Its not much, however there is more space on other partitions, which wont do you good for several upgrades.  Its real good for logging though.  I dont recall what it is, but its like 80 Gig or so. 

So, what I hear from Check Point (the first time) is that I need 2.5 Gig on the root directory for an upgrade to R75.40 from R75.30.  Turns out that now they tell me I need 4.9 Gig free (according to another TAC guy).  So, first, here are some commands that are useful in knowing about disk space on SPLAT:

Disk space in the current directory you are in:
[Expert@CP]# ls | xargs du -hs
0       1
796M    Check_Point_Upgrade_for_R75.40.Splat.tgz
0       CvpndAdmin.log
0       DEBUG
68K     DOCS
4.0K    License.txt
11M     bin
20M     boot
64K     dev
31M     etc
52K     home
4.0K    initrd
61M     lib
1.5G    log
16K     lost+found
32K     mnt
1.9G    opt
du: `proc/5355/task/5355/fd/3': No such file or directory
du: `proc/5355/fd/3': No such file or directory
0       proc
28K     root
14M     sbin
12K     scripts
0       sys
1.1G    sysimg
32K     tmp
76M     usr
7.0G    var

Overall disk space:
[Expert@CP]# df -h
Filesystem            Size  Used Avail Use% Mounted on
                      7.9G  7.1G  438M  95% /
/dev/sda1             145M   24M  114M  18% /boot
none                  501M     0  501M   0% /dev/shm
                       60G 1020M   56G   2% /var/log

So I did the upgrade, and I ran out of disk space.  It bombed said it completed, but failed to verify.  Essentially, I experienced this below:
[Expert@CP]# ./
Start Upgrading ..
Wait while creating upgrade image ... 23%
Creating upgrade image is ok
Verifying ..
Disk Space Error - 888960.97265625 Required, 0 AvailableVerification failed.

I was at 100% disk utilization on the root partition.  I didnt reboot because I was afraid it wouldnt come back up with the disk full like that.  So, I moved the /log directory (because it was 1.5 Gig) to another partition and then rebooted.  It didnt come back up from where I was (two floors up, physically), so I went down there to the server room and it says the following:
I have to admit, my heart sank a little when I saw this.  My first thought was "what am I about to encounter???"  I got my console cable out to see if I could see anything.  I got a blinking cursor that would respond if I hit enter, but I couldnt really do anything.  It was truly working on reverting back to R75.30.

So, in time, I finally go to this screen below for a few minutes:
When it got to the "loading" screen, thats when it started to reboot.  Below is what I saw on the console:
CPU Brand Name : Intel(R) Celeron(R) M processor          600MHz

  Memory Frequency For DDR2 400
IDE Channel 0 Master : None
IDE Channel 0 Slave  : None

SATA Channel 0 Master: WDC WD1601ABYS-01C0A0 06.06H05
SATA Channel 0 Slave : None
SATA Channel 1 Master: None
SATA Channel 1 Slave : None


Initializing Intel(R) Boot Agent GE v1.2.30
PXE 2.1 Build 084 (WfM 2.0)

                           Phoenix Technologies, LTD
                             System Configurations
| CPU T: Intel(R) Celeron(R) M processor  Base Memory       :    640K          |
| CPU I: 0695/45D                         Extended Memory   :1038336K          |
| CPU C: 600MHz                           Cache Memory      :    512K          |
| Diskette Drive A  : None                Display Type      : EGA/VGA          |
| Diskette Drive B  : None                Serial Port(s)    : 3F8 2F8          |
| Pri. Master Disk  : None                Parallel Port(s)  : None             |
| Pri. Slave  Disk  : None                DDR2 at Bank(s)   : 0 2              |
| Sec. Master Disk  : None                                                     |
| Sec. Slave  Disk  : None                                                     |

IDE Channel 2 . Master Disk  : LBA,ATA 100,  164GB

PCI device listing ...
Bus No. Device No. Func No. Vendor/Device Class Device Class               IRQ
    0       2         0     8086   2592   0300  Display Cntrlr               9
    0      29         0     8086   2658   0C03  USB 1.0/1.1 UHCI Cntrlr     11
    0      29         1     8086   2659   0C03  USB 1.0/1.1 UHCI Cntrlr     15

ACPI: Getting cpuindex for acpiid 0x1
ACPI: Getting cpuindex for acpiid 0x2
ACPI: Getting cpuindex for acpiid 0x3
├┐Red Hat nash version starting
  Reading all physical volumes.  This may take a while...
  Found volume group "vg_splat" using metadata type lvm2
  5 logical volume(s) in volume group "vg_splat" now active
INIT: version 2.85 booting
mount: proc already mounted
Configuring kernel parameters:  [  OK  ]
Setting clock  (utc): Wed May  9 15:59:07 GMT-5 2012 [  OK  ]
Starting udev: [  OK  ]
Setting hostname CPipacc:  [  OK  ]
Setting domain name  [  OK  ]
Initializing USB controller (ehci-hcd):  [  OK  ]
Your system appears to have shut down uncleanly
Press Y within 1 seconds to force file system integrity check...
Checking root filesystem
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/mapper/vg_splat-lv_current
/dev/mapper/vg_splat-lv_current: clean, 39865/1048576 files, 2093857/2097152 blocks
[  OK  ]
Remounting root filesystem in read-write mode:  [  OK  ]
Setting up Logical Volume Management: [  OK  ]
Finding module dependencies:  [  OK  ]
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /boot] fsck.ext3 -a /dev/sda1
/boot: recovering journal
/boot: clean, 78/38152 files, 29142/152586 blocks
[/sbin/fsck.ext3 (1) -- /var/log] fsck.ext3 -a /dev/mapper/vg_splat-lv_log
/dev/mapper/vg_splat-lv_log: recovering journal
/dev/mapper/vg_splat-lv_log: clean, 786/7864320 files, 1483684/15728640 blocks
[  OK  ]
Mounting local filesystems:  [  OK  ]
Activating swap partitions:  [  OK  ]
Enabling swap space:  [  OK  ]
INIT: Entering runlevel: 3
Applying Intel Microcode update: don't know how to make device "cpu/0/microcode"
/etc/rc3.d/S00microcode_ctl: microcode device /dev/cpu/0/microcode doesn't exist?
Checking for new hardware [  OK  ]
Updating /etc/fstab [  OK  ]
Starting WdHwSensors_init:  [  OK  ]
Starting lcdpanel_init:  [  OK  ]
Starting kdump:  [  OK  ]
Inserting vpntmod.2.6.18.cp.i686.noPAE: [  OK  ]
Starting s3500.boot:  [  OK  ]
CKP: Loading SecureXL:  [  OK  ]
CKP: Loading FW-1 IPv4 Instance 0:  [  OK  ]
CKP: Loading VPN-1     Instance 0:  [  OK  ]
Configuring network:  [  OK  ]
Starting SMBFS mounts:  [  OK  ]
Starting system logger: [  OK  ]
Starting kernel logger: [  OK  ]
Starting rmatool:  [  OK  ]
CPshell initialization:  [  OK  ]
Initializing random number generator:  [  OK  ]
Starting acpi daemon: [  OK  ]
Starting sshd:[  OK  ]
Starting arp:  [  OK  ]
Starting bp_init:  [  OK  ]
Starting crond: [  OK  ]
Running cp_http_server_wd: [  OK  ]
Running cpwmd_wd: [  OK  ]
Starting cpri_d:  [  OK  ]
Starting cpboot:  [  OK  ]
Starting cpboot_refetch:  [  OK  ]
Starting lcdpanel:  [  OK  ]
Starting led:  [  OK  ]
Starting ntp:  [  OK  ]

Check Point SecurePlatform R75.30
For Web User Interface access connect to https://X.X.X.X:4434/

When it came up, I tried to push policy and it would not push.  It just hung up on me.  
Again, for some reason, my disk space was full on the root partition.  There was a /log folder again, about 1.5 Gig in size, that I moved to the /var/log partition. I was then able to push policy after I moved the folder.  I also deleted the file for the R75.40 upgrade, which gave me more space.  At this point, I left it alone.  Im able to push policy and I have disk space.  Time to go home.