Grid Infrastructure Disk Issues– CHM DB file: crfclust.bdb

We had issues our Live production server grid Infrastructure disk usage increasing. 

I have come across an issue in 11gR2 RAC, where the GI file system GRID_HOME was mostly consumed by a single file called crfclust.bdb

This single file itself was about 132 GB of size.

crfclust.bdb is a Cluster Health Monitor (CHM) file, which collects the stats of Cluster as well as the OS statistics by means of the Cluster Health Monitor Service ora.crf.

-rw-r----- 1 root root   1814122496 May  3 16:32 crfhosts.bdb

-rw-r----- 1 root root 141540589568 May  3 16:32 crfclust.bdb

the below steps are followed to fix the issues,

adequately lower value. To do so, follow these steps:

1. Issue "$GI_HOME/bin/crsctl stop res ora.crf -init " on all the nodes of cluster.

2. Locate the config file $GRID_HOME/crf/admin/crf<hostname>.ora

3. Manually edit the crf<hostname>.ora file on every node of the cluster and

   change BDBSIZE tag entry and remove the value (set it to blank) or

   set it to a desired value, eg. 61511. Do not delete the BDBSIZE tag itself.

[root@vsnlmmdb06 admin]# less crfvsnlmmdb06.ora 

BDBLOC=default

BDBSIZE=67054080

4. Restart ora.crf daemon on every node.

[root@vsnlmmdb06 admin]# crsctl stop res ora.crf -init

CRS-2673: Attempting to stop 'ora.crf' on 'vsnlmmdb06'

CRS-2677: Stop of 'ora.crf' on 'vsnlmmdb06' succeeded

[root@vsnlmmdb06 vsnlmmdb06]# rm *.bdb

rm: remove regular file `crfalert.bdb'? yes

rm: remove regular file `crfclust.bdb'? yes

rm: remove regular file `crfconn.bdb'? yes

rm: remove regular file `crfcpu.bdb'? yes

rm: remove regular file `crfhosts.bdb'? yes

rm: remove regular file `crfloclts.bdb'? yes

rm: remove regular file `crfts.bdb'? yes

rm: remove regular file `repdhosts.bdb'? yes

[root@vsnlmmdb06 vsnlmmdb06]# crsctl start res ora.crf -init

CRS-2672: Attempting to start 'ora.crf' on 'vsnlmmdb06'

CRS-2676: Start of 'ora.crf' on 'vsnlmmdb06' succeeded

After started please check the location space will be reduced,