Thursday, January 14, 2016

Solaris/VxVM - Restore a Private Region on a VxVM disk.

PROBLEM: You are getting the following error when importing a disk group
# vxdg -C import kchdg
VxVM vxdg ERROR V-5-1-587 Disk group kchdg: import failed: Disk group has no valid configuration
A call was then opened with Veritas that requested the output of the following command which checks the configuration.
# vxdisk -o alldgs list
DEVICE       TYPE            DISK         GROUP        STATUS
Disk_0       auto:SVM        -            -            SVM
Disk_1       auto:SVM        -            -            SVM
EMC3_06B7    auto:cdsdisk    -            -            online
EMC3_054D    auto:cdsdisk    -            -            online
EMC4_0609    auto:cdsdisk    -            -            online
EMC4_0926    auto:cdsdisk    -            -            online
EMC5_2DEF    auto:cdsdisk    -            (kchquorumdg)    online
EMC5_2D70    auto:cdsdisk    -            (kchdg) online
EMC6_17A0    auto:cdsdisk    -            (kchquorumdg)    online
EMC6_27C6    auto:cdsdisk    -            (kchquorumdg) online
Get the disk names of the problem diskgroup. In this case it is EMC5_2D70
Get the underlying disks that are associated with this device.
# vxdisk list EMC5_2D70| tail -2
c3t5006048C52A863DDd144s2       state=enabled
c4t5006048C52A863D2d144s2       state=enabled

Take the cXtXdX name and run the following command
# /etc/vx/diag.d/vxprivutil dumpconfig /dev/rdsk/c3t5006048C52A863DDd144s2 |vxprint -D - -ht

This will dump the config of the private region on the disk, from this you can see if there are duplicates where there shouldn't be, if there is then your private region is corrupt and needs recreated.
First step to recreate and not have to reinitialize the disks etc. is to try and use a previous backup of the disk group. These are taken every time the disk group is updates so fingers crossed you have one.
# cd /etc/vx/cbr/bk/kchdg.1333032959.69.kchnode1
# ls -l
-rw-r--r--   1 root     root      655360 Apr  3 17:36 1333032959.69.kchnode1.binconfig
-rw-r--r--   1 root     root      655360 Apr  3 13:36 1333032959.69.kchnode1.binconfig.1
-rw-r--r--   1 root     root       15145 Apr  3 17:36 1333032959.69.kchnode1.cfgrec
-rw-r--r--   1 root     root       15070 Apr  3 13:36 1333032959.69.kchnode1.cfgrec.1
-rw-r--r--   1 root     root        1804 Apr  3 17:36 1333032959.69.kchnode1.dginfo
-rw-r--r--   1 root     root        1804 Apr  3 13:36 1333032959.69.kchnode1.dginfo.1
-rw-r--r--   1 root     root        2544 Apr  3 17:36 1333032959.69.kchnode1.diskinfo
-rw-r--r--   1 root     root        2544 Apr  3 13:36 1333032959.69.kchnode1.diskinfo.1

Try and restore from a previous config

# cat 1333032959.69.kchnode1.cfgrec |vxprint -D - -ht

# /opt/VRTS/bin/vxconfigrestore -p kchdg
Now the above command will either work or it won't, if it does then YAY if it doesn't then you will have to reinitialize the disks and volume group. But we are only touching the Private Region so all the data SHOULD be fine if we get our numbers correct.
Ok to reinitialize your private region you need a few imports bits of information, here is a sample output of the disk i am going to reinitialize.
# vxdisk list EMC5_2D70 | egrep -i '^public|^private'
public:    slice=2 offset=65792 len=4390528 disk_offset=0
private:   slice=2 offset=256 len=65536 disk_offset=0
Ok here is the command to initialize the disk, and the information for the command was found in the above output.
# vxdisk -f init EMC5_2D70 privoffset=256 privlen=65536 puboffset=65792 publen=4390528
Hope that makes sense?
Next you need to tell it what the subdisks are you can get this info from your previous config
# cat 1333032959.69.kchnode1.cfgrec |vxprint -D - -ht
Disk group: kchdg

DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
ST NAME         STATE        DM_CNT   SPARE_CNT         APPVOL_CNT
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY  DATAVOLS  SRL
RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
CO NAME         CACHEVOL     KSTATE   STATE
VT NAME         RVG          KSTATE   STATE    NVOLUME
V  NAME         RVG/VSET/CO  KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
SC NAME         PLEX         CACHE    DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO
EX NAME         ASSOC        VC                       PERMS    MODE     STATE
SR NAME         KSTATE

dg kchdg      default      default  32000    1141998863.35.acslsn

dm kchdg01    EMC5_2D70     auto     2048     35688576 -

v  kchbackup  -            ENABLED  ACTIVE   14714880 SELECT    -        fsgen
pl kchbackup-01 kchbackup ENABLED ACTIVE   14714880 CONCAT    -        RW
sd kchdg01-01 kchbackup-01 kchdg01 20972576 14714880 0      EMC5_2D70 ENA
pl acslsbackup-02 kchbackup ENABLED ACTIVE   LOGONLY  CONCAT    -        RW
sd kchdg01-02 kchbackup-02 kchdg01 20972048 528   LOG       EMC5_2D70 ENA

v  kchhome    -            ENABLED  ACTIVE   20971520 SELECT    -        fsgen
pl kchhome-01  kchhome    ENABLED  ACTIVE   20971520 CONCAT    -        RW
sd kchdg01-01 kchhome-01  kchdg01 528     20971520 0         EMC5_2D70 ENA
pl kchhome-02  kchhome    ENABLED  ACTIVE   LOGONLY  CONCAT    -        RW
sd kchdg01-02 kchhome-02  kchdg01 0       528      LOG       EMC5_2D70 ENA
Create a file that will make all your disk configuration, this is then applied to the diskgroup to set it up as before
# cat 1333032959.69.kchnode1.cfgrec |vxprint -D - -mvphsr > vxmake.out
Initialize the disk EMC5_2D70 to what the dm device was before which in this occasion was kchdg01
# vxdg init kchdg kchdg01=EMC5_2D70
Now update the diskgroup information with the vxmake.out file you created. This will restore it to its former glory
# vxmake -g kchdg -d vxmake.out
Start the volumes
# vxvol -g kchdg -f startall
When you run a vxprint -htg kchdg you will notice that not all the volumes will be online.
# vxvol -g kchdg      init active kchbackup 
# vxvol -g kchdg      init active kchhome
Now all should look good when you run
# vxprint -htg kchdg
And continue to mount your filesystems. You might have to perform fsck on it first but all should be fine…
# mount /dev/vx/dsk/kchdg/kchhome /export/home

No comments:

Post a Comment