Guys, help me in resolving the issue.
Environment:
NBU Master version 7.1.0.4 :: on VCS 5.0 :: 2 nodes :: RHEL5_x86_ 2.6
NBU Media servers verion :: 6.5.6 :: RHEL5 ::Serving as alternate client for snapshot backups.
Snapshot Method :: Legacy Shadowimage for Hitachi USPV -- Hitachi AMS
Client OS: RHEL5
VXVM 5.0mp3 on master, media and clients
Policy vaildation fails with below error:
There was an error found with snapshot configuration on clients specified in this policy.
Policy: MBS2_store5 -- client: xxxx-xxx-xxx --Snapshot could not be created with ShadowImage method - (Status 223).
Tried running the backup anyway resulting in 156.
Bpplinfo:
Policy Type: FlashBackup (20)
Active: yes
Effective: 01/05/2010 14:44:01
Follow NFS Mounts: no
Cross Mount Points: no
Client Compress: no
Collect TIR info: no
Policy Priority: 0
Ext Security Info: no
File Restore Raw: yes
Client Encrypt: no
Max Jobs/Policy: Unlimited
Mult. Data Stream: yes
Block Level Incremental: no
Perform Snapshot Backup: yes
Snapshot Method: ShadowImage
Snapshot Method Arguments: sync_at_start=0,keep_fi=0,timeout=240,async_resync=0
Perform Offhost Backup: yes
Backup Copy: 0
Use Data Mover: no
Data Mover Type: -1
Use Alternate Client: yes
Alternate Client Name: xxxxxxxxxxxx
Use Virtual Machine: 0
Hyper-V Server: (none)
Enable Instant Recovery: no
Collect BMR Info: no
Max Frag Size: 0 MB
Checkpoint Restart: no
Residence: xxxxxxxxxxxx-hcart-robot-tld-2
Volume Pool: MbsPool
Server Group: *ANY*
Data Classification: -
Residence is storage lifecycle policy: no
Granular Restore: no
Generation: 4
Ignore Client Direct: no
Exchange Source attributes: no
Application Discovery: no
Discovery Lifetime: 0 seconds
Recent changes: A new set of Luns were presented to both client (parent volume) and media servers(clone volume).
Backup process: Involves suspending mirror (Primary to clone), snapshot creation of volume (rdsk), importing VXVMdiskgroup _clone .
Bpfis on client :
7:12:53.278 [11154] <2> parse_ssm_parms: Found prompt Sync I/O delay in milliseconds (UNIX only)
17:12:53.278 [11154] <2> onlfi_vfms_logf: INF - vfm_configure_fi_one: configuring VxFI credentials
17:12:53.285 [11154] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
17:12:53.285 [11154] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bprd
17:12:53.326 [11154] <2> logconnections: BPRD CONNECT FROM 10.4.20.144.47911 TO 10.4.22.59.13724
17:12:53.449 [11154] <2> get_long: (2) premature end of file (byte 1)
17:12:53.449 [11154] <2> bprd_read_text_file: get_string() failed, Input/output error (5), premature end of file encountered
17:12:53.449 [11154] <2> onlfi_vfms_logf: INF - vfm_configure_fi_one: read credentials file failed
17:12:53.455 [11154] <2> onlfi_vfms_logf: INF - TimeFinder FIM $Revision: 1.54.8.4 $
17:12:53.457 [11154] <4> bpfis: INF - FIS_ID=dsmdc-mail-mbs3_1393974591
17:12:53.477 [11154] <4> onlfi_fim_dev_info: INF - Preparing freeze of /dev/vx/rdsk/mbs3store5dg/store5 using snapshot method ShadowImage.
17:12:58.038 [11154] <32> onlfi_fim_dev_info: FTL - VfMS error 11; see following messages:
17:12:58.038 [11154] <32> onlfi_fim_dev_info: FTL - Fatal method error was reported
17:12:58.038 [11154] <32> onlfi_fim_dev_info: FTL - vfm_freeze: method: ShadowImage, type: FIM, function: ShadowImage_freeze
17:12:58.038 [11154] <32> onlfi_fim_dev_info: FTL - VfMS method error 3; see following message:
17:12:58.038 [11154] <32> onlfi_fim_dev_info: FTL - set_devinfo: Could not find targ info for device /dev/sdad
17:12:58.038 [11154] <4> onlfi_thaw: INF - Thawing /dev/vx/rdsk/mbs3store5dg/store5 using snapshot method ShadowImage.
17:12:58.039 [11154] <8> do_thaw: WRN - VfMS error 2; see following messages:
17:12:58.039 [11154] <8> do_thaw: WRN - Feature is not supported
17:12:58.039 [11154] <8> do_thaw: WRN - vfm_fi_ctl: Function is not supported for this method
17:12:58.039 [11154] <4> onlfi_thaw: INF - do_thaw return value: 0
17:12:58.040 [11154] <16> bpfis: FTL - snapshot preparation failed, status 156
17:12:58.040 [11154] <4> onlfi_thaw: INF - Thawing /dev/vx/rdsk/mbs3store5dg/store5 using snapshot method ShadowImage.
17:12:58.040 [11154] <8> onlfi_thaw: WRN - /dev/vx/rdsk/mbs3store5dg/store5 is not frozen
17:12:58.040 [11154] <8> bpfis: WRN - snapshot delete returned status 20
17:12:58.046 [11154] <4> bpfis: INF - EXIT STATUS 156: snapshot error encountered
17:13:00.065 [11431] <2> logparams: bpfis delete -nbu -id dsmdc-mail-mbs3_1393974591 -bpstart_to 300 -bpend_to 300 -clnt dsmdc-mail-mbs3 -S dsmdc-bkp-mst-cls
17:13:00.068 [11431] <4> bpfis: INF - BACKUP START 11431
17:13:00.068 [11431] <32> fis_rebuild_from_db: FTL - cannot open /usr/openv/netbackup/online_util/fi_cntl/bpfis.fim.dsmdc-mail-mbs3_1393974591.0
17:13:00.068 [11431] <4> bpfis: INF - EXIT STATUS 0: the requested operation was successfully completed
BPBRM on Media server
17:12:53.016 [11845] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpcd
17:12:53.056 [11845] <2> logconnections: BPCD CONNECT FROM 10.4.22.55.38142 TO 10.4.20.144.13724
17:12:53.056 [11845] <2> vnet_connect_to_vnetd_extra: vnet_vnetd.c.182: msg: VNETD CONNECT FROM 10.4.22.55.33078 TO 10.4.20.144.13724 fd = 7
17:12:53.064 [11845] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.535: VN_REQUEST_CONNECT_FORWARD_SOCKET: 10 0x0000000a
17:12:53.105 [11845] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.552: ipc_string: /tmp/vnet-11151393974773103770000000000-BgmzUC
17:12:53.187 [11845] <2> bpcr_get_hostname_rqst: Server hostname length = 26
17:12:53.187 [11845] <2> bpbrm start_bpcd_stat: DATA_SOCK from bpcr = 6
17:12:53.187 [11845] <2> bpbrm start_bpcd_stat: NAME_SOCK from bpcr = 7
17:12:53.188 [11845] <2> pfi_start_client: command = /usr/openv/netbackup/bin/bpfis bpfis create -nbu -owner NBU -rhost dsmdc-bkp-med1 -id dsmdc-mail-mbs3_1393974591 -bpstart_to 300 -bpend_to 300 -b
ackup_copy 0 -ru root -pt 20 -clnt dsmdc-mail-mbs3 -rg root -fim ShadowImage:sync_at_start=0,timeout=240 -class MBS3_store5 -sched Full-Snap -fso -S dsmdc-bkp-mst-cls
17:12:53.188 [11845] <2> pfi_start_client: received bpcd success message
17:12:53.253 [11845] <2> pfi_start_client: read start message from dsmdc-mail-mbs3, msg=<INF - BACKUP START 11154>
17:12:53.253 [11845] <2> pfi_start_client: client_pid=11154
17:12:53.253 [11845] <2> write_file_names: buffering file name 'NEW_STREAM' for output
17:12:53.253 [11845] <2> write_file_names: buffering file name '/dev/vx/rdsk/mbs3store5dg/store5' for output
17:12:53.253 [11845] <2> write_file_names: successfully wrote buffer to COMM_SOCK
17:12:53.253 [11845] <2> bpbrm write_filelist: wrote CONTINUE on COMM_SOCK
17:12:53.457 [11845] <2> get_bpfis_msg: bpfis buff: INF - FIS_ID=dsmdc-mail-mbs3_1393974591
17:12:53.480 [11845] <2> get_bpfis_msg: bpfis buff:
17:12:58.039 [11845] <2> get_bpfis_msg: bpfis buff:
17:12:58.040 [11845] <2> get_bpfis_msg: bpfis buff:
17:12:58.040 [11845] <2> get_bpfis_msg: bpfis buff: FTL - snapshot preparation failed, status 156
17:12:58.041 [11845] <32> get_bpfis_msg: from client dsmdc-mail-mbs3: FTL - snapshot preparation failed, status 156
17:12:58.043 [11845] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
17:12:58.043 [11845] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpdbm
17:12:58.085 [11845] <2> logconnections: BPDBM CONNECT FROM 10.4.22.55.34826 TO 10.4.22.59.13724
17:12:58.171 [11845] <2> get_bpfis_msg: bpfis buff: WRN - /dev/vx/rdsk/mbs3store5dg/store5 is not frozen
17:12:58.171 [11845] <8> get_bpfis_msg: from client dsmdc-mail-mbs3: WRN - /dev/vx/rdsk/mbs3store5dg/store5 is not frozen
17:12:58.173 [11845] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
17:12:58.173 [11845] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpdbm
17:12:58.215 [11845] <2> logconnections: BPDBM CONNECT FROM 10.4.22.55.47037 TO 10.4.22.59.13724
17:12:58.300 [11845] <2> bpbrm kill_child_process: start
17:12:58.300 [11845] <2> bpbrm Exit: attempting to send mail to root on dsmdc-mail-mbs3
17:12:58.300 [11845] <2> hosts_equal: Comparing hosts <dsmdc-mail-mbs3> and <dsmdc-mail-mbs3>
17:12:58.300 [11845] <2> hosts_equal: names are the same
17:12:59.849 [11854] <2> decode_fim_attr: decode_fim_attr: fim=ShadowImage:alternate_backup_client=dsmdc-bkp-med1,sync_at_start=0,keep_fi=0,timeout=240,async_resync=0, client version=650000
17:12:59.849 [11854] <2> decode_fim_attr: decode_fim_attr: mapped fim: ShadowImage:sync_at_start=0,timeout=240, rfi_backup 1, keep_fi 0
17:12:59.849 [11854] <2> bpbrm Exit: backup_status: 156
17:12:59.849 [11854] <2> bpbrm start_bpcd_stat: bpbrm.c.19676: bpcd_client: dsmdc-mail-mbs3
17:12:59.849 [11854] <2> bpbrm start_bpcd_stat: bpbrm.c.19679: bpcd_client_hostname: dsmdc-mail-mbs3
17:12:59.849 [11854] <2> hosts_equal: Comparing hosts <dsmdc-mail-mbs3> and <dsmdc-mail-mbs3>
17:12:59.849 [11854] <2> hosts_equal: names are the same
17:12:59.868 [11854] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
17:12:59.868 [11854] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpcd
17:12:59.909 [11854] <2> logconnections: BPCD CONNECT FROM 10.4.22.55.54700 TO 10.4.20.144.13724
17:12:59.909 [11854] <2> vnet_connect_to_vnetd_extra: vnet_vnetd.c.182: msg: VNETD CONNECT FROM 10.4.22.55.41407 TO 10.4.20.144.13724 fd = 6
17:12:59.926 [11854] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.535: VN_REQUEST_CONNECT_FORWARD_SOCKET: 10 0x0000000a
17:12:59.968 [11854] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.552: ipc_string: /tmp/vnet-11429393974779967209000000000-iQ0xmU
17:13:00.051 [11854] <2> bpcr_get_hostname_rqst: Server hostname length = 26
17:13:00.051 [11854] <2> bpbrm start_bpcd_stat: DATA_SOCK from bpcr = 4
17:13:00.051 [11854] <2> bpbrm start_bpcd_stat: NAME_SOCK from bpcr = 6
17:13:00.052 [11854] <2> pfi_start_client: command = /usr/openv/netbackup/bin/bpfis bpfis delete -nbu -id dsmdc-mail-mbs3_1393974591 -bpstart_to 300 -bpend_to 300 -clnt dsmdc-mail-mbs3 -S dsmdc-bkp-
mst-cls
17:13:00.052 [11854] <2> pfi_start_client: received bpcd success message
17:13:00.068 [11854] <2> pfi_start_client: read start message from dsmdc-mail-mbs3, msg=<INF - BACKUP START 11431>
17:13:00.068 [11854] <2> pfi_start_client: client_pid=11431
17:13:00.068 [11854] <2> get_bpfis_msg: bpfis buff: FTL - cannot open /usr/openv/netbackup/online_util/fi_cntl/bpfis.fim.dsmdc-mail-mbs3_1393974591.0
17:13:00.068 [11854] <32> get_bpfis_msg: from client dsmdc-mail-mbs3: FTL - cannot open /usr/openv/netbackup/online_util/fi_cntl/bpfis.fim.dsmdc-mail-mbs3_1393974591.0