Thursday, December 24, 2009

Problem: DST: Workaround to EM Agent Restart Problem Due to a Daylight Savings Change


Applies to: 
Enterprise Manager Grid Control - Version: 10.1.0.3.0 to 10.2.0.3.0
This problem can occur on any platform.
Symptoms
Oracle Management Agent does not start.



Starting agent ….. failed.
The agentTZRegion value in /opt/oracle/GCAGENT/sysman/config/emd.properties is not in agreement
with what agent thinks it should be. Please verify your environment to make sure that TZ setting has not changed since the
last start of the agent. If you modified the timezone setting in the environment, please stop the agent and exectute ‘emctl
resetTZ agent’ and also execute the script mgmt_target.set_agent_tzrgn() to get the value propagated to repository. Consult the log files in: /opt/oracle/GCAGENT/sysman/log
The emagent.nohup contains the following errors:
If you modified the timezone setting in the environment, please stop the agent and exectute ‘emctl resetTZ agent’ and also execute the script ‘mgmt_target.set_agent_tzrgn’ to get the value propagated to repository —–
This note has been produced to provide a short term workaround in response to any changes in Daylight Savings which may affect the ability to start the EM Agent. It can generically be applied to any Daylight Savings change throughout the world. The workaround may also apply to the DB Console as it also has an EM Agent.
The optimal scenario is that there is enough time for both the OS vendor and Oracle to get patches created in time for the change but this is not always possible.
Changes
The agent runs on a server with a timezone that has recently changed the start or stop time of daylight saving time, DST. Either the OS or agent have not been patched resulting in the time in the agent not matching the OS.
Cause
When there is a change in the Daylight Savings dates an OS patch and an Oracle patch will need to be created and applied. For the EM Agent the OS patch and the Oracle patch will both need to be applied before the time zone changes take effect. Otherwise there will eventually be an offset mismatch for the changing timezone region and the Agent once stopped will not start.
There are 2 supported options for timezone changes.
1. Do nothing. ie. Do not apply the OS patch or the Oracle patch. In this case the system time will be wrong but the EM Agent software will continue to work. This is the preferred option when not all the patches are ready.
2. Apply the OS patch and the Oracle patch before the time zone changes take effect. If the time zone changes are already in effect, then apply the OS patch and the Oracle patch at the same time. Now the offsets will match and the Agent will start in the affected timezone.
If both the agent and OS have not been updated with the DST changes there is an offset mismatch and so the EM Agent will not start. The following short term workaround may provide a solution till patching can be arranged.
Solution
The steps described in this note are intended as a short term workaround and not a permanent solution to the mismatch between the OS time and the time calculated by the agent. Oracle strongly recommends that the agent and the OS have the appropriate DST updates. Refer to appropriate patching note in the reference at the end of this note. If the workaround does not resolve the agent start up problem then the agent Oracle Home must be patched.
If the OS patch has been applied but the Oracle timezone.dat patch has not been applied then you can use the following workaround.
From the $AGENT_HOME/sysman/admin/nsupportedtzs.lst choose the Etc/GMT** setting that matches your timezone. In the case of US/Pacific the value is Etc/GMT+8. Then to adjust for the daylight savings change we need to modify this value in indicate the change. Because we are starting DST earlier. ie. we are closer to GMT time so we subtract 1, this then becomes Etc/GMT+7. If we are on the other side of the world for example Etc/GMT-8 would become Etc/GMT-9.
And so using US/Pacific as an example the workaround goes….
export TZ=Etc/GMT+7
emctl resetTZ agent

Log into the OMS repository as user sysman the above command ‘emctl resetTZ agent’ tells you exactly what needs to be executed in the repository.
exec mgmt_target.set_agent_tzrgn(’‘, ‘Etc/GMT+7′);
commit;
emctl start agent – (or emctl start dbconsole)
Once the missing patch has been applied you can use the above workaround to reset the timezone back to the original region.

No comments: