Archive

Posts Tagged ‘Upgrade’

Upgrading Operations Manager Reporting Server to Windows 2008 R2 and SQL 2008

23/05/2010 Leave a comment

Customer requested to upgrade his Operations Manager reporting services to Windows server 2008 R2 & SQL 2008 SP1 to do so i followed :

http://technet.microsoft.com/en-us/library/cc540368.aspx
http://technet.microsoft.com/en-us/library/cc540369.aspx

The site DBA said that we don’t need to do an SQL upgrade and we can move the DB from SQL 2005 to SQL 2008.

the monitoring environment include:
RMS + SQL for OpsMgr DB (windows server 2008 SP2 with SQL 2005)
MS
GW
SRS + SQL for OpsMgrDW DB (Windows server 2003 SP2 with SQL 2005)

in order to get a fast rollback we got a new server and we will use the same server name (customer home-made scripts and reports uses server name and not a relative name).

The Upgrade Procedure:
1. Install new OS on the new server with temp name.
2. Backup of the OpsMgrDW DB + Jobs + Logins form the old server.
3. Stop all Opsmgr services (RMS,MS,GW) and shutdown the old reporting server.
4. Change the temp server name to the old server name.
5. Install SQL 2008 with SP1 + Configure the reporting services.
6. Verify that http://localhost/reports is showing info.
7. Start all services from step 3 and Install SCOM R2 Reporting Services. use the same Data Warehouse action account, OpsMgr Data reader and Writer.
8. repeat step 3 and Delete the new OpsMgrDW and restore the old DB.
9. Check Security settings (Logins) for OpsMgrDW DB for the users in step 7 and for the SDK service account.
10. Verify that the reporting services data source (connection string) is OK.
11. Start all services from step 3.
12. Verify that RMS and MS can store data in OpsMgrDW DB , check the OperationsManager event log for events 31570,31558,31554,31563,31551,31569,31552,10103 that indicates a connection problem.
13. run a report and check that everything is OK.

Rollback:
In case of rollback we need to shutdown the new server, restart the old server (join to domain) and that’s all.

We followed all steps and it seems that everything is fine, BUT i found that the setup procedure include an agent installation and the service “System Center Management” did not start. did i missed something ?

OK, i followed the move procedure again and i found that i need to “Uninstall the current Data Warehouse component” before i move to the new installation. maybe this part delete the agent from the OpsMgr DB? 

To fix this i deleted the old reporting server agent from the console, uninstalled the agent from the new server restart the server and then pushed a new client. now the agent started and we can monitor the new reporting server.

Categories: OpsMgr Tags: , , ,

Opsmgr 2007 R2 events id 20070 21016 21023 after upgrade

17/12/2009 Leave a comment

Here are some details about the second problem I had after upgrading to R2.

 

Some entities from a specific server are missing on console, when we try to install the agent again events 20070 and 21016 are logged on the agent.

Several agents do not download new management packs and event 21023 logged every 5 minutes in the operations manager event log.

 

 

These are 2 different problems, so I will start with 20070 and 21016, basically this 2 events indicate that the agent cannot connect to RMS, and the reason is that RMS is rejecting the agent. The agent is in the same domain as the RMS so what could be the problem?

 

I found that the server removed from SCOM several days before the upgrade, no one can explain how (deleted or Uninstalled?). To verify this I executed the following SQL query:

 

SELECT * FROM dbo.[BasemanagedEntity] where DisplayName Like ‘%ServerName%’

 

The result verified that the object is deleted from the DB.

 

 

The query give another sight about the problem, this server was managed by the management server and not by the root management server!!!  (search the path column)

 

 

Changing the primary management server (from RMS to MS), restarting the health service on the agent and the problem solved.

 

The second problem was much more difficult to solve. We tried to do the following with no luck.

·         Re-install MS XML Parser again.

·         Flush the agent cache.

·         Re-Install the agent.

 

Nothing helped to solve the problem….so what next?

Running again the previous SQL query showed us that the server object is still in deleted status.

So if we would wait several days until the grooming process will run, then the object will be deleted from the DB. After 1 week the object removed and we start the agent install process again.

 

SAME PROBLEM!!! Event 21023 is still logged every 5 minutes and the folder:

"C:Program FilesSystem Center Operations Manager 2007Health Service Statemanagement pack" is empty. The configuration process isn’t working.

OK, maybe the RMS is preventing the update. In RMS event log I found the following event:

Event Type: Information
Event Source: OpsMgr Connector
Event Category: None
Event ID:       21042
Date:            1/12/2009
Time:            09:01:57
User: N/A
Computer: RMS SERVER
Description: Operations Manager has discarded 1 items in management group MY-MG, which came from $$ROOT$$.  These items have been discarded because no valid route exists at this time.  This can happen when new devices are added to the topology but the complete topology has not been distributed yet.  The discarded items will be regenerated.

Finally I found the solution here:

http://social.technet.microsoft.com/Forums/en/systemcenter/thread/a7a21192-92f2-477f-a268-e1c355df0e98

I executed the following query:

DECLARE @BaseManagedEntityInternalId int
DECLARE @BaseManagedEntityId uniqueidentifier
DECLARE @ViewName sysname
DECLARE @Statement nvarchar(max)
SET @BaseManagedEntityInternalId = 0
WHILE EXISTS (SELECT * FROM BaseManagedEntity WHERE (BaseManagedEntityInternalId > @BaseManagedEntityInternalId))
BEGIN
SELECT TOP 1
@BaseManagedEntityInternalId = bme.BaseManagedEntityInternalId
,@BaseManagedEntityId = bme.BaseManagedEntityId
,@ViewName = met.ManagedTypeViewName
FROM BaseManagedEntity bme
JOIN ManagedType met ON (bme.BaseManagedTypeId = met.ManagedTypeId)
WHERE (bme.BaseManagedEntityInternalId > @BaseManagedEntityInternalId)
AND (bme.IsDeleted = 0)
ORDER BY BaseManagedEntityInternalId
SELECT @Statement = ‘IF NOT EXISTS (SELECT * FROM ‘ + QUOTENAME(@ViewName) + ‘WHERE BaseManagedEntityId = ”’ + CAST(@BaseManagedEntityId AS varchar(50)) + ”’)PRINT ”’ + CAST(@BaseManagedEntityId AS varchar(50)) + ‘ ‘ + @ViewName + ””
EXECUTE(@Statement)
END

And the result was:

image

Then executed the following :

select fullname
from basemanagedentity
where basemanagedentityid = ‘<GUID>’

And there it was: Microsoft.Windows.Computer:ServerName.XXX.corp

The server was not listed under managed agents and we couldn’t find the object anywhere in the console.

Before executing the above query please ensure that you have a backup of the database. Also note that you need to run this query only incase you do not see the object in the console.

So now we will need to delete the object from the DB.

update basemanagedentity
set isdeleted = 1
where basemanagedentityid = ‘<GUID>’

After executing this query, run the following stored procedure to make sure the DB is fixed.

exec p_Detectandfixinstancespaceinconsistencies

Once you done restart all the three OpsMgr services.

And that’s all, after a couple of minutes all agents will start download all new configuration.

Categories: OpsMgr Tags: ,

my notes regarding the upgrade of SCOM SP1 to R2

09/12/2009 Leave a comment

There are several good blog articles containing all the procedures required to perform this upgrade, one of the best ,in my opinion is Kevin Holman Blog,

http://blogs.technet.com/kevinholman/archive/2009/05/23/my-experience-upgrading-to-opsmgr-r2-rtm.aspx

After several successful upgrades I bumped into some problems, I will try to describe them and the solutions helped me.

The environment I tried to upgrade was:

Server A: RMS + Reporting Services + Web Console

Server B: MS + ACS

Server  C: Cluster SQL Server with both operations manager DB and DW

Agents: Domain and DMZ agents

I started with the prerequisite checker, everything looks good (I need to install the AJAX extension for the web console), I followed all the pre-install procedures according to http://technet.microsoft.com/en-us/library/dd789074.aspx .

Now I’m ready to start the upgrade, so let’s start.

OMSetup.exe is lunched on server A.

The setup procedure starts upgrading the server and after a couple of minutes an error popps up. "Error 25154. Failed to execute SQL script. Error Code -2147217900 (The affinity mask specified conflicts with the IO affinity mask specified. Use the override option to force this configuration.).

clip_image001

Oops….OK, the rollback started automatically. After several searches I found this http://www.dario.co.il/blog/?p=535.

it’s good to know that I’m not alone.

I followed Amit’s suggestion with no luck. My next step was to explore the mom.log file, the log file indicated that the problem is with "build_mom_db_upgrade.sql", the file is in the System Center operations Manager Installation folder under program files. The SQL script contains the following lines:

sp_configure @configname=clr_enabled, @configvalue=1

GO

RECONFIGURE

GO

Back to Amit’s post, the suggestion is to add the WITH OVERRIDE after the reconfigure statement.

I decided to rename "build_mom_db_upgrade.sql" to "build_mom_db_upgrade.sql.old"

And to place a copy in "C:Program FilesSystem Center Operations Manager 2007" which contains the fix .

sp_configure @configname=clr_enabled, @configvalue=1

GO

RECONFIGURE WITH OVERRIDE

GO

run SETUPOM.exe again. , now the "Create database object" step finished successfully.

clip_image002

After a couple of minutes the upgrade procedure failed with no errors and the auto rollback process started again.

Open mom.log again to search the problem…..now my problem is with the Web Console.  I start the uninstall procedure of the web console, and check to see if there are left over’s, the web console folder didn’t clean up, I deleted the files and executed SETUPOM.exe for the third time.
After 30 minutes the setup procedure completed successfully!!!

The next steps were: the MS upgrade, reporting upgrade, ACS upgrade and a new installation of Web Console.

It’s a good time to take a coffee break.

Do not forget to follow the post installation procedures http://technet.microsoft.com/en-us/library/dd789073.aspx .

Unfortunately, a few days after the upgrade we found one more annoying problem that i will describe in my next post.

Categories: OpsMgr Tags: , ,