Restarting all OBIEE 11G domain services in linux and common issues

Hello

Although there are a lot of articles out there that talks about how to restart OBIEE 11G, I find them tend to be less clear to beginners. Trust me, if you have just started using 11G, you will find out that there are some common issues that you will run into when you are restarting your OBIEE system. So today, I want to gets these things straighten out.

Let's start and assume that your OBIEE is already up and running, and you have made some configuration related changes or you have to go through some other system maintenance, which requires you to shut down OBIEE and start it again. Now, I am not going to explain why to do or how OBIEE architecture works,  I find that unhelpful to a lot of people, so let's start with steps.

To Stop OBIEE 11G domain components:
(Duh! restarting means stop, then start)

1. Stop Opmn

in your Linux terminal window, run the following command line:
Go to the following path:
$your OBIEE main folder/instances/instance1/bin/

Then type
./opmnctl stopall

2. Stop Managed Server

Go to the following path:

$Your OBIEE folder/user_projects/domains/bifoundation_domain/bin/

Then type
./stopManagedWebLogic.sh bi_server1

Enter the credentials for the WebLogic administrator (in my case it is weblogic/weblogic123) when prompted.

Let it run and it will be done eventually. (Duh!)

3. Stop Node Manager

Since there is no script that stops the node manager, you will have to kill it, so type the following in your terminal:

ps -ef |grep Node|grep nodemanager |cut -c10-15

It will show the Node Manager process, there is an PID that comes with it the result, note it down and then type:

kill

After having done so, you can re-execute : ps -ef |grep Node|grep nodemanager |cut -c10-15

This time, it shouldn't return any results, that means node manager has been stopped.

4. Stop WebLogic Domain

In the same Terminal go to the following path:

$Your OBIEE folder/user_projects/domains/bifoundation_domain/bin/

Then type:
./stopWebLogic.sh

Let it execute and eventually it will complete the stoppage.

Now starting all OBIEE domain components:
You will eventually realize that the sequence of starting each components is pretty much in reverse of the stopping sequence

A. Start WebLogic Domain

In the following folder:

$Your OBIEE folder/user_projects/domains/bifoundation_domain/bin/

Type:
./startWebLogic.sh

If you want to have it run in the background, which means the component remains started after you close the terminal session (you pretty much have to do it this way), type this:

./startWebLogic.sh &

Now if you are restarting, it is likely you will run into some issues at this step, so scroll down to where I provide the error handling

B. Start Node Manager

Go to the following path:

$Your OBIEE folder/wlserver_10.3/server/bin/

Type:
./startNodeManager.sh &
(for running in the background)

Let it run and keep the windows opened for a while

C. Start Managed Server

Go to the following path:

$Your OBIEE folder/user_projects/domains/bifoundation_domain/bin

Type:
./startManagedWebLogic.sh bi_server1 &
(for running in the background)

Just like step A, it is likely you will run into some common issues, if you do, please scroll to the error handling section of this post to see if the errors I mention here applies to you or not.

D. Start Oracle BI System Components (OPMN)

go to the following path:

$Your OBIEE Folder/instances/instance1/bin

Type:
./opmnctl startall &
(again the '&' is for running it in the background)



Error Handling:

Error 1:
There are 1 nested errors:

Weblogic.manaement.managementException: Unable to obtain lock on /path......../AdminServer.Lok. Server may already be running

It is pretty common that you will run into this error, it looks like this in my system:

When you see this error, just go to path which I highlighted and you will find the AdminServer.lok file or bi_server1.lok file and delete it:

Basically, AdminServer.Lok and BI_Server1.lok can be found here:
YourOBIEEFolder/user_projects/domains/bifoundation_domain/servers/AdminServer/tmp
YourOBIEEFolder/user_projects/domains/bifoundation_domain/servers/bi_server1/tmp

If AdminServer.lok exists, it will cause errors when you start Weblogic (In Step A)
If Bi_server1.lok exists, it will cause errors when you start Manage Server (In Step C)

Simply delete these files and execute the run script again, the errors will go away.

Error 2:

When you are running these starting scripts in the background as indicated in Step A and C, the system prompts you to enter the username, when you do so, you end up getting 'weblogic: comment not found' and the system doesn't get started. When you run the script without '&', everything worked normally. The behavior is seen as following in my system:

When you see a behavior like this, it has something to do with boot.properties file not found in the following directory:

/user_projects/domains/bifoundation_domains/servers/AdminServer/security --- This will cause above error when you run step A 

/user_projects/domains/bifoundation_domains/servers/bi_server1/security/ -- This will cause above error when you run step C

Now you can either create your own boot.properties file in both directory, or you can find the existing boot.properties file in the following directory:

/user_projects/domains/bifoundation_domains/servers/bi_server1/data/nodemanager as shown:

If you open this file, you will see that the weblogic admin user password has been encrypted. If you would, just copy the boot.Properties file from '/user_projects/domains/bifoundation_domains/servers/bi_server1/data/nodemanager ' to the following 2 directories:

/user_projects/domains/bifoundation_domains/servers/AdminServer/security --- This will take care of the above error when you run step A 

/user_projects/domains/bifoundation_domains/servers/bi_server1/security/ -- This will take care of the above error when you run step C

In Summary:

When you get these 2 errors handled, your restarting of OBIEE will be smooth 90% of the time.

Thanks

Until next time