OBIEE Project Case Study 1 --- Telecom Analytics

So for the past few weeks I have been very busy working with this new project, which is challenging and very unique. It makes me look back on my career and see the special things in every project I have done. It makes me realize that every project I did can be looked at as experience gained, because just like you will never find 2 persons with the exact same face, you will never find 2 projects with the exact same setups, environments and journeys. Therefore, I have decided to talk about some of the projects I have been or currently been doing as a way to  honor the experience, I hope it serves as a learning reference as well as understanding the journeys I take to overcome my own issues and challenges.

This project is for a telecom company. What makes this project uniquely challenging is the kind of report that I have to deliver. In order to understand the purpose of this project, it's better to throw away our normal understanding of BI Projects. Here, we are not creating reports for the clients internal operation, in other words, whatever we can think of that a business needs such as Financial reporting, HR, Supply Chain, Marketing etc, can be thrown out the windows. Basically, this client doesn't use OBIEE period.

Strange isn't it? Let me tell you what we do in this project and why I call it Telecom Analytics.

In order to understand the project at a high level, we should first understand what the client is trying to achieve overall. I know this is the basic consulting 101, but you will be surprised how many consultants I worked with did not have to the awareness to think at a higher level. Using the technical skills to create reports based on the requirement is one thing, but understanding the purpose of this project is something else worth spending time on. It will make everything you do much more proactive and efficient.

Enough said here. So in this telecom business, you have to first understand how things work. Why when a person dials a number, the other person's phone rings? I won't get into the detail of how signals are traveling from radio base stations through routers to another stations. Assuming it is done by a very complex router networks, therefore when customers complaints about the quality of their services, how would you even begin the solve the problem? How would you know at which router the signals gets dropped or at which traveling route the traffic is overloaded? So this is where OBIEE comes in.

As we all know, for any data warehouse projects, it is about extracting data from sources and loading them into target DB which provides BI reports. So in this project, the source we are looking at are those routes and network devices. In other words, instead of dealing with transactions, sales, orders, GL accounts, suppliers in your normal BI project, here we are dealing with Pseudo-wires, nodes, Thresholds, Utilization %, Area etc. Are these things making business sense? It won't. Do they make sense to network engineers? They have to.

Of course the raw data coming from these routers (realistically speaking, thousands of routers) won't make any sense for reporting. It probably looks like what you see in the Matrix movies. These data have to be converted into formats that OBIEE can work with, so how is this done? This is where the concept of big data, cloud computing comes in. For those who don't know what is Big Data, feel free to look it up online. Basically, this is the method that google has been using for years to make their search engine the best in the world.

The specific tool we are using to extract data from the routers is Hadoop, which is an open-source engine based on java programs. It goes after all these routers on hourly basics or daily basics and convert the data into CSV files. From there, we have decided to use OWB or Oracle Warehouse Builder to take the CSV files and load them into another DB, which is where OBIEE will be using to build the reports. The reason it has been decided this way is simply because of the price. Now, how much computing should be done at Hadoop and how much should be done at OWB or even OBIEE will be determined completely based on the requirement and our approach.

So having described the high level flow of this project, now on to the challenges.

Here are some of the challenges this project brings:

1. Lack subject matter expert for gathering business requirements. Since network engineers have no clue about how data warehouse works and most of the project folks have very limited understanding of telecom technology. Understanding the requirement is a constant challenge.

2. We are inventing the wheel here. In other words, there is no prebuild models to follow, there is no base practices to refer to and there is no study cases to look at for building telecom reports.

3.There is no way to build a clean data model. In the ideal world, we desire a clean relationship between facts and dimensions. However here, because we are dealing with configuration data from routers and network devices, it is impossible to form star schema.

4. The way dimensions are related to each other is never straightforward. There is rarely dimensional 1 to N relationship. For example, 1 Pseudo-wire can connect to multiple nodes yet 1 node can also have multiple Pseudo-wires going in and out. It can be bi-directional or non-directional. Having such tricky relationship among these devices greatly reduces our choices when it comes to building warehouse objects. De-normalization has to be done very carefully to make sure the data are aggregated correctly

5. Performance has to be very quick. Think about there are millions of signals traveling through mobile networks every second, what kind of data volume can you imagine we are dealing with.

6. Oracle support is not available for this project. Simple, because this project is not for the internal business, there is no production environment here. Therefore, good luck charlie.

7. Deadline and requirements are non-negotiable

Having said all these, you will start wondering with a thousand question marks. Most likely you will be wondering who is coming to look at these reports and how are we releasing it if there is no production environment.

Fair enough. Just think of what we are doing like creating software. Think of it like how blizzard entertainment developed "warcraft". They will only have a QA environment, the production environment will be the home computer of us and we are the users who are complaining about bugs. It is the same thing here at our project.

The reports will be created with the intention of analyzing the router network for the mobile companies, therefore our end product will have to be made into DVDs shipped along with other hardware products to big network carriers who we are paying our phone bills to.

Hopefully, you will get some idea about how this project works at the high level.

In order to succeed in this project, having good consulting skill is very important as requirement gathering is an never ending effort. It is also required to be able to think on your feet when it comes to design by throwing away the models of BI Apps. Finally, knowing telecom is also going to be very helpful. Yet, even if you don't know, this project will make you a pretty good expert on that.

The skills, knowledge and problem solving ability developed here is completely transferable to any other industry. It is not about what you know, it is about how you can pick up new things.

In the near future, I will go through some of the things I do to make this project work. It will cover functional requirement gathering to technical implementation of OBIEE as well as general consulting scenarios.

Stay tuned!