In BI system Fan trap and chasm trap are common problem When designing a Universe (semantic Layer) which is used by BI reports to dynamically generate query.
The query information is passed to other system using CWXML (common warehouse XML files) this makes query data inter-operable to desperate systems requiring various different software landscape.
Fan Trap: happen when Object A representing table is mapped to another object B in 1:m cardinality. Which is also mapped to Object C in 1:m cardinality.
This WHEN this happens it lead to double or triple values in summary numerical attributes measures when query generated over them.
A (5) –> B(5,5) = may have value 10 then B –> C may have over calculated summaries c (5,5,5)=15.
A (Emp, salary) now B(dept, emp), C(section, dept).. now you can clearly see 1:m relationship between tables. as explained above the Emp,dept,sum(salary) would have 3 times or more over calculated values when dynamic query is genrated by semantic Layer. this is Called Fan Trap:
This can be solved by using Aliases, aggragate awareness functions @object, @ functions..
Chasm trap: A –> B (1:m) and B –> C (m:1) when relationship between objects is of this type it leads to Chasm trap.
There Chasm trap can also be solved by using aggregate awareness functions @userObjects etc…
Along with this loops are common problem in BI designing ?
Loops are resolved by setting context, which actually defines the path query is going to takes thus resolving ambiguity of join suppose A,B,C are joined but there are in web of tables which join happens dynamically by BI system it can take path like
join path 1: A, B, D1, D2, C
join path 2: A, B, D3, D4, C
Which introduces loops in BI semantic layer is systems like SAP Business Objects, IBM cognos or Micro strategy. Because as report utilizes the semantic objects directly and tool creates query as defined by semantic layer by BI architect it leads to loops.
the two intermediary routes available to tool creates two different types of query and hence two different types of results which leads to distortions in values and duplicate values on reports.
There are resolved by defining path by setting context so suppose we say always when A,B,C join use path 1 by using context.
usually when section is created on a column having multiple values for respective data set.
Solution : By looking why multiple values are coming issue can be resolved.
#compute : Computation Error comes due to Errors in formulae and Also due to objects used in computation not present in data block.
Solution 1: if summary data is needed from detailed block then keep copy of objects and use Fold at break level in Desktop reporting. But this option is not present in Web Intelligence WebI Reports XI but version XI Release 3 onwards introduced again in WebI.
Synchronisation between data provider.
List of Error Codes:
Universe Development issue:
First few days should understand business otherwise cannot create effective reports.
9:00 -10am Meet customer to understands key facts which affect business.
10-12 prepare HLD High level Document containing 10,000 feet view of requirement.
version 1. it may refined later subsequent days.
12-1:30 attend scrum meeting to update status to rest of team. co-ordinate with Team Lead, Architect and project Manager for new activity assignment for new reports.
Usually person handling one domain area of business would be given that domain specific reports as during last report development resource already acquired domain knowledge.
And does not need to learn new domain..otherwise if becoming monotonous and want to move to new area. (like sales domain report for Chip manufactuers may contain demand planning etc…)
1:30-2:00 document the new reports to be worked on today.
2:30-3:30 Look at LLD and HLD of new reports. find sources if they exist otherwise Semantic layer needs to modified.
3:30-4:00 co-ordinate with other resource reports requirement with Architect to modify semantic layer, and other reporting requirements.
4:00-5:00 Develop\code reports, conditional formatting,set scheduling option, verify data set.
5:00-5:30 Look at old defects rectify issues.(if there is separate team for defect handling then devote time on report development).
5:30-6:00 attend defect management call and present defect resolved pending issue with Testing team.
6:00-6:30 document the work done. And status of work assigned.
6:30-7:30 Look at report pending issues. Code or research work around.
7:30-8:00 report optimisation/research.
8:00=8:30 Dinner return back home.
Ofcourse has to look at bigger picture hence need to see what reports other worked on.
Then Also needed to understand ETL design , design rules/transformations used for the project. try to develop frameworks and generic report/code which can be reused.
Look at integration of these reports to ERP (SAP,peopesoft,oracle apps etc ), CMS (joomla, sharepoint), scheduling options, Cloud enablement, Ajax-fying reports web interfaces using third party library or report SDK, integration to web portals, portal creation for reports.
So these task do take time as and when they arrive.
Lets understand how Cognos product works internally
Most of BI product Architecture are almost similar internally.
BI Bus: Enterprise service Bus which surrounds all the services/servers which tool provide.
Typical ESB from Oracle BEA Aqualogic Stack engulfing many Web services looks like:
Now you can compare this popular ESB with BI internal Architecture.
you can read more about ESB at : http://docs.oracle.com/cd/E13171_01/alsb/docs20/concepts/overview.html
Under 4 tier system: A client connects the Web server (which is protected by firewall) using dispatcher. Dispatcher connects to Enterprise Service Bus (ESB) which surrounds all the application server services (Web services). ESB in case of cognos is Cognos BI Bus surrounds Web services Servers (like Report Server, Job server, Content Management server etc ). Mediation Layer Cognos BI Bus interacts with Non Java , C++ code which could not to converted or purposefully kept in C++ for may be more flexibility and speed
In case of SAP Business Objects (BO) ESB was not properly developed so an intermediate layer was created which works for interfacing between multiple servers like Job server, report server, page server etc. BO XI R2 came in pervius version was more in C++ to C++ to java bridge was created in ESB layer. Since Java was preferred language for coarse grain interoperability provided by web services. Each server was developed using web services.
interaction between web server was routed through BI Bus.
In latest version here u find a pipe connecting all components call Business Objects XI 3.1 Enterprise Infrastructure. Earlier version had different names. here you can see its connecting all server like Crystal report server, IFRS input file repository server( storing template of reports), OFRS Output file repository services, Program Job server(storing all programs which can be published on Portal (Infoview) ). This ESB does mediation between different server and achieves interoperability yet control of different components of products. This is in competitor product Cognos is called Cognos BI Bus.
For latest BO uses in memory product SAP HANA more about its competitors follow:
In Micro-strategy there are two important server Intelligent server which creates cubes
More I will cover in later issues:
Oracle BI Architecture:
Implementation OF BI system is not related to these product Architecture :
A typical BI system under implementation haveing componets of ETL, BI, databases, Web server, app server, production server, test/development server looks like:
More details: http://www.ibm.com/developerworks/patterns/bi/product-s390-web.html
Big Data Architecture:
From components perspective of ETL to BI implementation Aspect is little different
Hadoop Architecture layers:
Just like UDDI registry is repository of Web