Detecting the Madoff Effect: Methodology for Fraud in Hedge Funds

As a result of the recent Bernard Madoff fraud scheme, pension funds and corporate finance managers have been put on the defensive, wondering how to detect this type of “under the radar” deceptive scam. The depth of the fraud in the case of Bernard Madoff and his ability to engage in a $50 billion undetected scheme employing “serial correlation” demonstrated the vulnerability of financial institutions to trusted individuals operating inside the security model. Bernard L. Madoff Investment Securities LLC (“Madoff”) engaged in a ponzi or pyramid fraudulent scheme in which investors were paid interests not from actual investments but from the funds deposited by other investors. But being on the defensive is not the ideal solution, as it forces financial institutions into a reactive mode, always trying to catch-up with the perpetrators, who somehow remain one step ahead. We would like to suggest a more proactive approach and corresponding methodology for detecting fraud in hedge funds.

Madoff was able to hide his scheme using a “serial correlation” reporting scheme. A serial correlation is a term used by MIT professor and hedge fund theorist Andrew Lo to describe the degree to which each month's returns in a fund mirror the results of the month before. Dr. Lo’s theory is that is a hedge fund has a nice smooth line in its rate of return every month. Upon close examination, any variation to the “smoothness” of the line constitutes a red flag, which should be look upon more carefully.

In the last year corporate finance departments, financial institutions, as well as public and private pension fund portfolios have already lost about 33% of their values due to overleveraged investment banks, the housing and credit crises. An effective, (no more than 1 to 3 weeks) and cost efficient proactive methodology to detect the Madoff effect in the hedge funds would be to apply the following methodology in the specified order to the relevant data available to you:

• Link Analysis – Use link analysis to determine in the network Madoff is categorized by, and create a subset of that network of hedge funds.
• Predictive Modeling – Use predictive modeling to create a score of all the hedge funds in your subset. Use Madoff’s variables as your training data.
• Clustering Analysis – Perform a cluster analysis which includes among other variables the predictive score. Since the predictive score is a multidimensional variable when used with one-dimensional or “flat” variables you will obtain a binocular vision (or binocular summation) of your analysis and increase by 1.4 times the ability to detect the serial correlation. See, Improving Search Engine Optimization by Incorporating Predictive Analytics at http://atomai.blogspot.com/2008/12/improving-search-engine-optimization-by.html
  
  For verification of the analysis you could use the following factors:
• The reputation of the independent auditors of the hedge fund identified through this methodology;
• Control Chart using standard deviation of the yearly returns over a 3-5 year period (exclude the current year);
• The ratio of total number of employees to the total amount of investments.
  
  Contact Alberto Roldan at atomanalytics@gmail.com or Sean Suskind at seansuskind@gmail.com
  

Improving Search Engine Optimization by Incorporating Predictive Analytics

As more companies increase the size of their databases search engine optimization (SEO) techniques can be adapted to data mining of commercial databases. In SEO link analysis is a measure of the quality and relevance of the set of links pointing to a given site. This is measured is achieved through an algorithm that maps the hyperlinks in a series of networks. The measurement creates a ranking of the strength of the inbound links to a particular network. The objective of link analysis is to detect patterns or trends that would make the search engine to bring to the top the most relevant web sites in any search.

Link analysis contains multiple variables that are analyzed. Google’s claims over 200 variables are analyzed in its link analysis for its ranking algorithm. Although I do not know which variables it uses, I surmise that they are the keystone of Google’s success. The core of any analysis is its variables. Let me suggest the utilization of predictive modeling as an additional variable that will improve SEO.

Using predictive modeling as another variable in link analysis could potentially increase SEO by 1.4 times by giving depth perception to the link analysis. In ophthalmology medicine it an established fact that binocular vision gives depth perception, and that depth perception (or binocular vision) increases the range of view 1.4 times greater than monocular vision. In other words, you can see better with two eyes than with one eye. The equivalent of depth perception in analytics is the addition of a predictive modeling (or scoring) variable to any pattern detection analysis.

A predictive modeling variable will improve the SEO because:

1. It gives an independent variable that acts as a spare variable in case that another variable is not working. In other words, you can use a predictive modeling variable in a correlation analysis as your independent variable against the other numerical variables in your link analysis.
2. A predictive modeling variable will widen the field of view of your networks from 160 degrees to 200 degrees.
3. Binocular summation (seeing with two eyes) will enhance faint but important networks and links within your data.

Among SEO scientists, statisticians, and business analysts it would increase stereopsis, or the keen sense that they have depth perception. In other words, it would give them another tool to do their work more efficiently.


Most of the variables used in link analysis are flat, or with one-dimension. A predictive modeling variable is multidimensional and hence a “depth variable”. The addition of a “depth variable” to any analysis statistically can be expressed as detecting the networks using two sensors instead of one. If each flat variable alone had a 0.6 probability of detecting a network, that probability has been calculated to be:

Pb = Pr + Pl - (Pr x Pl) = 0.6 + 0.6 - (0.6 x 0.6 ) = 0.84 (1)

The improvement from 0.6 to 0.84 represents a 1.4 fold improvement. This improvement can be achieved in any analytics technique by adding a multidimensional variable to a one- dimensional variable during analysis.

Contact: Alberto Roldan, CEO of R&R Analytics at atomanalytics@gmail.com

Depression Economics: America’s Economic Crisis

I have written many times in the last 12 months about this issue in the Business Analytics website at http://atomai.blogspot.com/. This interview from Paul Krugman in Newsweek gives us an insight into how deep is the ocean in the financial and credit crisis. If you look in page 2 of the interview Krugman talks about a $10 trillion (yes trillion with a “t”) shadow banking system that just went up on smoke. Hence, he argues that a $700 billion capital injection by the Federal Reserve and Treasury Department will be insufficient to make up for those $10 trillion. I agree with Krugman, but we need to consider that if we take into consideration the capital injection that have an effect in America’s economic crisis by other countries in Western Europe, Russia, China, and Japan. We are talking more than $3 trillion injected into the world economy when we take those capital injections into consideration. This still leaves us with an $7 trillion issue. The next issue is the valuation of those depleted assets, because it is the difference between the $7 trillion and the value of the depleted assets that are going to determine when we are going to touch bottom. I estimate that the minimum value of those assets will be about $3.6 trillion, and that is going to leave us with about $3.4 trillion that the international capital markets are going to deal with a combination of global stock market devaluations and injection of additional capital by central banks. The jobs program and other programs by president-elect Obama will have an effect on the economic crisis but it is too early to determine what that effect will be. If the administration of President-elect Obama injects into the economy $1 trillion through different programs, including a jobs program, we will still have $2.4 trillion to deal with. The bottom line is that companies need to prepare to cut 12% to 24% in additional expenses above what they have already trimmed. The role of cost-efficient business analytics decision-support systems at the operational level is going to become a cornerstone in the transformation of companies in how to increase the margin of profit with less resources.

The Krugman interview is at http://www.newsweek.com/id/171871/page/1

Successful Business Intelligence Projects: The Role of Managers and Leaders

Most BI projects fail because the leadership for those projects is wrong. An article on how Some Brains Wire for Change[i] helps explain the physiological reasons how some individuals can adapt easier to change than others. This article makes clear that people’s brains are different and that different does not mean “bad”. In today’s recession it is important for organizations to understand the role of a manager vis-à-vis the need for a leader in the area of analytics.

Every organization needs both managers and leaders in analytics. Managers are those individuals who supervise individuals who conduct analytics within an organization. Leaders are those individuals who guide or have commanding authority in the area of analytics within an organization.

I have found that organizations tend to have good managers in the area of analytics but lack leaders. Managers are efficient at maintaining the status quo and are adverse to risks. Leaders are risk takers and innovators, but not necessarily proficient at managing or maintenance of a department.

An organization that is satisfied with how its analytical capabilities are producing a lift in their revenues and profits, should be looking to improve how to efficiently manage those capabilities. On the other hand, an organization that is looking to improve revenues, costs, or profitability by using its analytical capabilities needs leadership in the area of analytics. A good manager realizes when he needs a leader, and a good leader acknowledges the need for a manager.

Managers and leaders of analytics have different roles, and although they are not mutually exclusive it is the role of executive management to define the priorities in the area of analytics at any given time. Sometimes organizations make the mistake of trying to make managers leaders or vice-versa. The results are that the analytics capabilities within an organization never bloom to its full potential in contributing to increased profits. In my experience managers contribute about 80% to 90% of the success of a business intelligence project, and leaders contribute 10% to 20% of the success of the project. Therefore, a successful BI project needs both managers and leaders.

Contact Alberto Roldan at R&R Analytics at atomanalytics@gmail.com
[i] http://www.livescience.com/health/081201-brain-personality.html

The Commoditization of Analytics

This is an article that argues that since analytics are based in long-standing math concepts they should be treated as a commodity. Hence, the difference in analytics is not the software vendor but:

1. The domain knowledge of the company doing the implementation of any decision-support system.
2. The total cost of the implementation should consider Software-as-a-Service as an alternative to reduce costs.
3. Visualization of analytics is an important component when dealing with large datasets.

This particular point of view is targeting the CFO and CIO from companies that are under pressure to deliver cost-effective solutions that have a direct impact on profitability during recessionary times.

Healthcare Payer Analytics

The link is to an article of the healthcare payer/insurance analytics available in the market. There are three categories of analytic services:

1. Outlier Detection - This area includes claims auditing, fraud and abuse, and Medicare Part D programs.
2. Predictive Modeling - This area includes HCC reconciliation and Outcomes Research.
3. Data Mining - This area includes claims processing, Medicare Advantage, and Care Management.

The architecture of the analytics engine allows for individual portals for payer staff, providers, members, and employers. There are over 300 reports. This architecture is compatible with Oracle (OBIEE), Microsoft (SQL 2005 & 2008), SAS Enterprise Data Miner, Business Objects, Hyperion, Cognos, and SPSS Clementine.

See: http://docs.google.com/Doc?id=dd87qswp_12xvh3mgc6&hl=en

If you have any questions contact Alberto at atomanalytics@gmail.com

Analytics In A Global Recession: Fixed Price Operational Dashboard

As the world economy moves towards a potential global recession companies are asking about how could they could optimized their investments in business intelligence, BPM, data mining, and analytics. The answer is simple: go to a fixed price operational dashboard model that requires vendors and service providers to:

1. Deliver within 90 days a proof of concept (POC) operational decision support system;

2. Operational means that information workers within the company can use the decision support system to meet a company's financial goals;

3. The POC should be measured in one of three simple key performance indicators: increase revenues, decrease costs, or increase the percentage of profits; and

4. Total price for POC should not be more than $25,000

In a recession companies must manage with a laser focus their revenues, expenditures, and investments. Strategic dashboards are nice, but operational dashboards reach down into the income producing worker. The goal of achieving a cost efficient and revenue enhancing operational dashboard is within all companies regardless of the technology that they use (i.e., Oracle/Hyperion, IBM/Cognos, Microsoft SQL 2005 and 2008, SAS, or Business Objects/SAP). Even small companies can use Software-as-a-Service analytics like Zementis to build operational dashboards that are cost efficient.

Let an analytics operational dashboard be the compass that guides your company in this stormy economic times. Contact me at atomanalytics@gmail.com If you would like to work with a technology provider who you can trust to work together in providing a fix cost operational dashboard in the marketing, retail, banking, insurance, manufacturing, CRM, or healthcare industries.

Alberto Roldan

See, Business Analytics at: http://www.linkedin.com/redirect?url=http%3A%2F%2Fatomai%2Eblogspot%2Ecom%2F&urlhash=AwT7&_t=disc_detail_link

IT and Enterprise Architecture Metrics: Managing in Tough Economic Times

How can IT demonstrate its value to the CFO in this times of economic downturn? How can IT make the case for funding for new projects? Lately I have seen this type of questions asked more often. The answer is that IT must have metrics that are directly connected to corporate revenues, expenditures, and profitability. In order to develop this metrics IT must partner with the business units. For example, if a business department wants a more efficeint application so that workers spend less time manually completing reports IT must ask to that department: how is the improvement in the application going to affect revenues, expenditures, or profitability. IT and business departments should be sitting down and making project-specific economic forecasts that are measurable in terms of the economic health of a company. If you need assistance in making these forecasts and its accompanying metrics let me know and I will assist you in this area.

The Democratization of Analytics - Microsoft Project Gemini

Last week I attended the Microsoft BI Conference. I learned about project Gemini. This project will allow analytics power users in companies to use Excel to do powerful analytics, while simultaneously allowing collaboration among all stakeholders using PerformancePoint. It allows Excel to load over 100 million rows (and about 6 columns) in just a few seconds and then create interactive pivot tables. They are still working on calculations but the demonstration was powerful. If you see Ted Kummert, Bruno Aziza, Kamal Hathi, Donald Farmer, or Amir Netz in a meeting make sure that you let them know that they are doing a great job.

Microsoft is doing a fantastic job at integrating Analysis Services, Integration Services, Reporting Services (the visualizations are robust in SQL 2008 RS), Excel Services, and PerformancePoint. This family of products are very different than SQL Server 2000. If you have followed this products for over 5 years you realize that the 2005 and 2008 products are totally different than the 2000.

A brief note about PerformancePoint and Reporting Services. For the first time I saw powerful visualizations that allow reports to be actionable. This is impressive and tells a lot about the folks working to develop these products.

Sometime ago I wrote: "If anticipation is part of journey, I can hardly wait for the sounds and tastes that will come in the next 12 months when business, technology, and science merge to produce masterpieces to help solve complex business problems." See, http://atomai.blogspot.com/2008/06/genetic-algorithm-grid-computing-and.html. Microsoft is providing the right tools for those who do analytics, to do their job in the most efficient manner.

In the Disney-PIXAR movie Ratatouille a Chef says that "anyone can cook". Microsoft is saying anyone can do analytics, and I agree. See, http://atomai.blogspot.com/2008/05/business-analytics-unleashing-power.html

Financial Markets & Credit Crisis: A Potential Solution

The failing of financial institutions are having a global impact on the financial and credit markets. The issues with Fannie Mae, Frddie Mac, Merryl Lynch, Lehman, AIG, and Countrywide were fairly obvious. Since March I have been predicting these type of failures by using the ratio of mortage based securities in a companies portfolio to total capital under management (mortage based securties/total capital under management). Also, I suggested that you use the Carlysle Fund ratio as your upper benchmark (or alarm). See, http://atomai.blogspot.com/2008/03/business-analytics-and-financial.html. If investors are being caught in these financial institutions failures and downgrades is because they are failling to do basic due diligence.

Also, I have suggested potential solutions that may have sounded fairly radical 6 months ago, but they are sounding more reasonable in today's financial markets. Let's allow more direct foreign investment in our capital markets. Second, target housing relief, for homeowners and financial institutions, to primary homes with a market value of up to $250,000. Lastly, let the market forces dictate what happens to properties valued at over $250,000. See, http://atomai.blogspot.com/2008/03/financial-markets-crisis-potential.html.

I have also suggested new metrics, using advanced analytics, as well as financial models that can be used to gauge risk and create a robust decision support system. See, http://atomai.blogspot.com/2008/05/bernanke-banks-must-get-better-at.html

These problems are not easy, but that is part of life...what we need is the will to implement these changes.

Microsoft BI Conference

Business Analytics Group Members,
I will be present at the Microsoft BI Conference in Oct 6-8 in Seattle, Washington, representing HP at our booth answering questions about BI design using SSAS in the the financial, marketing, internet, healthcare, CPG, life sciences, and retail industries. Also, I will be covering how to use advanced analytics for: improving view of the data by 40%; fraud detection; and improving server performance in large databases. If you are attending the conference let me know so we can meet.

I have met some of you, but most of the members of the group I have never meet in person. This is a great opportunity to meet, exchange opinions, and probably get some souvenir from HP. In the best case scenario you can claim that you saved thousand of dollars by consulting with me for free. In the worst case scenario you can claim that you "stumped the chump" (i.e., me) with your questions and knowledge.
Thanks,
Alberto
alberto.roldan@hp.com

Data mining detects signs of Lou Gehrig's disease in gene carriers long before symptoms appear

This data mining methodology of "pattern arrays" in life sciences could be adapted for business analytics. I would recommend this methodology for large retailers to examine consumer behavior in brick and mortar onsite shopping. Also, it could be used in online advertisement.

Evaluating Online Advertising: Media Measurement Framework

There is an article by Stephen Pretorius published in Adotas that correctly identifies the Media Measurement Framework for evaluating online advertisement. He correctly identifies five areas of concern:

1. Track everything – The higher the number of variables that greater the probability that you can build a robust decision support system.


2. Integrate the data – You need a data warehouse for your variables, metrics, and decision support system that is separate from your operational data warehouse.


3. Compare apples with apples – Meaning comparison requires comparing the same type of metrics.


4. De-duplicate conversions – “You need a conditional filtering system (like DoubleClick’s Floodlight tag tool) to ensure that you only ping performance media tracking tags when they were the ones sending you the customer.”


5. Make your metrics relevant to your business – Invest in creating meaningful metrics for your business.
   

Additionally, Pretorius discusses the engagement measurement within the context of some potential improvements to Microsoft’s Engagement Mapping (EMAP) methodology: assumptions by advertisers without factual prove, and more data is better at prediction than better data mining algorithms. See, http://atomai.blogspot.com/2008/04/more-data-usually-beats-better.html

Stephen Pretorius article is called Why CFOs Don’t Believe in Online Advertising and it can be found at: http://www.adotas.com/2008/07/why-cfos-don’t-believe-in-online-advertising/

Google Analytics Switch Data Processing Times or Reporting Errors

Over the course of a month or two, many Google Analytics users began distrusting the reports provided in Google Analytics. We now have more reports, this time via Search Engine Roundtable Forums of Google Analytics not reporting accurate numbers.
I noticed yesterday's reports for this site was down about 20%. I immediately thought, give it a day and see if the numbers are higher tomorrow. They were. The numbers reported for Tuesday on Wednesday, was at normal levels when I looked today, Thursday. So I thought, maybe Google Analytics changed the time they are processing these reports (i.e. pulling in less data because they are running the reports earlier).
For example, let's say, Google Analytics typically runs the reports at 3am every morning. If Google Analytics pushed the report runs to 10pm every night, then you will be missing out on 2 hours of traffic, from the previous day. That means, you would have to wait a two-day period to see your full traffic data for the previous day. Hope I explained that well.
In any event, I am also hearing these reports outside of the forums - so this seems fairly widespread. Is this an bug or a feature - now that is the question.
Forum discussion at Search Engine Roundtable Forums.

Mending Fences Between Microsoft and Yahoo

It seems that Microsoft and Yahoo may enter into some type of an agreement if Carl Icahn wins the majority of Yahoo’s shareholders votes on August 1. If that happens Microsoft and Yahoo management will need to mend fences in order to move efficiently forward. There are two ways to mend a broken fence: discard or repair. In this situation, Icahn has stated that he will discard Yahoo’s current board of directors for a different slate of directors willing to seriously consider a business agreement with Microsoft. This is an important, but small piece of fence that is broken between these two companies.

The next layer will be the executive management of Yahoo. Microsoft’s goal would be to identify the members of the executive team that could be repaired vis-à-vis—those that are beyond salvage. A definition of beyond salvage is going to be crucial at this early stage. I assume that those members of executive management beyond salvage are those that are not willing

1. to cooperate with Microsoft’s executive management;
2. to have the flexibility to change Yahoo’s current business strategy; or
3. to take the time to understand Microsoft’s culture and business strategy.

The key ingredient is willingness. Willingness can be explained as a voluntary eagerness or disposition to act without reluctance to accomplish the goal of integrating both cultures while advancing Microsoft’s business objectives. Willingness comes from within an individual, hence it can only the measured through the actions or inaction of a particular individual.

Microsoft management also has the task of mending fences, since most fences can be repaired. This will require small but consistent acts from Microsoft that show business maturity and personal understanding to difficult changes. It reminds me of the story of the young boys that used to play baseball in an alley and the old lady that would never give them the balls that came into her yard. One of those boys started to water her lawn and clean the leaves on her patio for an entire year. He did this voluntarily and without expecting anything in return. He saw a need and he acted according to his conscience. After an entire year of doing this, the old lady called him to her house and gave him all the stray baseballs that she had collected over the years. He realized that more important than the baseballs was that he had her trust. It takes business maturity and transparency to earn trust in this type of situation.

Microsoft and Yahoo management can expect a difficult road ahead. It will not be perfect, but the combination of these two companies should be rewarding and worthwhile.

8 Lessons of Leadership

Read this article in Time magazine. For those in business analytics it brings eight principles that we should be applying to provide leadership to our organizations:

1. Courage is not the absence of fear - it's inspiring others to move beyond it. I am sure that getting involved in changing the status quo brings fear of rejection. The issue is not whether changing the status quo brings fear or not, it is the leadership that we provide to inspire other to move beyond that fear.
2. Lead from the front - but don't leave your base behind. Your base is your co-workers and your clients (internal and external). Incremental changes make changes easier than drastic changes.
3. Lead from the back - and let others believe they are in front. Give your co-workers permission to try their own ideas, and permission to fail. When people in good faith unite for a common purpose the impossible become possible.
4. Know your enemy - and learn about his favorite sport. It is part of the natural order to have adversaries, but it is important to have something in common with everybody within an organization. Besides sports I recommend talking about family, since this is a common denominator to all of us.
5. Keep your friends close - and your rivals even closer. Your friends will help you reach your goals, but your rivals can block your progress. The person that opposes the initial objective the most is a great candidate to lead the second version of the product.
6. Appearances matter - and remember to smile. Always make sure that your appearance is professional. This is not the university, this is the business world and appearances do matter.
7. Nothing is black or white. Please learn the difference between what is illegal and unethical. Illegal is against the law. Nobody is perfect, including you.
8. Quitting is leading too. We can exercise leadership by changing course. Reach consensus on metrics and timetable up front in the process and they will be the best indicator when is time to change course or make adjustments during the process.
   

Data Mining Combined With Predictive Modeling Equal 3D Data Visualization

The interaction and cooperation between computers and the human brain is at a crossroad. There are some who believe that decision support systems should be completely automated. There are others who believe that there are many areas of business, technology, and science that have not been discovered yet, and, hence, only part of a decision support system can be automated. I subscribe to the latter proposition.

Computer science is, at its core, an attempt to replicate the processing, reasoning, and learning processes of the human brain. Therefore, an understanding of the human brain is fundamental to determine the next steps into advances in the area of business analytics (i.e. the fusion of technology, science, and business). See http://atomai.blogspot.com/2008/06/intersection-of-business-science-and.html; and http://atomai.blogspot.com/2008/05/attention-system-of-human-brain.html.

Visualization is an important method that the human brain uses to perceive and make decisions. See http://atomai.blogspot.com/2008/06/on-visualization-of-mathematics.html. Classification into groups with similar characteristics, or data mining, is another method to make decisions. Depth and movement perception based on prior experience for the human brain is the equivalent of what we call in science predictive modeling or forecasting. The amalgamation of these elements in a decision support system is equivalent to the way the human brain makes decisions.

History teaches us that some discoveries provide a quantum leap of understanding. The earth revolving around the sun, gravity, and the theory of relativity are examples of these types of discoveries. On the other hand, most decision making is incremental in nature. See http://atomai.blogspot.com/2008/05/kaizen-analytics-continuous-improvement.html. For example, buying securities collateralized with real estate might not be the best investment in the United States right now; buying short-term oil contracts in the commodities market might be a more profitable investment.

We have reached a point in our technological development that we can put together genetic algorithms, data mining techniques, grid or cloud computing, and visualization techniques using gaming technologies to automate some areas of decision support systems to reduce operational costs. For an interesting perspective on how gaming techniques use predictive modeling to forecast movement, see http://web.cs.wpi.edu/~claypool/courses/4513-B03/papers/games/bernier.pdf. Also, the joining of these techniques will facilitate a decision support system to make both incremental as well as quantum leap discoveries in many business areas.

Microsoft Announces Name, Pricing for Subscription Office

This is one important piece of the Microsoft strategy to integrate Software-as-a-Service as part of it Internet strategy. See, http://atomai.blogspot.com/2008/06/internet-business-models-and-analytics.html; and http://atomai.blogspot.com/2008/06/delivering-software-as-service.html. The price will need to be adjusted to an Internet model of less than $50, it should be on a month-to-month basis without limited restrictions, and it should incorporate Microsoft Analysis Services in order to be competitive. See, http://atomai.blogspot.com/2008/06/business-analytics-and-software-as.html.

On the other hand, it shows that Microsoft is aware of the basic components to have a sucessfull Internet strategy and the steps that it can take to make those changes. The failed adquisition of Yahoo would have made it easier for Micrsosoft in terms of the change management that need to occurr for them to make these changes. Yahoo already has an Internet business and culture that could more easily adapt to these type of changes than the Microsoft propietary software business culture.

Also, Microsoft adquistion of semantic search engine Powerset shows the understanding by Microsoft of the importance of Internet "search speed" as part of its strategy.

Genetic Algorithms, Grid Computing, and Visualization Techniques

In the beginning of the technological era, we used computers to more efficiently process information. Now we use computers to help us solve problems. The next phase is to have computers generate their own solutions to problems. The last frontier is to have man and computers cooperatively competing to find solutions to complex business problems.

John Koza at Genetic Programming provides an example of computers generating their own solutions (and coding) to complex problems. Koza, a professor at Stanford University, is a thought leader in this area. “Genetic programming (GP) is an automated method for creating a working computer program from a high-level problem statement of a problem. Genetic programming starts from a high-level statement of ‘what needs to be done’ and automatically creates a computer program to solve the problem.” See, http://www.genetic-programming.com/

In the past I have written about genetic algorithms and their practical applications in business analytics. See, http://atomai.blogspot.com/2006/08/artificial-intelligence-applied-to.html; http://atomai.blogspot.com/2006/08/segregative-genetic-algorithms-sega.html; and http://atomai.blogspot.com/2006/07/genetic-algorithm-based-optimization.html.

If we combine the power of genetic programming, grid computing, predictive modeling, and data mining with powerful dynamic three dimensional visualization techniques used by game developers, we can have a world where complex business problems can be solved by cooperatively competing between man and machine. See, http://atomai.blogspot.com/2008/06/on-visualization-of-mathematics.html. The Internet and Software-as-a-Service are going to be an integral part of that mix of ingredients as a way to lower IT costs.

The next step is akin to a chef putting together a gourmet meal by using the finest ingredients and tools of the trade. A conductor brings out the most expressive details of a classical piece while blending all the sounds of an orchestra as one exquisite experience.

If anticipation is part of journey, I can hardly wait for the sounds and tastes that will come in the next 12 months when business, technology, and science merge to produce masterpieces to help solve complex business problems. See, http://atomai.blogspot.com/2008/06/intersection-of-business-science-and.html

Unleashing the Power of the Mind™

VISUALISATION USING GAME ENGINES

This is a good introductory article that explains how to adapt game engines visualization to business analytics in mobile devices. The issue to create a visualization that captures the entire data set in 3D.

At the present time businesses are used to visualize the entire data set using Excel charts. One way to augment the Excel charts and create depth perception into the data set is to add a predict probability vector. SQL Analysis Services already provide this capability. If you combine the add-in Analysis Services capability to Excel, you can see your entire data set in 3D. In other words, you combine data mining and predictive analytics capabilities in the same chart. This allows the user to see the entire data set in 3D.

In the next twelve months we will see more efficient ways to see large data set in 3D using GIS graphs.

The Internet, Business Models and Analytics

The Internet is ready for another dramatic change within the next twelve months. Technology, science and business are converging on the Internet. See Intersection of Business, Science, and Technology: Business Analytics. The speed and capacity of computers has brought us to a place “where no one has gone before”. See High Performance Computing (HPC) is Changing the World; HP Delivers Real-Time Business Information with Enhanced Neoview Capabilities; and Math Grid Toolkit brings grid computing to business. Science has provided us with powerful data mining and predictive algorithms that are used in healthcare, financial services, energy, and government industries. See Business Objects Aims to Predict the Future; The New ADAPA iGoogle Predictive Analytics; and BI Conceptual Architecture using Microsoft Products.

Companies like IBM, SAS, Google, and Microsoft are investing not only in the latest science and technology, but on different business models that will make the most of the capabilities of the Internet. The next elements to this exciting pursuit of the next Internet generation are visualization, mobile devices, and Software-as-a-Service. See On the Visualization of Mathematics (Analytics); Mobile Devices and Business Analytics; and Delivering Software as a Service.

Companies that are successful in harnessing the reach of the Internet will need a powerful infrastructure that would support virtual machines using grid computing. Companies like Microsoft, IBM, Oracle, and SAS would need to add a different Internet business model to their current business model in order to overcome the advantage that Google has in providing Software-as-a-Service and analytics through the Internet. Google’s partnership with Zementis has all fundamentals to reach individuals, as well as small, medium, and large businesses. See Business Analytics and Software-as-a-Service: Controlling IT costs. Companies like Microsoft have the potential to challenge Google if they implement an Internet business model that incorporates business analytics and Software-as-a-Service elements.

The human brain has been waiting thousand of years for business, science and technology to come up to par to its capabilities. See The Attention System of the Human Brain. We can harness this power by using the reach of the Internet to bring together the minds of millions of people. See Kaizen and Analytics.


Unleashing the Power of the Mind™

Business Analytics and Software-as-a-Service: Controlling IT costs

During an economic downturn IT executives are looking for ways to control the cost of hardware and software while simultaneously providing their companies ways to improve its business analytics capabilities. I had a demo of the ADAPA Predictive Analytics Edition on Amazon EC2 and was impressed with its capacity to provide powerful analytics at a fraction of the investment for hardware and software for predictive modeling. This is a product for the current power users of analytics within an enterprise. You can get a subscription for $49 per month plus CPU usage to start with. If you are not satisfied with the product your investment will not have any impact in your budget.

Try this product and let me know your comments. http://www.zementis.com/howtobuy.htm

I think that this predictive analytics product and business model is the future. I recommend that this product add a cluster analytics functionality that will incorporate data mining into the suite of algorithms. Also, I recommend a dynamic 3D visualization to “see” the entire data set. Otherwise, this is a powerful business model to provide cost-efficient analytics to an enterprise.

Attensity - Text Analytics

We should be hearing more and more about Attensity text analytics products and solutions. The issue is how to do analytics with unstructured data, and they have developed a powerful product.

KXEN Analytic Framework Version 5.0 Accelerates Data Mining Automation Throughout the Analytic Enterprise

KXen product is powerful and its business model "to move beyond traditional cottage-industry analysis to large-scale factory analysis" is the correct approach.

Analyze This: Four Fundamentals of Business Analytics

Very general article but captures four basic ingredients needed to implement business analytics.

1. Leaders who "get it".
2. Staff who love numbers
3. Processess that revolve around facts
4. Technology to capture, clean, sort, and make sense of data
   

Double Feature Selection and Cluster Analyses in Mining of Microarray Data from Cotton

This technique can be applied to data mining in the financial services, banking, investment banking, and healthcare industries.

Nationwide and Ohio State Partner to Address Real World Business Problems

This is an example of a partnership between business and universities utilizing data mining and predictive modeling

On the Visualization of Mathematics (Analytics)

While reading articles on analytics and mobile devices, it occurred to me that I need to be clearer on explaining my vision on the visualization of mathematics. I believe that the visualization of mathematics, by using analytics techniques, will transform the way we do business the same way the invention of the printing press transformed the world. Mathematics is the universal language, and its visualization is a form of expression. This is where business, science, and technology meet.

Before the printing press was invented, knowledge was in the purview of a few individuals who knew how to read. Once the press was invented, knowledge and communication became available to all. With the advent of word processing software, more people could not only read but write within their own sphere of influence. Of course, the ability to read and write does not mean that everyone will be a Shakespeare, Neruda, or Solzhenitsyn. The same is true with the visualization of mathematics. The analytics power users will not disappear; if anything, they will become more visible within an organization.

Sometimes we tend to equate Excel reports with analytics. This is a mistake. Reports have a limited capacity to represent the full spectrum of analytics. The alphabet is the fundamental of writing, but only represents a very limited spectrum of the capacity of expression of written communication. A poem by Robert Frost, a novel by Garcia Marquez or J.K. Rowling can create thousands of similar but slightly different mental pictures to different readers.

It has been said that a picture is worth a thousand words. We are living in a world of large datasets. Companies, governments, and organizations are challenged everyday in storing large datasets that contain structured and unstructured data. We have mobile devices that have enlarged real-time communications in ways that 20 years ago were unimaginable. Science has progressed to bring us the capacity to store and analyze large volumes of data. We can combine the advances of business, science, and technology with the capacity of the human brain by visualizing analytics.

The human brain is made to perceive, understand and analyze three-dimensional geometrical configurations. See, The Attention System of the Human Brain at: http://atomai.blogspot.com/2008/05/attention-system-of-human-brain.html. Mathematics is the foundation of analytics, as the alphabet is the basis for written communication. Analytics can be turned into three-dimensional geometrical configurations that will change the way we make decisions by giving us an evidence-supported decision support system within our own sphere of influence. Analytics is not a report. The power of the visualization of analytics will allow each person to “see” the same analysis but will allow each person to have a slightly different interpretation of how to apply that knowledge in her own life. The visualization of analytics will transform the world by having the power of mathematics at the fingertips of everyone.

Delivering Software as a Service

This McKinsey Quarterly article is right on point as to this emerging business model. See, http://atomai.blogspot.com/2008/03/software-as-service-overview.html. The three areas that software vendors must pay attention in this area are:

1. Adjust the Revenue Model – Although initial sales costs are higher, a SaaS model target small and medium size businesses which are the driving force of revenues and profits during an economic recession. See, Tools During Economic Recession: Forecasting & Business Analytics; and Preparing Your Company For Recession.


2. Build a Platform – Do not take too long to recognize SaaS as a profit center otherwise you could make the Microsoft mistake of not recognizing the Internet as a powerful media and advertisement generating revenue center. The catch up business model has proven to be an inefficient way to growth a business.


3. Improve Internal Capabilities – “The biggest capability gap for software companies embracing the new model is in the operational and customer service skills necessary to deliver software online. The operational challenge is to host the software rather than shrink-wrap and ship it. Companies will have to develop capabilities to handle massive data center operations, systems and network monitoring, and billing.”Companies with a mature internet presence like Google, Microsoft, and Yahoo, have an advantage over smaller software vendors that do not the internal capabilities. On the other hand, the major companies must improve their decision making by making adjustments to those capabilities.

Business analytics allows companies to improve their internal capabilities efficiently. See, Explaining the Value of Business Analytics: Clarifying driving factors in a decision support system. Adjusting the revenue model is going to require executive management to allocate investment in a SaaS model. See, Enterprise Analytics: A Business Decision. The role of IT is to build an Internet-driven platform that supports real-time analytics. See, HP Delivers Real-time Business Information with Enhanced Neoview Capabilities.

A SaaS model is fundamentally based on the principle that every person in an enterprise can contribute to improve revenues and profits by making small incremental changes in their areas of responsibilities. See, Kaizen and Analytics: The Power of Each Employee to See Data. The next frontier of SaaS will be business analytics using mobile devices. See, Mobile Devices and Business Analytics.

Unleashing the Power of the Mind™

Mobile Devices and Business Analytics

1. The next frontier of business analytics is the incorporation of mobile devices into an enterprise decision support system. The key is to have the following functionalities:
   (a) It must give a visual representation of the entire dataset; and (b) the screens in mobile devices are small, hence the visualization of large datasets requires the ability to see the data three dimensionally. There are two potential solutions to approach this problem:

1. Excel type of flat bar charts – Use clustering analysis to separate the clusters with similar characteristics, and include as a variable in the cluster a vector that that will clarify the driving factors. Microsoft, SAS, and Business Objects already have these functionalities (i.e., clustering and create a vector using a predict probability). The issue is whether they can have a universal interface with mobile devices for these charts. See, INSTINCT GAINS INTELLIGENCE; Three-Dimensional Business Analytics: How Deep is the Ocean?; and Explaining the Value of Business Analytics: Clarifying driving factors in a decision support system

2. Three-dimension graphical representation – There are companies and academic institutions that are gravitating toward a 3D representation of all the data. I believe that in the future we will integrate this to the flat charts most currently in used nowadays. See as an example, http://thedmblog.wordpress.com/2008/06/04/visualizing-large-graphs/; and Real Time Scalable Visual Analysis on Mobile Devices

Another essential functionality is the ability to drill-down capability from a graph to a report – Some companies already has this functionality as part of the mobile device. See as an example, http://www.webalo.com/index.html

Unleashing the Power of the Mind™

Business Objects aims to predict the future with business intelligence

This is a powerful move for Business Objects and SAP. Business Objects have had for years the click-and-drag reporting capabilities, the functionality of standard deviation, and with the predictive analytics capabilities this will become a powerful force in business analytics. I have always been impressed by the products of these companies. The combination of these two companies, their technical functionalities and worldwide market reach put us closer to unleash the power of the mind by putting business analytics at the fingertip of every employee in a company.

Tools During Economic Recession: Forecasting & Business Analytics

The economic recession in the U.S., the housing and credit crisis, and the high oil prices are bringing business analytics to the forefront of companies. See Preparing your Company for a Recession at http://atomai.blogspot.com/2008/02/preparing-your-company-for-recession.html. In good economic times business analytics is important. In challenging economic times it may be the difference between success and surviving.

These are the best of times and the worst of time. Business, technology, and science have leap hundreds of years in the last couple of years. Risk and reward are closely intertwined with the vision of our business leaders and their ability to be at the forefront of change. See Evidence-based Enterprise Business Analytics Model: Turning Around in a Quarter. Companies must use all the resources available to become better at forecasting risk. See Bernanke: Banks must get better at foreseeing risk....

Those of us in business analytics have the ability and knowledge to incorporate forecasting and data mining techniques that separates the clusters of data and clarify the driving factors. See Business Case for Analytics: Explaining Cluster Analysis; and Explaining the Value of Business Analytics: Clarifying driving factors in a decision support system. Those of us in business analytics, data mining, and forecasting must do better in explaining the business value to our companies. See Enterprise Analytics: A Business Decision.

We must be specific and present a complete strategy to executive management whether is a workplace reorganization of restructuring, a marketing plan, an investment strategy, or an operational strategy. We cannot just give "the numbers" and not look at the big picture. See Best-laid plans . . .at http://www.app.com/apps/pbcs.dll/article?AID=/20080606/SPORTS06/806060350/1002/SPORTS

We have the ability and responsibility to present solutions. We can make a difference. The future is now. See Intersection of Business, Science, and Technology: Business Analytics.

Unleashing the Power of the Mind™

Real-Time Business Analytics

A new discovery about the human brain gives us a good insight in how to improve real-time business analytics and predictive modeling. The discovery is that the brain compensates the neural lag that everyone experiences by giving us a glimpse of events one-tenth of a second before they occur. See Key to All Optical Illusions Discovered. The brain predicts what is going to occur based on available data and experience. See The Attention System of the Human Brain. Imagine the implication for the financial services industry. Banks, investment banks, and brokerage houses could profit or lose billions during stocks and commodities trading, if they had a one-tenth advantage over their competitors. See Financial Services Business Analytics: Evidence-Based Model. Envision a system that reminds customers about an item that they may have forgotten before leaving the store in the retail industry. Also, picture a system that reminds network administrators to check a specific system, server, or pathway. Depict a system that reminds physicians to check for the symptoms of a specific disease, recommend a particular test, or reminds a surgeon to check a particular area of the body.

In an interdependent and continuously changing global economy, companies need to accurately forecast and anticipate trends. Companies must use the latest advances in science, technology, and business to gain leverage and improve profitability. See Intersection of Business, Science, and Technology: Business Analytics; and Business Analytics Technique: Borrowing From Medical Imaging at http://atomai.blogspot.com/2008/05/new-business-analytics-technique.html.

The advances in technology and sciences allow these tools to be implemented rapidly and in a cost efficient manner. See High Performance Computing (HPC) is Changing the World; HP Delivers Real-time Business Information with Enhanced Neoview Capabilities; and Business Competency Model: Turning Around in a Quarter.

Business Analytics: Unleashing the Power of the Mind™

Analytic Culture – Does It Matter?

This is a good article that explains a difficult issue in change management in the area of business intelligence and change management: the importance of understanding the culture within a company if you want to achieve change. Dave Wells has done an excellent job at explaining how to identify different cultures. The key for change is to adapt our product and services to the existing culture. This is one reason that Microsoft BI stack, or the iGoogle predictive analytics tool could become the products of the future. See, The New ADAPA iGoogle gadget: Google Predictive Analytics; Microsoft SQL 2005 Analysis Services: Ten Best Practices© at http://atomai.blogspot.com/2007/03/data-mining-and-microsoft-sql-2005_23.html. These products appeal to the broader cultures: people who already use Excel, or people who want to limit their cost by using Software-as-a-Service. See, Software as a Service overview . It is important to remember that it is not just the algorithm but the amount of data that makes business analytics a reality today. See, More data usually beats better algorithms . SAS and KXEN have powerful algorithms and business models, but I think that those products do not take into account the culture within a company as well Google and Microsoft.

High Performance Computing (HPC) is Changing the World

This short video is a powerful tool to explain how science and technology are interacting for advacements beyond our imagination, and the concept of HCP. "Powerful Beyond Imagination". The International Conference on High Performance Computing, Networks, Storage, and Analysis will take place on November 15-21, 2008, in Austin, Texas. http://sc08.supercomputing.org/ This is the 20 years celebration of the super computing conference!

These two events may be fun:

• a Music Room, conveniently located in the convention center lobby, where conference attendees can sit and listen to other attendees play during scheduled times throughout the conference. Basic orchestral and woodwind instruments will be available, including a Baby Grand piano, but you are encouraged to bring your own. (A reservation must be made in order to secure a time for you to play in the music room. More about signing up will be provided later).
• Digital Vibrations: Is it Real?, where attendees will visit a kiosk, listen to a series of songs, and identify ones they think are computer generated versus those they think are produced with actual instruments. A prize will be awarded to the person who correctly identifies all songs.

HP Delivers Real-time Business Information with Enhanced Neoview Capabilities

These are the type of advances in technology that makes business analytics a reality today, by imitating the capabilities of the human brain. The enhanced Neoview capabilities for business intelligence are:

1. It efficiently processes large and small transactions simultaneously
2. Forecast traffic to avoid delays and improve performance (i.e., it avoids those instances when you see a person but forget their name, or
3. Make sure that real-time information is up-to-date (i.e., avoid those moments when you go to a room to do something and ask yourself, "why am I here?")
   

Intersection of Business, Science, and Technology: Business Analytics

We are living in the future. Some time ago business, science, and technology were three distinct and separate disciplines. The advancements in these three disciplines during the last five years have created a quantum leap in how each directly affects the others. Businesses, large and small, make global transactions on a daily basis. Technology allows people from around the world to communicate internationally and storage large volumes of digital data. Science allows accurate predictions using large databases around the world.

What is business analytics? Business analytics is the intersection of business, science, and technology that allow us to unleash the power of the brain. Historically, mankind has attempted to replicate the decision-making prowess of the brain in order to progress. The financial services industry has transformed from local lenders and borrowers, to complex international financial transactions involving international and local banks, investment bankers and insurance companies. The healthcare industry has worldwide specialty surgeries, and the pharmaceutical industry cooperative bio-molecular research and genome research, facilities. The chemical and energy industries have found new ways to find and convert raw materials. The media industry has gone from the books to the internet.

The common characteristics of these advances are the way that business, technology, and science have cooperatively intersected to promote common objectives. The next frontier of business analytics is to tap into the collective power of the individual and unleash the power of the human brain. Companies like GE, Microsoft, Hewlett-Packard, IBM, SAS Institute, Exxon, and Google are among the thousand of companies around the world that are working to unleash the power of the mind by intersecting business, technology, and science. Business analytics is the future, and the future is now.

Business Analytics Technique: Borrowing From Medical Imaging

A study published in the journal Nature shows a scanning technique developed by GE Healthcare to detect cancer that can be adapted to business analytics in any industry. This technique involves the concept of separating the clusters in the data and clarifying the driving factors. See, Explaining the Value of Business Analytics: Clarifying driving factors in a decision support system.

The technique involves:
1. Cluster analysis to separate the data. See, Business Case for Analytics: Explaining Cluster Analysis.
2. Vector analysis using regression to clarify the driving factors. See, Three-Dimensional Business Analytics: How Deep is the Ocean?
3. A Human-Interaction Computer Visualization tool. See, Human Factors In Visualization Research; Visualization Projects for Data Mining; Information Visualization and Visual Data Mining.

Business analytics can adapt some of the state-of-the-art solutions from medical imaging techniques and use them for business forecasting, as well as to determine whether strategies are working well at an earlier stage. This would allow businesses to be flexible and adapt to changing an ever changing economic environment. See, Preparing Your Company For Recession; and Evidence-based Enterprise Business Analytics Model: Turning Around in a Quarter.

I envision in the future that business analytics techniques will be at the same level than medical imaging. See, NIH-NSF Visualization Research Challenges Report; and Kaizen and Analytics: The Power of Each Employee to See Data.

Unleashing the Power of the Mind!

iGoogle Predictive Analytics Gadget: Feedback

This is the feedback that I have gotten so far on this new predictive analytics tool

E. Harris - “I think the gadget is a boost to marketing campaign managers, small business website owners and SEO professionals. It gives an immediate ability to test and analyze content and campaigns real-time. I think there is a high degree of value here as the wait time for results will be Google dependent. There is also independence to the subscriber, if you will, to become less reliant on the high costs associated with statistic and ranking providers. We may also see an increase in the length of time remains on a website as it is evaluated predictively for value.”

D. Padmanabhan - “I tried a demo NN model and found it interesting. It will be interesting if the models can be shared among different users. I would also like to see additional outputs such as prediction errors or maybe even error charts.”

You can see my comments and Mike Zeller (CEO of Zementis that developed this iGoogle tool) response at: The New ADAPA iGoogle gadget: Google Predictive Analytics

The competition between Microsoft and Google in the area of analytics is unfolding. See, The Future of Business Analytics: Microsoft vs. Google. Microsoft SQL 2005 Server Analysis Services and the Excel 2007 add-in offer a proprietary model with the advantages that people are used to Microsoft products. The Google gadget for predictive analytics offers a new model of Software-as-a-Service, which could decrease the cost of predictive modeling in a company. See, Software as a Service overview; and Delivering Software as a Service.

The Attention System of the Human Brain

It is important to understand how the human brain functions in the area attention, at the cognitive and neuronal levels, so we can attempt to replicate these functions in any decision support and business analytics system. The concepts in this article cover the fundamentals that every person in the analytics field should understand. Among the interesting areas for practical application:

1. Data processing and analytics are separate functions that interact – A practical application is that we should not be processing and analyzing data in the same server.
   Orienting – Practical applications include: visualization tools, human-computer interaction (HCI), and the need for size, color and depth (right and left hemispheres) representation of the data. See, Business Case for Analytics: Explaining Cluster Analysis; Explaining the Value of Business Analytics: Clarifying driving factors in a decision support system ; and Three-Dimensional Business Analytics: How Deep is the Ocean?
2. Pattern Recognition – Practical applications include: utilization of colors in the analytics visualization tools; utilization of forecasting vectors to determine complex patterns; and data does not need to be 100% clean since the brain will fill the missing parts.
3. Detection – Practical applications include: the ability to visually represent data in its totality, while having the functionality to dynamically select specific areas of the data for further study; interactivity for further study depends n alertness and this depends on robust and fast system of analytics (i.e., the fastest the report creation the higher our alertness and detection); and speed in the system is directly correlated to lower quality of data and produces a higher error rate.

Knowing the anatomical reasons for decision support and analytics in brain function gives a higher importance to create decision support systems (which include predictive modeling) and make it available to everyone within a company or organization. See, Kaizen and Analytics: The Power of Each Employee to See Data; and Business Analytics: Unleashing the Power from Within .

The New ADAPA iGoogle gadget: Google Predictive Analytics

After talking with Mike Zeller the CEO of Zementis I decided to re post this blog with Mike comments. In the interest of full-disclosure, Mike is a professional colleague and I always enjoy our conversations. In a nutshell, Mike wants to make the point that:
1. This product is for those who are knowledgeable in the field of analytics.
2. Import/Export functionality using "browse" capability is part of the product.
3. Import/Export functionality for Excel files is already part of this product.

The fact that Google is moving toward Software-as-a-Service in analytics is impressive and the correct direction. Also, Mike points out that this is the first step in a journey to put analytics capabilities in the hands of everyone. The future is here and it is exciting! Enjoy the post, and by all means give this product a try and let me know your opinion (I do not post anonymous opinions).

You can install the engine as an iGoogle gadget by following the link below:http://www.google.com/ig/adde?hl=en&moduleurl=hosting.gmodules.com/ig/gadgets/file/115640297026242314759/adapawidget.xml

I was looking at the new ADAPA iGoogle gadget and this is a tool that would mostly be used by people that have technical knowledge about predictive analytics. There are some basic improvements that must be made so that analytics is available to "everyone" (see my post Business Analytics: Unleashing the Power from Within).

Mike's comments: "Yes, you are correct that ADAPA is a tool for people who are somewhat familiar with predictive analytics. The intention of the ADAPA iGoogle gadget is not to teach "everyone" about predictive analytics, but to deliver a tool that makes it easy to deploy and use such models.

ADAPA bridges the gap between the scientists developing the models, the IT folks who have to integrate & deploy the models, and the business who ultimately want to use the results of the models to drive the business process.

Predictive analytics uses mathematical models that assume a certain technical knowledge and for many users that will always remain a black box. Only in the context of a business application, most users will understand the outcome of a particular model, e.g., that a high risk score means that the related credit application is bad for business.

Following your vision, we should strive towards building many of these iGoogle Gadgets in a form that focuses on the business user and "hides" the technical details.

Our ADAPA iGoogle Gadget was just an example to demonstrate that is actually possible to deliver predictive analytics as service and make it available via simple iGoogle gadgets. We wanted to empower the scientists and engineers to easily build business applications that leverage the potential of predictive analytics."

My wish list for this product:

Keep it simple!

1. The Demo is talking about PMML standards, etc. Most users do not care about the technical aspects and standards, they just want to dig in and make it happen. Do not get a technical person to explain how it works, get a sales person to explain how it works. We usually complaint that sales people over simplify technical issues, but this is a case that "sometimes less is better than more". Go back to Google home page and compare it with Yahoo and MSN to understand this concept.
   Mike's comments: "In order to understand the basics about predictive analytics, it is critical to know that there is a standard to represent such models. If you want to represent a web page, you need to know that there is something like HTML. If you want to represent a predictive model, you need to know that the common language is PMML and that there is such a file format.
   Unfortunately, we still have much work to do in order to evangelize the PMML standard. Many people don't know about this standard which allows models to be shared between various software tools from different vendors."
2. The converter should be behind the scenes. Again, just make it happen and do not bore us with the details.
   Mike's comments: "We made a conscious decision to keep the PMML converter a separate product, so users can choose to only convert various PMML versions, but not be locked into using our ADAPA deployment platform. It is our support to the PMML community and will remain a free service."
3. Explanation for models should be simple. For example: "Linear regression takes numerical values only. Numerical means that if you can add those values and it means something. You can represent months in numbers but if you add those numbers it does not mean anything. Numerical representation of months are not a numerical value."
   Mike's comments: "'Explanation' are supposed to explain what the model does, this is an element of the PMML standard (description). Maybe it could be more descriptive, but it is not really the place to explain how a linear regression works."
4. I think that the future of analytics is to allow everyone to see the clusters in the data and clarify the driving factors. See, Kaizen and Analytics: The Power of Each Employee to See Data

Google must overcome the basic rule: People would rather live with a problem they cannot solve than accept a solution they cannot understand . I am sure that the science behind the models are good. On the other hand, the science behind this type of algorithms have been out in the market for a long time. The differentiators are: that is free and that it could be used by a company as a Software-as-a-Service (SaaS). See, Software as a Service overview.
Mike's comments: "We are on the same page here. The goal of this application is to make it easy to deploy, integrate, and execute predictive models.
It is the first step towards a more general adoption of predictive analytics. Now we can work towards the same simplicity in business applications that sit on top of the ADAPA deployment platform and draw their decisions from the predictive power of advanced scientific algorithms."

Predictive Analytics Engine available as an iGoogle gadget

This was a posting in a group that I have at Facebook (Analytics, Data Mining, Predictive Modeling, Artificial Intelligence). As predicted on May 12, 2008, the next battleground between Google and Microsoft is going to be in the realm of business analytics. The science in this Google product is good, as is the science behind Microsoft Analysis Services too.

In my opinion Google is correct in assuming that the future business analytics is in Software-as-a-Service (SaaS). I specifically addressed this topic on blog postings of March 10 and March 26

"If you are interested in executing data mining models such as neural networks, SVMs, logistic and linear regression, ... from your desktop, check out the new ADAPA iGoogle gadget. It allows for the uploading of data mining models expressed in PMML (Predictive Modeling Markup Language) for scoring. Model execution is done through Amazon EC2 and is free. A paid version which will allow for real-time scoring via web-services (as a SaaS - Software as a Service) is going to be launched in May. You can install the engine as an iGoogle gadget by following the link below:http://www.google.com/ig/adde?hl=en&moduleurl=hosting.gmodules.com/ig/gadgets/file/115640297026242314759/adapawidget.xmlAlso available is a converter gadget to convert old PMML to version 3.2:http://www.google.com/ig/adde?hl=en&moduleurl=hosting.gmodules.com/ig/gadgets/file/115640297026242314759/converterwidget.xmlDecision trees is next in the range of mining models that ADAPA supports. Have fun!Alex"

Business Analytics: Unleashing the Power from Within

Every person has the inherent ability to interpret the world around them in a physical and in an intuitive manner. This ability of the human mind to deduce facts and patterns based on both external and internal conditions is one of the main characteristics that separate us from other species. Business analytics have evolved to give every member of a company the ability to interpret known metrics like revenue and income, as well as internal evidence-based multidimensional analysis (like the predict probability for an event to occur) which clarifies the drivers in the data.

In my experience, executives understand the concept that people are the most important asset within an organization. Also, they understand the importance of business analytics to maintain a competitive advantage. Business analytics is an enterprise continuous improvement system that allows every person within an organization to detect patterns in the data that allow for incremental improvements on a regular basis. Companies are like large ships, they are difficult to turn around. The advantage of business analytics is that it allows for regular and incremental changes in a business that allows for change to take place.

Allowing every employee the ability to suggest incremental changes, based on business analytics, which will improve their own performance, is kaizen analytics. Kaizen analytics assumes that employees are the most important asset in an organization. Therefore, analytics and improvement should permeate through out the entire company.

In the last few years some of the best minds in software companies have developed the technology and science to put business analytics at the fingertip of every employee: from the entry-level to the executive suite. Companies have the capacity to unleash the power from within each employee by deploying business analytics. The implementation of business analytics allows companies to separate the cluster of data and clarify the driving factors.

Some companies already have the internal capacity to deploy business analytics, but lack the skills to implement a business analytics system. For example, companies that already have Microsoft SQL 2005 Server and Excel 2007 are in a position to quickly turn around a business analytics system. If you want to unleash the power from within your company contact me at Hewlett-Packard: alberto.roldan@hp.com

BI Enterprise Conceptual Architecture Solutions in the Financial Services Industry and Healthcare Industry: Microsoft SQL 2005 Server and Excel 2007

If you are a Microsoft shop and want to implement a business intelligent (BI) solution using SQL 2005 Server and Excel 2007, these are the high-level conceptual architectures for the financial services and healthcare industries. If you need any assistance in implementing these models please contact me at Hewlett-Packard: alberto.roldan@hp.com

Bernanke: Banks must get better at foreseeing risk

I have been writing about this issue since March. I have proposed a comprehensive forecasting system that can be available to everybody within a financial company, and can be deploy in three months. For more details about the business analytics model that I propose see the following articles that I have published since March 2008.
1. Financial Services Business Analytics: Evidence-Based Model
2. Three-Dimensional Business Analytics: How Deep is the Ocean?
3. Business Case for Analytics: Explaining Cluster Analysis
4. Explaining the Value of Business Analytics: Clarifying driving factors in a decision support system
5. Business Competency Model: Turning Around in a Quarter
6. Default Risk, Asset Pricing and Debt Control
7. DEVELOPING RICH INSIGHTS ON PUBLIC INTERNET FIRM ENTRY AND EXIT BASED ON SURVIVAL ANALYSIS AND DATA VISUALIZATION
8. People would rather live with a problem they cannot solve than accept a solution they cannot understand
9. The Financial Market Crisis and Risks for Latin America
10. Kaizen and Analytics: The Power of Each Employee to See Data

If you need assistance in implementing the financial services evidence-based model please contact me at Hewlett-Packard: alberto.roldan@hp.com

The Future of Business Analytics: Microsoft vs. Google

The battle over dominance of the Internet will be fought in the realm of business analytics. Microsoft and Google understand this and are investing in making their products the flagship of analytics. The reason is that analytics, like the Internet, have the potential to unleash the creative power of billions of minds.

Microsoft is depending on the combination of Excel and Analysis Services. In the business world people are used to Excel as a way explore trends. Adding the capabilities of Analysis Services allow individuals to gain insight into analytics in an incrementing manner. This is Microsoft version of kaizen analytics. Donald Farmer explains Microsoft’s vision in a recent article, Microsoft Sets Sights on Data Mining Dominance.

Google Analytics is been offered as a free software to wean people out of their Excel dependency. Google understand that Microsoft’s monopoly in Excel needs to be counteracted with a free and robust alternative. In the next few months Google will move into different industries using Google Analytics to unleash the power of the mind of billions of people. This is Google version of kaizen analytics. The users of Google product are associating the word “analytics” with Google Analytics.

Interestingly, both Google and Microsoft products have the capacity to separate or partition the data into categories or clusters. The next incremental step would be to add an automated vector analysis to clarify the driving factors in the data. I expect the competition to be head-to-head in the healthcare and Life Sciences industry in the next 12 months.

Kaizen and Analytics: The Power of Each Employee to See Data

Do companies really know how to unleash the power of employees to be the leaders in their industry? Toyota has shown a different approach to innovation, kaizen or continuous improvement approach rather than a technology leap approach. Instead of great technological breakthroughs, this approach goes for involving the entire workforce in a continuous improvement process. Hence, most of the improvements are small and process oriented (like making shelves more easily to reach) but the involvement of the entire workforce rather than a selected few keeps a vibrant and innovative enterprise. The best measurement of how this work is that the Toyota workforce gives managements one hundred times more suggestions for improvement than other auto manufacturers.

Businesses that want to improve their analytics capabilities should follow the kaizen approach and make business analytics available throughout the entire organization. It seems that in some companies analytics is only within the purview of the few like statisticians, physicians, molecular engineers, and actuaries. The concept behind this thinking is that analytics technology is expensive and difficult to interpret. This premise is no longer applicable since in the last three years mathematical science and computer technology have advanced to such a degree that this technology is now inexpensive and available to interpretation to anyone within an organization.

This technology is the work of dedicated professionals and scientist that over many years have worked to make this possible. The issue now has become whether companies want to institute a continuous improvement process that includes enterprise analytics or whether they want to leave business analytics in the hands of the few.

If you want to know how to do kaizen analytics in your company let Hewlett-Packard help you. Our Technology Services Group had over $30 billion in revenues last year, or contact me at alberto.roldan@hp.com