There are three reasons why it's hard to feed data to spend analysis systems:
Data feeds from different sources have incompatible formats;
Data feeds consist of files with imperfect relations
between them that cause transactions to be omitted from the dataset;
Data feeds need translation and repair before loading.
Traditionally, these problems are tackled with Microsoft
Access and other database and data manipulation technologies.
BIQ allows business users to load their own data
and build their own datasets, without "expert" assistance.
Unfortunately, business users usually lack sufficient skill
with such technologies to enable them to succeed, even if
they had the time to try (which they don't). That means
specially-trained resources are required in order to coerce
input data into a form that is compatible with the system,
and it typically means long lead times as those resources
struggle with tools that aren't really suited to the task.
This requirement for "data experts" means that traditional spend
analysis products are always sold "on a tether" — that is to say,
they are useless unless services personnel are packaged right
along with the product. So, vendor personnel typically perform
refreshes of data; they build and maintain datasets; and they
get involved whenever there is a change of any kind.
But the problems don't stop there. Traditional spend analysis
systems face several more hurdles even after raw data are loaded:
The "dataset build" process is complex. Creating new dimensions
and defining hierarchies requires the same "experts" that were needed
in the data loading process;
The build process takes many hours, and errors created by
imperfect relations may cause multiple re-starts of the many-hour process;
Once built, the dataset must be "published" to a server,
a process that requires IT personnel to accomplish.
We felt there had to be a better way.
The BIQ Difference
We designed BIQ so that business users can load their own data
and build their own datasets. Dozens of BIQ business users
accomplish this magic daily. How do we know this? Because
BIQ doesn't offer any "services on a tether." If you asked
us for them, we'd stare at you blankly. In fact, BIQ is so
easy to use that many of our users never took advantage of a
BIQ training class at all — they just picked up the manual
and taught themselves.
The Eureka moment, for us, was when we stopped thinking about
data in "database" terms. Real-world data is messy. So, BIQ
can easily transform data from one format to another. Joins
are imperfect. So, BIQ is very tolerant of poor joins.
Transactions must never be omitted. So, BIQ never omits
transactions under any circumstance (which gladdens every
accountant's heart), it simply groups them intelligently
under an "Other" category. And (you've probably already guessed
this) -- BIQ doesn't use traditional database technology to build its cubes.
Our build process is very fast — we average 2-3 minutes
for each 1M transactions on an ordinary desktop PC. Many changes to BIQ datasets take
place instantaneously (such as hierarchy and rules changes) —
as opposed to the multi-hour re-publish rigmarole that legacy
systems require, whenever the slightest alteration is made.
With BIQ, there isn't any "re-publish" or "cooking" step —
BIQ datasets are always good to go.
Defining new dimensions and hierarchies is also accomplished
by business users. BIQ flattens related files automatically
into a big, wide "virtual" record. Any column of this record
can be defined as a dimension or as a measure, and hierarchical
relationships can be established between columns — or externally,
within index files (BIQ already understands the most common types of computer-generated
hierarchy indexes). New columns can be created within the virtual
record, as functions of other columns. And it's all done with
point-and-click tools that are easy to use and easy to understand.
