Business Innovation Homepage > Information Management

An enterprise data warehouse can be a treasure trove of information for organizations, a place from which workers can access historical data as well as the most recent information about customers, products, market trends and technology developments.

But creating and maintaining an effective data warehouse can be a significant challenge for organizations. If there’s too much data in the warehouse, if the data is not kept up to date and cleansed to ensure accuracy and consistency, and if the tools used to access the data warehouse — such as business intelligence applications — are not well integrated, the value of the warehouse is limited.

An organization can take several steps to improve the process of data warehouse development and maintenance, says David Hatch, research director, business intelligence, at research firm Aberdeen Group.

For one thing, the organization should understand end users’ needs early in the implementation process. “Too many data warehouses are built as an IT initiative without first considering exactly how the end product will be used, and by which areas of the company,” Hatch says. “It is not always a best practice to stuff all of the data one can source within the organization into a data warehouse. While this might make it easier to find information (from one location as opposed to several), it will also make the data warehouse a very large, resource-intensive, complex asset.”

Once end user needs have been established, and the data residing in various systems has been identified as being required to fulfill those needs, organizations can ease the complexity and resource drain of a data warehousing project by automating the data collection (integration, cleansing, loading) steps used to populate the warehouse on an ongoing basis, Hatch says.

“Data warehouses are living things. An organization must have a method for incrementally adding new information — and archiving outdated information — to maintain the value of the resource,” Hatch says. Ongoing maintenance and support are among the pain points organizations identify in Aberdeen’s research, he says, and the degree to which automation is used to ease the tasks surrounding data collection dictates whether a company is taking the best approach to data warehousing.

Hatch also recommends that organizations establish query optimization and prioritization rules for data access. “One of the problems organizations face once a data warehouse has been deployed is the deterioration of system performance due to query load, and the complexity — and often poor design — of growing numbers of queries,” he says. “There are tools that are designed to monitor and prioritize queries in order to alleviate this problem.”

Companies should also consider investing in technologies that will ensure system responsiveness for users. “Otherwise, the efficiency and productivity improvements that were intended with the initial investment in the data warehouse may be lost due to its performance,” Hatch says.

Finally, enterprises should measure and plan for data and end-user growth. “Organizations that do not have a plan for dealing with data growth will have a difficult time managing their data warehouse on an ongoing basis,” Hatch says. “Data growth is being accelerated by the influx of new data streams that companies are beginning to leverage.” As more data becomes available, he says, companies must have a formal plan for whether to include, or exclude, specific data sets into the warehouse.

Click here for more Information Management articles