Up to 50 X SAS performance gains on large data volumes using scalable parallel computing.

Abstract

Turning large volumes of historical data into valuable information to support the decision making process of management is a growing need of corporations in the 90s. However, the ability to perform fast data reduction on large data volumes is limited by traditional computer systems that can take hours of CPU time and days of elapsed time to process multi-gigabyte data sets. This paper describes how one of my customers in the health care industry addressed the volume issue in health decision support processing and improved the end user's tool sets. The paper describes the problems encountered while trying to achieve their business goals and the hardware and software solution implemented to resolve these problems. Actual performance numbers are given for Tabulate, Select, 3-way Join, Nested Select Queries, and a multiple user test. With up to 50 X faster query turn-around times on large data volumes, productivity was increased by: 1) allowing users to extract valuable information with queries that could not be run before; 2) reducing the cost of data analysis; 3) providing users with more elaborate reporting and greater versatility due to SAS and the new client/server environment; and 4) improving overall business with more timely, accurate, and cost-effective results.