Elizabeth Varki -- ITR Project Page

ITR/IM: Data Management Using Smart Storage Systems

Elizabeth Varki
Department of Computer Science
University of New Hampshire
Durham, NH 03824

Eugene Freuder (Collaborator)
Cork Constraint Computation Centre
University College Cork
Cork, Ireland

Phone: (603) 862-2319
Fax: (603) 862-3493
Email: varki@cs.unh.edu
www: www.cs.unh.edu/~varki

WWW Page
www.cs.unh.edu/~varki/me/itr.html

Project Award Information

Award Number: 0082399
Duration: 9/1/2000 - 8/31/2005
Title: Data Management Using Smart Storage Systems

Keywords
storage data management, enterprise storage systems, disk arrays, constraints programming, performance evaluation, analytical performance models.

Project Summary
Storage systems have evolved from small disk systems under control of a file server to large independent disk-array systems. These storage devices have powerful controllers capable of running complex algorithms and making decisions regarding data placement. The storage controllers also have array caches capable of holding large amounts of data for quick access. As a result, the placement and movement of storage data are largely controlled by the storage controllers, independent of file systems. The performance of these smart storage devices is definitely superior to the performance of older storage devices. However, the performance of storage devices is far lower than that promised by storage manufacturers. That is, the performance of these large storage devices should be far superior given the current state of the hardware and software technology. Since storage devices are the slowest components in a computer system, an improvement in the performance of storage devices would result in an overall improvement in the performance of the computer systems accessing these devices. This project addresses this issue and attempts to improve the performance of enterprise storage systems.

We first analyzed enterprise storage systems and evaluated the internal algorithms. Based on our analysis, we concluded that storage software has not kept pace with storage hardware. As a result, storage devices are not able to perform to their potential. Some of the problem areas this project identified are: (a) the storage caching algorithms dealing with adaptive prefetching of sequential streams; (b) the load balancing algorithms; (c) the disk scheduling algorithms for mirrored disk configurations; and (d) the workload-dependent adaptive algorithms. Based on this analysis, new algorithms to address some of these areas are developed. Detailed performance models of enterprise storage devices and techniques to quickly evaluate the performance of storage devices are also developed. These algorithms and models can be implemented in storage devices so that they can quickly adapt to workload changes and deliver the performance promised by the hardware technology.

Publications and Products

E. Varki, A. Merchant, J. Xu, X. Qiu. ``Issues and challenges in the performance analysis of real disk arrays", IEEE Transactions on Parallel and Distributed Systems, Vol 15, No. 6, pp. 559 -- 574, June 2004.
E. Varki. Response time analysis of parallel computer and storage systems, IEEE Transactions on Parallel and Distributed Systems}, 12(11):1146-61, November 2001.
E. Varki, L.W. Dowdy, T.Z. Chang. Quick performance bounding techniques for computer and storage systems with parallel resources, under review.
M. Li, E. Varki, C. Chong. "I/O Data prefetching based on sequential stream recognition", under review.
A. Villa, E.Varki. "Mirrored disk scheduling - common or distributed queue?", under revision.
E. Varki, A. Merchant, J. Xu, X. Qiu. An integrated performance model model of disk arrays, IEEE MASCOTS, 2003.
E. Varki, H. Chen, "The M/M/1 fork-join queue with variable tasks" , under review.
E. Varki, "A note on arrival instant queue lengths", under revision.
A. Gandhi, E. Varki, S. Bhatia. Reader-Writer locks for Network Attached Storage and Storage Area Network, ISCA 17th International Conference on Computers and their Applications April 2002.
E. Varki, S.X. Wang. A performance model of disk array storage systems, The Computer Measurement Group's 2000 International Conference, Orlando, Florida, December 2000.
M. Sabin, R.D. Russel, E.C. Freuder, I. Miftode. ``Using constraints technology to diagnose configuration errors in networks managed with SPECTRUM'', IEEE International Conference on Telecommunications, Bucharest, Romania, June 2001.
E.C. Freuder, B. O'Sullivan. "Generating trade offs for interactive constraint-based configuration'', CP 2001, 590-594, December 2001.
E.C. Freuder, C. Likitvivatanavong, R.J. Wallace. "Deriving explanations and implications for constraint satisfaction problems'', CP 2001, 585-589, December 2001.
S. Epstein, E.C. Freuder. "Collaborative Learning for Constraint Solving'', CP 2001, 46-60, December 2001.
Remi Coletta, Christian Bessiere, Barry O'Sullivan, Eugene C. Freuder, Sarah O'Connell, Joel Quinqueton : Semi-Automatic Modeling by Constraint Acquisition in proc. of Proceedings of CP-2003 , 2003.
Barry O'Sullivan, Sarah O'Connell, Eugene C. Freuder : Interactive Constraint Acquisition for Concurrent Engineering in proc. of of the 9th International Conference on Concurrent Enterprising - ICE-2003 , 2003.
Eugene C. Freuder, Chavalit Likitvivatanavong, Manuela Moretti, Francesca Rossi, Richard J. Wallace: Computing explanations and implications in preference-based configurators in proc. of Recent Advances in Constraints, 2003.
Richard J. Wallace, Eugene C. Freuder and Marius Minca: Possibilistic Reasoning and Privacy/Efficiency Tradeoffs in Multi-Agent Systems in MICAI 2004: Advances in Artificial Intelligence. Proc. 3rd Mexican International Conference on Artificial Intelligence, 2004.
Christian Bessiere, Remi Coletta, Eugene C. Freuder and Barry O'Sullivan: Leveraging the Learning Power of Examples in Automated Constraint Acquisition in Proceedings of CP-2004, 2004.
Tudor Hulubei and Barry O'Sullivan, "Optimal Refutations for Constraint Satisfaction Problems", Proceedings of IJCAI-2005, vol. , (2005), p. 500.
Sarah O'Connell, Barry O'Sullivan and Eugene C. Freuder, "Timid Acquisition of Constraint Satisfaction Problems", Proceedings of ACM SAC-2005, vol. , (2005), p. 225.
Barry O'Sullivan, "Introduction to the Special Issue on User-Interaction in Constratint Satisfaction", Constraints Journal, Volume 9, Number 4, vol. , (2004), p. 340.
Tudor Hulubei, Eugene C. Freuder and Richard J. Wallace, "The goldilocks problem.", Artificial Intelligence for Engineering Design, Analysis and Manufacturing, vol. , (), p. .
Sarah O'Connell, Barry O'Sullivan, Eugene C. Freuder, "Query Generation for Interactive Constraint Acquisition", proc. of Proceedings of the 4th International Conference on Recent Advances in Soft Computing (RASC-2002), vol. , (2002), p. 11.
Sarah O'Connell, Barry O'Sullivan, Eugene C. Freuder, "Strategies for Interactive Constraint Acquisition", proc. of CP-02 Workshop on User-Interaction in Constraint Satisfaction, vol. , (2002), p. 100.

Project Contributions

(a) The detailed storage system model that can be used to evaluate the performance of 'real' storage devices.

(b) The queueing techniques developed for parallel systems.

(c) The identification of areas of storage devices that under-perform and reasons for this poor performance. Based on this analysis, we developed new algorithms for some of the areas we identified.

(d) the educational and research development of students who participated in this work and/or took related courses.

Area Background
Storage systems represent a growing market. In recent years there has been an explosion of applications (which include scientific ``grand-challenge'' programs, multi-media systems, and large transaction-based information systems) with varying performance needs that use enormous amounts of data. These applications have high Quality of Service (QoS) requirements from storage devices, irrespective of the location of the data and its users and the problems that could interfere with data access. It is very difficult to coordinate the storage, network, and computation resources required for these heterogeneous applications. A solution to this problem of storage data management is to have the storage system manage its data. This approach was first proposed by Gelb [Gelb89] who referred to it as system-managed storage. Attribute-managed storage [Borowsky98], a formalization of the system-managed approach, is currently being studied in Hewlett-Packard Storage Labs. In addition, companies like EMC, Veritas, and IBM and research labs like the NASD Lab in CMU are investigating the design and development of smarter storage systems.

Area References

Acknowledgement Of Support And Disclaimer

"This material is based upon work supported by the National Science Foundation under Grant No. ITR/IM 0082399. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation."