Model-Based and Model-Free Approaches to Autonomic Resource Allocation

A major goal of autonomic computing is to dynamically allocate computational resources so as to continually optimize high-level policy objectives. A key challenge to achieving this goal is to accurately estimate the impact of resource-level changes on application performance with respect to a Service Level Ageement (SLA). We compare two methodologies for accomplishing this: (i) developing a queuing-theoretic performance model for an application, and fitting its parameters online based on current state; (ii) using model-free reinforcement learning of resource valuation estimates based on trial-and-error learning. We describe these approaches in the context of a distributed architecture in which servers are allocated amongst multiple applications with independent time-varying loads. Each application has a local utility function, based on SLA payments as a function of relevant performance metrics. The overall system goal is to maximize the sum of local utility functions. Individual applications use one of the above methodologies to estimate resource valuations, which are then used by a resource arbiter to compute optimal allocations. We present empirical data illustrating the practicality and effectiveness of both methods in a realistic data center prototype. We highlight important tradeoffs between the methods, and point out potential benefits of a hybrid approach combining both methods.

By: Rajarshi Das; Gerald J. Tesauro; William E. Walsh

Published in: RC23802 in 2005


This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.


Questions about this service can be mailed to .