Software/Hardware Co-managed Cache Optimizations

Caches are traditionally managed by relatively simple hardware based replacement and placement schemes. Prior work has shown that such history based schemes are not optimal across a range of applications. Ideally, we would like to tailor the caching strategy for each individual application, but doing this in hardware alone is difficult because of the cost of added hardware complexity and the overhead of dynamic discovery of application characteristics. Instead, software can be used to determine the caching strategy by analyzing the application code and to provide hints to the hardware. In this paper, we introduce two such hardware/software assisted cache management techniques. These techniques improve upon prior work in that they better balance the flexibility in using available cache space with the hardware complexity. In our Priority-based replacement scheme, software assigns a relative priority for selected memory reference instructions, and hardware uses the priority information to make better replacement decisions. In our Virtual cache partitioning technique, software assigns selected memory reference instructions to certain virtual cache partitions to reduce conflict misses due to address based fixed mapping of data into cache arrays. For both these cache management mechanisms, we describe heuristics that the software can use to determine what cache strategy will be most effective for caching during the execution of a given application. Our preliminary experimental results show that the priority-based replacement can reduce the miss rate up to 7.1% (with software assigned priorities for approximately 1% of load/store instructions), and the virtual partitioning scheme can reduce the miss rate up to 5.6% (with software assigned partitions for less than 10 instructions in the application).

By: Rajiv Ravindran; Krishnan Kailas; Zehra Sura

Published in: RC23998 in 2006


This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.


Questions about this service can be mailed to .