VMware announced the beta availability of vSphere 5.5 that has a really cool feature for many organizations – vFRC (vSphere Flash Read Cache). It is important to understand some key aspects about vFRC, what is has to offer, and best practices around configuring it.
When a request is sent by the guest VM to vFRC enabled VMDK, the vFRC metadata is checked to find if entire data is available in cache. If data is available the cahce is read, data is fetched from the flash device and request is serviced.
When the request sent by guest VM could not be serviced in entirety by the vFRC cache because some data may have accessed for first time or expired from cache, the entire requested data is fetched from cache and simultaneously written to the flash cache.
vFRC Cache volatility
vFRC is a volatile cache and will be destroyed at the restart of a virtual machine. It will be recreated on boot. Other scenarios include suspend-resume, vMotion of VM without migrating the cache, snapshot consolidation, snapshot revert and so on.
Default Cache Value
The cache block sizes range from 4kb to 1mb. If a cache block size is 64Kb and a 4kb read I/O request is issued by guest VM and data is not available in cache, a 4kb read is issud to VMDK. when populating cache the algorithm looks for 64kb region to place the new 4kb data. If no free space is available, a 64kb region is evicted and space is used to hold the new 4kb data. The remaining 60Kb is marked as invalid. Thus not selecting the correct cache value has great effect on performance.
Cache friendly workloads
vFRC works best for workload that frequently reads data since it fetches data and returns that when queried. Write I/Os are always serviced by the underlying storage. If the workload only accesses unique data witout repeated access of any blocks vFRC merely stores data in flash only to evict it after some time. There is a slight overhead due to adding an extra layer in the I/O path for zero benefit. Thus vFRC is best for workloads with high amount of data re-access.
vFRC Cache block size
Cache block size is the minimum granularity of cache fills and evictions. For good performance vFRC places its metadata in memory and therefore the cache block size has a direct co-relation with memory usage. The higher the cache block size the lower the footprint of metadata for indexing the blocks and therefore resulting in smaller memory footprint. Consequently a smaller cache block size consumes a bigger memory footprint. However, higher block sizes are not better in terms of performance and efficient management of cache space. As cache evicts and cache fills happen, if the cache block size is much higher than typical I/O size there might be a situation where additional amount of cached data needs to be evicted to store a small amount of new data. In plotting the average per request latency of I/O trace under different conditions the 4kb cache block size appears most beneficial.
Setting the Correct Cache Size
- During vFRC creation specify an approximate value e.g. 20% of VMDK size (remember vFRC is set per VM)
- Collect vFRC stats using esxcli in real time after the cache gets warmed up. The numBlocks field in statistics represents the total number of blocks in cache.
- When running a workload if numBlocksCurrentlyCached is less than numBlocks it means the cache is over provisioned and can be reduced
- If the two fields are equal then cache size may be correct or under-provisioned
- The EvictavgNumBlocksPerOp is average amount of data evicted. If value is very high cache size is under-provisioned but if vFlash:cacheHitPercentage is very high then cache value might be correct
- If there are more evictions increase or decrease cache size and monitor change in evictions. After increasing cache size if cache hit percentage stays same the cache size may be decreased.
Finally, remember to choose the right SSD (though cost may be a barrier) – PCIe flash works best compared to SATA/SAS SSD. Not saying though that SSD may not be a good choice – it’s also a matter of what your infrastructure can support.
With this hope you enjoy using vFRC and take advantage of it.