Virtual Ascetic

everything about virtualization, storage, and technology

  • Home
  • About

Your likes and shares help increase my enthusiasm to share knowledge. Please leave a comment if you like any post

STO2496 – vSphere Storage Best Practices: Next-Gen Storage Technologies

August 30, 2014 By asceticadmin

This was a panel like session that wasn’t vendor specific but broadly gave pointers on new type of arrays – like all vSAN,SDRS, VVOLs, flash arrays, datastore types, and jumbo frame usage etc. It truly lived up to its name – not just by content but also by its duration. The session ran over its scheduled duration of 1 hour and actually finished in 1.5 hours but no one was complaining since there was a lot of interesting stuff.

Presenters – Rawlinson Rivera (VMware), Chad Sakac (EMC),  Vaughn Stewart (Pure Storage)

The session kicked off by talking about enabling simplicity in the storage environment. Some key points discussed were –

1) Use large datastores

  • NFS16Tb and VMFS 64Tb
  • Backup and restore times and objectives should be considered

2) Limit use of RDMs to when required for application support

3) Use datastore clusters and SDRS

  • Match Service Levels on all datastores on each datastore cluster
  • Disable SDRS IO Metric on all flash arrays and arrays with storage tiering

4) Use automated storage array services

  • Auto tiering for performance
  • Auto grow/extend for datastores

5) Avoid Jumbo frames for iSCSI and NFS

  • Jumbo frames provide performance gains with increased complexity and the improvement in storage technology no longer requires Jumbo frames

They spoke about the forms of Hybrid Storage and categorized them based on their key functionality –

  • Hybrid arrays – Nimble, Tintri, All modern arrays
  • Host Caches – PernixData, vFRC, SanDisk
  • Converged Infrastructure – Nutanix, vSAN, Simplivity

Benchmark Principles

Good Benchmarking is NOT easyYou need to benchmark over time – most arrays have some degree of behaviour variability over time

  • You need to look at lots of hosts, VMs – not a ‘single guest’ or ‘single datastore’
  • You need to benchmark mixed loads – in practice, all forms of IO will be flinging at the persistence layer
  • If you use good tools like SLDB or IOmeter – recognize that they are still artificial workloads, and make sure to configure them to drive out a lot of different workloads
  • With modern systems (particularly AFA’s  or all flash hyper-converged), its really, REALLY hard to drive sufficient load to saturate the system. Have a lot of workload generators (generating more than 20K IOPS out of a host isn’t easy)
  • Absolute performance more often than not is not the only design consideration

virtual disk formart can be IO bottleneck

 

Storage Networking Guidance

VMFS and NFS provide similar performance

  • FC, FCoE and NFS tend to provide slightly better performance than iSCSI

Always separate guest VM traffic from storage and VMkernel network

  • Converged infrastructures require similar separation as data is written to 1+ remote nodes

Recommendation: avoid Jumbo frames as risk via human error outweighs any gain

  • Goal is to increase IO while reducing host CPU
  • Ethernet is 1500 MTU
  • Jumbo frames are often viewed as 9000 MTU (9216)
  • FCoE auto negotiates to ‘baby/ – jumbo frame of 2112 MTU (2158)
  • Jumbo frames provide modes benefits in mixed workload clouds
  • TOE adapters can produce issues uncommon in software stacks

jumbo frame performance example

 

Jumbo Frame summary – Is it worth it ?

Large environments may derive the most benefit from Jumbo frames but are also the most difficult to maintain compliance

– All the steps need to align – on every device

Mismatched settings can severely hinder performance

– A simple human error will result in significant storage issue for a large environment

Isolate jumbo frames iSCSI traffic (e.g. backup/replication) – apply CoS/QoS

Unless you have control over all host/network/storage settings, best practice is to use standard 1500 MTU

The future – Path Maximum Transmission Unit Discovery (PMTUD) – It is an IP packet (L3 routers) whereas Jumbo frames are L2 (switches)

It is part of ICMP protocol (same protocol that has Ping, Traceroute, etc) and is available on all modern Operating Systems.

The speakers then got into Data Reduction technologies – they are the new norm (specially de-duplication in arrays)

Deduplication is generally good at reducing VM Binaries (OS and application files). Deduplication block size variances can be impacted by GOS file system fragmentation

  • 512B – Pure Storage
  • 4KB – NetApp FAS
  • 4KB – XtremIO
  • 16KB – HP 3Par
There is a major operational difference between Inline (Pure Storage, XtremIO type) and post-process (NetApp FAS, EMC VNX)
– The advice they provided is that try it yourself or talk to another customer (use VMUGs) – don’t take vendor claims seriously.
Compression is generally good at reducing storage capacity of applications
– Inline compression tends to provide moderate savings (2:1 common) but there are CPU/latency tradeoffs
Post process compression tends to provide additional savings (3:1 common)
Data reduction in Virtual disks
Thin, thick, and EZ-Thick VMDKs reduce to the same size
– Differences exist between array vendors but not between various disk types
T10 UNMAP is still not here in vSphere 5.5 – in the way people ‘expect’ – UNMAP is a SCSI command that allows to reclaim space from blocks that have been deleted by virtual machine.
– It is one of the rare cases where Windows is still ahead – but only in Windows Server 2012 R2
– Manual ‘vmkfstools -k’ option for vSphere 5.1 is available. See Cormac Hogan’s blog post by clicking on this link
– Manual ‘esxcli storage vmfs UNMAP’ in vSphere 5.5 can do > 2Tb volumes (a diagram depicting UNMAP of 15TB over 2 hours was displayed)
– Not all GOS zero properly which means you may not reclaim space properly via UNMAP
An entire set of Horizon specific and Citrix specific Best Practices to follow (vSphere config and GOS config)
Rawlinson who had stepped away from the stage as Chad and Vaughn spoke about storage stuff earlier, then came on to talk about VMware vSAN Best Practices
Network Connectivity
– 10GbE Preferred Speed (previously 1Gb connectivity used to be good enough. But vSAN works best if 10GbE connectivity is available – specifically because of the volume of data that travels over the network)
– Leverage vSphere Distributed Switches (vDS) – NIOC is not commonly used in most organizations but acts like SIOC where it performs QoS for the network traffic and throttles traffic to offer the best performance. Specifically, vDS offer the best flexibility and control over network performance with the feature set that is required in Enterprise environments.
Storage Controller Queue Depth – The Queue Depth setting is something that should not be setup manually anymore unless you are observing performance issues. VMware has specifically reviewed it and officially set it up at 256. In some environments however you may have a requirement to change. Just don’t change it for the sake of setting something up manually without allowing uninterrupted operation and associated monitoring of default values.
– Queue depth support of 256 or higher
– Higher storage controller queue depth will increase
  • Performance
  • Resynchronization
  • Rebuilding Operations
  • Pass-through node preferred
Disks and Disk Groups
  • Don’t mix disk types in a cluster for predictable performance
  • More disk groups are better than one

The session finally concluded at 6:30pm and after a few hand shakes everyone was on their way. But it was completely worthwhile and goes on to show why attending VMworld offers great insights that you cannot learn in a 4 day course. The structure and content of these sessions is not limited by any way.

 

Filed Under: General Tagged With: 2014, anil sedha, benchmark, best practices, chad sakac, clusters, compression, converged, data reduction, deduplication, Discovery, EMCElect, host cache, hybrid arrays, infrastructure, inline, iometer, jumbo frames, NFS, pass through, Path Maximum, performance, PMTUD, Queue Depth, Rawlinson Rivera, RDM, reclaim disk, SDRS, storage, Transmission Unit, UNMAP, Vaughn Stewart, vDS, vExpert, VMFS, vmworld, vsphere

Let’s Get Social

  • Google+
  • Linkedin
  • Twitter


Recent Posts

  • VMware’s Photon OS 1.0 Now Available!
  • VMware’s Photon OS 1.0 Now Available!
  • VMware’s Photon OS 1.0 Now Available!
  • VMware’s Photon OS 1.0 Now Available!
  • VMware’s Photon OS 1.0 Now Available!

Archives

  • June 2016
  • May 2016
  • April 2016
  • February 2016
  • September 2015
  • August 2015
  • February 2015
  • January 2015
  • October 2014
  • September 2014
  • August 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • October 2013
  • September 2013
  • August 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • November 2012
  • October 2012