My team at Dell is working feverishly to bring the new Dell and Nutanix partnership to fruition through new and exciting hardware platforms and solutions. Part of the process is learning and playing…a LOT.  A few weeks ago there was a very bad storm in Austin that knocked my team’s datacenter offline. Our four-node Nutanix cluster, along with everything else, went down hard. Fortunately three nodes came back up without issue, the 4th only partially. At this point we’re just doing POC work and functionality testing so there is no alerting configured. We had a failed Controller VM (CVM) on one of our nodes and didn’t know it. This is probably an extraordinary situation which is why I want to document it.

This cluster is running Server 2012 R2 Hyper-V with an SMB3 storage pool. The architecture is simple, all nodes and their disks participate in two layers of clustering: one private cluster controlled by and run for Nutanix, the other a failover cluster for the Hyper-V hypervisors. SMB resources cannot be owned by a cluster in Hyper-V anyway as there are no disks to add to the cluster, this is simply a namespace that you utilize via UNC pathing. The two clusters operate completely independent from one another. The physical disks are owned by the Nutanix CVMs and are completely obscured from Hyper-V. So even though our 4th node was fine from a Hyper-V perspective, able to run and host VMs, the CVM living on that node was caput as were its local disks, from a Nutanix cluster perspective.

In Prism, it was plain to see that this node was having trouble and the easy remediation steps didn’t work. Rebooting the CVM, rebooting the host, enabling the metadata store, all had no effect, neither did trying to start the cluster services manually. I removed the host from the cluster via Prism hoping I would be easily able to add it back. 

Once the disks had been completely removed in Prism, the remaining nodes could see that this CVM and it’s physical resources were gone. Unfortunately, I was unable to expand the cluster and easily add this node back into the mix. I could not open the prism console nor the cluster init page on this CVM. To clean up the metadata and “factory reset” this CVM I ran the following command in the CVM’s console:

cluster –f –s 127.0.0.1 destroy

Once complete, I tried to expand the cluster again in Prism and this time the CVM was discovered. Woot!

Cluster expanded successfully and all is well in the world again, right? Not quite. The disks of the 4th node never joined the pool even though Prism now saw four nodes in my cluster. Hmm. Running a “cluster status” on the CVM revealed this:

We’re not out of the woods yet. Enter the Nutanix CLI to check the disk tombstone entries. Both of my SSDs and three of my SATA disks had been tombstoned as part of the previous configuration, so were being prevented from being assigned now.

One by one these entries needed to be removed so that the CVM and these disk resources could again be free to join the pool.

Now, perform a “cluster start” to start the services on this CVM and voila, back in business.

Check the current activities in Prism which reflect the node getting added (for real this time). 

All disk resources are now present and accounted for.

Pretty simple fix for what should be a fairly irregular situation. If you ever run into this issue this is what solved it for us. Big shout to Mike Tao and Rob Tribe at Nutanix for the assist!

HTPC build for Plex three part series:

1. Hardware build and Storage Spaces
2. Plex server and clients
3. Performance

I’ve had a single disk Home Theater PC (HTPC) running Plex for a while now but with my ReadyNas nearing the end of its useful life, I decided to kill two birds with one stone: build an updated HTPC with drive redundancy to also take over the functions of my aging NAS. My HTPC stays on 24/7 anyway so this makes perfect sense. My design goals for this were to build a small, quiet, low power, data redundant and sufficiently performing box that I can stuff into my TV cabinet. The extent of the duties for this HTPC will be video streaming and file sharing/ backup, no gaming. This series details my build, Plex server, Plex clients, Windows Storage Spaces, as well as the performance of the storage subsystem.

Performance

In these performance tests I want to focus on the main activities my HTPC will be undertaking: file read/ write operations and streaming. I tested using both Parity and Mirror Spaces to demonstrate the capabilities of each.

WD Red Baseline (Control)

First to set a baseline for WD Red 3TB HDD, here is the observed write performance with a single 40GB file write operation (from C:\ to Z:\). The copy took around 4.5 minutes and the drive topped out around 145 IOPS. Subsequent read performance tests top out around the same level. Considering these disks have been reported to spin at 5400 RPMs, frankly that’s really not too bad.

Windows reported ~142MB/s transfer speed:

These drives produce absolutely ZERO audible noise. No clicking, no head seeking noise, no rotational noise…nothing. With the cover off of this box, all that is audible is the 3 spinning fans (case, CPU, PSU). Under the TV cabinet, my build is dead silent.

File copy between Mirror and Parity Space

Test constants:

C:\ = SSD
N:\ = Mirror space
M:\ = Parity space

This test demonstrates the performance capabilities of a Parity Space using the WD Red disks. Disk performance copying multiple files from a mirror space (N:\) to a parity space (M:\) was fairly poor, averaging 33 write IOPS on the parity space. Observed read performance on the source 2-way mirror space was low at 25 IOPS on average, but that’s all it was being asked to do. This is somewhat expected however, as one of the caveats with parity resiliency is reduced write performance.

Once again, the Windows file copy dialog tracks very closely to the observed maximum write performance in Perfmon.

Power consumption of the system under test varied between 50 to 75 watts. Very good!!

Write performance of Mirror and Parity Spaces:

Test constants:

C:\ = SSD
D:\ = Mirror space
M:\ = Parity space

Here you can clearly see the write performance disparity between a two-way mirror and parity space. Copying a 4.5GB file from SSD (with very high read performance) to my two-way mirror space yielded very reasonable write performance according to the Windows file copy dialog. The overhead of the mirror operation appears to be costing around 20 IOPS.

The write activity of the physical disks participating in the mirror.

That same 4.5GB file written from SSD to my parity space (M:\) yielded significantly lower results (almost 3.5x lower):

Here is a more detailed view of the write performance against the parity space:

Read performance of Mirror and Parity Spaces:

Test constants:

C:\ = SSD
D:\ = Mirror space
M:\ = Parity space

Copying the same 4.5GB file from each space back to the SSD yielded better results overall. The read performance of the parity space during this operation is better but still below what the mirror space or even native drive are capable of:

Copying the file from the mirror space to the SSD was higher performing than the parity space and on par with native disk read performance, but the gap isn’t as wide here.

Plex Streaming Performance: Single File

Test constants:

-Physical disk 1 and 2 comprise a mirror Space
-Plex clients in use: Roku
-Media in use: single file, highly compatible, 1080p

Streaming being a highly sequential mostly read operation, there isn’t a great tax put upon the system resources. Multiple streams and transcoding complicate this a bit. The following chart shows the performance of a single video stream from my HTPC using a mirror Space to my Roku client. Interesting to note that the C:\ drive is busier than both disks in the mirror space, but all reads are sourced from a single disk.

Plex Streaming Performance: Single File, skipping around

Test constants:

-Physical disk 1, 2 and 3 comprise a mirror Space
-Plex clients in use: Chromecast
-Media in use: single file, highly compatible, 720p

Trying to force spaces to utilize all disks within a vDisk, I manually skipped around four times within a sample media file. Here you can see that with every color change within the performance chart is where I jumped to a different time within the media file. Almost every time I did this, Spaces offered up a different disk to be read from. The one exception is during the period between 7:40 and 7:43 where I performed a jump but the same disk was used. Otherwise, all three disks in  my mirror Space were used to read the file.

Plex Streaming Performance: Multiple Streams from One File

Test constants:

-Physical disk 1 and 2 comprise a mirror Space
-Plex clients in use: Roku, Chromecast, PS3 (DLNA)
-Media in use: single file, highly compatible, 1080p

To satisfy my curiosity, regardless of how infrequently this situation might actually occur, I wanted to see what the impacts were if I kicked off multiple streams of the the same file. I chose a mostly compatible file so we get the benefits of Direct Play here. Three simultaneous streams requiring transcoding would probably not work so well. There is a CPU spike when the third stream is launched and minor spikes on disk 2 for each stream event. You’ll note here that all reads are coming from disk 2. Looking at each TV screen individually and subjectively, the playback was completely normal.

Plex Streaming Performance: Multiple Streams from One File, 3 Disk Mirror

Test constants:

-Physical disk 1, 2 and 3 comprise a mirror Space
-Plex clients in use: Roku, Chromecast, PS3 (DLNA)
-Media in use: single file, highly compatible, 1080p

Just to see if there is any impact to performance, I ran this same test again but with three physical disks in my mirror Space. I would have expected my reads to be coming from at least two disks but as evident here, disk 1 is doing almost all of the work. Interesting to note that overall spikes are much lower now with read performance hovering around 1 IOPS. The takeaway here is that this third disk in my two-way mirror Space doesn’t add much from a performance perspective.

Plex Streaming Performance: Multiple Streams from Multiple Files

Test constants:

The following chart shows the effects when additional streams are added using three different media files, one requiring transcoding. You can see that system resource usage is relatively low with the first two streams, things change dramatically with the introduction of the third file. Transcoding means heavy CPU usage as evident here. Disk activity with 3 different files being accessed simultaneously jumps up quite a bit too. 

Zooming in here looking at the disk IO, you can see once again that almost all read requests are satisfied from a single disk. Hmm, where’s my striping performance??!

Here is the Plex transcoder in action:

Network Performance

Network performance of the Plex server with a single HD media stream is very reasonable. There are spikes north of 11Mbps but for most of the duration the  transmitting bandwidth stayed around 2Mbps.

Conclusions:

• The WD Red HDDs are a very quiet, very low power option that yield decent disk performance. Performance is more than acceptable for an HTPC that doubles as a NAS.
• Two-way mirror Spaces are the highest performing option in Storage Spaces but come with the highest disk cost as well (2x). Read performance is on par to that of a native disk with Spaces adding no discernable overhead, writes cost ~20 IOPS in overhead. If you have HDD capacity to spare, this is the resiliency option you want!
  • While writes clearly hit all disks in a mirror Space, observed read operations from streams are satisfied from a single disk in the Spaces pool. Jumping around within a media file does force reads from all disks in a Space.
  • 3 disks in a Storage Space really doesn’t buy you much other than additional capacity. With a 1-column two-way mirror you still only have two copies of data, on two physical disks. Read and write performance is not increased here.
• Parity spaces will work fine for Plex and video streaming in general, as video streaming is sequential and read performance is acceptable. Parity spaces suffer massively in reduced write performance so copying files to or moving files around on your HTPC will be much slower. The trade off for poorer write performance in a parity space is more overall usable capacity (1.5x consumed in a parity space vs 2x in a mirror space).
• Plex makes excellent use of system resources and can handle just about anything you can throw at it. The Plex server doesn’t even break a sweat with highly compatible media types, even with multiple streams. Transcoding is CPU intensive and should work fine for a single stream, multiple streams may tax your system and degrade the viewing experience.

This post originated at Exit | the | Fast | Lane – weestro.blogspot.com

HTPC build for Plex three part series:

1. Hardware build and Storage Spaces
2. Plex server and clients (You are Here)
3. Performance

I’ve had a single disk Home Theater PC (HTPC) running Plex for a while now but with my ReadyNas nearing the end of its useful life, I decided to kill two birds with one stone: build an updated HTPC with drive redundancy to also take over the functions of my aging NAS. My HTPC stays on 24/7 anyway so this makes perfect sense. My design goals for this were to build a small, quiet, low power, data redundant and sufficiently performing box that I can stuff into my TV cabinet. The extent of the duties for this HTPC will be video streaming and file sharing/ backup, no gaming. This series details my build, Plex server, Plex clients, Windows Storage Spaces, as well as the performance of the storage subsystem.

Plex

So what is Plex and why do we need it? Plex is a very simple, very powerful media center server that can stream videos, music and photos to almost any device…most importantly TVs. Plex can be installed on many NAS devices as well as Windows/Mac/Linux based PCs. Without Plex, DLNA is available to watch supported content from network accessible locations and works fairly well most of the time, but this is simply a list of content. Plex can also talk DLNA but the media server greatly enhances the overall experience by providing art, posters, synopsis, which media has been watched/ which hasn’t, the ability to resume where you left off, remote streaming and transcoding. Plex also supports many file containers (AVI, MKV, MP4, etc), codecs and subtitles. I personally used the DLNA method for years before seeing the light on Plex and no way would I go back. I love Plex and you will too!!

First, create an account on http://plex.tv, you’ll need this later for remote access and it will make management much simpler. Setup is simple: install the Plex media server on the supported platform of your choosing, open the Plex console in a web browser and point Plex at the locations of your media by adding libraries. Media is best categorized by type which will make browsing, searching and updating content much easier. Plex makes this process very easy for you:

I use the following categories and will add one for photos soon:

Plex will then automatically connect to a variety of online sources, such as Freebase or The Movie Database, to download media information and art.  Once complete all of you media will be organized and accessible in the Plex console with full posters and even background music, if you want it.

Plex content can be browsed via normal folder structures if you like or you can use the Plex Dashboard where you will find “On Deck” and “Recently Added”. The latter should be self explanatory, On Deck is where you will find movies that you started and stopped or the next episode of a TV show in a series. This is Plex helping to keep your place in your media universe and it is incredibly useful. 

Plex provides three primary methods for playing content: Direct Play, Direct Stream and Transcoding. Bottom line is that Plex will play your media one way or another, you don’t have to worry about it, but this is how it does it.

• Direct Play is the best and exists in the ideal scenario. 100% file compatibility with very little resources consumed on the media server itself. The file is sent to the Plex client as-is, easy breezy.
• Direct Stream is used when a media file is mostly compatible but some element within is not and requires transcoding. The example Plex uses is a media file where compatible audio/video codecs are streamed direct and the non-compatible container is transcoded.
• Transcoding is used when either the audio or video format isn’t compatible and has to be converted into a compatible format. Audio transcoding is not very CPU intensive but video transcoding is.

Recommended Tweaks

Overall, you could change almost nothing post install and Plex will work fine right out of the box. Here are a few tweaks that I recommend to enhance your experience.

In the server settings, click Connect on the left, make sure to log into Plex (your online account) as this will allow you to connect to and stream media to any device anywhere you are. This is honestly one of the coolest features of Plex and you do not need a Plex Pass to do this! Once signed in on your server, you can now connect to your Plex environment from any smart phone, tablet or PC anywhere you have internet connectivity by using the IOS/ Android Plex app or by logging into Plex.tv. Traveling? As long as you have internet connectivity, you can browse and watch anything from your library. No USB thumb drives required!

To maximize ease of use when you add content, make sure the following settings are enabled. This will ensure that Plex will automatically scan your library when files are added vs having to run a manual or timed scan. Plex will see that you’ve added a file and will automatically add it to the library for playback and download related art.

Under the Web settings, click Player on the left, you can control the streaming quality based on where you will be streaming to. 1080p for local streaming and a lower setting for remote streaming, for example. I have found that setting the local quality to 4Mbps/ 720p greatly increases local streaming responsiveness and makes it easier to skip around in a given media file.

Further customization at this point is highly preferential and you’ll figure it out by clicking through the available options. Plex has made this all VERY easy. In Windows, Plex runs as a process launched by an interactive user so does require that a user account is logged into the desktop. There is no Windows service offering yet but there are ways to work around this. I have an account set to login automatically after a reboot then the screen locks after 1 minute for security. In lieu of a Windows service starting with the OS, this works fine.

Plex Clients

With the HTPC built and Plex installed, let’s talk about client connectivity. There are a number of ways to go here which is another benefit of Plex: extreme flexibility. The methods below are my top 3 in order by preference.

Smart TV app

If you own an internet connected Samsung Smart TV then you already have access to the best free Plex client available. Nothing is required externally, just install the Plex app from the Samsung app store on your TV, launch and it should find your Plex server on the network. Your TV remote will navigate the Plex menus and this will provide the richest experience you can get while navigating Plex on your TV screen. Sadly, if you own a Panasonic TV you will have to use an external client. There is no and will be no native Plex app in the foreseeable future.

Google Chromecast

Chromecast is a very simple and very cheap option that I personally use on my Panasonic plasma. $35 gets you an HDMI-connected dongle that simply casts or displays content on your TV via wifi. This is an important distinction to make when considering the Chromecast. There are no on-screen menus to navigate and there is no integration with your all-in-one remote. Your PC, tablet, or phone is your remote and where you select the content you want to display on your TV. This does require that you purchase the Plex app for IOS/ Android ($5) or if you choose to pay for PlexPass, you’ll get the app for free. Chromecast for Plex works great, the downside is the other services currently supported are a bit limited compared to Roku and the like (if you care). Chromecast is powered via a USB port on your TV or from a wall adapter.

Chromecast setup is very simple and pretty neat in what they do. Initially the Chromecast turns itself into a wifi Access Point (AP) by broadcasting a SSID that you connect to for configuration using the Chromecast app on your device. Once the local wifi network SSID and password are selected, Chromecast will disable its own AP and connect to yours.

Once Chromecast is configured, launch the Plex app on the device of your choosing and navigate to the content you want to watch. Connect Plex to the Chromecast by pushing the “Cast” button in the top right corner and select the Chromecast, this will also automatically change the input on your TV to the correct HDMI slot. Press play on the chosen content. This method is definitely different than the more traditional on-screen navigation approach but having used it for going on a year now, I really like the experience this solution provides. Your more technically challenged family members may not like or fully understand this method without some training.

Roku Streamstick

Roku is a favorite of many home theater aficionados and provides several client options with varying price points. The Roku Streamstick is the Roku answer to the minimalist HDMI dongle but with the majority of the value that Roku provides. I use this client on my older Samsung plasma that doesn’t have built-in wifi support. Roku Streamstick is powered via a USB port on your TV or from a wall adapter. Setup is simple and handled via the specialized Roku remote included. Connect Roku to your home wifi and you are good to go.

Roku uses the more traditional on-screen navigation approach and has an impressive catalog of connectable services you can use. One of the neat things Roku provides is an impressive search function that gives you multiple options as to where you can find/ buy/ play the content you’re looking for. Roku works using a concept of “channels” which are really apps that you add to your client. Plex, Netflix, Amazon, etc. Plex is one of those channels and costs $5 just like the IOS/ Android version. PlexPass members get this app for free as well. Unlike the Smart TV or Chromecast methods, Plex on Roku is a very minimalist affair that provides a very basic visual experience.

While Roku does ultimately provide the essentials for Plex, it is my least preferred method and comes with the following caveats:

• Separate non-IR Roku remote required to control on-screen navigation. No integration into traditional IR all-in-one remotes possible.
• Very slow boot up compared to Chromecast and takes a very long time to get to a point you can actually use it.
• $50 vs $35 for the Chromecast
• Least rich experience of the 3 options with a very basic user interface.

Please also see part one of this series which details the hardware build and part three which discusses performance.

References:

https://plex.tv/

Converting ISO, Video_TS and other disk image formats

HTPC build for Plex three part series:

1. Hardware build and Storage Spaces
2. Plex server and clients
3. Performance

I’ve had a single disk Home Theater PC (HTPC) running Plex for a while now but with my ReadyNas nearing the end of its useful life, I decided to kill two birds with one stone: build an updated HTPC with drive redundancy to also take over the functions of my aging NAS. My HTPC stays on 24/7 anyway so this makes perfect sense. My design goals for this were to build a small, quiet, low power, data redundant and sufficiently performing box that I can stuff into my TV cabinet. The extent of the duties for this HTPC will be video streaming and file sharing/ backup, no gaming. This post details my build, Plex server, Plex clients, Windows Storage Spaces, as well as the performance of the storage subsystem.

HTPC Build

Trying to keep my costs in line while building in performance where it mattered, I decided to go with an AMD-based mini-ITX (m-ITX) form factor system running Windows 8.1 x64. The motherboard and CPU were sourced a year ago but still provide more than adequate performance for my purposes. I went a little higher on the CPU than I really needed to but since video transcoding is very CPU intensive, more horsepower there is certainly not wasted. All parts were sourced from Amazon, who has officially conquered the PC parts space, New Egg can no longer compete (price or return policy). A decent 2-bay ReadyNas or Synology NAS will set you back at least this much, plus this way I get a full featured box without compromises. 

• CPU – AMD A4-5300 FM2, dual core, 3.4GHz, 65w, $50
• MOBO – MSI FM2-A75IA-E53, $100
• Case – Cooler Master Elite 130, $50
• RAM – Corsair 8GB DDR3/1333 Memory, $60
• PSU – Silverstone SFX 450w, $70
• Disk – 1 x Crucial M500 120GB SSD (OS), $72
          – 3 x WD Red 3TB (spaces), $122 each

Total: $768

This A-series 65w AMD APU (Trinity) includes a robust dual-core clocked at 3.6GHz with on-die Radeon HD 7480D for graphics. The RAM isn’t anything special, just trusty Corsair 4GB DIMMs clocked at 1333MHz. The MSI main board provides all the connections I need and pairs nicely with the Cooler Master case. 4 x SATA ports, 2 x DIMM slots, and a PCIe slot should I ever choose to drop in a full length GPU.

I had originally spec’d this build using the WD GP-AV disks which are purpose built for DVRs, security systems and 24/7 streaming operation. This was the disk I used originally a year ago which has served me faithfully and quietly. Online benchmarks show this drive to perform higher than the Green and Red drives from WD. Due to a parts sourcing issue I was forced to look for alternatives and landed on the WD Red disks, which are actually a bit cheaper. Low noise, low power consumption, built for 24/7 NAS duty… these drives are very favorably reviewed. The WD Reds are no where near the highest performing HDDs on the market, with a rotational speed of ~5400 RPM, but for the purposes of an HTPC/ NAS, their performance is perfectly sufficient.

Quality SSDs are cheaper than ever now and make a great choice for your OS disk when you separate your data (as you always should).  These disks are capable of thousands of IOPS and make install, boot up and OS operations lightning quick.

For power I once again turned to Silverstone who have a 450w 80 PLUS bronze efficient unit in SFX form factor. Plenty of power with a very minimal foot-print which is ideal when working with m-ITX. This non-modular PSU has only 3 SATA connectors on a single string so I used a Molex to SATA power connector for my 4th drive. This PSU provides plenty of juice for these power sipping components and doesn’t block the CPU fan.

Finding the “perfect” case for whatever project can prove challenging. For this project I wanted a small m-ITX case that can house 3 x 3.5” HDDs + a 2.5” SSD and Cooler Master had the answer. Well ventilated with plenty of working room, support for a full length GPU and full size PSU. You can mount a BD-ROM if you like or use that space for a 3.5” HDD. The finish is matte black inside and out with all rounded corners and edges. My only gripe is the included front case fan which, although extremely quiet, has no protection grill on its backside. You have to be very careful with you wires as there is nothing to stop them from direct contact with the fan blades. With the one-piece cover removed there is loads of working room in this case. With four SATA connections, cabling gets a bit tight on the rear side but it’s completely manageable.

With the Elite 130 case loaded you can see the three WD Red drives, one on the floor, one on the wall and one on the top in the 5.25 bay. The SSD is mounted below the 5.25 bay, you can see the grey-beige rear of its case.

The SFX PSU sticks out the back of the case a few inches which provides even greater clearance for the CPU fan. With this case it did require using the ATX bracket to fit perfectly. Both the CPU and PSU fans are completely unobstructed with fresh air brought in by the 120mm front fan. This case comes with an 80x15mm fan mounted on the side by the SATA ports. I found this fan to be noisy and in the way, so removed it.

When all was said and done, power consumption for this build is at 29w idle and up around 75w under load. Awesome!!! See the performance section at the bottom for more details.

Storage Spaces

First debuted in Windows 8/ Server 2012, Storage Spaces provides a new very interesting method for controlling storage. Part of the “software defined” market segment, Spaces supplants traditional software and hardware RAID by providing the same data protection capability with a ton of additional flexibility. The biggest value propositions for Spaces in Windows 8.1 is the ability to thin provision (create drives with more space than you actually have or need), use disparate drive media (create a pool from several different disks, internal and external USB) and choose from a few different resiliency methods to control performance or provide greater redundancy. Server 2012 R2 adds a few additional features to Spaces, namely the ability to provide performance tiering between SSDs and HDDs and post-process deduplication which can have a dramatic reduction on consumed capacity. At the end of this exercise I will have two Spaces called Docs and Media both with 2-way mirror resiliency.

Setting up Spaces in Windows 8.1 is a trivial affair once the drives have been physical added to your build. One of the only restrictions is that the disk where the OS is installed cannot be used in a Spaces pool. First open the Spaces management console by pressing Windows Key + Q and typing spaces.

Click create a new pool and select the unformatted disks you want to include. Additional disks can be easily added later.

Storage Spaces (aka virtual disks) can now be created within the physical disk pool and will be presented back to the OS as logical drives with drive letters. Give the new Space a name (for me I have functionally separated Spaces for Docs and Media), choose a drive letter and select a file system. Resiliency type will open an additional file system option called ReFS for mirrored Spaces which is the recommended file system for these Spaces types. NTFS will be the only option for all others. ReFS builds on NFTS by providing greater data integrity including self healing, prioritized data availability, high scalability and app compatibility.  Here is a quick breakdown of the available resiliency types:

• Simple (no resiliency) – just like it says, only one copy of your data so offers no protection. This is akin to RAID0 and can be useful if you simply want to thin provisioning a large logical drive with no data protection.
• Two-way mirror – writes two copies of your data on different physical disks thus protecting from a single disk failure. This requires a minimum of two disks.
• Three-way mirror – writes three copies of your data on different physical disks thus protecting you from two simultaneous disk failures. This requires a minimum of five disks.
• Parity – writes your data with additional parity information for recovery, similar to RAID5. This protects from a single disk failure and requires a minimum of three drives. Parity Spaces will give you more available capacity but at the cost of reduced IO performance. See the performance section at the end for more details.

The images below show the capacity impact of resiliency using a two-way mirror vs parity. The two-way mirror mode will consume 2x of the stated capacity for every file stored (50% reduction of available capacity). All spaces can be thin provisioned and adjusted later if need be. As you can see below, a 2TB logical Space requires 4TB of actual storage should it fill completely.

Parity will consume 1.5x of the stated capacity for every file stored. Based on the 2TB logical Space shown below, if that volume were to fill completely it would consume 3TB of total available capacity.

Once a Space has been created only the size and drive letter can be changed. If you wish to change resiliency type for a space, you’ll have to create a new Space and migrate your data to it.

New drives can be added later to an existing pool very easily and with no down time. Note that adding a drive to an existing Storage Space will destroy any data currently on that drive.

Once configured, you will have a view of your logical Spaces, consumption stats as well as the physical disks backing them in the pool. Here you can see my three disk pool and two Spaces, each with a two-way mirror. Notice that the actual data consumed on the physical disks is evenly spread across all three. Having tried both Parity and two-way mirror resiliency, I decided to take the capacity hit and use the mirror method on both since it performs much better than Parity. Once finalized, I moved all of my pictures and videos into the new M drive.

This is reflected in Windows File Explorer with the two new logical drives and their individual space consumption levels.

Slabs and Columns

Underneath it all Microsoft has a few interesting concepts, some customizable which can be used to achieve certain results. I’ve linked to many good information sources in my references section at the bottom if you’d like more info on these concepts. Data that is mirrored is done so in 256MB slabs which is what is actually copied to each physical disk in a two-way mirror. Half of each slab is allocated to two separate disks. In the event of a disk failure, Spaces will self heal by identifying all affected slabs that were on the failed disk and reallocate them to any suitable remaining disk in the pool. You can take a peek under the covers if you’re curious by running the optimize-volume –verbose command in PowerShell:

Another important concept in Storage Spaces is that of columns which define the number of underlying physical disks across which one stripe of data for a virtual disk is written. This takes a minute to wrap the head around. By default, Spaces creates each virtual disk with one column when there are two disks present. So one column means one stripe on two disks. The interleave value dictates how much data is written to a single column per stripe, by default 256KB. For larger JBODs with dozens of disks the column count can go as high as eight. In my case only having three disks, one column is fine. Two columns makes sense if you have four disks, for example. Generally speaking, more columns means higher performance. The downside of more columns is that this controls how you will scale out later. A two column virtual disk will require you to expand your storage pool by two physical disks at a time. Creating a virtual disk with more columns or a different interleave is done so in PowerShell using the New-VirtualDisk cmdlet.  You can use the get-virtual disk cmdlet to view the lay of the land for your Spaces:

In the end I have met all my initial goals around cost, size, noise, performance and power consumption. I’m very pleased with this mix of hardware, especially my alternate WD Red HDD choice. Spaces provides a robust and flexible subsystem which is a great alternative to legacy hardware/ software RAID. For more information about the inner workings of Spaces and additional design considerations please see the references section below. Please see parts two and three of this series for more information on Plex and the performance of this build.

References:

http://social.technet.microsoft.com/wiki/contents/articles/15200.storage-spaces-designing-for-performance.aspx

http://technet.microsoft.com/en-us/library/hh831739.aspx

http://blogs.msdn.com/b/b8/archive/2012/01/05/virtualizing-storage-for-scale-resiliency-and-efficiency.aspx

ReFS

Columns in Storage Spaces