Previously I took at look at a small three disk Storage Spaces configuration on Windows 8.1 for my Plex media server build. Looking at the performance of the individual disks, I grew curious so set out to explore Spaces performance in Server 2012 R2 with one extra disk. Windows 8.1 and Server 2012 R2 are cut from the same cloth with the Server product adding performance tiering, deduplication and means for larger scale.  At its core, Storage Spaces is essentially the same between the products and accomplishes a singular goal: take several raw disks and create a pool with adjustable performance and resiliency characteristics. The execution of this however, is very different between the products.

For my test bed I used a single Dell PowerEdge R610 which has 6 local 15K SAS disks attached to a PERC H700 RAID controller with 1GB battery-backed cache. Two disks in RAID 1 for the OS (Windows Server 2012 R2) and four disks left for Spaces. Spaces is a very capable storage aggregation tool that can be used with large external JBODs and clustered across physical server nodes, but for this exercise I want to look only at local disks and compare performance between three and four disk configurations.

Questions I’m looking to answer:

• Do 2 columns outperform 1? (To learn about columns in Spaces, read this great article)
• Do 4-disk Spaces greatly outperform 3-disk Spaces?
• What advantage, if any, does controller cache bring to the table?
• Are there any other advantages in Spaces for Server 2012 R2 that outperform the Windows 8.1 variant (aside from SSD tiering, dedupe and scale)?

No RAID

The first rule of Storage Spaces is: thou shalt have no RAID in between the disks and the Windows OS. Spaces handles the striping and resiliency manipulation itself so can’t have hardware or software RAID in the way. The simplest way to accomplish this is via the use of a pass-through SAS HBA but if you only have RAID controller at your disposal, it can work by setting all disks to RAID 0. For Dell servers, in the OpenManage Server Administrator (OMSA) console, create a new Virtual Disk for every physical disk you want to include, and set its layout to RAID-0. *NOTE – There are two uses of the term “virtual disk” in this scenario. Virtual Disk in the Dell storage sense is not to be confused with virtual disk in the Windows sense, although both are ultimately used to create a logical disk from a physical resource.

To test the native capability of Spaces with no PERC hardware cache assist, I set the following policy on each Virtual Disk:

Make some Spaces!

Windows will now see these local drives as assignable disks to a Storage Pool. From the File and Storage Services tab in Server Manager, launch the New Storage Pool Wizard. This is where creating Spaces in Windows 8.1 and Server 2012 R2 divest in a major way. What took just a few clicks on a single management pane in Win8.1 takes several screens and multiple wizards in Server 2012 R2. Also, all of this can be done in PowerShell using the new-storagepool and new-virtualdisk cmdlets. First name the new storage pool:

Add physical disks to the pool, confirm and commit.

You will now see your new pool created under the Storage Spaces section. Next create new virtual disks within the pool by right-clicking and selecting “New Virtual Disk”.

Select the pool to create the Virtual Disk within and name it. You’ll notice that this is where you would enable performance tiering if you had the proper disks in place using at least one SDD and one HDD.

Choose the Storage Layout (called resiliency in Win8.1 and in PowerShell). I previously determined that Mirror Spaces are the best way to go so will be sticking with that layout for these investigations.

Choices are good, but we want to save committed capacity, choose Thin Provisioning:

Specify the size:

Confirm and commit:

Now we have a pool (Storage Space) and a Virtual Disk, now we need to create a volume. Honestly I don’t know why Microsoft didn’t combine this next step with the previous Virtual Disk creation dialogs, as a Virtual Disk with no file system or drive letter is utterly useless.

Next select the size of the volume you want to create (smaller than or equal to the vDisk size specified previously), assign a drive letter and file system. It is recommended to use ReFS whenever possible. Confirm and commit.

Finally, a usable Storage Space presented to the OS as a drive letter!

Performance Baseline

First, to set a baseline I created a 4-disk RAID10 set using the integrated PERC H700 with 1GB cache (enabled). I copied a 6GB file back and forth between the C:\ and M:\ drives to test 100% read and 100% write operations. Write IO was fairly poor not even breaking 20 IOPS, the RAID10 double write penalty in full effect. Reads were much better, as expected, nearing 300 IOPS at peak.

3-Disk Mirror Space Performance

To force the server to go to disk for all IO operations (since I simply moved the same 6GB file between volumes), I used a little utility called Flush Windows Cache File by DelphiTools. This very simple CLI tool flushes the file cache and memory working sets along with the modified and purged memory standby lists.

Test constants: 3-disk Mirror space, PERC caching disabled, single column (default for 3 disks), 6GB file used to simulate a 100% read operation and a 100% write operation:

Here are a few write operations followed by a read operation to demonstrate the disk performance capabilities. Write IOPS spike ~30 with read IOPS ~170. Interestingly, the write performance is better than RAID10 on the PERC here with read performance at almost half.

Turning the PERC caching back on doesn’t help performance at all, for reads or writes.

4-Disk Space, 1 Column Performance

Test constants: 4-disk Mirror space, PERC caching enabled, single column, 6GB file used to simulate a 100% read operation and a 100% write operation:

When adding a 4th disk to a 3-disk Space, the column layout has already been determined to be singular, so this does not change. To use a two column layout with four disks, the space would need to be destroyed and recreated. Let’s see how four disks perform with one column.

You’ll notice in the vDisk output that the Allocated size is larger now with the 4th disk, but the number of columns is unchanged at 1.

Adding the 4th disk to the existing 3-disk Space doesn’t appear to have helped performance at all. Even with PERC caching enabled, there is no discernable difference in performance between 3 disks or 4 using a single column, so the extra spindle adds nothing here. Disabling PERC cache appears to make no difference so I’ll skip that test here.

4-Disk Space, 2 Column Performance

Test constants: 4-disk Mirror space, PERC caching enabled, two columns, 6GB file used to simulate a 100% read operation and a 100% write operation.

Using 4 disks by default will create Virtual Disks with 2 columns as confirmed in PowerShell, this is also the minimum number of disks required to use two columns:

WOW! Huge performance difference with four disks and two columns. The read performance is consistently high (over 700 IOPS) with the writes performing in a wider arc upwards of 300 IOPS. This is a 1000% performance increase for writes and 300% increase for reads. Literally off the chart! Spaces is allowing each of these participating disks to exhibit their full performance potential individually.

Thinking this is too good to be true, I disabled the PERC cache and ran the test again. This is no fluke, I got very close to the same results the second time with all disks participating. Incredible.

Plex Performance

Since I began this whole adventure with Plex in mind, fitting that I should end it there too. Streaming a compatible file across the network yielded the following results. Skipping around in the file, all four disks were read from at some point during the stream, half spiking ~110 IOPS, the other two ~80 IOPS. This is twice the amount of observed performance on my home 3-disk 7.2k SATA-based HTPC.

Conclusions:

Using 15K SAS disks here, which are theoretically capable of ~200 IOPS each, performance is very good overall and seems to mitigate much of the overhead typically seen in a traditional RAID configuration. Spaces provides the resilient protection of a RAID configuration while allowing each disk in the set to maximize its own performance characteristics. The 3-disk one column Spaces configuration works but is very low performing. Four disks with one column don’t perform any better, so there is significant value in running a configuration that supports two or more columns. Columns are what drive the performance differential and the cool thing about Spaces is that they are tunable. Four disks with a two column configuration should be your minimum consideration for Spaces, as the performance increase adding that second column is substantial. Starting with three disks and adding a fourth will persist your single column configuration so is not advised. Best to just start with at least four disks. PERC caching showed to neither add nor detract from performance when used in conjunction with Spaces, so you could essentially ignore it altogether. Save your money and invest in SAS pass-through modules instead. The common theme that sticks out here is that the number of columns absolutely matters when it comes to performance in Spaces. The column-to-disk correlation ratio for a two-way mirror is 1:2, so you have to have a minimum of four disks to use two columns. Three disks and one column work fine for Plex but if you can start with four, do that.

Resources:

Storage Spaces FAQs

Storage Spaces – Designing for Performance

This post originated at Exit.the.Fast.Lane | Weestro.blogspot.com

HTPC build for Plex three part series:

1. Hardware build and Storage Spaces
2. Plex server and clients
3. Performance

I’ve had a single disk Home Theater PC (HTPC) running Plex for a while now but with my ReadyNas nearing the end of its useful life, I decided to kill two birds with one stone: build an updated HTPC with drive redundancy to also take over the functions of my aging NAS. My HTPC stays on 24/7 anyway so this makes perfect sense. My design goals for this were to build a small, quiet, low power, data redundant and sufficiently performing box that I can stuff into my TV cabinet. The extent of the duties for this HTPC will be video streaming and file sharing/ backup, no gaming. This series details my build, Plex server, Plex clients, Windows Storage Spaces, as well as the performance of the storage subsystem.

Performance

In these performance tests I want to focus on the main activities my HTPC will be undertaking: file read/ write operations and streaming. I tested using both Parity and Mirror Spaces to demonstrate the capabilities of each.

WD Red Baseline (Control)

First to set a baseline for WD Red 3TB HDD, here is the observed write performance with a single 40GB file write operation (from C:\ to Z:\). The copy took around 4.5 minutes and the drive topped out around 145 IOPS. Subsequent read performance tests top out around the same level. Considering these disks have been reported to spin at 5400 RPMs, frankly that’s really not too bad.

Windows reported ~142MB/s transfer speed:

These drives produce absolutely ZERO audible noise. No clicking, no head seeking noise, no rotational noise…nothing. With the cover off of this box, all that is audible is the 3 spinning fans (case, CPU, PSU). Under the TV cabinet, my build is dead silent.

File copy between Mirror and Parity Space

Test constants:

C:\ = SSD
N:\ = Mirror space
M:\ = Parity space

This test demonstrates the performance capabilities of a Parity Space using the WD Red disks. Disk performance copying multiple files from a mirror space (N:\) to a parity space (M:\) was fairly poor, averaging 33 write IOPS on the parity space. Observed read performance on the source 2-way mirror space was low at 25 IOPS on average, but that’s all it was being asked to do. This is somewhat expected however, as one of the caveats with parity resiliency is reduced write performance.

Once again, the Windows file copy dialog tracks very closely to the observed maximum write performance in Perfmon.

Power consumption of the system under test varied between 50 to 75 watts. Very good!!

Write performance of Mirror and Parity Spaces:

Test constants:

C:\ = SSD
D:\ = Mirror space
M:\ = Parity space

Here you can clearly see the write performance disparity between a two-way mirror and parity space. Copying a 4.5GB file from SSD (with very high read performance) to my two-way mirror space yielded very reasonable write performance according to the Windows file copy dialog. The overhead of the mirror operation appears to be costing around 20 IOPS.

The write activity of the physical disks participating in the mirror.

That same 4.5GB file written from SSD to my parity space (M:\) yielded significantly lower results (almost 3.5x lower):

Here is a more detailed view of the write performance against the parity space:

Read performance of Mirror and Parity Spaces:

Test constants:

C:\ = SSD
D:\ = Mirror space
M:\ = Parity space

Copying the same 4.5GB file from each space back to the SSD yielded better results overall. The read performance of the parity space during this operation is better but still below what the mirror space or even native drive are capable of:

Copying the file from the mirror space to the SSD was higher performing than the parity space and on par with native disk read performance, but the gap isn’t as wide here.

Plex Streaming Performance: Single File

Test constants:

-Physical disk 1 and 2 comprise a mirror Space
-Plex clients in use: Roku
-Media in use: single file, highly compatible, 1080p

Streaming being a highly sequential mostly read operation, there isn’t a great tax put upon the system resources. Multiple streams and transcoding complicate this a bit. The following chart shows the performance of a single video stream from my HTPC using a mirror Space to my Roku client. Interesting to note that the C:\ drive is busier than both disks in the mirror space, but all reads are sourced from a single disk.

Plex Streaming Performance: Single File, skipping around

Test constants:

-Physical disk 1, 2 and 3 comprise a mirror Space
-Plex clients in use: Chromecast
-Media in use: single file, highly compatible, 720p

Trying to force spaces to utilize all disks within a vDisk, I manually skipped around four times within a sample media file. Here you can see that with every color change within the performance chart is where I jumped to a different time within the media file. Almost every time I did this, Spaces offered up a different disk to be read from. The one exception is during the period between 7:40 and 7:43 where I performed a jump but the same disk was used. Otherwise, all three disks in  my mirror Space were used to read the file.

Plex Streaming Performance: Multiple Streams from One File

Test constants:

-Physical disk 1 and 2 comprise a mirror Space
-Plex clients in use: Roku, Chromecast, PS3 (DLNA)
-Media in use: single file, highly compatible, 1080p

To satisfy my curiosity, regardless of how infrequently this situation might actually occur, I wanted to see what the impacts were if I kicked off multiple streams of the the same file. I chose a mostly compatible file so we get the benefits of Direct Play here. Three simultaneous streams requiring transcoding would probably not work so well. There is a CPU spike when the third stream is launched and minor spikes on disk 2 for each stream event. You’ll note here that all reads are coming from disk 2. Looking at each TV screen individually and subjectively, the playback was completely normal.

Plex Streaming Performance: Multiple Streams from One File, 3 Disk Mirror

Test constants:

-Physical disk 1, 2 and 3 comprise a mirror Space
-Plex clients in use: Roku, Chromecast, PS3 (DLNA)
-Media in use: single file, highly compatible, 1080p

Just to see if there is any impact to performance, I ran this same test again but with three physical disks in my mirror Space. I would have expected my reads to be coming from at least two disks but as evident here, disk 1 is doing almost all of the work. Interesting to note that overall spikes are much lower now with read performance hovering around 1 IOPS. The takeaway here is that this third disk in my two-way mirror Space doesn’t add much from a performance perspective.

Plex Streaming Performance: Multiple Streams from Multiple Files

Test constants:

The following chart shows the effects when additional streams are added using three different media files, one requiring transcoding. You can see that system resource usage is relatively low with the first two streams, things change dramatically with the introduction of the third file. Transcoding means heavy CPU usage as evident here. Disk activity with 3 different files being accessed simultaneously jumps up quite a bit too. 

Zooming in here looking at the disk IO, you can see once again that almost all read requests are satisfied from a single disk. Hmm, where’s my striping performance??!

Here is the Plex transcoder in action:

Network Performance

Network performance of the Plex server with a single HD media stream is very reasonable. There are spikes north of 11Mbps but for most of the duration the  transmitting bandwidth stayed around 2Mbps.

Conclusions:

• The WD Red HDDs are a very quiet, very low power option that yield decent disk performance. Performance is more than acceptable for an HTPC that doubles as a NAS.
• Two-way mirror Spaces are the highest performing option in Storage Spaces but come with the highest disk cost as well (2x). Read performance is on par to that of a native disk with Spaces adding no discernable overhead, writes cost ~20 IOPS in overhead. If you have HDD capacity to spare, this is the resiliency option you want!
  • While writes clearly hit all disks in a mirror Space, observed read operations from streams are satisfied from a single disk in the Spaces pool. Jumping around within a media file does force reads from all disks in a Space.
  • 3 disks in a Storage Space really doesn’t buy you much other than additional capacity. With a 1-column two-way mirror you still only have two copies of data, on two physical disks. Read and write performance is not increased here.
• Parity spaces will work fine for Plex and video streaming in general, as video streaming is sequential and read performance is acceptable. Parity spaces suffer massively in reduced write performance so copying files to or moving files around on your HTPC will be much slower. The trade off for poorer write performance in a parity space is more overall usable capacity (1.5x consumed in a parity space vs 2x in a mirror space).
• Plex makes excellent use of system resources and can handle just about anything you can throw at it. The Plex server doesn’t even break a sweat with highly compatible media types, even with multiple streams. Transcoding is CPU intensive and should work fine for a single stream, multiple streams may tax your system and degrade the viewing experience.

This post originated at Exit | the | Fast | Lane – weestro.blogspot.com

HTPC build for Plex three part series:

1. Hardware build and Storage Spaces
2. Plex server and clients (You are Here)
3. Performance

I’ve had a single disk Home Theater PC (HTPC) running Plex for a while now but with my ReadyNas nearing the end of its useful life, I decided to kill two birds with one stone: build an updated HTPC with drive redundancy to also take over the functions of my aging NAS. My HTPC stays on 24/7 anyway so this makes perfect sense. My design goals for this were to build a small, quiet, low power, data redundant and sufficiently performing box that I can stuff into my TV cabinet. The extent of the duties for this HTPC will be video streaming and file sharing/ backup, no gaming. This series details my build, Plex server, Plex clients, Windows Storage Spaces, as well as the performance of the storage subsystem.

Plex

So what is Plex and why do we need it? Plex is a very simple, very powerful media center server that can stream videos, music and photos to almost any device…most importantly TVs. Plex can be installed on many NAS devices as well as Windows/Mac/Linux based PCs. Without Plex, DLNA is available to watch supported content from network accessible locations and works fairly well most of the time, but this is simply a list of content. Plex can also talk DLNA but the media server greatly enhances the overall experience by providing art, posters, synopsis, which media has been watched/ which hasn’t, the ability to resume where you left off, remote streaming and transcoding. Plex also supports many file containers (AVI, MKV, MP4, etc), codecs and subtitles. I personally used the DLNA method for years before seeing the light on Plex and no way would I go back. I love Plex and you will too!!

First, create an account on http://plex.tv, you’ll need this later for remote access and it will make management much simpler. Setup is simple: install the Plex media server on the supported platform of your choosing, open the Plex console in a web browser and point Plex at the locations of your media by adding libraries. Media is best categorized by type which will make browsing, searching and updating content much easier. Plex makes this process very easy for you:

I use the following categories and will add one for photos soon:

Plex will then automatically connect to a variety of online sources, such as Freebase or The Movie Database, to download media information and art.  Once complete all of you media will be organized and accessible in the Plex console with full posters and even background music, if you want it.

Plex content can be browsed via normal folder structures if you like or you can use the Plex Dashboard where you will find “On Deck” and “Recently Added”. The latter should be self explanatory, On Deck is where you will find movies that you started and stopped or the next episode of a TV show in a series. This is Plex helping to keep your place in your media universe and it is incredibly useful. 

Plex provides three primary methods for playing content: Direct Play, Direct Stream and Transcoding. Bottom line is that Plex will play your media one way or another, you don’t have to worry about it, but this is how it does it.

• Direct Play is the best and exists in the ideal scenario. 100% file compatibility with very little resources consumed on the media server itself. The file is sent to the Plex client as-is, easy breezy.
• Direct Stream is used when a media file is mostly compatible but some element within is not and requires transcoding. The example Plex uses is a media file where compatible audio/video codecs are streamed direct and the non-compatible container is transcoded.
• Transcoding is used when either the audio or video format isn’t compatible and has to be converted into a compatible format. Audio transcoding is not very CPU intensive but video transcoding is.

Recommended Tweaks

Overall, you could change almost nothing post install and Plex will work fine right out of the box. Here are a few tweaks that I recommend to enhance your experience.

In the server settings, click Connect on the left, make sure to log into Plex (your online account) as this will allow you to connect to and stream media to any device anywhere you are. This is honestly one of the coolest features of Plex and you do not need a Plex Pass to do this! Once signed in on your server, you can now connect to your Plex environment from any smart phone, tablet or PC anywhere you have internet connectivity by using the IOS/ Android Plex app or by logging into Plex.tv. Traveling? As long as you have internet connectivity, you can browse and watch anything from your library. No USB thumb drives required!

To maximize ease of use when you add content, make sure the following settings are enabled. This will ensure that Plex will automatically scan your library when files are added vs having to run a manual or timed scan. Plex will see that you’ve added a file and will automatically add it to the library for playback and download related art.

Under the Web settings, click Player on the left, you can control the streaming quality based on where you will be streaming to. 1080p for local streaming and a lower setting for remote streaming, for example. I have found that setting the local quality to 4Mbps/ 720p greatly increases local streaming responsiveness and makes it easier to skip around in a given media file.

Further customization at this point is highly preferential and you’ll figure it out by clicking through the available options. Plex has made this all VERY easy. In Windows, Plex runs as a process launched by an interactive user so does require that a user account is logged into the desktop. There is no Windows service offering yet but there are ways to work around this. I have an account set to login automatically after a reboot then the screen locks after 1 minute for security. In lieu of a Windows service starting with the OS, this works fine.

Plex Clients

With the HTPC built and Plex installed, let’s talk about client connectivity. There are a number of ways to go here which is another benefit of Plex: extreme flexibility. The methods below are my top 3 in order by preference.

Smart TV app

If you own an internet connected Samsung Smart TV then you already have access to the best free Plex client available. Nothing is required externally, just install the Plex app from the Samsung app store on your TV, launch and it should find your Plex server on the network. Your TV remote will navigate the Plex menus and this will provide the richest experience you can get while navigating Plex on your TV screen. Sadly, if you own a Panasonic TV you will have to use an external client. There is no and will be no native Plex app in the foreseeable future.

Google Chromecast

Chromecast is a very simple and very cheap option that I personally use on my Panasonic plasma. $35 gets you an HDMI-connected dongle that simply casts or displays content on your TV via wifi. This is an important distinction to make when considering the Chromecast. There are no on-screen menus to navigate and there is no integration with your all-in-one remote. Your PC, tablet, or phone is your remote and where you select the content you want to display on your TV. This does require that you purchase the Plex app for IOS/ Android ($5) or if you choose to pay for PlexPass, you’ll get the app for free. Chromecast for Plex works great, the downside is the other services currently supported are a bit limited compared to Roku and the like (if you care). Chromecast is powered via a USB port on your TV or from a wall adapter.

Chromecast setup is very simple and pretty neat in what they do. Initially the Chromecast turns itself into a wifi Access Point (AP) by broadcasting a SSID that you connect to for configuration using the Chromecast app on your device. Once the local wifi network SSID and password are selected, Chromecast will disable its own AP and connect to yours.

Once Chromecast is configured, launch the Plex app on the device of your choosing and navigate to the content you want to watch. Connect Plex to the Chromecast by pushing the “Cast” button in the top right corner and select the Chromecast, this will also automatically change the input on your TV to the correct HDMI slot. Press play on the chosen content. This method is definitely different than the more traditional on-screen navigation approach but having used it for going on a year now, I really like the experience this solution provides. Your more technically challenged family members may not like or fully understand this method without some training.

Roku Streamstick

Roku is a favorite of many home theater aficionados and provides several client options with varying price points. The Roku Streamstick is the Roku answer to the minimalist HDMI dongle but with the majority of the value that Roku provides. I use this client on my older Samsung plasma that doesn’t have built-in wifi support. Roku Streamstick is powered via a USB port on your TV or from a wall adapter. Setup is simple and handled via the specialized Roku remote included. Connect Roku to your home wifi and you are good to go.

Roku uses the more traditional on-screen navigation approach and has an impressive catalog of connectable services you can use. One of the neat things Roku provides is an impressive search function that gives you multiple options as to where you can find/ buy/ play the content you’re looking for. Roku works using a concept of “channels” which are really apps that you add to your client. Plex, Netflix, Amazon, etc. Plex is one of those channels and costs $5 just like the IOS/ Android version. PlexPass members get this app for free as well. Unlike the Smart TV or Chromecast methods, Plex on Roku is a very minimalist affair that provides a very basic visual experience.

While Roku does ultimately provide the essentials for Plex, it is my least preferred method and comes with the following caveats:

• Separate non-IR Roku remote required to control on-screen navigation. No integration into traditional IR all-in-one remotes possible.
• Very slow boot up compared to Chromecast and takes a very long time to get to a point you can actually use it.
• $50 vs $35 for the Chromecast
• Least rich experience of the 3 options with a very basic user interface.

Please also see part one of this series which details the hardware build and part three which discusses performance.

References:

https://plex.tv/

Converting ISO, Video_TS and other disk image formats