Smaller HDD Arrays Give More Polyphony ?

[dawman]

I just condensed the content of 2 HDD's onto a single HDD, a 7200rpm Western Digital Caviar to see how it's performance compared to the 74GB, and 150GB Raptor. It's going to be my Cubase audio HDD eventually.

But I thought I would check out the difference, and it was quite shocking.

I unknowingly spread my sampled content across the 2 Raptors, as the 74GB HDD was filled and needed the 150 to hold other content. This seems to allow much more polyphony especially when I stacked multiple instruments in Gigastudio, one of it's great features IMO. But a certain GVI patch can actually stream from both drives simultaneously, as that is where the paths lead.

I did this out of my need to expand my content, not knowing it is the way to get extreme voice counts in Giga.

The single drive approach gags on most of the performances I loaded. I thought it might be the 7200rpm speed, but the memory meter, and CPU meter proved that false.

So I am convinced that these giant HDD's while fast, and sometimes cheap, will not give me the polyphony needed as compared to an array of 150's @ 10k rpm.

I assume this would work on all soft samplers that stream from the HDD's.

I still would love a giant RAM based soft synth, or some of those SSD's.

[pollux]

So I am convinced that these giant HDD's while fast, and sometimes cheap, will not give me the polyphony needed as compared to an array of 150's @ 10k rpm.

A disk array will *always* be more performant than single drives.

Also separating system/software, swap and data leads to performance gains... No matter how much free RAM you have, WIndoze will always swap (say you've 16 GB on the machine, 4 or 5 of them free, and a 3GB swap file???)

The dream configuration (huge rack and power supply needed ):

* a small RAID 1 array for they system and the software
* a small RAID 0 array for the swap file
* a huge RAID 0+1 array of at least 4 disks for the data, if mostly read/write, like a recording station, or RAID 5 of at least 3 disks if mostly read-only like a sampler (can even combine both )
* an even bigger bank account to buy all this (don't shoot the messenger)...

[kylie]

A disk array will *always* be more performant than single drives.

no. it depends on several aspects. think of a raid 5 done in software. no good idea...

No matter how much free RAM you have, WIndoze will always swap (say you've 16 GB on the machine, 4 or 5 of them free, and a 3GB swap file???)

you can disable the pagefile (or set it to zero), although it is strongly recommended by MS not to do so; others say, if you have a 32bit windows and 4GB ram, it doesn't matter, but wastes memory.

* a small RAID 1 array for they system and the software
* a small RAID 0 array for the swap file
* a huge RAID 0+1 array of at least 4 disks for the data, if mostly read/write, like a recording station, or RAID 5 of at least 3 disks if mostly read-only like a sampler (can even combine both )

a raid10 would be fine. remember, a fast raid volume is always distributed over as much spindles as possible.
you would need a powerful raid controller for all that, and a fast bus to hook it to. there is a reason why current array controllers capable of supporting more than 2 drives are PCI-X or PCIe 4x minimum...

the 10k drives also reach a higher i/o count than those large 7.2k, which get a decent sustained transfer rate if you just read a large uncluttered file from it, but as soon as you start to read parallel streams, the heads have to move rather quick between the different disk zones, and since the disk size itself is bigger the heads have a longer way to go. add missing command queueing capability (if that applies) to it, and it will get even worse.
the head movement is much less hectic with NCQ/TCQ support, which counts to the higher I/O rate of scsi and sas drives as well as of the raptors.

-greetings, markus-

[pollux]

no. it depends on several aspects. think of a raid 5 done in software. no good idea...

In software / system a RAID 5 is ok because it's mostly read-only. For many kinds of data it's ok too. RAID 10 or 0+1 is only recommended for throughput-hungry apps, like transactional databases or live recording systems.

Anyway, given the usual size of system/software, a RAID 5 is not needed, and a small raid 1 works perfectly.

a raid10 would be fine.

RAID 10 = RAID 0+1 (some purists say different, but in practice they are almost always implemented the same.. don't intend to start a religion war here about if you should mirror the stripes or stripe the mirrors)

you would need a powerful raid controller for all that, and a fast bus to hook it to. there is a reason why current array controllers capable of supporting more than 2 drives are PCI-X or PCIe 4x minimum...

I've been doing this for years, much before PCI-E exists. You can perfectly do it with standard PCI RAID cards, and if possible SCSI disks (much more reliable than SATA)

the 10k drives also reach a higher i/o count than those large 7.2k, which get a decent sustained transfer rate if you just read a large uncluttered file from it, but as soon as you start to read parallel streams, the heads have to move rather quick between the different disk zones, and since the disk size itself is bigger the heads have a longer way to go. add missing command queueing capability (if that applies) to it, and it will get even worse.
the head movement is much less hectic with NCQ/TCQ support, which counts to the higher I/O rate of scsi and sas drives as well as of the raptors.

True.. that's why it's also a very good practice to correctly tune the stripe sizes according to the needs, to minimize the I/O operations... nfortunately there is no "one size fits all" an it takes huge amount of time to correctly benchmark it and go live...

Cheers

Raul

[kylie]

In software / system a RAID 5 is ok because it's mostly read-only.

if intend to use it as read only data store you're maybe right. but you'll have a hard time, nevertheless, using it while it initializes or rebuilds.

a raid10 would be fine.

RAID 10 = RAID 0+1 (some purists say different, but in practice they are almost always implemented the same.. don't intend to start a religion war here about if you should mirror the stripes or stripe the mirrors)

no war
there ARE differences, but I always forget which is the "better" one and always have to look up the reasons why. fact is: if 2 certain drives fail, it's all gone. the discussion is mainly about the statistically higher probability of failure depending on how it is organized

you would need a powerful raid controller for all that, and a fast bus to hook it to. there is a reason why current array controllers capable of supporting more than 2 drives are PCI-X or PCIe 4x minimum...

I've been doing this for years, much before PCI-E exists. You can perfectly do it with standard PCI RAID cards, and if possible SCSI disks (much more reliable than SATA)

well, the transfer rate of scsi disks and controllers didn't start that big either, so conventional pci was sufficient for a long time
and btw, of course I referred to current technology. if you want to get maximum performance from your 15k drives, you won't hook them to pci, will you?
scsi is almost dead. it's SAS or FC. the raptors, as a derivate built with several of their features, mainly differ in their interface.

-greetings, markus-

[pollux]

if you want to get maximum performance from your 15k drives, you won't hook them to pci, will you?

Let me think... hmmm...

scsi is almost dead. it's SAS or FC.

Of course.. When I say "SCSI" I mean "server/enterprise class storage"


Anyway, it's fun to share these "hi-tech" points of view.. One can always learn new stuff


Cheers!

Raul

[kylie]

scsi is almost dead. it's SAS or FC.

Of course.. When I say "SCSI" I mean "server/enterprise class storage"

agreed

Anyway, it's fun to share these "hi-tech" points of view.. One can always learn new stuff

you bet! but I just realized we're not in the tech talk forum...
let's blame jimmy, he started it all!

-greetings, markus-

[Shroomz~>]

let's blame jimmy, he started it all!

Is it ok if we blame him for everything while we're at it?

[kylie]

let's blame jimmy, he started it all!

Is it ok if we blame him for everything while we're at it?

I'm fine with that. let's go

[pollux]

let's blame jimmy, he started it all!

Is it ok if we blame him for everything while we're at it?

I'm fine with that. let's go

Besides blaming Jimmy for this and the bad wether over here, we could move this topic to the tech forum in case it would be useful for anyone it'd be found easier

[kylie]

Besides blaming Jimmy for this and the bad wether over here, we could move this topic to the tech forum in case it would be useful for anyone it'd be found easier

contact a moderator. iirc we can't move it ourselves...

[grappa]

Without getting too technical the following applies (and a lot more besides):

1. Forget software RAID - good for fault tolerance RAID configs on a budget but if your gonna invest in performance you want a hardware RAID controller. Remember you can use one card to support a number of different logical RAID volumes based on requirements/budget.

2. The correct RAID configuration depends on what you intend to do with the array and whether fault-tolerance is a requirement.

3. RAID performance is determined also by controller cache and its configuration, the rotational speed of the HDD's, seek time, disk interface type, RAID controller interface....

4. RAID stripe size also has in impact on performance as does sector alignment.

The best performing RAID configuration for read and write (without any fault tolerance) is RAID 0.

This basically stripes each physical disk in the array with some of the data i.e. two disks get half, three a third etc. The greater number of disks the greater the potential throughput. The downside of this configuration is that if a single spindle fails ALL of the data is lost.

RAID 1 copies data on write to another disk offering fault tolerance but no performance improvement.

RAID 10 is a fault tolerant extension of RAID 0 whereby we create the same RAID 0 configuration with twice the number of disks. We then mirror the array. The problem with this is cost - half of the storage sits doing nothing unless a spindle fails.

RAID 5 is a scheme that writes parity data across the spindles in an array so that if one of the spindles fails the array can rebuild the data dynamically using the parity - the drive can then be replaced at a convenient time and be automatically rebuilt with no data loss. RAID 5 comes with a performance hit as parity data has to be calculated during writes - due to parity some of the disk space is also consumed although much more economically than RAID 10/1 and less so as more spindles are introduced.

For Audio applications RAID 0, decent RAID controller, decent disks (and a good backup regime) is the kiddy - fastest read/write performance, best performance/price ratio.

Anyone trying to get the most out of disk performance should do some research at http://technet.microsoft.com/en-us/libr ... 98219.aspx as disk alignment is very important. Bit techy I'm afraid.

Simon

[FrancisHarmany]

I wonder how iSCSI performs with audio related work!

[pollux]

I wonder how iSCSI performs with audio related work!

iSCSI is intended to be a replacement to FiberChannel, mostly for SANs.. so unless you have huge storage bays with terabytes and terabytes of storage capacity, and a decent server farm connecting to it, it's useless

For internal storage, or a simple storage bay attached, then SATAII or SAS are the best choices.

[kylie]

I wonder how iSCSI performs with audio related work!

our experiences with iSCSI ar not as satisfying as one could expect...
but then, it's way cheaper than FC, and if reliability is not a major concern it may be fine.
I see you don't want to have your disk array near you, do you?

-greetings, markus-

[grappa]

Our iSCSI implementation has been pretty succesful.

Hosting Exchange, SQL, FIle Shares etc with great results.

Need to use Jumbo Frames, dedicated iSCSI LAN, TCP/IP ofload etc but rock solid, great performance and much cheaper than SAN.

Add features like snapshots, replication, dynamic volume management and you get some serious bang for buck.

Great for enterprise environments but not really suitable for digital audio streaming at home due unless you've got some serious money and relevant skills

Simon

[FrancisHarmany]

I wonder how iSCSI performs with audio related work!

our experiences with iSCSI ar not as satisfying as one could expect...
but then, it's way cheaper than FC, and if reliability is not a major concern it may be fine.
I see you don't want to have your disk array near you, do you?

-greetings, markus-

Exactly! Also in the future I will have multiple DAWs so it might
be cool for such a setup

[dawman]

Success !!

Multiple single and dual platter HDD's that contain pre configured content works significantly better.

I just finished re doing the 3 Raptors where I have taken the time to catagorise them more efficiently.

It works better when I spread the content around. For example, some of my string and horn section libraries contain several different libraries. So having those heavily used multi library performances spread out like this:

First Call Horns D Drive
Chris Hein Horns E Drive
Quantum Leap Brass F Drive

The app just works so much better as I stacked and stacked and never got a glitch. On the single drive approach they would have crashed the app or gagged. Even the way I had them before, just paced wherever was in efficient. This approach is sound, and well worth the time and hassle.

I get to do it again next week too.

Would this approach also work with pre recorded tracks in Cubase ?

Say you have 128 tracks of MIDI and 128 tracks of audio. Could you spread the content to different drives?

Sorry if this gets something else going, I shall take my verbal thrashings in exchange for quality information.

[Me$$iah]

Great thread!

I think I just found a use for the remaining pair of SCSIs I have here.

I believe they shall soon be holding my 'drumkits' on my VST computer

Ahhh I love these general scope dicussions [/sarcasm]



Im gonna blame Jimmy, an say cheers for making my system better (I hope)





I hate the thread facsim here, but I guess I would probly miss it. No. In fact I take that back, I probly enjoy it in some perverse way

[hubird]

Ahhh I love these general scope dicussions [/sarcasm]

...the logic of some forum selfdiscipline is - except that it makes focussed reading possible - getting usefull results when using the search engine...