On PAE's inability to enable access to more than 4GB of ram

[valis]

In a discussion some time ago I recall mentioning that Windows (post Xp sp2) was no longer able to use PAE to enable addressing more than 4GB of physical ram on the 32bit variants of the OS, and that Microsoft justified this limitation via claims of driver instability (from companies that were still having a hard time adapting to post-16bit Windows.) Well, that's not incorrect but there is a bit more to the story.

http://www.geoffchappell.com/viewer.htm ... memory.htm

This article is particularly interesting since the transition between 32bit & 64bit systems is currently underway, with the main message people are getting about the 64bit transition is that it's *necessary* to get access to more than 4GB of ram. Of course users of NT before XP & in the early days of XP are aware of the ability to address more than 4GB of ram on a 32bit CPU (back to the PPro, mine was a dual 200mhz powerhouse!) But the article is interesting because it shows the change was actually a simple check rather than anything actually being *removed* from Xp (in Service Pack 2) so I thought I'd share... discuss?

[Neutron]

If i read that bit about server 2008 correctly, that means certain people who need lots of memory for romplers and etc on their DAWs, could theoretically run scope on 32 bit server 2008 and have up to 64 GB of RAM, now we just need a guinea pig

[valis]

I believe you're reading that correctly (as long as the system's BIOS supports enabling NUMA for handling that ram.) Of course no matter how much RAM is installed 32bit applications are still going to be limited on how much ram they can handle (per app, either 2GB or 3GB depending.) I think he also put instructions on the end for those who wanted to hack the Vista kernel as he did (ie, if someone was feeling REALLY adventurous.)

[kylie]

I'm kinda tempted to install w2k3 or w2k8 on a spare drive and see what happens...

[Neutron]

Does a VSTi count as a separate application? in that case it could still be useful for some people.

[dawman]

If anyone actually proves this in 32bit please report back.
I am dreading the idea of 64bit. Sure Cubase, Bidule, Gigastudio 4 and Kontakt 3.5 are all 64bit and many guys are enjoying it's benefits, but I prefer mature working 32bit versions still and the extra RAM is the ONLY reason to upgrade IMHO.
Even Quad CPU"s, while working w/ sequencers well, still haven't been optimized on powerful VSTI's yet. Most powerful VSTi's I see can have their Core assignments but do not use more than a single core. That is why you will see iMacs and even laptops outperforming the new Dual Quad i7's that run @ 2.26.
Please someone try this and report back.
I would love to use those new 4GB DDRII-DIMM's that are going for dirt cheap.
16GB's would be heaven w/ a fast E8600 again.

[cortone]

Jimmy, it's a minor point, but you don't optimize the cpus to run the VSTi, it's the other way around. Those laptops aren't outperforming the high-end procs from AMD or Intel, the software just happens to be better-optimized for the older processors.

Software engineers have had 8 or so years to deal with multi-threading in their apps. AMD really started that ball rolling and Intel played catch-up, well enough to be king (right now anyway, and probably for some time to come). Most applications and OSs are about 5 years behind at this point. Rather than optimizing with better coding techniques, they have relied on the ever-advancing hardware to make the s/w appear to be improving. Microsoft is especially guilty of this, and they would shift the whole world forward if they got it together (not in their business plan though), but they are definitely not alone in the s/w bloat category.

The hardware guys build it, the software guys just don't come until they're damn-well ready (broad generalization, there are some who do!) and they've finished kicking and screaming (and whining that the hardware guys are making them do it).

Cory

[iSiStOy]

At least for multi cores vsti are assigned to other cores by C5. So they might use also other memory pages then.

OK for CPUs but I thought the VST plugins were hold/encapsulated into the Cubase.exe memory space, meaning they are constrained to the CubaseSX.exe process max. in a 32 bits system process memory allocation...
... But I might be wrong.

[valis]

Kontakt on OSX supports something called 'Kontakt Memory Server' which launches another process and requires giving that process admin privs (presumably to talk to the 'original' process in your host.) Unfortunately this is Mac OS X only & I haven't any idea what Giga does under windows.

Jimmy, you should hit up the plogue forums and see what experiences people have had using Server2003/Server2008 with a 32bit kernel (I have no idea how to enable 32bit + NUMA with them during install, having never installed them.) Rather than worrying about how cubase handles it, it's probably more relevant to head right to Plogue (or whatever your host is atm) and see if you can get them to help you setup a few test cases. If Kontakt/Giga really were able to run separate memory management underneath Bidule I imagine there would be a LOT of people interested since 64bit drivers are still slow to mature for some things (or never will appear) though I suspect in most cases the plugins are limited by host application's memory paging.

[iSiStOy]

Exactly!
Several separated application instances for plugins, able to efficiently share memory, a way to overcome the problem of addressing it in 32 bit systems.

[valis]

OS X isn't artificially limited (as NT 5.x & 6.x are) to only addressing 4GB of PAE's 36bit space, presumably due to the fact that Apple exerts enough control over their platform that they rarely have to deal with braindead driver developers making assumptions based on previous OS versions.

[valis]

Incidentally I meant when using a 32bit kernel in OS X, which is actually the default still in Snow Leopard (10.6) regardless of all the 64bit marketing from Apple. Interestingly enough in both Leopard & Snow Leopard the 32bit kernel can host some 64bit processes, but to use the 64bit kernel (and allow full access to 64bit paging) you have the same restrictions that Windows does (64bit drivers are a necessity etc.)