- Reference Benchmarks :.
PC DAW O.S Shootout : Part II : XP 32 v Vista 64 - SONAR :.
Back again for another round of what will become an ongoing series.
Following my original report in regards to DAW performance across the competing O.S platforms, there had been a few concerns raised due to the choice of Audio applications – Steinbergs Nuendo 3.20 / Cubase 4.02 not being Vista optimized, so it wasn’t really a fair reflection on what Vista had to offer as a DAW O.S
With that in mind, I decided to port the methodology
across to Cakewalk SONAR 6.2.1 , which had both Native 32 and 64 bit
versions of the application, as well as some highly publicized Vista
Optimizations, especially in regards to MMCSS -Multimedia Class Scheduler
|Preparing for Battle :|
Firstly we needed to port
the methodology as closely as possible over to SONAR.
The overall participation and contribution over at Cakewalk forums was disappointing to say the least, but the ball was rolling , so we pressed on regardless.
This time I decide to bypass XP64 and to test only using XP32 and Vista 64, as most if not all Audio developers are concentrating purely on Vista 64 for their Native 64 bit applications.
I personally believe XP64 is potentially a far superior 64 bit DAW O.S solution, but unfortunately the dev's are not going to be ringing that bell too loudly, especially Cakewalk, who have been trumpeting the loudest about their Vista Optimizations.
With that in mind , I have streamlined the testing to native 32 Bit applications on XP 32, and Native 64 Bit applications on Vista 64.
|WDM V ASIO :|
Sonar is one of the few
Professional Audio application that use both WDM/KS and ASIO as low
latency driver models.
Not having a lot of experience with WDM, I stumbled on a few anomalies in profiling the Lynx Audio card that I used as the reference card in my first round of testing, which lead me to further investigate how WDM interacts with the buffer settings within the cards control panel.
I also decided to cross reference the testing this time with a RME HDSP card to further consolidate the results across the 2 driver models.
When using ASIO , the buffer settings that are set in the respective control panels of the audio hardware is reflected in the latency setting that SONAR displays, very simple, streamlined and uncomplicated. With WDM , the latency control is supposedly handle by the application after the card has been profiled , which then determines the appropriate lowest latency setting available-which can then be adjusted by a slider within the application.
What I wanted to understand was what if any correlation
did the buffer settings in the control panels have with the lowest available
setting within SONAR. With The RME card, the lowest available latency
would correspond to the initial buffer settings in the control panel.
ie : 032 – 1ms , 064 1.5ms, 128 – 2.9 ms.
With the Lynx card, no matter what the
initial buffer setting , once profiled the lowest available setting
was always 2.9 ms , however , if the initial buffer setting was set
to 032/064, the performance was markedly different to the buffer being
set to 128 to coincide with the lowest available setting with SONAR.
Confusing, you bet.
So despite the WDM profiling always reporting an available
2.9 ms , the initial buffer settings which should have been ignored,
were still impacting on the overall performance.
I asked Paul Erlandson from Lynx to not only confirm the behavior, but also to help me understand what the correlation is between the 2 settings. The information was delivered during a real-time messenger chat, so I will need to collate and paraphrase the information into a more cohesive format. Any technical discrepancies will therefore be mine, and not Paul's.
“ With ASIO the buffer size in the Lynx Mixer impacts the app's buffer and the transfer size. This is defined by the ASIO spec. With WDM/KS, the app buffer size is controlled by the app, and the buffer size in the Lynx Mixer is actually just the transfer size. The size of chunks that our hardware will swallow at a time.
There is a unique function call for Sonar where we send up what we think these values should be for given sample rates, so, the real buffer size as we're used to thinking about it, is the slider under Mixing Latency. The other values determine the size of chunks they are sending and the size of chunks we're receiving at a time, we do not report back up to Sonar a value for that buffer size, since that should be controlled by the app.
The reason setting the buffer size in the Lynx Mixer is not reflected there is because they are two different entities, buffer size vs transfer size,we could do a metric, i.e. 2X the transfer size should be the buffer size, but we don’t. So, when you set the buffer to 32 samples, that's just way too low, especially since we are being mounted as a multi-channel driver under WDM/KS.”
In respect to RME cards following the ASIO Buffer setting..
“It's not the ASIO buffer, but they must do some
function call making the Sonar buffer size relative to their transfer
size. Ultimately Sonar is in charge of this though. It yields little
to play with these transfer sizes, unless your operating on the edge.
With all that under my belt, I pressed on. Just a small note. I always matched the buffer settings with the respective hardware controls for both cards , to the buffer/latency setting I was testing in SONAR , just to ensure no buffer/transfer mismatches.
|Round One : SONAR 6.2.1 : Lynx 2 / RME HDSP : XP32 v Vista64 : ASIO :.|
I initially ran the tests with just the Lynx card, but I decided to cross reference the results with the RME HDSP this time around, to ensure that the performance variables could not be attributed to a specific audio interface/driver.
I first tested the system under XP 32 using the 32 bit version of SONAR, and as it was the first time I had officially put the test session thru its paces, I made special note of the inherent behavior of running thru the test session. Some interesting points
I wasn't able to scale the system to higher than about
70% CPU loadings before the session stopped playback with a drop out.
I couldn't get to the stage of audio break up with crackles and pops,
as Sonar stopped playback at the point of a drop out, which is great
for determining just when it hits the wall..
On the point of the first drop out, I hit stop and started playback again, and then managed to usually load another 2-3 Vintage Channels before the next drop out.
Any attempt at adding any more plugins after the second drop out resulted in playback instantly being stopped by another drop out.., so that's the score I have listed.
On to Vista 64, using SONAR x64 :
Well the numbers tell the story, what can I say except we have a Native 64 Bit Application , with specific Vista Optimizations, and it has totally tanked using ASIO.
On both audio cards tested.
Running the test was also nowhere near as smooth as in XP32 , slight glitches as plugins were enabled, which were not apparent under XP32, and also crackles and pops way before the final tank.., which made things even more difficult to get a quantifiable results as once it started glitching, moving back a few plugs to try and clear the pops and clicks, resulted sometimes in having to remove 10 or more.. ?
|Round Two : SONAR 6.2.1 : Lynx 2 / RME HDSP : XP32 v Vista64 : WDM :.|
under WDM were less consistent than the ASIO results across the 2 audio
cards , so having the cross reference is valuable in gauging a more
Playback was just a garbled mess no matter what buffer setting. I was unable to resolve the issue, so I unfortunately do not have a cross reference to the Lynx results.
None the less the Lynx results were enough for me to conclude that the performance on Vista 64 was again abysmal compared to XP32.
So is this a reflection of what to expect from a Native 64 bit application, under Vista 64, or is there more to it.
*It has been suggested that the PlugIns used in the test session, which were all SONAR bundled , were perhaps not x64, therefore we were seeing the performance penalty of using the Bitbridge that enables SONAR x64 to use 32 bit plugins.
I have no way of confirming this, and the Cakewalk dev's have not come forward to offer any explanation.
I would have hoped that with the amount of hype ventilated from Cakewalk in regards to their X64 Native version of the application over the last 18 months, that at least the native plugins bundled with the application are x64. ? !*
This rounds off my testing until at least the next batch of application updates, most importantly Steinbergs Native x64/ Vista optimized release later this year. SONAR 7 is also due for release in the next few months, but I have to admit that after my recent experience, I’ll be hard pressed getting excited.
Since I have gone public with the preliminary results of this recent round of testing I have had numerous concerns voiced, one being that the testing was unfair, as it was well known that SONAR X64 sucked, and that I should have tested the 32 Bit x86 version on Vista 64 as well.. ??
That would totally negate the whole reason why I embarked on this 2nd round of testing in the first place, which was to answer the concerns voiced that I should have been using a Native x64/Vista optimized application.., I just can’t win either way.
It has also been voiced that the results could be more
of a reflection of the audio cards/drivers than the O.S itself.
This is the reason I specifically used 2 reference cards this time around to negate those concerns. These are the most respected and widely used low latency cards on the market, so if we are to point the finger at the hardware being the cause of the issue within the Sonar/Vista paradigm, then I'd suggest we are in for a long haul finding some audio hardware that is going to perform in accordance to Cakewalks lofty claims.
I'd suggest the fault lies within Sonar / Vista , and that the so called Vista optimizations so loudly touted by the Cakewalk dev's do not amount to anything past hyperbolics...
There are no smoke and mirrors here, the test session is in the public domain , and I will be more than happy for anyone to post a differing experience..
I’ll close off again with exactly the same sentiment I closed off with last time.
Currently , there is no real advantage
for the vast majority to be going anywhere near Vista for audio application,
and to be brutally honest, I wouldn't be touching it with a barge
pole until all applications / plugins are ported natively across..
, in other words, talk to me in 2008.
*Update : It
has been confirmed that the Vintage Channel Plugin VC-64 is not x-64
Native, ( Go figure - a plugin called VC-64 is NOT Native x64 ) and
that the results are reflective of the performance penalty imposed
by the Bit Bridge.