HOME | BENCHMARKS | FORUM| CONTACT |
 
DAWbench - Reference Benchmarks :.

   

PC DAW O.S Shootout : Part II : XP 32 v Vista 64 - SONAR :.

   

Back again for another round of what will become an ongoing series.

Following my original report in regards to DAW performance across the competing O.S platforms, there had been a few concerns raised due to the choice of Audio applications – Steinbergs Nuendo 3.20 / Cubase 4.02 not being Vista optimized, so it wasn’t really a fair reflection on what Vista had to offer as a DAW O.S

With that in mind, I decided to port the methodology across to Cakewalk SONAR 6.2.1 , which had both Native 32 and 64 bit versions of the application, as well as some highly publicized Vista Optimizations, especially in regards to MMCSS -Multimedia Class Scheduler Service:

 
Preparing for Battle :    

Firstly we needed to port the methodology as closely as possible over to SONAR.

I approached the community over at the Cakewalk Forums for a show of hands in helping me with the preliminary development and testing.

The SONARbench DSP test session was developed with the assistance of one of the members there – Tarsier, who developed the session following my directions and methodology.

 

The overall participation and contribution over at Cakewalk forums was disappointing to say the least, but the ball was rolling , so we pressed on regardless.

This time I decide to bypass XP64 and to test only using XP32 and Vista 64, as most if not all Audio developers are concentrating purely on Vista 64 for their Native 64 bit applications.

 

I personally believe XP64 is potentially a far superior 64 bit DAW O.S solution, but unfortunately the dev's are not going to be ringing that bell too loudly, especially Cakewalk, who have been trumpeting the loudest about their Vista Optimizations.

With that in mind , I have streamlined the testing to native 32 Bit applications on XP 32, and Native 64 Bit applications on Vista 64.

         
WDM V ASIO :    

Sonar is one of the few Professional Audio application that use both WDM/KS and ASIO as low latency driver models.

While most other applications use ASIO as their preferred ( and only ) low latency solution, SONAR have always pushed the WDM /KS (Kernel Streaming) driver as their preferred model when available.

Not having a lot of experience with WDM, I stumbled on a few anomalies in profiling the Lynx Audio card that I used as the reference card in my first round of testing, which lead me to further investigate how WDM interacts with the buffer settings within the cards control panel.

I also decided to cross reference the testing this time with a RME HDSP card to further consolidate the results across the 2 driver models.

When using ASIO , the buffer settings that are set in the respective control panels of the audio hardware is reflected in the latency setting that SONAR displays, very simple, streamlined and uncomplicated. With WDM , the latency control is supposedly handle by the application after the card has been profiled , which then determines the appropriate lowest latency setting available-which can then be adjusted by a slider within the application.

What I wanted to understand was what if any correlation did the buffer settings in the control panels have with the lowest available setting within SONAR. With The RME card, the lowest available latency would correspond to the initial buffer settings in the control panel. ie : 032 – 1ms , 064 1.5ms, 128 – 2.9 ms.

That however didn’t guarantee that the system could cope with those settings, but they were available non the less.

 

With the Lynx card, no matter what the initial buffer setting , once profiled the lowest available setting was always 2.9 ms , however , if the initial buffer setting was set to 032/064, the performance was markedly different to the buffer being set to 128 to coincide with the lowest available setting with SONAR. Confusing, you bet.

So despite the WDM profiling always reporting an available 2.9 ms , the initial buffer settings which should have been ignored, were still impacting on the overall performance.

I specifically made the point of 032 sample buffer setting.

I asked Paul Erlandson from Lynx to not only confirm the behavior, but also to help me understand what the correlation is between the 2 settings. The information was delivered during a real-time messenger chat, so I will need to collate and paraphrase the information into a more cohesive format. Any technical discrepancies will therefore be mine, and not Paul's.

“ With ASIO the buffer size in the Lynx Mixer impacts the app's buffer and the transfer size. This is defined by the ASIO spec. With WDM/KS, the app buffer size is controlled by the app, and the buffer size in the Lynx Mixer is actually just the transfer size. The size of chunks that our hardware will swallow at a time.

There is a unique function call for Sonar where we send up what we think these values should be for given sample rates, so, the real buffer size as we're used to thinking about it, is the slider under Mixing Latency. The other values determine the size of chunks they are sending and the size of chunks we're receiving at a time, we do not report back up to Sonar a value for that buffer size, since that should be controlled by the app.

 

The reason setting the buffer size in the Lynx Mixer is not reflected there is because they are two different entities, buffer size vs transfer size,we could do a metric, i.e. 2X the transfer size should be the buffer size, but we don’t. So, when you set the buffer to 32 samples, that's just way too low, especially since we are being mounted as a multi-channel driver under WDM/KS.”

In respect to RME cards following the ASIO Buffer setting..

“It's not the ASIO buffer, but they must do some function call making the Sonar buffer size relative to their transfer size. Ultimately Sonar is in charge of this though. It yields little to play with these transfer sizes, unless your operating on the edge.
In terms of experienced latency, that buffer slider is where the action is, that initial setting is just a suggestion from the driver, nothing more”

O.K, hopefully that has made it all a little clearer .. :-)

With the Lynx profiling only allowing 2.9 ms, which I found quite conservative in respect to the ASIO performance of these cards, I tried to force the profile of the card to at least 064 samples in SONAR to see if I could get a 1.5 ms latency setting.

All seemed fine until I actually tried to play the session. I re profiled the card and the setting was again returned to 2.9ms despite the Lynx buffers being set to 064, so I resigned to the fact that 2.9 ms was the lowest setting that the Lynx card will run at under WDM.

With all that under my belt, I pressed on. Just a small note. I always matched the buffer settings with the respective hardware controls for both cards , to the buffer/latency setting I was testing in SONAR , just to ensure no buffer/transfer mismatches.

 

Round One : SONAR 6.2.1 : Lynx 2 / RME HDSP : XP32 v Vista64 : ASIO :.

I initially ran the tests with just the Lynx card, but I decided to cross reference the results with the RME HDSP this time around, to ensure that the performance variables could not be attributed to a specific audio interface/driver.

I first tested the system under XP 32 using the 32 bit version of SONAR, and as it was the first time I had officially put the test session thru its paces, I made special note of the inherent behavior of running thru the test session. Some interesting points

I wasn't able to scale the system to higher than about 70% CPU loadings before the session stopped playback with a drop out. I couldn't get to the stage of audio break up with crackles and pops, as Sonar stopped playback at the point of a drop out, which is great for determining just when it hits the wall..

 

On the point of the first drop out, I hit stop and started playback again, and then managed to usually load another 2-3 Vintage Channels before the next drop out.

Any attempt at adding any more plugins after the second drop out resulted in playback instantly being stopped by another drop out.., so that's the score I have listed.

On to Vista 64, using SONAR x64 :

Well the numbers tell the story, what can I say except we have a Native 64 Bit Application , with specific Vista Optimizations, and it has totally tanked using ASIO.

On both audio cards tested.


 

Running the test was also nowhere near as smooth as in XP32 , slight glitches as plugins were enabled, which were not apparent under XP32, and also crackles and pops way before the final tank.., which made things even more difficult to get a quantifiable results as once it started glitching, moving back a few plugs to try and clear the pops and clicks, resulted sometimes in having to remove 10 or more.. ?

Onto WDM..

         

Round Two : SONAR 6.2.1 : Lynx 2 / RME HDSP : XP32 v Vista64 : WDM :.    

The results under WDM were less consistent than the ASIO results across the 2 audio cards , so having the cross reference is valuable in gauging a more balanced viewpoint.

Under XP32 the RME and Lynx results were comparable to 2.9 ms /128 samples, with the RME card also being able to return a result at 1.5 ms/064 samples.

The overall results under WDM were better than the ASIO results on both cards.

Moving onto Vista 64 , using SONAR x64 we had few minefields to navigate in regards to the RME card being unable to profile correctly in stereo under Vista 64, the application reporting that only mono playback was available when attempting to profile the card.

 

Playback was just a garbled mess no matter what buffer setting. I was unable to resolve the issue, so I unfortunately do not have a cross reference to the Lynx results.

None the less the Lynx results were enough for me to conclude that the performance on Vista 64 was again abysmal compared to XP32.

So is this a reflection of what to expect from a Native 64 bit application, under Vista 64, or is there more to it.

*It has been suggested that the PlugIns used in the test session, which were all SONAR bundled , were perhaps not x64, therefore we were seeing the performance penalty of using the Bitbridge that enables SONAR x64 to use 32 bit plugins.

 

I have no way of confirming this, and the Cakewalk dev's have not come forward to offer any explanation.

I would have hoped that with the amount of hype ventilated from Cakewalk in regards to their X64 Native version of the application over the last 18 months, that at least the native plugins bundled with the application are x64. ? !*

         
Conclusion :

This rounds off my testing until at least the next batch of application updates, most importantly Steinbergs Native x64/ Vista optimized release later this year. SONAR 7 is also due for release in the next few months, but I have to admit that after my recent experience, I’ll be hard pressed getting excited.

Since I have gone public with the preliminary results of this recent round of testing I have had numerous concerns voiced, one being that the testing was unfair, as it was well known that SONAR X64 sucked, and that I should have tested the 32 Bit x86 version on Vista 64 as well.. ??

That would totally negate the whole reason why I embarked on this 2nd round of testing in the first place, which was to answer the concerns voiced that I should have been using a Native x64/Vista optimized application.., I just can’t win either way.

It has also been voiced that the results could be more of a reflection of the audio cards/drivers than the O.S itself.

 

This is the reason I specifically used 2 reference cards this time around to negate those concerns. These are the most respected and widely used low latency cards on the market, so if we are to point the finger at the hardware being the cause of the issue within the Sonar/Vista paradigm, then I'd suggest we are in for a long haul finding some audio hardware that is going to perform in accordance to Cakewalks lofty claims.

I'd suggest the fault lies within Sonar / Vista , and that the so called Vista optimizations so loudly touted by the Cakewalk dev's do not amount to anything past hyperbolics...

There are no smoke and mirrors here, the test session is in the public domain , and I will be more than happy for anyone to post a differing experience..

I’ll close off again with exactly the same sentiment I closed off with last time.

 

Currently , there is no real advantage for the vast majority to be going anywhere near Vista for audio application, and to be brutally honest, I wouldn't be touching it with a barge pole until all applications / plugins are ported natively across.. , in other words, talk to me in 2008.

Vin Curigliano
AAVIM Technology

Part I , Part III

*Update : It has been confirmed that the Vintage Channel Plugin VC-64 is not x-64 Native, ( Go figure - a plugin called VC-64 is NOT Native x64 ) and that the results are reflective of the performance penalty imposed by the Bit Bridge.

The test session has been revised with x64 Native Plugins only, and I have rerun the tests and updated the performance data in Part III.

I have also detailed the looming minefield of navigating the Bit Bridge and other variations of the same theme *

 

         
   

© AAVIMT 2007