PonoPlayer review: Neil Young’s new streaming device sounds no better than an iPhone.

Ignore the Audiophiles. Neil Young’s PonoPlayer Sounds No Better Than an iPhone.

Ignore the Audiophiles. Neil Young’s PonoPlayer Sounds No Better Than an iPhone.

Innovation, the Internet, gadgets, and more.
Feb. 17 2015 12:48 PM

Out of the Blue and Into the Wack

Neil Young’s PonoPlayer sounds no better than an iPhone—no matter what the audiophiles say.

150217_TECH_Pono
Should you pony up for the Pono?

Photo illustration by Slate. Photo via PonoMusic.

When Neil Young announced his plan to design a better portable music player, it sounded like a marvelous ambition. His Kickstarter appeal quickly lured $6.2 million in hopeful backing, with Young declaring he was on a quest “to revive the magic that has been squeezed out of digital music.” The promise behind his new PonoPlayer was that a music purist and all-around rock ’n’ roll mensch could—through sheer passion and the wonders of modern engineering—offer us a richer audio soundscape than we’d been getting from our iPods and smartphones.

Seth Stevenson Seth Stevenson

Seth Stevenson is a frequent contributor to Slate. He is the author of Grounded: A Down to Earth Journey Around the World.

Young’s Pono player went on sale this January for $399. I bought one immediately. I was planning to review it as a new consumer gadget. Instead, I wandered into a holy war I hadn’t known existed.

Those of us who remember downloading cruddy MP3s in the ’90s know from experience that the quality of a digital music file can make an enormous difference. Young’s announcement had me wondering: Which details, which nuances am I still missing when I listen to songs I’ve bought in the standard iTunes format? Might I achieve a more emotional, more soulful bond with my favorite tunes if I invested in high-resolution digital files and played them through more expensive hardware? Would I hear things I’d never heard before?

Advertisement

For me, these were merely curious questions worth investigating—with the possible bonus that I might delight in music even more than I already do. I soon learned, however, that for certain aural partisans—so-called golden-eared audiophiles on one side, skeptical nonbelievers on the other—the stakes are far higher. Their feud involves schismatic arguments, charges of heresy, and a standoff that has been going on for years.

But let’s just start with the music player at hand.

When my Pono arrived, in a handsome wooden box, I couldn’t wait to hear what it sounded like. I turned it on and eagerly listened to the one song that had come pre-loaded on the device: Neil Young’s “There’s a World,” off the remastered version of his 1972 album Harvest. The track sure did sound great. Beautiful song, faithful playback. Yet something nagged at me. My ears were slaked, but my mind was not blown. Never did I feel I was experiencing a brave new aural world.

To check my hunch, I purchased the same track on my iPhone 6 Plus in standard iTunes resolution. Then I switched back and forth between listening to the “high-quality” track on my Pono and “standard-quality” track on my phone. I used the same headphones with each device—a $130 pair of Polks that suit me just fine. (A note from Neil tucked inside the Pono box assures you that the Pono makes your music sound better no matter what headphones you connect to it.)

Advertisement

I listened as intensely as I could, to appreciate every subtle nuance. I opened my mind to the music, praying for sonic distinctions to emerge. But if there was any difference between the Pono and the iPhone, I simply couldn’t detect it.

I figured maybe my ears were faulty. So I ran an informal experiment. I asked several Slate colleagues to turn their backs while I played them the same 30-second clip on both devices, through headphones. Then I asked them to pick which clip they thought was higher resolution. I mixed up the order—at times playing the Pono first, and at times the iPhone. Some people borrowed my Polk headphones for the test while others used their own equipment. (The quality ranged from Apple earbuds to Klipsch in-ears to high-end Sony studio cans.)

Bottom line: Not one person had any clue whether they were listening to the Pono or to the “inferior” iTunes track. There was zero confidence in determining which was which. When forced to state a preference, six out of seven people actually picked the iPhone as the higher-quality experience. An eighth person refused to guess because he simply had no idea. These folks were in their 20s and 30s, all avid music listeners. A couple of them write about music professionally and one is a video producer.

You may wonder why I used “There’s a World” instead of some more recent jam. Partly, it’s because I assumed the lone track that came proudly pre-loaded on the player—a song by the founder of Pono, no less—would surely exhibit the product at its absolute finest. But also: For two days, I couldn’t figure out how to make the Pono sync with my MacBook Air, so this was the only track I could play.

Advertisement

Let’s get it all out there: In my experience, the Pono was difficult to sync—if the sync worked at all. The device has a kludgy interface, with the touch screen often failing to correctly register my choices. The on/off button is easy to press when you don’t mean to. The Pono’s form factor (people compare its triangle tube to a Toblerone chocolate bar) is rather bulky, and the device tends to jut uncomfortably out of a pants pocket.

Pono’s proprietary music store is also a hassle to navigate, and its high-res albums are more expensive than they are elsewhere. (Pono’s 96-kHz version of Beck’s Morning Phase costs $17.99, while the 44.1-kHz version on iTunes costs $9.99.) The Pono store’s selection is decent, but it seems to have gaps. The first group I searched for, Pavement, didn’t show up at all. After purchasing a song, transferring it from your computer to the player is a nonobvious operation. Anything you’d want to do could be done much easier on iTunes.

For these practical reasons alone, I strongly recommend you not buy this pricy gewgaw—which costs more than many smartphones yet does nothing except play music. More enraging than qualms about tetchy touch screens, though, is the thought that an expensive, high-end, dedicated audio component might not even sound better than playing your music on your phone. And the fact is I’m not the only listener who’s had trouble hearing any difference. In a recent review of the Pono on Yahoo (“Neil Young’s Pono Player: The Emperor Has No Clothes”), columnist David Pogue did an A/B test that was similar to but far more rigorous than my own. Pogue’s testers likewise professed, on balance, a minor preference for listening on an iPhone.

You can question the methodology of our tests. That’s fair. “These things are not designed to be heard through $100 headphones,” a friend who fancies himself an audiophile complained to me in an email. “They’re supposed to be hooked up to USB [digital-to-analog converters] and played on a good stereo system. The Radio Shack A/B box that Pogue used to compare iPhones with Pono has terrible circuitry that degrades the sound of anything hooked up to it.”

Advertisement

OK, fine. But what about the legendary 2007 study published in the Journal of the Audio Engineering Society? Authors David R. Moran and E. Brad Meyer let test subjects listen to recordings of their choosing—jazz, pop, acoustic—over expensive playback components. “People talk about the ‘depth’ and ‘definition’ of high-res audio,” says Moran now, “so we thought it was time that someone actually checked. If it’s so easy to hear and everyone can hear it, let’s do a blind study.” The subjects could switch back and forth between standard “CD-quality sound” (16-bit/44.1 kHz, which is what you’ll find in a lot of iTunes songs you buy today) or “high-resolution sound” (with beefier specs of the sort that the Pono purveys).

The listeners included audio engineers and hardcore audiophiles. It turned out neither they, nor anybody else, could reliably guess which was the higher-resolution recording. “Of course, the excuses we heard were that the people were all deaf, the gear sucked, and so forth,” says Moran. “So we said have at it and do your own test. No one’s been able to demonstrate that high-res sounds ‘smoother’ or ‘deeper,’ or has ‘more resolution’ or ‘more front to back.’ It’s all faith-based, religious words. No one’s been able to show it sounds different. They refuse to submit to fourth grade–level scientific principle.”

Mere mention of the Moran-Meyer study will still make steam shoot out of an audiophile’s ears. He’ll start spewing all sorts of tech-spec talk that’s meant to refute the study’s holdings. It’s easy to get lost in those nerdy weeds. But, for the sake of thoroughness, let’s briefly examine the improved specs that the Pono boasts for itself.

For instance: The iTunes version of “There’s a World” is 16-bit/44.1 kHz (identical to the CD quality of the Moran-Meyer test) while the Pono version of the song is 24-bit/192.0 kHz (typical of high-resolution recordings). What’s the difference?

Advertisement

The kHz number refers to the sampling rate, which is double the tippy-top pitch the recording can reproduce on the treble end.* So a 44.1 kHz sampling rate will equate to a 22.05 kHz top-end pitch. The Pono version indeed captures much higher pitches. But here’s the catch: According to people who study human physiology, 44.1-kHz recordings provide enough upper-end treble to exceed the capabilities of our ears. Anything higher and you’re spending money for sounds your dog can listen to but you can’t. And it’s worth noting that many of the albums Pono sells are only 44.1 kHz, anyway, including Steely Dan’s Aja and Nas’ Illmatic, to name two I searched for at random.

As for the 16-bit versus 24-bit difference, it relates to the “noise floor” of the recording—meaning the quietest noise that you can still hear on the playback. This might be especially relevant if, say, you need to hear the piccolo at the back of an orchestra. Pono does have a lower noise floor, technically. But the dynamic range offered by the 16-bit iTunes Advanced Audio Coding format is already “enormously large,” according to David Ranada, the former technical editor of Sound & Vision.* “It’s good enough to be perfect under most conditions.” And consider that the vast majority of the music the Pono store offers is loud rock—not acoustic chamber music where the quietest sounds really do come into play.

Oh, one last thing: file types. The Pono store sells Free Lossless Audio Codec files that are completely lossless—and also take up a lot of hard drive space, which limits how many of them your Pono can hold. Apple’s AAC format is not 100 percent lossless, but it squeezes files down to a much more compact size. The FLAC file for “There’s a World” is 100.51MB while the AAC file is 6.3MB. Does it matter? Again, you’re getting more information with the bigger file, but it’s not information you can hear. “AAC is extraordinarily good,” says Ranada. “In order to hear a difference between FLAC and AAC, you’d need to generate a special test tone. For any sounds you’d ever actually want to listen to, you’ll hear no difference.”

We could rap about specs all day. If you want to dive deeper—much deeper—into the weeds, I encourage you to read this highly technical, anti-Pono essay that Moran pointed me to. Basically, the whole split boils down to this: People who consider themselves empiricists, and who believe in A/B testing and in the known science of the human ear, are convinced that selling high-res music is a scam that separates fools from their money. One guy I talked to—an Ivy League physics professor who didn’t want me to use his name because he was wary of wading into this fray—used the term “snake oil.” These folks readily concede that older, outdated digital music formats could be audibly grungy, but they all assured me that these days the 16-bit/44.1-kHz AAC files you can find on iTunes are as good as anybody needs. By the way, they also take mischievous pleasure in noting that most of the self-proclaimed “golden-eared” high-res warriors are men who’ve reached an age at which hearing typically declines to an immense degree. “The human capacity for self-delusion is infinite,” says Ranada.

But listen closer: Do you hear the battle inching near? The rival squadron is homing in, with your humble reviewer caught in the crossfire.

“These people have an obstinate desire to stick with 1980s technology and declare it perfect,” says Michael Fremer, a contributing editor at Stereophile magazine, the editor of AnalogPlanet.com, and a guy who swears by the virtues of high-res music. “I don’t understand their mentality. If you can get a bigger file that captures more of the music, why wouldn’t you want that?”

For Fremer, assessing music with A/B testing is a farce. “You can’t switch back and forth and tell the difference. You need to listen for an hour and see how it makes you feel. Music is about emotion. You can listen to high-res music for hours and hours and enjoy it. Listen to a standard-resolution CD for 15 minutes and you get bored—it doesn’t compute.”

People like Fremer (read his frothing response to an recent anti-Pono Gizmodo piece) have a long list of excuses to explain away ordinary people’s inability to hear differences with the Pono. They’ll blame your “cheap” $100 headphones. They’ll retreat into ever-wonkier technical concepts like “aliasing” and “brick-wall filters.” They’ll pity you for having grown accustomed to lower-res, inferior digital formats: “It’s like you’ve been eating McDonald’s your whole life,” they’ll say. “Of course you can’t appreciate a great steak.”

And I feel for them. I want to believe. Nothing made me sadder than the moment when, just after I told Fremer about my Pono-debunking tests, he said with a sigh, “Well, if you didn’t hear it, you didn’t hear it.”

Besides, the audiophiles aren’t wrong, per se. It’s all about what makes you happy. An ancient, scratchy record wobbling on a turntable might sound warmer to you than a newly unwrapped CD. A decades-old tube amp might please your ears in a way that a brand-new solid-state amp never could—even though the newer amp has loftier specs. All you can do is be true to your audio self.

So maybe you get off on listening to high-res lute music through $600 headphones in a soundproofed room. By all means, if you feel you require this level of fidelity, and further believe you possess a hypersensitive sonic palate, go ahead and spend your cash.

If, on the other hand, you’re like me … or like everyone else I let listen to my Pono … or if you plan to play pop songs through $60 headphones, maybe even while riding in a loud subway car … buying a Pono means paying for sounds that you’ll never hear.

*Correction, Feb. 17, 2015: This article originally misspelled David Ranada’s last name. (Return.)

*Correction, Feb. 20, 2015: This article originally stated that sound files’ kHz figures refer to the highest pitch the recording can reproduce on the treble end. They in fact refer to the sampling rate, which is double the kHz of the treble top end in the recording. (Return.)