Blog

Hands-On Testing and Analysis

Disagreeing without being Disagreeable

Any one that knows DeepStorage knows that we’re opinionated and that we’re always willing to have a reasonable debate about our opinions. Unfortunately, as we’ve discovered in politics recently, some people take anything you say about them that isn’t a straight up compliment as an attack that must be responded to with as much force as possible.

Again, as in politics today, these people also have “Alternative Facts” insisting that their opinions must be accepted as facts, not the opinions they clearly are. A reasoned debate, however, has to have participants that can agree, as the great Daniel Patrick “Pat” Moynihan famously said, “’Everyone is entitled to his own opinion, but not to his own facts.”

A reasonable debate also requires the parties to avoid ad hominem attacks on the other party personally. We can debate positions all day and disagree without being disagreeable, but once you make it personal, there’s little chance of actually shedding light, rather than heat on the matter at hand.

So after writing what we thought was a sober comparison of the acquisition costs of comparable Nutanix clusters and Pure FlashStack converged infrastructure systems, Nutanix Blogger Josh Odgers wrote a 7000+ word screed attacking not only the content of the paper but our motivations and integrity.

I generally choose to take the high road on these things. I didn’t respond publically when Mr. Odgers insisted that my thoughts on data locality were a direct attack on Nutanix because any reference to locality in HCI MUST be about Nutanix.

We’re Not Perfect

Mr. Odgers did point out one factual error in our report. We misread the Nutanix spec sheet and used the power consumption of a block (4 nodes) in our calculations as a per node value. This does mean the Nutanix solution uses less power than the Flashstack.

As this was intended as just additional information for the cost of acquisition study, we did not include the cost of power in the main charts that compare the two systems’ cost.

After correcting our spreadsheet, power savings from the Nutanix solution amount to about 6% of the purchase price of the Nutanix cluster over 3 years assuming power costs at 20¢/KWH and a PUE of 2.0.

We regret the error, and we’ll be posting a revised version of the paper next week (week of Aug 21, 2017)

Facts Opinions and Assumptions

The Nutanix folks argue that any price comparison should be based exclusively on facts and be free of opinion. That would be nice, but there are always underlying assumptions, every TCO study assumes some number of person-hours at some hourly rate for the cost of labor. If the analyst assumes a $150,000 Ph.D. has to perform some task that will have a different effect on the bottom line than if he thinks a $50,000 tech can handle it.

I’m not going to present a full defense of the paper here. Frankly, a company that’s declared we should just “hang it up” doesn’t deserve the courtesy. I will point out two of our underlying assumptions, which like all assumptions are based on experience and opinion, that the Nutanix folks claim are factually wrong:

AHV

The Nutanix folks argue that our choice of vSphere was a factual error and that we should have instead chosen Nutanix’s own AHV hypervisor. After all, they argue, 23% of their customers now use AHV.

This places Nutanix in the curious position of arguing that over three-quarters of their customers are also making this factual error. We decided a cost analysis should follow the other 77% of Nutanix customers who use vSphere or Hyper-V instead of AHV.

Since Nutanix phrases it as 23% of customers, not 23% of workloads, nodes or clusters we assume that customers with mixed configurations like vSphere in production and AHV in test or even vSphere as their only hypervisor but Nutanix storage only nodes, which run AHV, are included.

RF3

It is my firmly held opinion that important workloads must be protected by at least a N+2 redundancy scheme. I’ve seen too many double drive failures and read errors on rebuilds to trust N+1. A few years ago at VMworld John Troyer had a game show format session. One question was “What’s the default FTT for VSAN”? The answer was “FTT=1, but @DeepStorageNet says it should be FTT=2”.

The paper lays out our logic and clearly identifies it as an opinion: “While each administrator must determine what level of resiliency he or she requires for their workloads, as relatively paranoid types our own standard for mission-critical workloads is that the storage system is able to survive…”

Again the Nutanix folks are presenting their opinion as fact by saying that RF2 (their N+1 2-way replication or single parity erasure coding method) is better than Pure’s N+2 RAID6ish approach and that our choosing RF3 could only be “to try and make the article sponsor’s product look more favorable.” not a long held opinion that’s common among storage professionals.

The Reaction

The response from Nutanix, a publically traded company, seemed a bit out of scale to me. Mr. Odgers can be a bit sensitive and I don’t view him as an official spokesperson for the company. I’m a big boy and I can take a bit of misdirected animus,

but then Nutanix’s President decided to chime in and tell the world DeepStorage should “hang it up”.

Is Nutanix’s official position, expressed publically by their President, and retweeted by a minimum of 10 Nutanix employees, that DeepStorage should “hang it up”?

I don’t really understand what he meant by “hang it up.” It could mean he would prefer that we stop writing about Nutanix, or he could mean that he would prefer that DeepStorage simply go out of business. Either way, it’s a curious thing for the President of a publically listed company with a multi-billion dollar market cap to say about a journalist/analyst in their field.

I’d appreciate some clarification.

Clearing up a True, not alternate, fact

Mr. Odgers’s summary concludes:

It’s important to note that even after DeepStorage has been made aware of many issues with the three reports, not a single item has been revised (at the time of writing).

This statement is simply untrue, and Mr. Odgers knows it. We did revise the Atlantis report that Mr. Odgers decided was an attack on Nutanix because we referred to Nutanix obliquely with “2.5 times the mailboxes of a leading HCI provider’s ESRP report” and “Five times the IOPS of that HCI provider’s system.

My response to Mr. Odgers at the time, which we know he read because he quoted it, said:

“We don’t pretend to be perfect, and after Josh’s post led us to take one more careful read of the report, we did identify some -things that needed correcting:”

The revised report was posted less than a week later. While we didn’t make all the changes Mr. Odgers wanted, to say we made no changes is untrue.

 

 

  • Just a note that I’ve muted Mr. Rogers on twitter.

  • == Disclaimer: Pure Storage employee ==



    Well said Howard.



    I can share from experience that vendors simply have a hard time accepting anything less than a glowing review of their technology. This is exceptionally hard when the review comes from a respected and knowledgeable analyst like yourself (and others). Many individuals pour their blood, sweat and tears into the products and services they bring to market and as such, critical or question reviews can be a challenge to come to grips with.

    

In the past, when on the receiving side of a less than desired review, it has been my experience to engage with the analyst / blogger / reviewer in an attempt to better understand one another. Often the analyst receives a better understanding of the technology, or technical goal and the vendor receives valuable product guidance and market insight.



    Nutanix had such an opportunity with this report and instead they chose an ad hominem attack; an attempt to discredit the author’s credibility.



    Why build bridges when one can simply blow them up? ;)



    There’s a lot of great people at Nutanix, they’ve founded the HCI market and serve their customers well; however, there’s something wrong with a culture that attacks analysts (Howard), pulls kit from analysts when results are below expectations (storagereviews.com 2015), and has a track record of sexism (women in bikinis on a Ferris wheel at .NEXT in 2016 and the ‘Nix Vblock’ online campaign of 2014).



    For the sake of the industry which we all share – vendor and analyst alike – I’m hoping all who read this understand here’s a lot to gain in taking the high road; to work with those who see your products and the market in a different light than you do.

Keep up the great work Howard. You’re one of the best and the industry appreciates your candid insight.


    — cheers,
    
v

    • Dallas

      An attempt to discredit an author’s credibility isn’t an ad hominem if it directly relates to inaccuracies in statements and biases or incentives that may cause him to make false or misleading statements. An ad hominem would be if somebody slammed the culture of a company and used that to suggest that the arguments of said company are invalid or incorrect.

      • And yet, the obvious and explicit bias of the person complaining isn’t relevant at all? The only person in this conversation wearing a logo….isn’t Howard. So when talking about inaccuracies in statements and biases or incentives that could cause someone to make false or misleading statements, why is it that you’d trust someone whose entire livelihood, income and personal brand are fixed on one specific vendor versus someone with the experience and independence of Howard?

        Of course since we don’t have any idea who you actually are, we have to guess at your biases as well, don’t we?

        • Dallas

          I never said it wasn’t and I never said I trusted him more, but he is at least he’s wearing his bias on his sleeve. One could even make the claim that Howard’s logos are hidden and can be bought. Independence does not mean without motive or bias. I ask you is it so hard to consider that Howard has just as much on the line as Josh? I do applaud his effort to respond to some of the points Josh made, but ya know muting the opposition doesn’t give much faith that he is truly unbiased.

          Of course you should consider what my biases might be. I don’t really care, I probably won’t be back on this site ever. I simply stumbled upon this pissing match through twitter and thought the way some of you were acting was ridiculous. Specifically, the one decrying ad hominem fallacy when committing it themselves to discredit the arguments of the other side. If anything Pure Storage should be more ashamed with the way their employee acted than Nutanix. At least most of Josh’s comments were pertinent to the discussion.

  • It’s sad that for many organizations, comparison and debate is seen as criticizing “the message”. It’s what EMC used to do. I’ve seen this personally with many attacks, while at the same time actually been doing work for other EMC divisions.

    In this instance, the initial report has to be put in context. All vendors want to put their products in the best light and will always show benchmarks and other testing in a way that favours them. It’s par for the course. When another vendor does a comparison, end users are intelligent enough to know what the implications of a sponsored report are; there’s a natural bias from the sponsoring company to pick a subject or area they feel their product will show stronger than the competition. However the idea here is not (just) to score points, but at least to get end users/customers to think a bit more about both their requirements and the implications of their technology choices, from a capex and opex perspective.

    Considered rebuttal is fine, but when we start getting into personal insults on the validity of someone’s technical ability, you better be ready for the slings and arrows to be coming your way. Josh Odgers is bringing his (publicly traded) company into disrepute, whether on his personal blog or not. There’s no distinction here just because the URL doesn’t have “nutanix” in it.

    I always believe there are consequences to actions. In this case I hope the report and response are shared widely so customers can get an idea of what kind of company they would be potentially buying from, when looking at Nutanix technology.

    • Dallas

      You have too much faith in the end user. There is a reason why marketing exists. If my company was the target of the same freelancer in three different reports each misrepresenting my side I would probably be pretty pissed too. I honestly think Josh showed decent restraint only targeting Howard’s expertise and then backing it up with his side of the story (not an ad hominem attack). Restraint would be appropriate the first time and maybe even the second, but three hit pieces by the same writer is ridiculous.

      • Actually for many years I was the end user. I’ve seen the tricks and tactics vendors use to make their RFP responses more attractive. I’ve worked with purchasing to force vendors to make bids more equitable. In my experience, (large enterprise), customers are more savvy than you think. Let them make decisions about the technical validity of reports, but take the personal attacks out of it. Otherwise we can start question judgements like this – https://www.nutanix.com/2016/06/24/next-in-vegas-and-the-wheel-at-the-party/ from a more personal perspective.

      • 1 – Makes oblique reference to a Nutanix report, less than 20 words in 1000+ word report.
        2 – Locality discussion in the abstract, doesn’t mention Nutanix by name, discusses other approaches including some elements of Simplivity

        These are hit pieces? They certainly weren’t intended as so.

  • ThomasLaRock

    Opinions and discussions are always welcome by rational adults. Attacking the person instead of focusing on the debate is what trolls do best. Thank you for writing this post and reminding people how to be professional.