The Errors That I Don’t See

On the Importance of Checking Your Work

Photo: Annie Sprat on Unsplash

My eyes were bloodshot.  I had spent most of the previous night preparing a set of exhibits that were now being reviewed by my case manager.  This was my first job out of college and I was learning what it was really like to be a litigation consultant.

“Have these been checked?” my manager asked as his eyes scanned each footnote.  “Yes,” I replied.  He immediately shot back, “Then why is the word ‘privileged’ misspelled on the bottom of Exhibit 2?”

Privileged and Confidential.  That was the golden phrase that was supposed to be in the footer of every exhibit we produced for our clients.  Litigation consulting 101.  I had included the phrase, but had accidentally misspelled the word privileged in what was the first mistake of my career.

“Oh, sorry about that.  Just a typo,” I said with a smile as I tried to brush it off.  My manager’s demeanor changed.  “Just a typo?” he said as he glared at me.  “Yeah, sorry.  No big deal,” I replied without hesitation.  I had just taken my first mistake and compounded it.

My manager leaned back in his chair and chuckled.  Though I was only 22 years old, I was about to receive some of the most important advice of my entire career.  My manager said:

Let me tell you a story about typos.  I once worked on a case involving a patent dispute and got invited to watch our attorneys depose the opposing side’s economic expert, Dr. Shaw [name changed].

Though I knew the issues in and out, I didn’t know exactly what our attorneys would ask of Dr. Shaw.  So, this became a kind of guessing game for me.  Would they ask about patent specifics?  Or maybe how Dr. Shaw calculated damages?  No and no.  Our attorneys took a different route:

Lead Attorney: “Dr. Shaw, please go to footnote 4.  What is a ‘transitsor’?”

Dr. Shaw: “That’s a typo.  Should be transistor.”

Lead Attorney: “Okay.  please go to Exhibit 6B.  Can you please define the word ‘proccesor’ in the third column from the right?”

Dr. Shaw: “Typo.  Should be processor.”

Lead Attorney: “Okay.  In Table 2, can you please tell me what a fibre optic cable is?”

Dr. Shaw (now visibly frustrated): “FIBER optic cable.  Do you have anything of substance to ask?”

The lead attorney paused and averted his gaze upward, locking eyes with Dr. Shaw.

Lead Attorney:  “Dr. Shaw, it’s not the errors that I do see that bother me.  It’s the errors that I don’t see.

Chills ran down my spine.  My manager went on to explain that the opposing side lost the case and Dr. Shaw never again worked as an economic expert.

The lesson I learned that day was two-fold:

  1. Build a reputation based on getting things right.
  2. Always check your work. 

Because even the most minuscule of errors call into question the quality of everything else you produce.

This lesson is even more important today in an era of information abundance.  With 2.5 billion GB of information being created each day, we have to continually ask ourselves, “What information can I trust?”  This is why becoming a purveyor of signal in a world drowned by noise is the ultimate reputational asset.  With it, you will easily be able to differentiate yourself from your peers.  How do I know?

Because most people don’t check anything.  For example, I recently saw a Twitter thread (h/t David Perell) detailing how the “40% of Americans can’t afford a $400 emergency expense” statistic was misleading.  Despite this, CNN Money, Forbes, and MarketWatch, among others, quoted it in their headlines unchanged.

Yes, this mistake was small, but there are plenty of examples where the consequences of improper checking have been much larger.  For example, consider how Facebook botched how they measure average video view duration (a key metric for advertisers) for over two years.  Or the Excel error made by two famous economists that affected economic policy across the globe?  Or the fat-finger Samsung dividend distribution that sent the stock crashing 11% in a day?

In all of these cases, important information, affecting billions of dollars, was sent out without being properly vetted.  Anecdotally, I know that this happens because of my friends who have left litigation consulting and now work at various Fortune 100 companies.  Many of them have told me privately, “No one checks my work anymore and it feels so weird.”  I recently experienced this feeling after some work I did for Michael Batnick made it on to CNBC:

Likelihood of seeing negative returns based on how often you check your portfolio.

Historically, the more often you check your portfolio, the more likely you will see negative returns.

Had I known that this was going on television I would have had it checked by someone other than myself.  Not that I made a mistake, but what if I had?

This is why an edge exists.  Because the cost of checking information, in terms of both time and mental processing, is high and getting higher.  And few people want to bear these costs, including me.  This is why I have my Dad check my blog post every single week.

So, if you are willing to put in the work, you can build a reputation where you are known and trusted for your accuracy.  And with more kinds of information becoming more ubiquitous every day, having a reputation for being accurate will only increase in value.  Though pursuing this career strategy is simple, it is far from easy…


The Battle That Never Ends

Despite my insistence on the importance of checking, unfortunately, even I don’t verify every fact I come across.  For example, my most viral tweet ever (which had over 1.2M impressions) was based on a completely fabricated video showing drones coming out of an Amazon airship (here is the original video).  I thought the video was real, but I never checked to make sure.

This might seem hypocritical, but as I have said before, “we all make mistakes.”  And in making mistakes I’ve come to realize that checking your work is the battle that never ends.  You never get to a point where you can stop.  Why?  Because you are human.  As Annie Duke explained in Thinking in Bets, we tend to form beliefs in the following way:

  1. We hear something;
  2. We believe it to be true;
  3. Only sometimes later, if we have the time or inclination, we think about it and vet it, determining whether it is, in fact, true or false.

As you can see, the checking part comes at the end, if at all.  This is what you and I will have to battle against for the rest of our lives.  Day in and day out.  There will be no finish lines, there will be no goal posts, and there will be no shortcuts.  But, if we remember to check, check, and then check again, we might stand a chance.  Thank you for reading!

➤ You can follow Of Dollars And Data via Twitter, Instagram, or my weekly newsletter (Sign up here!)

This is post 122. Any code I have related to this post can be found here with the same numbering: https://github.com/nmaggiulli/of-dollars-and-data

 
Print Friendly, PDF & Email

Full Disclosure: Nothing on this site should ever be considered to be advice, research or an invitation to buy or sell any securities, please see my Terms & Conditions page for a full disclaimer.

OfDollarsAndData.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com and affiliated sites.

17 Responses

  1. Anonymous commented on Apr 30

    You are, of course, right. Checking facts and interpretation are more important than ever. But, as a former newspaper man, I can tell you the last 30 years has proved that people aren’t willing to pay for it.

  2. Anonymous commented on May 01

    Nick, you’re missing a word in the second sentence. You need a “the” between “of” and “previous.”

    Loved the post and the story from your manager. Thanks for sharing.

  3. Anonymous commented on May 01

    Coming out (*of*) an Amazon airship
    Lol just giving you a hard time 😉 good article

    • Nick Maggiulli commented on May 01

      This article had 4 typos when posted. Most of them were not recognized by most people. Our brains automatically fill in what is missing.

  4. Anonymous commented on May 02

    Your examples illustrate the real issue: in many areas, people don’t check their work and/or facts because they have no incentive to. A bunch of media outlets publish an extreme claim without checking whether it’s true because they have nothing to gain by checking if it’s true; they would much rather publish it, get whatever traffic they can, and if it turns out to be false, later say “whoops, sorry, but everyone else published it too so don’t blame us”.

    Facebook inflates their video view metrics and gets paid more by advertisers, at least for a while. Do they suffer any real consequences for this error? I doubt it. Maybe a few refunds to important advertisers, but maybe not even that!

    The Excel-challenged economists did suffer a serious loss of credibility as a result of their error, but only because they eventually released their data and someone actually found the error! Had that not happened, they would have undoubtedly benefited from their mistake, by getting their names on an important and influential study. This error got caught because it was such a high-profile case and the error was in a simple calculation, but I would wager that most such errors are never discovered, either because nobody bothers to check the work, or the error is of a different type (e.g. data gathering) that is harder or impossible to check. So publishing work that is not thoroughly checked might even be a gamble worth taking for many researchers!

  5. Anonymous commented on May 07

    The referred-to Twitter-storm was mostly an argument over semantics. As I read it, the Fed said that >40% of people don’t have enough cash reserves to pay an unexpected expense WITH CASH. Millerd then goes off the rails based on his interpretation of what the phrases “can’t cover,” “can’t come up with,” and “can’t pay” mean. If the word CASH were attached in some way to each phrase, then would that satisfy Millerd’s legalistic interpretation? Furthermore, in Tweet #4, Millerd states:
    “4/ If you look at the 2017 version of the report, you only find this chart which seems to show that only 60% of people can meet an expected $400 charge with cash or equivalent.”
    Of course, the whole discussion is about UNexpected (emergency) expenses.

  6. Anonymous commented on May 12

    Great points, thanks Nick.
    Reminds me of the many space probes going to Mars in the past that have been lost due to some small software error. Such as the Mars Climate Orbiter which failed due to a unit conversion error from imperial to metric. Or the Mars Polar Lander which shut down it’s descent engines too soon and crashed. And it’s not just the US of course, but many other countries have failed at Mars. In fact, there are more failures than successes when sending probes to Mars. Which just highlights the fact that space is extremely unforgiving of any errors.

    Check out this link of the list of Mars missions and relative success vs failure… https://en.wikipedia.org/wiki/List_of_missions_to_Mars

We have the right to delete inappropriate and unfriendly comments.