People

A Theranos lab worker says blood tests were like ‘flipping a coin’

The testimony from whistleblower Erika Cheung could form a crucial piece of the prosecutors' fraud case against former Theranos CEO Elizabeth Holmes.

The former Theranos headquarters in Palo Alto.

The former Theranos headquarters in Palo Alto.

Photo: Andrej Sokolow via Getty Images

Did Theranos' blood-testing technology work? That was the key question prosecutors hammered away at as the fraud trial of former CEO Elizabeth Holmes continued Wednesday in a San Jose courtroom.

The company's proprietary Edison machines routinely failed quality control tests to the point that former lab employee Erika Cheung said she sometimes refused to run patient samples on the devices, she testified in court.

Patients "don't know the fact that behind closed doors we're having all these problems, and they think they're getting correct results," Cheung said. "It was starting to get very, very uncomfortable and very stressful for me working at the company."

Holmes now faces charges of fraud and conspiracy to commit fraud against investors and patients who paid directly for their tests. Cheung's testimony for the prosecution laid out the case that Theranos devices failed often and were not accurate compared to traditional blood-testing machines, which could prove crucial to the prosecution's claims that Holmes touted technology she knew didn't work — if they can prove she was aware of the problems.

Cheung eventually left Theranos and is now known as one of the whistleblowers who first sounded the alarm about the company. She is the second witness in the blockbuster case, following testimony from the company's former controller about the company's financials .

In March 2014, 25.6% of quality control runs on Edison devices failed, according to a spreadsheet shown in court. For one of the thyroid tests, the failure rate in quality control runs was 51.3%, according to the data.

"Was March an especially bad month for [quality control] at Theranos?" asked Assistant U.S. Attorney John Bostic.

"No, this was pretty standard," Cheung testified.

Erika Cheung Erika Cheung, a Theranos employee turned whistleblower, speaks at a film-industry event in 2019. Photo: Jeff Kravitz/FilmMagic for HBO/Getty Images

She called the level of failures "immensely concerning," since labs would want to see less than 1% failing. While Theranos did remove machines that were failing control tests from the lab until they were fixed, Cheung said that the level of problems that she found running the tests in the research lab made her concerned about the results patients would ultimately get.

"You would have about the same luck flipping a coin on whether your results were right or wrong," she said, describing her concerns about the thyroid test.

She also testified that the company had a habit of "cherry-picking data" and deleting outlier information. There was no standard for what outlier data was so the company would often delete two out of six in tests, typically when something was wrong, she said.

"There were no rules. There were no protocols," she said. (The defense did show later some of the company's validation reports that did disclose when outliers had been deleted or that contained all data.)

The company also prepared proficiency tests that compared the Edisons' performance to off-the-shelf testing devices. In the test, vitamin D levels measured by the Theranos Edison were three times higher than what the standard devices measured, Cheung said. On a different test, the company re-ran the numbers a second time, she testified. On the off-the-shelf devices, the values stayed the same each time it was run, but on the Theranos devices, it reported new values for the same blood, according to Cheung.

Cheung said she had tried communicating the challenges to many people at the company, including Sunny Balwani, whom she met with directly. Balwani, who is also facing fraud charges related to his time as chief operating officer at Theranos, is set for a separate trial in January. Cheung said Balwani seemed irritated by her discussing it and told her she should just run the tests.

She also had dinner with George Shultz, a Theranos board member and grandfather to Tyler Shultz, one of Cheung's co-workers. Cheung had shared data with Tyler Shultz that supported some of her concerns about the lab quality. Tyler Shultz ended up emailing Holmes with a detailed letter of his concerns about the company, but Cheung said she never contacted Holmes directly herself.

Holmes' defense has tried to distance the CEO from challenges in the lab and show they were the responsibility of a lot of people other than Holmes. In their cross-examination of Cheung, they asked her to explain who reported to whom at Theranos. A frequent question was whether a given Theranos employee had a Ph.D. (a degree neither Holmes nor Cheung have).

Cheung ultimately left Theranos in 2014. She had begun talking to a Wall Street Journal reporter the following year when she noticed a black SUV outside of her new employer. When she left work, a man jumped out of the SUV and hand-delivered a letter from David Boies, Theranos' attorney, warning her against sharing trade secrets and defamation or else it would pursue litigation. Undaunted, Cheung said she called the Centers for Medicare and Medicaid Services, which regulates laboratory testing, and ended up filing a letter with the agency outlining her concerns.

Beyond discussing the letter from Boies, Cheung wasn't asked much about her experience as a whistleblower. The judge blocked the prosecution from allowing Cheung to talk about her current job of running a nonprofit called Ethics in Entrepreneurship.

Her testimony will continue on Friday.

See Protocol's in-depth coverage of the Theranos trial for more.

Fintech

Judge Zia Faruqui is trying to teach you crypto, one ‘SNL’ reference at a time

His decisions on major cryptocurrency cases have quoted "The Big Lebowski," "SNL," and "Dr. Strangelove." That’s because he wants you — yes, you — to read them.

The ways Zia Faruqui (right) has weighed on cases that have come before him can give lawyers clues as to what legal frameworks will pass muster.

Photo: Carolyn Van Houten/The Washington Post via Getty Images

“Cryptocurrency and related software analytics tools are ‘The wave of the future, Dude. One hundred percent electronic.’”

That’s not a quote from "The Big Lebowski" — at least, not directly. It’s a quote from a Washington, D.C., district court memorandum opinion on the role cryptocurrency analytics tools can play in government investigations. The author is Magistrate Judge Zia Faruqui.

Keep Reading Show less
Veronica Irwin

Veronica Irwin (@vronirwin) is a San Francisco-based reporter at Protocol covering fintech. Previously she was at the San Francisco Examiner, covering tech from a hyper-local angle. Before that, her byline was featured in SF Weekly, The Nation, Techworker, Ms. Magazine and The Frisc.

The financial technology transformation is driving competition, creating consumer choice, and shaping the future of finance. Hear from seven fintech leaders who are reshaping the future of finance, and join the inaugural Financial Technology Association Fintech Summit to learn more .

Keep Reading Show less
FTA
The Financial Technology Association (FTA) represents industry leaders shaping the future of finance. We champion the power of technology-centered financial services and advocate for the modernization of financial regulation to support inclusion and responsible innovation.
Enterprise

AWS CEO: The cloud isn’t just about technology

As AWS preps for its annual re:Invent conference, Adam Selipsky talks product strategy, support for hybrid environments, and the value of the cloud in uncertain economic times.

Photo: Noah Berger/Getty Images for Amazon Web Services

AWS is gearing up for re:Invent, its annual cloud computing conference where announcements this year are expected to focus on its end-to-end data strategy and delivering new industry-specific services.

It will be the second re:Invent with CEO Adam Selipsky as leader of the industry’s largest cloud provider after his return last year to AWS from data visualization company Tableau Software.

Keep Reading Show less
Donna Goodison

Donna Goodison ( @dgoodison ) is Protocol's senior reporter focusing on enterprise infrastructure technology, from the 'Big 3' cloud computing providers to data centers. She previously covered the public cloud at CRN after 15 years as a business reporter for the Boston Herald. Based in Massachusetts, she also has worked as a Boston Globe freelancer, business reporter at the Boston Business Journal and real estate reporter at Banker & Tradesman after toiling at weekly newspapers.

Image: Protocol

We launched Protocol in February 2020 to cover the evolving power center of tech. It is with deep sadness that just under three years later, we are winding down the publication.

As of today, we will not publish any more stories. All of our newsletters, apart from our flagship, Source Code, will no longer be sent. Source Code will be published and sent for the next few weeks, but it will also close down in December.

Keep Reading Show less
Bennett Richardson

Bennett Richardson ( @bennettrich ) is the president of Protocol. Prior to joining Protocol in 2019, Bennett was executive director of global strategic partnerships at POLITICO, where he led strategic growth efforts including POLITICO's European expansion in Brussels and POLITICO's creative agency POLITICO Focus during his six years with the company. Prior to POLITICO, Bennett was co-founder and CMO of Hinge, the mobile dating company recently acquired by Match Group. Bennett began his career in digital and social brand marketing working with major brands across tech, energy, and health care at leading marketing and communications agencies including Edelman and GMMB. Bennett is originally from Portland, Maine, and received his bachelor's degree from Colgate University.

Enterprise

Why large enterprises struggle to find suitable platforms for MLops

As companies expand their use of AI beyond running just a few machine learning models, and as larger enterprises go from deploying hundreds of models to thousands and even millions of models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

As companies expand their use of AI beyond running just a few machine learning models, ML practitioners say that they have yet to find what they need from prepackaged MLops systems.

Photo: artpartner-images via Getty Images

On any given day, Lily AI runs hundreds of machine learning models using computer vision and natural language processing that are customized for its retail and ecommerce clients to make website product recommendations, forecast demand, and plan merchandising. But this spring when the company was in the market for a machine learning operations platform to manage its expanding model roster, it wasn’t easy to find a suitable off-the-shelf system that could handle such a large number of models in deployment while also meeting other criteria.

Some MLops platforms are not well-suited for maintaining even more than 10 machine learning models when it comes to keeping track of data, navigating their user interfaces, or reporting capabilities, Matthew Nokleby, machine learning manager for Lily AI’s product intelligence team, told Protocol earlier this year. “The duct tape starts to show,” he said.

Keep Reading Show less
Kate Kaye

Kate Kaye is an award-winning multimedia reporter digging deep and telling print, digital and audio stories. She covers AI and data for Protocol. Her reporting on AI and tech ethics issues has been published in OneZero, Fast Company, MIT Technology Review, CityLab, Ad Age and Digiday and heard on NPR. Kate is the creator of RedTailMedia.org and is the author of "Campaign '08: A Turning Point for Digital Media," a book about how the 2008 presidential campaigns used digital media and data.

Latest Stories
Bulletins