Intel Wrestled With Chip Flaws for Months
By Ted Greenwald and Jack Nicas
The disclosure of security flaws in computer chips dealt Intel Corp. what seemed like a sudden crisis, but behind the scenes it and other tech companies and experts have been grappling with the problem for months.
Intel, its rival Advanced Micro Devices Inc. and chip designer ARM Holdings this week said some of their processors have security holes that could be exploited by hackers, a problem that affects a variety of chips in computers, smartphones and other devices but so far hasn't been associated with any breaches.
The chip makers and their customers and partners including Alphabet Inc.'s Google, Amazon.com Inc., and Microsoft Corp. had been racing to address the vulnerabilities even before word of their existence leaked out this week, with a coalition of large tech companies working together to protect their servers and issue patches to customers' computers and smartphones.
The flaws, which could allow hackers to pilfer sensitive information like passwords, affect most modern chips from an array of companies. But Intel -- which has a dominant share of the market for chips in servers and personal computers -- was most directly affected, with its stock falling Thursday for the second consecutive day.
On June 1 last year, a member of Google's Project Zero security team notified Intel and other chip makers of the vulnerabilities. Even with the lead time, Intel and others are still trying to plug the security gaps. One issue is getting security updates to billions of devices. Another is that some security patches could slow performance, as the flaws affect chip features designed to speed up processors.
Companies as of Thursday said they hadn't seen evidence of attacks that exploit the flaws. Should hackers manage to do that, they are likely to compromise chips made by Intel. That is because it is the world's leading maker of microprocessors and the vulnerabilities are inherent in its chips going back at least a decade.
The issue called into question the security of Intel's products, and requires many customers to take action to protect their systems. It also highlights how the growing complexity in chips and the software that runs on them makes them difficult to lock down and allows them to harbor flaws that go undetected for years.
"People are re-evaluating core tenets about the security properties of modern hardware. A lot of our assumptions were violated and need to be reassessed," said security researcher Kenn White.
Nonetheless, the impact on Intel is likely to be minimal, said Jim McGregor of the technology strategy firm Tirias Research. "When you control that much of the market and you're that prevalent with customers, you get away with it," he said.
Intel said it expects by the end of next week to have issued software updates for more than 90% of processors introduced in the past five years. Other vendors already have issued patches. Google said Thursday it developed a patch with "negligible impact on performance." Intel said any performance degradations would be minimal for average users and diminish over time, adding that it plans redesigned chips within a year. The company said it didn't anticipate any financial impact from the security issue.
Intel shares fell to $44.43 in Thursday trading, leaving them down 5.2% from Tuesday's close, before news of the flaws emerged. Shares in rival chip makers including AMD and Nvidia Corp. rose.
Researchers have been closing in on the vulnerabilities for more than a year. In August 2016, at the Black Hat cybersecurity conference in Las Vegas, researchers Anders Fogh and Daniel Gruss demonstrated early signs of the flaws. Mr. Fogh wrote a blog post about them last July, which helped inspire other researchers to investigate.
Meanwhile, Jann Horn of Project Zero, a security-research team inside Google, already had uncovered the issue and notified Intel. Eventually three other teams of researchers from around the world contacted Intel about the same issues, and Intel connected them to compare notes and write joint papers.
Daniel Genkin, one of those researchers, said that they found the vulnerabilities in chips from as far back as 2010, and believe the issues go back even longer. "The general architectural principles that make this possible are decades old," he said. "I didn't try booting up a computer from 1995 and seeing what happened, but the principles are there."
Mr. Fogh said researchers simultaneously converged on the same longstanding flaws because previous research had laid the groundwork, including his 2016 presentation. Why didn't Intel spot the flaws earlier? "Good question. I can't really answer that," he said.
Intel is always looking for ways to improve the security of its products, said Stephen Smith, vice president in data center engineering.
While Intel worked with partners to resolve the issue, Intel Chief Executive Brian Krzanich in November sold $25 million of his Intel stock and options, according to regulatory filings, retaining 250,000 shares, the minimum the company requires him to own, according to the company's most recent proxy statement.
An Intel spokesman said the divestiture was unrelated to the security issue and followed a prearranged plan with an automated sales schedule.
Intel and a larger consortium of tech firms and researchers planned to disclose the issue on January 9. But this week, the tech-news site The Register broke the story by tracking discussions online between programmers studying patches for the flaws.
Intel, AMD and ARM Holdings gave some large customers advance notice, but the revelations caught smaller firms off guard. Josh Feinblum, chief security officer of cloud-computing provider DigitalOcean Inc., learned of the vulnerabilities late in the weekend from peers. He said he has been patching his systems as rapidly as possible and has been satisfied with the results so far.
Linus Torvalds, who programmed the core of Linux, blasted Intel for failing to recognize the problem and for initially saying the chips were operating as intended. "I think somebody inside of Intel needs to really take a long hard look at their CPU's, and actually admit that they have issues instead of writing PR blurbs that say that everything works as designed," he wrote on Wednesday to a mailing list for Linux programmers.
Mr. White said that as messy as Intel's disclosure has been, the company "probably did as good a job as possible" given the nature and scope of the problem. Researchers have said some of the patches required "creating new areas of computer science," he said. "There's so much complexity, it's mind-boggling."
The disclosure has sparked worries that attacks based on the newly revealed vulnerabilities could have been happening undetected for years. Some technologists say this is unlikely due to the complexity of the hacking techniques required. For instance, Google said it has been unable to design a way to exploit the flaws in Android phones, though it is still issuing security patches for those devices because of the risk someone may be able to.
Others say that if so-called White Hat hackers -- who spot security gaps so they can be fixed -- found the flaws, their Black Hat counterparts could, too.
Mr. White said that with the information now out there, it is incumbent on users to protect themselves. "These are very sophisticated attacks that require all kinds of background and understanding," he said. "But once it's out, it will be turned into attack code."
Corrections & Amplifications
This article was corrected on Friday, Jan. 5, 2017 at 09:52 a.m. ET because the original incorrectly said he sold $24 million in stock and options. Intel Corp. Chief Executive Brian Krzanich in November netted nearly $25 million by selling Intel stock and exercising options, according to regulatory filings.