Probing the Whole Internet for Weak Spots

Rapidly scanning the Internet has become vital to efforts to keep it secure.

When a major flaw in the encryption that secures websites was revealed this March, Zakir Durumeric, a research fellow at the University of Michigan, was the first person to know how serious it was. By performing a scan of every device on the Internet, he realized its full potential even before the researchers who had first identified the flaw, known as FREAK.

“There were questions as to the correct way to respond before we did the scan,” says Durumeric.

The scan showed that more than five million sites were affected, including those operated by the FBI, Apple, and Google. Facebook’s like button, a fixture on many popular sites, was also vulnerable. The results prompted an urgent, careful effort to inform key companies and organizations before the problem was announced publicly.

The FREAK flaw allows an attacker to break a secure connection between a Web browser and a vulnerable site, gaining access to encrypted data sent between the two. The attack works by forcing a site to fall back to a weak form of encryption mandated by the U.S. government in the 1990s.

Durumeric leads a team of researchers at the University of Michigan that has developed scanning software called ZMap. This tool can probe the whole public Internet in under an hour, revealing information about the roughly four billion devices online. The scan results can show which sites are vulnerable to particular security flaws. In the case of FREAK, a scan was used to measure the scale of the threat before the bug was publicly announced.

The ZMap team was contacted by Matthew Green, an assistant professor at Johns Hopkins University who had been alerted to FREAK by its discoverers, a team of researchers from Microsoft, the French Institute for Research in Computer Science and Automation, and Madrid’s IMDEA Software Institute.

Green says the scan results helped him decide who needed to be tipped off, ensuring the announcement wouldn’t leave large swaths of the Internet at risk. “We haven’t had really good data like this before,” says Green. “You can find out exactly who’s broken, and tell people exactly how bad something is. It was when Zakir did that scan I knew this was bad.”

Durumeric and colleagues developed ZMap late in 2013. Before that, the software used to scan the Internet took weeks or months to finish the job. “Existing tools were a thousand times too slow,” says Durumeric.

The first high-profile project for ZMap was tracking the impact of the Heartbleed bug, a flaw in a widely used piece of Web encryption software found in April 2014 (see “Many Devices Will Never Be Patched to Fix Heartbleed”). The researchers scanned regularly for systems vulnerable to the bug, and published a site listing the most popular unpatched websites along with information on how to fix the problem.

Durumeric says this effort helped pressure companies into fixing their systems. The group even sent automated e-mails informing companies that they had vulnerable infrastructure and offered guidance on what they should do. Controlled experiments showed that the notifications made a measureable difference, says Michael Bailey, a professor at the University of Illinois at Urbana-Champaign who also works on the project.

The team plans to issue similar notifications for FREAK soon. It is also using scans to track how long it takes for FREAK and similar major flaws to be mopped up. Almost a year after Heartbleed’s disclosure, says Durumeric, about 1 percent of the top one million websites are still vulnerable to it.

One reason well-known bugs linger is that companies fail to realize the extent of the problem, says HD Moore, chief research officer with security company Rapid7. Moore uses ZMap for his own scans. “Most enterprises are completely unaware of at least 10 percent of their assets on the public Internet,” he says. ZMap scans can help companies find vulnerable infrastructure.

Moore began scanning the Internet using software of his own design in 2012 (see “What Happened When One Man Pinged the Whole Internet”). He now runs a more formal scanning project at Rapid7, using ZMap as well as tools developed inside the company.

Green says that Google has also begun to perform its own Internet scans. The results are used to program the Chrome browser to connect more cautiously with sites that pose potential security risks, he says.

However, tools like ZMap can’t find everything. The software works by systematically contacting every possible numerical address for Internet devices using the most commonly used protocol, called IPv4. That misses the tiny but growing fraction of devices using addresses under a newer system called IPv6, which has too many possible addresses to scan comprehensively. ZMap’s scans also can’t reach inside private networks, such as corporate intranet sites, or devices on mobile networks.

Still, Green says, ZMap and other scanning software provides a much needed, if sometimes gloomy, picture of the state of Internet infrastructure. “We’re getting better all the time, but from a very bad place,” he says.

Gain the insight you need on emerging technologies at EmTech Digital.

Register today

View the original article here