Got a Phishy Looking URL?
Phishing, spear phishing and business email compromise (BEC)…however you identify the threat, the worry is always the same. As a security analyst how do you know for certain that the link or links in a suspected phishing email are indeed malicious?
There are several tactics you can employ. Conventional techniques include reviewing the MIME header of the email, decoding encoded URLs, or hovering over the link in the URL. All these, however, require some level of expert knowledge to be effective. Not only that, it’s time consuming. Depending on the number of emails you are analyzing, this could take hours of your time every day.
There is a simpler alternative, where you copy the URL link out of the email (without clicking it!) and paste it into a URL scanner. In this blog we are going to look at three popular URL scanning tools that’ll inspect a URL and determine if it is safe:
- CheckPhish.ai
- Urlscan.io
- ScamAdviser.com
We’ll review the pros and cons of these tools to assess their accuracy and usefulness. And along the way, we’ll examine two different approaches for inspecting a suspicious URL: 1) A simple scan to understand whether it is malicious or not and 2) A review of related threat intelligence to understand the context of a rendered verdict
For the comparison, we’ll use a hypothetical scenario to walk through the two use cases. In the scenario a user has reported an email that was delivered by Replit.com (an online, browser-based IDE). The user reports they don’t usually get emails like this and when they hover over the links, things look phishy. (see below).
Figure 1. Example of suspicious URL delivered via Replit.
Real-Time URL Scanning
CheckPhish.ai
While I agree, this URL looks odd and the user said it’s unusual to get an email from them, it’s hard to be certain. By copying out the link (right click > copy link address) and pasting it into CheckPhish.ai we get a verdict in seconds. It’s clean!
But how does CheckPhish.ai know it’s clean? Unlike other URL scanners, the tool actually goes out and analyzes the URL in real time. The backend of CheckPhish.ai is the same as the award-winning Bolster enterprise platform used by companies like Zoom, LinkedIn, and Dropbox. When scanning a URL, the tool launches a headless browser to view the site.
From there, it uses computer vision to identify logos and trademarks and then combines it with natural language processing to understand whether the intent of the site is malicious or not. What’s unique about CheckPhish is that it is not just an aggregation of open source threat feeds that are freely available. It’s a real-time expert analysis of the site with a false positive rate of 1/100,000.
Figure 2. CheckPhish output
CheckPhish also is able to handle scenarios that other scanners seem unable to process. For example, here, we can see a “source” and a “Redirected” URL on the left side, near the top. The redirected URL is the official replit.com domain and the path appears to lead to a careers page. In this instance, the reported email is legitimate and poses no threat. CheckPhish.ai is able to go to the redirected URL and complete the analysis. Not all of the scanners I tested were able to do this.
Scamadviser.com
Performing the same series of steps on ScamAdviser.com appears to lead to failure. It is apparently unable to handle redirected URLs, which is what we are working with in this case. Because of this, ScamAdviser.com is unable to assist in this investigation.
Figure 3. ScamAdviser output
Urlscan.io
Using urlscan.io, we can execute a successful scan on the URL, however we get erroneous results as reported by the Google safe browsing test (Malicious). Urlscan.io aggregates a large quantity of threat feeds, and it appears that Google Safe Browsing had at one point classified Repl.it as a malicious site. Urlscan appears not to have updated their data since the site is no longer classified as malicious by Google Safe Browsing.
Figure 4. Urlscan.io output
Threat Intelligence Gathering
Checkphish.ai
Because of the dynamic nature of URLs and domains, it’s important to gather as much contextual information as you can, which gives you (the analyst) better information to make decisions about the best remediation method. CheckPhish.ai provides threat intelligence that reviews historical data to identify trends or patterns. Using this threat intelligence we can see that the IP address currently hosting replit.com has at one point been involved in a phishing campaign.
You’re also provided all the relevant information needed if you want to take action against the site, for example initiating a site takedown. CheckPhish.ai provides you with the hosting provider, the IP address, and even the number of past phishing sites that have used that same IP address.
Figure 5. CheckPhish threat intelligence
Scamadviser.com
Because ScamAdviser has failed to convert the shortened URL in this demo, I have entered the “Redirected” URL (replit.com/site/careers) obtained from Checkphish.ai for this portion of the investigation.
As we can see here, ScamAdviser provides very little threat intelligence for an analyst to use in their decision making process. It does render a simple verdict, which is accurate, however, they did have the issue where they were unable to handle a redirected URL. Beyond the simple verdict, ScamAdviser does not appear to be designed for serious threat researchers or SOC analysts trying to assess a threat and determine a remediation method.
Figure 6. ScamAdviser threat intelligence
Urlscan.io
Urlscan.io provides an abundance of threat intelligence, but the method in which it is delivered makes it consumable only by more senior security analysts. For example, identifying the threat posed by Javascript Global Variables is a very challenging task that many SOC analysts would not know how to handle. If the primary objective of scanning a URL is to understand whether it is malicious or not, then urlscan.io misses the mark since it rendered an inaccurate verdict. The data it gathers and presents is comprehensive, but it is not all actionable. Most analysts would not find it very useful, and perhaps even misleading since the rendered verdict is not accurate.
Figure 7. Urlscan.io threat intelligence
Summary
We may be biased here, but it appears CheckPhish stands out in terms of both verdict accuracy and actionable threat intelligence. CheckPhish.ai is built on an enterprise platform used by some of the largest companies in the world. It leverages computer vision and AI to quickly determine the intent of a website in real time rather than relying on threat feeds that could be days or even weeks old.
This provides highly accurate, actionable, verdicts which in turn helps incident response teams to take swift action. What’s more it provides historical context that is important for understanding past phishing activity helping security teams better secure their enterprises. Check it out for yourself!
CheckPhish Real-Time URL Scanning: checkphish.ai
