Despite extensive research by the security community, cyberattacks such as phishing and Internet of Things (IoT) attacks remain profitable to criminals and continue to cause substantial damage not only to the victim users that they target, but also the organizations they impersonate. In recent years, phishing websites have taken the place of malware websites as the most prevalent web-based threat. Even though technical countermeasures effectively mitigate web-based malware, phishing websites continue to grow in sophistication and successfully slip past modern defenses. Phishing attack and its countermeasure have entered into a new era, where one side has upgraded their weapon, attempting to conquer the other. In addition, the amount and usage of IoT devices increases rapidly because of the development and deployment of 5G network. Although researchers have proposed secure execution environment, attacks targeting those devices can often succeed. Therefore, the security community desperately needs detection and prevention methodologies to fight against phishing and IoT attacks. In this dissertation, I design a framework, named CrawlPhish, to understand the prevalence and nature of such sophistications, including cloaking, in phishing attacks, which evade detections from the anti-phishing ecosystem by distinguishing the traffic between a crawler and a real Internet user and hence maximize the return-on-investment from phishing attacks. CrawlPhish also detects and categorizes client-side cloaking techniques in phishing with scalability and automation. Furthermore, I focus on the analysis redirection abuse in advanced phishing websites and hence propose mitigations to classify malicious redirection use via machine learning algorithms. Based on the observations from previous work, from the perspective of prevention, I design a novel anti-phishing system called Spartacus that can be deployed from the user end to completely neutralize phishing attacks. Lastly, inspired by Spartacus, I propose iCore, which proactively monitors the operations in the trusted execution environment to identify any maliciousness.

Despite an abundance of defenses that work to protect Internet users from online threats, malicious actors continue deploying relentless large-scale phishing attacks that target these users. Effectively mitigating phishing attacks remains a challenge for the security community due to attackers' ability to evolve and adapt to defenses, the cross-organizational nature of the infrastructure abused for phishing, and discrepancies between theoretical and realistic anti-phishing systems. Although technical countermeasures cannot always compensate for the human weakness exploited by social engineers, maintaining a clear and up-to-date understanding of the motivation behind---and execution of---modern phishing attacks is essential to optimizing such countermeasures.

In this dissertation, I analyze the state of the anti-phishing ecosystem and show that phishers use evasion techniques, including cloaking, to bypass anti-phishing mitigations in hopes of maximizing the return-on-investment of their attacks. I develop three novel, scalable data-collection and analysis frameworks to pinpoint the ecosystem vulnerabilities that sophisticated phishing websites exploit. The frameworks, which operate on real-world data and are designed for continuous deployment by anti-phishing organizations, empirically measure the robustness of industry-standard anti-phishing blacklists (PhishFarm and PhishTime) and proactively detect and map phishing attacks prior to launch (Golden Hour). Using these frameworks, I conduct a longitudinal study of blacklist performance and the first large-scale end-to-end analysis of phishing attacks (from spamming through monetization). As a result, I thoroughly characterize modern phishing websites and identify desirable characteristics for enhanced anti-phishing systems, such as more reliable methods for the ecosystem to collectively detect phishing websites and meaningfully share the corresponding intelligence. In addition, findings from these studies led to actionable security recommendations that were implemented by key organizations within the ecosystem to help improve the security of Internet users worldwide.