[CCT (Cyber Crime Tracker) ②] The Eyes That Track Cybercrime: A Step-by-Step Data Collection and Analysis Pipeline

In Part 1, we looked at how harmful sites—including illegal gambling sites—do not exist in independently, but form an organically interconnected “cybercrime ecosystem.”. Moreover, we also introduced the need for the CCT (Cyber Crime Tracker) framework to systematically track this ecosystem. (Link to Part 1)

In this post, we take a closer look at the step-by-step process—from data collection, the very first step and the “tracking eye” of the CCT framework, through to final analysis.

Cybercrime: All illegal activities occurring on a variety of illegal and harmful sites, including illegal gambling, adult content, and illegal webtoon piracy sites

Taking It a Step Further: No Data, No Analysis

To analyze the cybercrime ecosystem, we first need to secure the data to be analyzed. However, collecting data from harmful sites is a completely different from ordinary web crawling. Illegal gambling sites frequently change their URLs, dynamically render key information—such as sign-up buttons—using JavaScript, and actively rely on overseas servers and cloud infrastructure to evade blocking. In other words, this poses a significant technical challenge, since it requires continuously tracking targets that keep changing in order to avoid detection.

To solve this problem, the CCT framework has built a pipeline composed of three main modules.

URL & Content Collection Module: Collects gambling-site-related URLs and content based on static and dynamic web crawlers
AI-based Web Analysis Module: AI/XAI-based website classification and data analysis, and detection of keyword trend changes
Deep Intelligence Expansion Module: Builds intelligence using a variety of analysis tools

In this post, we will focus on the three modules—that is, the collection and analysis process.

Figure1: Conceptual diagram of CCT’s cybercrime site collection and classification

Step 1: URL & Content Collection Module — Crawling That Never Misses a Changing Site

Static Crawler: Finding New Clues from Already-Known Harmful Sites

In a previous post, we introduced the basic concepts and differences between static and dynamic crawling. In this step, we take a closer look at how these two types of crawlers are actually put to work in the process of collecting data on harmful sites.

The first collection method is static crawling. This process starts from the database of harmful sites we have already secured. The system first selects target URLs to explore from the existing database, then accesses those sites and collects the raw HTML source of the web pages. After then, the module focuses on analyzing the <a> tags that link to other pages within the collected data and extracts the TLD (top-level domain). By checking whether these newly discovered addresses already exist in the database, the system can effectively filter out new gambling-site URLs that have not yet been found, or that have recently changed addresses.

Dynamic Crawler: Executing JavaScript to Capture Core Content

Recently, illegal gambling sites often hide key information—such as sign-up buttons or deposit instructions—by rendering it dynamically with JavaScript, in order to evade detection by security solutions. To avoid missing this information, CCT also runs a dynamic crawler in parallel that operates within an actual web browser environment.

The dynamic crawler runs a web browser directly within a virtual environment to access target URLs. After accessing a site, it fully executes the JavaScript on the page to capture all resulting rendered HTML elements as well as visual content such as PNG images. Rather than simply capturing the screen, it uses its own rule set—based on tag attribute information—to automatically locate elements such as sign-up buttons, navigate directly to subsequent pages, and save all content on those pages as well.

One of the most important technical strengths of this collection process is that it goes beyond basic HTML text and metadata: it uses Optical Character Recognition (OCR) to extract text that is cleverly hidden inside image files. By diversifying keyword sources, the system can successfully capture information—including so-called human-identifiable information—that could never be caught through text analysis alone.

Figure2: Static/dynamic crawling-based data collection technology

Step 2: AI-based Web Analysis Module — Intelligently Classifying the Collected Data

Once URLs and web content have been successfully collected, the next step is to automatically determine what kind of site the data actually represents. CCT performs this task through an advanced classification system based on AI and XAI.

Deep Learning-based Harmful-Site Classification Model

Simply classifying collected sites into a binary “harmful or normal” category is not enough for an in-depth analysis of cybercrime. Therefore, CCT uses BERT (Bidirectional Encoder Representations from Transformers), a large language model (LLM), to apply a detailed classification scheme and accurately identify analysis targets. The analysis model classifies collected sites in detail—ranging from ordinary, normal sites to gambling sites (including casino and sports-betting sites), illegal webtoon-sharing sites, illegal TV and video streaming sites, and adult sites. In addition, it accurately identifies “link-aggregator” sites that serve as gateways to other illegal sites, as well as drug-related illegal sites. This precise classification is possible because BERT does not simply count how many times a specific word appears in a site’s text; rather, it learns the context and relationships between words in depth. The trained model calculates the probability that a given site belongs to each category (gambling, webtoon piracy, etc.) and clearly classifies it into the category with the highest score. By classifying the nature of sites across multiple categories in this multidimensional way, we can go beyond identifying a single illegal gambling site and clearly grasp the structure of the entire surrounding harmful-site ecosystem closely connected to it.

Explaining Classification Results with XAI

The classification results produced by deep learning models are powerful and accurate, but they have one critical shortcoming: they cannot explain the logical reasoning behind a decision—for example, “why was this site identified as gambling site?”. CCT resolves this so-called “black box” problem of AI by applying the SHAP (SHapley Additive exPlanations) algorithm.

By applying the SHAP algorithm to the system, CCT is able to calculate, as clear numerical values, the contribution of the specific keywords or features that had the greatest influence on the AI’s final classification decision. Through this process, users can see at a glance how much core gambling-related keywords—such as “casino,” “roulette,” “mini-game,” and “slot”—influenced the actual classification process. Furthermore, by continuously analyzing the results of this classification model, the system accumulates meaningful key keywords over time and can nimbly detect trend changes in the slang and keywords used by criminals.

As a result, when a particular site is classified as a gambling site, the system does not simply return a bare result; it automatically provides a clear explanation such as, “the keywords ‘casino,’ ‘sports betting,’ and ‘mini-game’ were decisive factors in classifying this site as gambling-related.”

Figure3: Harmful-site classification and explanation through AI and XAI models

Step 3: Deep Intelligence Expansion Module — Digging Beyond the Site to Uncover the Operating Organization

Even if the AI classification model in the previous step accurately identifies the nature of a site, this alone cannot reveal who is actually operating these illegal sites. To overcome this limitation, CCT’s Deep Intelligence Expansion Module thoroughly examines information both inside and outside the site, relentlessly tracking clues to the veiled operating organization.

Harmful-Site Intelligence based on Source Code, Images, and Keywords

Going beyond simple network-information collection, CCT thoroughly analyzes the content that makes up a site itself in order to uncover the hidden trail of its operating organization. After collecting the HTML tags and JavaScript code that make up a web page, the system calculates similarity between sites and clusters their structures. During this process, it analyzes not only the content displayed on screen, but also the various operational information embedded within HTML tags and scripts. For example, it extracts sign-up codes, referral codes, and affiliate identifiers found in sign-up links or advertising banners, and analyzes the relationships between operating organizations. It also collects developer notes, template information, and version information contained in HTML and JavaScript comments to infer a site’s development environment or CMS (Content Management System). In some cases, the same CMS templates or script structures are found repeatedly across sites, which serves as an important clue for identifying technical connections between different domains.

The numerous clues gathered in this way are robustly stored in separate, detailed databases for operators, CMS, individual content types (HTML, scripts, images), favicons, and more. There are moments when this data truly proves its worth. For example, if this analysis reveals that two gambling sites—which appear completely unrelated on the surface, with entirely different designs and names—internally share the same source-code comments, Telegram channel, and sign-up codes, this becomes very strong cross-evidence that the two sites are operated by the same organization.

Figure4: Process of building harmful-site operator-organization intelligence through various content analyses

Conclusion

In this post, we looked at how the CCT framework effectively collects illegal and harmful-site data, accurately classifies its nature using AI, and identifies clues to operating organizations and operational clusters from the collected content. The ultimate goal of the CCT framework is to go beyond simple URL collection and uncover traces of the organizations hidden deep within these sites.

In the upcoming Part 3, we will take a detailed look at techniques not covered in this post—“building network intelligence using OSINT tools” and “discovering security vulnerabilities and gathering information using OWASP”—to explore how we can track criminal organizations in an even more multidimensional way. We will also introduce how this meticulously accumulated data can be leveraged as intelligence to identify operating organizations. We look forward to your continued interest.

References

[1] 이경석, “디지털 질병! 불법 도박사이트의 특징을 파헤치다.”, CSRC Weblog, 2022.
[2] 임규민, 이경석, “온라인 불법 도박 사이트!! 어떻게 효과적으로 탐색하고 분류할 수 있을까?”, CSRC Weblog, 2022.
[3] 이경석, “도박사이트 파헤치기 1부 (네트워크형 사이버 불법/유해 사이트 특징)”, CSRC Weblog, 2023.
[4] 이경석, “도박사이트 파헤치기 2부 (불법 도박사이트의 기술적 진화)”, CSRC Weblog, 2023.
[5] 최규현, “대규모 언어 모델을 이용한 유해사이트 분류하기 1부”, CSRC Weblog, 2023.
[6] 최규현, “대규모 언어 모델을 이용한 유해사이트 분류하기 2부”, CSRC Weblog, 2024.
[7] 박상류, “유해사이트 군집화를 통한 유해사이트 조직적 운영 특징 분석”, CSRC Weblog, 2023.
[8] Choi Gyu-hyeon, ” [Cyber Crime Tracker (CCT) ①] Rethinking Harmful Websites: The Cybercrime Ecosystem Perspective and the Need for the CCT Framework,” CSRC Weblog, 2026.
[9] G.H Choi et al., “Detecting and Classifying Harmful Websites with Specific Korean Keywords Using a Large Language Model”, 2024 IEEE International Conference on Big Data and Smart Computing, 2024.