Szczegóły publikacji

Opis bibliograficzny

What if, behind the curtain, there is only an LLM? : a holistic evaluation of TinyLlama-generated synthetic cyber threat intelligence / Zuzanna Pietrzak, Krzysztof Mączka, Marcin NIEMIEC // Electronics [Dokument elektroniczny]. — Czasopismo elektroniczne ; ISSN  2079-9292 . — 2025 — vol. 14 iss. 24 art. no. 4971, s. 1–22. — Wymagania systemowe: Adobe Reader. — Bibliogr. s. 20–22, Abstr. — Publikacja dostępna online od: 2025-12-18. — M. Niemiec – dod. afiliacja: Klaipeda University, Lithuania

Autorzy (3)

Słowa kluczowe

cyber threat intelligencecybersecurityAI-generated contentmisinformation detectionlarge language modelsthreat intelligence analysis

Dane bibliometryczne

ID BaDAP165246
Data dodania do BaDAP2026-01-12
Tekst źródłowyURL
DOI10.3390/electronics14244971
Rok publikacji2025
Typ publikacjiartykuł w czasopiśmie
Otwarty dostęptak
Creative Commons
Czasopismo/seriaElectronics

Abstract

The generation of synthetic cyber threat intelligence (CTI) has emerged as a significant area of research, particularly regarding the capacity of large language models (LLMs) to produce realistic yet deceptive security content. This study explores both the generative and evaluative aspects of CTI synthesis by employing a custom-developed detection system and publicly accessible LLMs. The evaluation combined automated analysis with a human study involving cybersecurity professionals. The results indicate that even a compact, resource-efficient fine-tuned model can generate highly convincing CTI misinformation capable of deceiving experts and AI-based classifiers. Human participants achieved an average accuracy around 50% in distinguishing between authentic and generated CTI reports. However, the proposed hybrid detection model achieved 98.5% accuracy on the test set and maintained strong generalization with 88.5% accuracy on unseen data. These findings demonstrate both the potential of lightweight models to generate credible CTI narratives and the effectiveness of specialized detection systems in mitigating such threats. The study underscores the growing risk of harmful misinformation in AI-driven CTI and highlights the importance of incorporating robust validation mechanisms within cybersecurity infrastructures to enhance defense resilience.