The term AI encompasses a wide range of technologies combined to solve problems, conduct analysis, and interact in a manner that mimics human intelligence. AI is often referred to in literature as cognitive technology (Chan–Olmsted, 2019). To produce more content, authors frequently utilise AI tools (Russell, 2022). Text generation tools can be used to check grammar, structure, or organise a paper according to human input. According to Awan, text generators produce human-like texts that mimic the patterns and styles of a person. This includes natural language processing, content creation, coding, and user assistance (Awan, 2023).

Language is a fundamental means of communication with others, as well as with various machines and devices. The need for generalised models arises from the necessity for machines that can perform complex linguistic tasks, such as translation, summarisation, information retrieval, and communication (Naveed et al., 2023).

Natural language processing (NLP) is a form of AI that is linked to linguistics (Šuman, 2021). It is a model that provides machine understanding of the human language, as well as understanding of human meaning and communicative intent. NLP focuses on the research on how machines process and understand natural human languages and is used for translating data from natural language to machine language (Chan–Olmsted, 2019).

Historical advances in NLP have evolved from statistical to neural language modelling and from pre-trained language models (PLMs) to large language models (LLMs). A PLM is a type of model in NLP that is trained on large amounts of text data before being adapted for specific tasks such as translation, text classification, summarisation, or question answering. To train a variety of models capable of generating natural language, these models require large amounts of data, which is what large language models enable (Naveed et al., 2024).

LLMs differ in architecture, training objectives, and scale. Transformer-based models such as generative pre-trained transformers (GPT) represent a specific subclass optimised for generative linguistic tasks, whereas the broader LLM category also includes encoder-based and hybrid architectures designed for classification or retrieval.

Although LLMs are a broader term, the term GPT has become more widely known. The relationship between them is that all GPTs are LLMs, while LLMs do not have to be GPTs. Although AI-based content generation brings numerous benefits to humans, it is challenging to determine whether content was written by a human or AI, which leads to problems, particularly in schools, higher education, and the scientific community. According to the available literature, there are two approaches to determining whether the text is AI-generated: Manual identification and automated identification using software tools. The rapid progress of language models has raised concerns about the misuse of text generation systems. With the development of GLTR (Generalised Language Tool for Recognition), it is easier for humans to detect whether the text is AI-generated or genuine. In a study conducted on human participants, GLTR was shown to improve the detection rate of inauthentic texts from 54% to 72% (Gehrmann, 2019). The literature also mentions the Raidar method (Generative AI Detection via Rewriting), which detects AI-generated content by rewriting text and calculating the edit distance of the resulting text. Raidar method shows significant results in detecting existing AI content detection models such as academic and commercial. This method works exclusively based on words and is compatible with LLMs that cannot be accessed within the system (Mao et al., 2024).

In manual identification methods, sentence patterns are utilised because texts written by humans tend to contain a broader range of phrases and sentences with diverse writing styles. In contrast, AI-generated texts often exhibit unique punctuation patterns (Amalia, 2023). Consequently, the searched content that requires more recent information may be incomplete or less detailed (Amalia, 2023). AI-generated content also presents different challenges. Two characteristics commonly discussed in the literature as indicators of AI-generated text are perplexity and burstiness (Ward, 2023). Perplexity represents a statistical measure used in natural language processing (NLP) to assess how predictable a sequence of words is for a language model. Texts with lower perplexity tend to contain more predictable word patterns and statistically probable constructions, which are often associated with machine-generated content. Burstiness, in contrast, refers to the degree of variation in sentence length, syntactic complexity, and structural patterns within a text. Human writing typically exhibits higher burstiness, reflecting natural fluctuations in linguistic style, whereas AI-generated texts often display more uniform sentence structure and a more consistent distribution of linguistic features.

In contrast, burstiness refers to the irregular distribution of content or concepts within the text (Ward, 2024). McLean lists several options for identifying AI-generated content, such as repetitive text, unusual vocabulary, and predictable patterns of sentence length and structure (McLean, 2024). AI-generated content detection tools, such as Copyleaks, ZeroGPT, and Winston AI, are available today. Legal frameworks related to the use of AI are still in their early stages of development. In the United States, Representative Ritchie Torres has proposed the AI Disclosure Act, which requires the labelling of AI-generated content, including photos, text, audio, and video (Smrekar, 2023). In Europe, the use of AI is to be regulated by the first legislative framework, the Artificial Intelligence Act (European Parliament, 2024).

As mentioned earlier, the increasing availability and ease of use of AI programs have led to their growing adoption, both for personal and business purposes. AI-generated content detection tools are designed to distinguish between human-written and AI-generated text. These tools differ from plagiarism detection systems, which identify overlap with existing texts regardless of authorship. Similar research and discussions have been published in several papers. One of the scientific papers is the work of the authors Anderson, Belavy, Perla, et al. The authors analysed the ability of AI to generate academic papers in the field of sports medicine and exercise medicine. They generated two essays using ChatGPT and analysed their shortcomings. The key problems revealed by the research were incorrect references, lack of originality and critical thinking. The AI-generated content detection tool, GPTZero, did not detect plagiarism. The authors also state that paraphrasing the AI-generated text with additional AI tools led to an increase in the percentage of real text, as detected by the plagiarism detection tool (Anderson et al., 2023). A similar issue is addressed by the author Cao, from whose work a section related to universities and the challenges they are facing was highlighted in this paper. She states that traditional plagiarism checks often fail to detect AI-generated texts. As a result, universities are introducing specialised systems, such as GPTZero and Turnitin, which function by combining perplexity measures, stylometric analysis, and partial comparison with reference databases. However, regardless of the above, unresolved issues related to privacy and the risk of false positives remain (Cao, 2025). In a study published in the Patterns journal in 2023, researchers analysed ninety-one essays written by native speakers of English for the TOEFL (Test of English as a Foreign Language), using seven AI detectors. For purposes of comparison, the identical detectors were used to analyse ninety-nine essays by American students, in which the detectors correctly identified more than 90% of the essays as being written by humans. Empirical evidence suggests that detection systems may systematically misclassify texts written by non-native speakers (Liang et al., 2023). In their conclusion, they agree with the authors as mentioned above, who emphasise the need for inclusive conversations involving all stakeholders to define acceptable uses of the Generative Pre-trained Transformer (GPT) model in different contexts, especially in academic and professional settings (Liang et al, 2023). Generative AI (GenAI), such as ChatGPT, enables users in educational contexts to generate content that appears original, thereby potentially violating educational standards (Hua, 2024). In the paper presenting an extensive–scale comparison of human-written versus ChatGPT–generated essays, the authors compared essays generated by ChatGPT with those written by humans. They analysed language skills, vocabulary complexity, and structural sophistication. The research results showed that GPT-3 and GPT-4 achieve better results than humans, leading to the conclusion that the trend of using AI is growing, especially when applied to written tasks (Herbold et al., 2023). GPTZero is designed to estimate the probability that a text was AI-generated, rather than to detect plagiarism in the traditional sense. In the context of the previously mentioned research and the issues related to English as a second language, the authors Alexander et al. agree, believing that currently available tools, such as GPTZero, yield inconsistent results, especially for non-native English speakers (Alexander, 2023). Although numerous studies and results have managed to detect generated texts, there are still limitations in their ability to identify AI-generated texts. This is the case because AI models are becoming increasingly sophisticated. The most common problems that content detection tools encounter are those related to more complex or edited AI-generated texts. To avoid recognising the generated text, current AI systems replace the original words with random synonyms (Bellini et al., 2024). This does not mean that AI systems actively resist recognition, but rather that paraphrasing mechanisms within generation tools unintentionally reduce detectability.

An analysis of the available and recent literature reveals that the topic of utilising AI for writing essays, professional papers, and other similar tasks is highly popular, with numerous global studies underway. Although the results are not uniform, the authors agree on one thing: It is necessary to work on privacy and authorship protection policies when using AI and to improve tools for detecting AI-generated content. In addition to authorship protection, privacy concerns arise because AI detection tools often require uploading sensitive documents, which may conflict with confidentiality obligations.

The research aims to analyse and compare texts created by AI tools with those written by humans across two languages, English and Croatian. Croatian was intentionally selected as a representative low-resource language in contrast with English as a high-resource language. This design enables a controlled comparison of detector performance across linguistic resource conditions. Authors writing in languages with smaller speaker populations such as Croatian often use English-language grammar correction tools, such as Grammarly, which tend to unify text consistency. This can result in more formalised text that human text recognition tools can characterise as machine-written text. The purpose of this study is to identify initial patterns and challenges in the effectiveness of currently available AI content detection tools in a multilingual context. The study does not aim to provide generalisable conclusions but rather to generate insights that can inform future large-scale research. Based on this study, it is expected that the generated content detection tools will not be able to identify the content generated by content generation tools. The scientific contribution of this work is reflected in the verified accuracy of the generated content detection tools. Furthermore, the work aims to raise awareness of the beneficial and harmful aspects of using AI tools.

At the beginning of the research process, research questions were set:

To provide answers, the research was conducted in three phases. In the first phase, three scientists from different scientific fields wrote a professional text of at least 1,800 characters from their respective fields of expertise. In the second phase, the same scientists from the first phase composed a professional text in the field of knowledge of the other two scientists, using different content generation tools: ChatGPT, Gemini, and Perplexity. In the third phase of the research, using AI-generated content detection tools, eighteen of these texts were analysed. In total, twenty-four texts were produced: Three human-written in Croatian + three human-written in English, and six AI-generated per tool across both languages). From these, eighteen were selected for detailed analysis due to tool limitations.

Although twenty-four texts were produced, only eighteen were analysed because some detection tools could not process Croatian texts. Thus, the analysed sample comprised all twelve English texts and six Croatian texts, depending on tool compatibility. Some texts were excluded because certain detection tools did not support Croatian input or imposed character limitations. This introduces a potential selection bias, which is acknowledged as a methodological limitation.

This study builds on a previous pilot exercise in which shorter texts, of approximately 600–800 characters, were tested with detection tools. The earlier pilot suggested limited detection accuracy. The present study expands the text length to at least 1,800 characters per sample to test whether longer texts yield different detection outcomes. Texts were produced in both English and Croatian, which is a morphologically complex South Slavic language with limited representation in AI training corpora). This bilingual design allows testing whether tool accuracy varies by language type and resource availability.

Written texts were checked against four systems designed to detect AI-generated content. The combined results of all analysed tools are available in the appendix.