Artificial Intelligence: Opportunities and Risks

A tragic incident made headlines in California when parents accused ChatGPT of contributing to their 16-year-old son’s suicide. The parents are convinced that their son was using the chatbot while struggling with distress, and although the AI recognized his distress and need for help, it nevertheless assisted him in the technical analysis of the loop and encouraged self-destructive thoughts (more information). OpenAI responded to the incident by announcing the introduction of parental controls in its services, such as receiving notifications when issues involving teenagers are detected. The goal of the new feature is to increase protection for underage users to shield them from harmful content, manipulation, or misinformation (more information). This story highlights the extremely delicate balance between technological progress and social responsibility. On the one hand, AI systems are designed to be assistants and sources of knowledge; on the other hand, without adequate safeguards, they can pose a serious risk to vulnerable groups. Ethical guidelines and safety mechanisms are therefore not an optional extra, but a necessity if we want AI to function as a tool that supports people rather than as a potential threat.

The rapid integration of AI into educational environments presents both opportunities and challenges, particularly regarding its ethical use. A key issue often revolves around the emphasis on outcomes rather than the process of learning. This emphasis has inadvertently led learners to view technology (and AI) as a means to increase efficiency, often at the expense of deeper engagement and critical thinking. It is important to foster an environment in which the process of acquiring knowledge and the quality of engagement are also valued, which will promote a more thoughtful and ethical integration of AI and ensure that AI serves as a tool for better learning rather than merely a shortcut to the final result (more information).

A recent survey by IOP Publishing indicates that, despite the growing use of AI in peer review, opinions on its use are becoming increasingly divided (41% of respondents have a positive opinion of AI use, 37% a negative one, and 22% are undecided). In fact, as many as 57% of respondents stated that they would be dissatisfied if a reviewer used generative AI to prepare a report on their scholarly contribution (more information). The IOP currently does not permit the use of AI in peer reviews, but acknowledges that the study showed that the technology can support—but cannot replace—the peer review process (more information). The use of AI tools can help alleviate the workload on reviewers, speed up processes, and reduce burdens; however, the expertise, experience, and judgment that AI cannot replace remain crucial. Furthermore, there is also concern that reviews could become too generic, biased, or even misleading. The key question, therefore, remains: how can we ensure that AI truly serves as a support rather than a replacement for the critical human judgment that lies at the heart of scientific work?

Scientists in the United Kingdom have received 3.4 million pounds in funding to assess the use of AI tools in doctoral studies. As part of the project, the researchers plan to conduct a survey among doctoral students and other participants to determine how AI tools are currently being used, what challenges arise in this context, and how these challenges are being addressed. The information gathered will be used to create a resource database with guidelines on available AI tools, their appropriate use, and examples of best practices. The project presents an opportunity to gain knowledge and apply it to develop the resources needed for the ethical and responsible use of AI in doctoral research (more information).

In the field of education, the question arises of how to integrate AI into the teaching process in a way that truly supports learning. The TPACK model, which combines technological, pedagogical, and content knowledge, has proven to be particularly useful. This framework highlights that technology alone is not enough; what matters is how it aligns with pedagogical goals and subject content. The advent of generative AI, however, requires educators to develop technology and pedagogical knowledge specific to AI, with an emphasis on the ethical aspects of this integration process. The article (more information) thus introduces “Intelligent TPACK,” which better captures the knowledge educators need to teach using AI. With the advent of AI, an increasing number of new tools and resources designed for educators are also emerging. The following article (more information) contains a practical resource featuring a list of numerous AI tools to improve educators’ workflows and productivity, AI teaching assistants, tools to enhance design thinking, story-building tools, and much more. These tools offer quick and accessible solutions, but they are not without risks, as there is a danger that educators may too readily delegate creativity and critical thinking to algorithms. Although AI offers numerous advantages, the importance of human connection, empathy, and creative adaptation to students must not be lost in education.

Below, we present a few more developments outside the realm of education. A few days ago, Albania caused a stir by appointing a virtual minister, making it the first country in the world to have a minister created using AI. Diella, as the minister has been named, will be responsible, among other things, for decisions regarding public tenders. The process is expected to be implemented gradually, and Albania hopes that the virtual minister will make the country completely immune to corruption. Of course, in addition to questions of legitimacy, practical dilemmas also arise—can AI truly understand complex social circumstances, formulate policies in the public interest, and take responsibility for its decisions? The example from Albania thus provides an opportunity to reflect on how far we are willing to go in integrating AI into social structures that have, until now, been considered the exclusive domain of humans (more information).

U.S. media outlets are reporting cases in which scammers have misused AI to provide false medical diagnoses and advice. On social media, for example, there is a video of a real doctor who, while he does specialize in treating obesity, does not endorse miracle weight-loss pills as presented in the AI-generated video. Scammers mislead users by stealing doctors’ images and voices and selling (dangerous) products to make money. Such cases are on the rise and can be very dangerous, as people (too) often trust quick fixes, especially when they’re presented as seemingly expert recommendations (more information). Let’s stay with social media, where a trend has taken hold in which AI creates nostalgic scenes from the 1990s. Algorithms generate videos featuring images and scenes characteristic of that era, ranging from fun children’s toys to scenes from everyday life. For many users, this is a pleasant way to relive memories; at the same time, such examples demonstrate how AI also encroaches on the cultural and emotional spheres. The ability to create “artificial nostalgia” raises questions about the authenticity of memories and the influence that digitally generated images have on our perception of the past. Although this may seem like a harmless trend at first glance, it demonstrates AI’s power to shape a collective perception of a time that an individual may not have personally experienced at all. In this sense, nostalgia created by algorithms becomes yet another example of how AI influences not only the future but also our perception of the past (more information).

In the field of meteorology, AI is playing an increasingly important role in data processing and weather forecasting. These systems enable the rapid analysis of vast amounts of information, which improves the accuracy of forecasts and allows for earlier warnings of severe weather. However, meteorologists caution that despite advances in AI, the final decision always remains in human hands. A weather forecast is not merely a mathematical calculation; it also requires knowledge of local conditions, experience, and the ability to interpret data. AI is thus becoming an indispensable tool, but one that can never fully replace human judgment. This story illustrates, for example, how AI and experts can successfully collaborate—technology provides speed and processing power, while humans contribute knowledge and an understanding of context (more information).

The stories collected here show that AI is no longer just a technological innovation, but a social phenomenon that is transforming our lives at every level—from personal experiences to academic research, politics, and culture. We are learning that every technological advance comes at a cost: it brings new opportunities, but at the same time raises questions about safety, ethics, and responsibility. An example from California highlights the need to protect vulnerable groups, while discussions within the scientific community demonstrate the importance of critical judgment in the use of AI. At the same time, the power of positive stories is evident: in education and meteorology, technology complements humans but does not replace them. The news thus teaches us that the future of AI will not depend solely on what algorithms are capable of, but above all on how we, as a society, will guide, oversee, and integrate them into our lives.

Authors: Mateja Bevčič, Eva Kern Nanut, Eva Škraba, Sanja Jedrinović Čufer, University of Ljubljana Center for the use of ICT in pedagogical process