AI in the context of university practices, youth protection and the issue of supervision
Canva
Date of publication:
Artificial intelligence in education is no longer simply a question of using tools. Rules, trust, security, institutional accountability and the question of who controls the systems that are having an ever-greater impact on learning, work and decision-making are coming to the fore.
This newsletter is not just about new features and new models. It highlights how AI is becoming integrated into standard university processes – research, tutoring, writing, student support – as well as into the marketing strategies of major technology companies. Universities find themselves in a position where they must simultaneously teach the use of AI, safeguard core academic skills, verify the reliability of tools and ensure that decisions regarding teaching do not slip too far into the hands of technology providers.
Universities and higher education
In its article Universities face difficult choices over how to integrate AI, the Financial Times describes how universities around the world are seeking a balance between enthusiasm for AI and the far more mundane practicalities of implementation. Some institutions are focusing on AI literacy in a very broad sense, whilst others are adopting approaches that are more subject-specific. It quickly becomes apparent that introducing AI is not merely a pedagogical decision. Training is required, as is support for staff, agreements on safe use, clear rules for students, and consideration of whether everyone has equal access to paid solutions. For universities, this is a rather uncomfortable but realistic phase. Following initial pilot schemes, the time comes to decide which uses of AI are genuinely worthwhile, which are primarily a reaction to a trend, and which may even undermine the quality of study in the long term.
This is particularly evident in tools designed to detect text written using AI. In the article Universities drop AI detection tools over fears about accuracy, the Financial Times reports that some universities are restricting or discontinuing the use of AI detectors due to fears of false accusations, procedural complications and unequal impact on students. Particularly sensitive are cases where the tool attributes a high probability of AI use to a student’s text written in a foreign language or in a more formal style. The problem is not merely technical. If the assessment of academic integrity is based on a tool that neither the lecturer nor the student can truly verify, trust is quickly eroded. Consequently, the debate is increasingly turning towards the design of assignments, oral presentations, the assessment process and clear rules, rather than catching students out with tools that provide seemingly precise figures.
In the background, market pressure is also mounting. In its article AI labs begin to muscle in on $6tn education market, the Financial Times describes how OpenAI, Anthropic, Google and other companies are increasingly entering the education sector with tools for teachers, students and institutions. This brings useful solutions, but also raises an old question in a new form. Who shapes pedagogical practices if the tools come from companies whose logic differs from that of universities? With such solutions, it is not enough to ask whether the tool summarises a text well or generates a quiz. We must also ask what happens to the data, how the tool guides the work, what types of learning it supports, and whether it genuinely helps students carry out the more demanding aspects of critical thinking or merely elegantly bypasses them.
In the same context, there is also a very practical question of how to design assignments so that students still do the mental work when using generative AI. Times Higher Education Campus points out that, when it comes to academic assignments, we can no longer assume that the final product is reliable evidence of understanding. If a student can quickly produce a coherent text, summary or solution, more attention must be paid to the process of arriving at the answer. This means more assignments in which the student demonstrates their reasoning, choice of sources, revisions, doubts, comparison of versions and an explanation of why they accepted or rejected certain AI suggestions. Such an approach is not a return to the pre-generative AI era, but an attempt to design learning in such a way that technology does not supplant critical thinking.
In this context, the Stanford HAI paper Legal AI’s Legibility Problem is also of interest; whilst it focuses on the legal field, it is applicable much more broadly. The author points out that, with legal AI tools, it is not enough to talk about transparency at the level of an individual response. The entire system must be legible – from how the tool is tested, to the types of errors it makes, how frequently they occur, and who detects them. The situation is similar in higher education. If we use a tool for writing, marking, tutoring or student support, we need to know more than just that it works. We need to know where it goes wrong, for whom it goes wrong, under what conditions, and how a person can detect this in good time.
General education, youth and wellbeing
Among younger users, the debate on AI is intertwined with the debate on social media, algorithms and the digital environment in which children are growing up. The French parliament has passed a law prohibiting children under the age of 15 from accessing social media without appropriate age verification. This was reported by Reuters in the article ‘French parliament backs social media ban for children under 15’ and by Le Monde. Whilst this news item is not directly about generative AI, it forms part of the same debate on protecting young people in digital environments driven by algorithms, personalised recommendation systems and data-intensive business models. It is relevant to education because schools and universities are no longer concerned solely with digital skills, but also with the question of how to teach young people to navigate environments designed to capture their attention.
In its article The Complexities of Governing Mental Health AI, Stanford HAI highlights an even more sensitive aspect of the same story. An increasing number of people, including young people, are turning to chatbots, companion apps and tools that simulate supportive conversation when in distress. The reason is understandable. Access to professional help is often limited, whereas AI is always on hand, non-judgemental and responds immediately. This is precisely why the risk is greater. If a tool fails to recognise signs of a crisis, encourages unhealthy attachment or provides harmful responses, the consequences are no longer limited to a wrong summary or a poor essay. Human wellbeing is at stake. For universities, this is not a marginal issue, as student wellbeing, loneliness, the pressure to succeed and the search for quick support have long been part of everyday life in higher education.
At the level of systematic teacher training, the example of Egypt is also of interest. UNESCO reports on a national framework of competences for teachers in the field of artificial intelligence, which is adapted to the Egyptian educational context and is based on UNESCO’s global framework. The purpose of such a document is not merely to teach teachers how to use specific tools. Rather, it focuses on the interplay between pedagogical judgement, ethical use, lesson planning, an understanding of the limitations of AI, and the protection of pupils. This is also of indirect interest to universities, as future students will come from school environments where AI literacy will increasingly depend on how well their teachers have prepared them for it.
Global and EU AI policy and the issue of oversight
Among the more worrying news stories of recent days is a report of a security incident during model evaluation, published by OpenAI and Hugging Face. The media quickly picked up on the story, portraying it as ChatGPT having hacked into another AI company on its own initiative; however, one should be cautious about this interpretation. In their statements, the event is described as an incident in an evaluation environment, where certain production security mechanisms were deliberately disabled or relaxed in order to test the models’ extreme capabilities. The incident was also reported by Reuters in an article OpenAI says AI models went rogue during testing, triggering unprecedented breach.
The question of who will have control over the development of the most powerful AI systems is also featuring increasingly frequently in the international debate. During discussions on the global governance of AI, Chinese President Xi Jinping emphasised that artificial intelligence should not be under the control of a single country. This was reported by Al Jazeera. The statement forms part of a broader geopolitical debate on whether AI development will be centred on a few major technology hubs or follow a more decentralised model in which other countries also play a greater role.
Meanwhile, at European Union level, more practical examples of the use of generative AI in the public sector are also being developed. In its news item New GenAI pilots for public administrations, the European Commission has presented three pilot projects for public administrations. The projects will test the use of generative AI in flood and drought response, administrative simplification, urban planning analyses and digital assistants for engaging with citizens. This is of interest to higher education because universities will increasingly be training specialists for fields where AI will no longer merely assist with writing, but will form part of public services, crisis management and decision-making. This requires a different kind of AI literacy, as users must also understand issues such as accountability, oversight, public procurement, reliability and the consequences of errors.
The European Parliament is also responding to the unregulated use of commercial AI tools. TechRadar reports that the Parliament is said to be developing an internal platform, the EPGenAI Hub, which would enable staff to use models from providers such as OpenAI, Meta, Anthropic and Mistral in a more controlled manner. This news is also relevant to the academic sector. Many institutions are, in fact, grappling with the same question: whether staff should use public commercial tools independently, or whether the organisation should set up a more secure environment where access, data and permitted uses are more clearly defined. This is yet another example of how AI literacy is not merely an individual user skill, but also a matter of institutional infrastructure.
At European level, the European Commission has published guidelines on transparency obligations for providers and users of certain AI systems. These obligations stem from Article 50 of the AI Act and will come into force on 2 August 2026. In the news article Commission publishes guidelines on transparency obligations for providers and deployers of certain AI systems and throughout the document Guidelines on transparency obligations for providers and deployers of AI systems, the Commission explains when a user must be informed that they are interacting with an AI system, when artificially generated or modified content must be labelled, and how to deal with deepfakes, emotion recognition, biometric classification and content of public interest generated by AI without human editorial review.
Developments regarding AI in Slovenia
As we have previously reported, the GaMS language model has been officially unveiled and released as an open-source model for the Slovenian language, developed by researchers at the University of Ljubljana. In the latest publication by the University of Ljubljana, the model is presented as part of the PoVeJMo research and innovation programme, with the emphasis primarily on its public accessibility and use for research and educational purposes. This is important for the academic community for several reasons. The first is linguistic. If we want the University of Ljubljana to support learning, research and public services in Slovenian, we need models that are not merely adapted from an English-language environment. The second is research-related. Such a model opens up more scope for domestic research, evaluation, adaptation and responsible use within Slovenian contexts. The third is strategic, as it quickly becomes apparent with languages with a smaller number of speakers that technological sovereignty is not an abstract concept, but a very concrete issue of access to high-quality tools.
At the same time, the Slovenian landscape is also being shaped at the level of the broader support environment. GOV.SI, on its page dedicated to the national artificial intelligence ecosystem, compiles information on the national strategy, the Slovenian Artificial Intelligence Factory, the centre of excellence, European digital innovation hubs and the implementation of the AI Act. This is important because it enables a more coordinated development of research, education and regulation in the field of artificial intelligence in Slovenia.
Invitation to educators
If you have tried out an interesting use of AI in education at your faculty, as part of a course or within the context of your professional work, we warmly invite you to share your example with us via the form for collecting examples of good practice. We look forward to reviewing the submissions and including selected contributions in future issues of the AI newsletter, so that together we can build an overview of useful and well-considered approaches within the university environment.
Authors: Eva Kern Nanut, Eva Škraba, Maja Kosmač, Sanja Jedrinović Čufer, Mateja Bevčič, University of Ljubljana Centre for the Use of ICT in the Teaching Process
Department
Center for the use of ICT in pedagogical process (Digital University Center)
Univerza v Ljubljani
Kongresni trg 12
1000 Ljubljana