1. Introduction

The 17 Sustainable Development Goals (SDGs) form the cornerstone of the 2030 Agenda for Sustainable Development, unanimously adopted by all United Nations Member States in 2015. They address critical global chal­lenges, including poverty alleviation, hunger eradication, quality education, gender equality, climate action, responsible consumption, and ecosystem preservation. These goals currently represent the most comprehensive and widely endorsed framework designed to measure and advance global progress toward sustainability across social, economic, and environmental dimensions. Tracking and assessing progress towards achieving the United Nations’ SDGs currently presents a considerable challenge due to the vastness and complexity of relevant data [1, 2, 3, 4]¹. Traditional manual analysis tech­niques can no longer cope with the rapid, large-scale data generation characteristic of today’s interconnected global environment [5].

In response, machine learning models have emerged as indispensable tools capable of processing and analysing extensive textual data from diverse sources, including institutional reports, news articles, social media platforms, and official documents [6, 7, 8]. These models facilitate the automatic detec­tion and categorization of content associated with individual SDGs, thereby supporting real-time monitoring and evaluation of the effectiveness of SDG-related initiatives.

Most research in this area primarily focuses on determining whether a given text segment is related to a specific SDG. This task is typically for­malized as a binary classification problem, wherein the objective is to assess whether a text pertains to a particular SDG. However, merely associating a text segment with an SDG often lacks the granularity needed to fully un­derstand the implications and broader context of the described activities or events. For example, a text discussing a new policy designed to reduce hunger in developing regions and another describing an emerging famine crisis would both be associated with SDG 2 (“Zero Hunger”), despite reflecting funda­mentally different impacts on sustainability goals: one potentially positive and the other clearly negative.

Consequently, binary classification alone proves insufficient for many prac­tical purposes, as it does not capture the direction or nature of the impact described. In numerous real-world scenarios, it is essential to further analyse whether the activities or events described in the texts have positive, neu­tral, or negative implications for progress toward achieving the respective SDGs. This more detailed analysis can significantly enhance researchers’ and policymakers’ abilities to evaluate and respond appropriately to devel­opments affecting global sustainability objectives [5]. This task aligns with a well-established problem in Natural Language Processing (NLP) known as polarity detection [9, 10]. In this context, polarity detection refers to iden­tifying the effect (positive, neutral, or negative) of the actions or activities described in a text with respect to a specific SDG. It is crucial to note that this task is distinct from traditional sentiment analysis [11], which aims in­stead to determine the overall positive or negative tone of a text, regardless of its relevance to any particular SDG. While sentiment analysis focuses on subjective opinions or emotional tone, polarity detection in the context of SDGs is concerned with evaluating the actual impact of described actions in relation to sustainability targets (i.e., whether a given text expresses ev­idence of progress toward a specific SDG or conveys an intention to pursue such progress). As we will discuss in more detail in Section 3, it is therefore possible for a text to exhibit a positive sentiment while conveying a negative polarity, or vice versa.

Despite its evident importance, polarity detection in the context of SDGs remains largely underexplored, primarily due to the absence of comprehen­sive, annotated datasets necessary to effectively train and evaluate relevant NLP models. Addressing this data gap constitutes a critical step toward en­abling more detailed, actionable analyses of texts concerning sustainable de­velopment. Concurrently, recent progress in NLP has largely been propelled by the introduction and advancement of large language models (LLMs), which have established new benchmarks across diverse tasks [12]. These models demonstrate exceptional capabilities in capturing intricate linguistic structures and semantic nuances, achieving superior results in tasks includ­ing text classification, sentiment analysis, polarity detection, and general language comprehension [13, 14].

In this paper, we address the task of SDG polarity detection by introduc­ing a novel benchmark and conducting a comprehensive evaluation of LLMs. Specifically, we present SDG-POD (SDG POlarity Detection), a new bench­mark dataset developed to support the training and evaluation of LLMs for this task. The dataset contains 6,400 texts, each annotated with a polarity label. We evaluate six state-of-the-art LLMs on the SDG-POD dataset under both zero-shot (ZSL) and fine-tuned conditions. Furthermore, we investigate the effect of augmenting the fine-tuning data with synthetic examples gener­ated by multiple LLMs. To enhance the quality of these examples, we apply a majority voting strategy to select the most reliable synthetic instances, thereby improving the robustness of the training set. In our experiments, we exclusively employed open-source models rather than proprietary alter­natives, since the study’s primary aim was to evaluate the performance of this class of LLMs on the specified task and within the designated domain. Specifically, we conducted all experiments using a diverse set of models, rang­ing from BERT variants with only a few hundred million parameters to the QWQ-2.5 model with 32 billion parameters.

Our results suggest that the task remains challenging for the current gen­eration of LLMs. This difficulty stems from the subtle and abstract nature of the text, which human experts can interpret effectively but which remains much more challenging for LLMs. Consequently, the SDG-POD represents a significant challenge for future research and models. We hope that the com­munity will continue to adapt and make progress on this task. Nevertheless, some fine-tuned models, particularly QWQ-32B, achieve fair performance, especially on specific Sustainable Development Goals (SDGs) such as SDG-9 (Industry, Innovation and Infrastructure), SDG-12 (Responsible Consump­tion and Production), and SDG-15 (Life on Land). Moreover, our analysis underscores the effectiveness of data enrichment techniques in mitigating the difficulties inherent to this domain, which is characterized by limited resources for training.

In summary, the main contributions of this paper are as follows:

• SDG-POD, a novel benchmark for polarity detection in the context of the SDGs, which integrates both manually annotated and synthetically generated data;
• A comparative evaluation of several LLMs on SDG-POD, considering both ZSL and fine-tuning settings;
• An investigation into the use of synthetic data, generated through an agentic architecture employing five LLMs, to enhance the performance of polarity detection models;
• The complete codebase of the experiments, made publicly available to facilitate reproducibility and enable the community to reuse the trained models².

The remainder of this paper is organized as follows. Section 2 reviews the related literature. In Section 3, we describe the specific task addressed in this study. Section 4 introduces the proposed benchmark. Section 5 outlines the experimental methodology. The results are presented and analyzed in Section

6. Finally, Section 7 concludes the paper and offers recommendations for researchers seeking to develop solutions that effectively balance performance and resource efficiency.

2. Related Work

Sentiment analysis [11], which involves understanding the emotional tone of a text to determine whether it expresses a positive, negative, or neutral sentiment, is one of the most dynamic research lines in the field of Nat­ural Language Processing (NLP). Initial approaches to sentiment analysis often relied on techniques based on predefined lexicons of positive/negative terms or on traditional machine learning algorithms (e.g., logistic regression, Naïve Bayes, Support Vector Machines) applied on manually extracted fea­tures from text [15]. Such conventional methods have shown limitations in capturing the linguistic nuances and complex context of natural texts.

A key advance has been the introduction of Transformer-based mod­els [12], in particular pre-trained Language Models such as BERT [13] and its derivatives [16], which have rapidly advanced the state of the art in sentiment analysis [17, 18]. Transformer models, thanks to the attention mechanism, can effectively capture long-range relationships in text and their fine-tuning on sentiment datasets has led to state-of-the-art results on many sentiment analysis benchmarks, often with a significant margin over previous meth­ods [19]. For example, Kokab et al. [20] proposed a generalized Transformer-based model for sentiment analysis on social media data, capable of handling noisy texts, out-of-vocabulary words, and context loss.

A very recent emerging trend is the application of generative LLMs, such as GPT [21] or LLaMA models [22], to the task of sentiment analysis. These models, trained on huge amounts of generic text data, demonstrate a surpris­ing ability to generalize to specific tasks via simple prompting (i.e., instruc­tions in natural language) without the need for further supervised training. Krugmann and Hartmann [23] performed a pioneering study in which they directly compared the ability of generative LLM models to classify sentiment with traditional specialized transfer learning models. The results show that the latest LLMs, despite operating in zero-shot mode (i.e., without having been specifically trained on the target dataset), can match and sometimes exceed the performance of conventional fine-tuned models on sentiment data, in terms of classification accuracy. This represents a paradigm shift: general-purpose generative models can perform sentiment analysis tasks with com­parable effectiveness to that of purpose-built models.

As an extension of sentiment analysis, another task has recently begun to emerge with increasing interest, called stance detection [24], which, instead of analyzing the simple emotional tone of a text (analyzed by sentiment anal­ysis), deals with determining the explicit or implicit position expressed to­wards a specific target. It differs from sentiment analysis because it does not necessarily coincide with the emotional tone of a text: for example, a tweet with a positive tone can still oppose a certain proposal, and vice versa [25]. Initially, the best results in stance detection were obtained with traditional supervised classifiers [26] (e.g., SVM or logistic regressors) based on manual features such as discriminant terms, n-grams, and lexical indicators. Early approaches with neural networks on short and noisy data (tweets) proved to be less effective, due to the scarcity of annotated data and the infor­mal language of social media [27]. In recent years, however, the adoption of pre-trained deep learning models has led to significant progress. In par­ticular, Transformer models (e.g., BERT, RoBERTa) fine-tuned on specific collections have rapidly outperformed manual feature models, becoming the standard for stance detection on texts [26].

In addition to sentiment analysis and stance detection, there is a third type of analysis capable of providing information on a dataset from the point of view of the degree to which the contents of a text (or a set of texts) are divided into opposing or extreme positions, named olarity detection. In other words, it involves evaluating how strongly a text expresses polarized positions, that is, clearly positive vs. negative, favorable vs. contrary, or adhering to opposing sides on a topic. This concept is applicable in any context (political, media, academic, social, etc.) and indicates the presence of a clear dichotomy in the opinions or feelings expressed in written language. A highly polarized textual discussion tends to cluster around two (or more) distinct ideological or emotional poles, with little presence of moderate or neutral tones in the middle. Textual polarization analysis, therefore, consists of identifying and measuring these divisions within the language, highlighting how much the texts are unbalanced towards opposing extremes rather than distributed along intermediate positions [9]. In recent years, research on the analysis of polarization in texts has seen an increasing use of deep learning techniques and Transformers-based models to detect and quantify ideological divisions in linguistic content. A recent review study has highlighted how over a third of works in the last two years adopt machine learning approaches for this purpose [10].

Pre-trained Transformer models such as BERT and RoBERTa have be­come key tools to classify the ideological orientation or pole of a text doc­ument [28]. One of the most well-known research strands of this type of analysis focuses on the automatic identification of the political position ex­pressed in social media or news texts, treating the problem as a variant of text classification [29]. For example, a BERT-based model has been pro­posed to detect political ideology in tweets about COVID-19, as an indicator of polarization in discussions about vaccines and masks [30]. In this study, the use of BERT allows to capture the linguistic context of tweets, while the integration of emotional signals improved the accuracy in distinguishing con­servative vs. progressive users. In parallel, other works combine sentiment and emotion analysis with deep learning models to identify forms of affective polarization (i.e. hostility towards the opposite group); these approaches recognize that expressions of strong positive/negative sentiment towards an issue can signal polarized positions [31].

In the last decade, since the definition of SDG in 2015, several scientific works have been published related to the SDG domain from various perspec­tives. For example, Schmidt-Traub et al. [32] introduced the SDG Index as an analytical tool to assess countries’ baselines for the SDGs, which can be applied by researchers in the interdisciplinary analyses needed for implemen­tation. Similarly, Vanderfeesten et al. [33] developed the well-known “SDG Classifier”, a tool that allowed to automatically map the scientific literature to the SDGs, based on BERT³. At the same time, Pradhan et al. [34] studied and identified the interactions between different SDGs, to discover synergies and trade-offs between several SDG pairs. Rosemberg et al. [35] instead de­veloped the sentiment analysis of Twitter data on climate change, which is transversal to several SDGs. Gennari and D’Orazio [36] finally created an approach based on statistical methods to evaluate the progress towards the sustainable development goals.

Despite significant advances in the fields of sentiment analysis, stance detection and polarization analysis, to date no study has specifically addressed the concept of “SDG polarization” as a polarity detection task. In other words, there is still no analysis that verifies whether a text refers to events or intentions indicative of an advancement or not with respect to a specific sustainable development goal. The few similar works found in the literature concern, for example, the use of advanced NLP methods to identify how much companies contribute positively to the SDGs by analyzing the texts of their sustainability reports (CSR) [37]. In this study, the sustainability reports of 1000 companies (2010–2019) were used to train classification models (lo­gistics, SVM, and neural network) in order to predict the alignment of each company to the 17 SDGs. The approach combines thematic dictionaries on SDGs with word embeddings (Word2Vec, Doc2Vec) to represent the text, im­proving the classification performance. The best model (SVM with Doc2Vec embedding) achieves over 80% accuracy in distinguishing whether a company is aligned with the SDGs (therefore indicating actions progressing towards the goals). Other works, such as that of Funk et al. [38], apply Aspect-Based Sentiment Analysis (ABSA) to Voluntary National Reviews (VNR), the reports with which 166 countries periodically describe their progress on the SDGs, to measure Sentiment in national reports on SDG progress. The method extracts for each country a sentiment score for each of the 17 SDGs, indicating how positively (or negatively) the country speaks of progress on each goal; these textual scores (SDG sentiment score) are then compared with official UN indicators. The reported results showed that for most of the SDGs the correlation is not significant, indicating that the contents of the VNRs do not always reflect the real trend.

For this reason, we developed an innovative analysis based on the use of LLMs, described and explored in the following sections, aimed at developing automatic analysis systems capable of supporting government bodies and companies in measuring the progress of projects and monitoring the activities of the various actors involved in achieving the objectives of the UN Agenda 2030.

3. Task Definition

In this paper, we formalise the task of polarity detection as a single-label, multi-class classification problem. The goal is to determine whether a given text contains indications of a progress state with respect to a specific SDG provided as input alongside the text. The three possible class values of this task are:

• Positive polarity: Represents the case in which the text contains indications related to a progress state or the intention to bring about a progress state with respect to the specific SDG provided alongside the For example, a document describing a new initiative aimed at promoting gender equality would be classified as having positive polarity with respect to SDG 5.
• Neutral polarity: Represents the case in which the text does not contain any indication or intention regarding either a progress state or a state of regression with respect to the specific SDG provided alongside the text.
• Negative polarity: Represents the case in which the text contains indications related to a state of regression or the intention to bring about a state of regression with respect to the specific SDG provided alongside the For example, a text reporting an increase in global famine would exhibit negative polarity with respect to SDG 2.

While classifying a text according to the relevant SGD is a largely solved task, the analysis presented in this paper demonstrates that detecting the polarity of SDG remains a surprisingly difficult challenge, even for the cur­rent generation of LLMs. This difficulty appears to stem primarily from the nuanced interpretation required when relating a piece of text to the often abstract goals of an SGD.

Below, we provide representative examples of texts corresponding to each of the three polarity categories, drawn from the OSDG community dataset⁴. This dataset will be further discussed in Section 4.

• Positive polarity with respect to SDG-1 (“End poverty in all its forms everywhere”):
  • “The Brazilian Instituto de Pesquisa Economica Aplicada (Insti­tute of Applied Economic Research, IPEA) has noted that every Real (BRL) invested in the programme increases GDP by BRL

1.44. The 16 million children and adolescents whose school atten­dance is monitored by the programme show lower rates of truancy and are performing at a level equal to the average student in the public school system, despite their impoverished economic condi­tion. This will lead to a future for these children far different from the situation of exclusion suffered by their parents and grandpar-ents”.

As it can be noted, the text above reflects positive progress towards SDG-1 by highlighting how strategic investments in education and eco­nomic development can create a virtuous cycle of growth, improved social outcomes, and a break from the longstanding cycles of poverty.

• Neutral polarity with respect to SDG-14 (“Conserve and sustainably use the oceans, seas and marine resources for sustainable development”):
  • “Seafood imports are primarily under the jurisdiction of the Food and Drug Administration. These figures represent total US tariff revenues for imports of edible fish and shellfish products. Since most fishery imports are duty-free, the majority of these amounts are accounted for by imports of a handful of processed products such as canned tuna, sardines and oysters, smoked salmon, and frozen crabmeat. The figures for each year are therefore inflated by approximately 33%”.

This text, on the other hand, simply outlines how seafood import tar­iff figures are calculated, without providing any insight into the actual management, sustainability, or conservation of marine resources. It fo­cuses on accounting practices and trade data rather than environmental outcomes, so it neither demonstrates progress nor indicates regression towards SDG-14.

• Negative polarity with respect to SDG-15 (“Protect, restore and pro­mote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, halt and reverse land degradation, and halt bio­diversity loss”):
  • “In MEA and other initiatives the focus has often been on de­sertification in dry regions and areas in the tropics, far from the Nordic Degraded land is however indeed also present in the Nordic region and concern have risen during the last decades as increased pressures on nature values and biodiversity have been documented in most ecosystems, although at different levels and intensities”.

In contrast, this text shows a regression in relation to SDG-15 because it reveals that existing initiatives have focused on desertification in traditionally vulnerable regions, neglecting land degradation issues in the Nordic countries. The rising concerns over increasing pressures on biodiversity and nature in these regions indicate that terrestrial ecosystems are deteriorating, that is a trend contrary to the cited SDG.

We would like to emphasise that the characteristics of this task do not necessarily coincide with those of the standard sentiment task. For instance, it is possible that a text exhibits characteristics typical of “positive” senti­ment, but indicates a regression with respect to a specific SDG. This is the case in a text such as:

“With enthusiasm and hope, we celebrate every daily achievement, while acknowledging that our progress towards Sustainable Development Goal 13 (Climate Action) has fallen behind expectations.”

In other cases, the text exhibits characteristics typical of “negative” sen­timent, but actually indicates progress with respect to a specific SDG, as in a sentence such as:

“Despite significant improvements in access to quality water resources (Sustainable Development Goal 6: Clean Water and Sanitation), the per­sistent sense of injustice and inequality casts a shadow of deep bitterness.”

4. A LLM-based solution for training data generation

In this paper, we pursue two main objectives: 1) to investigate the task of SDG polarity detection, and 2) to evaluate whether a synthetic train­ing set, generated using a simple agentic architecture composed of multiple LLM-based annotators, can serve as a high-quality resource for fine-tuning models. The second objective is motivated by the increasing use of synthetic data generated by LLMs in recent years. This approach has been particularly valuable in specialized domains and tasks that suffer from a lack of annotated data. To address both objectives, we developed and publicly released a new benchmark, SDG-POD. The training set of SDG-POD was generated using the proposed agentic architecture, while the test set was independently anno­tated by human experts in accordance with established annotation guidelines.

The remainder of this section describes the methodology followed in the creation of the SDG-POD benchmark. We begin by presenting the source data used for building the dataset, then introduce the agentic architecture employed to generate the synthetic training set, and conclude with an expla­nation of the procedure adopted to create the human-annotated test set.

4.1 Source data

SDG-POD includes documents derived from a subset of the dataset re­leased in October 2023 as part of the OSDG community dataset⁵, developed by the OSDG initiative. This dataset is the product of a collaborative effort by more than one thousand volunteers from over 110 countries and contains more than 40,000 text excerpts classified by human volunteers with respect to 16 SDGs. Each excerpt is 3 to 6 sentences long, averaging about 90 words, and is sourced from publicly available documents such as reports, policy doc­uments, and publication abstracts. A significant portion of these documents, over 3,000, originates from UN-related sources such as SDG-Pathfinder⁶ and SDG Library⁷.

To construct SDG-POD, a total of 6,400 texts were randomly sampled from the OSDG dataset, with 400 texts selected for each SDG. This col­lection was then divided into two subsets: a training set containing 5,824 texts (364 per SDG), which was automatically annotated using LLMs, and a test set comprising 576 texts (36 per SDG), which was evaluated by human annotators.

4.2 Synthetic training set generation with multiple LLM

The training dataset was automatically labelled using a majority voting system based on the classification provided by five LLMs of different features and sizes.

The five models chosen were as follows:

1. Meta-Llama-3.1-8B-Instruct: Meta’s LLaMa-3.1 is a decoder-only Trans­former that comprises a series of generative text models, which have been both pre-trained and fine-tuned. These models range in size from 8 billion to 405 billion parameters. They operate within an auto-regressive framework and leverage an optimized transformer architec­ture, enabling efficient language processing. The fine-tuning procedure employs both Supervised Fine-Tuning (SFT) and Reinforcement Learn­ing with Human Feedback (RLHF), ensuring that the models align well with human preferences for helpfulness and safety. Additionally, LLaMa-3.1 supports a context length of 128K tokens and demonstrates robust reasoning capabilities. To further enhance its performance in chat and dialogue applications, Meta introduced an innovative instruc­tion fine-tuning approach that combines supervised fine-tuning with techniques such as rejection sampling, proximal policy optimization (PPO), and direct preference optimization (DPO)⁸.
2. Mixtral-8x7B-Instruct-v0.1: Building on the success of its inaugural model, named Mistral, Mistral AI has unveiled its second language model, named Mixtral [39]. This high-quality sparse mixture of ex­perts (SMoE) features open weights and is available under the Apache 2.0 license. Mixtral-8x7B retains key characteristics of its predecessor, including Sliding Window Attention with a training context of 8,000 tokens and Grouped Query Attention, which enhances inference speed. Additionally, it continues to use the Byte-fallback BPE tokenizer for precise character recognition. The Mixtral-8x7B-Instruct-v0.1 variant has been fine-tuned for chat-based inference applications, optimizing its performance for instruction-driven interactions.
3. Phi-3-mini-4k-instruct: The Phi-3 series [40], launched by Microsoft in April 2024, is a set of compact, decoder-only language models that combine affordability with high performance in language processing, reasoning, coding, and math­ematics, often outperforming larger models. Notably, the Phi-3-mini of­fers both 4K and an unprecedented 128K token context length without significant quality loss, making it ideal for analyzing extensive texts. Developed under Microsoft’s Responsible AI Standard, these models underwent rigorous safety assessments, red-teaming, and enhancements such as reinforcement learning from human feedback. Their small size also makes them well-suited for resource-constrained environments, in­cluding on-device and offline applications. Additionally, the Phi-3-mini-4k-instruct variant, with its 4-bit OmniQuant quantization, brings pow­erful AI chatbot capabilities to devices like iPhones, iPads (with at least 6GB of RAM), and Macs.
4. Gemma-1.1-7b-it: Gemma [41] is a series of lightweight, open models developed by Google DeepMind and other Google teams, leveraging the same foundational technology as the Gemini models. They build on recent advances in sequence modeling, transformers, and large-scale distributed training, while also drawing from Google’s legacy of influential open models like Word2Vec, BERT, and T5. These models, trained on 8,192-token con­texts, incorporate enhancements such as Multi-Query Attention, ro­tary positional embeddings, GeGLU activations, and RMSNorm for improved stability.
5. Qwen2.5-7B-instruct: The Qwen models [42], developed by Alibaba Cloud, represent a fam­ily of LLMs with significant advancements in both pre-training and post-training. Qwen 2.5 introduces substantial improvements over the previous version, starting with an expanded pre-training dataset, scal­ing from 7 trillion tokens to 18 trillion tokens, enhancing the model’s common sense, expert knowledge, and reasoning abilities. Addition­ally, the post-training process includes a rigorous supervised fine-tuning with over 1 million samples, along with multistage reinforcement learn­ing, which refines human preference alignment and significantly boosts performance in long text generation, structural data analysis, and in­struction execution.

The following prompt was used across all LLMs to perform the classifi­cation task using a zero-shot approach:

f”””Given the following input text, between triple quotes, with its associated classification label with respect to one of the Sustainable Development Goals (SDGs), further classify the text with respect to the three labels defined below:

• “positive”: the text implies or explicitly states that what is affirmed or described leads to a significant advancement in favor of the goal indicated by the SDG under which it was previously
• “negative”: the text implies or explicitly states that what is affirmed or described leads to a significant advancement against the goal indicated by the SDG under which it was previously classified.
• “neutral”: the text does not imply or explicitly state that what is affirmed or described leads to a significant advancement either in favor or against the goal indicated by the SDG under which it was previously

Return the result in JSON format with 4 keys:

• “label”: The assigned label for the input text [this key/value pair is mandatory].
• “explanation\_1”: A description of the reasoning behind the classification decision [this key/value pair is mandatory].
• “explanation\_2”: Additional description of the reasoning behind the classification decision [this key/value pair is optional].
• “explanation\_3”: Another additional description of the reasoning behind the classification decision [this key/value pair is optional].

input = “INPUT TEXT: {input_text}”
SDG\_CLASSIFICATION: “SDG-{sdg}” “””

To assign a single polarity classification to each item based on the outputs of five LLMs across three polarity categories (positive, negative, and neutral), we adopted a set of carefully designed heuristics.

The distribution of label agreements among the five models was as follows:

• 774 texts received identical labels from all five We refer to this level of agreement as “Platinum”.
• 2,338 texts were labelled identically by four out of the five models. This case is denoted as “Gold”.
• 2,429 texts showed agreement among three models, which we label as “Silver”.
• 283 texts were classified with only two matching We refer to this lower agreement level as “Bronze”.

In the “Platinum”, “Gold”, and “Silver” cases, the assignment of the final label is straightforward, as a clear majority label emerges. However, the “Bronze” cases require a more nuanced approach, as no label reaches a simple majority. To address these cases, we implemented the following rules:

• When two models predicted a positive label, two a negative label, and one a neutral label, the final label was set to neutral.
• When two models predicted a positive label, two a neutral label, and one a negative label, the final label was set to positive.
• When two models predicted a negative label, two a neutral label, and one a positive label, the final label was set to negative.

This procedure led to the creation of a training dataset with the following characteristics:

• 2,218 texts labeled as positive (of which 426 “Platinum”, 888 “Gold”, 784 “Silver”, and 120 “Bronze”)
• 2,757 texts labeled as neutral (of which 287 “Platinum”, 1,212 “Gold”, 1,221 “Silver”, and 37 “Bronze”)
• 849 texts labeled as negative (of which 61 “Platinum”, 238 “Gold”, 424 “Silver”, and 126 “Bronze”)

4.3 Test set construction with human annotators

The test dataset of SDG-POD was manually annotated by a team of six human evaluators, divided into two groups of three members. Each group was assigned 288 texts for annotation. To enable the computation of inter-annotator agreement, 48 texts were annotated in common by all members within each group. The remaining 240 texts were evenly divided, with each member independently annotating 80 unique texts not shared with the others in their group. A majority voting system was used to determine the final labels for the 48 texts annotated in common. These shared texts were also employed to calculate the inter-annotator agreement and the Cohen’s Kappa coefficient within each group. The purpose of this procedure was to assess the consistency among evaluators.

The agreement values for each group are presented in Table 1, while the corresponding Cohen’s Kappa coefficients are reported in Table 2.

In this case as well, it was necessary to apply a majority voting mechanism to choose the labels for the shared portion among the evaluators. Since the evaluators worked in groups of 3, the possible cases of label combinations in the shared part of the test dataset and the corresponding distributions of texts were as follows:

• 59 texts were classified with 3 labels of the same type by the 3 evaluators (this classification was called “Gold”)
• 35 texts were classified with 2 labels of the same type by the 3 evaluators (this classification was called “Silver”)
• 2 texts were classified with 3 different labels by the 3 evaluators (this classification was called “Bronze”)

As with the majority voting mechanism applied to the LLMs, while in the “Gold” and “Silver” cases the assignment of the final label is straightforward, since one label appears more frequently than the others, in the “Bronze” case it was decided that the most appropriate label would be the neutral one.

The final version of the test set includes 220 texts labeled as positive, 219 as neutral, and 137 as negative.

5. Experimental Methodology

The analysis presented in this paper has two main objectives. First, to demonstrate that modern LLMs can effectively perform the task of SDG polarity detection, especially if fine-tuned on relevant data. Second, to show that a training set composed of synthetically generated data from LLMs can be used to fine-tune models and enhance their performance.

To achieve these goals, we designed an experiment to evaluate the per­formance of various LLMs in polarity detection, both in ZSL setting and after fine-tuning on synthetic data from the SDG-POD benchmark. The underlying hypothesis is that if the fine-tuned models outperform their non-fine-tuned counterparts, this would indicate that the synthetic training data provides a valuable contribution to the classification task. The fine-tuning procedure was standardised for all models by employing five training epochs and applying the same prompt as in the zero-shot setting.

We selected a set of six language models that are different from those used to generate the synthetic training set. These models are described in detail below.

1. QwQ-32B-Preview-unsloth-bnb-4bit: Alibaba Cloud’s Qwen models form a robust suite of LLMs, with Qwen 2.5 representing a significant leap forward⁹. The upgrade expands the pre-training dataset from 7 trillion to 18 trillion tokens, enhancing com­mon sense, expert knowledge, and reasoning capabilities. Post-training improvements include extensive supervised fine-tuning with over one million samples and multi-stage reinforcement learning, which boost long text generation, structural data analysis, and instruction follow­ing. The model excels in mathematics, programming, and scientific benchmarks like GPQA and MATH-500. Additionally, the QwQ-32B-Preview is a causal language model built on advanced transformer ar­chitecture. It incorporates features such as Rotary Positional Embed­ding, SwiGLU, RMSNorm, and Attention QKV bias, with 64 layers and 40 attention heads to support deep reasoning. Its extended context length of 32,768 tokens allows it to handle large inputs and complex, multi-step problems.
2. Phi-4: Phi-4¹⁰ is a 14B parameter open model from Microsoft, trained on a blend of synthetic datasets, curated public domain content, and aca­demic Q&A sources. Designed for high-quality reasoning capabilities, the model underwent extensive fine-tuning, including supervised train­ing and direct preference optimization to improve instruction adherence and safety. The training data, an extension of the data used for Phi-3, includes a variety of sources, such as educational materials, code, syn­thetic “textbook-like” content, and high-quality chat data reflecting human preferences. Multilingual data comprises approximately 8% of the total dataset, focusing on content that enhances reasoning abilities and correct knowledge alignment.
3. Mistral-Nemo-Instruct-2407: Mistral-Nemo-Instruct-2407¹¹ is a 12B parameter instruct model with a 128k context length, jointly developed by Mistral AI and NVIDIA. It significantly outperforms comparable models in reasoning, world knowl­edge, and coding, thanks to advanced fine-tuning and alignment. De­signed for global, multilingual use, it supports FP8 inference through quantisation awareness and features the new Tekken tokenizer, which compresses text more efficiently than previous tokenizers. Overall, it excels in following precise instructions, managing multi-turn conversa­tions, and generating code.
4. BERT: BERT [43], introduced by Google in 2018, is an encoder-only Trans­former model that set new standards in NLP tasks like language com­prehension, question answering, and named entity recognition. It comes in two main variants: BERTbase (12 layers, 12 attention heads, 110 million parameters) and BERTlarge (24 layers, 16 attention heads, 340 million parameters). Trained over four days on a massive corpus from Wikipedia and the Google Books Corpus (https://www.english-corpora.org/googlebooks/), BERT employs a bidirectional masked lan­guage modeling technique, masking 15% of words to predict them based on context, which, combined with transfer learning, enables effective fine-tuning for specific tasks.

5. ESG-BERT: ESG-BERT [44] is a language model specifically engineered for text mining in sustainable investing. Leveraging the BERT architecture, it has been fine-tuned to accurately identify and classify content related to Environmental, Social, and Governance (ESG) issues. In classifica­tion tasks, it outperforms the standard BERT-base model by achiev­ing a 90% F1-score compared to 79%. With 110 million parameters, ESG-BERT was rigorously trained using domain-specific data, reach­ing 100% accuracy in Next Sentence Prediction and 98% in Masked Language Modeling. Additionally, it supports classification across 26 ESG-related categories, from Business Ethics to GHG Emissions.

All models were evaluated on the manually annotated test set of SDG-POD using standard evaluation metrics for single-label, multi-class classifi­cation: precision, recall, and F1 score.

6. Results

Table 3 reports the results of the experiments conducted on the SDG-POD test set under the zero-shot learning (ZSL) setting. BERT and ESG-BERT are excluded from this evaluation, as they require fine-tuning and are therefore incompatible with a zero-shot setup.

In the case of zero-shot experiments, it was observed that the best open source performing model was the Phi4 model, which has the smallest number of parameters (only 4B), with a measured F1 score of 59.8%. The Phi4 good result was achieved thanks to its optimized architecture and efficient training methodologies, which provide a competitive advantage over other models, even those with a larger number of parameters, in situations with limited available information.

Table 4 instead shows the results of the classification experiments con­ducted on the models after they were fine-tuned. Unlike before, in this case the best open source performing model was the one with the largest number of parameters (QWQ-32B), although the Phi-4 model was fine-tuned for 10 epochs (whereas QWQ-32B was trained for only 5) in order to improve its classification metrics, with a measured F1-score of 61.6%, an increase of 3.8 percentage points compared to the performance of the non-fine-tuned ver­sion of the same model and a gap of 1.8 percentage points compared to the performance of the best non-fine-tuned model.

Figure 1. F1 comparison by SDG for QWQ-32B and Phi4-8B LLMs.

Although the overall F1-score gains between zero-shot and fine-tuned settings may seem modest at first glance, a closer examination reveals the substantial benefits of fine-tuning, particularly in reducing critical errors and improving reliability across specific SDG classes.

Figure 1 shows a side-by-side comparison of F1-scores, broken down by each SDG, for the best zero-shot model (Phi-4) and the best fine-tuned model (QWQ). When Phi-4 edges out QWQ, the margin is almost negligible, with the lone exception of SDG-2. In contrast, when QWQ outperforms Phi-4, the gaps are substantially larger. This pattern underscores that fine-tuning can deliver pronounced gains in specific cases.

Comparing the confusion matrices for each SDG (Tables 5 and 6) reveals that the non-fine-tuned Phi-4 model frequently mistakes negative examples for positive (and vice versa). The fine-tuned QWQ model, however, commits these “critical” errors far less often, at worst confusing a positive or negative label with the neutral class. Since confusing positive and negative labels is more severe than assigning them to neutral, this demonstrates that fine-tuning reduces the most serious misclassifications.

In light of these considerations, we compared the performance of the non-fine-tuned model with that of the fine-tuned version using the error-weighted F1 metric, which is commonly employed when certain types of misclassifications are more critical than others [45]. Specifically, we assigned heavier penalties (twofold and tenfold) to severe errors involving the incorrect prediction of negative labels as positive and vice versa, in order to emphasize which model was more robust to such critical mistakes. Table 7 reports both macro and micro weighted F1 scores. The results indicate that as larger weights are applied to the most consequential errors, the F1-score gap between the non-fine-tuned and fine-tuned models increases markedly, from approximately 1–2 percentage points to about 10–11. This trend underscores the substantial performance improvements yielded by fine-tuning.

The evaluation indicates that fine-tuning on synthetic data generated with the novel technique proposed in this paper, which integrates the outputs of multiple LLMs, leads to consistently improved results. The models obtained not only achieve higher overall performance but also demonstrate greater robustness, particularly by reducing the occurrence of severe mis­classifications where polarity is reversed. It also highlights that the task of SDG polarity classification remains highly challenging and far from solved. Given the importance of monitoring SDG concepts, this task constitutes a valuable benchmark for future systems and an important avenue for further exploration.

Moreover, in addition to the analyses described above, we also applied the McNemar statistical test to compare the performance of our fine-tuned models against the baseline. The test reveals that, for the positive class, the improvements achieved by both Phi4 and Mistral are statistically significant, with p-values of 0.04 and 0.05, respectively. For the neutral and negative classes, statistical significance could not be established, which is likely at­tributable to the relatively smaller number of data points in these categories, limiting the test’s sensitivity.

7. Conclusions

In this work, we explored the task of polarity detection for SDGs in news text, leveraging LLMs to assess how different architectures and train­ing strategies perform in this classification scenario. Our analysis shows that fine-tuned models substantially outperform their zero-shot counterparts, par­ticularly when we account for the severity of classification errors such as confusing positive and negative sentiments. These improvements are espe­cially evident when using weighted evaluation metrics that penalize critical misclassifications more heavily.

To support this study, we introduced SDG-POD, a benchmark dataset specifically designed for polarity detection in the SDG domain. By combining manually and automatically annotated data, SDG-POD enables systematic evaluation of LLMs on a task where annotated data is limited and costly to obtain.

Our experiments indicate that SDG polarity detection remains highly challenging, even for the latest generation of LLMs that achieve strong re­sults on a wide range of NLP tasks. The difficulty stems from the subtle and abstract nature of the text, which is readily interpretable by human experts but considerably less accessible to LLMs. As a result, the SDG-POD rep­resents a demanding benchmark for future research, and we expect that the community will continue to adapt and advance in addressing this task.

As an initial step in this direction, we demonstrated that synthetic data generated with the novel method introduced in this paper, which integrates the outputs of multiple LLMs, yields consistently improved performance. The resulting models not only achieve higher overall accuracy but also ex­hibit greater robustness, particularly through a reduction in severe misclassi­fications involving polarity reversal. Notably, QWQ-32B trained on this data surpassed all competing approaches.

Beyond aggregate performance, our per-SDG analysis and confusion ma­trix comparisons show that fine-tuning also mitigates the most consequential errors. These results underscore the importance of careful model adaptation and targeted evaluation when applying LLMs in socially significant domains such as sustainability.

Overall, this work provides both methodological insights and practical tools for advancing polarity detection in SDG-related news content. In future research, we aim to expand the benchmark to include multilingual data and explore real-world deployment settings for policy monitoring, media analysis, and decision support in the sustainability domain.

Notes

¹ https://unece.org/sites/default/files/2021-04/2012761_E_web.pdf

² Codebase of the experiments: https://github.com/vincenzodeleo/sdg_polarit y_detection/tree/main

³ https://zenodo.org/records/6487606

⁴ https://zenodo.org/records/5550238

⁵ https://zenodo.org/records/5550238

⁶ https://sdg.iisd.org/news/oecd-tool-applies-sdg-lens-to-international-organizations-policy-content/

⁷ https://www.sdglibrary.ca/

⁸ https://ai.meta.com/blog/meta-llama-3-1/

⁹ https://qwenlm.github.io/blog/qwq-32b-preview/

¹⁰ https://huggingface.co/microsoft/phi-4

¹¹ https://mistral.ai/news/mistral-nemo

References

[1] T. Hirai, F. Comim, Measuring the sustainable development goals: A poset analysis, Ecological Indicators 145 (2022) 109605. doi:https://doi.org/10.1016/j.ecolind.2022.109605.
URL https://www.sciencedirect.com/science/article/pii/S1470160X22010780

[2] D. Ziegler, S. Wolff, A.-B. Agu, G. Cortiana, M. Umair, F. de Durfort, E. Neumann, G. Walther, J. Kristiansen, M. Lienkamp, How to Measure Sustainability? An Open-Data Approach, Sustainability 15 (4) (2023) 3203–.
URL https://ideas.repec.org/a/gam/jsusta/v15y2023i4p3203-d1063481.html

[3] M. Mishra, S. Desul, C. A. G. Santos, S. K. Mishra, A. H. M. Kamal, S. Goswami, A. M. Kalumba, R. Biswal, R. M. da Silva, C. A. C. dos Santos, K. Baral, A bibliometric analysis of sustainable development goals (sdgs): a review of progress, challenges, and opportunities, Environment, Development and Sustainability (2023) 1–43 doi:10.1007/s10668-023-03225-w.

[4] M. La Fleur, Using large language models to help train machine learning sdg classifiers, Working Paper 180, UN DESA, New York (November 2023).

[5] M. Nilashi, O. Keng Boon, G. Tan, B. Lin, R. Abumalloh, Critical Data Challenges in Measuring the Performance of Sustainable Development Goals: Solutions and the Role of Big-Data Analytics, Harvard Data Science Review 5 (3), https://hdsr.mitpress.mit.edu/pub/9n4uzkg3 (jul 27 2023).

[6] J. E. Guisiano, R. Chiky, J. De Mello, Sdg-meter: A deep learning based tool for automatic text classification of the sustainable development goals, in: N. T. Nguyen, T. K. Tran, U. Tukayev, T.-P. Hong, B. Trawiński, E. Szczerbicki (Eds.), Intelligent Information and Database Systems, Springer International Publishing, Cham, 2022, pp. 259–271.

[7] T. Matsui, K. Suzuki, K. Ando, Y. Kitai, C. Haga, N. Masuhara, S. Kawakubo, A natural language processing model for supporting sustainable development goals: translating semantics, visualizing nexus, and connecting stakeholders, Sustainability Science 17 (3) (2022) 969–985. doi:10.1007/s11625-022-01093-3.
URL https://doi.org/10.1007/s11625-022-01093-3

[8] J. Guisiano, R. Chiky, Automatic classification of multilabel texts related to sustainable development goals (sdgs), in: TECHENV EGC2021, Montpellier, France, 2021.
URL https://hal.archives-ouvertes.fr/hal-03154261

[9] F. Bianchi, M. Marelli, P. Nicoli, M. Palmonari, SWEAT: Scoring polarization of topics across different corpora, in: M.-F. Moens, X. Huang, L. Specia, S. W.-t. Yih (Eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2021, pp. 10065–10072. doi:10.1007/s11625-022-01093-3.
URL https://aclanthology.org/2021.emnlp-main.788/

[10] R. Németh, A scoping review on the use of natural language processing in research on political polarization: trends and research prospects, Journal of Computational Social Science 6 (1) (2023) 289–313. doi:10.1007/s42001-022-00196-2.

[11] B. Pang, L. Lee, S. Vaithyanathan, Thumbs up? sentiment classification using machine learning techniques, in: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing – Volume 10, EMNLP ’02, Association for Computational Linguistics, USA, 2002, p. 79–86. doi:10.3115/1118693.1118704.
URL https://doi.org/10.3115/1118693.1118704

[12] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Curran Associates Inc., Red Hook, NY, USA, 2017, p. 6000–6010.

[13] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, in: J. Burstein, C. Doran, T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics, Minneapolis, Minnesota, 2019, pp. 4171–4186. doi:10.18653/v1/N19-1423.
URL https://aclanthology.org/N19-1423

[14] T. Brown, et al., Language models are few-shot learners, in: H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin (Eds.), Advances in Neural Information Processing Systems, Vol. 33, Curran Associates, Inc., 2020, pp. 1877–1901.
URL https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

[15] K. L. Tan, C. P. Lee, K. M. Lim, A survey of sentiment analysis: Approaches, datasets, and future research, Applied Sciences 13 (7) (2023) 4550. doi:10.3390/app13074550.

[16] P. Xia, S. Wu, B. V. Durme, Which *bert? a survey organizing contextualized encoders., in: B. Webber, T. Cohn, Y. He, Y. Liu (Eds.), 29 EMNLP (1), Association for Computational Linguistics, 2020, pp. 7516–7533.
URL http://dblp.uni-trier.de/db/conf/emnlp/emnlp2020-1.html#XiaWD20

[17] S. Alaparthi, M. Mishra, Bert: a sentiment analysis odyssey, Journal of Marketing Analytics 9 (2) (2021) 118–126, published on June 1, 2021. doi:10.1057/s41270-021-00109-8.
URL https://doi.org/10.1057/s41270-021-00109-8

[18] H. Batra, N. S. Punn, S. K. Sonbhadra, S. Agarwal, Bert-based sentiment analysis: A software engineering perspective, in: C. Strauss, G. Kotsis, A. M. Tjoa, I. Khalil (Eds.), Database and Expert Systems
Applications, Springer International Publishing, Cham, 2021, pp. 138–148.

[19] Y. Wu, Z. Jin, C. Shi, P. Liang, T. Zhan, Research on the application of deep learning-based bert model in sentiment analysis, Applied and Computational Engineering 67 (2024) 280–286. doi:10.54254/2755-2721/67/2024MA.
[20] S. T. Kokab, S. Asghar, S. Naz, Transformer-based deep learning models for the sentiment analysis of social media data, Array 14 (2022) 100157. doi:10.1016/j.array.2022.100157.

[21] OpenAI, J. Achiam, et al., Gpt-4 technical report (2024). arXiv:2303.08774.
URL https://arxiv.org/abs/2303.08774

[22] A. Grattafiori, et al., The llama 3 herd of models (2024). arXiv:2407.21783.
URL https://arxiv.org/abs/2407.21783
[23] J. O. Krugmann, J. Hartmann, Sentiment analysis in the age of generative ai, Customer Needs and Solutions 11 (2024) 3. doi:10.1007/s40547-024-00143-4.

[24] D. Küçük, F. Can, Stance detection: Concepts, approaches, resources, and outstanding issues, in: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, Association for Computing Machinery, New York, 30 NY, USA, 2021, p. 2673–2676. doi:10.1145/3404835.3462815.
URL https://doi.org/10.1145/3404835.3462815

[25] A. AlDayel, W. Magdy, Stance detection on social media: State of the art and trends, Information Processing & Management 58 (4) (2021) 102597. doi:10.1016/j.ipm.2021.102597.

[26] N. Alturayeif, H. Luqman, M. Ahmed, A systematic review of machine learning techniques for stance detection and its applications, Neural Computing and Applications 35 (7) (2023) 5113–5144, epub 2023 Jan 28. doi:10.1007/s00521-023-08285-7.

[27] A. ALDayel, W. Magdy, Stance detection on social media: State of the art and trends, Information Processing & Management 58 (4) (2021) 102597. doi:https://doi.org/10.1016/j.ipm.2021.102597.
URL https://www.sciencedirect.com/science/article/pii/S0306457321000960

[28] A. Chriqui, I. Yahav, Hebert and hebemo: A hebrew bert model and a tool for polarity analysis and emotion recognition, INFORMS Journal on Data Science 1 (1) (2022) 81–95. doi:10.1287/ijds.2022.0016.
URL http://dx.doi.org/10.1287/ijds.2022.0016

[29] S. Gode, S. Bare, B. Raj, H. Yoo, Understanding political polarization using language models: A dataset and method, AI Magazine 44 (3) (2023) 248–254. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/aaai.12104, doi:https://doi.org/10.1002/aaai.12104.
URL https://onlinelibrary.wiley.com/doi/abs/10.1002/aaai.12104

[30] M. Y. Kabir, S. K. Madria, A deep learning approach for ideology detection and polarization analysis using covid-19 tweets, in: Lecture Notes in Computer Science, vol. 13607, Springer, 2022, pp. 209–223. doi:10.1007/978-3-031-17995-2\_15.

[31] P. Badjatiya, S. Gupta, M. Gupta, V. Varma, Deep learning for hate speech detection in tweets, in: Proceedings of the 26th International Conference on World Wide Web Companion, WWW ’17 Companion, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 2017, p. 759–760. doi:10.1145/3041021.3054223.
URL https://doi.org/10.1145/3041021.3054223

[32] G. Schmidt-Traub, C. Kroll, K. Teksoz, D. Durand-Delacre, J. D. Sachs, National baselines for the sustainable development goals assessed in the sdg index and dashboards, Nature Geoscience 10 (8) (2017) 547–555. doi:10.1038/ngeo2985.
URL https://doi.org/10.1038/ngeo2985

[33] M. Vanderfeesten, R. Jaworek, L. Keßler, Ai for mapping multi-lingual academic papers to the united nations’ sustainable development goals (sdgs) (1.0), Zenodo (2022). doi:10.5281/zenodo.6487606.
URL https://doi.org/10.5281/zenodo.6487606
[34] P. Pradhan, L. Costa, D. Rybski, W. Lucht, J. P. Kropp, A systematic study of sustainable development goal (sdg) interactions, Earth’s Future 5 (11) (2017) 1169–1179. arXiv:https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1002/2017EF000632, doi:https://doi.org/10.1002/2017EF000632.
URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2017EF000632

[35] E. Rosenberg, C. Tarazona, F. Mallor, H. Eivazi, D. Pastor-Escuredo, F. Fuso-Nerini, R. Vinuesa, Sentiment analysis on twitter data towards climate action, Results in Engineering 19 (2023) 101287. doi:https://doi.org/10.1016/j.rineng.2023.101287.
URL https://www.sciencedirect.com/science/article/pii/S2590123023004140

[36] P. Gennari, M. D’Orazio, A statistical approach for assessing progress towards the sdg targets, Statistical Journal of the IAOS 36 (4) (2020) 1129–1142. arXiv:https://journals.sagepub.com/doi/pdf/10.3233/SJI-200688, doi:10.3233/SJI-200688.
URL https://journals.sagepub.com/doi/abs/10.3233/SJI-200688

[37] M. Chen, G. Mussalli, A. Amel-Zadeh, M. O. Weinberg, “nlp for sdgs:Measuring corporate alignment with the sustainable development goals”, The Journal of Impact and ESG Investing 2 (3), spring 2022 (2022). doi:10.3905/jesg.2021.1.035.

[38] C. Funk, E. Tönjes, R. Teuber, L. Breuer, Finding common development paths in voluntary national reviews reporting on sustainable development goals using aspect-based sentiment analysis, PLoS One 19 (8) (2024) e0307886, published on August 14, 2024. doi:10.1371/journal.pone.0307886.

[39] A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. de las Casas, E. B. Hanna, F. Bressand, G. Lengyel, G. Bour, G. Lample, L. R. Lavaud, L. Saulnier, M.-A. Lachaux, P. Stock, S. Subramanian, S. Yang, S. Antoniak, T. L. Scao, T. Gervet, T. Lavril, T. Wang, T. Lacroix, W. E. Sayed, Mixtral of experts (2024). arXiv:2401.04088.

[40] M. Abdin, et al., Phi-3 technical report: A highly capable language model locally on your phone (2024). arXiv:2404.14219.

[41] G. Team, T. Mesnard, et al., Gemma: Open models based on gemini research and technology (2024). arXiv:2403.08295.
URL https://arxiv.org/abs/2403.08295

[42] Qwen, :, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, Z. Qiu, Qwen2.5 technical report (2025). arXiv:2412.15115.
URL https://arxiv.org/abs/2412.15115

[43] J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirectional transformers for language understanding, in: J. Burstein, C. Doran, T. Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), Association for Computational Linguistics, 2019, pp. 4171–4186. doi:10.18653/V1/N19-1423.
URL https://doi.org/10.18653/v1/n19-1423

[44] S. Mehra, R. Louka, Y. Zhang, ESGBERT: language model to help with classification tasks related to companies environmental, social, and governance practices, CoRR abs/2203.16788 (2022). arXiv:2203.16788, doi:10.48550/ARXIV.2203.16788.
URL https://doi.org/10.48550/arXiv.2203.16788

[45] S. A. P. Parambath, N. Usunier, Y. Grandvalet, Optimizing f-measures by cost-sensitive classification, in: Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, K. Weinberger (Eds.), Advances in Neural Information Processing Systems, Vol. 27, Curran Associates, Inc., 2014.
URL https://proceedings.neurips.cc/paper_files/paper/2014/file/5c0314ec1b57fcd36bbb013f3f025868-Paper.pdf