Step I: metadata-based subcorpus creation

1To answer RQ1.1, we want to understand what is the distribution of topics per gender,²⁵ i.e., which are more dominant topics and which are less dominant topics in speeches delivered by women MPs and men MPs respectively. First, we need two subsets of data that will represent the speeches by the two groups.

2As explained under Research data, we want two subcorpora with the following parameters:

• gender: woman/man
• status: MP
• role: regular speaker

3These three parameters are part of metadata categories (also called Text types in noSketch Engine; see Metadata). In noSketch Engine, a subcorpus based on metadata is created as follows:

1. From the dashboard, click Manage corpus, then find the Subcorpora tile and click the red button Create subcorpora (Figure 35).

   

   
   
2. Define Subcorpus name and, by expanding the drop-down lists, select the required values to limit the speeches by metadata (Figure 36). In our case (see Research data for explanation):
   1. subcorpus name = regularMP_F (or whatever feels convenient for you)
   2. speech.speaker_mp = MP
   3. speech-speaker_role = regular
   4. speech.speaker_gender = F

   

   
   

3. 1Click the red button in the corner to save the subcorpus. You are redirected to the Subcorpora overview page which shows the size of the subcorpus as well as the parameters used to create it under the information symbol. We can thus confirm that the corpus of speeches produced by women MPs acting as regular speakers has been successfully produced.

   2Moreover, we see that the two subcorpora considerably differ in their sizes: regularMP_M is almost three times the size of regularMP_F which corresponds with the seat distribution between men and women in the parliament. This is important to keep in mind for any subsequent quantitative comparisons of the two subcorpora (see Step III).

4. Now we repeat the previous steps to create the corpus of speeches produced by male MPs acting as regular speakers. From the Subcorpora overview page, we first click Create subcorpora and then select the following parameters:
   1. subcorpus name = regularMP_M (or whatever feel convenient for you)
   2. speech.speaker_mp = MP
   3. speech-speaker_role = regular
   4. speech.speaker_gender = M

Step II: frequency list creation

1To answer RQ1.1 which foresees examining the distribution of topics by gender across European parliaments as a whole, we will exploit the speech topic metadata category (see Section Topic information for more detail on the annotation procedure). We will compute the frequencies for the 21 topic categories, available in ParlaMint, to understand which topics dominate the speeches produced by women MPs and which by men MPs.

2In noSketch Engine, frequency information can be computed in several ways. Since we are interested in the distribution of topics for one of the metadata categories, we can use a dedicated function in noSketch Engine as follows:

1. From the dashboard, click Text type analysis (Figure 37).

   

   
   

2. 1In the list on the left, we select the desired metadata category, in our case speech.topic, and one of the two subcorpora in focus, here first regularMP_F. We switch the Show parameter to Token coverage, so the frequencies reflect the number of tokens belonging to speeches annotated as addressing one of the topics (Figure 38).

   2For our purpose, frequencies reflecting the number of speeches (instead of tokens) would also work, but note that due to a bug,²⁶ when a subcorpus is selected, the tool does not behave as expected when we switch the Show parameter, and always returns the number of tokens. When no subcorpus is selected, the tool returns correct values.

   

   
   
3. We download and save the results locally (Figure 39) in order to analyse them more thoroughly later on. We either leave the tab open or save the URL for quick access, since we will use it soon to dig deeper into the data.

   

   
   
4. Now we repeat the steps (optionally in a separate window) to create the frequency list for topics addressed in speeches by male MPs. Note that this time, we select the subcorpus regularMP_M.

3For the purpose of this analysis, distribution can be assessed either based on token frequencies or speech frequencies. However, did you wish to obtain frequency data on speeches only, you need to:

1. Go to the Text type analysis page and select the analyse multiple text types option on the left.
2. Define as many metadata categories as needed, in this case speech.speaker_role, speech.speaker_mp, speech.speaker_gender and speech.topic. This recreates the conditions used during subcorpus creation and stratifies results per topic.
3. The results then need to be downloaded and sorted/analysed locally in a spreadsheet.

Step III: frequency list interpretation

1The results (Figure 40) present the topics in descending order of frequency within the speeches of one of the two groups under scrutiny. The frequencies reflect the absolute number of tokens that compose the speeches addressing a given topic. Topics circled in Figure 40 indicate the ones that Ljubešić et al. (n.a.), in line with previous studies, identified as being more frequently debated by one of the gendered groups, namely health, civil rights, education and social welfare for women (red circles), and macroeconomics, transportation, defence and energy for men (blue circles).

2A quick glance at Figure 40 reveals that topics typically associated with women’s speeches (red circles) indeed appear higher in the ranking of the regularMP_F subcorpus than in regularMP_M. Likewise, the highest-ranked topic in the regularMP_M subcorpus (macroeconomics) corresponds to one of the domains most often identified as male-dominated in previous research. This corroborates earlier findings and hints at particular gendered patterns in parliamentary discourse.

3However, topic ordering alone cannot reliably point to gender differences or similarities. Based on our results, one might even argue that the two distributions appear broadly similar: most of the highly ranked topics, e.g., law and crime, international affairs, education, occur frequently in both corpora, albeit in slightly different positions. The challenge lies in the fact that the two subcorpora differ substantially in size (see Step I). Without normalisation, any comparison risks overestimating the prominence of certain topics in the larger corpus while underestimating it in the smaller one.

4Normalisation is achieved by calculating relative frequencies, which makes it possible to assess not only whether women and men debate the same themes (which is expected since this depends on the session agenda), but also whether they devote proportionally more or less attention to specific areas. For instance, observation based on absolute frequency values, according to which topics are ranked in Figure 40, suggests that health and social welfare occupy a relatively stronger position in women’s speeches than in men’s, while macroeconomics and defence are more salient in men’s discourse. The next step in the interpretation of the results is therefore to adjust for corpus size and examine whether proportional frequencies confirm the patterns observed based on the absolute counts.

5Relative frequency is calculated by dividing the number of tokens for a given topic by the total number of tokens in the subcorpus and then multiplying the result by 100 or, as in our case, 1 000 000, depending on the desired scale. This normalises the data across subcorpora of different sizes and enables direct comparison between them.

1. To perform this step, we open the data, which we downloaded during Step II, in a spreadsheet editor.
2. The total number of tokens can be retrieved either from the results page (see Total frequency in the upper-right box) or from the information page under the Subcorpus sizes section (accessible via the information icon at the top of the page next to the corpus selection box).
3. To obtain the relative frequency, we divide the absolute counts for, e.g., regularMP_F with the total number of tokens from that corpus (327,631,659) and multiply by 1,000,000. We do the same for the regularMP_M corpus, just that this time the total token count is 855,066,379.

6Table 3 shows the predominant topics of women’s speeches, and Table 4 illustrates the predominant topics of men’s speeches, based on the ratio of relative token frequencies ²⁷. We observe that the patterns established based on absolute counts do not fully translate when relative frequencies are taken into account. Consider, for example, the first two topics in the listing above (Figure 40) that are shared between the two subcorpora: Health and Macroeconomics. In absolute terms, Health is more frequently represented in the regularMP_M subcorpus than in regularMP_F (50 million tokens vs. 32 million tokens, respectively). This might lead to a quick but misleading conclusion that health is a more dominant topic for men than for women. However, relative frequencies reveal the opposite: women address health almost twice as often as men. By contrast, Macroeconomics is clearly very prominent in the regularMP_M subcorpus and much less so in regularMP_F. Relative frequencies confirm this trend, showing that macroeconomic issues are discussed nearly 1.5 times more often in men’s speeches. Yet the absolute counts also show that, despite this difference, Macroeconomics remains a major theme in the regularMP_F subcorpus, ranking second overall.

7This comparison illustrates the value of considering both absolute and relative measures. Absolute frequencies highlight the overall weight of a topic within a subcorpus, while relative frequencies uncover how strongly each gendered group emphasises a given issue compared to the other group. Taken together, they provide a more nuanced view: women clearly privilege topics such as health and social welfare, while men devote proportionally greater attention to macroeconomics, defence, and government operations. At the same time, overlap in high-ranking topics across both groups reminds us that gendered differences occur within a largely shared parliamentary agenda.

8Yet even when a topic appears highly ranked in both subcorpora, this does not necessarily mean it is addressed in the same way. Subtle differences in framing, sentiment, or argumentative stance may reveal distinct gendered approaches to these issues. Our second research question, which we tackle in the next steps, addresses exactly this aspect.

Step IV: concordance-based subcorpus creation

1To answer RQ1.2 which foresees exploring the typical subtopics within the two topics we identified in the previous step for each gendered group, we will now look into the typical vocabulary used in the speeches on the topics of Health and Macroeconomics. Typical vocabulary of one group against the other can be obtained through an analytical technique, called keywords. To be able to compare two groups, we first need to create the appropriate subcorpora. We could compute the keywords for the two subcorpora already created, i.e., regularMP_M and regularMP_F, but we are not interested in the differences in their overall vocabulary, but just in the differences in the vocabulary on two specific topics. For this reason, we need to create subcorpora of speeches that only cover the topics in focus for either of the two gendered groups.

1. To open all concordance lines, in our case speeches, marked as belonging to one of the two topics, we can continue from the Text type analysis page (see Step II). We shall start with the topic-distribution results for the regularMP_F subcorpus.
2. In the lower part of the page, we see the list of topics which are followed by frequency information and three dots which leads us to concordance lines (Figure 41).

   

   
   
3. We are redirected to the Concordance view which allows interacting with the data, i.e., the actual transcripts of speeches. In the upper right corner, we see a + symbol which is used to create a subcorpus from all the listed concordance lines (Figure 42). However, due to a limitation of the tool which does not allow creating a subcorpus based on concordance lines that were computed based on another user-defined subcorpus (in our case regularMP_F), we need to re-display the desired concordances through the search functionality or the so-called Concordance function.

   

   
   

4. 1noSketch Engine enables different types of searches, from simple to advanced. The interface allows to formulate search queries that combine tokens, different levels of linguistic tags and metadata. The search option that allows most complex queries is called CQL which uses a particular syntax and requires a bit of getting-used-to.²⁸ However, there are some shortcuts in noSketch Engine that help the user formulate the CQL query, like the CQL builder available in the interface, or reusing previous queries displayed in the tool which is what we will do in our case.

   2In the list of concordances, look at the upper left corner where you’ll find the CQL query used to display the concordance lines on the topic of Health (Figure 43). Copy this to your clipboard, i.e., <speech topic=="Health">[].

   

   
   
5. As shown in Figure 44, open the Concordance function via the menu on the left, choose the Advanced tab and the CQL option in the list. Paste the copied query on the line and define the three parameters we used in Step I to create the subcorpus (in our case, we first select the combination MP, regular, F) and click the red button Go.

   

   
   
6. Now you’re back at the page with concordance lines and this time the + symbol in the upper right corner is enabled. Click the symbol, define the subcorpus name, in our case health_F, choose speech as a structure so that the entire speeches are saved in the subcorpus, and hit Create subcorpus (Figure 45). You can check that the operation was successful on the Corpus Information page, accessible via the information symbol next to the corpus selection box at the top of the page.

   

   
   
7. Repeat the steps and change the metadata parameters accordingly to create a subcorpus of health-related speeches produced by male MPs. Next, repeat the steps to create the two subcorpora of speeches on Macroeconomics (<speech topic=="Macroeconomics">[]). Your list of subcorpora should include the entries as shown in Figure 46.

Step V: keywords list creation

1Now that we have created the necessary subcorpora, we can proceed to creating keywords lists which will show us the typical vocabulary of the focus group in comparison to the reference group. The calculation is based on simple math and basically leverages relative frequencies to determine which items are comparatively more frequent in the focus corpus. The result is the keyness score which can be displayed via Change view settings and is, by default, used to sort the keywords. The keywords can be actual tokens, but can also be different tags. In this step, we will exploit both options. First, we will use semantic annotations, available in the machine-translated ParlaMint, to compute the typical semantic domains used by men and women MPs in speeches on macroeconomics. Next, we will take another approach and produce a list of typical lemmas used by men and women MPs in speeches on health.

Step VI: keywords list analysis and interpretation

1The analysis may take several directions, but in this showcase, in line with our second research question, we focus on the comparison of subtopics addressed in speeches on a common central topic, i.e. Macroeconomics and Health. To this end, we first analyse the common and unique semantic domains present on lists of speeches by women and men respectively and then perform close reading of a random sample of 20 concordances for each of the first 50 key semantic domains to establish the prevailing topic patterns.

2The two thresholds (20 and 50) are heuristic rather than fixed parameters. They are set to keep the qualitative inspection manageable while still covering the main semantic patterns that characterise the corpus. These thresholds can be adjusted depending on the research goals, corpus size, and how much manual work is feasible. You can also base your selection on keyness scores instead of just taking the top N keywords, but there’s no universal rule for what counts as a “good” score – it depends on your corpus and research questions.

Step I: composing simple and CQL search queries

1In this section, we focus on a set of practical operationalisations of RQ2.1 that can be implemented using simple and CQL‑based searches in the NoSketch Engine interface. These examples illustrate one possible way of approaching the research question; however, they do not exhaust all potential operationalisations. Other analytical pathways are possible, but for the purposes of this showcase we will concentrate on the following two options.

2Possible operationalisations of RQ2.1

• Does the distribution of the lemma EU and the sequence European Union in sentences spoken by MPs and annotated as “Negative”, “Neutral” and “Positive” in ParlaMint-GB v. 5.0 differ between the Reference subcorpus and the COVID and COVID, War subcorpora?
• Does the distribution of the lemma Europe in sentences spoken by MPs and annotated as “Negative”, “Neutral” and “Positive” follow the same patterns as those observed for EU and European Union?

1. After selecting the corpus of interest, as shown in Figure 62 above, for a simple query, click the Concordance icon on the vertical menu bar and select Advanced search option. Simple query is the default query type in the drop-down menu (Figure 63).

   

   
   
2. Write EU | European Union in the search line and limit your search by selecting the built-in Reference subcorpus and MP speakers under Text types. Click Go and view your results. To view details about the selected text types or to modify the selection, click the three dots next to the number of text types in the top left corner. To change your search criteria, click the far-left icon in the top horizontal toolbar. (Figure 64).

   

   
   
3. Alternatively, you can opt for a CQL search by clicking the Concordance icon on the vertical menu bar, selecting Advanced search option and the CQL query type in the drop-down menu (Figure 65).

   

   
   
4. To proceed with a CQL query, write [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <speech subcorpus="Reference" & speaker_mp="MP"/> – all in the CQL line (Figure 66).

   1➽ noSketch Engine tip: searching for lemma and lower-case lemma (lemma_lc) may return somewhat different results, since these are distinct attributes. While the lemma attribute preserves capitalisation distinctions introduced by the lemmatiser, particularly for acronyms and named entities, the lowercase lemma flattens these differences, including tokens with inconsistent or non-standard capitalisation and tokens whose lemma was normalised differently upstream. The same distinction applies to word and lower-case word attributes. Use lower case attributes, when you are interested in case-insensitive matching and unified frequency counts (see Lowercase version for more detail).

   

   
   
5. In fact, you can also combine the CQL format for searches of lexical (and grammatical) structures with metadata limitations activated through the Text type selector (Figure 67).

   

   
   
6. Either way, after clicking the Go button, you will arrive at the Concordance page, where you can view concordance lines from the parliamentary transcripts. Click the information icon to view numerical details of your query (Figure 68).

Step II: retrieving sentiment frequency and density

1. To access frequency results for the sentiment annotation of the retrieved concordance lines, click the Frequency button on the Concordance page, select the Text types preset and scroll down to the sentiment metadata (Figure 69).

   

   
   
2. Alternatively, you can click the Frequency button on the Concordance page, select the Advanced tab, define one or more sentiment attributes of interest in the dropdown list and click the Go button. For the purposes of this step, we will focus on the senti_3 scale (Figure 70) but the senti_6 or the senti_n data can be selected alternatively or added to the search via the white and blue Plus icon in the Advanced version of the Frequency page.

   

   
   
3. On the Frequency page, the default frequency of tokens is raw, or absolute, frequency but you can also choose to see relative frequency in text types and relative density (Figure 71). Relative frequency is explained above in the textbook. Relative density is provided in the NoSketch Engine concordancer as an indicator of a value being either more frequent or less frequent in the text type compared to the whole corpus. In other words, values below 100% indicate underuse, and above 100% indicate overuse relative to the baseline.

1Figure 71 shows the absolute frequency, the relative frequency within the text type and the relative density of uses of the abbreviation EU and the phrase European Union by MPs in sentences annotated as neutral, negative, or positive in the Reference subcorpus of ParlaMint-GB. Mentions of the European Union/EU in neutrally annotated sentences appear to be more frequent and more characteristic than those in sentences annotated as negative or positive.

2To answer RQ2.1, complete the following tasks.

1. Adjust Step 1 and Step 2 to extract frequency information regarding explicit references to the European Union by MPs after Brexit (in the combined COVID and COVID, War subcorpora) and save these results for subsequent comparison.
2. Adjust Step 1 and Step 2 to extract frequency information regarding explicit mentions of Europe by MPs before and after Brexit (in the combined COVID and COVID, War subcorpora) and save these results for subsequent comparison.

Interpreting the results

1. Compare the retrieved relative frequency of references to the European Union/EU in the Reference subcorpus with those retrieved from the COVID and COVID, War subcorpora. Consider whether the pattern of predominantly mentioning the European Union in neutrally marked sentences changes after Brexit.

   1For ease of comparison and interpretation, it is advisable to visualise the results using simple bar charts, as in Figure 72 and Figure 73, or line graphs.

   

   
   
2. Compare the retrieved relative frequencies on mentions of Europe (Figure 73) with those of the European Union (Figure 72) across the two periods. Does the tendency for mentions of the European Union to occur in neutrally annotated sentences (Figure 72) also apply to mentions of Europe (Figure 73)? Do you observe any other similarities or discrepancies in the relative frequency patterns under analysis before and after Brexit?

   

   
   

   1➽ noSketch Engine tip: because relative frequency and relative density represent different aspects of distribution, they should not be combined or interpreted as if they were directly comparable. As a reminder, relative frequency expresses how often an item occurs in relation to the size of a given text type or subcorpus, whereas relative density shows how frequent a word or phrase is in a particular text type compared with its frequency in the whole corpus, taking into account the size of that text type. It therefore indicates how typical or characteristic the item is of that text type. Since these measures capture distinct dimensions of usage, students must keep them analytically separate and avoid drawing conclusions by comparing one measure from one subcorpus with the other measure from another.

3. Compare the retrieved relative density of mentions of the European Union (Figure 74) across the two periods. Remember that sentiment is annotated at the sentence level, and a sentence labelled as “Negative”, “Neutral”, or “Positive” does not necessarily express a negative, neutral, or positive stance toward the EU itself, although it is mentioned in those sentences. However, you can make some preliminary observations about more typical or less typical sentiment contexts for mentions of the European Union before and after Brexit. What is the most noticeable change in relative density after Brexit, in your view?

   

   
   
4. Compare the retrieved relative density of mentions of the European Union (Figure 74) across the two periods with those of Europe (Figure 75).Which mentions – of the European Union or of Europe – show a stronger shift in sentences with the sentiment attribute after Brexit, and in which direction?

1In answering RQ2.1, the numerical data obtained thus far indicate that, after Brexit, references to the EU occur far less frequently in ParlaMint-GB. Although such references continue to appear characteristically in neutral contexts, there is a moderate shift towards their occurrence in sentences annotated as negative. As with the EU, mentions of Europe also decline sharply after Brexit across all sentiment categories, suggesting a general reduction in Europe‑related discourse in the UK Parliament. Unlike the EU, however, Europe tends to be used in more evaluative (non-neutral) contexts and displays a more pronounced shift towards negative density in the post‑Brexit period.

2That said, further interpretation and explanation of these results require the use of additional techniques of both distant and close reading, together with a critical evaluation of the broader sociopolitical context. For instance, negative sentences mentioning Europe may relate to the Covid‑19 pandemic or to Russia’s war of aggression against Ukraine, as illustrated in the examples below:

“My hon. Friend the Member for South Suffolk (James Cartlidge) referred to the disappointing news that Edinburgh is regarded as the covid capital of Europe. I will not be political on this one; I just think that it is a disappointment and that all colleagues across the House will hope that, with our heated debate and constructive criticism, we will get a better result quickly.” (ParlaMint-GB, Gagan Mohindra, 2021-07-07)

“There is a war in Europe now. Russia has invaded Ukraine. It is the most ghastly act—a number of speakers have said how unacceptable it is—but it is a war.” (ParlaMint-GB, Alan William John West, 2022-02-25)

3To develop a robust interpretation, you should therefore consider how different contexts shape the sentiment of the sentence as a whole, rather than assuming that a negative sentence necessarily reflects a negative attitude towards Europe (or any other concept of interest). You are encouraged to reflect on how you might incorporate these contextual factors into your own analysis and to think critically about what additional evidence would be needed before drawing firm conclusions.

Step III: calculating ratio of relative frequencies as an indicator of political polarisation in the UK parliament

1In Steps III-V, we focus on seeking answers to RQ2.2: Do polarity-bearing attitudes by MPs mentioning the European Union in the UK Parliament split along party lines and, if so, did Brexit influence this division?

2First, we need to operationalise it for our further exploration of ParlaMint-GB via the NoSketch Engine concordancer. The operationalisations below represent one possible way of approaching RQ2.2; however, they do not exhaust all potential analytical pathways. Other approaches are certainly possible, but for the purposes of Steps III-V we will concentrate on the following two options, which do not include the lemma Europe:

3Possible operationalisations of RQ2.2

• Does the ratio of relative token frequencies of the lemma EU and the sequence European Union in negatively annotated sentences spoken by MPs affiliated with various parliamentary parties differ between the Reference subcorpus and the combined COVID and COVID, War subcorpora?
• Does the ratio of relative token frequencies of the lemma EU and the sequence European Union in positively annotated sentences spoken by MPs affiliated with various parliamentary parties differ between the Reference subcorpus and the COVID and COVID, War subcorpora?

4Review Steps I and II, then compose the appropriate queries to extract the required relative token frequencies. Revisit Showcase I for an illustration of how the ratio of relative token frequencies is calculated and interpreted. Try to complete these searches independently; if you encounter difficulties, consult the guidelines provided below.

1. Write the following complex query in the CQL search line on the Concordance page::
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Negative"/> within <speech subcorpus="Reference" & speaker_mp="MP"/> and click the Go button.
2. Once the concordance has been generated, navigate to the Speech.speaker_party_name section on the Frequency page (Figure 76), using either the Basic or the Advanced tab. Note that the parliamentary parties are already ranked by their absolute frequencies. Select the top five parties (Conservative, Labour, Liberal Democrat, Scottish National Party, Crossbench) and save the relative frequencies of references to the EU/European Union in negatively annotated sentences in the Reference subcorpus. These data will be needed for subsequent comparisons.

   

   
   
3. Repeat the same procedures for the other three queries:
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Positive"/> within <speech subcorpus="Reference" & speaker_mp="MP"/>
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Negative"/> within <speech subcorpus="COVID|COVID,War" & speaker_mp="MP"/>
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Positive"/> within <speech subcorpus="COVID|COVID,War" & speaker_mp="MP"/>
4. Once you have extracted the relative frequencies of EU‑related mentions in positively and negatively annotated sentences for the selected parliamentary parties in both the Reference and the combined COVID and COVID, War subcorpora, calculate the ratio of these relative frequencies and report the results in a table.
5. After completing the table, examine the patterns that emerge across parties and across time. Consider how these ratios might reflect broader political dynamics and discuss what additional contextual information would be required before drawing firm conclusions about party‑political attitudes towards the European Union.

Interpreting the results

1For reference, Table 9 shows that, across both periods, members of the Scottish National Party, Labour, the Liberal Democrats, and the Crossbench group consistently used EU-related references more frequently in negative than in positive contexts, with these ratios increasing after Brexit. By contrast, Conservative members maintained a ratio slightly below 1 in both periods, indicating a tilt towards positive sentences and a generally more balanced distribution of positive and negative sentences containing EU references compared with their counterparts.

2That said, it remains uncertain whether the expressed sentiment is directed at the EU itself or, for example, at the UK’s withdrawal from it. This is because sentiment annotations applied at the level of the whole sentence do not necessarily capture the speaker’s attitude towards specific concepts mentioned within that sentence (see the Caution note above).

Step IV: reading raw sentiment scores and concordance lines

1Given that sentiment annotations do not reliably indicate the speaker’s attitude towards the EU itself, a close reading of selected concordance lines becomes essential. This procedure enables researchers to examine the linguistic context surrounding EU-related mentions and to determine whether the polarity reflects approval, criticism, or something else entirely. It also helps distinguish between attitudes directed at the EU as an institution and those directed at specific events, such as Brexit. Without this qualitative step, quantitative sentiment ratios risk oversimplifying or misrepresenting the speakers’ actual positions.

1. To illustrate the point, we randomly select concordance lines from negatively annotated sentences in the post-Brexit period, spoken by MPs from the parties with the highest and lowest ratios in Table 9 above, i.e. the Scottish National Party and the Conservative Party. Using the senti_n scale, which provides raw sentiment values, including scores below 0 and above 5 (see Sentiment information), we examine lines with sentiment scores below zero (see Figure 77). We hypothesise that these instances are more emphatically negative than sentences annotated within the 0–5 range and therefore offer a promising starting point for the close-reading procedure.
2. To retrieve negative sentences by SNP and Conservative Party members that mention the European Union, compose the appropriate query and enter it into the search line. For your reference, you may consult the following CQL query:
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Negative"/> within <speech subcorpus="COVID|COVID,War" & speaker_mp="MP" & speaker_party="CON|SNP"/>
3. After running the query, go to the Frequency page and open the concordance lines of interest. You can do this either by clicking the three dots and selecting Only this option in the drop-down box to open the concordance lines on the same page, or by using the pop-out icon to open them in a new page (Figure 77). Alternatively, you can tick the relevant item and click Selected or use the corresponding pop-out icon (Figure 78).

   

   
   

   

   
   
4. The concordance lines can then be grouped according to whether they were produced by SNP or Conservative members using the Frequency page (Figure 79) or alternatively filtered by party name via the Filter page (Figure 80).

2To demonstrate the benefit of integrating quantitative sentiment analysis with a close examination of concordance lines, we analyse instances of the terms EU and European Union in sentences produced by SNP and Conservatives that share an identical raw sentiment score of –0.043.

Interpreting the results

1A close reading of concordance lines in Figure 81 reveals that SNP members portray the EU as a positive force, emphasising its protective role in areas such as cultural exchange, food standards, and regional identity. Their utterances convey a sense of grievance and loss, with phrases like “dragged out… against our will” and “a kick in the teeth” signalling frustration at the perceived consequences of Brexit for Scotland. In these examples, the EU is framed as a guarantor of rights and quality, and its absence is associated with tangible harm to Scottish interests.

2In contrast, Conservative MPs (Figure 82) adopt a more critical and confrontational tone towards the EU. Their references include accusations of inflexibility, manipulation of trade protocols, and economic inefficiency. The EU is portrayed as obstructive or adversarial, with the wording such as “conning,” “imposing a border,” and “threat to disrupt exports”, and its representatives as irritable and overreacting, “as if they had been stung by wasps”. Although some lines acknowledge the need for continued negotiation, the overall sentiment leans towards scepticism and strategic distancing. Conservative speakers also take the opportunity to criticise the Scottish Government for “waiting and following the European Union.”

3This contrast underscores a fundamental divergence in how the two parties frame the EU: SNP members tend to lament its loss and foreground its perceived benefits, whereas Conservative members are more likely to question the EU’s intentions and emphasise themes of sovereignty and control.

4A close reading of the concordance lines makes clear that quantitative sentiment data, taken in isolation, may be misleading and cannot capture nuanced rhetorical strategies or the different evaluative stances embedded in each party’s discourse.

Step V: extracting collocates

1While close reading helps uncover the contours of meaning and speaker intent, collocation analysis can map the broader terrain by identifying statistically significant word pairings that signal prevailing evaluative patterns across large corpora, thereby offering a valuable analytical lens for examining wider discourse tendencies.

1. To compare collocates of the EU/European Union used in negatively annotated sentences by SNP and Conservative Party in the post-Brexit period, use a respective complex query for each party separately.
2. Compose and run a CQL query for SNP:
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Negative"/> within <speech subcorpus="COVID|COVID,War" & speaker_mp="MP" & speaker_party="SNP"/>
3. Click the Collocations icon on the Concordance page, set the collocation parameters and push the Go button (Figure 83). The collocation parameters specified here ensure that the retrieved collocates of the EU/European Union are lowercase lemmas occurring within a span of 5 tokens to the left and 5 tokens to the right from the search terms. Candidate collocates should appear at least 5 times in the whole corpus and at least 3 times within the given range. Collocates are computed and ranked using the Log Dice statistical measure. Each of these collocation parameters may be adjusted to suit the requirements of your own research design.

   

   
   
4. You can download and save the results locally (Figure 84). Also, it is recommended to keep the browser tab with these results open or to save the URL for easy return, as we will revisit these collocates shortly to explore the data further.

   

   
   
5. Repeat procedures 1-4, subsequently querying negatively annotated sentences used by Conservative Party and positively annotated sentences used by SNP and Conservative Party, all in the post-Brexit period:
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Negative"/> within <speech subcorpus="COVID|COVID,War" & speaker_mp="MP" & speaker_party="CON"/>
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Positive"/> within <speech subcorpus="COVID|COVID,War" & speaker_mp="MP" & speaker_party="SNP"/>
   • [lemma_lc="EU"] | ([lemma_lc="European"] [lemma_lc="Union"]) within <s senti_3="Positive"/> within <speech subcorpus="COVID|COVID,War" & speaker_mp="MP" & speaker_party="CON"/>
6. Compare the Log Dice values across the retrieved collocate lists and select collocates for further examination.
7. Closely read contexts of the selected collocates in which they are used by clicking on the three dots and opening respective concordance lines.

Interpreting the results

1For illustrative purposes, we focus here on the top 10 collocates in each of the retrieved lists and select collocates with the Log Dice value above 7 for closer analysis (Figure 85 and Figure 86), as such collocates are regarded to have strong associations with the search terms (Jaworska and Kinloch, 2018: 122).

2Probably, the most obvious observation, when comparing the collocate lists in Figure 85 and Figure 86 above, is the polar difference in sentiment related to the lemma rejoin. It emerges as the strongest collocate of the EU/European Union in positively annotated sentences uttered by SNP members and in negatively annotated sentences uttered by Conservative Party members in the post-Brexit period.

3A close reading of concordance lines helps clarify this difference in sentiment. While SNP members discuss rejoining the EU as a desired scenario for Scotland (Figure 87), Conservative MPs criticise predominantly the SNP, but also members of some other parliamentary parties, for expressing a desire to rejoin the EU (Figure 88).

4A close reading of concordance lines containing the lemma drag, which is the strongest collocate of the EU/European Union in negatively annotated sentences uttered by SNP members in the post-Brexit period, reveals a recurring narrative of involuntary separation (Figure 89). The patterns “dragged out of” in the left-hand contexts and “against … will / wishes” in the right-hand contexts appear repeatedly, emphasising Scotland’s lack of agency and the perceived violation of its electoral mandate. This framing casts Brexit not as a collective national decision but as an imposed rupture, often linked to economic harm, political betrayal, and cultural loss. The collocate drag thus functions as a rhetorical anchor for grievance, reinforcing the SNP’s broader argument for autonomy and re-engagement with the EU.

5This reading is also corroborated by examining sentiment information of all the concordance lines from Figure 89 above according to the 6-level scale (s.senti_6). As seen in Figure 90 below, 13 out of 14 instances containing the lemma drag are annotated as negative and only one as mixed negative.

6On the other hand, the four strongest collocates of the EU/European Union in positively annotated sentences uttered by Conservative Party members in the post-Brexit period are departure, leave, outside, exit. Their prominence indicates that references to the European Union by Conservative Party members remain strongly framed through the lens of withdrawal. However, unlike in the case of drag used by SNP members, sentiment of the sentences containing the lemmas departure, leave, outside, exit co-occurring with EU/European Union in positively annotated sentences uttered by Conservative Party members in the post-Brexit period is relatively evenly distributed between positive and mixed positive according to the 6-level scale (Figure 91).

Step VI: comparing keywords to capture sentiment differences

1To capture more nuanced differences between positive and mixed positive sentences containing the lemmas departure, leave, outside, exit that collocate with the EU/European Union used by Conservative Party members in the post-Brexit period, we can resort to keyword analysis. Our actions are as follows:

1. Create a subcorpus from the positive sentences displayed in Figure 91 above by opening the respective concordance lines in a new window, clicking the Plus icon, naming your subcorpus and saving it, as shown in Figure 92 below (see also concordance-based subcorpus creation).

2. Repeat the procedures above to create a subcorpus from the selected mixed positive sentences.
3. Navigate to the Keywords page by clicking the respective icon on the vertical toolbar, select one of the newly created subcorpora as your focus subcorpus, ParlaMint-GB 5.0 as your reference corpus and the other newly created subcorpus as your reference subcorpus, as shown in Figure 93. Identify keywords as lowercase lemmas and click the Go button.

4. Save the results.
5. Switch between the focus and the reference subcorpora by clicking the icon with rightwards arrow over leftwards arrow in the top left corner.
6. Save the results.
7. Compare the top 20 keywords from the positive_… subcorpus with the top 20 keywords from the mixed-positive_… subcorpus.

Interpreting the results

1In Step VI, we obtain two keyword lists to better understand the range of positive sentiment in sentences containing the terms EU/European Union and their four strongest collocates departure, leave, outside, exit, which were uttered by Conservative MPs in the post-Brexit period. The list on the left contains keywords that are more typical of the positive_CON_post-Brexit_collocates-departure-left-outside-exit subcorpus and the list on the right contains keywords that are more typical of the mixed-positive_CON_post-Brexit_collocates-departure-left-outside-exit subcorpus (Figure 94).

2A close examination of the keyword list representing the positive_… subcorpus, coupled with an inspection of selected concordance lines, suggests that it includes words encoding positive emotional stance (proud, delighted, confident), positive evaluation (excellent, quality), intensifiers (really), lexis associated with goals, progress, or future‑oriented positivity (ambitious, ambition, vision, pursue, chance), expressions of centrality (at the heart of), as well as items indicating economic development (freeports), improvement, strengthening, or proactive engagement (boost, engage, attract).

3By contrast, the keyword list representing the mixed-positive_… subcorpus is dominated by items related to governance, law, regulation, and institutional processes (regulations, court, responsibility, appropriate, guarantee, function, political, SI, i.e. Statutory Instrument), along with lexis referring to specific policy areas (emission, vehicle, excise) as well as other common words and proper nouns that are predominantly neutral.

4Taken together, the two keyword lists reveal a palpable distinction in how positive sentiment is constructed across the two subcorpora. The positive_… subcorpus relies heavily on explicitly affective and evaluative lexis, including emotional adjectives and aspirational vocabulary, alongside policy‑linked terms framed as opportunities. By contrast, the mixed‑positive_… subcorpus is dominated by governance‑, regulation‑, and institution‑related terminology, supplemented by technical policy‑domain items and largely neutral procedural vocabulary. This suggests that positive sentiment in the positive_… subcorpus is conveyed primarily through overtly positive lexical choices, whereas in the mixed‑positive_… subcorpus it emerges more subtly from contextual framing within otherwise neutral or administrative language.

5To finalise answering RQ2.2, complete the following tasks.

1. Repeat Step V to extract collocates of EU/European Union in positively and negatively annotated sentences by SNP and Conservative Party members uttered before Brexit.
2. Repeat Step VI for selected collocates before Brexit.
3. Compare your results with the results discussed in Step V and Step VI. What similarities or discrepancies do you observe across both periods?
4. Additionally, you can obtain, analyse and compare sentiment data for other parties represented in the UK Parliament (for instance, the Labour Party), or select any other parliament corpus (corpora) from the rich ParlaMint family and develop your own research questions.

6To conclude, this showcase has examined the challenges and opportunities of applying sentence-level sentiment annotations to word-level expressions in parliamentary discourse. By tracing sentiment variation over time and across party lines in debates concerning the European Union, we have highlighted the importance of aligning linguistic granularity with analytical aims. While quantitative data and corpus techniques such as collocation analysis, keyword analysis and concordance reading offer valuable insights into political polarisation, they must be contextualised within broader rhetorical and ideological frameworks to avoid misinterpretation. As with all corpus-assisted research, methodological rigour and political awareness are essential to ensure that sentiment analysis contributes meaningfully to our understanding of parliamentary language and its evolving dynamics.