Do state-of-the-art natural language understanding models care about word order - one of the most important characteristics of a sequence? Not always! We found 75% to 90% of the correct predictions of BERT-based classifiers, trained on many GLUE tasks, remain constant after input words are randomly shuffled. Despite BERT embeddings are famously contextual, the contribution of each individual word to downstream tasks is almost unchanged even after the word's context is shuffled. BERT-based models are able to exploit superficial cues (e.g. the sentiment of keywords in sentiment analysis; or the word-wise similarity between sequence-pair inputs in natural language inference) to make correct decisions when tokens are arranged in random orders. Encouraging classifiers to capture word order information improves the performance on most GLUE tasks, SQuAD 2.0 and out-of-samples. Our work suggests that many GLUE tasks are not challenging machines to understand the meaning of a sentence. 3.3). 3. In sentiment analysis (SST-2), the polarity of a single salient word is≥ 60% predictive of an entire sentence’s label (Sec. 3.4.1). 4. BERT-based models trained on sequence-pair GLUE tasks used a set of self-attention heads for finding similar tokens shared between the two inputs (Sec. 3.4). 5. Encouraging RoBERTa-based models to be more sensitive to word order improves the performance on SQuAD 2.0 and most GLUE tasks tested (i.e. except for SST-2) (Sec. 3.5). Despite their superhuman scores, most GLUE- trained models behave similarly to Bag-of-Words (BOW) models, which are prone to naive mistakes (Fig. 1b–d). Random shuffling methods To understand model behaviors across varying degrees of word- order distortions, we experimented with three tests: randomly shuffling n-grams where n ={1, 2, 3}. Shuffling 1-grams is a common technique for an- alyzing word-order sensitivity (Sankar et al., 2019; Zanzotto et al., 2020). We split a given sentence by whitespace into a list of n-grams, and re-combined them, in a random order, back into a “shuffled” sentence (see Table 1 for examples). The ending punctuation was kept intact. We re-sampled a new random permutation until the shuffled sentence was different from the original sentence. 2.3.1); and (2) dev-s i.e. a clone of version of dev-r but that each example has one sentence with re-ordered words (Sec. 2.3.2). 2.3.1 Selecting real examples For each pair of (task, classifier), we selected a subset of dev-set examples via the following steps: 1. For tasks with either a single-sequence or a sequence-pair input, we used examples where the input sequence to be modified has only one sentence 2 that has more than 3 tokens (for shuffling 3-grams to produce a sentence different from the original sentence). 2. We only selected the examples that were cor- rectly classified by the classifier (to study what features were We shed light into this question by testing model performance when word order is increas- ingly randomized. If a task strictly requires words to form a se- mantically meaningful sentence, then randomly re-positioning words in correctly-classified sen- tences will cause model accuracy to drop from 100% to 50% (i.e. the random baselineb for binary- classification tasks with two balanced classes). Thus, to compare model-sensitivity across tasks, we use a Word-Order Sensitivity score (WOS): s = (100−p)/(100−b) (1) wherep∈ [50, 100] is the accuracy of a GLUE- trained model evaluated on adev-s set (described in Sec. 2.3.2) ands∈ [0, 1]. Here,b = 50. Consistently across three different BERT “base” variants and a RoBERTa “large” model (Table A5), our results suggest that word order and syntax, in general, are not necessarily required to solve GLUE. 2-noun swaps Besides shuffled n-grams, we also repeated all experiments with more syntactically- correct modified inputs where only two random nouns in a sentence were swapped (Table 1; Q s). This is a harder test for NLU models since the meaning of a sentence with two nouns swapped often changes while its syntax remains correct. We found the conclusions to generalize to this setting. That is, the models hardly changed predictions al- though the meanings of the original sentence and its swapped version are different (Table 2b; 2-noun swap vs. 1-gram). 3.2 How sensitive are models trained to predict the similarity of two sentences? An interesting hypothesis is that models trained explicitly to evaluate the semantic similarity of two sentences should be able to tell apart real from shuf- fled examples. Intuitively, word order information is essential for understanding what an entire sen- tence means and, therefore, for predicting whether two sentences convey the same meaning. the highest LIME-attribution score) per example by looking it up in the Opinion Lexicon (Hu and Liu, 2004) of 2,006 positive and 4,783 negative words.∼57% of these top-1 words were found in the dictionary and labeled either “positive” or “negative” (see Table A3). Results We found that if the top-1 word has a positive meaning, then there is a 100% probability that the sentence’s label is “positive”. For exam- ple, the word “thrilling” in a movie review indi- cates a “positive”sentence (see Fig. 3). Similarly, the conditional probability of a sentence being la- beled “negative” given a negative top-1 word is 94.4%. Similarly, the conditional probability of a sentence being labeled “negative” given a negative most important word (by LIME Ribeiro et al. 2016) is 94.44%. 1159 LIME attributions (negative -1, neutral 0, positive +1) CoLA example. Groundtruth: “acceptable” S Medea denied poisoning the phoenix. “acceptable” 0.99 S1 poisoning the phoenix denied Medea. “acceptable” 0.53 S2 phoenix Medea denied the poisoning. “acceptable” 0.99 S3 Medea the poisoning phoenix denied. “unacceptable” 0.95 S4 phoenix Medea denied the poisoning. “unacceptable” 0.99 S5 Medea phoenix poisoning the denied. “unacceptable” 0.96 Figure A2: Each CoLA example contains a single sentence.