In my recent blog post I explained the politics of autocomplete, a feature that suggests queries while they are being typed. We may wonder how this is affected by the trend of creating tailor-made search engine results for specific audiences. How do Google´s suggestions differ from country to country (and language to language)?
Methodology and research question
I tried to gain insights into this question during a small project at the summer school of the Digital Methods Initiative (DMI) this year. With a tool provided by the Digital Methods team I was able to conduct a limited cross-cultural comparison for the query “9/11”. On July 4th and 5th 2012, I crawled the query in 4 languages (English, Arabic, Hebrew, German) in 12 country versions (Australia, Canada, UK, US, Egypt, Lebanon, Palestinian Territories, Iraq, Israel, Germany, Austria, Switzerland). Before I noticed that Google ranked websites with alternative accounts of 9/11 (commonly described as “conspiracy theories”) fairly high (see my post here). Thus, I wanted to know whether this is also reflected in the autocomplete suggestions and if there are differences between the language versions. While the used tool is very helpful to retrieve the suggestions, it does not provide any help for interpreting them. The output comes in form of numerous tables, simply listing the suggestions in the order of their appearance in Google, together with some additional information (see below). Google´s API allows for retrieving ten results (the suggestions have partly been limited to four in the regular search interface).
Output of the DMI autocomplete tool
To interpret this vast amount of data, I categorized each suggestion with Grounded Theory oriented content analysis. That means these categories resulted from the data itself, rather than forcing them onto the data before analysis. The outcome is eight categories which helped to structure the data.
Categories for the suggestions
To get a summarizing overview of all the data, I created a word cloud (with Wordle) which shows all the suggestions in relation to their frequency (the bigger the word, the more often it appeared in the suggestions) and in the color of the categories. The rare cases of suggestions in non-Latin script (Arabic/Hebrew) were translated into English and brackets indicating the original script were added.
Results
The word cloud immediately reveals a striking result: Google´s suggestions for 9/11 over-represent alternative accounts of the event, most notably the word “conspiracy” which was suggested most often. Additionally, a number of “conspiracy-related” queries appear, for example, “truth” (pointing to the so-called “9/11 truth movement” which advocates alternative accounts) or the catch phrase “[was an] inside job”.
Overview: Word cloud of all suggestions
On the contrary, only few suggestions refer to the mainstream account or “neutral” facts. Another frequently suggested term is “memorial”, probably pointing to the 9/11 memorial museum in New York City. Less visible, but also striking are the rare suggestions of queries which are not related to September 11, 2001. These are more common in the Arabic countries, for example in regard to a section in the Quran. The following tables show the categorized results for all the analyzed language and country versions.
Google´s suggestions for the query “9/11” in English
Google´s suggestions for the query “9/11” in German
Google´s suggestions for the query “9/11” in Hebrew
Google´s suggestions for the query “9/11″ in Arabic
The following figure shows only the categories but gives a better overview for direct comparison.
Categorized suggestions in comparison
The differences across the language versions are significant, especially between the Arabic and the Western countries, while there are only slight variations between countries with the same language version. Alternative accounts seem to play a major role in the Arab world, followed by German-speaking countries, Hebrew and finally the English-speaking countries. The latter ones apparently have a more heterogeneous interest in 9/11, including jokes which otherwise only appear in Hebrew but in none of the Arabic or German language versions. In all Western countries “memorial” is the most popular query, whereas all Arabic countries suggest the slogan “was an inside job” first.
Discussion
Google´s suggestions for the query “9/11” dominantly associate it with the events of September 11, 2001. Given the massive global impact of this incident this was expectable, although we can think of other meanings for this query (for example the car Porsche 911, other events on September 11 or the emergency number). The general popularity of alternative accounts is striking. The observation that they are particularly relevant in the Arabic world is supported by a representative study which showed that many people in the Arabic world see other forces behind 9/11 than Al Qaeda. However, we must be careful when we draw conclusions about a society´s opinion based on Google´s suggestions. First of all, we cannot be sure if they really represent what users actually search for, although that´s what Google claims. Secondly, even if autocomplete represents previous queries, they give us only insights into a very specific part of a population, namely those who actively search for the term “9/11” with Google. This also implies a certain language bias. Although the term 9/11 is also commonly related to the September 11 attacks in non-English countries, it is still mostly used in English queries. Only in the German-speaking countries we find combinations where 9/11 is paired with a local expression and related to the incident (e.g. “9/11 wahrheit”, “9/11 ablauf”). Therefore, it might be useful to conduct comparative studies with queries in the local language.
Still, these results give interesting insights into the local differences of Google´s search engine. They show that the autocomplete suggestions vary significantly between language versions but rather slightly between countries within one language.