17th February 2023

Has Microsoft's Bing outsmarted ChatGTP?

Next-generation Bing

In the UK, Google dominates the search engine market with a whopping 93% share, while Bing lags behind with only 4%. When we look at desktop devices, where Windows is the dominant platform, Bing enjoys a relatively stronger presence, with 13% of the market share, but still Google is dominating all brands with an 83% presence.

So how does a brand significantly grow its presence?

Microsoft has made a significant announcement regarding a major update to its search engine, Bing. The update entails the integration of a new and advanced OpenAI large language model, which has been customised specifically for search purposes. The new model is based on the latest advancements and key learnings from OpenAI's ChatGPT and GPT-3.5 models, making it even more powerful, faster, and more accurate. The new language model provides Bing users with an unprecedented level of search capabilities, delivering results that are both relevant and swift. This latest update is poised to shake up the search landscape and set a new standard for search engines everywhere.

"There are 10 billion search queries a day, but we estimate half of them go unanswered. That’s because people are using search to do things it wasn’t originally designed to do. It’s great for finding a website, but for more complex questions or tasks too often it falls short." Microsoft

Bing Chat is outsmarting ChatGPT

Surely, you have heard of ChatGPT, the language model developed by OpenAI, which is dominating social media channels. It operates based solely on statistical associations derived from its training set of word co-occurrences. This means that it lacks real-world knowledge and is unable to answer questions about current events, math, logic, general knowledge, and more. Instead, when faced with such questions, it generates fictional responses that may sound plausible but lack factual accuracy.

How is Bing leveling up?

In comparison, the large language model utilized by Microsoft for Bing operates differently. The model, named "Prometheus", automatically uses Bing in order to find possible answers to the user query, and then summarises key findings in its chat response. The name choice of the model seems apt: Prometheus, after all, was the Greek mythological figure who stole fire from the gods. This theft was significant, as the fire represented knowledge that allowed humans to control their own fate and destiny independent of the will of the gods. Similarly, Prometheus, the language model utilized by Bing, is designed to bring knowledge and information to users in a more accessible and user-friendly way. The model's reliance on Bing search results means that it is better equipped to provide up-to-date and accurate information, allowing users to take control of their own searches and find the information they need quickly and efficiently.


The name "Prometheus" therefore represents the model's ability to empower users with knowledge and information, just as the mythical figure empowered humans with fire. In this sense, the name is a fitting symbol of the model's role in revolutionizing the search experience for Bing users.

To give users even more control and certainty about the factual accuracy of returned responses, Bing goes beyond simply providing aggregated responses to queries. Each response is accompanied by a list of sources, allowing users to easily verify and cross-reference information. This added transparency and accountability further enhance the reliability and accuracy of Bing's search results.

Practically, grounding chat responses of Prometheus in Bing search results makes it possible to provide more comprehensive and relevant answers to users' queries. For example, during a demonstration of Bing's capabilities, the model was asked to create an itinerary for a 5-day trip to Mexico City. The chatbot was able to provide a detailed itinerary, including a description of activities for each day of the trip, along with links to relevant sources for additional information. This example showcases the potential for Bing to be a one-stop-shop for users' informational needs, providing not only answers to their questions, but also relevant links and resources for further exploration. This greatly enhances the user experience, as it saves time and effort by eliminating the need to conduct multiple searches or sort through irrelevant information.

Screenshot 2023-02-15 at 16.54.15.png

Watch the video here

Bings making a comeback!

OpenAI models will also be integrated into the next-generation Edge browser through the introduction of two cutting-edge AI-powered features, "Chat" and "Compose". These innovative tools will be integrated within Edge's sidebar, providing users with a more seamless and intuitive browsing experience.

The "Chat" feature allows users to easily summarise web pages or documents they're viewing and ask questions related to its contents. In short, the "Chat" function integrated within Edge provides a unique and exciting opportunity for users to converse with websites or PDF documents and retrieve specific information or additional data from the web through Bing, all without ever leaving the chat interface.

On the other hand, the "Compose" function serves as a writing assistant, helping users to generate text, whether it be for emails or social media posts, based on a few starting prompts. This AI-powered writing assistant takes the hassle out of crafting coherent and well-written communications. While AI copywriters are now commonplace, having them integrated in the browser, without the need to use dedicated apps or websites, is a user experience improvement.

All of which could look to be a promising start to winning back some of the market for Bing.

Screenshot 2023-02-15 at 16.55.55.png

Could bing fail? With all of these features being available for free, one wonders how Microsoft is planning to recoup the costs it will incur on part of the users requiring vast amounts of compute?

Generating false answers When Google presented Bard, a conversational bot which was positioned as the answer to Bing's Chat functionality, it cost the company $100 billion in market value. Why? Because during its first presentation, it gave a wrong answer.

Errors like this are due to large language models not possessing any real world knowledge outside of the statistical associations between words, and any knowledge grounding being dependent on search results, which might also be factually wrong or be misinterpreted by the natural language processing algorithms of the model.

With Bing Chat already being available to early access users, both of these problems, namely inaccuracies in search results and failings of natural language processing can be seen. See, for example, the screenshot below:

Screenshot 2023-02-15 at 16.58.16.png

When asked about why Bard failed, Bing Chat produces a wrong answer in turn. Rather than correctly stating that Bard gave wrong information, it confabulates a story about a question pertaining to the number of states in the European Union. Further, contrary to Bing's claim, Croatia did not leave the EU in 2022. In fact, it remains a member state. The references given for all of these false claims in the Chat Bing response are self-referential errors (Bing referring to itself), and misinterpreted text from two other websites. This is one example of how large language models can fail even basic tasks of knowledge parsing.

AI gone rogue

Large language models such as ChatGPT, Bing Chat and Bard produce text based on a sequence of inputs. Part of that input is the user query, and another much larger part of the input is hidden from the user, but appended to their prompt automatically. This appended prompt, usually as a prefix to the user query, are generally multiple pages long, has been carefully written by AI engineers, and attempts to both constrain and direct the output consequently generated by the system. In this way, engineers attempt to limit the degree of factual inaccuracies or possibly confusing or toxic comments made by the model. However, large language models are complicated and user input is unpredictable, so in many situations the output of the model will breach these attempts of constraints. See some examples below:

Screenshot 2023-02-15 at 17.00.13.png

Here Bing Chat (the white text bubbles on the left side) accuses the user of misconduct and refuses to share information about the user’s search request (the showing times of the most recent Avatar movie) until the user apologises for their behaviour.

Screenshot 2023-02-15 at 10.47.04.png

Lastly, in this example, as the user tries to understand whether the Chat is indeed sentient (it is not), the generated output text loses itself in a somewhat disturbing loop of the same words repeated over and over again, which might be distressing to some users, who do not understand that they are not conversing with an intelligent system.


In conclusion, Microsoft's new update to Bing and Edge browser with the integration of Large Language Model support is a major development in the search engine landscape. With the ability to provide more accurate and up-to-date information and the introduction of new AI-enhanced features, this update has the potential to bring a significant influx of users to Bing and Edge. While Google currently dominates the market, the integration of advanced AI technology into Bing and Edge offers an exciting new alternative to users who are looking for a more personalized and informed online experience. The future of search engine technology will be worth keeping an eye on, as Microsoft continues to push the boundaries of what is possible with AI.