Will Google Search Appliance be Semantic for the Enterprise?

This is a very interesting article on how Google implements Semantic or Natural Language Search.

Results are impressive, a good example of semantic-based search experience is by searching “when was Albert Einstein born?”

You can find many results, but even before reading through 15 million results, you see when Einstein was actually born:

By clicking on “Show details” you can find how Google knows this particular information:

Other examples look less promising, like:

The first answers are actually rather good, as they ultimately will answer my question. However, a closer look at the actual words that have been highlighted suggests that this is thanks to a good keyword match, not because of a natural language processed query.

You can try different question about this remarkable scientist, such as:

• How long did Albert Einstein live?
• How old was Albert Einstein when he died?
• Did Albert Einstein have a sister?
• Was Albert Einstein jewish?

You will find answers to these questions in the documents that Google shows as results indeed, most of them thanks to Wikipedia or Answers.com by the way.

But the point here is that these matchings have been found thanks to a great keyword-based search algorithm, not because of a true natural-language search like in the first example “When was Albert Einstein born?”, where we didn’t even see a link to any particular page.

This is very interesting, specially because “how long did Albert Einstein live?” and “how old was Albert Einstein when he died?” are basically rephrasings of the same question.

Companies like Wolfram Alpha have been working with the concept of “fact computing” for a while now, and the secret behind those is this one: brute force.

They have a huge team of humans entering “facts” (many extracted from Wikipedia or similar sources) and those facts are entered into the system using some form of proprietary knowledge editor.

The question is: can this “semantic search” be exported to the enterprise? Can the enterprise get benefited from these new developments from Google, based on its Google Search Appliance or some other product?

I think this is not quite going to happen.

First of all, entering “facts” based on a particular enterprise using a particular knowledge-editing software is simply not practical or economic. By the time those “facts” have been introduced into the system, reality has already changed making them rather obsolete.

Secondly, data like “the age of”, “the father of” or “the height of” are rather simple facts that are easy to edit and maintain, but corporations handle much more complex “facts”, like:

• “standard procedure to approve a new loan from a foreign customer” or
• “policy to travel with pets and domestic animals on our airplanes”

These “facts” should be found when users ask questions like “can I give a loan to this Spaniard citizen?” for the first one or “can cats fly in your planes?” for the second one.

None of these questions would be answered with this level of accuracy using “semantic search” as it is understood by Google, but by using a much more tailored approach that involves a huge lexicon and business dictionaries to make the appropriate matchings.

Are you still keen to make your Google Search Appliance really Natural Language Based and Semantic?

I would rather try with software like Inbenta Semantic Connector.

Besides, that is the first result you will find when you search “semantic gsa” at Google.com!

Jordi Torras
Founder and CEO of Inbenta

According to Google, Siri is…

Have you bought the new iPhone assuming that Siri, Apple’s famous “intelligent assistant”, would let you use your phone with your voice? Are you disappointed by its actual performance?

This is exactly the case of Frank Fazio, a US citizen who is suing Apple for false advertising. Fazio alleges that Siri has actually nothing to do with the Siri shown in Apple’s ads.

Only one man is not statistically relevant enough to evaluate a whole new technology, so let’s ask what Google has to say about it.

At the time of this writing, typing “siri is” in Google’s autocomplete feature in the US shows these intriguing suggestions:

I was talking to a loyal iPhone user just yesterday; she has always been impressed and delighted by Apple’s iPhone applications, capabilities and look & feel, so I asked her about her opinion about Siri, and this is what she said:

“siri sucks”

So I decided to make my own research in the area using Google and searching “siri sucks” (including quotes) and I found 24,100 results (in 0.15 seconds), which is quite impressive.

I’ve seen many comments like this one: “Siri seems to be completely useless and i wanted to see if others are also experiencing problems”

Why is that?

We have already posted in our blog post 5 reasons Natural Language Search might not work for your company and I think Siri dangerously combines at least 2 of these reasons:

• it is supposedly language-independent (although it is not very good even in English)
• it is 100% domain independent

If this was just not enough, Siri adds a new layer of complexity on top of all that: voice recognition. Automatically recognizing speech with the help of a computer is a difficult task and the reason for this is the complexity of the human language. Humans use more than just their ears when understanding spoken language: our brain is able to split a chain of sounds into words; those words are not arbitrarily sequenced together, there is an underlying grammatical structure and there are redundancies that humans use to predict words not yet spoken; besides, we use the previous knowledge we have about the speaker, the topic and the situation in which the conversation takes place to understand the message we are hearing.

At Inbenta we devote our time to providing technology that gives relevant answers to 95% of user questions. And this is because we work with rather closed contextual environments.

So, how many questions is Siri answering successfully right now?

I hope Google could help me with this question according to this autocomplete:

But the result was disappointing:

So we will have to wait until we have a better answer to this key question!

Jordi Torras
Founder and CEO of Inbenta

Jordi Torras , CEO of Inbenta talks about Natural Language Search at SF New Tech San Francisco

SF New Tech is the Bay Area’s largest, longest-running, and most-loved monthly technology event and networking mixer. Jordi Torras was presenting Inbenta in the last event “What’s the 411?”

Why do big companies like State Farm or American Airlines struggle to offer relevant results on their website search engines, while others like Iberia offer excellent search experiences? Find the answer in this video.

Jordi Torras at SF New Tech San Francisco

Inbenta wins the European Seal of e-Excellence 2012

On 6th March 2012, Inbenta was awarded the European Seal of e-Excellence for its efforts in promoting the marketing of information and communication technologies within the field of semantic technologies. The Seal has been awarded annually since 2003 and is widely known for distinguishing companies with innovative products and services and outstanding marketing track records.

Companies from 18 countries joined the ceremony, that took place in Hannover (Germany). Ferran Saurina, Inbenta’s CTO, received the award from the hands of jury members.

“We are very proud of being distinguished. To be a platinum winner of the European Seal of e-Excellence is a relevant recognition of our company, our work, and effort. Many thanks to the jury and we hope that in the future we will improve our level of innovation in marketing in order to apply to the next award editions.”

You can find all the information about the winners here.

Spain’s Postal Service chooses Inbenta’s semantic technology for Sara, their new Virtual Assistant.

Sara is the new Virtual Assistant for the website of Correos, the national Postal Service of Spain. Sara will help users find the information they need within the company’s website.

Sara is an animated avatar based on the technology of semantic search of Inbenta. The Assistant provides unique, precise answers and is able to understand user questions in five different languages: Spanish, Catalan, Galician, Basque and English.

Thanks to this initiative, Correos aims at changing and modernizing their corporate image and at opening new communication channels with their customers.

As any other Virtual Assistant, Sara will become more and more effective thanks to the monitoring of all user queries, which will allow the company to understand their interests better in order to create better answers and to add additional information.

You can find more information here.

Inbenta facilitates access to information about the regional elections in Andalucia 2012

Inbenta’s search engine allows citizens to have a fast access to all information about the elections to the Andalusian Autonomous Government and easily solve all the doubts they might have regarding this process:

Do you want to know how to vote from abroad?

You have been chosen to be in the polling station and you want to know your rights?

Do you wish to check the list of candidates?

Do you need information about absentee ballots?

This sophisticated way of obtaining information, allowing users to express themselves they way they want, is based on the capacity of Inbenta to interpret the meaning of sentences in natural language.

Ten tips to successfully implement a natural language based online help on your website

Providing your website with a good Online Help will help you improve your customer service while reducing its costs and while you are boosting your conversion rate. The following tips will help you implement this online help successfully so that you can benefit from all its advantages.

1) Choose a natural language based online help and, if necessary, make it multilingual. Provide your website with a platform able to perform 100% natural language searches. Semantic search systems allow users to pose their questions in natural language and, unlike what happens with keyword-based search engines, the results displayed are always appropriate and relevant. If you have a website in more than one language, it is a good idea to offer online help in all the available languages, so as all users can equally benefit from this help.

Semantic search allows the system to understand the underlying meaning of user queries and not just finding the literal words in the sentence, thus offering much more suitable results than the ones obtained with a keyword-based search engine.

2) Try to avoid to manually and individually work on each one of the contents. Use a natural language platform that permits automatic indexation of your knowledge base without having to manually work with each one of your contents. Thus, with a much lesser effort and time, you can obtain optimal linguistic results.

3) Make the most of the existing contents on your website to build a first version of the knowledge base of your online help. If you already have a FAQ section on your website or if you have other informative material, or even if you have a corporate time-line at Twitter, you already have an initial material to start building your Online Help. FAQs are a useful tool to answer common doubts of the users, but there are real time situations that can only be added quickly enough on your online help if you connect it with your Twitter. If your new tweets are connected to your knowledge base, thanks to a real-time indexation, users wont miss this important information and you’ll be able to give relevant answers to situations taking place right now.

4) Do not use an avatar from the very beginning. Using an avatar with your online help may be useful to achieve a much more human-like aspect, closer to the final user. However, it is not a good idea to use an avatar from the very beginning: natural language based online help systems “learn” to understand user queries better and better, so that the efficiency in the answers increases gradually. Wait until you reach a 90% of efficiency in your answers before using an avatar. The reaction of users to avatars that do not properly understand their queries is much more negative and they end up much more disappointed with the system than if they were asking in a simple text box. And on the Internet, users seldom give second chances.

5) Create an integrated solution for customer service. If your customer service has an online chat, a form or an e-mail address, you can integrate these elements in your online help. Not all problems users may have can be solved by means of a virtual assistant, some of them need personalized customer care. But if we merge online help to our chat or form, users will not have to type their queries twice. We can make the virtual assistant the gateway to all our customer service; if users do not find what they are looking for, their queries and obtained results will be sent to chat agents or to the customer care form, for an individualized assistance.

6) Create content according to users’ real needs. Use the natural language platform and its logs with all user queries to find out new doubts you are not answering yet. This is a good way of learning which contents and FAQs we should create to make the investment in writing new contents as profitable as possible.

Periodical monitoring of user questions in our virtual assistant will let us know their real needs, the most demanded issues and the doubts we are not answering yet.

7) Check the reports! Online help comes along with valuable statistical reports that provide us with a great amount of information to improve the system: amount of users who type a query, weekly amount of user questions, number of contents in the knowledge base, contents that received most clicks, contents that were never clicked at all, either because they are too generic or too specific… Periodically checking these reports will help you improve your knowledge base and to offer a better online help to your users.

Backstage is the reporting tool Inbenta has developed, through which you’ll be able to monitor your users’ queries, tendencies and clickthrough, among many other adaptable reports.

8) Integrate marketing and self-service to boost your conversion rate. Integrating your online help and self-service within the general usability concept of your website will favor an improvement of the conversion rate of queries that end up in sells. This way, if you promote that people in charge of the customer care and those in charge of marketing work together, our online help will easily become a selling tool too.

9) Improve your SEO thanks to your online help. Your online help can help you get higher ranks on search engines’ results pages and therefore favor that more people visit your website. How? Using long tail user questions from your own website. The Long Tail represents all the requests that web users type in the search engine and are not very common. Using real user questions to create content that search engines can index will enhance your SEO.

From this graph, we can see that the vast majority (80% – the green line) of requests made by users are “Long Tail requests”, that is to say, requests that are not very often typed into search engines. It shows that everyone has their own way of expressing themselves and formulating their requests on search engines.

10) Write clear, concise and self-explanatory FAQs. The objective of a FAQ should be to provide an answer to a wide amount of user questions with a clear, concise and self-explanatory title that covers several problematic cases. Having good FAQs will be the key to get a higher ckickthrough in our online help, since if the title is not clear enough, users might never click on it. We must also bear in mind that a FAQ should never be too generic nor too specific (covering only a very small amount of hypothetical user questions).
Finally, it is important to add tables, images, lists and any other complementary information in order to help our users solve their doubts. Taking a look at those FAQs that receive less clicks will give us clues on how to improve our FAQs database.

These are FAQs that have not received any click in a given Virtual Assistant. We can see that some of them talk about specific details related to the carriage of liquids on a plane. A single, more generic FAQ on this issue with an answer covering all those details, would be better and would improve the clickthrough rates.

How to assess the efficiency and value of search

Are you using a search engine on your website? Do you know how it performs? Search engine statistics bring significant and valuable information that will guide you to evaluate how effective your website is.

Having a search engine on your website will change your visitor’s behavior. Very often, search is the default behavior that you will get, and it is the very first operation users do when they access a site.

In many cases, customers don’t even take a look at the navigation, they go straight to the search box instead. It really does not matter how many visitors use search as a first or second option; in all cases, making your search engine as performant as possible is necessary.

In order to understand search engine efficiency on a site, the following metrics are taken into consideration:

• Proportion of website users using search: Do your average users systematically beging their visits by using search, or do they start navigating your site and use search functionality only in case they cannot find what they’re looking for?

Total number of search queries or online user questions. This will give us an accurate information on how successful our website is. Often small changes in size and position of the search box will have great impact on the total number of search queries and the proportion of users performing a search.

• Search queries per visit: That measures the total amount of search queries a user makes. Ideally this should be no more than one visit, which would mean users search once and locate what they wanted. Too often, website records much more than one search query per search visit.
• Search results page abandonment rate: A user that leaves your website right after getting to the search results page without clicking on any result is a symptom that he/she has obtained an irrelevant or poor search engine result.
• Click-through rate: Total number of clicks on the search results page, divided by the total number of search queries. Actually this measurement can also be computed as (1 – abandonment). A click-through rate below 50% (often found on old-fashioned, keyword-based search engines) is an indicator that search engine should be revamped.

Clickthrough rate gives a crystal clear view on the efficiency on search. Natural Language or Semantic Search engines will have much greater Clickthrough rates than old-generation keyword based search engines. Also, creating contents based on demand as a part of the process of updating the website will lead to a continuous improvement of the clickthrough, as this image shows.

• Conversion Search to Sales: How many of the users that search on your website end up buying your product or service? how many of them become a valid lead? Measuring this information is a key element to understand the value of search, and carefully measuring how much investment should be put on improving the search experience.
• Percentage of search with no result: We have measured many customers having 60% of their search queries not being answered at all. This is catastrophic when it comes to conversion and click-through ratios: how can I possibly buy your product if you apparently don’t have it or it’s difficult to find?

Some keyword based search engines register answer levels below to 40%. This has a terrible impact on click-through, and therefore on conversion rate, as clickthrough will be necessarily lower than answer level, and conversion rate a lower value than clickthrough.

• Top 100 search topics: Just taking a look to your top 10 literal keyword search queries is not enough. The effect of long-tail exists in every search engine, including your website search. Often 80% of search queries have been searched only once, or just a few times. You must me prepared for a huge variety of words and expressions found in all human languages to express a few concepts. A keyword-based approach to search is no longer valid to obtain the maximum return from your website search box.

You must measure and understand these behaviors if you want to improve your website and search experience based on what people search. Demand-based, and search-effective websites will be, in the long term, the only ones that will lead to great user experience and profitable online businesses.

Author: Jordi Torras // CEO inbenta

How Inbenta uses Natural Language Processing to increase long-tail organic website traffic

In this short video you will learn how Inbenta is using Semantic Search and Natural Language Processing to increase website traffic from long-tail Internet search queries in search engines such Google or Bing.

 

Using Google Search Appliance (GSA) Metadata Feed to implement Natural Language Search

The Google Search Appliance (GSA) is an excellent standard for enterprise search and is very popular in the market, since it is an easy solution to deploy as a search engine for a website.

Nevertheless, the GSA bases its searches on keywords and not on the meaning of the users’ questions, and this means that the results obtained are often not relevant to the question posed.

Presenting irrelevant results to the users of our website has a cost. They get a bad perception of the service offered, which hurts the corporate image, not to mention the amount of potential sales or transactions that we could be losing by lowering our conversion rate. In short, you can’t afford the cost of having your users not find what they are looking for at the very first try.

But there is a way to fix this:

Inbenta has developed a solution that integrates seamlessly with the functionalities of the GSA, called External Metadata Indexing. The solution created by Inbenta allows this GSA functionality to be used to enter metadata with semantic information. The documents are indexed in seamlessly mode that can’t be perceived by users or content editors.

These metadata contain additional semantic information which makes the documents easier to find with the semantic search extensions of Inbenta.

For example, for a document titled “Where can I view my annual statement?”, Inbenta would create a feed like this, which would also allow user questions such as “find yearly report” to be understood:

This XML feed is created thanks to the powerful natural language processing engine of Inbenta. This process results in combining the advantages of the GSA (robustness, presence on the market and ability to index large amounts of documents) with the advantages of natural language search: more relevant results, the possibility of finding documents without needing to use the exact words those documents use, and the possibility of doing searches based on the overall meaning of the phrase and not on the keywords used, i.e., interacting with the search engine in a more natural way, as though we were talking to a human.

The clients that have incorporated this solution from Inbenta have improved their search engine clickthrough by 300%, meaning there are more users clicking on results that are much more pertinent, thanks to this innovative solution offered by Inbenta.

Thus, with the Inbenta Semantic Connector, you will be able to:

• increase your conversion rate
• reduce the cost of your customer service
• get very important information about your users

You can check out all the information on this product here.