Another dot in the blogosphere?

Posts Tagged ‘search engines

Video source

Finally. An episode on how search engines use AI to help (or not help) us find answers to questions.

The narrator likened search engines to library systems: They had to gather data, organise them, and find and present answers when needed.

The gathering of data is done by web crawlers — programmes that find and download web pages. The data is then organised by reverse indexes (like those at the back of textbooks).

The indexed web content is associated with numbers. Each time we search with an engine, these numbers are then linked to associated web content.

Example of indexing.

Since there is so much content, it needs to be ranked by accuracy, relevance, recency, etc. We help the AI to this with bounces (returning to the search) to click-throughs (staying with what we were presented).

The narrator also explained how we might be presented with immediate answers and not just links to possibly relevant web resources. AIs use knowledge bases instead of reverse indexes.

Knowledge bases might be built with NELL — Never Ending Language Learner. The video explains this better than I can.

NELL — Never Ending Language Learner.

Fair warning: Search engines still suck at questions that are rarely asked or are nuanced. AI is still limited by what data is available. This means that it is subject to the bias of people who provide data artefacts.

The next episode is about dealing with such bias. Now the series gets really interesting!


Usage policy

%d bloggers like this: