WebTech and Security Lab

Current Projects
SearchScraper

What are the main trends in Web search? We have been collecting search result data for nearly two years now to give us insights into the major influences in search.

We are looking at some of the following questions:

  • What types of queries have the most changeable top 10 or top 50? (Commercial, navigational, informational, transactional) (popular versus non-popular)
  • How different are search results for the different interfaces (desktop, mobile)?
  • Is one search engine really copying the results on another search engine?
  • How long does the average URL stay in the top ten or top 50?
  • How many URLs remain in the top 10 or top 50 after three months/6 months/one year?
  • For any given query, will every URL in the top 10 or top 50 eventually be replaced? If so, how long will that take? If not, what class of query is it?
  • What is the 'half-life' of the average URL in the top 10 or top 50?

What we hope to do is discover what are the main influences for different types of query in different contexts, and to hence be able to predict the trajectory of some types of websites. We are not looking at how to influence the trajectory of websites in search results - that belongs to search engine optimisation companies.

We have published our first paper on a week-by-week analysis of the change and half-life of search results sets (Truran, Schmakeit and Ashman, see the publications page).