Some background on search relevancy
We've been getting feedback from staff asking about search relevancy in our new website. Here's some background on how it works.
The search engine we use with our new website has a relevancy tool which we can use to affect the relevancy of keyword search results. Relevancy tuning is an ongoing process and can be tweaked based on performance and user feedback.
Using this tool allows us to compensate for the difference between the nature of our data vs. the way users search, e.g. Brown, Dan vs Dan Brown, books about J.K Rowling vs books by J.K. Rowling, or title and author vs subject.
For example, searching for To Kill a Mockingbird in Endeca retrieves the book itself as the top result, while works about the novel are ranked lower. The same keyword search in iBistro does not produce equally useful results.
Try this link to see the search results in our new website:
http://beta.torontopubliclibrary.ca/endecaApp/pages/search.jsp?Ntt=To+Kill+a+Mockingbird+&N=0
We can achieve these types of results by manipulating a number of variables. We developed a strategy that works the best with a number of search types, based on our users’ needs.
Generally speaking, records that match the exact phrase a user has typed in will appear first. Where multiple records match that exact phrase, they will be sub-sorted based on the strategy that gives more weight to Title, Author and Subject, in that order.
We have also added a variable based on the number of copies the Library has of a title to help display popular titles and and formats with the most copies at or near the top of search results.
We found that the number of copies helps push up popular titles to the top of the results list.
Currently, on iBistro, if you search for The Lost Symbol, the print version with 800 copies appears 5th on the list below the various electonic versions and the large print edition. This may help with users who accidently put holds on the Large Print version of a title when they don’t meant to.
Try this link to see the results on the new site:
http://beta.torontopubliclibrary.ca/endecaApp/pages/search.jsp?N=&Nf=&eT=&Ntt=the+lost+symbol
Better relevance is a continuous process and our relevance strategy should be monitored on an ongoing basis in order to achieve optimal results. Better relevance will improve searching succes which in turn should have a positive impact on user satisfaction and circulation statistics.
13 thoughts on “Some background on search relevancy”
I tried the website at the NYCL-ST desk today and had good results with questions. There was 1 problem in trying to locate a work by Danish film director Lars Trier. Since I did not have the name of the work I used the search term Lars Trier and pulled up all his works and then limited by type (movies). The desired DVD Dogville was not in the limited listing. When I repeated the search and looked at the result without applying the limiter, Dogville was there. There are 2 problems: 1) the linking of the catalogued record to the format limiter. There might be a 2nd dvd with this problem 2) having to scroll quite a bit to to find the work.
Hi Joan,
We checked and there is a problem with the coding used for Format in the MARC record for Dogville. It should be fixed at the source tomorrow and display correctly on Wednesday. Once fixed, Dogville will display when you refine by Movies & Videos. That should mean less scrolling once you refine your search.
You can also change the display order of your search results list which may help with scrolling. For example, you can display your search results by Title, Author or Date instead of by Best Match. Depending on your search results, this may or may not mean less scrolling.
Catherine
Coding was fixed this morning, it should display correctly tomorrow (Thursday) morning
Hi, a question about the Related Internet Links in Search results, which I presume are from the VRL. When advancing to the next page of search results, the content of this sidebar does not change. The search I used was ‘genealogy canada’. However when selecting the link External Websites from the TYPE list, the VRL sites are displayed in the main centre display and do change with Next and Previous as expected. Is the intended behaviour, i.e. the Related Internet Links sidebar remains the same throughout?
Hi Andrew,
Yes, this is the intended behaviour at the moment, although we are looking at ways of improving the Related Internet Links.
Thanks for your comment.
I have concerns about the discovery of online resources through search results lists on the new website:
– i think links to the material in databases should appear higher up in the presentation of filter options e.g. If a search is done from the main search box and Online/Download filter is selected – Options are: eBooks eAudiobooks, eVideo eJournals.
If a search is done for “Stephen Harper” the only eJournals that appear are journals that are free on the internet – the valuable journals in our databases are not presented.
I think a link to “Databases” should be offered among the filter options here as well.
– when users select “Databases” as a filter option, i think we should provide a link to “Search several databases at once” (or similar language) on the right side, near the top of the list.
Users could then chose to go to the “Article Search” interface or stay with the alphabetical listing to use databases individually.
– i think the the phrase beside the search box that says “Search inside articles” should say “Search inside Databases”
thanks
I was about denounce the horiziontally scrolling images on the home page as far too distracting, when i noted that there was a start/pause feature in lower left. Could this button be coloured in a different way to make it more obvious. How frequently will the images be changed?
Thanks
Hi Alan,
In response to feedback from staff, a decision has been made to go with a static banner on the homepage. The new version of the banner will be loaded on the beta site before the public beta release.
Thanks for your comment.
Hi Joanne,
We’re still looking at ways to improve the presentation of databases on the new website.
The problem is that the new search engine is not searching the content of the databases, so a search for “Stephen Harper” does not produce any database matches. The refinement options that appear in the left column are based on the search results set – “databases” will not appear unless there is one or more database records in the results. This is fundamental to the way faceted searching works.
We face real challenges in getting users to understand what our subscription databases are and how they can benefit from using them. Our most recent round of usability testing reminded us once again of some basic truths:
– Users experience the website as if it’s a billboard they are driving by at 100 km/hour – nuances are lost on them
– Increasing the number of options on a web page makes it less likely that users will notice any of them
– Users will not click on a link if they don’t understand what it’s for
I’d be happy to meet with you to talk some more about ideas for access points to the databases. Like you, I would love to see more customers discover and use them.
Thanks for being such a staunch advocate on this issue.
Hello Web Team,
First of all, I think the simple search feature is working well as a library catalogue. The various ways of limiting searches by location, language and whether or not they circulate seem handy.
However, since the home page no longer contains links to library services, I think they should be easy to find through the basic search. This doesn’t seem to be the case. For example, in Special Collections, we are often asked about digitization services, but the keywords “digitization” and “reproduction service” only lead people to the Preservation and Digitization Dept.’s pages via the not-very-noticeable “Library Info” link. Once there, the link appears in a list of things that are not very relevant.
This is even more true if you search for gallery information. Most people will just put in a simple keyword like “Gallery”, but even if you find the Library Info button, the first page is all hits related to Kids Space. A search on “Baldwin Room” on the other hand, is at the top of the page for Library Info. It would be ideal if other library departments and collections could be made to appear at the top as well.
The advanced search is a marvel of post-literate simplicity–but some people might like that. I see that it is in construction, so perhaps you are adding more later. Right now it really only offers limits that are already mostly available in the side bars of the basic search. I hope that people who actually want to do an advanced search are able to find the link to the current catalogue. A subject field search is essential, at the very least.
It’s difficult to please everyone in this sort of project, but I do think it is essential to make the links available in the “library info” sidebar more prominent. Otherwise it is very difficult to find specific library departments and services. Are you creating a “Find Your Way” category to library departments? Good luck fitting it all in!
Hi Peggy,
You make some good points! We have had a few comments from staff wondering where to find information about services. The plan is to include them in the “Using the Library” section, although not all the pages have been built yet. Based on staff feedback, we are considering changing the label of that section to include the word “Services”.
In addition, it will become much easier to find information about services through search. A lot of the content describing services has not been added to the site yet. As we create the pages, we will add metadata to them, and then we will have to test and tweak our search relevance strategies based on the metadata to ensure that common searches (like the “gallery” and “baldwin room” examples you provide) produce good results.
Subject browse will not be possible in the new website – staff and customers who miss it will have to continue using the old catalogue. Authority records for subjects and authors have not been included in the new website’s data set (for several reasons, including the cost of licensing and the challenge of figuring out how to do it when no other libraries had tried it yet). That said, we are adding field-specific searching to Advanced Search, so you will be able to do a “keywords in subject” search.
We have considered creating an “A to Z” list of all library services and collections. However, we have found in past usability testing that customers do not think in terms of “departments,” “collections,” and “services.” That’s why we’ve taken the more topic-oriented approach you see in the Find Your Way section. As mentioned above, the capability to locate specific collections, services, and departments through search will improve in the near future.
Thank you very much for taking the time to send us such detailed feedback – we appreciate it.
Could you add in the drop down menu in Advance search browse searches linked to the old catalogue?
McMaster University added browse searches linked to the old catalogue right under the keyword searches on the main page of the catalogue
The website looks very clean and streamlined to me, even though a lot is packed into it. Since people can customize the font size, I don’t think the current size is a problem—and certainly not for many under 50.
I didn’t like the moving graphic so much at first, but am getting used to it, and it does punch up the site. I see you’ve made a decision to make it static, but perhaps a compromise might be what’s been done on the http://www.toronto.ca site, where the picture is static for 7-8 seconds, then slides over to show the next image. Also, on that site, each image relates to a specific high profile program, and clicking beside the image takes you to info for that. It is also possible to repeat and/or reverse the order of the images. Maybe something to consider for TPL too?
The more I use the Search, the more I like it. It does seem to be much finer tuned than iBistro, and including events and websites is excellent. And once I’d mastered the categories down the side, I found the refinements they offer very useful. There are some specific searches that have brought up some questions/problems though. I will send those examples on the email, rather than clutter up the blog. And I find the fonts and spacing on the catalogued records (e.g. the shelf locations) much clearer and easier to read than in iBistro.
Once you do a Search, the next screens offer the “Search inside articles” option. It would be good if this was on the first page too.
One suggestion for recommended websites—maybe they could open in new windows, so the patron doesn’t lose their place in the TPL site. Maybe that’s even a possibility for other library info (like events), so the search isn’t lost. Or maybe this just isn’t working yet.
There were some blog comments about directing users to special departments and collections. I would really like to see that under one of the Find Your Way bullets—to highlight the work that the R & R levels do. Maybe:
(Find Your Way–)
To Research, Special [or Specialized, or Specialty] and Rare Materials [or Books, or Collections]
The language is hard, but we should try to put that up front for the serious researchers and scholars, and as a quick way into–as well as a promotion of– things like the Merrill and Osborne collections, the Baldwin Room and Urban Affairs (among others.)
This is huge project, but it really is looking good, and I think it will be a great boon to the public.