Having met hundreds of users at client engagements, through user groups like the London APE and from my own personal experience as a Confluence Administrator, I know the search facility in Confluence can be a real source of frustration for end-users. Receiving an email titled 'Confluence Search Sucks' from a Senior Manager whilst I was in my last job lead me to researching the cause of such frustrations and I thought I'd share my findings with you, as well as what we did to overcome them.
In addition to observing users, my research saw me looking at the most commonly searched for things on our Intranet and looking in to where the the results the most visited pages came in typical searches. The data for this was provided by Adaptavist's Stats Analysis plugin and through looking at this I was able to identify several problems with searching for content in our Confluence instance. Below are my findings:
Firstly, we found that Confluence's search does not suck. In fact, it's very good. It's based on Apache's Lucene, which is an Enterprise standard, high-performance, full-featured text search engine. There were some functional limitations which I will come on to, but the majority of the problems were in the way content was being created, titled, labelled (or not!) and organised.
Titles are the most important part of a page, they are the key determining factor for where a page will be shown in search results and it is the highest weighted field in a page. Their importance is only heightened by people's heavy use of the Quick Nav which puts even greater emphasis on the title when determining results. However, page names are often given little thought by end-users. As someone who has worked with Websites for years it's second-nature to think about who the page is being written for and what people will most likely search for when trying to find that particular page. However, it was clear from my research that this is not something that other users considered, or perhaps knew to do at all. A good example was a page containing driving directions for one of our offices named 'Getting here'. Getting where and how I thought. It's clear from the name that the eventual reader was not given much thought and consequently people were struggling to find the page.
To overcome this it's important that content creators are aware of how important titles are. Blog posts and email newsletters with tricks and tips aimed at highlighting this and a user training course detailing how to 'Write for the web' were all things we began to do. This helped get the message out that they could help their customers to find their content.
The majority of the pages I looked at were not labelled and if they were, they weren't labelled well. Labels are almost as important as titles when it comes to influencing a page's findability and are a good way to ensure that searches on synonyms and related words still find the page. However, a lot of our users didn't understand what they were for and why they should add them. When they did, they only added one or two and often they just repeated words already in the page title.
User education was again at the heart of improving this. We encouraged users to think for keywords that described the page they had created and think about what they might search for if they were trying to find the page, suggesting that they add 6-8 labels on a typical page. We also helped with this by installing Adaptavist's Synonym plugin, which helps to ensure that a set of labels are associated with content in the search index. This is especially useful when searching using technical language and acronyms. Andrew Frayling also blogged recently about seeding your Wiki with labels to help take some of the work out of labelling for end-users, which could also help.
We had spent a long time deciding upon our Information Architecture whilst implementing Confluence as an Intranet - we decided to move from a structure based around teams and functions, to the more cross-functional subjects and topics. We undertook card-sorting exercises and ran focus groups to try and understand how our users thought. However, something we overlooked was tailoring this to how Confluence itself works. We ended up creating large spaces containing many hundreds of pages about whole ranges of topics. This did not work well for Confluence, especially given that Spaces are included in Quick Nav results. We found that creating smaller, more focussed spaces meant that people could navigate using the Space results more easily. We also had Adaptavist customise the Quick Nav results to include the space name in addition to the page title, something which I'm glad to see is now included in Confluence by default. This really helped with the Quick Nav and deciding upon which of the many pages with the same title was actually the one I was looking for.
The age of content also plays a small part in where pages appear within search results. We found that a lot of popular pages were landing pages for topic areas, which we had set to be mostly self-updating with RSS feeds and Advanced Search lists. However, this meant that they often went months without the page being edited. We changed this to ensure that they were edited regularly.
We started to use the Confluence Awesome Search plugin, which helps finesse search results by factoring in how recently you visited a page, whether it is a favourite, whether you have contributed to it and whether it was a good match for a previous searcher with the same query. It adds a personal factor to search results.
However, we also found that sometimes we just wanted to manually make sure that popular pages were always at the top of specific search queries, so Adaptavist developed a 'Featured Search Result' plugin for us, which put results for specific searches at the top, a bit like Google's featured search results.
We found that usernames like 'Support' and 'IT Helpdesk' which existed in our Active Directory and had thus had a Confluence account created automatically also caused a problem. Since people are considered the most important type of 'content' in search results, these accounts often occupied high ranking positions in search results - so renaming them was a must.
Having implemented these changes, reports about problems with search began to reduce. We even got the complaining Senior Manager on board. Of course people didn't really notice the improvements made, but that's how search should be - it should 'just work'. And with a combination of user-education, plugins and re-organising content, we made it do just that.