Researching Research: what academics want from the Web James A. J. - - PowerPoint PPT Presentation

Researching Research: what academics want from the Web

James A. J. Wilson

James.wilson@oucs.ox.ac.uk

From the RDN to 'Intute'

• The Resource Discovery Network, 1999-2006.

R.I.P

• Intute, 2006-

– From 8 hubs to 4 'Subject Groups' – Humbul to merge with Artifact

• A big 'thank you' to 'The Big Idea' for coming up

with the name 'Intute'

– We did a quick user survey on that, too

The Research Support Theme

• Each Subject Group expected to manage one or

two cross-group 'themes'

• July 2005: Mike Fraser writes draft proposal for

the RDN 'Research Support' theme

– Recommend priorities and emphases which should be given to collection and service development – Help implement operational and communication strategies to encourage participation of researchers – Make recommendations to the RDN training theme – Raise awareness of RDN within research councils, JCSR, and other policy makers for research support

Research Survey

• On March 12th, 2006, the RDN Research Survey is

launched on the WWW

– http://www.humbul.ac.uk/rdn/survey/

• Instant response from excited researchers

– 350 responses in little over a month

• Possible conclusions:

– Researchers have too much time on their hands – Researchers are underpaid and motivated principally by Amazon vouchers

Recognition

• Recognition of the RDN or any of its eight Hubs

was not especially high

Ye s, I use a t le a st one of the se se rvice s re gula rly Ye s, I use a t le a st one of the se se rvice s occa siona lly I've trie d a t le a st one of the se se rvice s be fore , but don't use it now I've he a rd of a t le a st one of the se se rvice s, but ha ve ne ve r trie d it

• No. I ca n't re ca ll ha ving he a rd of

a ny of the se se rvice s

Recognition

• Why is recognition not as high as we'd like it to

be?

– Lack of targeted publicity – Lack of word-of-mouth recommendation

• Encouragingly, only 22% of respondents who have

used the RDN services have since abandoned us

• But there's clearly room for improvement

Resource Discovery

• How researchers find online resources for their

research

0% 20% 40% 60% 80% 100% G

• g

l e Y a h

• A

n

• t

h e r g e n e r a l s e a r c h e n g i n e T h e R D N

• r
• n

e

• f

i t s h u b s P

• r

t a l

• r

w e b l i n k s p a g e Y

• u

r u n i v e r s i t y ' s w e b p

• r

t a l P u b l i c i t y m a t e r i a l s W

• r

d

• f

m

• u

t h O t h e r

What do researchers look for in a website?

• Here are some common responses:

– “Is the author genuine - or a nutcase?” – “How comprehensive is the coverage?” – “Authoritativeness, currency, quality of searching” – “Who runs it, who contributes to content, theoretical

• rientation”

– “Relevance, reliability, trustworthiness” – “Accessibility of information, design of site” – “Is the material peer-reviewed”

• What do these have in common?

Strength of Intute

• Google cannot guarantee any of them!

– Well OK, the cream is likely to rise to the top in some subjects, but not so much in those that are of general interest or taught in schools

• So why is everyone using Google?

– Because it's really good

• But we can do things that Google can't

– We're an 'authoritative mentor'

Review preferences

• Not every Hub reviews online resources in the

same manner

– Some are more evaluative than other – Some prefer shorter or longer descriptions

• Researchers vary in their preferences:

Esse ntia lly de scriptive S om e e va lua tive re m a rks S trongly e va lua tive

20 40 60 80 100 120 140 Not us e ful Not particularly us e ful M ode rate ly us e ful Us e ful V e ry us e ful!

De ta ile d Re vie w Brie f Re vie w

Alpha be tica l By num be r of tim e s a ke yw ord occurs By popula rity (num be r of 'click-throughs') By use r ra ting of re source By spe cia list ra ting of re source Othe r / Don't know

Search result preferences

• Researchers were also divided as to their

preferred way of ordering search results:

• The devious ones recommended a choice of

methods

– seems sensible given these results

Value-added services

• Respondents were asked to rate 24 potential

'value-added' services on a scale of 1 (not important) to 5 (very important)

– These services were suggested by the researchers who were interviewed in the preliminary stages of the study

• Almost all the suggested services were received

enthusiastically, with most respondents rating them towards the 'important' end of the spectrum

The top 10 most wanted

Rank Service Points 1 A directory of bibliographic databases in each subject 1271 2 1213 3 A database of sources of funding and scholarships 1205 4 A Directory of regular conferences, with indicators of size and prestige 1175 5 1166 5 Ability to filter out resources not directly applicable to research 1166 7 1157 8 National directory of researchers and research 1155 9 1121 10 1104 A directory of journals, with indicators of subscription numbers and prestige Searching and browsing filtered by the time period which the resource concerns Online events calendars - for conferences and other events in each subject area Data that can be exported for use on other Web sites, or for other uses elsewhere List of resources for graduate skills training - publishing, networking, thesis writing, etc.

User contribution

• Some of the suggested value-added services

would depend on user contributions

– Online events calendar – Directory of researchers – User rating or review feature

• Turn Intute from a provider into a community?

– Amazon is probably the most significant model – Means to establish market dominance – So much for 'authoritative mentors'

Yes Maybe N o

Willing and able

• Surprisingly, a majority of respondents said they

would be willing to contribute

Would you be willing to contribute your details to a national directory of researchers? Would you contribute website ratings if we implemented a user-review feature?

Yes Yes, if large Maybe N o Yes If notable N o

Would you be willing to contribute details of events you were

• rganising to an online events calendar?

Able and willing?

• Of course, they may have been lying

– 27% of respondents said that they didn't put their research on the Web because they didn't have time, 26% 'Just never got around to it' – And people filling in online surveys are likely to be the keenies in any case

Next steps

• Split out Hub by Hub findings
• Persuade Intute to actually respond to some of

the findings

– Some (such as the directory of bibliographic databases) are easy hits

• Because we've already got this data for some Hubs

– Others (such as evaluating journals) are going to prove extremely tough – Yet others might require a leap of faith and an initial kick start

Lessons learned

• On the whole, the online survey has been a

success

– Initial interviews were a good idea

• Needed better coordination of publicity

– Some Hubs drew many responses, some very few

• Would have benefited from tracking referrals

– Measuring proportion of respondents arriving via Hubs would have given more accurate picture of recognition