Project Computing

Research published in Australian University Repositories
Last updated: 23 Apr 2008

A Google Custom Search Engine searching the full text of research published in Australian university repositories:

What does this do? How does it work?

Most Australian universities manage repositories of their publications. Google may crawl those repositories and index the text. The Google Custom Search Engine lets anyone scope a search over just a specific part of the entire world wide web. This tool uses the Google Custom Search Engine to limit a search to just some Australian university repositories, more or less.

How does it know where to look?

I've given it a list of URL patterns (see below). This list is almost certainly incomplete, so if you know how I can improve it email me at kent.fitch@projectcomputing.com. The original list was seeded from links found here and then by running Google searches such as site:xxx.edu.au filetype:pdf and looking for interesting patterns.

How is this related to ARROW?

It isn't, really. ARROW is program aimed at helping Australian universities establish repositories which also regularly collects the metadata describing the contents of those repositories (not the full text), and "pushes" what it finds to Google, which reads and indexes those contents and other pages it discovers as a result of that processing.

The ARROW Discovery services lets you search the collected metadata. Its results are relevance ranked based on the occurrence of your search term in the collected metadata. But it doesn't "harvest" or index the full text. So unless your search term appears in the metadata of a resource, it won't find it.

This tool, however, searches the full-text of resources Google has crawled and their link metadata (the text contents of hyperlinks on the web which point to these resources). Relevance ranking is based on Google-magic, which includes occurrence of your search term in the full text contents and incoming links as well as the number of incoming links and their pagerank. For result sets comprised of resources with few incoming links, ranking is rather poor.

The two approaches will find slightly different sets of resources and present them in a different order, for example, compare:

Can the two approaches be combined?

Of course! If searching on a subset of research outputs from the university sector by national boundary is important, I guess they will be!

Why do people want to limit searches to Australian university outputs placed in specific Australian university repositories?

Apart from bean-counters, probably they don't. It is very important that research "outputs" are made public and can be discovered - this is an important goal of ARROW. But whether it is useful to restrict discovery to some fraction of materials produced within a national boundary, reduced further to those materials uploaded to an Australian university repository allied with the ARROW service, then discovered and indexed by Google, is another question...

URL patterns search by this tool

Note: Google does not (yet) index the contents of all these URLs. Some may be excluded from Google's view with a robots.txt configuration. Others may not be crawlable, or may not be linked from the outside web, which is one of the tasks ARROW is hoping to perform.


ACT

www.library.unsw.edu.au/~thesis/adt-ADFA/uploads/*	
erl.canberra.edu.au/*	_
thesis.anu.edu.au/uploads/	
dspace.anu.edu.au/bitstream/*
dspace.anu.edu.au/html/*	
dlibrary.acu.edu.au/digitaltheses/*	

NSW

epubs.scu.edu.au/cgi/*	
library.uws.edu.au/adt-NUWS/uploads/*	
arrow.uws.edu.au:8080/vital/access/manager/Repository/*	
*.une.edu.au/*article*.pdf	
*.une.edu.au/*publications*.pdf	
*.une.edu.au/*Preprints*.pdf	
*.une.edu.au/*Report*.pdf	
www.researchonline.mq.edu.au:9080/vital/access/manager/Repository/*	
www.library.uow.edu.au/adt-NWU/uploads/*	
ro.uow.edu.au/cgi*	
www.library.unsw.edu.au/~thesis/adt-NUN/uploads/*	
unsworks.unsw.edu.au/vital/access/manager/Repository/*	
unsworks.unsw.edu.au/vital/access/services/Download/*	
epress.lib.uts.edu.au/dspace/html/*	
epress.lib.uts.edu.au/dspace/bitstream/*	
ses.library.usyd.edu.au/bitstream/*	
www.newcastle.edu.au/services/library/adt/uploads/*	
ogma.newcastle.edu.au:8080/vital/access/manager/Repository*	
csu.edu.au/research/*.pdf	

QLD

eprints.usq.edu.au/*.pdf	
eprints.usq.edu.au/*	
adt.library.qut.edu.au/adt-qut/uploads/*	
eprints.qut.edu.au/archive/*	
adt.library.uq.edu.au/public/*	
eprint.uq.edu.au/archive/*	
eprints.jcu.edu.au/*.pdf	
eprints.jcu.edu.au/*	
www4.gu.edu.au:8080/adt-root/uploads/*	
www98.griffith.edu.au/dspace/html/*	
www98.griffith.edu.au/dspace/bitstream/*	
research.usc.edu.au/vital/access/manager/Repository/*	
library-resources.cqu.edu.au/thesis/*	
acquire.cqu.edu.au:8080/vital/access/manager/Repository/*	
epublications.bond.edu.au/context/*	
epublications.bond.edu.au/cgi/*	

VIC

eprints.infodiv.unimelb.edu.au/archive/*	
digthesis.ballarat.edu.au/adt/uploads*	
wallaby.vu.edu.au/adt-VVUT/uploads/*	
eprints.vu.edu.au/archive/*	
adt.lib.swin.edu.au/uploads/*	
researchbank.swinburne.edu.au/vital/access/manager/Repository/*	
researchbank.swinburne.edu.au/vital/access/services/*	
adt.lib.rmit.edu.au/adt/uploads/*	
arrowprod.lib.monash.edu.au/vital/access/services/Download/*	
eprint.monash.edu.au/*	
alpha3.latrobe.edu.au/thesis/uploads/*	
tux.lib.deakin.edu.au/adt-VDU/*	

WA

espace.lis.curtin.edu.au/archive/*	 
espace.lis.curtin.edu.au/archive/*.pdf	 
adt.curtin.edu.au/theses/available/*	 	
ro.ecu.edu.au/rqf_submissionsfedrt/*	 
portal.ecu.edu.au/adt-public*	 	
wwwlib.murdoch.edu.au/adt/pubfiles/* 	
*.uwa.edu.au/*article*.pdf	 
theses.library.uwa.edu.au/adt-*	 

SA

digital.library.adelaide.edu.au/dspace/bitstream/*	 	
digital.library.adelaide.edu.au/dspace/html/*	 	
dspace.flinders.edu.au/dspace/bitstream/*	 	
dspace.flinders.edu.au/dspace/html/*	 	
catalogue.flinders.edu.au/local/adt/uploads/*	 	


TAS

eprints.utas.edu.au/*	 	
eprints.utas.edu.au/*.pdf	 	

Thanks to the following people for updates:

Kent Fitch, Project Computing


Project Computing Pty Ltd ACN: 008 590 967 contact@projectComputing.com