RSS Feeds for MetaLib

Tags: advanced, metalib, nelli-tuning
Last Updated: Oct 30, 2009 01:03

• Author: Ere Maijala
• Additional author(s):
• Institution:
• Year: 2008

  ---

• License: MPL 1.1 / GPL 2.0
• Short description: Use, modification and distribution of the code are permitted provided the copyright notice, list of conditions and disclaimer appear in all related material.
• Link to terms: MPL 1.1, GPL 2.0

  ---

• Skill required for using this code: advanced (due to requirement of a separate MySQL and additional Perl modules)

  ---

Description

Implementation of alerts as enhanced RSS Feeds. In addition to the number of results
the user will get changed records among the first 100 retrieved records. Feeds are very easy to create from the search results screen by clicking the well-established RSS icon.

Screenshot:

State

Stable

Author(s) homepage

http://wiki.helsinki.fi/display/Nelli/Etusivu

Download

• MetaLib:Version 1.4

Working example

Not publicly available due to it requiring login.

Using the following Ex Libris API

X-Server

Changes

Version 1.4

• Added possibility to open SFX links in a new window
• Added possibility to define an icon for SFX links
• Fixed updating of an outdated feed found with an alias
• Fixed retrieving results if a database in the set is not searchable

Version 1.2

• Added SFX links to citations
• Added search terms to the feed (channel) title
• Added gray RSS button for guests redirecting to login
• Added link tag advertising the feed to html headers

Version 1.1

• Initial public release.

Release notes

A separate MySQL and X-Server license for MetaLib are required. Additionally, the
following Perl modules are needed:
DBI (with DBD mysql), CGI, URI::Escape, LWP::UserAgent, HTTP::Request::Common, Digest::MD5,
XML::DOM, Time::Local, POSIX, Encode, Cwd, File::Basename, Fcntl and HTML::Entities.
Some of the modules are probably already present, but some may need to be installed from
CPAN. By default the system Perl is used, but the path can be changed from the scripts.
Development and testing has been done in SUN Solaris 10, but there should be nothing
preventing use on other platforms.

Installation instructions

MySQL

Install MySQL somewhere. These scripts don't care if it resides on the same
server as MetaLib or not, as long as it can be connected to. For instance
/exlibris/nelli/mysql is fine. Development and testing has been done with MySQL 5, though no exotic features (just some InnoDB tables) are used so other versions will probably work. Details on how MySQL is installed are out of scope of these instructions.
Using mysql with root privileges, create the user and database structure with the commands described in database.txt in the package.
Just change the '...' to a quality password.

Perl

Verify that all the required Perl modules are installed. By default nellirss.pl will search
for modules in /exlibris/nelli/lib/perl5/sun4-solaris-64int in addition to system defaults.
This can be changed in the beginning of nellirss.pl.

Scripts

For documentation purposes we assume the installation directory is /exlibris/nelli/rss.
Another directory may be used, but the settings in nellirss.pl must be changed accordingly.

1. Untar the scripts (nellirss.pl, metalib_x_server.pm) to /exlibris/nelli/rss
2. Make sure that nellirss.pl is executable (chmod +x nellirss.pl)
3. Copy rss-icon.gif to MetaLib instances' icon_xxx directory (e.g. /exlibris/metalib/m4_2/ins00/icon_eng).
4. Create an empty directory 'data' under /exlibris/nelli/rss. It will be used
   for storing the feed files
5. Change settings in nellirss_config.pl.
   

   Setting	Description
   useragent	Just the user agent used when requesting a URL (mostly from MetaLib)
   timeout	HTTP request timeout
   own_url	The URL pointing to the script itself
   metalib_url	MetaLib's base url (without any trailing /V)
   x-server	X-Server address
   x-user	X-Server user name
   x-password	X-Server password
   x-debug	Whether to log debugging information regarding X-Server queries (writes to STDERR)
   debug	Whether to log debugging information regarding different operations (writes to the log file)
   logfile	File name used for generic logging
   log_requests	Whether to log any HTTP request
   db_dsn	DSN used for connecting to MySQL. See e.g. documentation at CPAN for more information
   db_user	MySQL user name
   db_password	MySQL password
   max_age	Maximum feed age before it's updated automatically when the feed is requested (minutes)
   max_age_batch	Maximum feed age before it's updated during the batch update initiated via crontab (minutes)
   item_retention_period	How long older articles are kept in the feed (days)
   passivation_time	How long after the last time the feed was accessed it is passivated (= not updated during the batch update)
   sfx_base_address	List of SFX addresses, one for each MetaLib institute. Used to create the SFX links. A list to be added here can be created automatically by running the provided get_sfx_addresses.pl as the metalib user. SFX links are not created if the address is not found in this table

   
   There are also texts in multiple languages in nellirss_texts.pl. The language codes are the same as in MetaLib. Languages can be added and texts can be customized as desired.

Apache

1. Add ScriptAlias for /rss pointing to nellirss.pl into the proper VirtualHost
   (or global if no vhosts are used) in httpd.conf of MetaLib's Apache
   (e.g. /exlibris/metalib/m4_2/apache/conf/httpd.conf):

ScriptAlias "/rss" "/exlibris/nelli/rss/nellirss.pl"

MetaLib Templates

1. Copy the following files to www_v_xxx (e.g. /exlibris/metalib/m4_x/ins00/www_v_eng):
   • find-db-rss
   • find-db-rss-guest
   • meta-rss
   • meta-rss-guest
   • quick-rss
   • quick-rss-guest
   • rss-head
   • rss-head-guest
   • helper-data
   • helper-data-guest

     helper-data and helper-data-guest are shared with other extensions. Overwrite only if never

2. Add <include>find-db-rss to MetaLib's head section of find-db-6-body after the number of results like this:

   <START_SECTION><!-- head -->
   <div class="record_list_header">
   <div class="containerTop">
     //Search for\\ "<strong title="$0100">$0100</strong>" //in\\ <strong>$0200</strong>
   <include>find-db-rss
   </div>
   

3. Add <include>meta-rss to sections caption and head-combined-i of meta-3-body after the number of results like this:

   <START_SECTION><!-- caption -->
       <div class="containerTop">
         <div class="alignLeft">
           //Results for\\ <strong title="$0200">"$0200"</strong> //in\\ <strong>$0300</strong> ($0400 //hits\\)
   <include>meta-rss
         </div>
   

   
   and:
   

   <START_SECTION><!-- head-combined-i -->
   <div class="containerTop">
   <div class="alignLeft">
           //Combined results for\\ "<strong title="$0200">$0200</strong>" ($0400 /
   /hits\\)
   <include>meta-rss
   </div>
   

   
   

4. Add <include>quick-rss to head-summary-open section of quick-3-body after the number of results like this:

   <START_SECTION><!-- head-summary-open -->
   <div>
     <div class="record_list_header">
           <div class="containerTop">
             <div class="alignLeft">//Search for\\ "<strong title="$0100">$0100</strong>" //in\\ "<strong>$0200</strong>" //found\\ <strong>$0300</strong> //results\\
   <include>quick-rss
             </div>
   

5. Add <include>rss-head to find-db-6-head, meta-3-head and quick-3-head like this:

    <include>wait-js\Searching\Retrieving
    <include>rss-head
    </head>
   

6. Add link to rss into history-1-body right after the alert link (or wherever desired):

               <td>
                 <a href="&server_vir?func=history-1-delete&amp;sequence=$0500" title="//Delete\\ //Query number\\ $0100"><img src="&icon_path_&lng/v-icon_delete.png" alt="//Delete\\ //Query number\\ $0100"></a>
                 <a href="&server_vir?func=history-1-alerts&amp;sequence=$0500" onclick='javascript:winPop(event,"&server_vir?func=history-1-alerts&amp;sequence=$0500","no", 570, 400);return false;' title="//Alert\\ //Query number\\ $0100">
                 <img src="&icon_path_&lng/v-icon_alert.png" alt="//Alert for Query number\\ $0100"></a>
                 <span class="history-rss"><a href="&server_httpd/rss?feed=&metalib_session/$0500" title="//Create RSS Feed\\"><img src="&icon_path_&lng/rss-icon.gif" alt="//RSS\\" title="//Create RSS Feed\\"></a></span>
               </td>
   

7. Copy rss-icon.gif and rss-icon-locked.gif to icon_xxx (e.g. /exlibris/metalib/m4_x/ins00/icon_eng)

cron

A couple of scheduled tasks are needed. One takes care of initializing new feeds by
default every five minutes and the other updates feeds one a day or so. Initialization will
retrieve initial search results for the feed.

1. Add the following lines to your crontab and change them according to your needs:

# Initialize any new RSS feeds every 5 minutes
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /exlibris/nelli/rss/nellirss.pl init

# Update RSS feeds every night
0 20 * * * /exlibris/nelli/rss/nellirss.pl update

CSS

The following classes can be used to modify the display of the RSS icons:

An example to hide the gray button from guest users:

.find_db_rss_guest, .quick_rss_guest, .meta_rss_guest
{
  display: none;
}

Upgrading to Version 1.4 from Version 1.2

• Create a backup of nellirss.pl.
• Replace nellirss.pl with the new version and make sure it's executable (chmod +x nellirss.pl)
• Optional: Add 'open_sfx_links_in_new_window' setting into nellirss_config.pl if desired
• Optional: Add 'Record OpenURL Link Image' into nellirss_texts.pl if desired. You may want to take out 'Record OpenURL Link Title' if the image is used

Upgrading to Version 1.2 from Version 1.1

• Create a backup of nellirss.pl.
• Copy files as described in the installation (there are changes at least in *-rss-guest, rss-icon-locked.gif, rss-head*, nellirss*.pl)
• Copy settings from the old nellirss.pl to nellirss_config.pl (not a straight cut and paste, but the settings are the same) and add SFX addresses if desired
• Add <include>rss-head to head files as described in the installation

Command Line Usage

nellirss.pl has a couple of commands that can be used from the command line.

TO DO list

• Implement a mechanism for purging old (dead, what's dead needs to be defined) feeds
• Create statistics on the searches made via X-Server so they could be subtracted from overall stats

Known issues

None.

Comments

Any feed that has not been accessed in a predefined time is deactivated, which means it
will not be automatically updated unless someone requests it. Active feeds are updated via
a cron job so that accessing them is as fast as possible.