Introduction

Solr is a Java-based search server which Sigmah relies on to carry out search over fields like projects, contacts and org-units. An instance of the Solr server needs to be installed on the server-side, and a Solr “core” (an index) needs to be setup for a particular organization to facilitate searching within an instance of Sigmah.

This installation guide describes the technical procedure to install Solr and set up a Solr Core, covering both Windows and Linux platforms.

For better performance and security, it is recommended to maintain an installation environment for Sigmah with latest versions of the following 4 key components:

• Java 1.8 and above
• PostgreSQL 9 and above
• Tomcat 7 and above

Refer to the instructions for installation of Sigmah for the above 3.

• You will also require the latest version of Sigmah (v2.2) with the search feature included.

Currently, you can find the code for this here: https://github.com/sigmah-dev/sigmah/pull/97. Although yet to be merged, it contains a working version of the search feature.

The latest Solr version is available at http://lucene.apache.org/solr/mirrors-solr-latest-redir.html

(The version of Solr used for testing Sigmah's search feature is 6.5.0 )

For Linux/Unix/OSX systems, download the .tgz file. For Windows systems, download the .zip file. When getting started, all you need to do is extract the Solr distribution archive to a directory of your choice. When you're ready to setup Solr for a production environment, please refer to the instructions provided at https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production. To keep things simple for now, extract the Solr distribution archive to your local home directory.

For example, on Linux, do:

$ cd ~/
$ tar -zxf solr-x.y.z.tgz

You can start Solr by running the following command from the Solr directory:

On Linux:

$ bin/solr start

On Windows:

bin\solr.cmd start

This will start Solr in the background, listening on default port 8983. To change the port Solr listens on, you can use the -p parameter when starting, such as:

$ bin/solr start -p 8984

Solr can be stopped with (add specific port number if it is not running on the default port):

$ bin/solr stop -p 8983

To confirm that Solr is running, open the admin console in a browser: http://localhost:8983/solr/

Download the postgresql jar from here: https://jdbc.postgresql.org/download.html and paste it in {solr installation directory}/solr-6.x.0/contrib/dataimporthandler/lib.

Each organization has a separate core or index which has to be linked to the Sigmah instance. Download/clone the latest version of Sigmah containing the search feature.

Go to “sigmah/src/main/resources/solr_config” and copy the folder “Test_Sigmah”. This folder is a core with the necessary configuration files for Sigmah.

Paste it inside {solr installation directory}/solr-6.x.0/server/solr/.

Alternately in Linux:

$ cp sigmah/src/main/resources/solr_config/Test_Sigmah ~/home/{your-username}/solr-6.x.0/server/solr/

In this copy, open the file Test_Sigmah/conf/data-config.xml in a text editor. You will see the lines:

<dataSource driver="org.postgresql.Driver"
  url="jdbc:postgresql://localhost:5432/sigmah_demo?zeroDateTimeBehavior=convertToNull"
  user="sigmah"
  password="hamsig" />

In place of “sigmah_demo”, write the name of the sigmah database linked to the organization. Replace the values in user and password with the username and password of the database.

Restart the Solr Server in case it was still running.

Go to the admin panel, click on “Core Selector” dropdown on the left-hand side, and you should see “Test_Sigmah”. Click on it.

Alternately, open http://localhost:8983/solr/#/Test_Sigmah in your browser.

If either of these failed, it means that Solr has not been installed correctly in the first step.

In order to change the name of the core from Test_Sigmah to something more appropriate:

Open {solr installation directory}/solr-6.x.0/server/solr/Test_Sigmah/core.properties. You will see:

name=Test_Sigmah

Replace Test_Sigmah with the name of your choice.

Change the name of the folder Test_Sigmah to the same name. Then, restart the Solr server.

The name of the core is important. It forms a part of the url using which an admin connects to the Solr server.

Before going any further, carry out a full data import. This basically takes all the required tables in your database and puts them into the Solr index.

Go to the 'Data Import' tab in the options below 'Test_Sigmah' and press the 'Execute' button for full-import command. Press 'Refresh Status' until the UI says 'Indexing Completed. Added/Updated X documents…'

If it doesn't say this after a minute or so and seems to be stuck in a loop, check the 'Logging' tab in the Solr admin, if it says there are errors, then there could be some mistake in the config files.

Also check the connection with the database and the database version.

First of all, check that the schema of the database is upto-date, and migrate any changes. The changes to the schema made for including the search feature can be found here (as of now):

https://github.com/indianauthority97/sigmah/blob/bec9e807d9a725b308c9da154574b9e68489aa5d/src/main/resources/db/migration/V21__solr_core_url_field.sql

Compile and run the war file on the Tomcat Server.

When you open the new instance of Sigmah, you won't see any search bar. It will probably complain with a message like “Solr Server connection is not available.”

In any case, the first thing to do is to connect Sigmah with the solr core you just created.

Go to Administration → Settings → Solr Search Settings. It has a text field called “Solr Core Url”.

Enter the full URL for the solr core, http://{server-ip:port}/solr/{solr-core-name}.

For example if the solr core is on your local system, enter:

http://localhost:8983/solr/Test_Sigmah

Save the changes.

If it gives a message saying “Successfully updated Solr Core!”, you're good to go.

Otherwise, if it says “Failed to update Solr Core!”, re-check the url you entered, and that the solr server is up and running.

The next step is to permit the user to carry out search.

Go to Administration → Users → Profiles.

Edit the profiles for which you want to permit searching. In the popup for editing, go to Global Authorisations → Others. A new permission “Search” should be there. Tick it and save the changes.

Reload Sigmah on your browser.

A few seconds after loading, the search bar should appear. If it doesn't, try logging out and then logging back in.

An automatic background job on the server side keeps indexing the database as well as files every 10 minutes to Solr. This ensures that any changes made to databases and files keep showing up in Solr, and ultimately to the user carrying out the search.

However, this operation can also be carried out manually by the admin from the front-end, and should be done right after entering the solr core url.

In Administration → Settings → Solr Search Settings, the “Manual Index” button is for this purpose.

• https://cwiki.apache.org/confluence/display/solr/Installing+Solr
• http://www.sigmah.org/issues/view.php?id=535

Work is still going on for the search feature at:

• https://github.com/sigmah-dev/sigmah/pull/97