frank
Goto Top

Ubuntu: Build Redisearch v2 module for Redis from source code

article-picture
Here is a tutorial to compile the Redisearch module v2.x for Redis by yourself. At the moment you can get it as binary only as subscriber of the Enterprise Version. With "apt-get" you can only install the Redisearch module v1.x. Version v1.x works fine, but has much less features. Without the subscription you can only build the v2.x version from the source code. The official documentation for the RediSearch module can be found here.

RediSearch is the search engine or more precisely, the search index for the Redis database. It is currently the fastest search index you can install for free.


back-to-topWhy not Docker?


The Redis company recommends a Docker install (redis-stack) for RediSearch and Redis. However, I want the server to run natively on the machine since our production is already virtualized (KVM). A virtualization (Docker) in a virtualization (KVM) works of course, but a lot of performance is lost and the operation is not that comfortable especially in development. The alternative installation method via "Docker" can be found on the RediSearch Quickstart page.

back-to-topPreparation, prerequisites for compiling


  • Tested under Ubuntu 22.04
  • All commands are executed under the Linux bash and as user "frank" and not directly as root! The hostname of the system is: "nexus". If "root" is needed, I use "sudo" for it.
  • The home directory ~/redis is used for the source code and compilation
  • All tools and programs needed for development under Ubuntu 22.04 must be installed before:

sudo apt update 
sudo apt install build-essential g++ make git

back-to-topLoad source code via Git


mkdir ~/redis 
cd ~/redis 
git clone --recursive https://github.com/RediSearch/RediSearch.git
cd RediSearch

Tip: The "--recursive" is used to load submodules.

If you want to load a specific release via Git you can control this with the "--branch" option. For example, if we want to load the Redisearch release 2.4.15:

git clone --recursive https://github.com/RediSearch/RediSearch.git --branch v2.4.15

This has the advantage that the correct version number of Redisearch also appears in the Redis log and in "redis-cli". Without the "--branch" specification a 99.99.99 (Git=master-xxx) appears as version.

back-to-topCheck and install dependencies


Here the user password is requested (because internally "sudo" is called)

make setup

The result should look like this or similar:

~/redis/RediSearch$ make setup
# Using CC=gcc
Setting up system...
[sudo] Passwort für frank: 
sudo apt-get -qq update -y
# readies version: ed8290a
sudo apt-get -qq install --fix-missing -y ca-certificates
sudo apt-get -qq install --fix-missing -y curl wget unzip
sudo /home/frank/redis/RediSearch/deps/readies/bin/enable-utf8
sudo apt-get -qq install --fix-missing -y git gawk jq openssl rsync unzip
sudo apt-get -qq install --fix-missing -y patch
sudo apt-get -qq install --fix-missing -y libatomic1
/home/frank/redis/RediSearch/deps/readies/bin/getgcc --modern
sudo apt-get -qq install --fix-missing -y libtool m4 automake libssl-dev
sudo apt-get -qq install --fix-missing -y python3-dev
sudo /usr/bin/python3 /home/frank/redis/RediSearch/deps/readies/bin/getcmake --usr
/usr/bin/python3 /home/frank/redis/RediSearch/deps/readies/bin/getrmpytools --reinstall --modern
sudo apt-get -qq install --fix-missing -y lcov
/usr/bin/python3 -m pip install --disable-pip-version-check --user pudb awscli
/usr/bin/python3 -m pip install --disable-pip-version-check --user gevent
/usr/bin/python3 -m pip install --disable-pip-version-check --user -r /home/frank/redis/RediSearch/tests/pytests/requirements.txt

back-to-topBuild and linking


make build

A sequence from 1% to 100% appears with a few "deprecated:" warnings. These can be ignored.
The output should start like this and end like this:

# Using CC=gcc
# RediSearch root: /home/frank/redis/RediSearch
# RediSearch binroot: /home/frank/redis/RediSearch/bin/linux-x64-release
# RediSearch static libstdc++: on
-- The C compiler identification is GNU 10.3.0
-- The CXX compiler identification is GNU 10.3.0
-- Detecting C compiler ABI info

....

[100%] Linking CXX executable rstest
[100%] Built target rstest

back-to-topInstall Redis server (or does it already exist)?


For the next step Redis must be installed. If you already have the Redis server installed on your system, you can skip this step and go straight to "Test RediSearch Module".

Here is a short command sequence on how to install the Redis server using the official "redis.io" packages on Ubuntu 22.04:

curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list  
sudo apt-get update
sudo apt-get install redis

After that the Redis-Server and the Redis-Cli is installed and started automatically. You can test the installation with the call "redis-cli ".

redis-cli 

The following should appear with a blinking cursor:

127.0.0.1:6379> 

With the command "info" you can display all kinds of information about redis:

127.0.0.1:6379> info
# Server
redis_version:7.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:68bf11aad5b039df
redis_mode:standalone
os:Linux 5.15.0-48-generic x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:11.2.0
process_id:790003
process_supervised:systemd
run_id:cac9eb4a3f82d6450dadc58079473c087027be3f
tcp_port:6379
server_time_usec:1665166255039660
uptime_in_seconds:72
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:4221871
executable:/usr/bin/redis-server
config_file:/etc/redis/redis.conf
io_threads_active:0
...

With "exit" you get out of the Redis-Cli again.

With the "service" commands "start", "restart", "stop" and "status" you can start, restart, stop and read the current status of the Redis server. The log file can be found under "/var/log/redis/redis-server.log".

Redis Start:
service redis-server start

Redis Stop:
service redis-server stop

back-to-topTest RediSearch module


Now, before going any further, you need to stop the Redis server:

service redis-server stop

With "status" you can read out the current status. Active: inactive (dead)" should appear.

service redis-server status
    redis-server.service - Advanced key-value store
     Loaded: loaded (/lib/systemd/system/redis-server.service; disabled; vendor preset: enabled)
     Active: inactive (dead)
       Docs: http://redis.io/documentation,
             man:redis-server(1)

Now test the newly created RediSearch module in the directory "~/redis/RediSearch":

make run 

Output:

# Using CC=gcc
801022:C 07 Oct 2022 20:19:53.449 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
801022:C 07 Oct 2022 20:19:53.449 # Redis version=7.0.5, bits=64, commit=00000000, modified=0, pid=801022, just started
801022:C 07 Oct 2022 20:19:53.449 # Configuration loaded
801022:M 07 Oct 2022 20:19:53.450 * Increased maximum number of open files to 10032 (it was originally set to 1024).
801022:M 07 Oct 2022 20:19:53.450 * monotonic clock: POSIX clock_gettime
                _._                                                  
           _.-``__ ''-._                                               
      _.-``    `.  `_.  ''-._           Redis 7.0.5 (00000000/0) 64 bit  
  .-`` .-```.  ```\/    _.,_ ''-._                                    
 (    '      ,       .-`  | `,    )     Running in standalone mode  
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379  
 |    `-._   `._    /     _.-'    |     PID: 801022  
  `-._    `-._  `-./  _.-'    _.-'                                     
 |`-._`-._    `-.__.-'    _.-'_.-'|                                    
 |    `-._`-._        _.-'_.-'    |           https://redis.io         
  `-._    `-._`-.__.-'_.-'    _.-'                                     
 |`-._`-._    `-.__.-'    _.-'_.-'|                                    
 |    `-._`-._        _.-'_.-'    |                                    
  `-._    `-._`-.__.-'_.-'    _.-'                                     
      `-._    `-.__.-'    _.-'                                         
          `-._        _.-'                                             
              `-.__.-'                                                 

801022:M 07 Oct 2022 20:19:53.450 # Server initialized
801022:M 07 Oct 2022 20:19:53.450 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. 
801022:M 07 Oct 2022 20:19:53.455 * <search> Redis version found by RedisSearch : 7.0.5 - oss
801022:M 07 Oct 2022 20:19:53.455 * <search> RediSearch version 99.99.99 (Git=master-f7f5957c)
801022:M 07 Oct 2022 20:19:53.455 * <search> Low level api version 1 initialized successfully
801022:M 07 Oct 2022 20:19:53.455 * <search> concurrent writes: OFF, gc: ON, prefix min length: 2, prefix max expansions: 200, query timeout (ms): 500, timeout policy: return, cursor read size: 1000, cursor max idle (ms): 300000, max doctable size: 1000000, max number of search results:  1000000, search pool size: 20, index pool size: 8, 
801022:M 07 Oct 2022 20:19:53.456 * <search> Initialized thread pool!
801022:M 07 Oct 2022 20:19:53.456 * <search> Enabled role change notification
801022:M 07 Oct 2022 20:19:53.456 * Module 'search' loaded from /home/frank/redis/RediSearch/bin/linux-x64-release/search/redisearch.so  
801022:M 07 Oct 2022 20:19:53.456 * Ready to accept connections

The module was loaded and the Redis server started. In the output there should be the entry: "Module 'search' loaded from /var/lib/redis/modules/redisearch.so". If not an error has occurred somewhere. Then repeat the first steps of the tutorial again.

With "Ctrl "+"c" you stop the Redis server.

back-to-topLoad RediSearch module automatically when starting Redis


The newly created RediSearch module is located after the creation under:

/home/frank/redis/RediSearch/bin/linux-x64-release/search/redisearch.so

We now create a module directory under "/var/lib/redis/" and copy the newly created library "redisearch.so" into the created directory:

sudo mkdir /var/lib/redis/modules
sudo cp /home/frank/redis/RediSearch/bin/linux-x64-release/search/redisearch.so /var/lib/redis/modules/.

You can check the copy process with:

sudo ls -la /var/lib/redis/modules/

The module should now appear under "/var/lib/redis/modules/".

Now we adjust the Redis server configuration file (/etc/redis/redis.conf). To do this, we open the file and add to it in the MODULES section:

sudo vi /etc/redis/redis.conf

Tip: If you don't know how to use the "vi", you can replace it with the command "nano". So "sudo nano /etc/redis/redis.conf".

################################## MODULES #####################################

# Load modules at startup. If the server is not able to load modules
# it will abort. It is possible to use multiple loadmodule directives.
#
# loadmodule /path/to/my_module.so
# loadmodule /path/to/other_module.so

loadmodule /var/lib/redis/modules/redisearch.so

################################## NETWORK #####################################

Save the "redis.conf" file.

Tip: You can copy the Redis library "redisearch.so" wherever you want. You just have to change the path under "loadmodule".

Restart Redis server:

service redis-server start

With "status" you can test the Redis server and see if everything is running. Status: "Ready to accept connections" should appear.

service redis-server status
redis-server.service - Advanced key-value store
     Loaded: loaded (/lib/systemd/system/redis-server.service; disabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-10-07 20:22:52 CEST; 3s ago
       Docs: http://redis.io/documentation,
             man:redis-server(1)
   Main PID: 801862 (redis-server)
     Status: "Ready to accept connections"  
      Tasks: 5 (limit: 18977)
     Memory: 2.9M
        CPU: 98ms
     CGroup: /system.slice/redis-server.service
             └─801862 "/usr/bin/redis-server 127.0.0.1:6379" "" "" "" "" "" "" ""  

Okt 07 20:22:51 nexus systemd[1]: Starting Advanced key-value store...
Okt 07 20:22:52 nexus systemd[1]: Started Advanced key-value store.

In the logfile "/var/log/redis/redis-server.log" should appear the entry: "Module 'search' loaded from /var/lib/redis/modules/redisearch.so".

Here is the snippet about it:

817526:M 07 Oct 2022 20:55:26.760 * <search> Redis version found by RedisSearch : 7.0.5 - oss
817526:M 07 Oct 2022 20:55:26.760 * <search> RediSearch version 99.99.99 (Git=master-f7f5957c)
817526:M 07 Oct 2022 20:55:26.760 * <search> Low level api version 1 initialized successfully
817526:M 07 Oct 2022 20:55:26.760 * <search> concurrent writes: OFF, gc: ON, prefix min length: 2, prefix max expansions: 200, query timeout (ms): 500, timeout policy: return, cursor read size: 1000, cursor max idle (ms): 300000, max doctable size: 1000000, max number of search results:  1000000, search pool size: 20, index pool size: 8,
817526:M 07 Oct 2022 20:55:26.760 * <search> Initialized thread pool!
817526:M 07 Oct 2022 20:55:26.760 * <search> Enabled role change notification
817526:M 07 Oct 2022 20:55:26.760 * Module 'search' loaded from /var/lib/redis/modules/redisearch.so  

back-to-topTest Redis and RediSearch module in operation


The Redis server should now be running, the RediSearch module created, copied and activated. Now we test this at runtime using the "redis-cli":

redis-cli
127.0.0.1:6379> 
127.0.0.1:6379> info modules
# Modules
module:name=search,ver=999999,api=1,filters=0,usedby=,using=,options=

# search_version
search_RedisSearch_version:7.0.5

# search_index
search_number_of_indexes:0

# search_fields_statistics

# search_runtime_configurations
search_concurrent_mode:OFF
search_enableGC:ON
search_minimal_term_prefix:2
search_maximal_prefix_expansions:200
search_query_timeout_ms:500
search_timeout_policy:return
search_cursor_read_size:1000
search_cursor_max_idle_time:300000
search_max_doc_table_size:1000000
search_max_search_results:1000000
search_max_aggregate_results:-1
search_search_pool_size:20
search_index_pool_size:8
search_gc_scan_size:100
search_min_phonetic_term_length:3
127.0.0.1:6379> 

If this appears, you have done everything right! Congratulations face-smile

If you are interested in Redis and RediSearch 2 you should have a look at the following:


There are also several clients for almost all programming languages (PHP, Ruby, Javascript, Go, etc.):


back-to-topTroubleshooting


If an error occurs when starting or stopping the Redis server, you can solve this using the commands:

systemctl status redis-server.service

or

journalctl -xeu redis-server.service

check exactly. Also the Redis logfile often gives an answer to errors:

cat /var/log/redis/redis-server.log


I am always happy about feedback or improvements. If you like the tutorial please click on the "heart", as a small thank you face-smile

Greetings
Frank

Content-Key: 4707820878

Url: https://administrator.de/contentid/4707820878

Printed on: April 25, 2024 at 07:04 o'clock