Exploring Elasticsearch with Laravel Scout

Searching in Laravel is very easy with Laravel Scout, but it only ships with an Algolia driver. Because Elasticsearch is fun, powerful and can also be self-hosted I created a Elasticsearch driver for Laravel Scout, it is called Explorer.

What is what?

Elasticsearch is "a distributed, open source search and analytics engine for all types of data" using REST and JSON. The key concepts being are the index (a collection of documents), full-text searches and data ingestion (the processing of raw data). Kibana is the interface to visualise and navigate through your elasticsearch indexes. I recommend that you set up both to make learning to work with Elasticsearch much more fun.

A very good way to get started with Elasticsearch is by creating a docker container and connecting it to your Laravel application. You could use Tighten's Takeout for the Elasticsearch container, however it comes without Kibana. To get started with Elasticsearch and Kibana, create a docker-compose.yml file and paste this configuration from Elastic's documentation. After that, run docker-compose up -d and you will have an elasticsearch instance running at localhost:9200 and a Kibana instance at localhost:5601

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.3
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic

  kib01:
    image: docker.elastic.co/kibana/kibana:7.9.3
    container_name: kib01
    ports:
      - 5601:5601
    environment:
      ELASTICSEARCH_URL: http://es01:9200
      ELASTICSEARCH_HOSTS: http://es01:9200
    networks:
      - elastic

volumes:
  data01:
    driver: local

networks:
  elastic:
    driver: bridge

Now that this is done, the application still needs the right tools. Install Laravel Scout and Explorer ia Composer:

composer require laravel/scout jeroen-g/explorer

For Explorer you will need the configuration file to define your indexes:

php artisan vendor:publish --tag=explorer.config

Now that Scout is installed, set the driver in your .env file to SCOUT_DRIVER=elastic. After that, you can define your first index in config/explorer.php:

'indexes' => [
    \App\Models\Post::class
],

Upon saving the file, run php artisan elastic:create to create this index, and php artisan scout:import "App\Models\Post" to add your posts as documents to the index. This of course assumes that you have a Post model (with an ID and title attribute) and a couple of entries of them in your database. As mentioned before, Laravel Scout also has a few requirements explained in its documentation for your models. Here is one way the Post model could look:

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
use JeroenG\Explorer\Application\Explored;
use Laravel\Scout\Searchable;

class Post extends Model implements Explored
{
    use HasFactory;
    use Searchable;

    protected $fillable = ['title', 'published'];

    public function mappableAs(): array
    {
        return [
            'id' => 'keyword',
            'title' => 'text',
            'published' => 'boolean',
            'created_at' => 'date',
        ];
    }
}

Regarding the mapping in mappableAs(), this is to tell Elasticsearch what type the fields should be. The power of Elasticsearch lies in its view of data as a document that can efficiently be searched through in its entirety. For this, Elasticsearch maps the given parts of the document, such as a Post's ID and title, to types it can work with. For example, the ID can be mapped to keyword, the created_at to date and the title to text. If you feed Elasticsearch raw data without a mapping, it will try to infer the types by itself. A complete overview of all possible types can be found here. I recommend that you read the documentation on arrays in Elasticsearch in particular.

You could use a Factory and Seeder to quickly fill your database with a few example posts. To query the posts, use Scout's search method to find stuff, for example:

$ipsum = App\Models\Post::search('Lorem')->get();

Explorer adds a few custom functions to Laravel Scout to harness the power of Elasticsearch. You might for example want to write a query to get all posts that are published; have "lorem" somewhere in the document and have "ipsum" in the title. That query would look like this:

$posts = Post::search('lorem')
    ->must(new Matching('title', 'ipsum'))
    ->filter(new Term('published', true))
    ->get();

There are a lot of possibilities with Elasticsearch and Explorer, be sure to check out the repository if you are interested! For now, enjoy and let's hope you will find what you're looking for ;)