» Rust: Building Full-Text Search API with ElasticSearch » 2. Index Documents » 2.3 Send Index Requests

Send Index Requests

Let's add the API for indexing books, then we can put in some test data for later use.

Add async_trait dependency to use async fn in traits:

cargo add async_trait

Create src/domain/gateway/book_manager.rs:

use std::error::Error;

use async_trait::async_trait;

use crate::domain::model;

#[async_trait]
pub trait BookManager: Send + Sync {
    async fn index_book(&self, b: &model::Book) -> Result<String, Box<dyn Error>>;
}

Again, we‘re using 4 Layers Architecture Pattern. It will pay off when your project grows bigger and bigger. Read more here.

A type is Send if it is safe to send it to another thread. A type is Sync if it is safe to share between threads.

Add src/domain/gateway/mod.rs:

mod book_manager;

pub use book_manager::BookManager;

Remember to create mod.rs files for all sub-folders to re-export symbols.

Create src/infrastructure/search/es.rs:

use std::error::Error;

use async_trait::async_trait;
use elasticsearch::http::transport::Transport;
use elasticsearch::{Elasticsearch, IndexParts};
use serde_json::Value;

use crate::domain::gateway::BookManager;
use crate::domain::model;

const INDEX_BOOK: &str = "book_idx";

pub struct ElasticSearchEngine {
    client: Elasticsearch,
}

impl ElasticSearchEngine {
    pub fn new(address: &str) -> Result<Self, Box<dyn Error>> {
        let transport = Transport::single_node(address)?;
        let client = Elasticsearch::new(transport);
        Ok(ElasticSearchEngine { client })
    }
}

#[async_trait]
impl BookManager for ElasticSearchEngine {
    async fn index_book(&self, b: &model::Book) -> Result<String, Box<dyn Error>> {
        let response = self
            .client
            .index(IndexParts::Index(INDEX_BOOK))
            .body(b)
            .send()
            .await?;
        let response_body = response.json::<Value>().await?;
        Ok(response_body["_id"].as_str().unwrap().into())
    }
}

By default, Elasticsearch allows you to index documents into an index that doesn‘t exist yet. When you index a document into a non-existent index, Elasticsearch will dynamically create the index for you with default settings. This can be convenient for certain use cases where you don‘t want to manage index creation explicitly.

If you met unexpected transfer-encoding parsed error:

  • Try to disable proxies, including system proxies. System proxies are enabled by default.
  • Or, use TransportBuilder‘s disable_proxy() method.
let url = Url::parse(address)?;
let conn_pool = SingleNodeConnectionPool::new(url);
let transport = TransportBuilder::new(conn_pool).disable_proxy().build()?;

Add toml dependency:

cargo add toml

Tip: TOML stands for Tom's Obvious Minimal Language.

Create src/infrastructure/config/mod.rs:

use std::fs;

use serde::{Deserialize, Serialize};

#[derive(Debug, Deserialize, Serialize)]
pub struct Config {
    pub app: ApplicationConfig,
    pub search: SearchConfig,
}

#[derive(Debug, Deserialize, Serialize)]
pub struct SearchConfig {
    pub address: String,
}

#[derive(Debug, Deserialize, Serialize)]
pub struct ApplicationConfig {
    pub port: i32,
}

pub fn parse_config(file_name: &str) -> Config {
    let content = fs::read_to_string(file_name).expect("Failed to open TOML config file");
    toml::from_str(&content).expect("Failed to parse TOML config file")
}

Create config.toml:

[app]
port = 3000
page_size = 10

[search]
address = "http://localhost:9200"

Caution: Do not directly track your config.toml file with Git, as this could potentially leak sensitive data. If necessary, only track the template of the configuration file.
e.g.

[app]
port = 3000
page_size = 10

[search]
address = ""

Create src/application/executor/book_operator.rs:

use std::error::Error;
use std::sync::Arc;

use crate::domain::gateway;
use crate::domain::model;

pub struct BookOperator {
    book_manager: Arc<dyn gateway::BookManager>,
}

impl BookOperator {
    pub fn new(b: Arc<dyn gateway::BookManager>) -> Self {
        BookOperator { book_manager: b }
    }

    pub async fn create_book(&self, b: model::Book) -> Result<String, Box<dyn Error>> {
        Ok(self.book_manager.index_book(&b).await?)
    }
}

Create src/application/wire_helper.rs:

use std::sync::Arc;

use crate::domain::gateway;
use crate::infrastructure::search;
use crate::infrastructure::Config;

pub struct WireHelper {
    engine: Arc<search::ElasticSearchEngine>,
}

impl WireHelper {
    pub fn new(c: &Config) -> Result<Self, Box<dyn std::error::Error>> {
        let engine = Arc::new(search::ElasticSearchEngine::new(&c.search.address)?);
        Ok(WireHelper { engine })
    }

    pub fn book_manager(&self) -> Arc<dyn gateway::BookManager> {
        Arc::clone(&self.engine) as Arc<dyn gateway::BookManager>
    }
}

Create src/adapter/router.rs:

use axum::{
    extract::{Json, State},
    http::StatusCode,
    response::IntoResponse,
    routing::{get, post},
    Router,
};
use std::sync::Arc;

use serde_json::json;

use crate::application;
use crate::application::executor;
use crate::domain::model;

pub struct RestHandler {
    book_operator: executor::BookOperator,
}

async fn create_book(
    State(rest_handler): State<Arc<RestHandler>>,
    Json(book): Json<model::Book>,
) -> Result<Json<serde_json::Value>, impl IntoResponse> {
    match rest_handler.book_operator.create_book(book).await {
        Ok(book_id) => Ok(Json(json!({"id": book_id}))),
        Err(err) => Err((StatusCode::INTERNAL_SERVER_ERROR, err.to_string())),
    }
}

async fn welcome() -> Json<serde_json::Value> {
    Json(json!({
        "status": "ok"
    }))
}

pub fn make_router(wire_helper: &application::WireHelper) -> Router {
    let rest_handler = Arc::new(RestHandler {
        book_operator: executor::BookOperator::new(wire_helper.book_manager()),
    });
    Router::new()
        .route("/", get(welcome))
        .route("/books", post(create_book))
        .with_state(rest_handler)
}

It is common to share some state between handlers. For example, a pool of database connections or clients to other services may need to be shared.

The three most common ways of doing that are:

Here, we use State to share rest_handler between handlers.

Replace src/main.rs with the following code:

mod adapter;
mod application;
mod domain;
mod infrastructure;

const CONFIG_FILE: &str = "config.toml";

#[tokio::main]
async fn main() {
    let c = infrastructure::parse_config(CONFIG_FILE);
    let wire_helper = application::WireHelper::new(&c).expect("Failed to create WireHelper");
    let app = adapter::make_router(&wire_helper);
    let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
    axum::serve(listener, app).await.unwrap();
}

Remember to add these mod.rs files:

  • src/adapter/mod.rs
  • src/application/executor/mod.rs
  • src/application/mod.rs
  • src/domain/gateway/mod.rs
  • src/domain/mod.rs
  • src/infrastructure/mod.rs
  • src/infrastructure/search/mod.rs

Changes in Cargo.toml:

@@ -6,7 +6,10 @@ edition = "2021"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
 [dependencies]
+async-trait = "0.1.80"
 axum = "0.7.5"
+elasticsearch = "8.5.0-alpha.1"
 serde = { version = "1.0.198", features = ["derive"] }
 serde_json = "1.0.116"
 tokio = { version = "1.37.0", features = ["full"] }
+toml = "0.8.12"

Run the server again and try it with curl:

cargo run

Example request:

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"The Da Vinci Code","author":"Dan Brown","published_at":"2003-03-18","content":"In the Louvre, a curator is found dead. Next to his body, an enigmatic message. It is the beginning of a race to discover the truth about the Holy Grail."}' \
  http://localhost:3000/books

Example response:

{"id":"gnRLFI8Be1nlzJXaEsQR"}

Put in more test data

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"Harry Potter and the Philosopher\u0027s Stone","author":"J.K. Rowling","published_at":"1997-06-26","content":"A young boy discovers he is a wizard and begins his education at Hogwarts School of Witchcraft and Wizardry, where he uncovers the mystery of the Philosopher‘s Stone."}' \
  http://localhost:3000/books

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"To Kill a Mockingbird","author":"Harper Lee","published_at":"1960-07-11","content":"Set in the American South during the Great Depression, the novel explores themes of racial injustice and moral growth through the eyes of young Scout Finch."}' \
  http://localhost:3000/books

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"The Lord of the Rings","author":"J.R.R. Tolkien","published_at":"1954-07-29","content":"A hobbit named Frodo Baggins embarks on a perilous journey to destroy a powerful ring and save Middle-earth from the Dark Lord Sauron."}' \
  http://localhost:3000/books

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"The Catcher in the Rye","author":"J.D. Salinger","published_at":"1951-07-16","content":"Holden Caulfield narrates his experiences in New York City after being expelled from prep school, grappling with themes of alienation, identity, and innocence."}' \
  http://localhost:3000/books

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"The Alchemist","author":"Paulo Coelho","published_at":"1988-01-01","content":"Santiago, a shepherd boy, travels from Spain to Egypt in search of a treasure buried near the Pyramids. Along the way, he learns about the importance of following one‘s dreams."}' \
  http://localhost:3000/books

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"The Hunger Games","author":"Suzanne Collins","published_at":"2008-09-14","content":"In a dystopian future, teenagers are forced to participate in a televised death match called the Hunger Games. Katniss Everdeen volunteers to take her sister‘s place and becomes a symbol of rebellion."}' \
  http://localhost:3000/books

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"1984","author":"George Orwell","published_at":"1949-06-08","content":"Winston Smith lives in a totalitarian society ruled by the Party led by Big Brother. He rebels against the oppressive regime but ultimately succumbs to its control."}' \
  http://localhost:3000/books

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"The Girl with the Dragon Tattoo","author":"Stieg Larsson","published_at":"2005-08-01","content":"Journalist Mikael Blomkvist and hacker Lisbeth Salander investigate the disappearance of a young woman from a wealthy family, uncovering dark secrets and corruption."}' \
  http://localhost:3000/books

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"title":"Gone Girl","author":"Gillian Flynn","published_at":"2012-06-05","content":"On their fifth wedding anniversary, Nick Dunne‘s wife, Amy, disappears. As the media circus ensues and suspicions mount, Nick finds himself in a whirlwind of deception and betrayal."}' \
  http://localhost:3000/books
PrevNext