Docker Compose
Docker Compose is a tool for defining and running multi-container Docker applications. It allows you to use a YAML file to configure your application's services, networks, and volumes, and then spin up all the containers required to run your application with a single command.
Note:
The easiest and recommended way to get Docker Compose is to install Docker Desktop. Docker Desktop includes Docker Compose along with Docker Engine and Docker CLI which are Compose prerequisites.Install Compose if needed: https://docs.docker.com/compose/install/
Add compose/docker-compose.json:
services:
lr-event-books-web-node:
build:
context: ../
dockerfile: src/web/Dockerfile
ports:
- 3000:3000
volumes:
- ./config-web.json:/usr/src/app/src/web/config.json
depends_on:
mysql:
condition: service_healthy
kafka:
condition: service_healthy
lr-event-books-trend-node:
build:
context: ../
dockerfile: src/trend/Dockerfile
ports:
- 3001:3001
volumes:
- ./config-trend.json:/usr/src/app/src/trend/config.json
depends_on:
redis:
condition: service_started
kafka:
condition: service_healthy
lr-event-books-rec-node:
build:
context: ../
dockerfile: src/recommendation/Dockerfile
ports:
- 3002:3002
volumes:
- ./config-rec.json:/usr/src/app/src/recommendation/config.json
depends_on:
mongo:
condition: service_started
kafka:
condition: service_healthy
redis:
image: docker.io/bitnami/redis:7.0
environment:
- REDIS_PASSWORD=${REDIS_PASSWORD}
ports:
- 6379:6379
mysql:
image: docker.io/bitnami/mysql:5.7.43
environment:
- MYSQL_DATABASE=lr_book
- MYSQL_USER=test_user
- MYSQL_PASSWORD=${MYSQL_PASSWORD}
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
ports:
- 3306:3306
healthcheck:
test:
[
"CMD",
"mysqladmin",
"ping",
"-h",
"localhost",
"-u",
"root",
"-p$MYSQL_ROOT_PASSWORD",
]
timeout: 20s
retries: 10
volumes:
- ~/lr-mysql-data:/bitnami/mysql/data
mongo:
image: bitnami/mongodb:latest
environment:
- ALLOW_EMPTY_PASSWORD=yes
ports:
- 27017:27017
volumes:
- ~/lr-mongodb-data:/bitnami/mongodb
kafka:
image: bitnami/kafka:latest
environment:
- KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true
- KAFKA_CFG_NODE_ID=0
- KAFKA_CFG_PROCESS_ROLES=controller,broker
- KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,EXTERNAL:PLAINTEXT,PLAINTEXT:PLAINTEXT
- KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka:9093
- KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
ports:
- 9092:9092
healthcheck:
test:
[
"CMD-SHELL",
"kafka-topics.sh --list --bootstrap-server localhost:9092",
]
interval: 10s
timeout: 10s
retries: 3
It‘s not easy to find a working yet not annoying healthcheck test for MySQL and Kafka in Docker Compose.
Fix Topic Issues
In order to get rid of errors like “KafkaJSProtocolError: This server does not host this topic-partition“.
You may need to create a topic before subscribing it.
Do it in shell:
bin/kafka-topics.sh --create --topic lr-book-searches --bootstrap-server localhost:9092
Or programmatically:
const kafka = new Kafka(...)
const admin = kafka.admin()
// remember to connect and disconnect when you are done
await admin.connect()
await admin.createTopics({
validateOnly: <boolean>,
waitForLeaders: <boolean>
timeout: <Number>,
topics: <ITopicConfig[]>,
})
await admin.disconnect()
Add Config Files for Docker Compose
Add compose/config-web.json:
{
"app": {
"port": 3000,
"page_size": 5,
"templates_dir": "src/web/adapter/templates"
},
"db": {
"dsn": "mysql://test_user:test_pass@mysql:3306/lr_event_book?charset=utf8mb4"
},
"mq": {
"brokers": ["kafka:9092"],
"topic": "lr-book-searches"
},
"remote": {
"trend_url": "http://lr-event-books-trend-node:3001/trends",
"rec_url": "http://lr-event-books-rec-node:3002/recommendations?uid="
}
}
You need a special version of the config file for Docker Compose because all the hosts, paths, and urls vary in different hosting environments.
Add compose/config-trend.json:
{
"app": {
"port": 3001
},
"cache": {
"host": "redis",
"port": 6379,
"password": "test_pass",
"db": 0,
"timeout": 5000
},
"mq": {
"brokers": ["kafka:9092"],
"topic": "lr-book-searches",
"groupId": "trend-svr"
}
}
Add compose/config-rec.json:
{
"app": {
"port": 3002,
"pageSize": 10
},
"db": {
"uri": "mongodb://mongo:27017",
"dbName": "lr_event_rec"
},
"mq": {
"brokers": ["kafka:9092"],
"topic": "lr-book-searches",
"groupId": "rec-svr"
}
}
Add compose/.env:
REDIS_PASSWORD=test_pass
MYSQL_PASSWORD=test_pass
MYSQL_ROOT_PASSWORD=test_root_pass
Caution:
.envfiles should be ignored in.gitignore.
Run it:
cd compose
docker compose up
You should see something like this:
[+] Running 7/7
✔ Container compose-redis-1 Created 0.0s
✔ Container compose-mysql-1 Recreated 0.2s
✔ Container compose-mongo-1 Recreated 0.1s
✔ Container compose-kafka-1 Recreated 0.2s
✔ Container compose-lr-event-books-rec-node-1 Created 0.2s
✔ Container compose-lr-event-books-web-node-1 Created 0.2s
✔ Container compose-lr-event-books-trend-node-1 Created 0.1s
Attaching to kafka-1, lr-event-books-rec-node-1, lr-event-books-trend-node-1, lr-event-books-web-node-1, mongo-1, mysql-1, redis-1
kafka-1 | kafka 07:58:07.37 INFO ==>
kafka-1 | kafka 07:58:07.37 INFO ==> Welcome to the Bitnami kafka container
...
redis-1 | redis 13:24:52.38
redis-1 | redis 13:24:52.39 Welcome to the Bitnami redis container
...
mongo-1 | mongodb 13:24:52.60 INFO ==>
mongo-1 | mongodb 13:24:52.60 INFO ==> Welcome to the Bitnami mongodb container
mongo-1 | mongodb 13:24:52.61 INFO ==> ** Starting MongoDB setup **
...
mysql-1 | mysql 13:24:52.61
mysql-1 | mysql 13:24:52.62 Welcome to the Bitnami mysql container
mysql-1 | mysql 13:24:52.63 INFO ==> ** Starting MySQL setup **
...
You don't need to manually install or setup those databases and message queues anymore. They're all in good hands with docker compose.
Anyway, you need to make sure that you have the MySQL databaselr_event_bookin your mysql docker container.CREATE DATABASE lr_event_book CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
Put in some test books if there isn‘t one.
curl -X POST -H "Content-Type: application/json" -d '{"title": "To Kill a Mockingbird", "author": "Harper Lee", "published_at": "1960-07-11", "description": "A novel set in the American South during the 1930s, dealing with themes of racial injustice and moral growth."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "1984", "author": "George Orwell", "published_at": "1949-06-08", "description": "A dystopian novel depicting a totalitarian regime, surveillance, and propaganda."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "Pride and Prejudice", "author": "Jane Austen", "published_at": "1813-01-28", "description": "A classic novel exploring the themes of love, reputation, and social class in Georgian England."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "The Catcher in the Rye", "author": "J.D. Salinger", "published_at": "1951-07-16", "description": "A novel narrated by a disaffected teenager, exploring themes of alienation and identity."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "The Lord of the Rings", "author": "J.R.R. Tolkien", "published_at": "1954-07-29", "description": "A high fantasy epic following the quest to destroy the One Ring and defeat the Dark Lord Sauron."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "Moby-Dick", "author": "Herman Melville", "published_at": "1851-10-18", "description": "A novel exploring themes of obsession, revenge, and the nature of good and evil."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "The Hobbit", "author": "J.R.R. Tolkien", "published_at": "1937-09-21", "description": "A fantasy novel set in Middle-earth, following the adventure of Bilbo Baggins and the quest for treasure."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "The Adventures of Huckleberry Finn", "author": "Mark Twain", "published_at": "1884-12-10", "description": "A novel depicting the journey of a young boy and an escaped slave along the Mississippi River."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "War and Peace", "author": "Leo Tolstoy", "published_at": "1869-01-01", "description": "A novel depicting the Napoleonic era in Russia, exploring themes of love, war, and historical determinism."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "Alice’s Adventures in Wonderland", "author": "Lewis Carroll", "published_at": "1865-11-26", "description": "A children’s novel featuring a young girl named Alice who falls into a fantastical world populated by peculiar creatures."}' http://localhost:3000/api/books
curl -X POST -H "Content-Type: application/json" -d '{"title": "The Odyssey", "author": "Homer", "published_at": "8th Century BC", "description": "An ancient Greek epic poem attributed to Homer, detailing the journey of Odysseus after the Trojan War."}' http://localhost:3000/api/books
Now, if you visit your page at http://localhost:3000/, you should be able to see all the features from these 3 microservices.
Search with terms like “love“, “peace“ and “Odyssey“, then refresh the page and see what happens.
Your event-driven microservices work like a charm! 📢
Loading...
> code result goes here