Skip to content
Android notifications 🔔, contact attachments 🖼️ and more 🎉 CHT 5.1 is here!
Host
CHT Core
Production Docker
Logs

Viewing server logs in CHT

What to do when you need to find server side errors in CHT

CHT has the following services running via Docker and each can have its logs queried:

  • nginx
  • sentinel
  • api
  • haproxy
  • couchdb
  • healthcheck
  • upgrade-service

Setting log level

To control log verbosity, set the LOG_LEVEL environment variable for the api and sentinel containers. The default value is info.

From version 5.1.0 onward, use LOG_LEVEL to control log verbosity. The NODE_ENV value no longer affects logging detail.

Configuration

Add LOG_LEVEL to the .env file used by your Docker Compose deployment:

LOG_LEVEL=debug

Then start or recreate the containers:

docker compose up -d

Supported values:

  • error — Only error messages
  • warn — Warnings and errors
  • info — General information (default, recommended for production)
  • debug — Detailed diagnostic information (troubleshooting only)
Using debug level generates significantly more logs and may impact performance. Use it only for troubleshooting and switch back to info or warn for normal operations.

Applying changes to running containers

To change the log level for already running containers, update the LOG_LEVEL value in your .env file and recreate the containers:

docker compose up -d
docker compose restart does not pick up environment variable changes. Use docker compose up -d to recreate the containers with the new configuration.

Viewing logs

First, find the actual names of the containers with the docker ps --format '{{.Names}}' command which should show something like this:

cht_nginx_1
cht_sentinel_1
cht_api_1
cht_haproxy_1
cht_healthcheck_1
cht_couchdb_1
upgrade-service-cht-upgrade-service-1

You can then use the docker logs command to view the logs of any given container. For example, if we call docker logs cht_nginx_1 it will show ALL the logs from that container. To show only the last 5 lines, you can use the --tail flag to specify the number of lines like this docker logs cht_nginx_1 --tail 5. The result will look like this:

10.131.161.1 - - [15/Feb/2023:21:08:35 +0000] "GET /medic/_changes?feed=longpoll&heartbeat=10000&since=115-g1AAAAH5eJyF0LENwjAQBVCLRIAEFBTMgESBCA0lrACJBzgnRXSKoKJmClaAxEswRZbIDCTHZ4GzXPzCT-d_rowx8zIqzDK_3fOycKdkf9jucJIKVyMybmVtxhQp6E-s23jfME00B-LdUWQIOBBxmJkG3gXJHHtfM8WaA2ncQ6RnGmsOZLjG5mLtk2mmSKAUCPHGSkxT3dZAiK_IR28A1AMhzta2wbko2iJe3ndMC92iaIfAzwrTmn8as5aY&limit=25 HTTP/1.1" 499 0 "https://10-131-161-159.local-ip.medicmobile.org/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0"
10.131.161.1 - - [15/Feb/2023:21:08:35 +0000] "GET /medic-user-medic-meta/_changes?include_docs=true&feed=longpoll&heartbeat=10000&since=13-g1AAAAH5eJyF0LENwkAMhWETKGkoWIICERpKWAESD3BOiugUQUXNFKwAiZdgiiyRGUjMY4FYV_zFfbJ8VxPRspqXtCpu96Iq5ZTuD9sdTlrjKgkka-Y8htkE-hOWjWrrOBCVo9noOBATzMwcB5JLVG0cB9LKw2xwHMh4XdCF-TktgTIg5I0nubYBQr5mnxiSaTsAIWfmzp2LRTvkpdq7Fov2CH7WYOMP5CCWMg&limit=25 HTTP/1.1" 499 0 "https://10-131-161-159.local-ip.medicmobile.org/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0"
10.131.161.1 - - [15/Feb/2023:21:08:35 +0000] "GET /fontawesome-webfont.woff2 HTTP/1.1" 304 0 "https://10-131-161-159.local-ip.medicmobile.org/styles.css" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0"
10.131.161.1 - - [15/Feb/2023:21:08:35 +0000] "GET /fonts/NotoSans-Bold.ttf HTTP/1.1" 304 0 "https://10-131-161-159.local-ip.medicmobile.org/styles.css" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0"
10.131.161.1 - - [15/Feb/2023:21:08:35 +0000] "GET /fonts/NotoSans-Regular.ttf HTTP/1.1" 200 221787 "https://10-131-161-159.local-ip.medicmobile.org/styles.css" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0

Sometimes you may want to search the logs for a specific string. To search, use the pipe (|) and grep commands to do this. Here we search for all the times HA Proxy thought CouchDB wasn’t reachable (DOWN) with this call docker logs cht_haproxy_1 2>&1 | grep 'DOWN':

<145>Feb 15 20:52:06 haproxy[25]: Server couchdb-servers/couchdb is DOWN, reason: Layer7 wrong status, code: 0, info: "via agent : down", check duration: 208ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 045/205206 (25) : Server couchdb-servers/couchdb is DOWN, reason: Layer7 wrong status, code: 0, info: "via agent : down", check duration: 208ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 045/205601 (25) : Server couchdb-servers/couchdb is DOWN, reason: Layer7 wrong status, code: 0, info: "via agent : down", check duration: 207ms. 0 active and 0 backup servers left. 5 sessions active, 0 requeued, 0 remaining in queue.
<145>Feb 15 20:56:01 haproxy[25]: Server couchdb-servers/couchdb is DOWN, reason: Layer7 wrong status, code: 0, info: "via agent : down", check duration: 207ms. 0 active and 0 backup servers left. 5 sessions active, 0 requeued, 0 remaining in queue.

If you want to watch the logs for a specific container in real time, you can use the --follow flag. This command would watch the requests come into API in realtime: docker logs cht_api_1 --follow. It’s nice to couple this with the --tail command so you only see the last 5 lines of the existing logs before watching for new lines with docker logs cht_api_1 --follow --tail 5 which would show this:

RES d17d71f5-2dcb-4ebb-bb0e-7874b3000570 10.131.161.1 - GET /medic/_design/medic-client/_view/reports_by_subject?keys=%5B%22557e79b8-2d99-4bd1-a4d6-a44491d483d8%22%5D HTTP/1.0 200 - 12.452 ms
RES e43c5d7f-4e32-433a-a96d-ef991f4298a3 10.131.161.1 - GET /medic/_design/medic/_view/doc_summaries_by_id?keys=%5B%22557e79b8-2d99-4bd1-a4d6-a44491d483d8%22%5D HTTP/1.0 200 - 31.226 ms
REQ c656ecc7-e6af-4564-ad63-2cab2c42844a 10.131.161.1 - GET /medic/_all_docs?include_docs=true&startkey=%22target~2023-02~557e79b8-2d99-4bd1-a4d6-a44491d483d8~%22&endkey=%22target~2023-02~557e79b8-2d99-4bd1-a4d6-a44491d483d8~%EF%BF%B0%22 HTTP/1.0
RES c656ecc7-e6af-4564-ad63-2cab2c42844a 10.131.161.1 - GET /medic/_all_docs?include_docs=true&startkey=%22target~2023-02~557e79b8-2d99-4bd1-a4d6-a44491d483d8~%22&endkey=%22target~2023-02~557e79b8-2d99-4bd1-a4d6-a44491d483d8~%EF%BF%B0%22 HTTP/1.0 200 - 11.153 ms
2023-02-15 21:54:49 DEBUG: Checking for a configured outgoing message service

Related Content

Community > Contribute > Code > Debugging > Sharing Logs

How to easily share logs from your CHT instance to get support

Community > Contribute > Code > Debugging > Browser and Phone Logs

How to obtain Android and browser client logs

Last updated on

Did this documentation help you ?

Change passwordBackup & Restore