Hosting the CHT on Amazon EC2
Fixing couch2pg Memory Errors
Some times when couch2pg is replicating documents to postgres, it encounters very large info docs that are larger than the memory allocation of the document sync array and causes out-of-memory errors. To fix this, we need to delete this document so that couch2pg can proceed. Below are steps to follow to achieve this.
- Reduce the size of the replicated docs to a value of say 4 in the couch2pg.conf file so that you can get within the range of the failing document.
- Clone the existing couch2pg repo so that you can run couch2pg locally
- Edit the file
lib/importer.jsin the couch2pg source code to be able to log the doc-id of the problem doc
- Edit just logs doc_ids to the console, around line 100 of importer.js
- Get the remote couch2pg environment variable settings and export them into your profile terminal
- Create an ssh reverse tunnel from the postgres server to your laptop
- Run couch2pg locally so that you can see the doc-ids on console till it fails
- From the ids printed on console try loading the docs in the couchdb web access(Futon or Fauxton), the problem doc is usually big and won’t load
- This will help you identify the problem doc
- Curl the document to your pc and back it up
- Back up the document for further analysis
- Delete the document using curl
curl --head "<HOST>/<DB>/<DOC_ID>"This returns something like
HTTP/1.1 200 OK Cache-Control: must-revalidate Content-Length: 307 Content-Type: application/json Date: Tue, 25 Jun 2019 11:58:29 GMT ETag: "2-6beeb38da9b096bacfe2fa769e5171be" Server: CouchDB/2.3.1 (Erlang OTP/21) X-Couch-Request-ID: e4aa7a8696 X-CouchDB-Body-Time: 0
The ETag is the rev. Delete document with curl:
curl -X DELETE "<HOST>/<DB>/<DOC_ID>?rev=<THE_REV>"
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.