Richard Friedman: ElasticSearch Client with Request Signing for AWS Recap

If you are trying to use AWS ElasticSearch and want to keep it secure you will hit the ‘brick-wall’ of frustration pretty quickly. AWS ElasticSearch can be secured via

IP
IAM Resource or Role

However, getting things to work with common out of the box mechanisms is not there.

Problem for securing via IP

If you are running your client from a single server with a fixed IP, then it is easy. If you are running from spot instances or lambdas then control of your IP is complicated. Plus changing the security policy on AWS Elastic Search is minutes before it is in effect.

Problem for securing via IAM or Role

Most of us use the standard ElasticSearch client in our language of choice. By default, this client talks direct HTTP with some built-in authentication mechanisms. The client does not support AWS request signing, however, it does have a pluggable mechanism.

Solutions for IP Mechanism

Elad Nava wrote the go to post on getting it done with a nginx proxy behind a VPC.

https://eladnava.com/secure-aws-elasticsearch-service-behind-vpc/

Now your connection flow

client(s) --> PROXY in VPC --> AWS ElasticSearch

can be secured via the proxy and standard AWS security groups. While your Proxy and AWS ElasticSearch are hardened to a single IP.

Solution for using IAM or Roles

While the default client does not support signing requests, it does allow a handler to modify the request. If someone develops a handler for each language specific client then you can access AWS ElasticSearch from the standard client library! However, I could not find a reference or link to these handlers on AWS or ElasticSearch docs, instead scattered around mentions in forums. I pulled together the ones I required and linked them below.

Special thanks to the folks that got the ball rolling on these clients and opened them up.

PHP