If you are trying to use AWS ElasticSearch and want to keep it secure you will hit the ‘brick-wall’ of frustration pretty quickly. AWS ElasticSearch can be secured via
- IAM Resource or Role
However, getting things to work with common out of the box mechanisms is not there.
Problem for securing via IP
If you are running your client from a single server with a fixed IP, then it is easy. If you are running from spot instances or lambdas then control of your IP is complicated. Plus changing the security policy on AWS Elastic Search is minutes before it is in effect.
Problem for securing via IAM or Role
Most of us use the standard ElasticSearch client in our language of choice. By default, this client talks direct HTTP with some built-in authentication mechanisms. The client does not support AWS request signing, however, it does have a pluggable mechanism.
Solutions for IP Mechanism
Elad Nava wrote the go to post on getting it done with a nginx proxy behind a VPC.
Now your connection flow
client(s) --> PROXY in VPC --> AWS ElasticSearch
can be secured via the proxy and standard AWS security groups. While your Proxy and AWS ElasticSearch are hardened to a single IP.
Solution for using IAM or Roles
While the default client does not support signing requests, it does allow a handler to modify the request. If someone develops a handler for each language specific client then you can access AWS ElasticSearch from the standard client library! However, I could not find a reference or link to these handlers on AWS or ElasticSearch docs, instead scattered around mentions in forums. I pulled together the ones I required and linked them below.
Special thanks to the folks that got the ball rolling on these clients and opened them up.
- Jonathan Eskew - https://github.com/jeskew/amazon-es-php
- David Muller - https://github.com/DavidMuller/aws-requests-auth
- Geoff Wagstaff - https://github.com/TheDeveloper/http-aws-es
- However, I am starting from a fork since the pull-requests have not been pulled in