Remote connection methods to EC2 Linux instances on AWS: comparison and best practices

Remote administration has become a cornerstone for the success of cloud infrastructures. In today's world, where mobility and flexibility are essential, having the ability to manage servers and resources from any location has become indispensable for businesses and IT professionals.

If you're looking to enhance the security and flexibility of your remote connections, this article will review all possible methods and provide you with the necessary knowledge to make informed decisions, selecting the most suitable method according to your needs. We will explore how some of these methods complement each other, allowing us to progressively strengthen security, reduce operational burden, and simplify the administration of our infrastructure.

Connection using SSH long lived key pairs

To remotely connect to a server from our computer, we can use Secure Shell (SSH). SSH is a network protocol that provides a secure and encrypted connection for executing commands on a server. Security is based on the use of asymmetric cryptography, which involves a pair of keys. In this system, the user on their computer possesses the private key, while the remote server possesses the public key. This key pair is used to verify the legitimacy of the user attempting to establish the connection with the server.

Infraestructure diagram to show how to deploy an EC2 instance to be able to connect using SSH

Before proceeding with the deployment of our Amazon Elastic Compute Cloud (EC2) instance, it's vitally important to generate a pair of keys in the AWS region where we intend to implement it. Once this step is completed, we will download the private key of the key pair to our computer. This private key will be used to perform connections to the server. During the EC2 instance deployment process, it will be necessary to specify the use of the newly created key pair, enabling the deployment of the instance with the corresponding public key. This type of SSH key pairs are known as long-lived key pairs, as they have an extended lifespan, with the same key being reused for multiple connections over time.

To be able to establish an SSH connection with an EC2 instance, it's essential to ensure that the instance is accessible over the Internet. To achieve this, the EC2 instance must be deployed in a public subnet with a properly configured route table. This route table must allow all traffic destined for 0.0.0.0/0 to be routed to an internet gateway associated with the Amazon Virtual Private Cloud (VPC). Once these steps are taken, the instance will have a public Internet Protocol (IP) address and will be accessible from any point on the Internet.

To enable SSH connections on our instance, we must ensure it's associated with an appropriate Security Group (SG), as SSH connections are established through Transmission Control Protocol (TCP) traffic on port 22. To achieve this, we will add an inbound rule to the security group, allowing TCP traffic on port 22 with a source of 0.0.0.0/0, meaning from any point on the Internet. However, if we want to restrict SSH connections only to specific and known IPs, we can configure the security group to accept SSH traffic only from those particular IPs. This measure enhances the security of our instances, as any attempt to establish an SSH connection from unauthorized IPs will be rejected by the security group. This way, we can have more precise control over who can access our instance, reducing potential risks.

Once these steps are completed, we will be ready to execute the following command from our computer, substituting the appropriate parameters, to establish the connection:

chmod 400 [PATH-TO-PRIVATE-KEY]
ssh -i [PATH-TO-PRIVATE-KEY] [USER-NAME]@[SERVER-ADDRESS]

The security of this long-lived key pairs scenario relies on the proper generation and protection of SSH keys. It's vital that the private key remains securely stored on each user's computer. Furthermore, it's essential to establish policies and governance procedures governing the secure management of keys, including periodic rotation and revocation in case of compromise or loss.

When performing independent SSH connections for each of our servers, we lack a unified system to audit SSH connections made by users. This necessitates accessing each of the servers individually to review the logs generated by each one. The lack of centralization makes detecting and analyzing suspicious activities difficult, making the auditing process more complex and labor-intensive.

If you want to practically test this scenario in your AWS account and connect to an EC2 instance using the method described in the diagram, I have created this CloudFormation template for you to deploy the necessary resources to test it: LongLived-SSHKeyPair-InstanceWithPublicIP.yaml

Connection using SSH ephemeral key pairs with EC2 Instance Connect

When establishing SSH connections to our instances in the traditional manner, the use of key pairs can become complex and tedious, especially when generating, distributing, and securely storing private keys. Additionally, managing multiple keys for different instances can become a cumbersome and error-prone task. EC2 Instance Connect eliminates the need to manage key pairs as this service uses a new key pair each time a connection is established; these types of keys are known as ephemeral key pairs.

Infraestructure diagram to show how to deploy an EC2 instance to be able to connect using EC2 Instance Connect

EC2 Instance Connect can be used from the AWS Management Console, providing us with a browser-based SSH console experience, or by using the AWS Command Line Interface (CLI). When establishing the connection using either of these methods, EC2 Instance Connect makes an Application Programming Interface (API) call to AWS and sends the public SSH key to the instance, which is stored in the instance metadata for 60 seconds.

To enable the sending of the public key, the user must be associated with an Identity Access Management (IAM) policy that allows it. Unlike a traditional SSH connection, where users gain access to instances by sharing the private key, with EC2 Instance Connect, we can simply grant permissions through IAM. This gives us greater control over which users can access which instances and allows us to revoke those permissions at any time.

To establish the connection, the user must have the "ec2-instance-connect:SendSSHPublicKey" and "ec2:DescribeInstances" actions enabled in their IAM policy. We can customize policies to provide the user access to specific instances or even instances with a particular tag, providing flexibility in access permission management.

{
  "Version": "2012-10-17",
  "Statement": [{
      "Effect": "Allow",
      "Action": "ec2-instance-connect:SendSSHPublicKey",
      "Resource": [
        "arn:aws:ec2:[REGION]:[ACCOUNT-ID]:instance/[INSTANCE-ID] "
      ],
      "Condition": {
        "StringEquals": {
          "ec2:osuser": "[AMI-USER-NAME]"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:DescribeInstances",
      "Resource": "*"
    }
  ]
}

By using EC2 Instance Connect, all SSH connections are logged in CloudTrail, AWS's auditing service, allowing for detailed and centralized tracking of who, when, and from where instances have been accessed.

It's important to note that EC2 Instance Connect only supports connections using IPv4 and is not compatible with the use of IPv6. Additionally, this functionality is not available in all AWS regions. To use EC2 Instance Connect, it's essential to verify if it's enabled in the region we are utilizing.

Another crucial aspect is that to use EC2 Instance Connect, the corresponding package must be installed on the instance. By default, this package comes pre-installed in Amazon Linux 2 AMIs (version 2.0.20190618 or later) and Ubuntu (version 20.04 or later). It's possible to manually install the package on Amazon Linux 2 AMIs (any version) or Ubuntu (version 16.04 or later).

EC2 Instance Connect using AWS Management Console

If we wish to establish the connection through the AWS Management Console, it's important that the instance is deployed in a public subnet, allowing access from the Internet. Additionally, we must associate a security group that permits incoming traffic on port 22 to enable SSH connections.

EC2 Instance Connect, when using the AWS Management Console, utilizes specific IP address ranges for SSH connections. Hence, it's possible to restrict the security group to allow traffic only from these specific IPs. By using this method, the process of sending the public key to the server and using the private key to establish the connection happens transparently, without needing to worry about generating a pair of ephemeral keys.

EC2 Instance Connect using AWS CLI

In the case that we want to establish the connection via AWS CLI, there are two possible scenarios. If we want to connect to our instances from the Internet, we'll need to deploy the instance in a public subnet and enable incoming traffic on port 22 from any IP or restrict it to a fixed group of IPs from which we know we'll establish the connection.

If we have private connectivity to our instance (we have a VPN or Direct Connect deployed), it won't be necessary for our instances to be in public subnets. They can be in private subnets and we can connect through their private IP address.

The first thing we need to do to initiate an SSH connection is to generate a new pair of keys or use a pair of keys that we already have generated on our computer. We can generate an SSH key pair with the following command.

ssh-keygen -t rsa -f [KEY-NAME]

Once we have our pair of ephemeral keys, we will use the following AWS CLI command to send the public key to the instance's metadata with which we want to establish the connection.

aws ec2-instance-connect send-ssh-public-key \
--region [REGION] \
--availability-zone [AVAILABILITY-ZONE] \
--instance-id [INSTANCE-ID] \
--instance-os-user [USERNAME] \
--ssh-public-key file:// [KEY-NAME].pub

During the 60 second period, we can initiate the SSH connection with the private key associated with the public key that has been sent to the instance. To establish the SSH connection, we can use the following command.

ssh -o "IdentitiesOnly=yes" -i [KEY-NAME] [USER-NAME]@[SERVER-ADDRESS]

SSH Connection through a Bastion Host

If we wish to keep our instances in private subnets of the VPC and avoid exposing port 22 to the Internet, an effective solution is to use a Bastion Host. This approach involves using an intermediate instance between the source and the final instance. The Bastion Host acts as an entry point exposed to the Internet, which we access to establish the connection. Once inside, we can initiate a new connection using the private IP address of the final instance. This way, we can ensure greater security by restricting direct access from the Internet to our instances, while facilitating remote administration.

Infraestructure diagram to show how to deploy an EC2 instance to be able to connect through a Bastion Host

To implement this solution, we will associate an Internet Gateway with our VPC. Next, we will create a public subnet and configure a route table so that traffic destined for 0.0.0.0/0 is directed to the Internet Gateway, while traffic destined within the VPC range is routed locally. Within the public subnet, we will deploy our EC2 instance, which will assume the role of a Bastion Host. For this instance, we will define a security group that allows incoming traffic on port 22, either from any IP or restricted to specific IPs if we know their origin. To restrict the connections the Bastion Host can initiate, we will add an outbound rule to the security group to allow only outgoing SSH traffic to the security group associated with the instances we want to establish a connection to.

Now, we will proceed to create a private subnet with a route table that will only route traffic within the VPC. In this subnet, we will deploy our final EC2 instances that we wish to access via SSH. It's important to associate these instances with a security group that allows incoming traffic on port 22, but only from the security group previously associated with our Bastion Host instance. This way, we will ensure that our virtual machine is only accessible via SSH through the Bastion Host, and that direct connections from the Internet cannot be established.

To be able to establish the connection with the Bastion Host, we will need to have a key pair. In turn, our Bastion Host must have the necessary key pairs to subsequently connect to each of the servers in the private subnets that we wish to access.

If we are using AMIs compatible with EC2 Instance Connect, we can combine the Bastion Host solution with EC2 Instance Connect. By doing this, we eliminate the need to manage private and public keys both for connecting to the Bastion Host and for connecting from the Bastion Host to the final instances.

To access our Bastion Host via EC2 Instance Connect, we will have the option to do so both from the AWS Management Console and by using AWS CLI. However, to access the final instances in private subnets from the Bastion Host, we can only use AWS CLI, and it will be necessary to target the private IP addresses of our servers.

To allow the Bastion Host to use AWS CLI and connect via EC2 Instance Connect to the final servers, it's essential to associate it with an IAM Role that includes a policy enabling the establishment of connections. This way, we will ensure that the Bastion Host has the appropriate permissions to interact with the instances in private subnets in a secure and controlled manner. Additionally, we will have to add a new outbound rule to the security group to allow outgoing traffic from port 443 to destination 0.0.0.0/0. This is necessary so that when we run the CLI command from our Bastion Host, we can reach the EC2 Instance Connect API endpoint.

If you want to practically test these two scenarios in your AWS account and connect to an EC2 instance using the methods described in the diagrams, I have created these CloudFormation templates for you to deploy the necessary resources to test them: LongLived-SSHKeyPair-BastionHost.yaml and Ephemeral-SSHKeyPair-BastionHost.yaml

SSH connection without need for public IP with AWS Client VPN

Using an EC2 instance as a bastion host offers the advantage of not directly exposing our final instances to the internet. However, this solution also comes with some significant disadvantages. Firstly, additional costs are incurred to keep the instance running, and the responsibility of managing and maintaining it must be taken on, including applying security patches and performing recurring administrative tasks. Introducing an additional instance adds complexity to the environment and can complicate configuration and troubleshooting efforts. It's important to note that the Bastion Host becomes a single point of failure, and the only way to address this is by incorporating additional Bastion Hosts in other availability zones, further increasing the operational burden.

AWS Client VPN is a managed VPN service based on OpenVPN that allows secure access to your EC2 instances. By implementing this method, an encrypted tunnel is established between your computer and the VPC, providing visibility to the instances via a private IP. This eliminates the need to expose any server in a public subnet accessible from the Internet. As a service managed by AWS, the operational and administrative burden present in the Bastion Host scenario is eliminated.

Infraestructure diagram to show how to deploy an EC2 instance to be able to connect through a ClientVPN

AWS Client VPN offers three authentication methods for the user when connecting to the endpoint. For the sake of simplicity in this example, we will use certificate-based mutual authentication. These certificates are digital forms of identification issued by a certificate authority. The first step will be to create a certificate and key for the server and another certificate and key for the client.

When creating a Client VPN endpoint, we will specify the subnet to which we want to associate it. In this subnet (or subnets), Elastic Network Interfaces (ENIs) will be created and assigned a security group. For greater control, we will add an outbound rule to the security group that only allows outgoing traffic to the security group of the final instances we wish to establish a connection with. Additionally, we will associate a route table with the endpoint, specifying that traffic destined within the VPC's Classless Inter-Domain Routing (CIDR) is routed locally. We can also utilize available Authentication Rules to allow access to the entire VPC range through a single rule for all groups.

Once we have deployed and configured our VPN, we must download the Client VPN endpoint configuration file and distribute it among users so they can access it via an OpenVPN-compatible client.

With private connectivity to the VPC through the VPN in place, we can establish an SSH connection in a traditional manner using the key pair with which we deployed the EC2 instance, connecting to the instance's private IP address. If we are using EC2 Instance Connect compatible AMIs, we can forget about using long-lived SSH key pairs and connect to our instances through AWS CLI using ephemeral SSH key pairs, just as was the case with the Bastion Host scenario.

Infraestructure diagram to show how to deploy an EC2 instance to be able to connect through ClientVPN and EC2 Instance Connect

SSH connection without need for public IP with EC2 Instance Connect Endpoint

Deploying a VPN from our computer to the VPC for remote instance administration purposes can be an excessive approach. While we no longer have an instance acting as a Bastion Host, which would exempt us from its management, AWS Client VPN, as a managed service, still involves configuration, administration, and user authentication concerns, regardless of the chosen method from the three possible ones.

Recently, AWS has introduced a new feature called EC2 Instance Connect Endpoint (EIC Endpoint). Thanks to this functionality, we can now establish connections to our instances in private subnets without the need for an intermediary Bastion Host, a public IP, or even an Internet Gateway in our VPC.

Infraestructure diagram to show how to deploy an EC2 instance to be able to connect through a Instance Connect Endpoint

The EIC Endpoint is a TCP proxy that incorporates an authentication system, allowing connection authentication through IAM policies. Below, we provide an example of how to allow a user to connect to a specific EIC Endpoint, using only port 22 and setting a time limit for the connection to a particular instance.

{
  "Version": "2012-10-17",
  "Statement": [{
      "Sid": "EC2InstanceConnect",
      "Action": "ec2-instance-connect:OpenTunnel",
      "Effect": "Allow",
      "Resource": "arn:aws:ec2:[REGION]:[ACCOUNT-ID]:instance-connect-endpoint/[EIC-ENDPOINT-ID]",
      "Condition": {
        "NumericEquals": {
          "ec2-instance-connect:remotePort": "22"
        },
        "IpAddress": {
          "ec2-instance-connect:privateIpAddress": "[SUBNET-CIDR]"
        },
        "NumericLessThanEquals": {
          "ec2-instance-connect:maxTunnelDuration": "[TIME]"
        }
      }
    },
    {
      "Sid": "Describe",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceConnectEndpoints"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

We have two methods to establish a connection to the final instances. In the first method, we use AWS CLI to generate a secure WebSocket tunnel from our machine to the endpoint, using our IAM credentials. Once the tunnel is established, we simply SSH to the local address (127.0.0.1 or localhost) and connect as usual. In the second option, we use the AWS Management Console, which transparently sets up the tunnel for us and initiates an SSH connection in a browser based terminal.

When deploying an EIC Endpoint, we need to specify in which subnet(s) we want to implement it. In these subnets, a new ENI will be created, to which we must associate a security group. To enhance the security of the environment, we'll add an outbound rule allowing SSH traffic to the security group associated with the final instances with which we want to establish a connection.

On the side of our final instances, it will be necessary to add an inbound traffic rule on port 22 from the security group associated with the ENI. This way, we ensure proper access through the EIC Endpoint to our instances.

When establishing connections via EIC Endpoint using AWS CLI, we have two options. The first involves opening the tunnel and then establishing an SSH connection in the traditional way using long-lived keys. This option ensures compatibility with any Linux AMI we're using. To do this, we can use the following command:

ssh ec2-user@[INSTANCE] \
-i [SSH-KEY] \
-o ProxyCommand='aws ec2-instance-connect open-tunnel \
--instance-id %h'

On the other hand, if we're using EC2 Instance Connect-compatible AMIs, we can take advantage of the ability to connect via SSH using ephemeral keys. To do so, we can use the following command:

aws ec2-instance-connect ssh --instance-id [INSTANCE]

Agent based connection with Systems Manager Session Manager

What if I told you there's a much simpler way to access our remote instances that doesn't require exposing servers to the internet, doesn't require opening port 22, and doesn't use SSH, thus eliminating the need to worry about keys? All of this is possible thanks to AWS Systems Manager (SSM) Session Manager.

Infraestructure diagram to show how to deploy an EC2 instance to be able to connect using SSM Session Manager

SSM Session Manager operates through an agent, this agent is compatible with Linux, Mac, and Windows. Not only does it work for your EC2 instances in AWS, but you can also install it on virtual machines you have locally. The agent is responsible for establishing communication between the server and the Session Manager service. This communication flows through bidirectional tunnels over an HTTPS connection.

To initiate a remote session, you can use the AWS Management Console or the CLI. When you request a session, SSM generates a session token and establishes a secure connection. For the agent installed on the instance to communicate with the SSM API, you need to assign it permissions through a role and allow HTTPS connectivity to the following addresses:

ec2messages.[REGION].amazonaws.com
ssm.[REGION].amazonaws.com
ssmmessages.[REGION].amazonaws.com

If your instances are in a public subnet with an Internet Gateway linked to the VPC and an outbound rule on port 443 in the security group, they will be able to access these addresses. If you prefer to keep instances in private subnets, you have two options: implement a NAT Gateway to provide them internet access, or configure a VPC Endpoint in the VPC, giving them private connectivity to the SSM API.

All sessions initiated through Session Manager are recorded in CloudTrail. This means a detailed record is maintained of who accesses which instances, when connections are established, which commands are executed, and when sessions are closed. These CloudTrail logs provide a comprehensive view of activities, simplifying the detection of potential vulnerabilities and ensuring compliance with regulations.

If you want to initiate connections using the AWS CLI, it's essential to have the Session Manager plugin for AWS CLI installed on your local machine. Once installed, you can execute the following command:

aws ssm start-session --target [INSTANCE-ID]

Thanks for reading! Catch you in the next one.

Buy me a coffee