Boosting CI/CD with GitHub Actions and EC2 Self-Hosted Runners

By Charan Sagili
4 mins Read

In the world of modern software development, CI/CD (Continuous Integration and Continuous Delivery) tops the list. It automates the build, test, and deployment process, ensuring frequent releases and improved code quality. GitHub Actions, a built-in CI/CD solution within your GitHub repositories, simplifies this process further. But what if you have specific requirements that extend beyond the capabilities of GitHub's hosted runners? This is where self-hosted runners on Amazon's EC2 service come into play.

What is a Self-Hosted Runner?
A self-hosted runner is a system you deploy and manage on your own infrastructure, typically an EC2 instance. It acts as an agent for GitHub Actions, executing workflow jobs triggered by events in your repository. Unlike GitHub's hosted runners, which provide standardized environments, self-hosted runners offer greater control and customization.

Why Use a Self-Hosted Runner?
While GitHub offers a variety of hosted runners catering to common build environments, self-hosted runners on EC2 can be advantageous in specific scenarios:

  • Customization: Need a specific software environment or tools unavailable on hosted runners? A self-hosted runner lets you install and configure the exact environment your project needs.
  • Performance: For resource-intensive tasks like building large codebases or running complex tests, a dedicated EC2 instance can provide superior performance compared to shared hosted runners.
  • Security: For highly sensitive projects, running the CI/CD pipeline in a controlled environment on your own EC2 instance offers an extra layer of security.

Security Considerations: A Top Priority
Security is paramount when dealing with CI/CD pipelines. Here are key points to remember when using self-hosted runners:

  • Least Privilege IAM User: The IAM user running the workflow should have the minimum permissions necessary for its tasks. Avoid granting broader access for security reasons.
  • Secure SSH Keys: Don't store SSH keys used for connecting to the EC2 instance in plain text. Utilize strong passphrases and consider managing them with tools like AWS Secrets Manager.
  • GitHub Secrets: Store sensitive information like AWS credentials, API keys, or passwords as GitHub secrets. This eliminates the need to hardcode them in your workflow scripts, enhancing security.
  • Regular Updates: Keep your EC2 instance and installed packages updated with the latest security patches to mitigate vulnerabilities.

Installation and Configuration on a Linux Server with Continuous Execution
Here's a step-by-step guide to installing and configuring a self-hosted runner on your Linux server (assuming Ubuntu/Debian-based distribution) to keep it constantly running and checking for new code:

Prerequisites:
  • An EC2 instance with a compatible Linux distribution (Ubuntu, Debian, etc.) and internet access.
  • An IAM user with least privilege access for the runner.
  • A generated GitHub Personal Access Token with repo permissions for the workflow.

Installation Steps:
  1. Establish SSH Connection - Securely connect to your EC2 instance using SSH with a strong passphrase or key pair. Avoid storing SSH keys in plain text.
  2. Update and Upgrade Packages - Ensure your system is up-to-date with the latest security patches: $ sudo yum update && sudo yum upgrade -y
  3. Create a folder - $ mkdir actions-runner && cd actions-runner
  4. Download the latest runner package - $ curl -o actions-runner-linux-x64-2.317.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.317.0/actions-runner-linux-x64-2.317.0.tar.gz
  5. Extract the installer - $ tar xzf ./actions-runner-linux-x64-2.317.0.tar.gz

Configuration:
If you prefer managing the runner as a service, you can use a service management tool like systemd after downloading and extracting the runner:
  1. Create the runner and start the configuration experience - $ ./config.sh --url https://github.com/your-github account/your-repo-name --token your-token
  2. Run the script - ./run.sh &
Note: Store your GitHub Personal Access Token securely. Do not hardcode or you can use GitHub Secrets.

Conclusion:
Imagine you're a developer working on a cutting-edge project. You need a robust CI/CD pipeline to ensure frequent updates and maintain high code quality. GitHub Actions offers a built-in solution, but your project requires a specific software environment and top-notch performance.

This is where a self-hosted runner on an EC2 instance comes in as your hero. It's like having your own dedicated workstation in the cloud, customized to your project's exact needs. No more limitations of shared environments!

Think of the self-hosted runner as a tireless builder, working away on your codebase in a secure and controlled space (your EC2 instance). It can handle complex builds and tests with ease, thanks to the dedicated resources of the EC2 instance.

But here's the catch: this tireless builder needs a nudge every now and then. By incorporating nohup, you equip it with the ability to keep checking for new code updates, even if your connection drops. It's like giving it a constant reminder to stay vigilant and ready to build!

In the end, using a self-hosted runner is like having a powerful and dedicated teammate on your development journey. It provides the customization, performance, and security you need to deliver high-quality code at lightning speed. Just remember to keep it informed (using nohup) so it can continuously build and test your project, ensuring a smooth and successful development process.