Jenkins on AWS EC2 (Master/Slave setup using EC2 plugin)

1) Create user ciadmin
useradd ciadmin
passwd ciadmin

2) Create a file system /apps/jenkins
chown -R ciadmin:ciadmin jenkins

3) Install JDK 1.8, Apache Tomcat 9

4) Download and Copy Jenkins.war to webapps directory

5) Update .bash_profile
PATH=$PATH:$HOME/.local/bin:$HOME/bin
JAVA_HOME=/apps/jenkins/jdk1.8.0_77
JAVA_OPTS="$JAVA_OPTS -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses"
PATH=$JAVA_HOME/bin:$PATH
export PATH JAVA_HOME JAVA_OPTS

6) Update /etc/hosts for the tomcat to start properly (example below)
172.x.y.x ip-172-x-y-z ip-172-x-y-z.us-west-2.compute.internal


7) Hit Jenkins URL: http://ip:8080/jenkins

8) Install Amazon EC2 plugin
Manage Jenkins -> Manage Plugins -> Available > Cluster Management and Distributed Build > Amazon EC2 plugin > Install

9) Create 2 AWS IAM roles

9.1) jenkins-master-role
Attach the custom policy

{
    "Version": "xxxxx",
    "Statement": [
        {
            "Sid": "Stmtxxx",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeSpotInstanceRequests",
                "ec2:CancelSpotInstanceRequests",
                "ec2:GetConsoleOutput",
                "ec2:RequestSpotInstances",
                "ec2:RunInstances",
                "ec2:StartInstances",
                "ec2:StopInstances",
                "ec2:TerminateInstances",
                "ec2:CreateTags",
                "ec2:DeleteTags",
                "ec2:DescribeInstances",
                "ec2:DescribeKeyPairs",
                "ec2:DescribeRegions",
                "ec2:DescribeImages",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Verify trust relationship

{
  "Version": "xxxxxx",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}


9.2) jenkins-slave-role

Verify trust relationship

{
  "Version": "xxxxxx",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

10) Setup EC2 Slave Config

Manage Jenkins > Configure System > Cloud section > Amazon Ec2

Name: aws-dev-vpc

Access Key: 

Secret Key: 

Use EC2 instance profile (Checked)

Region: us-west-2

EC2 Key Pair's Private Key: Place your private key

Test your AWS connection!!

AMIs

Description: celebrity-jenkins-slave

AMI ID: ami-xxxx

Instance Type: M4Large

EBS Optimized: Checked

Availability Zone: us-west-2a

Security group names: my-security-group

Remoe FS root: /mydir/subdir

Remote user: ciadmin

AMI Type: unix

Idle termination time: 60 

Note: Please note that slave will get terminated automatically after idle termination time with no activities kicked off by Jenkins

Stop/Disconnect on Idle Timeout: Checked

Note: If you do not want the slave to get terminated automatically but to stop, check the above.

Subnet ID for VPC: subnet-xxxx

Tags:
Name: project Value: myproject
Name: Name Value: myjenkins-slave

Instance Cap: 3
IAM Instance Profile: arn:aws:iam::MyAccountNo:instance-profile/cmyjenkins-slave-role

Connect by SSH Process: Checked

Save

11) Start a Slave
Manage Jenkins > Manage Nodes > provision via aws-de-vpc
Check the logs