Monday, March 28, 2016

AWS Codedeploy

Using Codedeploy to publish files from S3 to EC2 server


Preparing a POC application to use with Codedeploy
  • Create index.html with some sample content
  • Create appspec.yml with the following content
----------------------------------------------------------
version: 0.0
os: linux 
files:
 - source: /index.html
   destination: /home/ec2-user/myapp/
----------------------------------------------------------
  • Please note all other unused config has been stripped down from original appsec.yml. It is important to remove all unused syntax. Otherwise, the application will not deploy properly using Codedeploy.
  • Zip just the files index.html and appspec.yml. Name the zip file as myapp-poc.zip
  • Please note when the zip is open, appspec.yml should be at the root level without any sub directories
Create S3 bucket and upload deploy artifacts
  • Create S3 bucket myapp ( Left the default Grantee myaccount with all permissions)
  • Upload myapp-poc.zip to S3 myapp S3 bucket ( Left the default Grantee myaccount with all permissions)

Create IAM policy and roles
  • Create a policy myapp-codedeploy-ec2-policy S3 bucket myapp ( Left the default Grantee myaccount with all permissions)
  • Upload myapp-poc.zip to S3 myapp S3 bucket ( Left the default Grantee myaccount with all permissions)
  • Contents of myapp-codedeploy-policy
----------------------------------------------------------
{
   "Version": "xxxx",
   "Statement": [
{
   "Sid": "Stmtxxx",
   "Effect": "Allow",
   "Action": [
"s3:Get*",
"s3:List*"
   ],
   "Resource": [
"arn:aws:s3:::myapp/*",
"arn:aws:s3:::aws-codedeploy-us-east-1/*",
"arn:aws:s3:::aws-codedeploy-us-west-2/*",
"arn:aws:s3:::aws-codedeploy-us-west-1/*"
   ]
}
   ]
}

----------------------------------------------------------
  • Create a role myapp-codedeploy-service-role 
  • Choose AWSCodeDeploy from AWS Service Roles
  • Attach the policy AWSCodeDeployRole (AWS managed) to myapp-codedeploy-service-role
  • Edit trust relationship of myapp-codedeploy-service-role to read as 
----------------------------------------------------------
{
 "Version": "xxxx",
 "Statement": [
   {
     "Sid": "",
     "Effect": "Allow",
     "Principal": {
"Service": [
 "codedeploy.us-west-2.amazonaws.com",
 "codedeploy.us-west-1.amazonaws.com",
 "codedeploy.us-east-1.amazonaws.com"
]
     },
     "Action": "sts:AssumeRole"
   }
 ]
}
----------------------------------------------------------
  • Create a role myapp-codedeploy-ec2-role
  • Choose Amazon EC2 from AWS Service Roles
  • Attach the policy myapp-codedeploy-ec2-policy (AWS managed) to myapp-codedeploy-ec2-role
  • Edit trust relationship of myapp-codedeploy-ec2-role to read as 
----------------------------------------------------------
{
 "Version": "xxxx",
 "Statement": [
   {
     "Effect": "Allow",
     "Principal": {
"Service": "ec2.amazonaws.com"
     },
     "Action": "sts:AssumeRole"
   }
 ]
}
----------------------------------------------------------

Create an EC2 instance
  • Spin up an EC2 instance with desired instance type.
  • In "Configure Instance Details" section, choose proper vpc, subnet. choose the IAM role as myapp-codedeploy-ec2-role
  • Please note that if you use Codedeploy wizard, it creates EC2 in the default vpc. If the default vpc does not exist, it fails. That is the reason behind spinning up a custom ec2 instance,
  • Create tags on the EC2 instance: Ex: Tag name: project Tag value: myapp
  • Please note tags are used by Codedeploy to discover instances.
Deploy Codedeploy agent on EC2 instance
    • Login into EC2 instance
    • Run the following commands
      • sudo yum update
      • sudo yum install ruby
      • sudo yum install wget
      • cd /home/ec2-user
      • wget https://bucket-name.s3.amazonaws.com/latest/install
      • chmod +x ./install
      • sudo ./install auto
      • sudo service codedeploy-agent status
    Create Codedeploy config
    • Create new application 
      • Application name: myapp
      • Deployment group name: myapp-deploy-stage
      • Tags: Amazon EC2: Key: project Value: myapp (The no. of instances discovered will be displayed)
      • Choose Service Role myapp-codedeploy-service-role
      • Leave rest of them as default
      • Create application
    • In the deployment group, select the deployment myapp-deploy-stage
      • Actions: Deploy new revision
      • Create New Deployment
      • Application: myapp
      • Deployment Group: myapp-deploy-stage
      • Revision Type: My application is stored in Amazon S3
      • Revision Location: 
    • Go to S3 console, select myapp-poc.zip. Copy the complete https url link and ETag url
      • Form the revision URL as follows as an example
      • https://s3-us-west-2.amazonaws.com/myapp/myapp-poc.zip?etag=962c02cb729b2f36745acbf4102129e1
    • Paste the above URL with ETag in the Revision Location field
    • Deploy

      Tuesday, March 22, 2016

      Enable forensic log in Apache within AWS Beanstalk

      1) Change (or Uncomment) the line in file /etc/httpd/conf/httpd.conf
      #LoadModule log_forensic_module modules/mod_log_forensic.so
      to
      LoadModule log_forensic_module modules/mod_log_forensic.so

      2) Update /etc/httpd/conf.d/elasticbeanstalk.conf to include the line
      ForensicLog /var/log/httpd/forensic_log 

      3) Update /etc/httpd/conf.d/elasticbeanstalk.conf to add %{forensic-id}n at the end
      LogFormat "%h (%{X-Forwarded-For}i) %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %{forensic-id}n"
      This step is optional and help with forensic id correlation with access log.

      4) Stop apache, check no http process is out there using ps-ef | grep http and start apache
      /usr/sbin/apachectl stop
      ps -ef | grep http (check for no http process)
      /usr/sbin/apachectl start

      Wednesday, March 9, 2016

      CodeCommit

      1) Create a userid and grant codecommitfullaccess policy for the poc purpose
      2) Create a repository named jrepo in AWS CodeCommit
      3) Copy the https url (example: https://git-codecommit.us-east-1.amazonaws.com/v1/repos/jrepo)
      4) Setup AWS CLI
      5) Install AWS Tools from https://aws.amazon.com/powershell/


      6) Run windows command utility as administrator
      7) cd %PROGRAMFILES(X86)%\AWS Tools\CodeCommit
      8) git-credential-AWSS4.exe -p jcodecommit
          Note: jcodecommit is profile name stored in AWS config or credentials file  (under users home .aws dir)
          Choose Yes to generate sig
      9) Run git config --global --edit and you should see a similar block
      [credential]
      helper = !'C:\\Users\\j\\AppData\\Roaming\\GitCredStore\\git-credential-AWSSV4.exe' --profile=jcodecommit
      UseHttpPath = true

      10) Create a local directory named codecommitrepos
      11) cd c:\codecommitrepos
      12) git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/jrepo local-jrepo
      13) git config --local user.name "developer1"
      14) git config --local user.email developer1@email.com

      15) cd c:\codecommitrepos\local-jrepo
      15)  Create files index.html and index2.html
      16) git add index.html
      17) git commit -m "Added index.html'
      16) git add index2.html
      17) git commit -m "Added index2.html'

      18) git push -u origin master

      Voila! The files are now pushed to AWS CodeCommit.