Continuous deployment with Jenkins and Salt LinuxCon CloudOpen - PowerPoint PPT Presentation

Continuous deployment with Jenkins and Salt LinuxCon CloudOpen ContainerCon North America 2015, Seattle Anirban Saha

About me Techie… Traveller… Thinker… Author of ‘Salt Cookbook’

Deployment Should be… Simple • Stable • Fast (good to have…) • Reliable • One click •

Problems faced Code distribution • Additional tasks (commands, migrations) • Latency • Parallel execution • Batch deployments • Ensuring uptime •

Methods available Deployment Configuration management tools : • Puppet, Chef, Ansible, Salt Remote execution tools: • SSH, Parallel SSH, Rundeck Packages: Code packaged as RPM or DEB files •

Code storage • GIT server (Github, Gitlab, Bitbucket, etc.) • Object storage (Amazon S3, Swift, etc.) • Package repositories (YUM/Debian) • Configuration management repositories • File servers managed manually

Problems Deployment: Remote execution tools : • Time consuming (over SSH) • Need to maintain inventory of hosts • Code storage: GIT repositories: • Usually centrally located, latency when deployed from GIT • Package repositories (RPM/DEB): • Problems in simultaneously updating repo metadata • Rollback: Version manipulation is a problem with most configuration • management and remote execution tools

Salt Provides… Agent based communication • Fast execution • Orchestration • Batch execution • Strong Integration with providers and • services

Salt features Targeting deployment nodes : • By hostname • ‘ myappweb *’ • By specific grains • - G ‘ node_type:webserver ’ • By nodegroups • -N webgroup

Salt features Task dependencies : Salt requisites to make tasks dependent on each other • get_archive: module.run: . . deploy_app: module.run: . . - require: - module: get_archive

Salt features Orchestration : webgrp2_deploy: salt.state: - tgt: ' webgrp2' - tgt_type: nodegroup - sls: - webserver.deploy - require: - salt: webgrp1_deploy webgrp1_deploy: salt.state: - tgt: 'webgrp1' - tgt_type: nodegroup - sls: - webserver.deploy

Salt features Batch Execution : • By number of hosts • --batch-size 5 • Executes on 5 hosts at a time • By percentage of hosts • --batch-size 25% • Executes on 25% of total target hosts at a time

Salt features Salt API: # curl -H "Accept: application/json" -d secretkey= "mysupersecretkey" -k https://10.0.0.2:8080/hook/deploy {"success": true}

Tools To be used… GIT for code repository • Jenkins for CI • Ant for build and tasks • Amazon S3 for code archive storage • Salt for deployment • Shell scripts (good old bash) for • additional tasks

Salt deployment methods • Salt hosted on the Jenkins server. Jenkins calls Salt binary • Salt hosted independently. Jenkins calls Salt via SSH • Jenkins calls Salt via Salt API

Objective

Steps • salt-cloud used to launch instances • Post install actions used to synchronize EC2 grains • Salt reactor used to run states on new instances , perform deployment and register with load balancer • Jenkins build job used to build new code, create tags and push new deployment ready archive to Amazon S3 • Jenkins deploy job used to fetch new code version and deploy code on target servers

Steps explained Salt cloud is used to spawn instances • Following parameter is provided in the profile to push custom EC2 grains, • sync_after_install: grains • The node is registered and deregistered from the load balancer with the • following module definition, register: module.run: - name: boto_elb.register_instances - m_name: mywebapp - instances: - {{ grains['ec2']['instance_id'] }} Here ['ec2']['instance_id'] is one of the custom grains pushed to the node •

Steps explained Fetching archive: Salt s3 execution module used to get code archive from Amazon S3 fetch_app_archive: module.run: - name: s3.get - bucket: mywebapp-us - path: mywebapp-{{ app_version }}.tar - local_file: /tmp/mywebapp-{{ app_version }}.tar

Steps explained Setting App version grain: After every deploy, a grain is set to record the app version deployed for tracking, app_version: grains.present: - value: {{ app_version }} - require: - cmd: deploy_app

Steps explained Deployment based on App version: At every deploy, it is checked if the version to be deployed is already on the node using the app_version grain, {% if grains['app_version'] != app_version %} deregister: fetch_app_archive: deploy_app: register: {% endif %}

Lets do it then !!!

Demo repository Get the demo repository at the following location, https://github.com/rosesnthornz/ cloudopen-na-2015

Questions ?

Contact Email : sahaanirban1988@gmail.com Twitter : @rosesnthornz

Thank You !!!