Microservice Deployment Software Engineering II Sharif University - - PowerPoint PPT Presentation

Microservice Deployment

Software Engineering II Sharif University of Technology MohammadAmin Fazli

Deployment

Topics

 Continuous Integration & Microservices  Continuous Delivery  Deployment Artifacts  Custom Images  Environments  Service Configuration  Service-to-Host Mapping  Physical to Virtual  Reading:

 Building Microservices-Sam Newman-Chapter

VI

2

Deployment

Continuous Integration

 With CI, the core goal is to keep everyone in sync with each

• ther, which we achieve by making sure that newly checked-in

code properly integrates with existing code.

 A CI server detects that

 The code has been committed  Checks it out  Carries out some verification

 Making sure the code compiles  Making sure tests pass

 As part of this process, we often create artifact(s) that are

used for further validation, such as deploying a running service to run tests against it.

3

Deployment

Continuous Integration

 Benefits

 Fast Feedback about code quality  Automatic binary artifacts’ creation  All the code is version controlled and all artifacts can be recreated  Traceability from deployed artifacts to the code  Traceability over what tests were run over codes and artifacts

 CI checks: Jez Humble’s three questions he asks people to test

if they really understand what CI is about

 Do you check in to mainline once per day?

 You need to make sure your code integrates.

 Do you have a suite of tests to validate your changes?

 Check if our code semantically integrates

 When the build is broken, is it the #1 priority of the team to fix it?

 Please avoid piling up non-integrated changes 4

Deployment

CI & Microservices

 When thinking about microservices & CI, we need to think

about how our CI builds map to individual microservices.

 1st solution: One repository+Monolithic build

5

Deployment

CI & Microservices

 1st solution:

 Benefits:

 Fewer repositories to worry about  Simpler build

 Downsides:

 A one-line change to a single service cause all the other services get verified

and built and can be very time consuming

 If this one-line change to a service breaks the build, no other changes can be

made to the other services until that break is fixed.

 If a change cause break, which team is in charge? 6

Deployment

CI & Microservices

 2nd Solution: One code repository + Individual Build

 Check in/Check out process can be simpler as we have one

repository to worry about

 Can be easily get into the habit of checking in source code for

multiple services at once which can slip into making changes that couple services together.

7

Deployment

CI & Microservices

 3rd Solution: Individual Repository + Individual Build

 When making a change, we run only the build and tests I need to.  We get a single artifact to deploy.  Alignment to team ownership is more clear.  If you own the service, you own the repository and the build.  Making changes across repositories can be more difficult in this

world

8

Deployment

Continuous Delivery

 The build pipeline concept gives us a nice way of tracking the

progress of our software as it clears each stage, helping give us insight into the quality of our software.

 Ex: Running fast and slow tests together does not get us a fast

• feedback. If fast tests fail waiting for slow tests is useless. So it is

better to run them in different stages.

 CD builds according to the pipeline concept.

 Continuous delivery is the approach whereby we get constant

feedback on the production readiness of each and every check-in, and furthermore treat each and every check-in as a release candidate.

9

Deployment

Continuous Delivery

 In CD we must model the process of getting our software

from check-in to production and know where any given version of the software is.

 Different software stages must be modeled both manual or

automated

 We really want a tool that embraces CD as a first-class

concept.

 Tools that fully support CD allow you to define and visualize these

pipelines, modeling the entire path to production for your software.

 By modeling the entire path to production for our software, we

greatly improve visibility of the quality of our software, and can also greatly reduce the time taken between releases

10 10

Deployment

Platform Specific Artifacts

 Most technology stacks have some sort of first-class artifact, along

with tools to support creating and installing them.

 Rubby gems, Java Jars, Python eggs

 Depending on the technology stack, these artifacts may not be

enough by themselves.

 Example: sometimes they need Apache/Nginx installation and configuration  We may need some way of installing and configuring other software that we

need in order to deploy and launch our artifacts.

 This is where automated configuration management tools like Puppet and

Chef can help.

 These artifacts are specific to a certain technology stack. This may

make deployment more difficult when we have a mix of technologies in play

 Automation can go a long way toward hiding the differences in the

deployment mechanisms of the underlying artifacts.

11 11

Deployment

Operating System Artifacts

 One way to avoid the problems associated with technology-specific

artifacts is to create artifacts that are native to the underlying

• perating system.

 Rpm packages for CentOS, MSI for windows, deb for Ubuntu

 Advantages of using OS-specific artifacts:

 From a deployment point of view we don’t care what the underlying

technology is. We just use the tools native to the OS to install the package.

 The OS tools can also help us uninstall and get information about the

packages too,

 The OS tools may even provide package repositories that our CI tools can

push to

 Downsides:

 Difficulty in creating the packages  There is a great overhead when deploying on different OSes 12 12

Deployment

Custom Images

 Preparing machines for a service is very time consuming even

if automated configuration tools (Puppet, Chef, Ansible) are used.

 Installing a JVM needs some minutes

 Automatic tools can be smart and will avoid installing software

that is already present. This does not mean that running the scripts on existing machines will always be fast, unfortunately, as running all the checks takes time.

 Moreover, we don’t want to allow for too much configuration

• drift. So we should have many machine preparations and it is a

real drag.

 Configuration Drift is the phenomenon where running servers in an

infrastructure become more and more different as time goes on, due to manual ad-hoc changes and updates, and general entropy.

13 13

Deployment

Custom Images

 One approach to reducing this spin-up

time is to create a virtual machine image that bakes in some of the common dependencies we use.

 When we want to deploy our software,

we spin up an instance of this custom image, and all we have to do is install the latest version of our service.

 Because you build the image only once,

when you subsequently launch copies of this image you don’t need to spend time installing your dependencies, as they are already there.

14 14

Deployment

Custom Images

 Drawbacks:

 Building images can take a long time. This means that for developers

you may want to support other ways of deploying services to ensure they don’t have to wait half an hour just to create a binary deployment.

 Some of the resulting images can be large. This could be a real

problem if you’re creating your own images.

 Moving a huge image around a network is a real problem

 The tool chain required to build such an image varied from platform

to platform.

 Building a

VMWare image is different from building an AWS AMI, a Vagrant image, or a Rackspace image.

 Some of the tools are multi-platform like packer.io which has support for

VMWare, AWS, Racksapce Cloud, Digital Ocean, Vagrant and …

15 15

Deployment

Immutable Servers

 By storing all our configuration in source control, we are

trying to ensure that we can automatically reproduce services and hopefully entire environments at will.

 But once we run our deployment process, what happens if

someone comes along changes things independently of what is in source control?

 Configuration Drift-the code in source control no longer reflects the

configuration of the running host.

 Immutable Servers Pattern: we can ensure that no changes are

ever made to a running server. Instead, any change, no matter how small, has to go through a build pipeline in order to create a new machine.

 We can disable SSH

16 16

Deployment

Environments

 Different environments in CD stages:

 Development Environments and Unit Testing  One for Slower Tests  One for UAT  One for Performance Testing  One for Production

17 17

Deployment

Environments

 Our microservice should be the same throughout, but

the environment will be different.

 Separate distinct collection of configuration and hosts  Differences can be more than just some configurations

 Ex. our production environment for our service might consist of multiple

load-balanced hosts spread across two data centers, whereas our test environment might just have everything running on a single host  As you move from development environment to build server

to UAT environment all the way to production, you’ll want to ensure that your environments are more and more production-like to catch any problems associated with these environmental differences sooner.

18 18

Deployment

Service Configuration

 Our services need some configuration

 Ex. Username and password for database connection in different

environments

 Configuration that changes from one environment to another

should be kept to an absolute minimum.

 The more your configuration changes fundamental service

behavior, and the more that configuration varies from one environment to another, the more you will find problems only in certain environments, which is painful in the extreme.

19 19

Deployment

Service Configuration

 Handling service configurations in CD

 Building one artifact per environment, with configuration inside the

artifact itself.

 Let’s imagine I build a Customer-Service-Test and Customer-Service-Prod

• artifacts. If my Customer-Service-Test artifact passes the tests, but it’s the

Customer-Service-Prod artifact that I actually deploy, can I be sure that I have verified the software that actually ends up in production?

 There is the additional time taken to build these artifacts.  We need to know at build time what environments exist.  Problems with sensitive data: I don’t want information about production

passwords checked in with my source code, but if it is needed at build time to create all those artifacts, this is often difficult to avoid

 Creating one single artifact, and manage configuration separately

 This could be a properties file that exists for each environment, or different

parameters passed in to an install process.

 Using a dedicated system for providing configuration

 Useful when dealing with a large number of microservices 20 20

Deployment

Service-to-Host Mapping

 How many services per machine?  Solutions

 Multiple services per host  Application containers  Single service per host  Platform as a service

 Host: A generic term which defines an operating system onto

which we can install and run services

 They can be physical or virtual machines

21 21

Deployment

Multiple Services per Host

 Benefits

 It is simpler. If more services are packed on to a single host, the host

management workload doesn’t increase as the number of services increases.

 It has less cost. Even if you have access to a virtualization platform

that allows you to provision and resize virtual hosts, the virtualization can add an overhead that reduces the underlying resources available to your services.

 This model is also familiar to those who deploy into some form of an

application container.

22 22

Deployment

Multiple Services per Host

 Challenges

 It can make monitoring more difficult.

 What can we do to trace CPU for one service?

 If one service is under significant load, it can end up reducing the

resources available to other parts of the system.

 Deployment of services can be somewhat more complex too, as

ensuring one deployment doesn’t affect another leads to additional headaches.

 If we use Puppet to prepare a host, but each service has different (and

potentially contradictory) dependencies

 This model can also inhibit autonomy of teams.  Efforts to target scaling to the service most in need of it can be

complicated.

 This model can limit our deployment artifact options.

23 23

Deployment

Application Containers

 The idea is that the application container your services live in

gives you benefits in terms of improved manageability, such as clustering support to handle grouping multiple instances together, monitoring tools, and the like.

 Like IIS for .NET applications and Servlet container for Java

applications

24 24

Deployment

Application Containers

 Downsides

 They inevitably constrain technology choice.

 They also force the implementation technology

 Some of them have ability to manage clusters to support shared in-

memory session state, which is in contradiction with scalability

 Their monitoring abilities are not sufficient  Attempting to do proper lifecycle management of applications on top

• f platforms like the JVM can be problematic, and more complex

than simply restarting a JVM.

 Analyzing resource use and threads is also much more complex, as

you have multiple applications sharing the same process.

 They add resource overheads

25 25

Deployment

Single Service per Host

 Benefits:

 Simpler monitoring  Simpler remediation  Reduced single points of failure  Easy Scaling one service independent from others  This opens up the potential to use alternative deployment techniques

such as image-based deployments or the immutable server pattern

 Downside

 We have more servers to manage  It has more cost

26 26

Deployment

Platform as a Service

 Using a platform as a service (PaaS), you are working at a higher-

level abstraction than at a single host.

 Taking technology specific artifacts like java WAR files and Ruby gems and

automatically provision and run them.

 Some of them handle scaling the system up and down 27 27

Deployment

Automation

 One of the pushbacks against the single-service-per-host setup

is the perception that the amount of overhead to manage these hosts will increase.

 It is true if everything is done manually

 2X servers, 2X work

 Many things can be automated

 Automation enables developers for self-service-provision

 Ideally, developers should have access to exactly the same tool chain

as is used for deployment of our production services so as to ensure that we can spot problems early on.

28 28

Deployment

Move from Physical to Virtual

 One of the key tools available to us in managing a large

number of hosts is finding ways of chunking up existing physical machines into smaller parts.

 Traditional Virtualization  Vagrant  Linux Containers  Docker

29 29

Deployment

Traditional Virtualization

 Virtualization allows us to slice up a physical

server into separate hosts, each of which can run different things.

 Type 2 virtualization: A hypervisor run on the

OS whose jobs are:

 It maps resources like CPU and memory from the

virtual host to the physical host.

 It acts as a control layer, allowing us to manipulate

the virtual machines themselves.

 Technologies like AWS, VMWare,

VSPhere, Xen, KVM and …

 In type 1, VMs run directly on hardware.  The problem is that the hypervisor here needs

to set aside resources to do its job.

30 30

Deployment

Vagrant

 Vagrant is a very useful deployment platform, which is normally used

for dev and test rather than production.

 Vagrant provides us with a virtual cloud on your laptop.  It allows us to define a set of VMs in a text file, along with how the

VMs are networked together and which images the VMs should be based on.

 This text file can be checked in and shared between team members.

 We can spin up multiple VMs at a time, shut individual ones to test

failure modes, and have the VMs mapped through to local directories so you can make changes and see them reflected immediately.

 Downside:

 Running lots of VMs can tax the average development machine. If we have

• ne service to one VM, you may not be able to bring up your entire system
• n your local machine.

31 31

Deployment

Linux Containers

 In Linux processes can spawn other processes

 The Linux kernel’s job is to manage the tree of processes

 For Linux users, there is an alternative to virtualization. Rather

than having a hypervisor to segment and control separate virtual hosts, Linux containers instead create a separate process space in which other processes live.

 Many forms of this technology

 Solaris Zones  OpenVZ  LXC is the most popular

 Available on most modern Linux

distributions

32 32

Deployment

Linux Containers

 We don’t need a hypervisor.

 More available resources

 Although each container can run its own

• perating system distribution, it has to

share the same kernel

 Because the kernel is where the process

tree lives

 Ex. Our host operating system could run

Ubuntu, and our containers CentOS, as long as they could both share the same kernel.

 Linux containers are much faster to

provision than full-fat virtual machines.

 Due to the lighter-weight nature of

containers, we can have many more of them running on the same hardware than would be possible with VMs.

33 33

Deployment

Docker

 Docker is a platform built on top of lightweight containers. It

handles much of the work around handling containers for you.

 In Docker, you create and deploy apps, which are synonymous with images in

the VM world, albeit for a container-based platform.

 Docker manages the container provisioning, handles some of the networking

problems for you, and even provides its own registry concept that allows you to store and version Docker applications.

 The Docker app abstraction is a useful one for us, because just as

with VM images the underlying technology used to implement the service is hidden from us.

 Rather than using Vagrant to host multiple independent

VMs, each one containing its own service, we can host a single VM in Vagrant that runs a Docker instance.

 We then use

Vagrant to set up and tear down the Docker platform itself, and use Docker for fast provisioning of individual services.

34 34

Deployment

Docker

 Docker is a simple PaaS for a single machine  Different technologies for Docker:

 CoreOS: A stripped-down Linux OS that provides only the essential

services to allow Docker to run. This means it consumes fewer resources than other operating systems, making it possible to dedicate even more resources of the underlying machine to our containers.

 Kubernetes & CoreOS Cluster Technology: Help to manage services

across multiple Docker instances

 Deis: Heroku like PaaS on top of Docker  Container as a Service (CaaS): Docker with an appropriate

scheduling layer sits between IaaS and PaaS solutions

35 35

Deployment

Deployment Interface

 Ex: we’re developing locally and want to deploy our catalog service

into our local environment.

 Ex: Once I’ve checked in, our CI build service picks up the change

and creates a new build artifact, giving it the build number b456.

 Ex: our QA wants to pull the latest version of the catalog service

into an integrated test environment to do some exploratory testing, and to help with a showcase.

36 36

Deployment

Environment Definition

 Ex: An

environment definition file

37 37

Deployment

Environment Defintion

 1: We varied the size of the instances we used to be more cost

effective. You don’t need a 16-core box with 64GB of RAM for exploratory testing!

 2: Being able to specify different credentials for different

environments is key. Credentials for sensitive environments were stored in different source code repos that only select people would have access to.

 3:We decided that by default if a service had more than one

node configured, we would automatically create a load balancer for it.

38 38

Deployment

Environment Definitions

39 39

 1: This was the name of the Puppet file to run.