Solve the paradox Less Downtime More Security LinuxCon Berlin, - - PowerPoint PPT Presentation

▶
solve the paradox
SMART_READER_LITE
LIVE PREVIEW

Solve the paradox Less Downtime More Security LinuxCon Berlin, - - PowerPoint PPT Presentation

Sep 21, 2023 214 likes •999 views

Solve the paradox Less Downtime More Security LinuxCon Berlin, Germany October 4, 12:10 13:00 Hannes Khnemund SUSE Product Management Downtime Considerations for your digital architecture Take a holistic approach - End-users

slide-1
SLIDE 1

Solve the paradox Less Downtime – More Security

LinuxCon Berlin, Germany October 4, 12:10 – 13:00

Hannes Kühnemund SUSE Product Management

slide-2
SLIDE 2

Downtime

Considerations for your digital architecture Take a holistic approach …

  • End-users (Business) are interested service availability
  • Application, OS, Cluster, VM, Server, Network, Storage, People, Processes...

... because we understand that components will fail, ...

  • Failure tolerant architecture, identify weak links

... acceptance of any downtime is decreasing and it is critical to ...

  • Seek to reduce both planned and unplanned service downtime

... strike a balance.

  • Cost of IT continuity vs. business impact

2

slide-3
SLIDE 3

Downtime Quiz

Regular cadence

  • monthly
  • quarterly
  • yearly

On the weekend In alignment with all stakeholders Combination of Taks

  • software updates / configuration
  • hardware exchange of defect parts
  • datacenter maintenance / AC

Optimizable with

  • SUSE Manager

3

planned

slide-4
SLIDE 4

Downtime Quiz

Regular cadence

  • monthly
  • quarterly
  • yearly

On the weekend In alignment with all stakeholders Combination of Taks

  • software updates / configuration
  • hardware exchange of defect parts
  • datacenter maintenance / AC

Optimizable with

  • SUSE Manager

4

planned unplanned

slide-5
SLIDE 5

Downtime Quiz

Regular cadence

  • monthly
  • quarterly
  • yearly

On the weekend In alignment with all stakeholders Combination of Taks

  • software updates / configuration
  • hardware exchange of defect parts
  • datacenter maintenance / AC

Optimizable with

  • SUSE Manager

5

planned No cadence unplanned

slide-6
SLIDE 6

Downtime Quiz

Regular cadence

  • monthly
  • quarterly
  • yearly

On the weekend In alignment with all stakeholders Combination of Taks

  • software updates / configuration
  • hardware exchange of defect parts
  • datacenter maintenance / AC

Optimizable with

  • SUSE Manager

6

planned No cadence Usually on Christmas Day unplanned

slide-7
SLIDE 7

Downtime Quiz

Regular cadence

  • monthly
  • quarterly
  • yearly

On the weekend In alignment with all stakeholders Combination of Taks

  • software updates / configuration
  • hardware exchange of defect parts
  • datacenter maintenance / AC

Optimizable with

  • SUSE Manager

7

planned No cadence Usually on Christmas Day No alignment with stakeholders unplanned

slide-8
SLIDE 8

Downtime Quiz

Regular cadence

  • monthly
  • quarterly
  • yearly

On the weekend In alignment with all stakeholders Combination of Taks

  • software updates / configuration
  • hardware exchange of defect parts
  • datacenter maintenance / AC

Optimizable with

  • SUSE Manager

8

planned No cadence Usually on Christmas Day No alignment with stakeholders Only one particular problem fixed unplanned

slide-9
SLIDE 9

Downtime Quiz

Regular cadence

  • monthly
  • quarterly
  • yearly

On the weekend In alignment with all stakeholders Combination of Taks

  • software updates / configuration
  • hardware exchange of defect parts
  • datacenter maintenance / AC

Optimizable with

  • SUSE Manager

9

planned No cadence Usually on Christmas Day No alignment with stakeholders Only one particular problem fixed Optimizable with

  • Various technologies available

unplanned

slide-10
SLIDE 10

Minimize Unplanned Downtime

10

Load Balancer

RAID

Virtualization

UPS RAS

System Rollback

High Availability and GEO

Live Patching

slide-11
SLIDE 11

Strike the balance?

11

slide-12
SLIDE 12

Strike the balance?

12

No Downtime Security

slide-13
SLIDE 13

13

But what about the non-disclosed

  • nes?

Since 2005, more than 75 data breaches in which 1,000,000 or more records were compromised have been publicly disclosed.

slide-14
SLIDE 14

Vulnerabilities

14

Year # vulnerabilities 2010 4258 2011 3532 2012 4347 2013 4794 2014 7038 2015 8822

2000 4000 6000 8000 10000

2010 2011 2012 2013 2014 2015

38% 16% 18% 28%

Vulnerability type 2015

Operating System Browsers Mobile Devices Applications

Rank Operating System # vulnerabilities 2015 1 Apple OS X 384 2 Microsoft Windows Server 2012 155 3 Canonical Ubuntu Linux 152 4 Microsoft Windows 8.1 151 ... 11 The Linux Kernel 77

Source: [http://www.cvedetails.com] & [https://nvd.nist.gov/] & [http://www.gfi.com/blog/2015s-mvps-the-most-vulnerable-players/]

slide-15
SLIDE 15

15

In a data center, not so long ago …

slide-16
SLIDE 16

In a data center, not so long ago …

16

Linux Kernel Nov-11, 2015

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-17
SLIDE 17

In a data center, not so long ago …

17

Linux Kernel Nov-11, 2015

 CVE-2015-6937  CVE-2015-7872  CVE-2015-7990

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-18
SLIDE 18

In a data center, not so long ago …

18

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

Linux Kernel Nov-11, 2015

 CVE-2015-6937  CVE-2015-7872  CVE-2015-7990

CVE: Common Vulnerabilities and Exposures It is a standard naming scheme used by the NVD NVD: National Vulnerability Database (https://nvd.nist.gov/)

slide-19
SLIDE 19

In a data center, not so long ago …

19

Linux Kernel Nov-11, 2015

 CVE-2015-6937  CVE-2015-7872  CVE-2015-7990

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-20
SLIDE 20

In a data center, not so long ago …

20

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015

 CVE-2015-6937  CVE-2015-7872  CVE-2015-7990

Reboot December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-21
SLIDE 21

In a data center, not so long ago …

21

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015

 CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2016-0728  CVE-2016-0728

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-22
SLIDE 22

In a data center, not so long ago …

22

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016

 CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2016-0728  CVE-2016-0728

Reboot December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-23
SLIDE 23

In a data center, not so long ago …

23

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2016-0728  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2016-0728  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-24
SLIDE 24

In a data center, not so long ago …

24

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2016-0728  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2016-0728  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660

Reboot December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-25
SLIDE 25

In a data center, not so long ago …

25

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-2384  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-2384  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-2384  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-2384

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-26
SLIDE 26

In a data center, not so long ago …

26

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-2384  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-2384  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-2384  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-2384

Reboot December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-27
SLIDE 27

In a data center, not so long ago …

27

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-1583  CVE-2016-3134

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-28
SLIDE 28

In a data center, not so long ago …

28

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-1583  CVE-2016-3134

Reboot December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-29
SLIDE 29

In a data center, not so long ago …

29

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-4997  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-4997  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-4997  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-4997  CVE-2016-1583  CVE-2016-3134  CVE-2016-4997  CVE-2016-4997

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-30
SLIDE 30

In a data center, not so long ago …

30

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-4997  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-4997  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-4997  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384  CVE-2016-3134  CVE-2016-4997  CVE-2016-1583  CVE-2016-3134  CVE-2016-4997  CVE-2016-4997

Reboot December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-31
SLIDE 31

In a data center, not so long ago …

31

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-0758  CVE-2016-1583  CVE-2016-2053  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-32
SLIDE 32

In a data center, not so long ago …

32

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016 Linux Kernel Sep-12, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-0758  CVE-2016-1583  CVE-2016-2053  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829

Reboot December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-33
SLIDE 33

In a data center, not so long ago …

33

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016 Linux Kernel Sep-12, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2016-0758  CVE-2016-1583  CVE-2016-2053  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2016-6480  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829  CVE-2016-6480

Sample data taken

  • n Sept-15, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-34
SLIDE 34

That reminds me of ...

34

slide-35
SLIDE 35

CVEs...? So what...?

  • CVE-2016-0728

‒ gain privileges or cause a denial of service

  • CVE-2015-8660

‒ local users can bypass intended access restrictions

  • CVE-2015-8539

‒ gain privileges or cause a denial of service

  • CVE-2015-7990

‒ allows local users to cause a denial of service

  • CVE-2015-7872

‒ local users can cause a denial of service (OOPS)

  • CVE-2015-6937

‒ local users can cause a denial of service (NULL pointer dereference and system crash)

  • CVE-2013-7446

‒ local users to bypass intended AF_UNIX socket permissions or cause a denial of service (panic)

  • ...

35

slide-36
SLIDE 36

36

Can’t we patch software while it runs? Mankind already flew to the moon …

slide-37
SLIDE 37

Dynamic Software Updates

Trinity Test 1945 (Manhattan Project)

  • IBM punch card automatic calculators

were used to crunch the numbers

  • A month before the Trinity nuclear

device test, the question was: “What will the yield be, how much energy will be released?”

  • The calculation would normally take

three months to complete – recalculating any batches with errors

  • Multiple colored punch cards introduced

to fix errors in calculations while the calculator was running

37

slide-38
SLIDE 38

kpatch

Modern history of kGraft and other DSU technologies

  • DSU: Dynamic Software Updates
  • the goal is to be able to fix bugs and add features either by
  • changing some functions or
  • replacing the whole program
  • kGraft developed as Open Source project by SUSE Labs
  • Upstream project „klp“
  • Takes best of both kGraft (SUSE) and kpatch (Red Hat)
  • Still in catch up w.r.t. to features required by enterprises

38

1990 2015 2000 1995 2005 2010 PoDUS Gupta Erlang Ginseng UpStare Ksplice Kitsune kGraft klp

slide-39
SLIDE 39

ftrace: return address modification mechanism

39

slide-40
SLIDE 40

Common Pitfalls

  • Function Inlining

→ DWARF to the rescue

  • Static Symbols

→ kernel keeps list: kallsyms

  • IPA-SRA (optimization like -O2)

→ using gcc optimization log

  • Multiple functions / dependencies

→ consistency model

  • Eternal sleepers (getty console 10)

→ send fake signal SIGKGRAFT / ignore

  • State transformation (req. for complex fixes)

→ not in kGraft right now

  • 3rd party kernel modules

→ depends on what the module does ...

40

slide-41
SLIDE 41

Consistency

Requirement: ensure system consistency when deploying live patches

41

Freezing the system (kpatch, ksplice) Lazy migration (kGraft)

slide-42
SLIDE 42

Consistency

Requirement: ensure system consistency when deploying live patches

42

Freezing the system (kpatch, ksplice) Lazy migration (kGraft)

stop_kernel(); check all stacks, whether any thread is stopped within a patched function If yes, resume kernel and try again later If not, flip the switch on all functions and resume the kernel

slide-43
SLIDE 43

Consistency

Requirement: ensure system consistency when deploying live patches

43

Freezing the system (kpatch, ksplice) Lazy migration (kGraft)

stop_kernel(); check all stacks, whether any thread is stopped within a patched function If yes, resume kernel and try again later If not, flip the switch on all functions and resume the kernel For each thread separately: Present the old version of functions to the thread until it leaves the kernel then give it the updated version Wake sleeping threads up by a special signal Prevent the signal from reaching userspace Once all threads have exited the kernel at least once we're DONE

slide-44
SLIDE 44

Consistency

Requirement: ensure system consistency when deploying live patches

44

Freezing the system (kpatch, ksplice) Lazy migration (kGraft)

stop_kernel(); check all stacks, whether any thread is stopped within a patched function If yes, resume kernel and try again later If not, flip the switch on all functions and resume the kernel For each thread separately: Present the old version of functions to the thread until it leaves the kernel then give it the updated version Wake sleeping threads up by a special

  • signal. Prevent the signal from reaching

userspace Once all threads have exited the kernel at least once we're DONE

Do you have better ideas than those two? Join SUSE as Live Patching developer https://jobs.suse.com/job/prague/live-patching-developer/3486/2529381

slide-45
SLIDE 45

Consistency model for KLP?

The chosen model is a merge of kpatch and kGraft

  • Combines stack checking and per-thread changes
  • Non-intrusive, fast finishing
  • Works well already but requires both:

45

Reliable stack unwinder (needed by kpatch)

  • Worked on by Josh Poimboeuf @ Red Hat
  • Currently needs FRAME POINTER
  • up 10% slowdown of kernel execution
  • Could use DWARF
  • complex, being developed by SUSE
  • speed is a concern
  • initial implementation removed from

upstream → Takes time Kernel thread model cleanup (needed by kGraft)

  • Worked on by Petr Mladek @ SUSE
  • Touches both kthreads and workqueues
  • These parts are the critical core
  • Needs a lot of good planning and review

→ Takes time

slide-46
SLIDE 46

Live Patching on ppc64le?

46

[ http://mpe.github.io/posts/2016/05/23/kernel-live-patching-for-ppc64le/ ]

slide-47
SLIDE 47

47

In a SUSE data center, today ;-)

slide-48
SLIDE 48

In a SUSE data center, today ;-)

48

Linux Kernel Nov-11, 2015

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-49
SLIDE 49

In a SUSE data center, today ;-)

49

Linux Kernel Nov-11, 2015

 CVE-2015-6937  CVE-2015-7872  CVE-2015-7990

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-50
SLIDE 50

In a SUSE data center, today ;-)

50

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-51
SLIDE 51

In a SUSE data center, today ;-)

51

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015

 CVE-2016-0728  CVE-2016-0728

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-52
SLIDE 52

In a SUSE data center, today ;-)

52

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-53
SLIDE 53

In a SUSE data center, today ;-)

53

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016

 CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-54
SLIDE 54

In a SUSE data center, today ;-)

54

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-55
SLIDE 55

In a SUSE data center, today ;-)

55

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016

 CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-2384  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-2384  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-2384  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0774  CVE-2016-2384

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-56
SLIDE 56

In a SUSE data center, today ;-)

56

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-57
SLIDE 57

In a SUSE data center, today ;-)

57

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016

 CVE-2016-1583  CVE-2016-3134  CVE-2016-1583  CVE-2016-3134  CVE-2016-1583  CVE-2016-3134  CVE-2016-1583  CVE-2016-3134  CVE-2016-1583  CVE-2016-3134

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-58
SLIDE 58

In a SUSE data center, today ;-)

58

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-59
SLIDE 59

In a SUSE data center, today ;-)

59

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016

 CVE-2016-4997  CVE-2016-4997  CVE-2016-4997  CVE-2016-4997  CVE-2016-4997  CVE-2016-4997

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-60
SLIDE 60

In a SUSE data center, today ;-)

60

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-61
SLIDE 61

In a SUSE data center, today ;-)

61

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016

 CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-62
SLIDE 62

In a SUSE data center, today ;-)

62

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016 Linux Kernel Sep-12, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-63
SLIDE 63

In a SUSE data center, today ;-)

63

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016 Linux Kernel Sep-12, 2016

 CVE-2016-6480  CVE-2016-6480  CVE-2016-6480  CVE-2016-6480  CVE-2016-6480  CVE-2016-6480  CVE-2016-6480  CVE-2016-6480

Sample data taken

  • n Sept-15, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-64
SLIDE 64

Sample data taken

  • n Sept-15, 2016

In a SUSE data center, today ;-)

64

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016 Linux Kernel Sep-12, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-65
SLIDE 65

In a SUSE data center, today ;-)

65

Linux Kernel Nov-11, 2015 Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016 Linux Kernel Aug-16, 2016 Linux Kernel Sep-12, 2016

 CVE-2013-7446  CVE-2015-6937  CVE-2015-7872  CVE-2015-7990  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0728  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2013-7446  CVE-2015-8019  CVE-2015-8539  CVE-2015-8660  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2015-8709  CVE-2015-8812  CVE-2015-8816  CVE-2016-0758  CVE-2016-0774  CVE-2016-1583  CVE-2016-2053  CVE-2016-2384  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2016-0758  CVE-2016-1583  CVE-2016-2053  CVE-2016-3134  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-4997  CVE-2016-5829  CVE-2016-6480  CVE-2016-6480  CVE-2016-0758  CVE-2016-2053  CVE-2016-4470  CVE-2016-4565  CVE-2016-5829  CVE-2016-6480

Sample data taken

  • n Sept-15, 2016

December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016

slide-66
SLIDE 66

Key Solution Highlights

66  Available for SLES 12 onwards (x86-64)  Provides fixes for Kernel bugs which affect

 Security  Stability  Data Integrity

 No runtime performance impact  No interruption of applications while patching  Allows full review of patch source code  Build-in PTF support  Patches available for most recent maintenance

kernels (last 12 months)

 Currently based on kGraft OpenSource project

slide-67
SLIDE 67

Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

67

slide-68
SLIDE 68

Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

68

(c) creativecommons.org/licenses/by/3.0

slide-69
SLIDE 69

Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

69

(c) creativecommons.org/licenses/by/3.0 http://cdn.slashgear.com/wp- content/uploads/2012/10/google-datacenter-tech-21.jpg

slide-70
SLIDE 70

Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

70

(c) creativecommons.org/licenses/by/3.0 http://cdn.slashgear.com/wp- content/uploads/2012/10/google-datacenter-tech-21.jpg (c) openSUSE.org

slide-71
SLIDE 71

Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

71

(c) creativecommons.org/licenses/by/3.0 http://cdn.slashgear.com/wp- content/uploads/2012/10/google-datacenter-tech-21.jpg (c) openSUSE.org FUJITSU PRIMEQUEST 2800B, (c) Fujitsu

SAP HANA

slide-72
SLIDE 72

Outlook

72

SLE Live Patching for ppc64le SLE Live Patching for IBM z Systems User Space Live Patching SLE Live Patching for Aarch64 Virtualization Live Patching

slide-73
SLIDE 73

Further Information

73

Join SUSE as Live Patching developer

https://jobs.suse.com/job/prague/live-patching-developer/3486/2529381

SUSE Linux Enterprise Live Patching – 60 day Eval

www.suse.com/products/sles-for-sap/

Forrester – Linux vs. Unix Hot Patching – have we reached the tipping point?

http://blogs.forrester.com/richard_fichera/16-05-20- linux_vs_unix_hot_patching_have_we_reached_the_tipping_point

7-11 November, 2016

www.susecon.com

slide-74
SLIDE 74

Thank you

74

Hannes Kühnemund SUSE Product Management hkuehnemund@suse.com @hakuehnemund www.linkedin.com/in/hanneskuehnemund

slide-75
SLIDE 75

Backup

75

slide-76
SLIDE 76

References

One hour of downtime costs $100k for 95% of all enterprises

http://itic-corp.com/blog/2013/07/one-hour-of-downtime-costs-100k-for-95-of- enterprises/

Kernel Live Patching for ppc64le

http://mpe.github.io/posts/2016/05/23/kernel-live-patching-for-ppc64le/

Forrester – Linux vs. Unix Hot Patching – have we reached the tipping point?

http://blogs.forrester.com/richard_fichera/16-05-20- linux_vs_unix_hot_patching_have_we_reached_the_tipping_point

Using Live Patching to patch a running SAP HANA system with zero interruption

https://www.youtube.com/watch?v=E9KwTfWeVLg

76

slide-77
SLIDE 77