ARCHITECTURE, ADMINISTRATION AND SECURITY CONSIDERATIONS IN A SAS - - PowerPoint PPT Presentation

▶
architecture administration
SMART_READER_LITE
LIVE PREVIEW

ARCHITECTURE, ADMINISTRATION AND SECURITY CONSIDERATIONS IN A SAS - - PowerPoint PPT Presentation

Mar 22, 2024 22 likes •280 views

ARCHITECTURE, ADMINISTRATION AND SECURITY CONSIDERATIONS IN A SAS BASED PROJECT Western Australian SAS User Professionals (WASUP) May 2017 CRICOS Provider Code 00301J ASSESSMENT QUALITY PROCESS (AQP) Background Curtin University is ranked

slide-1
SLIDE 1

CRICOS Provider Code 00301J

ARCHITECTURE, ADMINISTRATION AND SECURITY CONSIDERATIONS IN A SAS BASED PROJECT

Western Australian SAS User Professionals (WASUP) May 2017

slide-2
SLIDE 2
  • Curtin University is ranked in the top two per cent
  • f universities worldwide in the prestigious

Academic Ranking of World Universities 2016

  • The Assessment Quality Process is integral to

Curtin’s assessment quality (AQ) assurance process as outlined in the Assessment and Student Progression Manual.

  • The AQP unit survey is conducted twice per year

and the report constructed from each unit's survey data helps to identify AQ and policy compliance issues to ensure consistent and fair assessment practices at Curtin.

ASSESSMENT QUALITY PROCESS (AQP)

Background

slide-3
SLIDE 3
  • Data processing
  • Input data: 3 surveys + supporting data
  • ETL: calculate measures, conform and load to Data Warehouse
  • ETL: run daily during survey period (2 months, twice a year)
  • Output: 3 VA Reports
  • Faculty: Available to anyone; Aggregated data
  • School: Available to AQ panel members; Individual unit details of panel member’s school
  • Unit Coordinator: Available to Unit Coordinator and AQ panel member; Units they were the UC of, or belong to

the school they are a panel member of

  • Project Lifecycle
  • Prototype of ETL (SAS Enterprise Guide)
  • Prototype and beta test reports (SAS Visual Analytics)
  • Design, build, verify and schedule ETL (SAS Data Integration Studio & LSF Platform Manager)
  • Publish VA reports (SAS Visual Analytics)
  • Monitor and improve system performance and AQ process

ASSESSMENT QUALITY PROCESS (AQP)

Project overview (Requirements, Design & Lifecycle)

slide-4
SLIDE 4
  • Four main workspace servers
  • SASDI - heavy workload overnight
  • SASBI – most workload during office hours
  • SASEG – unpredictable workload
  • SASOLAP – minimum workload
  • Infrastructure is provided by Curtin IT Services (CITS)
  • Virtual Machines with low maximum specifications (2–4 core, 2-8GB RAM)
  • Unable to support specifications required for SAS Visual Analytics

SAS ARCHITECTURE AT CURTIN

Curtin Architecture history and constraints

slide-5
SLIDE 5
  • Two SAS Environments:
  • SAS Visual Analytics environment (VA):
  • 1x SAS server (Metadata, Workspace, Web & Web Services)
  • 5x LASR & HADOOP server
  • Total 6x AWS EC2 r3.8xlarge instances (32vcpu, 244GB RAM)
  • Total LASR memory available: 1.2TB
  • SAS 9.4 Enterprise Data Warehouse & Business Intelligence environment (EDW/EBI)
  • 5 x Compute Servers (4vcpu, 32GB RAM, in a SAS Grid)
  • 1 x Metadata server
  • 2 x Mid tier servers
  • 2 x Web server
  • 1 x Environment Monitoring Server
  • 1x PC Files Server

SAS ARCHITECTURE AT CURTIN

Current Curtin Architecture

slide-6
SLIDE 6

SAS ARCHITECTURE AT CURTIN

Benefits of SAS Grid

Allows us to have one SAS workspace server (SASAPP) distributed across 5 physical servers Whichever server is least busy will be selected by the grid controller to run the job

slide-7
SLIDE 7

SAS ARCHITECTURE AT CURTIN

Benefits of SAS Environment Manager

slide-8
SLIDE 8

SAS ARCHITECTURE AT CURTIN

Benefits of SAS Grid Manager (SAS Environment Manager Add-in)

slide-9
SLIDE 9

SAS ARCHITECTURE AT CURTIN

Benefits of SAS Grid Manager – High Availability Settings

slide-10
SLIDE 10

SAS ARCHITECTURE AT CURTIN

Benefits of SAS Grid Manager – LSF Configuration (Hosts & Queues)

slide-11
SLIDE 11

SAS ARCHITECTURE AT CURTIN

Benefits of SAS Grid Manager – Queue Priority, Hosts, Limits & Dispatch windows

slide-12
SLIDE 12
  • Data processing
  • Input data: 3 surveys + supporting data
  • ETL: calculate measures, conform and load to Data Warehouse
  • ETL: run daily during survey period (2 months, twice a year)
  • Output: 3 VA Reports
  • Faculty: Available to anyone; Aggregated data
  • School: Available to AQ panel members; Individual unit details of panel member’s school
  • Unit Coordinator: Available to Unit Coordinator and AQ panel member; Units they were the UC of, or belong to

the school they are a panel member of

  • Project Lifecycle
  • Prototype of ETL (SAS Enterprise Guide)
  • Prototype and beta test reports (SAS Visual Analytics)
  • Design, build, verify and schedule ETL (SAS Data Integration Studio & LSF Platform Manager)
  • Publish VA reports (SAS Visual Analytics)
  • Monitor and improve system performance and AQ process

ASSESSMENT QUALITY PROCESS (AQP)

Recap

slide-13
SLIDE 13
  • Metadata (Users & Groups)
  • Daily maintenance (add/change/remove) is performed in the SAS EDW/EBI Environment
  • Data
  • Data Warehouse is hosted in Curtin’s (on-site) Oracle DB

MULTIPLE SAS ENVIRONMENTS

Synchronising metadata and data between SAS environments

slide-14
SLIDE 14

Using scheduled (hourly) DIS Jobs in the EDW environment:

  • Extract Users & Groups metadata using the %MDU macros
  • Compress output
  • Upload compressed output to VA environment

15 minutes after the hour in VA environment

  • De-compress output
  • Import Users & groups metadata using the %MDU macros

MULTIPLE SAS ENVIRONMENTS

Synchronising metadata between SAS environments

slide-15
SLIDE 15

Using scheduled (daily) DIS Jobs in the EDW environment:

  • Extract Data from Oracle Data Warehouse
  • Using the ‘Load data into LASR Server’ transform, send data directly into the VA environment via

appropriate SAS/Connect

MULTIPLE SAS ENVIRONMENTS

(Ideally) Synchronising data between SAS environments

slide-16
SLIDE 16

Using scheduled (daily) DIS Jobs in the EDW environment:

  • Extract data from Oracle Data Warehouse
  • Compress output
  • Upload compressed output to VA environment

VA environment:

  • De-compress output
  • Perform ‘autoload’ of data into LASR (built-in, configurable VA scripts)

MULTIPLE SAS ENVIRONMENTS

Synchronising data between SAS environments

slide-17
SLIDE 17
  • Data processing
  • Input data: 3 surveys + supporting data
  • ETL: calculate measures, conform and load to Data Warehouse
  • ETL: run daily during survey period (2 months, twice a year)
  • Output: 3 VA Reports
  • Faculty: Available to anyone; Aggregated data
  • School: Available to AQ panel members; Individual unit details of panel member’s school
  • Unit Coordinator: Available to Unit Coordinator and AQ panel member; Units they were the UC of, or belong to

the school they are a panel member of

  • Project Lifecycle
  • Prototype of ETL (SAS Enterprise Guide)
  • Prototype and beta test reports (SAS Visual Analytics)
  • Design, build, verify and schedule ETL (SAS Data Integration Studio & LSF Platform Manager)
  • Publish VA reports (SAS Visual Analytics)
  • Monitor and improve system performance and AQ process

ASSESSMENT QUALITY PROCESS (AQP)

Recap

slide-18
SLIDE 18
  • Unit Level Report
  • Report shows an individual unit’s progress towards meeting the standards
  • For Unit Coordinators & AQ panel members
  • A unit coordinator may only see data related to the units they coordinated
  • An AQ panel member may only see data related to the units offered by the schools they are panel members of
  • School Level Report:
  • Report shows aggregated unit compliance across the schools and reflection/response plan for the schools
  • Ability to drill down to unit level
  • For AQ panel members and executives
  • An AQ panel member may only see data related to the units offered by the schools they are panel members of
  • Faculty Level Report
  • Report shows aggregated unit compliance across the faculty and schools and reflection/response plans for the schools and

faculties

  • For executives and AQ panel members
  • An AQ panel member may only see data related to the units offered by the schools they are panel members of

ROW LEVEL SECURITY

Report Security, Privacy and Confidentiality Requirements

slide-19
SLIDE 19
  • Three groups/types of users
  • Unit coordinators
  • AQ panel members
  • Executives
  • Three reports
  • Unit Report: Detailed data on individual units
  • School Report: Aggregated & detailed data of units across the school & school reflections
  • Faculty Report: Aggregated data of schools across the faculty & school reflections & faculty reflections
  • Two subsets of data
  • Detailed unit responses (used in unit and school report)
  • Aggregated responses + school & faculty reflections (used in school and faculty report)
  • Row Level security required on Data
  • A Unit coordinator should only see units they coordinate
  • An AQ panel member should only see schools they are panel members of

ROW LEVEL SECURITY

Design

slide-20
SLIDE 20
  • Three groups of users:
  • Define 3 groups in SAS Metadata Console (SMC)
  • DIS jobs which utilise the %MDU macros to import/update users via Active Directory & Curtin HR Database

ROW LEVEL SECURITY

Implementation

slide-21
SLIDE 21
  • Three reports
  • Create and apply ACTs with SMC to which allow only the relevant groups to access the relevant reports
  • Two subsets of data
  • Create and apply ACTs with SMC to which allow only the relevant groups to access the relevant data

ROW LEVEL SECURITY

Implementation

slide-22
SLIDE 22
  • Row Level security
  • Create additional column in each row of data which contains the staff ID of the unit coordinator
  • Create additional column in each row of data which contains the staff ID of all AQ panel members
  • Using VA’s Row Level Security (conditional grants), create and define rules which allow the logged in user to access specific

rows of data

ROW LEVEL SECURITY

Implementation

slide-23
SLIDE 23

ROW LEVEL SECURITY

Implementation

slide-24
SLIDE 24
  • Streamlined the AQ process
  • Reduced the overall time spent on the process
  • Improved and standardised the reports
  • Previously used print-out reports
  • Improved security/confidentiality of data
  • Printed/emailed reports were easier to misplace and be seen by others

ASSESSMENT QUALITY PROCESS (AQP)

Project Outcomes

slide-25
SLIDE 25

ARCHITECTURE, ADMINISTRATION AND SECURITY CONSIDERATIONS IN A SAS BASED PROJECT

Herbert Au – BI Applications Administrator