Self Aw areness, Cultural Aw areness Consistency and Intelligent - - PowerPoint PPT Presentation

8-Apr-09 / 1

Self Aw areness, Cultural Aw areness Consistency and Intelligent Self Assessment (ISA) Scales

Joseph Psotka*, Peter Legree, Colleen E. Miller U.S. Army Research Institute for the Behavioral and Social Sciences

* The views presented here do not represent official Army positions or doctrine.

8-Apr-09 / 2

Hobgoblins or Wisdom

• A foolish consistency is the hobgoblin of little minds. –

Emerson

• Is there a wise consistency and can we make use of it?
• This project believes we can and it is related to self

awareness

• We have developed a technology to exploit it as an ability

measure of cultural knowledge – Batchelder’s Cultural Consensus

• Psychophoresis.

A wise consistency Is the hallmark of Great minds

8-Apr-09 / 3

Example 1

• People offer self-ratings that they are very conservative

(or very liberal) in their political views.

• They rate a series of politically oriented questions about

issues like the role of bigger government, taxes, constitution, law, and justice , etc. on a psychophysical scale.

• Their answers are compared to the group of self –

professed, politically conservative (liberal) respondents (CBA) using Principal Components Factor analysis.

• Their factor scores should correlate with cognitive ability
• measures. (The distance from the conservative (liberal)

extreme should be a measure of self awareness and cultural awareness)

8-Apr-09 / 4

Psychophoresis of Self-Aw areness (Notional)

Ability Measure e.g. SAT, ACT

Knowledge, Skill or Attribute Being Self-Appraised (e.g. Political Attitude)

8-Apr-09 / 5

Psychophoresis of Self-Aw areness

Ability Measure

Polar/ Dialectic Attribute Being Self-Appraised e.g. Political view - Liberal vs Conservative

8-Apr-09 / 6

SAT Verbal and Math Vs Psychophoresis of Political View s

3 00000 2 00000 1 00000 0 00000

• 1 00000
• 2 00000
• 3 00000
• 4 00000

700 400

S...

700 400

S...

700 400

S...

700 400

S...

700 400

S...

R Sq Linear = 0.133 R Sq Linear = 0.003 R Sq Linear = 8.48E-4 R Sq Linear = 0.082...

600 300

SA...

600 300

SA...

600 300

SA...

600 300

SA...

600 300

SA...

R Sq Linear = 0.325 R Sq Linear = 0.016 R Sq Linear = 0.011 R Sq Linear = 0.172 R Sq Linear = 0.049

Math Verbal R = .3 R = .0 R = .0 R =-.4 R = .4 R = .0 R = .0 R =-.5 Factor Scores of Political Views

8-Apr-09 / 7

Actual Math scores correspond to Self appraisals.

• Std. Dev. =

National Mean

8-Apr-09 / 8

Another Data Set

A factor score for Political Views from ten questions and full scale IQ (WAIS) yields a modest correlation with an interesting tail at the other end.

8-Apr-09 / 9

Full Scale IQ by Political View Stratified by Self Appraised Religiosity

__

Full Scale IQ (WAIS)

(r ~ -.6) (r ~ -.5) (r ~ -.3) (r ~ -.4) (r ~ -.2) (r ~ .2)

Looks most like overall dataset

8-Apr-09 / 10

Consensus Based Assessment

• Psychophoresis is founded on Consensus based

assessment (CBA) on subjective (not factual) knowledge.

• CBA is based on theory and demonstrations that

for many domains:

– Opinions become more consistent with level of expertise – Errors in opinion tend to be random over expertise and not systematic – Scoring standards from journeymen and experts are consistent provided adequate sample sizes

• Much ill-defined knowledge is cultural and

dependent on cultural norms or “Personality”.

8-Apr-09 / 11

• Need better data for Psychophoresis

– Self report on a general measure: political, extroverted, reflective; sports fan, aggressive, friendly, etc. – Series of subordinate questions that are complex and ambiguous, not just knowledge but values and cultural norms, personal attributes, and personality. – Create Psychophoresis by analyzing the extremes where respondents are certain, show interest, etc.

• E.g. only test on baseball if they are fans.
• Inter-correlate multiple Psychophoresis measures

to establish a broader assessment of intelligence. Future Directions

8-Apr-09 / 12

Individual Self Aw areness Measures

• Imagine Psychophoresis (self awareness) scores available

for all sorts of subjective measurers: Political, religious, sports, introversion, extraversion, social, academic attributes - virtually any cognitive or personality attribute you could define: – What would the overall correlation be for each individual, and how would that compare with traditional achievement and ability measures? – What would their overall intercorrelation matrix be for groups?

8-Apr-09 / 13

Future Directions

• If you have data or want to collect better data, please give me a call:
• Joseph Psotka PhD

US ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES Physical address: Room 4126, 2530 Crystal Drive (FEDEX and visits) Mail address: 2511 JEFFERSON DAVIS HIGHWAY ARLINGTON VA 22202-3926

• Phone is 703-602-7945

Fax is 703-602-7710

• email: joseph.psotka@hqda.army.mil
• psotka@msn.com

8-Apr-09 / 14

Selection and Assignment Outcomes

ARI Overarching Research Goals

• Individual
• Collective
• Institutional
• Self-development
• Mentoring
• Operational experience

Training and Leader Development Performance

• Knowledge
• Skills
• Leadership

Attitudes

• Army values
• Warrior ethos
• Career intent

Selection Tests

• Develop new measures and methods to support Army achievement of

recruiting, selection, classification, and retention goals.

• Develop effective methods to train Soldiers and units, and grow adaptive

leaders.

8-Apr-09 / 15

15 ARI’s Network Science Research

Problem/Questions

PKT & U Colorado – Automatic technologies to create a network model of actions

• U. Michigan - Impact of Network Structure on Organizational Behavior

CMU – How to improve automated systems for network analysis Boeing - The Impact of Prior Knowledge on Trust Development USMA – The implications of Social Network Analysis of Officers’ email ARI (ILIR) - Impact of Collective network analysis of social networks as feedback

Significance/Potential Impact

Fuller understanding of relationships among action and knowledge. (Landauer & Foltz) Improved understanding for the creation of a new science of collaboration. (Olson) Dynamic network analysis tools that understand knowledge. (Carley) Social network analyses’ implications for rapid trust in teams. (Handel) Better understanding organization of friendly command and control. (McCulloch ) Improving group performance through better SNA (Horn)

Technical Barriers

Complexity of human behavior overwhelms human task analysis . Few effective collaboratories are only beginning to develop Dimension reduction techniques for SNA poorly understood. Theoretical relationships between trust and social networks unexplored. Commonsense knowledge links and ontologies rudimentary. The effects of awareness of social network structure on command

• rganization has not been investigated.

Massively multiplayer games (MMOGs) create scale problems for analysis Simulations and MMOGs demand hundred-fold increase in expense

• ver paper and pencil.

Applications

Army Network Science Organizing Forum Validate behavior models in field (DARCAAT) Apply collaboratory wizard to NSTEC Apply DNA to fielded systems: TIGRNET, CPOF SBIR’s to develop professional forums, MMOGs Develop Officers ready to apply SNA, DNA Collaborate with DARPA, ICT, ICB, NSF on complex networked environments Testbed analysis of MMOGs by Booz Allen (DARPA funded)

Network Science and Human Behavior

8-Apr-09 / 16

16

Group Dynamics Models of Massively Multiplayer Online Games (MMOGs)

• Dr. Noshir Contractor - Northwestern
• Dr. Scott Poole – UIUC, NCSA
• Dr. Dmitri Williams – USC
• Dr. Jaideep Srivastava – U Minn

Problem There is little empirical data on the emergence and evolution of ad hoc teams in large populations of

• humans. MMOGs provide a rich, unique source of

data to validate and test network theories. Significance Massively Multiplayer Online Games (MMOGs) provide a unique testbed to evaluate theories of human networks. Understanding how ad hoc teams form and evolve in virtual worlds can enable prediction about population level group dynamics in real world contexts. Technical Challenges Mining enormous amounts of data Inferring social/communication relations from behavior

Future Opportunities Application to other Army MMOG efforts (e.g., Forterra Systems). Application to mining large non-MMOG datasets for social network data to predict and inform ad hoc team formation and evolution.

8-Apr-09 / 17

17

Social Netw ork Analysis of Officers Technical Relevance and formulation

Dollars

($M) PE Project 07 08 09 10 11 12 13 6.1 128

Start Date: Feb 2007

Problem

Develop and validate predictive model for network structure of individual and group behavior among Army Officers within a functional organization.

Significance

Understanding Organization’s Performance Evaluating Friendly Command and Control Networks Monitoring Terrorist Communication Networks

Technical Barriers

How to use Outlook e-mail systems to gather important and relevant information. Privacy issues, costs, and consent: West Point presents a very unique opportunity to collect data

• ver the course of an entire year.

Graph and social network theory, Empirical research on human behavior, groups,

• rganizations, and societies

Future Opportunities

Continue to Monitor Ikenet Email Network Compare Data across multiple years Blackberry study (adds location, etc) Improvements in collection, and more data Applications to DARPA Strategic Collaboration: TIGRNET; CPOF Army officers are increasingly more competent and trained to use these tools in training and operation.

8-Apr-09 / 18

18

Social Netw ork Analysis of Officers

Connection to the Broader Community

Others Working This Problem

The “network” or “relational” perspective is now a fundamental lens through which to view phenomena for a variety of different areas including social, computer, metabolic, biological, and ecological networks.

Leveraging Other’s Efforts

Dyad Analysis: Study mutual, asymmetric, and null relations to identify statistically significant relationships in the group. Behavior Change Due to Awareness: Investigates automated measures of changes in social behavior resulting from awareness of one’s position in a social network. Validity of E-mail Data: Compares self-reported social networks based on survey data to e-mail communication data to assess validity of e-mail traffic analysis of social networks. Cadet Education: Research opportunities for cadets and faculty.

Collaboration Activities

Carley, Kathleen, CMU CASOS Horn, Dan ARI Moxley, Fred USMA James, John, USMA Many USMA Cadets

IkeNet Data (Email over 24 weeks, 07)