V, pensiero! - Fly, thought! Experiment for interactive internet - - PowerPoint PPT Presentation

„Và, pensiero!“ - Fly, thought!

Experiment for interactive internet based piece using Csound6 Tarmo Johannes Kita Toshihiro 2013

WHY?

• Csound community is strong and active, spread
• ver the world
• Chances to meet physically are rear
• Are there ways to meet musically over distance?
• Csound6 run-time compilation of orchestra and

score offer vast possibilities for internet based interactive music making

„Và pensiero“- Fly, thought!

• Dedicated to developers of Csound6 and all bright Csounders
• People join to „choir“ - writing everybody their voice as an Csound

instrument

• One player plays on a MIDI keyboard that triggers the „voices“
• Participants can change their instruments in real time, depending on

the current situation. Nobody knows exactly how the choir will sound in the performance :)

• The result (should be) streamed into internet, so the players and

public can hear it

Software

• pensiero-server – written in C++, Qt framework:

– sets up listener for incoming messages, loads csd file and plays it, compiles Csound

code from the messages, displays output

• va_pensiero_orc.csd – collection of the „voices“, triggering midi instrument,

definitions of chords (the melody is orchestrated automatically) etc

• Clients for sending TCP messages:

– pensiero-client.csd – works in CsoundQt with PythonQt support (sends selected text) – pensiero_client.ipynb – iPython notebook (requires iPython works in any web

browser, send Csound code from iPython cells.)

Software

• pensiero-server – written in C++, Qt framework:

– sets up listener for incoming messages, loads csd file and plays it, compiles Csound

code from the messages, displays output

• va_pensiero_orc.csd – collection of the „voices“, triggering midi instrument,

definitions of chords (the melody is orchestrated automatically) etc

• Clients for sending TCP messages:

– pensiero-client.csd – works in CsoundQt with PythonQt support (sends selected text) – pensiero_client.ipynb – iPython notebook (requires iPython works in any web

browser, send Csound code from iPython cells.)

pensiero-server

• Incoming connections are handled in separate

threads

• Received messages are forwarded to Csound in a

separate thread to compile and play, result is sent back to client

• The messages are displayed as chat conversation

between users and Csound

Networking

• pensiero-server must be visible from network

(open TCP port)

• The public server handling/forwarding the

messages is now 54.228.204.118 (in Ireland)

• The messages can be tunneled via ssh to another

computer even if it is behind a firewall (like in Boston) – the actual performance place can be anywhere with internet connection

Streaming the result

• Needs more work and testing
• We tried vlc (VideoLAN media player) for streaming – works otherwise well

with Csound, but it blocked somehow compiling new code (compileOrc )

• Working solution:

icecast <- darkice <- jackd <-Csound

• Listen with any media player (for example):

mplayer -nocache http://johannes.homeip.net:8000/pensiero (currently the stream is ogg encoded)

• TODO: minimize lag between sound production and listening
• Automatic connection of darkice and Cound jack output

Demo 1 send messages to test orchestra

• test_orc.csd – plays instr 100 (pluck sound) every 5 seconds
• HOST: 54.228.204.118, port: 5005
• Use CsoundQt or iPython Notebook, command line or anything that can send a string as

TCP message

• Command line example:

echo „giVal init 12“ | nc 54.228.204.118 5005

• Prepared clients are up on: http://tarmo.uuu.ee/varia/failid/cs/pensiero-files/
• Send your name first in the form ';name YOUR_NAME' for clearer output
• You can send new instruments, run instruments with event_i or scoreline_i opcode,

(re)define tables etc – everything you can do within the orchestra header

• Please don't send scoreline_i „e“ ! (Check against that not implemented yet ...)

Demo 2 Forwarding live playing (test)

• Does not work properly yet!
• Tarmo plays on a midi keyboard at home, note
• n/off are sent as event_i / turnoff2 to pensiero-

server

• Problem: right timings get easily disturbed –

messages don't arrive exactly enough

• Some synchronisation system or other solution is

necessay.

Demo 3: Streaming

• Tarmo plays „Va pensiero“ on MIDI keyboard at

home, streams up on http://johannes.homeip.net:8000/pensiero

• pensiero-server runs in Tarmo's computer
• Send messages to

HOST: johannes.homeip.net, PORT 5005

Demo 4

• Keyboard is played and sound produced on spot
• (Messages to 54.228.204.118, port: 5005)
• Everybody can play along via Csound messages!

Possibilities for future

• Remote teamwork (for example developing new

Csound instruments - needs extension for saving the code)

• Interactive, intercontinental performances
• Csound improvisations, jam sessions, live coding
• another way for community events
• TODO: better streaming, improvements in server,

tests, tests, tests

Thank you!

Have a wonderful conference!

Tarmo, Kita tarmo@otsakool.edu.ee kita@ield.kumamoto-u.ac.jp