Sound Module

From OrbSWARM
Jump to: navigation, search

Software and Content

Sounds for GoogleIO and Makerfaire

A few musical multi-track pieces and some new ambient/interaction joystick sounds will be used.

New Compositions

Contact Mark A and Lee Sonko for more info about this:

Mark's thoughts from 5-5-09

Erik, Lee,
 
I agree, we should get some fresh content into line for the Google gig. Obviously I'll be servicing that orb with the broken voicebox on Thursday ;-)
 
Personally, given the timeframe I'm not optimistic about kickoff-timing for multitrack pieces, nor about choreography, but maybe we could do something with synchronized light and sound events. If you think it's worth going after the former and you've got ideas, don't let me stand in your way... I'm just trying to think in bang-for-buck mode.
 
Two thoughts in this vein:
 
1. Assuming joystick-driven orbs and joystick-driven sounds, let's think in terms of the number of sounds we have available through that interface, and narrow our focus to that. If that only gives us access to 20 sounds per orb, we've got a much tighter space in which to make a creative statment. It's actually exactly the same size space as we've had all along... the idea is to just think about it a little more intentionally. Here's one proposal for the joystick-driven sound load: Half the orbs get the same set of 20 ratchet-clank sounds, the other half the orbs get the same set of 20 birdcalls. Then we've got a set of naturalesque birds vs. a set of mechanical monstrosities. Thoughts? 3 groups: ratchetclank, modem sounds and bircalls? 4 groups: ratchetclank, modem sounds, analog synthesizers and bird calls?
 
2. We've never bound light events to sound events before. I think this could be cool. Maybe it's something like this: No change to Joystick programming; the joystick sends the same old sound commands to its orb. When the SPU receives them, it says, "OK, I need to put a light event with this. They're asking for BIRD1.mp3, so when I issue that command to the soundmodule, I'm also going to issue RED_FULL_0.5s_OFF to all 4 of the lights." We could establish a lighting events table with a 1:1 mapping of soundfiles to lighting events. Or we could have the SPU choose a random number and kick off a random lighting event (probably not as cool unless we put some constraints on this, a la Simran).
 
But the point of number 2 is to give the orbs more OOMF in their gestures. Pairing light and sound may help this.
 
...and if we do the ratchetclank vs. birdcalls, the former lighting events would be harsh, acidulous and saturated; the latter lighting events would be tastefully soffeted and incorporate fades.
 
I'll sign up for production of 20 ratchetclanks and 20 birdcalls (or somesuch similar thing we're all into).
 
Thoughts? All-ears, but looking for bounded scope.
 
-mark

Mark's thoughts from 3-12-09

As the robot voices idea slowly anneals, I've had a few more ideas/concepts on it:

  • Though all the orbs have their own unique voices, there's a couple sounds that all of them share. One would be a kind of data-ey sound, like they're bursting paragraphs of data amongst each other. They can say single words to each other in their unique clanks, grunts and pings, but there's this other protocol they can use to presumably pass robot-ey data around. Two orbs approach each other, then "peee-p-p-peep... peee-beeep-p-peep," back and forth between them, and then they go on their way.


  • Taking that a bit further, there's a concept we can take from the Wall-E sound design approach: Each actuator on Wall-E has a sound assigned to it. When he moves his shoulder actuator, the corresponding sound is played. OK, simple concept; how does it apply to the orbs? Simply that we assign a handful of sounds to certain orb behaviors/actions, and those sounds would be more-or-less common amongst the orbs (maybe there are slight variations per-orb... but the same basic sound). But then beyond that, each orb has its own sounds that are unique to them. This language is more what you and I were brainstorming at the box-shop -- the perky but calculating robot vs. the dullard who brain dnt.. w3rk... 50.... f@5t.


  • Mapping of orb personalities: who are the family members? Analogs from Teletubbies or other kids' shows could be a good starting point:
    • smart loud enthusiastic and happy (Lala)
    • slow and dull but loveable oaf (Iron Giant? Lennie?)
    • exciteable, ambitious and upset (Ren Hoek)
    • shy, generous and good-natured (?)


I think 4 characters is a good number for two reasons. One, we've rarely been able to field more than 4 orbs. Two, the personalities can be more distinct with 4 vs. more, which may turn out to be important (or maybe not... maybe it's a more-the-merrier sort of thing, seeing as we're a "swarm" and all). When fielding 6 orbs, we just double up 2 of the personalities.

And then there's sound reinforcement. This hasn't been terribly compelling in the past. But thinking about the framework above, it gets more interesting. If certain orb behaviors have sounds associated with them, and were effectively foley-ing the actions, then blasting those sounds over a full-range stationary loudspeaker system (PA) might add a cool immersive feel to the performance. Plus it gets us around the 100dB, 90Hz+ limitations of the sound module, which simply can't compete in most of our venues.

Mark and Denise's thoughts from 5-19-2009

More brainstorms on Orb Interaction sounds and/or set-pieces
    - Sounds should contain tone, cadence and inflection
        * built from machine noises; sounds the orbs could voluntarily create to "talk"
        * text-to-speech
            - Speak&Spell <-- anyone know where we can get one to create samples?
            - The Little Professor
    - Interaction templates
        * Knock-knock joke
        * Joey-calls (I don't know but I been told...)
        * Big-Booty / Wales Tales game
            - as orbs screw up and get out, other orbs laugh and the one leaves the circle
        * Drug deal / Shakedown

Sounds for Scottsdale

As of 12-16-08 Orb 0, 3, 4 and 5 are loaded with the new Scottsdale sounds. Those thumbdrives don't have any Timeline or speech tracks on them.


Sound for upcoming events

Brainstorm here!

Have an orb record local sounds and then play them back at will. This avoids the "no human sounds" guideline because it's obvious the orb didn't make the sound itself. And the orb will demonstrate that it doesn't REALLY know what it's saying. Someone at the Hacker Dojo grand opening party suggested this to Lee. He also said it had been used during one of the Predator movies.

http://www.freesound.org Freely available, Creative Commons sound effects

  • This is awesome:

http://www.mrc-cbu.cam.ac.uk/~mattd/sine-wave-speech/ Can synthesize stuff like this with a formant tracker. (from Jon)

In the RobotHighSchool newsletter, the author writes "You can download it (for free) here: http://www.robothighschool.com/music/rhsremix.mp3"


  • E:\Music2 - No Backup\aaa-Charlotte's Music\Hildegard von Bingen
  • The old Wargames speech synthesizer, "Votrax"

Sound for Burning Man

How to play sounds on orbs

(from Simran) if by "coordinated musical compositions" you mean a piece that plays something synchronized on all the orbs, perhaps with lights (and in the fyoocher, movement) also synchronized, then you'll want to make a timeline.

there are two ways to make a timeline that does that:

  1. make a six (or however many) channel piece of music in 6 sound files that when you play them together does the right thing. Then copy one of the timelines that does something similar, and modify it.

For example, ToccataFugue.tml plays a six channel version of bach's toccata & fugue in d minor. (all the timelines are in the timelines directory in svn: ../trunk/mothernode/java/resources/timelines/).Also a line for put your new timeline in the timelines list file: timelinest.list

2. arrange a bunch of smaller sounds in a timeline so they get played when you want, or according to events like when-button-4-gets-pressed or when-orb-3-goes-over-there or when-anybody-gets-near-orbs-2-or-4.

The timelines Trekkies.tml or AreasTest.tml or Throadsinging.tml are good examples here.

3. ??

4. profit!


To get new sounds into the mix, you'll need to set up the sounds and the compiled sounds directory on your machine (I'll write something to the wiki about that later), run the sound compiler, and then turn the simulate-sounds flag on in the mothernode software 

java -jav swarmcon.jar --simulateSounds true


Simran


(from Jon) It is possible to put any sound file on the thumb drive and play it with a serial command of the form {64 <VPF filename.mp3>}. If we know the filename, we can put it in the list that's sent by the remote control.


From Jon:

I'm considering remote control of Orb illumination based on audio input. It looks like this could be pretty straightforward using PureData (an open-source relative of Max/MSP) http://puredata.info/downloads/ . An extension to PureData (http://iem.kug.ac.at/ritsch/programmier/pd_externals/) can generate serial data, and PureData has an audio input and analysis as native functions. Hooking them together to generate Illuminator serial commands in response to audio data should be relatively straightforward.

Since PD speaks OSC, it is entirely possible to use things like an iPhone as a front end. http://createdigitalmusic.com/2008/05/06/control-music-and-visuals-with-iphoneipod-free-via-pd/

How to Design Sound for SWARM

(from Mark A) The type of music we can most easily integrate at this point is pre-composed pieces, tracked out to 4, 6 or 12 voices. The ideal setup for this would be an artist/composer who's producing their music in Ableton or Protools or some other multi-track environment, and can readily bounce us 4, 6 or 12 tracks that when played together at the same time result in the full mix.

A few more technical details: Each orb has stereo, so with 6 orbs you've got 12 tracks to work with. The stereo sound of each orb probably isnt' all that differentiable in a field of 6 unique sources, so I think in most cases we'll get the best bang-for-buck if we disregard the stereo ability and treat each orb as a mono source (copy the mono track into both left and right of the sound file).

Also, it's a rare performance where all 6 orbs are involved. Limiting track count to just 4 tracks gives you the ability to perform without a missing voice, even with 1 or 2 orbs out of commission. If you've got 5 or 6 orbs rolling, just double the two of the tracks to those orbs.

As far as file formats, the MP3 modules are happiest with encoding at 128k, with NO VBR. Ideally the file size across all tracks is uniform; the closer the file sizes are to each other, the more consistent their start-time latency will be.

Hardware

Consider $4 2 gig thumb drives here

The Sound Module designed by Mark Alexander consists of a Vinculum VMUSIC-2 serial-controlled MP3 player amplified by a Sonic Impact T-Amp integrated amplifier through Blaupunkt speakers. The VMUSIC takes 9600 baud serial commands generated by a custom control board, which receives serial commands from the SPU at 384000 baud.

SwarmSoundNotes 4.jpg


Specs for tthe various componenents:

Vinculum VMUSIC2 MP3 player

http://www.vinculum.com/prd_vmusic1.html

The data sheet is mirrored here DS_VMUSIC2-1.pdf

Spec says 128kb/sec .mp3 files maximum but they can handle more. But Variable Bit Rate certainly doesn't work.

VMUSIC2 command documentation


It turns out you can upgrade the firmware really easily. Download new firmware from here http://www.vinculum.com/downloads.html#vfirmware

Get the "Reflash (VTD)" for the VMUSIC 2. This will be a long-named .ftd file. Rename that file FTRFB.FTD 

When the Flash Disk is inserted and detected by the Vinculum VNC1L it checks for a file called ‘FTRFB.FTD’ in the root  
directory of the disk. If this file is found it will verify the format of the file matches a Vinculum VNC1L firmware upgrade 
file. If the file format is correct then it will upgrade the firmware with the firmware image from the file. FTD files are 
supplied on the Vinculum website but will require renaming to ‘FTRFB.FTD’ to be recognised as firmware upgrade files

The new firmware fixes a lot of bugs, for example this one: 

 Fix VPF command not returning to prompt when it attempts to play a non-existing file.

I'm pretty sure this one was responsible for a lot of the sound system flakiness we saw under development. Also there are some new commands, e.g. VBR to play a random file (good for remotes!)

Full command documentation is here. http://www.vinculum.com/documents.html#vfwspecs

Good idea to mirror it locally in case it changes. Latest version is http://www.vinculum.com/documents/fwspecs/Vinculum%20Firmware%20User%20Manual%20V2.4%20Rev%203.pdf

There's a new version rev205 out now I think. Latest firmware on the Orbs is "VMSC1FUL_V3_63.ROM" as of June 2008 (I think) Jtfoote 01:45, 16 May 2009 (UTC)

Sonic Impact T-Amp Amplifier

Sonic Impact T-Amp: http://www.tnt-audio.com/ampli/t-amp_e.html

As of 2008 this amp is unfortunatly no longer in production. For new designs we recommend the Ramsey UAM4 40 watt class-D amp. http://www.ramseyelectronics.com/cgi-bin/commerce.exe?preadd=action&key=UAM4

Isolation Transformer

We found a ground loop issue between the VMUSIC2 and the T-Amp amplifier. An isolation transformer solved the problem; we used a Radio Shack part number 273-1374.

Control Board

SwarmSoundNotes 5.jpg SwarmSoundNotes 1.jpg

Sound Module Serial Interface Commands

Speakers and Enclosure

Blaupunkt PCx352 3-1/2" ProComponent speakers http://sonicelectronix.com/item_2134_Blaupunkt+PCx352+3-1-2-+ProComponent+Series.html

Sound Design

Sound design was principally done by Simran Gleason. Each Orb is equipped with a MP3 player with 2600 samples. Filenames are hashed to an 8-digot integer and are thus not very descriptive.

List of Orb Sounds



See the archival Orb Sounds for design process and decisions.

Retrieved from "http://wiki.orbswarm.com/index.php?title=Sound_Module&oldid=5479"