#1 2009-10-22 02:14:56

**_barry.b_**

how do they do that? (a feature of Sonic Foundry's Mediasite tool)

Hi guys

I'm looking at a recorded videoconf played by Mediasite. I'm trying to work how they do something in it (so I can see if it can be done by either Connect Pro or a Flex app with the CoCoMo componentry)

for those that haven't seen Mediasite in action: (note: this is not how Mediasite works, but a translation into how ConnectPro might do it)

imagine a bunch of presenters' webcam videopods that are off each presenters presentation area. Imagine a single videopod "view" that is _within_ the presenters area that all attendees can see.

when a presenter speaks, the webcam of that person is then displayed within the single videopod in the presentation area. The switching between one video pod from "private" to "public" happens (it seems) automatically, based on whoever is speaking.

it's just like multiple TV cameras in a TV studio with an online editor doing the video switching to the live feed, except automatically triggered. There is a delay in the camera switch but, because all the audio is heard, I'm not worried if it's slow to switch - it's no worse than some heated current affairs shows where politicians shout each other down!

From what I can work out, there are various conditions to make the switch from one videopod to another

- when there is silence (or previous person stops talking for a while) and someone starts talking (camera moves to them)
- when someone is talking, someone else starts over the top of them and the first person stops (camera moves to the second person)
- when someone is talking, someone else starts over the top of them and it turns out to be a shouting match (camera moves to the loudest person)

a couple of last points:
- all audio from all presenters are broadcast at all times
- only the vision is switched (depending on the automation rules) into the "live feed" video pod to be broadcast.

anyone seen anything like this in the "Connect" world? Is it indeed possible?

any thoughts, opinions, etc?

thanx
barry.b

**_Jorma_at_RealEyes_**

Re: how do they do that? (a feature of Sonic Foundry's Mediasite tool)

This is something that could be built in a custom application for Connect.

For the video portion of this app, you would need an external FMS to run the video feeds (as this is not possible in the Connect Video pod), or you could run all the video you want into a video board, that controls the output video into the broadcasting computer (I believe this was done by University of Washington).

For the audio, the problem of getting that functionality comes down to how the audio is delivered. Connect's Audio tool is a single thread audio stream. This means that everyone's audio is "slammed" into a single audio thread, so if more than 4-5 people are talking at once it just turns into noise (if the group is well behaved, you can have more people with hot mics). Again, with an external FMS, there is the option of giving people their own audio thread, with their video stream, but this again requires the building of a custom application to run within Connect.

I'd be happy to discuss options further with you should you want.

**_barry.b_**

Re: how do they do that? (a feature of Sonic Foundry's Mediasite tool)

"This is something that could be built in a custom application for Connect"

interesting...

"For the video portion of this app, you would need an external FMS to run the video feeds (as this is not possible in the Connect Video pod)"

fair 'nuff - it makes sense (see below)

" or you could run all the video you want into a video board, that controls the output video into the broadcasting computer"

this is a non-starter since the various presenters can be (are) separated by much distance - hence trying to do this with medium/low quality and not high quality with dedicated video lines

"For the audio, the problem of getting that functionality comes down to how the audio is delivered. Connect's Audio tool is a single thread audio stream. This means that everyone's audio is 'slammed' into a single audio thread"

the audio (as used as the trigger for automating the vision-switching) sounds like the biggest issue.

"so if more than 4-5 people are talking at once it just turns into noise (if the group is well behaved, you can have more people with hot mics)"

well, at least with experienced presenters, having hot mics isn't so much of an issue and it's a limitation (a shouting match, noise, interruptions) that's worth the compromise (IMHO)

"Again, with an external FMS, there is the option of giving people their own audio thread, with their video stream, but this again requires the building of a custom application to run within Connect."

it actually sounds like three FMS apps working together
- inputs: audio and video stream from each presenter
- outputs: single audio stream (mixed) and video stream (switched)

- #1: a presenter app that takes in the audio and video feed, perhaps also showing (for monitoring and cueing purposes) the other presenters. The app would probable be similar to a bunch of videopods from a typical Connect room
- #2: a public broadcast app that shows the result of the audio mix and switched video
- #3: a controller app that combines the hot mic audio feeds into a single audio stream, uses each presenters audio feed to run the vision-switching logic (based on silence/audio levels), and pushes the switched vision to the public broadcast app.

I suspect to do a good job it would have audio trim control on presenter inputs (in addition to their own settings) before it gets to the vision-switching automation (to fine-tune the triggering) as well as a master output audio trim before being broadcast.

hmmm... not a small amount of work...

**_Jorma_at_RealEyes_**

Re: how do they do that? (a feature of Sonic Foundry's Mediasite tool)

No, Not a small amount of work, but doable.

So for the 3 needs here is how I would see this best working out.

- #1: Each presenter would use Flash Media Live Encoder (FMLE), which is free and can encode their video to Sorenson Squeez, ON2VP6, or h.264 codecs to the the FMS.
- #2: A custom pod (SWF) in the Connect room that displays the output from the FMS and tool #3.
- #3: This can be something that is either built into the custom pod in the room, where only hosts or hosts and presenters have the ability to see and manage the app, or it could be built into an external controller application (built in AIR) that runs the pod behind the scenes.

As far as the automatic switching is concerned, that is just something that would have to be explored in the controller piece of this tool.

Just out of curiosity, how does the Sonic Foundry tool deal with multiple people speaking? Does it show multiple videos, or just the person who is the loudest?

**_barry.b_**

Re: how do they do that? (a feature of Sonic Foundry's Mediasite tool)

re: switching

from what I've been able to work out - or have interpreated as what happens.

- when there is silence (or previous person stops talking for a while) and someone starts talking, the video switching uses their camera (person talking wins)
- when someone is talking, someone else starts over the top of them and the first person stops the video switches to the second person (person who remains talking wins)
- when someone is talking, someone else starts over the top of them and it turns out to be a shouting match video switches to the loudest person (loudest person wins)
- once switched, the video stays with that person for X number of seconds before it is allowed to switch again (to cut down on continuously switching from one person quickly and making the audience "sea-sick"

NOTE: there is a short but noticable delay before the switching of cameras takes effect - between 1 and 2 seconds. But because all the mic's are hot, at least the conversation is heard, even if the video takes a second or so to catch up - a small compromise (an effect not unlike an online editor in a TV studio who has had a rough night and needs lots of coffee!)