Live Streaming with Flash swf: working; Quicktime question

mrreynolds · Post by **mrreynolds** » Sun Dec 04, 2005 11:27 pm

Hello all,

As this is my first post, kudos to Phil for a great tool!

I recently started evaluating ZM for a project needing reasonable quality live streaming over consumer broadband internet connections, as well as record, playback and download. I also need compatibilty with browsers on Win and Mac. So .asf live streaming was not acceptable for me since, as far as I can tell, it does not work on a Mac "out of the box."

So I've been trying other live stream formats, but ZM kept trying to load them into WMP no matter what I tried. Looking at the code in zm_html_view_watchfeed.php I found (starting line 96):

Code: Select all

<?php
if ( $mode == "stream" )
{
        if ( ZM_VIDEO_STREAM_METHOD == 'mpeg' && ZM_VIDEO_LIVE_FORMAT )
        {
                $stream_src = getStreamSrc( array( "mode=mpeg", "monitor=".$mid, "s
cale=".$scale, "bitrate=".ZM_WEB_VIDEO_BITRATE, "maxfps=".ZM_WEB_VIDEO_MAXFPS, "for
mat=".ZM_VIDEO_LIVE_FORMAT ) );
                if ( isWindows() )
                {
?>
<object id="MediaPlayer" width="<?= reScale( $monitor['Width'], $scale ) ?>" height
="<?= reScale( $monitor['Height'], $scale ) ?>"
classid="CLSID:22D6F312-B0F6-11D0-94AB-0080C74C7E95"
codebase="http://activex.microsoft.com/activex/controls/mplayer/en/nsmp2inf.cab#Ver
sion=6,0,02,902"
standby="Loading Microsoft Windows Media Player components..."
type="application/x-oleobject">
<param name="FileName" value="<?= $stream_src ?>">
<param name="autoStart" value="1">
<param name="showControls" value="0">
<embed type="application/x-mplayer2"
pluginspage = "http://www.microsoft.com/Windows/MediaPlayer/"
src="<?= $stream_src ?>"
name="MediaPlayer"
width="<?= reScale( $monitor['Width'], $scale ) ?>"
height="<?= reScale( $monitor['Height'], $scale ) ?>"
autostart="1"
showcontrols="0">
</embed>
</object>
<?php
                }
                else
                {
?>
<embed type="video/mpeg"
src="<?= $stream_src ?>"
width="<?= reScale( $monitor['Width'], $scale ) ?>"
height="<?= reScale( $monitor['Height'], $scale ) ?>"
autostart="1"
showcontrols="0">
</embed>

As I read this, if the browser is on Windows, MediaPlayer is loaded to try and read the stream no matter what the actual format. Otherwise, mime type "video/mpeg" is selected on all other platforms, again no matter what the actual format.

Since I'm trying formats compatible out of the box with both Win and Mac, I've focused on Quicktime (.mov, .mp4) and Flash (.swf). And of course I need the browser to load the correct player. So I have changed the above code as follows:

Code: Select all

<?php
if ( $mode == "stream" )
{
        if ( ZM_VIDEO_STREAM_METHOD == 'mpeg' && ZM_VIDEO_LIVE_FORMAT )
        {
                $stream_src = getStreamSrc( array( "mode=mpeg", "monitor=".$mid, "scale=".$scale, "bitrate=".ZM_WEB_VIDEO_BITRATE, "maxfps=".ZM_WEB_VIDEO_MAXFPS, "format=".ZM_VIDEO_LIVE_FORMAT ) );
//MRR Following switch code to match embed type to file types being used
                switch ( ZM_VIDEO_LIVE_FORMAT )
                {
                        case "asf" :
                                print "<embed type=\"video/x-ms-asf\" ";
                                break;
                        case "swf" :
                                print "<embed type=\"application/x-shockwave-flash\" ";
                                break;
                        case "mp4" :
                                print "<embed type=\"video/mp4\" ";
                                break;
                        case "mov" :
                                print "<embed type=\"video/quicktime\" ";
                                break;
                        default :
                                //MRR Note - leaving blank forces browser to determine mime type from suffix of the src attribute
                                print "<embed ";
                                break;
                }
?>
src="<?= $stream_src ?>"
width="<?= reScale( $monitor['Width'], $scale ) ?>"
height="<?= reScale( $monitor['Height'], $scale ) ?>"
autostart="1"
showcontrols="0">
</embed>

This works fine to load and stream to WMP (.asf) or Flash (.swf) players on Win; on Mac .swf works great also. As a bonus, Flash avoids some of the other problems reported here with .asf: there's no delay in starting up the stream and at least in my limited testing, it does not lose time when run continuously.

Although Quicktime loads, I have not succeded yet to get live streaming working with .mp4. And with .mov I get this error from zms:

Code: Select all

Unable to determine mime type for 'mov' format, using 'video/mpeg' as default

If someone has sorted this out, please le me know.

Thanks again for the fine project.

regards,
Mickey

caseystone · Post by **caseystone** » Mon Dec 05, 2005 4:02 am

Why not just use the streaming jpeg instead of video? It works perfectly well on OSX with Firefox (not so good with Safari). I guess since the mpeg streaming has never worked for me on my mac, I've never tried it. Does it have some great advantage over the streaming jpeg?

-Casey

SyRenity · Post by **SyRenity** » Mon Dec 05, 2005 9:11 am

Hi.

This is very interesting - there is another thread, speaking about playing back events in Flash (http://www.zoneminder.com/forums/viewto ... ight=flash), but AFAIK no one tried yet streaming a live video Flash.

How's the image quality, compared to WMP? Also, was the visible fps equal to the one of asf?

Regarding MP4, I checked this matter for using MF plugin (http://mediaframe.org/), and it seems that the current version of FFMPEG (at least the one that I got from CVS around a month ago), can't create MP4 streams using avcodec library (which ZM uses for live streaming). Manual generation (i.e. via running the FFMPEG) works fine although. Perhaps the situation has changed since then.

The most noticeable advantage to MPEG is simply speed - streaming JPEG, no matter how fast the connection, will never look like a flowing video.

mrreynolds · Post by **mrreynolds** » Mon Dec 05, 2005 2:22 pm

Re:

Why not just use the streaming jpeg instead of video? It works perfectly well on OSX with Firefox (not so good with Safari). I guess since the mpeg streaming has never worked for me on my mac, I've never tried it. Does it have some great advantage over the streaming jpeg?

Wish I could; client wants "more realistic" motion, yet internet upload bandwidth is limited in this case to 384 kb/s. Motion jpeg consumes almost all of it for 1 320x240 image even on quality rating = 30 at 5 fps. But with swf I can stream 2 images at 10 fps with (some) BW to spare.

Re:

How's the image quality, compared to WMP? Also, was the visible fps equal to the one of asf?

Image quality not quite as good as asf but acceptable. Visible fps seems about the same, although you can notice a slight pause and recover in the stream. You can try it out yourself here: http://reynoldshouse.net:8081. Use uid=View, pwd=view and set at medium bandwidth to get higher fps. Note that one issue I discovered after the original post is the Flash player consumes substantial memory (about 100 MB on my system) and additional streams all use additional memory space - unlike with WMP.

Re:

Regarding MP4, I checked this matter for using MF plugin (http://mediaframe.org/), and it seems that the current version of FFMPEG (at least the one that I got from CVS around a month ago), can't create MP4 streams using avcodec library (which ZM uses for live streaming). Manual generation (i.e. via running the FFMPEG) works fine although. Perhaps the situation has changed since then.

I'm also exploring MPEG4IP (http://mpeg4ip.net/) which can stream mp4 to Quicktime using ffmpeg (using hinted sdp files), which caused me to explore it through ZM.

Thanks for the follow-up.

Mickey

SyRenity · Post by **SyRenity** » Mon Dec 05, 2005 7:13 pm

Hi Mickey.

Thanks for the update regarding the memory usage with Flash. I noticed that it also doesn't work with Montage? I'm getting black screens for every camera. Have you experienced something similar?

Post by **zoneminder** » Mon Dec 05, 2005 7:13 pm

This is interesting. Can you tell I don't actually know anything about video formats (

). I'll try and incorporate some of your ideas into the next release. The mov mime-type thing is from ffmpeg as it keeps a store of known mappings but I can probably override it.

Phil

caseystone · Post by **caseystone** » Mon Dec 05, 2005 7:46 pm

Thanks for putting up the URL for observation. YES! I see what you mean, that is nice video at a low bandwidth. Wow, now I want it too.

Please post your methods and final mods when you get it working to your satisfaction.

Thanks.

-Casey

mrreynolds · Post by **mrreynolds** » Mon Dec 05, 2005 7:56 pm

Hi SyRenity,

For Montage to work, you need to make the same changes to zm_html_view_montagefeed.php - otherwise it tries to load the swf stream into WMP.

BTW, sorry if you tried to reach my demo and could not, my ISP was down the last 3 hours.

Phil,

I tried to trace the mov mime problem through the source code... of course you know the error mentioned above is from zm_mpeg.cpp. But I could not determine where the problem originates... I traced it back through the include files zm_mpeg.h and from there to ffmpeg/avformat.h, but did not see anything odd about mov. But I don't really know C/C++ so I may have missed something. The odd thing is I can use ffmpeg to encode .mov files from my bttv card which play on Quicktime with no problem, for example:

ffmpeg -s 320x240 -t 10 -an test1.mov

Mickey

victor_diago · Post by **victor_diago** » Mon Dec 05, 2005 8:30 pm

Mr Reynolds.

That is AMAZING. Man Much thanks about that, i will be moving to swf by now.

I have only two questions, 1st how many seconds do you have of delay by watching live view ?

2nd How fast in kbps is your uplink ?

Thanks Again

Victor Diago

mrreynolds · Post by **mrreynolds** » Mon Dec 05, 2005 9:21 pm

Hi Victor,

From my limited trials, there's practically no delay when watching a live stream. On my LAN the stream from my live TV demo is ahead of the WinTV card in my workstation.

From a distant site the stream seems to keep up with real time as long as the bandwidth use is not hitting the upload limit. But it will stall if too many packets are dropped.

My ISP limits my upload BW to 225 kb/s, so I've set the encoding bitrate to 100 kb/s at 10 fps for medium BW in ZM. This is so the peak demand shown on my Win XP task manager Networking tab mostly stays below 200 on the high update speed setting.

Be sure to note the large memory footprint for Flash that I mentioned above... this makes it impractical to display more than 2 feeds at once, for example in Montage. There must be a way to manage this, however, as Google video (video.google.com) runs Flash in your browser with no substantial memory load... appreciate if anyone knows the trick to this.

Rgds,
Mickey

Post by **zoneminder** » Mon Dec 05, 2005 9:37 pm

As far as I know the problem (if it is a real problem) with the ffmpeg library is purely outputting the appropriate mime header for .mov, actually creating the video is fine. I suspect it's probably an element of ffmpeg that most propel don't use and I'm probably being lazy using it at all rather than just generating my own mime headers. One way or the other I'll try and crack it, as I've just about finished the polygonal zones changes.

Phil

victor_diago · Post by **victor_diago** » Mon Dec 05, 2005 9:50 pm

Ok Thanks.

I will need it in montage. I will try and see what happen, but if you set fps to lower does it help ?

I will search something about memory usage in flash player under firefox.

Actually, what happen when you try to do a montage in swf mode ?

Cheers

Victor Diago

jameswilson · Post by **jameswilson** » Mon Dec 05, 2005 9:52 pm

ill also give up on writing the mpg4 bit for zm4ms and go straight to this too, I think its easier in .net to impliment as well.

victor_diago · Post by **victor_diago** » Mon Dec 05, 2005 10:04 pm

Yeah.

We really need something better than Motion Jpeg to really become a professional surveillance system, bandwidth is very important and low delay online monitoring is also important.

Victor Diago

PS: Lets get skype each other

My Skype -> victor_diago

Post by **zoneminder** » Mon Dec 05, 2005 10:19 pm

I have modified the mime header handling to become more independant of ffmpeg and thus easier to amend. I will also incorporate the header changes suggested early in the thread into the relevant php files. I am planning to do a 1.22.0 release very shortly so they will be in that, though as a .0 release it will be pretty much a beta until I'm sure that the other changes don't break too much.

Phil