Mods to Improve streaming / montage

[maclellan]

Hello (Phil) and others ,

I am rewriting Cambozola applet and need some help and suggestions for mods to streaming code in ZM particularly files such as Remote_camera.c

I want to improve montages and general streaming to:

1) Allow cycling or selection of monitors without re-initializing the applet. To do this start a stream then allow command to switch monitors while keeping the stream open.

2) Allow live view and montage to have image sizes different that the recording size. For example I want to stream at 320X280 and record at 640X480. I want the image scaled down just before streaming to significantly reduce the bandwidth requirements.

3) Montage up to 16 monitors with scaled down images using only 4 live streams (4 images cycling per stream).

4) Montage using a single applet for the images. (done)

5) Low priority. De-interlace filter image before streaming. Transcode can do this and simultaneously scale image. Could I employ this within the streaming code?

I know this is a tall order but I am halfway there! I do need some advice or suggestions as how to handle this in the streaming code. That means opening a stream generically then allow another command to switch monitors or cycle through a set of monitors sending a single frame for each, on that stream. The header contents need additional mionitor information. This has to not affect existing functionality.

When I'm done this I'll release my new applets and new code changes. This will really inprove live viewing and make montaging 16 monitors a reality.

Thanks!

-scott

[jameswilson]

wow impressed. good luck

[cordel]

Actualy you might want to find out what Phil has going as he i think has started working on his own client. I'm not sure if it is based on Andrew Wilcock's code or not but I do know the intention was to make it lighter wieght and for better performance.
Regards,
Cordel

[zoneminder]

When I ever get round to starting it my approach was to extend the Java applet to have more two way interaction as well as simplifying the streaming (by maybe using a raw stream and not http jpg image type streaming). Thus the applet could have PVR type functionality including signalling back that the stream should pause, rewind, go half speed, switch monitors etc. This is obviously quite a major change and requires back end modifications to support so it's still a way off before I get time to start it.

In the meantime just rewriting the applet to add more functionality is probably the right thing to do. In fact knowing how it might be extended in the future might help direct a current rewrite and make it easier to use as a platform for the future changes.

Some answers to questions,

1) Allow cycling or selection of monitors without re-initializing the applet. To do this start a stream then allow command to switch monitors while keeping the stream open.

The easiest way of doing this is obviously opening a new stream connection for each change. However a more efficient way would be for zms to support cycling directly. As soon as the next release is out in the next couple of days I will look at adding this functonality in.

2) Allow live view and montage to have image sizes different that the recording size. For example I want to stream at 320X280 and record at 640X480. I want the image scaled down just before streaming to significantly reduce the bandwidth requirements.

This shold be possible just by scaling the image in the applet I would assume.

3) Montage up to 16 monitors with scaled down images using only 4 live streams (4 images cycling per stream).

See (1).

5) Low priority. De-interlace filter image before streaming. Transcode can do this and simultaneously scale image. Could I employ this within the streaming code?

I'm not sure I understand this one fully. What do you mean by 'filter image'?

Phil

[maclellan]

Hello Phil,

Thaks for the response. I'll elaborate in more detail.

1)ZMS mods: Scaling images for streaming only.
This means taking the buffer contents and scaling/copying to another temporary location then sending out to stream. Nothing else is affected. So I like to save 640x480 on the disk and use that for motion detection as usual. But I stream 320X240 rather than scale the image at the applet. I reduce the data content 75% and streaming will be much more reliable for montage displays especially. This could be done by code bashing or maybe using Transcode. I only mentioned delinterlacing filter since it is trivial if you are already using transcode.


Transcode is very powerful and has ability to do several operations to jpegs (or most other formats) at once. Because jpegs show deinterlacing artifacts - the chopped horizontal lines on moving object, it can filter this out and dramatically improve the image. It can rescale at the same time and runs on its own thread. But I do not know if its fast enough for us. Maybe we could run the buffer data from memory to do this since transcode is very flexible. It could also be called from c.

here is an example(in Perl):
transcode -i image_list.txt -x imlist,null -I 5 -z -k -g ${width}x${height} --use_rgb -y ffmpeg,null -F mjpeg -o $video_file -H 0 -f $frame_rate > mjpeg.log";

Advantages:
- Dramatic picture improvement.

BUT: If Transcode is too slow then you need code to scale the image and that may not be too difficult anyway. Maybe some of the base64 code you have written could be altered to scale data on the fly.

2) ZMS mods - monitor selection:
-Call ZMS with url as usual but specify a board or device. Then add another call "setMonitor" that select the actual monitor buffer to stream with. The applet could call "setMonitor" in a similar way as starting the stream. Now the applet can select monitors without re-starting the applet. A button on the applet could initiate this. Note that the scale of the monitor could be included here as well to support above. Big improvement.

- same as above but include "setMonitorList" to cycle through monitors in zms. The list cooresponds only to inputs for the device in which the stream was opened. zms adds the monitor to the header so the applet can identify. Each time ZMS streams a buffer frame it goes to the ext monitor on the list.

Advantage to this a huge reduction in computer resources in server and remote. Even for 4 monitors making 4 image montage very reliable. Add earlier scaling and montage of 16 smaller images would be a reality and can be viewed from a single applet.



3) Applet changes:(done)

I have rewritten cambozola by breaking it down to 2 applets one for CamStream and the other viewer. When I launch the main viewer it will launch a Camstream applet for each required stream. This improves threading with most browsers.

Additionally I support multiple CamStreams objects that scale and display on a single viewer. the result is a montage on a single window not separate iframes in a browser. This window can be scaled and looks neater more responsive.

The next step is to allow multiple monitors on a single stream. This is done but I cant test it since I need the ZMS mods detailed above. Essentially I parse the monitor from the header and decide where on the main image window to paint it.



-Scott

[jameswilson]

this sounds great. I would also like to incorporate some of this into my windows viewer. Mainly the single stream multiple images. The way i would envisage it working is 1 stream made of a cycle of all images. the more monitors used the longer the refresh between frames. This is a good thing and would really enable 16 way steams per site.

Good luck