MLAPI weird polygon drawing

[ibrewster]

See the attached image for an example - the polygon drawn in the image should be covering the yard (the cutout goes around the bush), and I'm fairly certain the "dog" detection box should be around the bee. It appears as though the polygon/detection box is drawn on the full-size image, but using the scaled coordinates.

3_682897_0.jpg (876.96 KiB) Viewed 6778 times

[asker]

Yup so it seems. As always, logs.

[ibrewster]

OR, we could do the smart thing and look at the code that draws the polygons, where we see that:

a) there is no relevant logging around the code that draws the polygons on the image (pyzm/helpers/utils.py draw_bbox, or the place in zm_detect.py it is called from)
b) you get the newh,neww of the image on line 93 of utils.py, but never use the information (probably not an issue)

AND, most significantly we can also see that the problem is a mismatch between the mlapiconfig.ini file, where (in my case) the resize option was set to 1024 (thereby returning scaled polygons), and the /etc/zm/objectconfig.ini used on line 174 of zm_detect.py, where (again, in my case obviously) resize was set to 'no'

As a result it was drawing the full-size image with the scaled polygons returned from MLAPI. Setting the two values to match "fixes" the issue.

Of course, with the possibility of MLAPI running on a separate machine, these two config options have to be separate, but ideally perhaps zm_detect should consider the 'resized' value of the image_dimensions object returned by MLAPI when deciding if it needs to resize the image on line 174?

Alternately, I see on line 112 that if resize is *not* 'no', zm_detect will actually send the already resized image to MLAPI, so an alternate approach would be to have zm_detect do all the scaling, and for MLAPI to always behave as though resize is set to no.

You may also be able to come up with an even better option, given your superior knowledge of the code and data flow, but regardless, that's the issue: MLAPI is using its value of resize, which says to resize the polygons (correctly), but zm_detect was getting a different value of resize, 'no' in this case, so it didn't know that the polygons had been resized for a smaller image, and therefore didn't size down the image to match.

On a related note, I personally would like it if the image written back to zoneminder was *always* the full-size image, even if we resized it down for detection sake. But that's a feature request, not a bug report

[asker]

OR, we could do the smart thing and look at the code that draws the polygons, where we see that:

a) there is no relevant logging around the code that draws the polygons on the image (pyzm/helpers/utils.py draw_bbox, or the place in zm_detect.py it is called from)
b) you get the newh,neww of the image on line 93 of utils.py, but never use the information (probably not an issue)

It’s not useful to go looking for logs at draw_bbox. We know it’s not drawing right. What do you expect from logs there? What matters more is the context of logs leading to it. This is also why I ask folks to post full logs and not try and make intelligent decisions on which logs are relevant and which are not. It is great that you took the effort to trace it down as a next step of posting an image.

AND, most significantly we can also see that the problem is a mismatch between the mlapiconfig.ini file, where (in my case) the resize option was set to 1024 (thereby returning scaled polygons), and the /etc/zm/objectconfig.ini used on line 174 of zm_detect.py, where (again, in my case obviously) resize was set to 'no'

And that is useful information that I bet you traced logs to find out. Tracing logs is much smarter than reading code first without context.

Of course, with the possibility of MLAPI running on a separate machine, these two config options have to be separate, but ideally perhaps zm_detect should consider the 'resized' value of the image_dimensions object returned by MLAPI when deciding if it needs to resize the image on line 174?

Alternately, I see on line 112 that if resize is *not* 'no', zm_detect will actually send the already resized image to MLAPI, so an alternate approach would be to have zm_detect do all the scaling, and for MLAPI to always behave as though resize is set to no.

You may also be able to come up with an even better option, given your superior knowledge of the code and data flow, but regardless, that's the issue: MLAPI is using its value of resize, which says to resize the polygons (correctly), but zm_detect was getting a different value of resize, 'no' in this case, so it didn't know that the polygons had been resized for a smaller image, and therefore didn't size down the image to match.

Possible. I’ll have to look at the code flow to see - from your description it sounds like it.

[ibrewster]

AND, most significantly we can also see that the problem is a mismatch between the mlapiconfig.ini file, where (in my case) the resize option was set to 1024 (thereby returning scaled polygons), and the /etc/zm/objectconfig.ini used on line 174 of zm_detect.py, where (again, in my case obviously) resize was set to 'no'

And that is useful information that I bet you traced logs to find out.

You'd loose that bet . I traced your code to figure that out. The exact procedure was:

• find the section of code responsible for drawing the boxes (draw_bbox)

• after looking at that code and seeing that it just drew whatever it was told to, go back to where it is called from

• Starting from where you get the response back from MLAPI, trace forward till I got to line 174

At that point, it was pretty obvious what was going on: looking at the matched_data object returned from MLAPI, I could see that the polygons were resized, but when I looked at what you get on line 174 from the `options.get('resize')` command, it was getting 'no' and therefore not resizing the image. A quick look at my configs later, and viola!

Note that at no point do you log if you do or do not resize the image in zm_detect, so I would not have been able to figure out that that was the problem from the logs, at least not as easily as I was able to from simply tracing the code. The thing about logs is they only log what you have thought to log. You have very good logging - better than almost any other package I have worked with. Even so, there are holes. Looking at the code shows you everything that is happening - and can often be faster and easier than looking at the logs, especially if the logs don't log every little thing. At least, that's my experience after many, many years of being a full-time software engineer

[ibrewster]

I owe you an apology. When I submitted this issue, I approached it from the mindset that I approach all the programing bugs I deal with every day - debug the code and see what it is actually doing vs what it is supposed to be doing. I then made the assumption that you would - and should - approach the problem the same way. I realize now that that assumption was incorrect and inappropriate - everyone handles issues differently.

You have obviously spent a lot of time and effort developing this software, and you deserve better than having random outsiders such as myself tell you how you should do things. Please accept my apologies.

[asker]

That's fine.

I've committed a fix, if you'd like to test.
https://github.com/pliablepixels/zmeven ... f3ae2a3803

[ibrewster]

Looks good, thanks!