Advanced search

Message boards : News : Alpha version of new video watching interface up for testing.

1 · 2 · 3 · Next
Author Message
Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 2257 - Posted: 27 Nov 2013, 7:53:36 UTC
Last modified: 27 Nov 2013, 8:12:45 UTC

Hi Everyone,

I think the new video watching interface is getting close to ready, or at least close enough to ready enough to let you all start trying it out.

You can access it for different species/locations:

Belden:
http://volunteer.cs.und.edu/wildlife/watch2.php?species=1&location=1

Blaisdell:
http://volunteer.cs.und.edu/wildlife/watch2.php?species=1&location=2

Lostwood:
http://volunteer.cs.und.edu/wildlife/watch2.php?species=1&location=3

You'll probably notice there's a lot more to the interface than there was before. Also, the videos are going to be longer -- no more shorter clips -- but don't worry, the new interface saves what video you're on as well as your progress so you can leave and come back to them if you want, and it still uses your min/max video durations.

The way observations are done now is also totally different. Instead of yes/no/unsure for different categories, now you can add events, and specify their start time, end time, leave comments and tag them with various additional information if need be (we're still working on adding some tags so there will be more on that later).

If you click on either of the time boxes, it will automatically enter in the current time in the video for you, which should speed up doing that, you won't have to type it in manually.

This obviously is going to change how validation works (and once the interface is all set I'll get to working on that), but a lot of the issues with unsure markings will go away. Either people have marked events or not. Also, accuracy is going to be redone as well.

With the new interface, I'll be tracking:

valid events marked
invalid events marked
missed events

And instead of having an accuracy value that goes back to the dawn of time, it will be based on how many missed/invalid events you've marked in the last X videos (X is still undecided). So hopefully there will be less worry about accuracy, and more incentive to just mark whatever events you can find -- especially as there are now a whole lot of different events you can mark.

One major thing I want to add to it before it really goes live is a way to post events in the video discussion forum. The old version just posts the entire video -- what I'd like to do here is have the option to discuss an event (which has a start and end time marked), which would then show that clip of the video in the discussion forums, kind of like how some youtube comments work. Hopefully have that up and going in a day or so, depending on whatever kinks I run into.

Let me know what you think, and I'll be working on it over thanksgiving break -- it's nice to finally have some time to really sit down and devote to working on the project again.

I should also have something up and running tomorrow to start re-converting all those slow loading videos. It'll take awhile to get them all converted however, but it's in the works.

cheers and happy thanksgiving!
--Travis

edit:

A couple usage notes! Click "New Event" to start entering in a new event. When you change anything in the event (like what type of event it is, it's start time, end time, comments, etc) a green update icon will show up on the right. Click that to submit your changes to our server. You can keep adding, changing, updating and removing events as much as you want for a video until you click the "next video" button.

The page will also save what video you are working on (for each site/species combo), so if you have to leave the page or your computer crashes, etc; all the events you've saved to the server will still be there when you come back so you won't lose any of your progress.

Profile STE\/E
Avatar
Send message
Joined: 5 Apr 13
Posts: 416
Combined Credit: 29,783,819
DNA@Home: 2,634,206
SubsetSum@Home: 735,231
Wildlife@Home: 26,414,382
Wildlife@Home Watched: 53,380,530s
Wildlife@Home Events: 9,349
Climate Tweets: 0
Images Observed: 0

          
Message 2260 - Posted: 27 Nov 2013, 8:25:52 UTC
Last modified: 27 Nov 2013, 8:39:14 UTC

I'll reserve judgement but it looks more than a bit complicated, are you sure you want to dump something like this on the Project just before a Video Challenge is supposed to take Place ??? The old system or system we use now is is hard enough for some to comprehend, the New Proposed System looks way more complicated to figure out, one will need to Print out a sheet to look & see if something is on the list of Events to Report.

One thing is it'll sure make watching the Video's more Challenging & or Interesting to keep track of any Events ... ;)

hiigaran
Avatar
Send message
Joined: 25 Sep 13
Posts: 131
Combined Credit: 0
DNA@Home: 0
SubsetSum@Home: 0
Wildlife@Home: 0
Wildlife@Home Watched: 3,895,814s
Wildlife@Home Events: 2
Climate Tweets: 0
Images Observed: 0

  
Message 2261 - Posted: 27 Nov 2013, 9:02:30 UTC
Last modified: 27 Nov 2013, 9:15:26 UTC

Interesting idea. I'll play around with it for a few hours and offer my opinions on it during.

I've already noticed the 16x speed option. Definitely a plus, if we are going to have longer videos. Speaking of which, how long is long? Is 20 minutes still the max, or are there longer ones?

Regarding the saving of video times, how does that work?

How will validity work here? What will be classified as missed and invalid, and how will they be classed as such?

EDIT:

- Camera Interaction > Inspect should be clarified a bit. What could classify as a physical, non aggressive interaction.

- Error > Too Dark should be Too Dark/Too Blurry

- All Parent Behaviour events really need to be either clarified or simplified. Is it necessary to be reporting when the parent is walking or flying? And we would need clarification for sitting and standing, especially with thermal regulation for eggs.

Profile STE\/E
Avatar
Send message
Joined: 5 Apr 13
Posts: 416
Combined Credit: 29,783,819
DNA@Home: 2,634,206
SubsetSum@Home: 735,231
Wildlife@Home: 26,414,382
Wildlife@Home Watched: 53,380,530s
Wildlife@Home Events: 9,349
Climate Tweets: 0
Images Observed: 0

          
Message 2262 - Posted: 27 Nov 2013, 9:17:09 UTC
Last modified: 27 Nov 2013, 9:40:12 UTC

I don't think there's anything to see yet, just real short Video's & one's that are Video Issue's. The Event Button doesn't work yet either, I'm just getting an Error when I press on it ... The 16x Option is only as good as your Cache Speed, right now 9x seems to be the Max with slower Cache Speed's than that normally.

hiigaran
Avatar
Send message
Joined: 25 Sep 13
Posts: 131
Combined Credit: 0
DNA@Home: 0
SubsetSum@Home: 0
Wildlife@Home: 0
Wildlife@Home Watched: 3,895,814s
Wildlife@Home Events: 2
Climate Tweets: 0
Images Observed: 0

  
Message 2263 - Posted: 27 Nov 2013, 9:25:16 UTC
Last modified: 27 Nov 2013, 9:33:35 UTC

Your cache speed? You mean internet speed, or actual hard drive performance?

Would we need to make events for the beginning of the video? Suppose the nest is empty when the video starts. Do we make an event for that, or leave it alone, until there is a change in events that would warrant a report?

And yes, I've only recently noticed the syntax error on the event button. If it's relevant, I'm on Chromium, Lubuntu.

EDIT: So far, I'm in agreement that the added complexity with the interface might throw a lot of people off. I too would suggest holding off on a release until after the 5th of January, since the aforementioned challenge will be from the 22nd of December to that date. I'd also suggest holding off the release for this time because there still seem to be a lot of things that apparently need ironing out. I'm more than happy to do extensive testing for this, to make sure everything is as it should be, and both quick and easy for people to use.

Profile STE\/E
Avatar
Send message
Joined: 5 Apr 13
Posts: 416
Combined Credit: 29,783,819
DNA@Home: 2,634,206
SubsetSum@Home: 735,231
Wildlife@Home: 26,414,382
Wildlife@Home Watched: 53,380,530s
Wildlife@Home Events: 9,349
Climate Tweets: 0
Images Observed: 0

          
Message 2264 - Posted: 27 Nov 2013, 9:43:02 UTC

How fast you can Download the entire Video, so far on Chrome using the 9x Setting a lot of times the Download speed can't keep up to the Video Speed your trying to play it at ... 5x or 7x is the Max a lot of Video's ...

hiigaran
Avatar
Send message
Joined: 25 Sep 13
Posts: 131
Combined Credit: 0
DNA@Home: 0
SubsetSum@Home: 0
Wildlife@Home: 0
Wildlife@Home Watched: 3,895,814s
Wildlife@Home Events: 2
Climate Tweets: 0
Images Observed: 0

  
Message 2265 - Posted: 27 Nov 2013, 9:59:26 UTC - in response to Message 2264.

It varies, but given my 100mbps connection, and taking into account the bandwidth loss from the distance between me and the server, it is very well possible for me to have a 20 minute video loaded in a matter of 20 to 30 seconds, assuming I don't get one of those slow videos.

Profile STE\/E
Avatar
Send message
Joined: 5 Apr 13
Posts: 416
Combined Credit: 29,783,819
DNA@Home: 2,634,206
SubsetSum@Home: 735,231
Wildlife@Home: 26,414,382
Wildlife@Home Watched: 53,380,530s
Wildlife@Home Events: 9,349
Climate Tweets: 0
Images Observed: 0

          
Message 2266 - Posted: 27 Nov 2013, 11:43:21 UTC

At best it takes me 2 Min's with Comcast, 2-3 Min's is more like it. I'm not fortunate enough to have a 100mpbs Connection ...

hiigaran
Avatar
Send message
Joined: 25 Sep 13
Posts: 131
Combined Credit: 0
DNA@Home: 0
SubsetSum@Home: 0
Wildlife@Home: 0
Wildlife@Home Watched: 3,895,814s
Wildlife@Home Events: 2
Climate Tweets: 0
Images Observed: 0

  
Message 2267 - Posted: 27 Nov 2013, 12:07:18 UTC
Last modified: 27 Nov 2013, 12:20:53 UTC

Doesn't look you like you need it, though!

EDIT: You know what I would really like to see with future videos? The ability to look for videos based on specific bird sites. That is, to search by radio frequency (the first six numbers in the video name). Also, something to aid new users:

All videos should have a rating similar to the 'interesting' one, which rates how easy it is to make observations in that video, without affecting your points/accuracy. If the bird is very clearly seen, rate it 'easy'. If it takes a bit of concentration to find, rate 'normal'. If the bird is completely obscured, and requires looking for nest movement, or the almost unnoticeable movements in some night videos, rate 'difficult'. Then, these community ratings can be applied to all videos of the same site/bird radio frequency, and then users can define the maximum difficulty level they would like to have. That way, new users can remain accurate, and the veterans can maintain the overall project accuracy by doing the harder ones themselves. Simple to implement, doesn't strain the servers when searching for one extra criterion, accuracy should improve overall. Everyone benefits.

Profile STE\/E
Avatar
Send message
Joined: 5 Apr 13
Posts: 416
Combined Credit: 29,783,819
DNA@Home: 2,634,206
SubsetSum@Home: 735,231
Wildlife@Home: 26,414,382
Wildlife@Home Watched: 53,380,530s
Wildlife@Home Events: 9,349
Climate Tweets: 0
Images Observed: 0

          
Message 2268 - Posted: 27 Nov 2013, 13:06:06 UTC
Last modified: 27 Nov 2013, 13:11:02 UTC

Doesn't look you like you need it, though!


Well I'm Retired & don't sleep much because of all the Medication I have to take so I can spend a lot of boring time ;) watching the Video's. It's getting into winter here in Michigan now so not much else to do other than shovel snow when it snows.

I don't really understand how the Once Again New Validation System will even work with all the different Events that could be marked.

What if you View a Video that you think no Event Happened & I View the same Video & determine the Bird let a Fart ... lol ... How will the Validator Handle that, with so many Events to mark Validation will be Murky at best I feel & Reporting & Proving your side will be even Murkier.

A better Event Example would be the Bird Wiggles around & does one of those Half Stands re-situating it's self, did it stand or didn't it stand will be viewed differently by different people I think & Marked differently also ...

hiigaran
Avatar
Send message
Joined: 25 Sep 13
Posts: 131
Combined Credit: 0
DNA@Home: 0
SubsetSum@Home: 0
Wildlife@Home: 0
Wildlife@Home Watched: 3,895,814s
Wildlife@Home Events: 2
Climate Tweets: 0
Images Observed: 0

  
Message 2269 - Posted: 27 Nov 2013, 13:18:29 UTC

Yes, that is my main concern as well. I have no idea how these would be handled, and since we are also stamping the events with times, what's the margin of error here? If a bird leaves at 12:30:50, and someone marks it as such, but then others mark it ten seconds too early/late, what happens there?

Profile Defender
Avatar
Send message
Joined: 4 May 13
Posts: 103
Combined Credit: 1,607,715
DNA@Home: 26,301
SubsetSum@Home: 21,008
Wildlife@Home: 1,560,405
Wildlife@Home Watched: 1,089,206s
Wildlife@Home Events: 343
Climate Tweets: 4
Images Observed: 8

          
Message 2271 - Posted: 27 Nov 2013, 14:15:11 UTC - in response to Message 2269.

It doesn't work. If I want so start a new event, it says "SyntaxError: Unexpected token M"

OS X 10.9, Google Chrome
____________
Proud member of SETI.Germany

Becca Eckroad
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Send message
Joined: 26 Oct 12
Posts: 59
Combined Credit: 0
DNA@Home: 0
SubsetSum@Home: 0
Wildlife@Home: 0
Wildlife@Home Watched: 81,289s
Wildlife@Home Events: 0
Climate Tweets: 0
Images Observed: 0

  
Message 2272 - Posted: 27 Nov 2013, 15:46:59 UTC

I am working on creating a tutorial on how to use the new interface. Hopefully I'll be able to have it finished by the end of the holiday weekend. This also should give teams the opportunity to become familiar with the new system well before the competition begins in mid-December.

I've noticed there are already some comments, and will take those into consideration. Please let me know if you run into issues/need more clarity on how to use the new system. Unfortunately I can't answer many questions about the validator but I can tell you that there will be a couple seconds worth of buffer for event start and end times.

I know there are still some kinks that need to be worked out, but we will be able to gather some really neat data using this new interface.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 2273 - Posted: 27 Nov 2013, 16:14:11 UTC - in response to Message 2262.

I don't think there's anything to see yet, just real short Video's & one's that are Video Issue's. The Event Button doesn't work yet either, I'm just getting an Error when I press on it ... The 16x Option is only as good as your Cache Speed, right now 9x seems to be the Max with slower Cache Speed's than that normally.


This is probably a settings issue, a lot of the videos are 1-3 hours long, so if your max time is 60 minutes it'll just be showing you the few shorter videos that were captured (which also tend to have more errors).

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 2274 - Posted: 27 Nov 2013, 16:14:22 UTC - in response to Message 2271.

It doesn't work. If I want so start a new event, it says "SyntaxError: Unexpected token M"

OS X 10.9, Google Chrome


That was a weird bug. Should be fixed now.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 2275 - Posted: 27 Nov 2013, 16:15:25 UTC - in response to Message 2269.

Yes, that is my main concern as well. I have no idea how these would be handled, and since we are also stamping the events with times, what's the margin of error here? If a bird leaves at 12:30:50, and someone marks it as such, but then others mark it ten seconds too early/late, what happens there?


That's something we'll need to work out to find something that's workable for everyone. I was thinking of having a 5 second buffer, as a starting point to see how it works. So if two people marked the same event and the start/end times were within 5 seconds of each other they would validate against each other.

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 2276 - Posted: 27 Nov 2013, 16:19:36 UTC - in response to Message 2267.

All videos should have a rating similar to the 'interesting' one, which rates how easy it is to make observations in that video, without affecting your points/accuracy. If the bird is very clearly seen, rate it 'easy'. If it takes a bit of concentration to find, rate 'normal'. If the bird is completely obscured, and requires looking for nest movement, or the almost unnoticeable movements in some night videos, rate 'difficult'. Then, these community ratings can be applied to all videos of the same site/bird radio frequency, and then users can define the maximum difficulty level they would like to have. That way, new users can remain accurate, and the veterans can maintain the overall project accuracy by doing the harder ones themselves. Simple to implement, doesn't strain the servers when searching for one extra criterion, accuracy should improve overall. Everyone benefits.


This is a cool idea -- and ideally we want to have something like this. The problem is that right now, we don't know if a video will be difficult to watch until someones watched it already. So there's a bit of a chicken and the egg problem.

Long term goal of the project is to get our computer vision techniques to the point where a lot of the video will be automatically gone through, which should filter out a lot of the videos which have nothing going on or other problems; which should hopefully leave the more interesting videos to all of you.

I do think it's a good idea to have a way to rate a video as to how difficult its was, which could help tweak validation. If everyone reported the video was difficult, then the validator could be a bit more forgiving (and we could flag it for additional review by the project's biologists).

Travis Desell
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 16 Jan 12
Posts: 1813
Combined Credit: 23,514,257
DNA@Home: 293,563
SubsetSum@Home: 349,212
Wildlife@Home: 22,871,482
Wildlife@Home Watched: 212,926s
Wildlife@Home Events: 51
Climate Tweets: 21
Images Observed: 774

              
Message 2277 - Posted: 27 Nov 2013, 16:26:43 UTC - in response to Message 2268.

Doesn't look you like you need it, though!


Well I'm Retired & don't sleep much because of all the Medication I have to take so I can spend a lot of boring time ;) watching the Video's. It's getting into winter here in Michigan now so not much else to do other than shovel snow when it snows.

I don't really understand how the Once Again New Validation System will even work with all the different Events that could be marked.

What if you View a Video that you think no Event Happened & I View the same Video & determine the Bird let a Fart ... lol ... How will the Validator Handle that, with so many Events to mark Validation will be Murky at best I feel & Reporting & Proving your side will be even Murkier.

A better Event Example would be the Bird Wiggles around & does one of those Half Stands re-situating it's self, did it stand or didn't it stand will be viewed differently by different people I think & Marked differently also ...


Hi Steve,

What I'm thinking for the new system is that events marked will be more important than overall seconds watched. Anyhow, in the case you describe what would happen would be:

1. user 1 marks:
sitting from time 1:00:00 to 2:00:00
2. user 2 marks:
sitting from time 1:00:00 to 1:30:00
standing from time 1:30:00 to 1:30:05
sitting from time 1:30:05 to 2:00:00
3. user 3 marks:
sitting from time 1:00:00 to 1:30:00
standing from time 1:30:00 to 1:30:05
sitting from time 1:30:05 to 2:00:00

In this case, user 1 would get 1 valid event (it mostly matches up to what other people say), and 1 missed event; while users 2 and 3 get 3 valid events.

Then for awarding watched video time, I was thinking we could award full video time (in terms of watched seconds) so long as the user has at least 1 valid event (things are set up so at least every video should have at least 1 event).

Profile STE\/E
Avatar
Send message
Joined: 5 Apr 13
Posts: 416
Combined Credit: 29,783,819
DNA@Home: 2,634,206
SubsetSum@Home: 735,231
Wildlife@Home: 26,414,382
Wildlife@Home Watched: 53,380,530s
Wildlife@Home Events: 9,349
Climate Tweets: 0
Images Observed: 0

          
Message 2279 - Posted: 27 Nov 2013, 16:37:26 UTC - in response to Message 2275.
Last modified: 27 Nov 2013, 16:40:00 UTC

Yes, that is my main concern as well. I have no idea how these would be handled, and since we are also stamping the events with times, what's the margin of error here? If a bird leaves at 12:30:50, and someone marks it as such, but then others mark it ten seconds too early/late, what happens there?


That's something we'll need to work out to find something that's workable for everyone. I was thinking of having a 5 second buffer, as a starting point to see how it works. So if two people marked the same event and the start/end times were within 5 seconds of each other they would validate against each other.


If somebody is watching a Video @ 16x Speed & happens to notice something a couple minutes could go by before they could react & press the Event Button. Personally I don't know how anybody could notice much at all at those speeds, I know it's easy to miss things at 9x Speed ...

Also will you be able to scroll back if you do Notice something & slow the Video down to get a more Accurate Time when you press the Event Button ???

Profile STE\/E
Avatar
Send message
Joined: 5 Apr 13
Posts: 416
Combined Credit: 29,783,819
DNA@Home: 2,634,206
SubsetSum@Home: 735,231
Wildlife@Home: 26,414,382
Wildlife@Home Watched: 53,380,530s
Wildlife@Home Events: 9,349
Climate Tweets: 0
Images Observed: 0

          
Message 2280 - Posted: 27 Nov 2013, 16:41:55 UTC - in response to Message 2273.

I don't think there's anything to see yet, just real short Video's & one's that are Video Issue's. The Event Button doesn't work yet either, I'm just getting an Error when I press on it ... The 16x Option is only as good as your Cache Speed, right now 9x seems to be the Max with slower Cache Speed's than that normally.


This is probably a settings issue, a lot of the videos are 1-3 hours long, so if your max time is 60 minutes it'll just be showing you the few shorter videos that were captured (which also tend to have more errors).


Probably, I didn't think of that, I only have mine set to 0-9 Min's, will Bump it up later & see if I get longer Video's ...

1 · 2 · 3 · Next
Post to thread

Message boards : News : Alpha version of new video watching interface up for testing.