Microsoft API For Multimedia Tips
I was recently asked by an artist friend to produce some software for an exhibit that would detect the presence of a viewer and play a sound clip in response.
After contemplating interfacing a motion detector to a laptop I decided to use a webcam and implement a simple differencing motion detection algorithm. I opted to use the full version of MSVC 2005 as the 2008 express version was a little feature light for me and I was developing for the gallery's windows machine.
Some of the issues that I came across may help others, so I've decided to document them here:
_Capturing data from a webcam_
This is A LOT harder in windows than it is in Linux. I've done quite a lot of Linux-based camera manipulation for my robots and it only really took me an afternoon to learn, and of course there was a plethora of open-source products that I could take apart and look under the hood to identify how to best perform the task.
I used the now slightly out-dated "Video for Windows" (VWF) interfaces, as I wasn't sure what version of windows that the target machine would be running and this, by MS's own admission, is the version that has the best chance of portability across versions. I was (eventually) able to download an example project that did everything that I wanted to do from the "Code Guru" website, (google "Ken Varn's Video for windows class interface"). It worked pretty well, but it took me a while to identify the monstrous way that VWF passes bitmaps of still frames. When you capture an image you are given a struct of type PBITMAPINFO. This struct contains all the gubbins for displaying a bitmap, the colour palette and whatnot, but not the actual bitmap data. This, under the MS API is stored contiguously in memory to the struct, with no direct pointer to it! This seems obscene to me, but I battled on and everything worked.
_Playing Sounds_
The API for playing wav files in MSVC seemed as easy as it possibly could be. The "PlaySound" function simply takes the filename of the relevant file and a series of flags that tell the OS what the source is, (file-based or memory-based) and how to play it etc etc. Seemed simple enough... but no.
PlaySound has a strict limitation that isn't brilliantly documented. When playing sound synchronously the file size has a 100kb limit, (which at the sampling rate that we were using equates to about 5 seconds). This was too restrictive for my app, so I needed a work round. It turns out that you can lengthen the memory limit if you choose to play the sound asynchronously, (i.e. pass the SND_ASYNC flag.) In this case it will play any file size that will fit into the available memory - but there's another catch. Because the sound is played asynchronously, there's no sensible way that you can check if the sound is finished, which was a problem for my application. In the end I played sounds with the SND_NOSTOP flag too, which meant that if a sound was already playing, it aborted trying to play the new sound. For my application this was the ideal behaviour, but I did notice that on the MS forums people were unhappy about this. My tip for checking if the sound has completed or not is a dirty hack, but it works.... simply play the sound of silence, (a short, pre-recorded wav file, that contains no sound) using the NOSTOP flag. If it plays successfully, the function will return TRUE and you can be assured that the other sound has finished. If it fails, (returns FALSE) then the sound is not finished. A bit dirty I know, but it does the job effectively.
_Application Distribution_
I foolishly just copied the release version of my program on to a memory stick, which was a stupid thing to do, but I took my chances. This didn't work, as the runtime libraries that I needed weren't on the destination machine. Instead I built an installer and the IDE made sure that all the relevant dependencies were packaged up with a minimum of fuss. This was really easy and worked very well.
One word of warning though - the machine that I was developing for wasn't internet ready and didn't have access to a connection. The installer NEEDED internet access as it installed some of the .NET runtime stuff via the MS website. In the end the only solution was to move the machine to someone's house with internet and quickly hop on for the install.
Apologies for the techy post, but this might help someone in the future, (probably me!) Ta!
After contemplating interfacing a motion detector to a laptop I decided to use a webcam and implement a simple differencing motion detection algorithm. I opted to use the full version of MSVC 2005 as the 2008 express version was a little feature light for me and I was developing for the gallery's windows machine.
Some of the issues that I came across may help others, so I've decided to document them here:
_Capturing data from a webcam_
This is A LOT harder in windows than it is in Linux. I've done quite a lot of Linux-based camera manipulation for my robots and it only really took me an afternoon to learn, and of course there was a plethora of open-source products that I could take apart and look under the hood to identify how to best perform the task.
I used the now slightly out-dated "Video for Windows" (VWF) interfaces, as I wasn't sure what version of windows that the target machine would be running and this, by MS's own admission, is the version that has the best chance of portability across versions. I was (eventually) able to download an example project that did everything that I wanted to do from the "Code Guru" website, (google "Ken Varn's Video for windows class interface"). It worked pretty well, but it took me a while to identify the monstrous way that VWF passes bitmaps of still frames. When you capture an image you are given a struct of type PBITMAPINFO. This struct contains all the gubbins for displaying a bitmap, the colour palette and whatnot, but not the actual bitmap data. This, under the MS API is stored contiguously in memory to the struct, with no direct pointer to it! This seems obscene to me, but I battled on and everything worked.
_Playing Sounds_
The API for playing wav files in MSVC seemed as easy as it possibly could be. The "PlaySound" function simply takes the filename of the relevant file and a series of flags that tell the OS what the source is, (file-based or memory-based) and how to play it etc etc. Seemed simple enough... but no.
PlaySound has a strict limitation that isn't brilliantly documented. When playing sound synchronously the file size has a 100kb limit, (which at the sampling rate that we were using equates to about 5 seconds). This was too restrictive for my app, so I needed a work round. It turns out that you can lengthen the memory limit if you choose to play the sound asynchronously, (i.e. pass the SND_ASYNC flag.) In this case it will play any file size that will fit into the available memory - but there's another catch. Because the sound is played asynchronously, there's no sensible way that you can check if the sound is finished, which was a problem for my application. In the end I played sounds with the SND_NOSTOP flag too, which meant that if a sound was already playing, it aborted trying to play the new sound. For my application this was the ideal behaviour, but I did notice that on the MS forums people were unhappy about this. My tip for checking if the sound has completed or not is a dirty hack, but it works.... simply play the sound of silence, (a short, pre-recorded wav file, that contains no sound) using the NOSTOP flag. If it plays successfully, the function will return TRUE and you can be assured that the other sound has finished. If it fails, (returns FALSE) then the sound is not finished. A bit dirty I know, but it does the job effectively.
_Application Distribution_
I foolishly just copied the release version of my program on to a memory stick, which was a stupid thing to do, but I took my chances. This didn't work, as the runtime libraries that I needed weren't on the destination machine. Instead I built an installer and the IDE made sure that all the relevant dependencies were packaged up with a minimum of fuss. This was really easy and worked very well.
One word of warning though - the machine that I was developing for wasn't internet ready and didn't have access to a connection. The installer NEEDED internet access as it installed some of the .NET runtime stuff via the MS website. In the end the only solution was to move the machine to someone's house with internet and quickly hop on for the install.
Apologies for the techy post, but this might help someone in the future, (probably me!) Ta!
posted by Bob at 11:13 AM
3 comments
