IkMijn pagina voor
prettig werken op de PC
www.pcpret.nl
Homepage Stuur bericht Gastenboek Website Blog Eindhoven

Video file saving in Windows Media video format for the DirectX.Capture class library

This article shows a coding example to save video as Windows Media video file. A number of enhancements to the DirectX.Capture class are described to save the captured video to a file. The DirectX.Capture class is originally written by Brian Low. He did a tremendous job to make DirectX available to C#. As result everyone can start with a working example to learn and use DirextX, DirectShow and C#. And also a lot of people wrote modifications for this class that could be used by others. An interesting webiste is DirectShow for C#, made by David David Wohlferd(?). At this site he provides (temporarily) some interesting examples using DirectX and Windows Media functionality. You can also find here an interesting modified version of the DirextX.Capture class example, named CaptureTest. He added much future functionality that was mentioned by Brian Low before. Very interesting is the way the new CaptureTest program can store program settings in a configuration file.

I wish you much pleasure with learning and using the DirectX.Capture class library and I hope you will enjoy my article also.
Hans Vosman

Sample Image - Capture Test

Introduction

This article is a follow up of my previous article Audio file saving for the DirectX.Capture class. This article describes the saving of video in de Windows Media file format (Wmv). Saving video is more complex than saving audio. This because saving video can mean saving video with and without audio. Because of that, there is a bigger chance that video saving fails due to a conflict with the selected save format. Furthermore, there are many more predefined video formats than audio formats, so it would be nice to have a user friendly solution of selecting a specific video format that suites your needs the best.

The DirectX.Capture class example was a big help for me in finding out how video file saving in the Windows Media format could be done. Because the DirectX.Capture class example supports Avi file saving only, so providing Windows Media file saving is an interesing enhancement. This because the files may become (much) smaller. The article C# Windows Media Format SDK Translation by Idael Cardoso gave me insight information on the Windows Media Format SDK. It took much effort to find out how the IWMProfile interface could be used for retrieving useful information. This article will provide an example how Windows Media can be used in an application.

Using profiles ...

Why using profiles? Well, each Windows Media format is represented by a profile. There is a list of system profiles for Windows Media that can be used, most interesting is that it is also possible to make a profile or to modify an existing profile. The system profiles can be found in the file WMSysPrx.prx in the Windows directory. A prx file contains the description of the profile in XML format. And maybe you already knew, Microsoft provides in the Windows Media Encoder software package a Windows Media profile editor, named WMProEdt.exe. So it is worth to look at that. Here a part of a WM Encoder profile:

<profile version="589824" 
        storageformat="1" 
        name="Higher quality video (VBR 97)" 
        description=""> 
                <streamconfig majortype="{73646976-0000-0010-8000-00AA00389B71}" 
                        streamnumber="1" 
                        streamname="Video Stream" 
                        inputname="Video409" 
                        bitrate="100000" 
                        bufferwindow="-1" 
                        reliabletransport="0" 
                        decodercomplexity="" 
                        rfc1766langid="en-us" 
                        vbrenabled="1" 
                        vbrquality="97" 
                        bitratemax="0" 
                        bufferwindowmax="0"> 
                <videomediaprops maxkeyframespacing="80000000" 
                        quality="0"/> 
                <wmmediatype subtype="{33564D57-0000-0010-8000-00AA00389B71}"  
                        bfixedsizesamples="0" 
                        btemporalcompression="1" 
                        lsamplesize="0"> 
                        <videoinfoheader dwbitrate="100000" 
                                dwbiterrorrate="0" 
                                avgtimeperframe="333667"> 
                ...
</profile> 
It is a XML file, containing the name, the description, the Windows structure names, the attribute names and their values of the profile. So a profile provides all details about the supported audio and video streams. A profile is used to configure the Asf file writer. The Asf file writer is the filter that will handle the actual multiplexing, the encoding of the video and/or audio and the acutal file saving! By configuring the Asf file writer, the video (and/or audio) will be saved in the chosen format. But much more important for me, the profile can also be used as information source for an user friendly solution of selecting a Windows Media file format.

A good start for learn more about Windows Media and profiles is to read Windows Media Format 9.5 SDK. This SDK contains samples, header files and, programs and documentation. The good news is that the Windows Media Format SDK can also be downloaded.

Problems with using the Asf file writer

In my previous article about audio saving, Audio file saving for the DirectX.Capture class, I mentioned that the Asf file writer gave conflicts when it was added to the graph but not connected. Because of that I could not use the default property window for selecting a Windows Media format for file saving. A new form is needed to provide similar functionality as the default property window. That is not that bad, because now more information can be shown about the profile itself then the default property window does. For example, information about supporting audio and video, the bitrate and the description of a profile can be shown. It is even possible to show the framesize, but that is some more work because the framesize is a little bit harder to retrieve. To have the specific profile information, is handy, because you will know in advance if audio and/or video can be saved. Anoter good reason is that the program can use this information also. If a selected format does not match the rendered audio and/or video streams to the file writer the program can detect that and can give a warning.

IWMProfile

To get a profile the IWMProfile interface needs to be used, however more interfaces are needed: First the profile manager (IWMProfileManager and IWMProfileManager2 interface), needs to be initialized. The Windows Media version needs to be set, Windows Media version 8 will be used in here. The IWMProfileManager2 interface is used because via this interface the profiles can be retrieved by using GetSystemProfileCount(), LoadProfileByID() and LoadSystemProfile().

If the profile is found, then the name can be retrieved via GetName() and description can be retrieved via GetDescription(). To get the audio and video stream information some more actions are needed. With GetStreamCount() the number of streams can be retrieved. With GetStreamByNumber() the stream configuration information can be retrieved (IWMStreamConfig interface). Then it is possible to check for each stream if it is an audio or an video stream. The framesize is more difficult, for this the video info header information (VideoInfoHeader) is needed. There is no direct interface that retrieves that information but there is a workaround: use the IWMWriter interface.

IWMWriter

The IWMWriter interface gives access to the Windows Media writer. This writer can be used for writing audio and video streams to file. While the Asf file writer is a filter, the Windows Media writer is not a filter and that is the big difference. Why using the Windows Media writer for this? Well, it can take the same profile as the Asf file writer and can save files in the same format. Furthermore, the Windows Media writer can access the video info header information indirectly via an input property. An input property can be retrieved via GetInputProps() of the Windows Media writer.

If a video input property is found, then via GetType() and GetMediaType() of the IWMInputMediaProps the video information can be retrieved, stored in a AMMediaType structure. Via the data in the AMMediaType data structure the VideoInfoHeader can be accessed, if available. Because of its complexity and additional research of releasing allocated COM objects, the use of IWMWriter is not supported in this example yet.

Another interesting feature might be the use of own-made profiles. Using the StreamReader, IWMProfileManager, IConfigAsfWriter2 interfaces and the functions LoadProfileByData and ConfigureFilterUsingProfile it is possible to load your own profile. An example is described in the Fun with DVR-MS MSDN article written by Stephen Toub.

Using the code

For saving video in a Windows Media format, four new classes are introduced: WMLib, WMProfileData, AsfFormat and AsfForm. The class WMProfileData is used to store profile data. The class AsfFormat is used to store the profiles, using the class WMProfileData. The class AsfFormat has a number of functions that will be used in this coding example. The class AsfForm enable the user to select a profile. The class AsfForm uses AsfFormat to store and retrieve profile information. The class WMLib provides the Windows Media Format SDK interfaces that are really needed in this coding example.

I wrote the WMLib class because I encountered problems with the WMF SDK described in the article C# Windows Media Format SDK Translation by Idael Cardoso. When I finished the example, I did not spend time on using the Yeti Windows Media SDK again. Feel free to make it working, and maybe it will work right away. The main problem is that the libraries are mostly untested. I did not want to use untested interfaces that are not needed in the DirectX.Capture class example. Also some structures and interfaces looked very similar to the structures and interfaces in DShowNet, and this confuses me at least. So, I could not make a decision which library can be used best and I made my own version. Because of that the code in the WMLib.cs might look quite similar to the Yeti version and the unofficial Sourceforge Windows Media library at cvs: directshownet/windowsmedialib, sourceforge.net.

WMProfileData

The class WMProfileData declares the attributes that holds the data of a profile. The application can use this data to retrieve for example the name of the profile, its description and some stream information.

public class WMProfileData
{
   ///  Name of the profile 
	protected string name;

	///  Guid of the profile 
	protected Guid guid;

	///  Description of the profile 
	protected string description;

	///  Audio bit rate 
	protected int audioBitrate;

	///  Video bit rate 
	protected int videoBitrate;

	///  Indicates whether this profile supports an audio stream 
	protected bool audio;

	///  Indicates whether this profile supports an video stream 
	protected bool video;

	///  Indicates whether this profile is the one currently in use 
	protected bool enabled;
}

AsfFormat

This class uses the WMProfileData class to build a list of profile information items. The next sample shows how the stream type and the bitrate is retrieved.

hr = profile.GetStreamCount(out streamCount);

if((hr >= 0)&&((streamCount > 0))
{
	IWMStreamConfig streamConfig = null;
	Guid streamGuid = Guid.Empty;
	audio = false;
	video = false;
	audioBitrate = 0;
	videoBitrate = 0;

	for(short i = 1;(i <= streamCount)&&(hr >= 0); i++)
	{
		hr = profile.GetStreamByNumber(i, out streamConfig);
		if((hr >= 0)&&(streamConfig != null))
	   {
			hr = streamConfig.GetStreamType(out streamGuid);
			if(hr >= 0)
			{
				if(streamGuid == MediaType.Video)
				{
					video = true;
					hr = streamConfig.GetBitrate(out videoBitrate);
					if(hr < 0)
					{
						videoBitrate = 0;
					}
				} 
				else
					if(streamGuid == MediaType.Audio)
				{
					audio = true;
					hr = streamConfig.GetBitrate(out audioBitrate);
					if(hr < 0)
					{
						audioBitrate = 0;
					}
				}
				hr = 0; // Allow possible unreadable bitrates
			}
		}
	} // for i
}

The AsfFormat class has a number of functions that can be used by the application. With the constructor the class can be initialized. With the function GetProfileFormatInfo(), the profile information can be updated (e.g. show all video formats, or show all video formats having no audio). This functionality is very useful if the application switches from audio file saving to video file saving. For video file saving the menu should show a list of profiles which do support at least a video stream. While for audio file saving the menu should show a list of profiles which support an audio stream and do not support a video stream.

AsfForm

The AsfForm class provides a form that provides (useful) information about the selected profile. In addition, the user is able to select a different profile. The class AsfForm is added as file AsfForm.cs to CaptureTest.

Code changes in Capture.cs

The following code shows the possible use of the AsfFormat constructor and the GetProfileFormatInfo() function in the Capture class.

switch(recFileMode)
{
	case RecFileModeType.Wmv:
		if(asfFormat == null)
		{
			asfFormat = new AsfFormat(AsfFormat.AsfFormatSelection.Video);
		} 
		else
		{
			asfFormat.GetProfileFormatInfo(AsfFormat.AsfFormatSelection.Video);
		}
		break;
	case RecFileModeType.Avi:
		break;
	default:
		// Unsupported file format
		return;
}

In addition, there is a one liner that changes the extension of the file name.

// Change filename extension
this.filename = Path.ChangeExtension(this.filename, RecFileMode.ToString().ToLower());

In the function renderGraph() the actual file saving is initialized. The original Avi file saving is still possible. To prevent problems with the Windows Media formats, the video compressor is used for Avi file saving only. For a Windows Media format, no video compressor will be used. For a Windows Media format the profile information must be retrieved to configure the Asf file writer. Furthermore, the stream information must be retrieved.

// Render the file writer portion of graph (mux -> file)

// Record captured audio/video in Avi, Wmv or Wma format
Guid mediaSubType; // Media sub type
bool captureAudio = true;
bool captureVideo = true;
IBaseFilter videoCompressorfilter = null;

// Set media sub type and video compressor filter if needed
if(RecFileMode == RecFileModeType.Avi)
{
	mediaSubType = MediaSubType.Avi;
	// For Avi file saving a video compressor must be used
	// If one is selected, that one will be used.
	videoCompressorfilter = videoCompressorFilter;
}
else
{
	mediaSubType = MediaSubType.Asf;
}

// Intialize the Avi or Asf file writer
hr = captureGraphBuilder.SetOutputFileName( ref mediaSubType, Filename, out muxFilter, out fileWriterFilter );
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

// For Wma (and Wmv) a suitable profile must be selected. This
// can be done via a property window, however the muxFilter is
// just created. if needed, the property windows should show up
// right now!
// Another solution is to configure the Asf file writer, the
// use interface must ensure the proper format has been selected.
if((RecFileMode == RecFileModeType.Wma)||
	(RecFileMode == RecFileModeType.Wmv))
{
	if(this.AsfFormat != null)
	{
		this.AsfFormat.UpdateAsfAVFormat(this.muxFilter);
		this.AsfFormat.GetCurrentAsfAVInfo(out captureAudio, out captureVideo);
	}
}

For saving video, the video stream check is performed. This check will prevent video file saving if an audio only file save was requested. For Avi only, the video compressor value will be used.

// Render video (video -> mux) if needed or possible
if((VideoDevice != null)&&(captureVideo))
{
	// Try interleaved first, because if the device supports it,
	// it's the only way to get audio as well as video
	cat = PinCategory.Capture;
	med = MediaType.Interleaved;
	hr = captureGraphBuilder.RenderStream( ref cat, ref med, videoDeviceFilter, videoCompressorfilter, muxFilter ); 
	if( hr < 0 ) 
	{
		med = MediaType.Video;
		hr = captureGraphBuilder.RenderStream( ref cat, ref med, videoDeviceFilter, videoCompressorfilter, muxFilter ); 
		if ( hr == -2147220969 ) throw new DeviceInUseException( "Video device", hr );
		if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );
	}
}

For saving audio, the audio stream check is performed. This check will prevent audio file saving if a video only file save was requested.

// Render audio (audio -> mux) if possible
if((AudioDevice != null)&&(captureAudio))
{
	// If this Asf file format than please keep in mind that
	// certain Wmv formats do not have an audio stream, so
	// when using this code, please ensure you use a format
	// which supports audio!
	cat = PinCategory.Capture;
	med = MediaType.Audio;
	hr = captureGraphBuilder.RenderStream( ref cat, ref med, audioDeviceFilter, audioCompressorFilter, muxFilter );
	if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );
}
isCaptureRendered = true;
didSomething = true;

Code changes in CaptureTest.cs

In CaptureTest.cs the updateMenu(). function needs to be modified for adding a menu with the possible audio/video recording modes. A new function menuAVRecFileModes_Click() is added, this function is called when the user selects an item in the menu.

menuAVRecFileModes.MenuItems.Clear();

// Fill in all file modes, use enumerations also as string (and file extension)
for(int i = 0; i < 3; i++)
{
	m = new MenuItem(((DirectX.Capture.Capture.RecFileModeType)i).ToString(), new EventHandler(menuAVRecFileModes_Click));
	m.Checked = (i == (int)capture.RecFileMode);
	menuAVRecFileModes.MenuItems.Add(m);
}
menuAVRecFileModes.Enabled = true;

Points of Interest

This example has additional support to get audible sound when using a TV-card that provides audio via the Pci bus (so no wired connection). For this the TV-card must have an audio driver. During testing I noticed that there were no audios sources listed. If the sources were available I noticed that the sources were invalid, so a source could not be selected. For the last problem I found two solutions. The first solution is to reload the AudioSources and VideoSources and then reload the AudioSource and VideoSource. The second solution is: do not release the crossbar object via a Marshal.ReleaseComObject in CrossbarSource.cs. Both solutions look bad, in this example the second solution is chosen because of its simplicity. To get the list of AudioSources, a few more modifications were needed. These modifications are needed only when AudioSources can not be found via the audio device or the audio device is found via the video device.

I did also some testing in saving files in Mpeg2 format. There are already some interesing articles on this subject, Preview and Record with MPEG2 Capture device and Working with Stream Buffer engine - TIME SHIFT on Windows XP Service Pack 1 written by BeCapture. The second article gives a very good explanation how the Mpeg2 demultiplexer should be configured. Still, I could not get the last example working. Looking to some problems I got, I am wondering if this is a working example. Nevertheless, these examples helped me to get a working C# solution. The concern I have is that the solution depends very much on the Mpeg2 encoder and decoder filters that are available. Graphedt is the big help to check whether a combinations may work, but still it is tough to get a good result. Sometimes a design seems to work but the file stays zero bytes ... Sometimes the property page information of a filter needs to be modified to get a good result. The easiest one to use for file saving, I found was a Nero audio/video encoder. Using the hardware Mpeg2 encoder is also not to difficult.More difficult is finding a proper dump filter. For the moment, I do not think an article on this subject will give new information. But I can imagin, that for pulling the pieces together, some help will be very useful. So, if you have questions on this subject, please let me know.

During testing I did some investigations on the good old GetCurFile() function that ispart of the (Asf) file writer filter. Via this function the filename and the video information can be retrieved. The filename is returned in a string, the video information in a AMMediaType structure. More information on AMMediaType can be found at Specifying Stream Formats. When checking the formatPtr, formatSize and formatType attributes, I found out that the information seems to be unusable. I checked this function because I had the idea that maybe I could access the VideoInfoHeader via this interface. That would simplify the code a little bit and the IWMWriter interface would not be needed. Unhappily, the format data does not look usable. For changing the filename in this example, the video information is not needed. I also noticed (on MSDN) that GetCurFile() might use a null pointer. According to me, using null instead of the AMMediaType structure with the formatPtr, looks much safer.

I noticed that the DirectX.Capture class example (so also my version of this example) gives usually a good picture when the program is started and the preview is selected first time. I do not know what is really causing the slow down of the video preview (usually a black picture). What I noticed is that problems occur when mediaControl.Stop is called. Actually the preview pin seems to be handled a little bit different then the capture pin. Using the capture pin instead of the preview pin, gives the best result. Here a sample of the original implementation for rendering video to the default video renderer.

cat = PinCategory.Preview;
med = MediaType.Video;
hr = captureGraphBuilder.RenderStream( ref cat, ref med, videoDeviceFilter, null, null ); 
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );
Unfortunately the file save functionality needs the capture pin. Using VMR9 for video rendering or using a lower video preview resolution, will quite often also give a good picture. Here an example for using the Video Mixing Renderer 9 as the video renderer.
cat = PinCategory.Preview;
med = MediaType.Video;
IBaseFilter VMRfilter = (IBaseFilter) new VideoMixingRenderer9();
hr = graphBuilder.AddFilter(VMRfilter, "Video mixing renderer 9");
if( hr < 0 )
{
	Marshal.ThrowExceptionForHR( hr );
}
hr = captureGraphBuilder.RenderStream(ref cat, ref med, videoDeviceFilter, null, VMRfilter); 
if( hr < 0 )
{
	Marshal.ThrowExceptionForHR( hr );
}
I did not find a solution that worked on all my capture cards (Hauppauge PVR150, Pinnacle PCTV, MX460 Vivo, Radeon 8500 Vivo). Esspecially the Hauppauge cards caused me problems. On one system it works great, on another system I usually get a black picture.

Feedback and improvements

I hope this code helps you in understanding the structure of the DirectX.Capture class. And I also hope I provided you an enhancement that might be useful to you. Feel free to post any comments and questions.

History

April 19, 2006 - First release

February 1, 2007: Added support for capturing audio via video device filter, added TV finetuning. DirectShow - TV finetuning using the IKsPropertySet in C# describes these modifications in more detail.

Back to the beginning.


Back to the Software Development page,
the Homepage or the Hobby activities page.


Contact me? Go back to/look at the Software Development page.
Date: February 5, 2007