Mijn pagina voor prettig werken op de PC www.pcpret.nl

Homepage

Stuur bericht

Gastenboek

Website Blog

World standard teletext on a PC, the Windows way

• Introduction
• How to use
• Software
• WST codec or VBI codec?
• Video port (VP, VPVBI) pin
• WST in Amcap
• VBI codec format
• Packet 30
• WST in Windows Vista
• Problems
• Interesting links

How to use
When looking around in Directshow not much information can be found. With help of MSDN I found out that the VBI pin was involed, the term WST came up and than there was the IAMWstDecoder interface. Digging around in DirectShowLib (using just grep, the best tool on earth?), I found what I needed. Only I could not find any example code to start with on this subject. Than of course, graphedt is the second best tool on earth. I figured out that when adding the WST Decoder Directshow filter to the graph, the VBI codec was added automaticly. When I did not specify anything, the WST Renderer is added together with the WST codec and a color filter. There is a third way, just use the WST codec and WST decoder. These filters needs to be added first before it gives the graph that is wanted.
To create a WST (Teletext) filter graph with Renderstream() just do an Add filter for the WST decoder and do the Renderstream(). Then Teletext will become visible, also on graphedt, just run the graph. The initial page is 100, to select another page the property page needs to be modified. In graphedt this works with a right click on the WST decoder. In the DirectX.Capture some additional code needs to be added. When the WST decoder filter is added, the corresponding property can be found by 'addFromGraph' by calling 'addIfSupported'. These functions can be found in the file 'PropertyPageCollection.cs'.
WST renderer (default graphedt choice on Windows XP, and it is quite sure that Media Center is using it). WST renderer offers nice functionality as 'fastext' (the red, green, yellow and blue button on the remote control ...). Disadvantage of WST renderer is that the functionality in wstrender.ax is not documented. At least I could not find any information on that.

Software
Well, because there was no working example, this was real new development. At least for me. Adding the VBI and WST filters via RenderStream gave me a lot of insight information. Because the use overlay mixer problem became more complicated and I had to do a lot of searching (on MSDN and internet) how to get a working graph. Finally I got a working graph by adding the filters and connecting the filters with an own written functions. A little later I found out that also two calls with RenderStream() would do the job. Yeh, if it is working then it is easier to understand why it works. The special connection function I needed for connecting the overlay mixer with the video renderer, so it was not spilled time to write it.
Than of course I want to select my own pages and I did not want to use the Vfw oriented property page for that. I had to understand how to use SetCurrentPage in the IAMWstDecoder interface. Should the special 'pucPageData' pointer be intialized with a valid IntPtr? I found out that the null pointer (= IntPtr.Zero) was okay. Of course the page number and subcode needs to be specified to get the page wanted. But after filling in 101 (an valid page in my test scenario), nothing happened. By accident, I tried a different number, and that was 1000. In normal cases this is a wrong number for a Teletext Page, which is in the range from 100 till 899. However, it was the key to find the solution! It is a nice experience to figure that out, so you may ask me, but I do not write the solution down here.
Next it was nice to read the page from the WST decoder (via the 'GetCurrentPage', which seems to do that job). Than the page could be saved in a file and the contents might be used for any other purpose. The page numbers in the page could be used for surfing through the Teletext pages. Special interesting pages could be the EPG (or TV guide) information pages. Ofcourse also the Radio guide information pages can be read too! For that still the TV tuner is needed ... The Media Center does not cover that yet. Well the initial code did compile correctly, but did not work. When calling 'GetCurrentPage' a fatal crash occured. Digging around, showed me that the IntPtr that was causing the initial problems needed to be Marshaled, however, that was only a part of the problem. That IntPtr pointed to the memory the Page data would be stored (temporary). When I figured out how to convert from COM to C# and reverse, the code still failed. An interface error 'Invalid parameter' was given. After some more digging I got the idea that the pointer to the WstPage structure was probably not seen as the right pointer. Okay the final solution was also Marshaling the pointer to the structure. In the Problems chapter, more information can be found on this.

WST codec or VBI codec?
In the figure at the right a WST graph is shown for the Hauppauge PVR150 TV-card. This graph the VBI codec is used. In some cases the VBI codec is not available. In that case rendering of the WST decoder with RenderStream() fails and WST can not be used. In such case the WST codec can be used instead of the VBI codec. And maybe there are also other suitable codecs. During testing I encountered several strange problems when using RenderStream(). For that reason the filters were added and connected seperately because that seems to solve the problem.

Video port (VP, VPVBI) pin
The normal teletext graph will not work on a capture device with a video port and a VBI pin. The reason for this is that the video port is used different by the computer than the preview pin on a normal capture device. Looking around on MSDN provided me the solution: add the VBI surface allocator to the graph and connect this to the VPVBI pin of the capture device and the overlay mixer. In addition, the preview pin MUST be connected with the overlay pin via the input0 pin. In the figure at the right a WST graph is shown for a NVidia MX460 Vivo videocard.

WST in Amcap
November 2006, I tried to get WST working in the Amcap program (Delphi as well as the Windows C version). I thought this would be easier then the managed version I got working earlier. And yes, it was a little easier, but still there were some pitfalls. How to use the 'pucPageData' pointer? How to allocate memory for this pointer? Could it work with a normal character array? Would the 'pucPageData' pointer also loose its value as in the C# version? How to get access to the IAMWstDecoder interface? Would this WST implementation also crash the program upon releasing the graph when the graph was created by RenderStream?

The memory allocation was the most difficult part. Because the 'WstDecoder' filter is a COM object, the IMAlloc interface needs to be used. The straight array solution did not work, then some strugling with pointers to get the proper data readable.
In the Amcap (Delphi as well as C) the Amcap program crashes also when the graph is released. For the moment the Delphi version seems to be more stable. Also some stability problems seen when accessing the WST Decoder property page. A little bit strange because the use of FindInterface and QueryInterface is quite straight forward.

VBI Codec format
DirectShow offers the VBI Codec (and sometimes also the WST Codec) filter. It is quite obvious that the VBI/WST Codec filter should be connected with the capture device. How this should be used, what goes in, what comes out is not documented. With help of the Sample Grabber and the World System Teletext specifications in hand I started the investigation.
It is possible to imagin what goes in and what comes out. Going in are at least the video lines 0 - 22 and 320 - 355 which holds Teletext data lines. Coming out would be the Teletext data in bytes as docuented is the World System Teletext Specifications. Okay, the data does not come out that way but there is much resemblance. The filter sends a bunch of data, most are bytes with zero value. It would be nice to know if any error checking of data conversion was done already. Some checking is done, but most of the real data seems to be unchanged and some data is removed. The VBI codec sent packets of 860 bytes, the size is the same in all Windows versions I used for testing. The header and trailer have zero's, also somewhere in between the packet data there are zero's.
I analysed samples of the data and I noticed that there are several type of packets passing. Most of the packets are the normal Teletext packets 0-23. Also some 24, 26, 27, 30 and 31 packets are seen. Having the packets it was possible to read and to sort them. I re-used "old" software from previous development for rendering the Teletext. The first byte of a packet starts with framing code 0x64 (official it is 0xE4). Then the magazine and packet address bytes follows, after that the 40 bytes data. But it seems that some packets are a little shorter ...
Than the real testing testing could start on the other Windows versions. The software worked on Vista Home Premium but the did not work on Windows XP. In Windows XP the data format differs a bit. I do not know the reason for that, and of course it is not documented. The framing code has been replaced by a byte I do not know its purpose, maybe error checking. This is a pity because data consistency checking becomes less safe. Furthermore there is an extra byte added, probably error checking. I do not know if the packet size is 40 bytes or not. I have to find out ... for now it seems to be 40 bytes. And I got it working for Windows XP too. Next step is to get it work for Digital TV.
It took some effort but I am able to get Teletext pages out of a TS stream, a MPEG2 Demultiplexer or a dump file with VBI data or PES packets. The MPEG2 Demultiplexer seems to be a problem for getting Teletext. Most of the data is lost before it can be used. This has to do with the bad buffering the MPEG2 Demultiplexer seems to do. When a MPEG2 Demultiplexer is configured without PSI parser and Mpeg2 and Sections, then most of the Teletext data is there. Unfortunately still some data is missing. But there is another problem, without parser the Demultiplexer must be configured manually ... So for Digital TV this approach is not usable. The solution is to use a Teletext slicer that gets the Teletext data direct from the TS stream. The NRK page is an example of a Teletext page that gave from a TS stream saved in a file. The TS stream I had to analyse for a customer. During testing a Windows problem showed up. The fonts I wanted to use, were not loaded at all. According to Microsoft (found by Googling on the web) it seems to be a GDI+ error. It is strange error because the problem shows up in .Net and not in GDI applications. Looking to the type of file, I noticed that all .FON font files are ignored by .Net ... For the moment it is not a big problem, I use the Teletext font for testing.

Packet 30
Packet 30 format 1 and 2 are interesting packets to look at. The first byte specifies the format. The following six bytes specify the initial teletext page. It also specifies the magazine bits which needs to be used to complement the bits in packet 27. Packet 27 is used for fasttext, this packet holds the page numbers that could be selected quickly under the red, green, yellow and blue button of the remote control (at least that is the idea). This makes packet 30 essential for fasttext.
The format 1 packet provides the program identification data, the date and time, and the title of the current broadcasted TV program. The program identification could be used for automatic TV broadcast searching if the values are documented. For the Netherlands I found out that the first byte has value 0x8c and the value of the second byte seems to identify the channel. A value of 0x80 is 'Nederland 1', 0x40 is 'Nederland 2' and 0xC0 is 'Nederland 3'. The commercial broadcast stations have, of course, other values, eg 0x20 is 'RTL 4'. Interesting is that the time offset code seems to be wrong for all Dutch channels. The value specifies the number of half hours between local time and Co-ordinated Universal Time (UTC). The documentation is pretty clear on the meaning, usually less clear what the real bit sequence is ... It seems the bits needs to be reversed ... The first 6 bits (bits 0-5) gives the actually value, but bit 2 is the 1 value, not bit 0! Bit 6 indicates the number is a negative number if the bit is set to 1. For the Netherlands the value 4 came in, after bit reversal and shifting, the actual value becomes 8. Wow, that is three hours too much!
Packet 30 format 2 is a real challenge to analyse, esspecially the 36 bits part ...

WST in Windows Vista
Windows Vista does not support the WST Decoder anymore. As result Teletext for personal use is not possible unless the Media Center is available. An alternative is to write your own WST Decoder This is handy also for Windows XP, because the current WST Decoder implementation is not that useable. E.g. the WST Decoder does not support fasttext and only one page can be looked up at a time. Windows Vista supports the VBI Codec filter so that filter is used for testing.
Research, described in section VBI Codec format, showed that the Teletext data can be decoded the same as for Windows Media Center 2005. There is only one but, I tested this only on Windows Home Premium ... During testing an interesting problem showed up, that are the fonts, the fonts looks good but there there are lines in it. Probably new fonts will be needed. I used the Teletext, TeletextDH, TeletextLineDraw fonts, these fonts came from the Matrox Marvel software.

Problems
View a teletext page: The WST decoder offers a way to select a teletext page and display it via a videorenderer. It is undocumented how the page number should be put in the WSTPage structure. and After that it is obvious that SetCurrentPage() needs to be called. By trial and error, I found the answer how to fill in. Happily the 'pucPageData' parameter could be zeroed with IntPtr.Zero. The WST interface (tested with an C# example and the VBI codec), seems to be unstable. Forward searching of Teletext pages most of the times works. Backward searching takes much more time and sometimes just does not work (at least within minutes). This is strange because usually every 20 seconds a page will pass. Sometimes the software crashes, sometimes the removal of the filters cause problems.
Saving a teletext page: The WST decoder offers a solution for getting the teletext page out of the decoder and save it as real data on disk. But the big question was how to use? By trial and error the answers could be found. The original interface GetCurrentPage() in DirectShowLib did not work, the program crashed because a COM pointer was not initialiazed properly.

// DirectShowLib v1.2, v1.3
int GetCurrentPage([In, Out] WSTPage pWstPage);

// HV version
int GetCurrentPage([MarshalAs(UnmanagedType.Struct)] out WSTPage pWstPage);

/// 

/// From AM_WST_PAGE
/// 
[StructLayout(LayoutKind.Sequential)]
public struct WSTPage
{
   [MarshalAs(UnmanagedType.U4)]
   public int	dwPageNr ;
   [MarshalAs(UnmanagedType.U4)]
   public int	dwSubPageNr ;
   public IntPtr pucPageData; // BYTE	*
}

Interesting links
Interesting links, well: Microsoft MSDN, here you can find a lot of information on almost any subject related to Microsoft Windows.

Back to the beginning? Click here.

Back to the Software Development page,
the Homepage or the Hobby activities page.