[mythtvnz] Extracting EPG from sky

[Nick Rout]

Good expanation of DVB for the uninitiated, thanks David.

Back to EPG alternatives, there is a good screenscraper (ie it downloads web pages and analyses the content into xmltv data) at reven.co.nz.

It is designed for windows, but is written in .NET and runs under linux
with mono. I know it does because I had it working for a while. However
the data throughput is high (a lot of web pages to download) and it also
uses an inordinate amount of memory when running on my mythtv box. I
suspect that it opens all it's current data in RAM and then parses and
sorts it. Whatever the reason, it takes ages and sometimes doesn't
complete. I must try it again, it is often updated. I was being greedy
in requesting 14 days of data at a atime, the author recommends 5. It is
closed source, but the guy is friendly - friendly enough to tweak it to
make it run in mono when a couple of people asked. Unusually for a
(primarily) windows program it runs on the command line.

It also has a handy web based command line builder - choose your
channels, number of days and preferred sources on a web page and it
produces a command line to copy and paste into your terminal or conf
file.


On Fri, 03 Feb 2006 14:45:53 +1300
David Zanetti wrote:

> On Fri, 2006-02-03 at 14:17 +1300, Steve Hodge wrote:
> > On 2/3/06, David Zanetti <david.zanetti@catalyst.net.nz> wrote:
> > > FWIW, the DVB spec _does_ allow EPG data to be encrypted. Yes, it'll be
> > > a last step, but one they can probably do with little fanfare.
> > 
> > Video and audio decryption is offloaded, but what are the chances data
> > decryption can be handled the same way? I guess if the spec allows
> > encrypted EPG data there is a reasonable chance.
> 
> They're all just PIDs, so there's no technical difference.
> 
> Big bit of DVB coming, if you don't care for the details, please skip :)
> 
> Everything in the TS you get from the transponder (or a broadcast tower
> doing DVB-T, it's all the same) is broken down into seperate streams
> identified by PIDs. There's special PIDs like 0x0, containing the
> Programme Association Table (tells you where to find the PIDs describing
> end-user services), and 0x1 for the Conditional Access Table (what CAs
> are in use, and their PIDs for private data), and so forth. Video and
> audio are also PIDs, as is the guide data (0x12 for Event Information
> Table).
> 
> So the standard DVB CA interface is based around telling the decryption
> engine to decrypt several PIDs, and arranging the flow of those streams
> over the engine. It's possible the engine has a low limit for the number
> of PIDs it will decrypt at once, but it should certainly be greater than
> 2.
> 
> In the spec, everything except the system PIDs and tables can be
> encrypted. EIT is technically a system PID, but it's explicitly allowed.
> 
> IIRC. :)
> 
> -- 
> David Zanetti <david.zanetti@catalyst.net.nz>
> Team Leader, Systems Administration
> Catalyst IT Limited
> +64-4-8032233 +64-21-402260

-- 
Nick Rout <nick@rout.co.nz>