[mythtvnz] No EPG, what am i doing wrong

[Stephen Worthington]

On Tue, 5 Apr 2022 21:19:57 +1200, you wrote:

>OK it's not my setup... the guide data on http://epg.org.nz/freeview.xml.gz
>is old, most recent listing from the March 29th.
>
>So... what is everyone else doing for guide data?
>
>Regards,
>Dan.

You can use mhegepgsnoop to get the MHEG5 EPG data for you.  My
updated Python 3 version of mhegepgsnoop is available from my web
server:

http://www.jsw.gen.nz/mythtv/mhegepgsnoop-0.7.1.py

To install it, download it to /usr/local/bin and rename it to
mhegepgsnoop.py, and do chmod +x.

A stripped down version of the script I use to run mhegepgsnoop is
here:

http://www.jsw.gen.nz/mythtv/do_mhegepgsnoop.sh

This downloads the EPG data and puts it a file listings-mheg5.xml on
my internal web server.  I then do processing on the raw EPG data
using xmltv-proc-nz to produce a fully processed version, also on my
internal web server, as file freeview.xml.gz.  The processing done by
xmltv-proc-nz is important as it fixes various problems with the EPG
data and my version also includes Wade's code that processes the
description data to extract the season and episode numbers where
possible.  Hads wrote the original version of xmltv-proc-nz and it is
what is used for the epg.org.nz data.

My current version of xmltv-proc-nz is available here:

http://www.jsw.gen.nz/mythtv/xmltv-proc-nz-jsw

To use it, download it to /usr/local/bin, rename it to xmltv-proc-nz
and do chmod +x.  Note that I have not gotten around to converting
xmltv-proc-nz to Python 3 yet, so you will need to have both Python 2
and Python 3 installed.

The JSON data used by xmltv-proc-nz is here:

http://www.jsw.gen.nz/mythtv/xmltv-proc-nz-json.tgz

I have that installed on my internal web server - to use it that way,
just go to the top level directory of your web server data and untar
the xmltv-proc-nz-json.tgz file there.  It will create a json
directory containing the data.  Then you may need change the ownership
so that the web server owns that subdirectory, or just do:

chmod -R a+r json

I have put a modified version of do_mhegepgsnoop.py here:

http://www.jsw.gen.nz/mythtv/do_mhegepgsnoop2.sh

This has extra code to run xmltv-proc-nz on the EPG data and create a
freeview.xml.gz file.  I have not tested those modifications, but they
are based on what I do in my main EPG script, so they should work
unless I have done some nasty typo.

My modified version of tv_grab_nz-py downloads the freeview.xml.gz
file from my internal web server instead of from epg.org.nz.  It is
easy to find the place at the start of your tv_grab_nz-py where the
download location is specified.  If you do not run your own internal
web server, you should be able to use file:/// URLs instead of http://
URLs and just have the EPG file somewhere on your MythTV box
filesystem.  You will need to change the URLs in all the above
scripts.  But I have never tried that, as I need my EPG data on a web
server so that it can be downloaded from other PCs on my network that
do TV recording.  And if you have MythWeb installed, you should
already have a suitable Apache 2 web server installed that you can use
for the EPG data.

The do_mhegepgsnoop2.sh script will need to be run daily from a cron
job or systemd timer unit, before the time that you have
mythfilldatabase run automatically by mythbackend.  Alternatively, you
can bypass using tv_grab_nz-py and just run mythfilldatabase from the
end of the do_mhegepgsnoop2.sh script.  That would also bypass the
need to use a web server.

The other problem with using mhegepgsnoop.py is that it needs access
to a DVB-T tuner whenever it is run.  I have not tested it for ages
for this, but I think it will work if the tuner you tell it to use is
already in use.  But it will then not change the tuner to the
frequency specified and will get the EPG data from the frequency the
tuner is already on.  That should not be a problem as the same EPG
data is supposed to be sent on all frequencies.  I do not have this
problem as I have an 8 DVB-T tuner card and can just assign one of
them for EPG gathering and 5 others for MythTV to use for the 5 NZ
muxes.

If you decide to try using mhegepgsnoop.py, I am happy to help with
any problems.  It may take a while to get it working, so if you want
access to my EPG server in the mean time to download my EPG data, I
have password access set up for that and if you (or anyone else) email
me, I should be able set up a login for you.  To use my EPG data, you
will need to be using the same xmltvid values in MythTV that I am for
my channels.  Or you could write a sed script to change my xmltvids to
yours.  To show what I am using, I have put the output of this SQL
command:

select
chanid,channum,freqid,sourceid,callsign,name,xmltvid,mplexid,serviceid,visible,deleted
from channel where sourceid=1 order by channum+0;

on my web server here:

http://www.jsw.gen.nz/mythtv/sourceid1.txt

It would format well and would be very hard to read if I put it in
this post.

I should also be able to provide SQL to change your xmltvid values to
mine, which will work if you have used the new MythTV scan code which
updates the channel.name and channel.callsign values to match the data
broadcast for the channel names.