[mythtvnz] No EPG, what am i doing wrong

[Curtis Walker]

On Wed, 6 Apr 2022 at 00:14, Stephen Worthington
<stephen_agent at jsw.gen.nz> wrote:
>
> On Tue, 5 Apr 2022 21:19:57 +1200, you wrote:
>
> >OK it's not my setup... the guide data on http://epg.org.nz/freeview.xml.gz
> >is old, most recent listing from the March 29th.
> >
> >So... what is everyone else doing for guide data?
> >
> >Regards,
> >Dan.
>
> You can use mhegepgsnoop to get the MHEG5 EPG data for you.  My
> updated Python 3 version of mhegepgsnoop is available from my web
> server:
>
> http://www.jsw.gen.nz/mythtv/mhegepgsnoop-0.7.1.py
>
> To install it, download it to /usr/local/bin and rename it to
> mhegepgsnoop.py, and do chmod +x.
>
> A stripped down version of the script I use to run mhegepgsnoop is
> here:
>
> http://www.jsw.gen.nz/mythtv/do_mhegepgsnoop.sh
>
> This downloads the EPG data and puts it a file listings-mheg5.xml on
> my internal web server.  I then do processing on the raw EPG data
> using xmltv-proc-nz to produce a fully processed version, also on my
> internal web server, as file freeview.xml.gz.  The processing done by
> xmltv-proc-nz is important as it fixes various problems with the EPG
> data and my version also includes Wade's code that processes the
> description data to extract the season and episode numbers where
> possible.  Hads wrote the original version of xmltv-proc-nz and it is
> what is used for the epg.org.nz data.
>
> My current version of xmltv-proc-nz is available here:
>
> http://www.jsw.gen.nz/mythtv/xmltv-proc-nz-jsw
>
> To use it, download it to /usr/local/bin, rename it to xmltv-proc-nz
> and do chmod +x.  Note that I have not gotten around to converting
> xmltv-proc-nz to Python 3 yet, so you will need to have both Python 2
> and Python 3 installed.
>
> The JSON data used by xmltv-proc-nz is here:
>
> http://www.jsw.gen.nz/mythtv/xmltv-proc-nz-json.tgz
>
> I have that installed on my internal web server - to use it that way,
> just go to the top level directory of your web server data and untar
> the xmltv-proc-nz-json.tgz file there.  It will create a json
> directory containing the data.  Then you may need change the ownership
> so that the web server owns that subdirectory, or just do:
>
> chmod -R a+r json
>
> I have put a modified version of do_mhegepgsnoop.py here:
>
> http://www.jsw.gen.nz/mythtv/do_mhegepgsnoop2.sh
>
> This has extra code to run xmltv-proc-nz on the EPG data and create a
> freeview.xml.gz file.  I have not tested those modifications, but they
> are based on what I do in my main EPG script, so they should work
> unless I have done some nasty typo.
>
> My modified version of tv_grab_nz-py downloads the freeview.xml.gz
> file from my internal web server instead of from epg.org.nz.  It is
> easy to find the place at the start of your tv_grab_nz-py where the
> download location is specified.  If you do not run your own internal
> web server, you should be able to use file:/// URLs instead of http://
> URLs and just have the EPG file somewhere on your MythTV box
> filesystem.  You will need to change the URLs in all the above
> scripts.  But I have never tried that, as I need my EPG data on a web
> server so that it can be downloaded from other PCs on my network that
> do TV recording.  And if you have MythWeb installed, you should
> already have a suitable Apache 2 web server installed that you can use
> for the EPG data.
>
> The do_mhegepgsnoop2.sh script will need to be run daily from a cron
> job or systemd timer unit, before the time that you have
> mythfilldatabase run automatically by mythbackend.  Alternatively, you
> can bypass using tv_grab_nz-py and just run mythfilldatabase from the
> end of the do_mhegepgsnoop2.sh script.  That would also bypass the
> need to use a web server.
>
> The other problem with using mhegepgsnoop.py is that it needs access
> to a DVB-T tuner whenever it is run.  I have not tested it for ages
> for this, but I think it will work if the tuner you tell it to use is
> already in use.  But it will then not change the tuner to the
> frequency specified and will get the EPG data from the frequency the
> tuner is already on.  That should not be a problem as the same EPG
> data is supposed to be sent on all frequencies.  I do not have this
> problem as I have an 8 DVB-T tuner card and can just assign one of
> them for EPG gathering and 5 others for MythTV to use for the 5 NZ
> muxes.
>
> If you decide to try using mhegepgsnoop.py, I am happy to help with
> any problems.  It may take a while to get it working, so if you want
> access to my EPG server in the mean time to download my EPG data, I
> have password access set up for that and if you (or anyone else) email
> me, I should be able set up a login for you.  To use my EPG data, you
> will need to be using the same xmltvid values in MythTV that I am for
> my channels.  Or you could write a sed script to change my xmltvids to
> yours.  To show what I am using, I have put the output of this SQL
> command:
>
> select
> chanid,channum,freqid,sourceid,callsign,name,xmltvid,mplexid,serviceid,visible,deleted
> from channel where sourceid=1 order by channum+0;
>
> on my web server here:
>
> http://www.jsw.gen.nz/mythtv/sourceid1.txt
>
> It would format well and would be very hard to read if I put it in
> this post.
>
> I should also be able to provide SQL to change your xmltvid values to
> mine, which will work if you have used the new MythTV scan code which
> updates the channel.name and channel.callsign values to match the data
> broadcast for the channel names.
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz at lists.ourshack.com
> https://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

Thanks Steven - this is helpful.

I'm giving it a go setting it all up using your files and instructions
(having used tv_grab_nz-py in years gone by, and relying on auto EIT
more recently, which seems to be broken as of 0.32).

One question: the do_mhegepgsnoop2.sh script refers to a channel map file:

# Channel map file.
MAP_FILE=/home/stephen/bin/mhegepgsnoop_channel_map.txt

What is this? And where might mine be?

Cheers,
Curtis