[mythtvnz] Integrating epgsnoop and mhegepgsnoop

[Robin Gilks]

Greetings all

Last weeks dropout of nzepg prompted me to look at mhegepgsnoop again - it
works great! I'm trying to merge its output with the DVB-S epgsnoop to
cover all my listings.

First off here is a patch to get the unicode standard the same in both
files...


--- storage/packages/mhegepgsnoop-0.5.1.py      2012-07-11
21:16:42.000000000 +1200
+++ /usr/local/bin/mhegepgsnoop-0.5.1.py        2013-03-19
23:44:49.000000000 +1300
@@ -1393,7 +1393,7 @@
 def map_channels():
        for line in chaninfo:
                if len(line) == 2:
-                       chan = unicode(line[0], "UTF-8")
+                       chan = unicode(line[0], "utf-8")
                        xmltvid = line[1]
                        channels.append([xmltvid, chan])
                        ch = unicodedata.normalize('NFKD',
chan).encode('ascii', 'replace')
@@ -1500,7 +1500,7 @@
        # Write the xml doc to disk
        outfile = open(OUT_FILE, "w")
        # Add manual declaration and doctype headers because it's tedious
to do any other way
-       outfile.write('<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE tv
SYSTEM "xmltv.dtd">')
+       outfile.write('<?xml version="1.0" encoding="utf-8"?><!DOCTYPE tv
SYSTEM "xmltv.dtd">')
        ET.ElementTree(root_element).write(outfile, encoding="utf-8")
        outfile.close()


and here is my script for getting and merging the two output files. It all
pretty much works as expected.


#!/bin/bash

export HOME="/home/mythtv"
export PATH=$PATH:/usr/local/bin:/usr/bin:/bin

LISTINGS_SKY="/tmp/listings-sky-$$.xml"
LISTINGS_DVBT="/tmp/listings-dvbt-$$.xml"
LISTINGS_ALL=$HOME/public_html/epg/listings-all.xml

epgsnoop --quiet --adapter=1 --lnb=11300 --tune=12671 --tune-retries=7
--config-dir=$HOME/.epgsnoop | iconv -c -f ISO_8859-1//TRANSLIT//IGNORE -t
UTF-8 -o $LISTINGS_SKY
mhegepgsnoop-0.5.1.py -d "/dev/dvb/adapter3/demux0" -p -f
$HOME/.mhegsnoop/channel_map -o $LISTINGS_DVBT

tv_cat --output $LISTINGS_ALL $LISTINGS_SKY $LISTINGS_DVBT
gzip -f $LISTINGS_ALL

rm -f $LISTINGS_SKY
rm -f $LISTINGS_DVBT


When I run tv_grab_nz-py I get the following error but no clue as to where
in the file it occurs :(

INFO - Downloading data from gilks.ath.cx...
INFO - Done
INFO - Parsing listings...
Traceback (most recent call last):
  File "/usr/local/bin/tv_grab_nz-py", line 278, in <module>
    time.strptime(element.get('start')[:-6], '%Y%m%d%H%M%S')
  File "/usr/lib64/python2.7/_strptime.py", line 454, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/usr/lib64/python2.7/_strptime.py", line 325, in _strptime
    (data_string, format))
ValueError: time data '20130320' does not match format '%Y%m%d%H%M%S'

Can someone assist in more debug to pin down where the error is?

Passing the xmltv file into mfdb manually for each of the sources works OK
and populates the listings, its just the top level grabber stuff is
failing.

Cheers

-- 
Robin Gilks