[mythtvnz] cron.daily mythfilldatabase environment issue

[Stephen Worthington]

On Thu, 10 Sep 2015 20:47:58 +1200, you wrote:

>I have been trying to setup a daily cron job to udpate my listings
>from http://epg.org.nz/freeview.xml.gz
>
>My script runs fine if I run in manually but it fails when run from
>cron with these errors:
>
>2015-09-10 07:55:52.485142 W  This application expects to be running a
>locale that specifies a UTF-8 codeset, and many features may behave
>improperly with your current language settings. Please set the LC_ALL
>or LC_CTYPE, and LANG variable(s) in the environment in which this
>program is executed to include a UTF-8 codeset (such as
>'en_US.UTF-8').
>2015-09-10 07:55:52.485154 W  Cannot locate your home directory.
>Please set the environment variable HOME
>2015-09-10 07:55:52.485158 E  Failed to init MythContext, exiting.
>
>Here is my script:
>#! /bin/bash
>echo "$(date) : starting epg import " >> /tmp/epggrubber.log
>PATH=/usr/sbin:/usr/bin:/sbin:/bin
>HOME=/home/mythtv
>LANGUAGE=en_NZ:en
>LANG=en_NZ.UTF-8
>wget -O /tmp/freeview.xml.gz http://epg.org.nz/freeview.xml.gz >>
>/tmp/epggrubber.log 2>&1
>gunzip /tmp/freeview.xml.gz >> /tmp/epggrubber.log 2>&1
>mythfilldatabase --update --file --sourceid 2 --xmlfile
>/tmp/freeview.xml --update >> /tmp/epggrubber.log 2>&1
>rm /tmp/freeview.xml
>echo "$(date) : finished epg import " >> /tmp/epggrubber.log
>exit
>
>As you can see I am setting the environment variables for home and language.
>
>Anyone know what I need to do to make it work?
>
>Cheers,
>Daniel Hughes

Doing this properly is a tricky problem that took me days to get
working to my satisfaction.  So I will set out what I did and why.

mythfilldatabase needs to be run by a mythtv group user.  If you want
to run it from a cron job, then you need to run it from a user cron
job, not a system one.  The easiest user to do this from is normally
the one you use for mythfrontend.  The mythtv user is often not fully
set up properly, and its crontab may not work.  Any cron job needs to
have it full environment set up as part of the script that is being
run, as it will not inherit any environment from cron.

So, login as the mythfrontend user from ssh or a terminal window.  Put
your script in say /home/<mythfrontend user>/bin.  Make sure your
EDITOR variable points to your favourite command line editor (eg vi,
nano), then run the command "crontab -e" which will bring up the user
crontab for that user using $EDITOR.  Edit the crontab to add your
script, save the crontab file and exit.

Note that manually trying to edit the user crontab file does not work
properly - you need to use the crontab command to do it.

I use two scripts for doing my EPG.  By setting things up with the two
scripts, the first script can log all the output of the second script
to a log file, which makes it easy to see exactly what has gone wrong
when the EPG processing fails.  My first script (epg_temp) is run by
cron, and sets up the environment for the second script, and also
rotates its log file.  The second (epg_temp.sh) does my EPG processing
and runs mythfilldatabase.  The cron script (epg_temp) uses a helper
script logrotate.sh that has been added to sudoers so that it can
rotate the logs.

I have put epg_temp and logrotate.sh on my web server so you can see
how they work:

  http://www.jsw.gen.nz/mythtv/epg_temp
  http://www.jsw.gen.nz/mythtv/logrotate.sh

The logrotate.sh file lives in my /home/stephen/bin directory, but is
chown root:root in order to be able to access the log files in
/var/log/mythtv.  You also need a logrotate.conf file in
/home/<mythfrontenduser>/.logrotate, which also needs to be chown
root:root:

    http://www.jsw.gen.nz/mythtv/logrotate.conf

There needs to be a file using the name of the mythfilldatabase user
in /etc/sudoers.d to allow logrotate.sh to work.  This is what I have
in my /etc/sudoers.d/stephen file:

stephen ALL=NOPASSWD:/home/stephen/bin/logrotate.sh

The sudoers file needs to be chown root:root and readonly for user and
group only, or it will not work.  It can not have any write
permissions at all.

I have not posted my epg_temp.sh file, as it is too specific to how I
do EPG and not a good example to follow (it really needs to be
re-written).