[mythtvnz] cron.daily mythfilldatabase environment issue

[Paulgir]

On Fri, 11 Sep 2015 01:13:48 +1200, Stephen Worthington  
<stephen_agent at jsw.gen.nz> wrote:

> On Thu, 10 Sep 2015 20:47:58 +1200, you wrote:
>
>> I have been trying to setup a daily cron job to udpate my listings
>> from http://epg.org.nz/freeview.xml.gz
>>
>> My script runs fine if I run in manually but it fails when run from
>> cron with these errors:
>>
>> 2015-09-10 07:55:52.485142 W  This application expects to be running a
>> locale that specifies a UTF-8 codeset, and many features may behave
>> improperly with your current language settings. Please set the LC_ALL
>> or LC_CTYPE, and LANG variable(s) in the environment in which this
>> program is executed to include a UTF-8 codeset (such as
>> 'en_US.UTF-8').
>> 2015-09-10 07:55:52.485154 W  Cannot locate your home directory.
>> Please set the environment variable HOME
>> 2015-09-10 07:55:52.485158 E  Failed to init MythContext, exiting.
>>
>> Here is my script:
>> #! /bin/bash
>> echo "$(date) : starting epg import " >> /tmp/epggrubber.log
>> PATH=/usr/sbin:/usr/bin:/sbin:/bin
>> HOME=/home/mythtv
>> LANGUAGE=en_NZ:en
>> LANG=en_NZ.UTF-8
>> wget -O /tmp/freeview.xml.gz http://epg.org.nz/freeview.xml.gz >>
>> /tmp/epggrubber.log 2>&1
>> gunzip /tmp/freeview.xml.gz >> /tmp/epggrubber.log 2>&1
>> mythfilldatabase --update --file --sourceid 2 --xmlfile
>> /tmp/freeview.xml --update >> /tmp/epggrubber.log 2>&1
>> rm /tmp/freeview.xml
>> echo "$(date) : finished epg import " >> /tmp/epggrubber.log
>> exit
>>
>> As you can see I am setting the environment variables for home and  
>> language.
>>
>> Anyone know what I need to do to make it work?
>>
>> Cheers,
>> Daniel Hughes
>
> Doing this properly is a tricky problem that took me days to get
> working to my satisfaction.  So I will set out what I did and why.
>
> mythfilldatabase needs to be run by a mythtv group user.  If you want
> to run it from a cron job, then you need to run it from a user cron
> job, not a system one.  The easiest user to do this from is normally
> the one you use for mythfrontend.  The mythtv user is often not fully
> set up properly, and its crontab may not work.  Any cron job needs to
> have it full environment set up as part of the script that is being
> run, as it will not inherit any environment from cron.
>
> So, login as the mythfrontend user from ssh or a terminal window.  Put
> your script in say /home/<mythfrontend user>/bin.  Make sure your
> EDITOR variable points to your favourite command line editor (eg vi,
> nano), then run the command "crontab -e" which will bring up the user
> crontab for that user using $EDITOR.  Edit the crontab to add your
> script, save the crontab file and exit.
>
> Note that manually trying to edit the user crontab file does not work
> properly - you need to use the crontab command to do it.
>
> I use two scripts for doing my EPG.  By setting things up with the two
> scripts, the first script can log all the output of the second script
> to a log file, which makes it easy to see exactly what has gone wrong
> when the EPG processing fails.  My first script (epg_temp) is run by
> cron, and sets up the environment for the second script, and also
> rotates its log file.  The second (epg_temp.sh) does my EPG processing
> and runs mythfilldatabase.  The cron script (epg_temp) uses a helper
> script logrotate.sh that has been added to sudoers so that it can
> rotate the logs.
>
> I have put epg_temp and logrotate.sh on my web server so you can see
> how they work:
>
>   http://www.jsw.gen.nz/mythtv/epg_temp
>   http://www.jsw.gen.nz/mythtv/logrotate.sh
>
> The logrotate.sh file lives in my /home/stephen/bin directory, but is
> chown root:root in order to be able to access the log files in
> /var/log/mythtv.  You also need a logrotate.conf file in
> /home/<mythfrontenduser>/.logrotate, which also needs to be chown
> root:root:
>
>     http://www.jsw.gen.nz/mythtv/logrotate.conf
>
> There needs to be a file using the name of the mythfilldatabase user
> in /etc/sudoers.d to allow logrotate.sh to work.  This is what I have
> in my /etc/sudoers.d/stephen file:
>
> stephen ALL=NOPASSWD:/home/stephen/bin/logrotate.sh
>
> The sudoers file needs to be chown root:root and readonly for user and
> group only, or it will not work.  It can not have any write
> permissions at all.
>
> I have not posted my epg_temp.sh file, as it is too specific to how I
> do EPG and not a good example to follow (it really needs to be
> re-written).
>
> _______________________________________________
> mythtvnz mailing list
> mythtvnz at lists.linuxnut.co.nz
> http://lists.ourshack.com/mailman/listinfo/mythtvnz
> Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

I run the following:

#! /bin/bash
wget http://epg.org.nz/freeview.xml.gz
gzip -d freeview.xml.gz
mythfilldatabase --update --file --sourceid 1 --xmlfile freeview.xml
rm freeview.xml
exit

It's been working for several years

-Paul