[mythtvnz] New install setup - recording problem

[Stephen Worthington]

On Fri, 26 Aug 2016 10:47:09 +1200, you wrote:

>On Thu, 25 Aug 2016 21:02:40 +1200, Stephen Worthington  
><stephen_agent at jsw.gen.nz> wrote:
>
>> I have now got my 16.04 test box running again, and spent some time
>> this afternoon looking at the log files.
>>
>> On Wed, 17 Aug 2016 09:40:37 +1200, you wrote:
>>
>>> On Tue, 16 Aug 2016 20:23:03 +1200, Stephen Worthington
>>> <stephen_agent at jsw.gen.nz> wrote:
>>>
>>>> On Sun, 14 Aug 2016 10:32:48 +1200, you wrote:
>>>>
>>>>> Here's a link to a text file with the log snippets you requested:
>>>>> https://drive.google.com/open?id=0B5RRWPLTmilhLUFhLTFkN3dEeUU
>>>>
>>>> I have finally had a free moment to start looking at the log files,
>>>> starting with the syslogs, and the thing that right away I can see as
>>>> a problem is that the hostname of the PC has changed from 12.04 (myth)
>>>> to 16.04 (mythpc).  That screws up the MythTV database quite badly, as
>>>> lots of settings are per hostname and many other tables have a
>>>> hostname field.  When changing the hostname of a PC, you have to
>>>> follow this procedure:
>>>
>>>
>>> My original install of 16.04 had the same hostname as the 12.04
>>> install.This was the one where I restored a DB backup from 12.04 onto
>>> 16.04.When I discovered that this led to a typo error in the command I
>>> used to change the permissions on the storage drive,I did a reinstall of
>>> 16.04 with a new hostname to avoid that mistake.
>>> On the new install I did not restore a 12.04 DB backup onto it.So the  
>>> logs
>>> you have seen are from a system with a pristine DB.
>>>
>>>
>>>>
>>>> https://www.mythtv.org/wiki/Hostname_change#Change_the_hostname_of_a_MythTV_frontend_or_backend
>>>>
>>>> and also see this:
>>>>
>>>> https://www.mythtv.org/wiki/Backend_migration
>>>>
>>>> I am not sure how well the hostname change options on
>>>> mythconverg_restore.pl work - I have never used them.  If they change
>>>> all the hostnames in all the tables, everything should work a lot
>>>> better than before.  I believe the right sequence would be to do a new
>>>> restore of the 12.04 database to the 16.04 setup, then run the
>>>> hostname change command, then run mythtv-setup to upgrade the database
>>>> schema, then start mythbackend and run mythfrontend to upgrade any
>>>> other plugin database that needed it.
>>>>
>>>> I think you said you are just installing 16.04 on a new SSD in the
>>>> same PC.
>>>
>>> That is correct.
>>>
>>>  If so, then it might be easier to just do a reinstall of
>>>> 16.04 keeping the old hostname, and then do an ordinary restore of the
>>>> database.  Once things are all working again, you could then look at
>>>> changing the hostname.
>>>
>>> As of yesterday ,I somehow killed the 16.04 system's ability to
>>> boot,setting up Myth Welcome.I ran boot repair but it did not correct  
>>> the
>>> problem.
>>> I could probably find the problem and fix it but I have decided to
>>> reinstall as that will be much quicker.
>>> I will use the same hostname as 12.04 ,in case I decide to do a DB  
>>> restore
>>> from 12.04.
>>> I will probably not do a restore,though as I don't have a huge number of
>>> recordings on 12.04 and may just move them to /Videos.
>>>
>>> I want to,initially, only install the HDHR tuners to see if I have the
>>> same start delay.
>>
>> The HDHR tuners will need a reasonable start delay as they need to
>> have networking fully up before they can be used.  This page gives how
>> to do it:
>>
>> https://www.mythtv.org/wiki/Systemd_mythbackend_Configuration
>>
>> in the "Delay starting the backend until network has initialized"
>> section.  But that information is generic, not specific to Ubuntu
>> 16.04.  What I think you need to do first is to try setting things up
>> so that you can wait for NetworkManager to say the network is up. Only
>> if that does not work should pingnetwork.service be needed.  So the
>> first thing to do is this command to see if the right NetworkMangager
>> target is enabled in systemd:
>>
>> systemctl status NetworkManager-wait-online.service
>>
>> If that does not show that the NetworkManager-wait-online.service is
>> enabled and working, then enable it:
>>
>> systemctl enable NetworkManager-wait-online.service
>>
>> Then mythbackend must be made to wait for that target.  In your
>> mythtv_backend_override.conf, in the [Unit] section, add a new line:
>>
>> After=NetworkManager-wait-online.service
>>
>>>>
>>>> However, that problem would not affect you if you were just running a
>>>> new database on 16.04, and I have not checked all the emails to see if
>>>> the latest tests (for the log files) were done with a clean database,
>>>
>>> Yes it was clean.
>>>
>>>> or using a restored one.  I may be able to work it out from the logs,
>>>> but please let me know if you can remember.  If the tests were done
>>>> with a database with wrong hostnames, there is not much point in
>>>> looking at the mythbackend logs until that problem is fixed.
>>>>
>>>> One other minor thing I can see is that the language locale seems to
>>>> be en_gb on the 12.04 setup, but en_us on the 16.04 one.  Since en_NZ
>>>> is a variant of en_gb, lots of things show en_gb when the locale is
>>>> set to en_NZ, so check the output of the "locale" command on 12.04 to
>>>> see if it is really set to en_NZ.  If so, 16.04 should be set up the
>>>> same way.  There can be problems with MythTV themes and other subtle
>>>> things if the locales do not match between the operating system and
>>>> the MythTV database.
>>>
>>> 12.04:
>>> ~$ locale
>>> LANG=en_NZ.UTF-8
>>> LANGUAGE=en_NZ:en
>>> LC_CTYPE="en_NZ.UTF-8"
>>> LC_NUMERIC="en_NZ.UTF-8"
>>> LC_TIME="en_NZ.UTF-8"
>>> LC_COLLATE="en_NZ.UTF-8"
>>> LC_MONETARY="en_NZ.UTF-8"
>>> LC_MESSAGES="en_NZ.UTF-8"
>>> LC_PAPER="en_NZ.UTF-8"
>>> LC_NAME="en_NZ.UTF-8"
>>> LC_ADDRESS="en_NZ.UTF-8"
>>> LC_TELEPHONE="en_NZ.UTF-8"
>>> LC_MEASUREMENT="en_NZ.UTF-8"
>>> LC_IDENTIFICATION="en_NZ.UTF-8"
>>> LC_ALL=
>>>
>>>
>>>>
>>>> When I have some more time I will look further at the logs.
>>>>
>>>> BTW My test PC is down for a few days - my mother's MythTV box power
>>>> supply fan suddenly became very noisy, so I had to use my spare power
>>>> supply from the test PC to keep her MythTV going until we can get a
>>>> new power supply for it.  So I am unable to do any testing with 16.04
>>>> at the moment.
>>
>> I think I know why systemd was timing out waiting for the /dev/dvb
>> devices.  It seems I forgot to tell you about creating an
>> /etc/udev/rules.d/99-mythbackend.rules file:
>>
>> #
>> # Create systemd device units for capture devices
>> #
>> SUBSYSTEM=="video4linux", TAG+="systemd"
>> SUBSYSTEM=="dvb", TAG+="systemd"
>> SUBSYSTEM=="firewire", TAG+="systemd"
>>
>> What that does is to make udev send systemd messages when those
>> devices are created.  Without it, systemd never received any messages
>> to tell it the devices were ready, hence the timeout.  I think I must
>> have created that file a while ago, so when I was working out how to
>> get systemd to wait for the DVB devices, I completely forgot it was
>> needed.  I really think that file should be installed automatically by
>> the MythTV packages.  Like the other files in that directory, it needs
>> to be chown root:root and chmod u=rw,g=r,o=r.
>>
>> With those changes, systemd should be able to bring up mythbackend
>> only after both the HVR-2200 and HDHR tuners are available.
>>
>> From the logs, I am pretty sure that the second firmware file for the
>> HVR-2200 tuners (dvb-fe-tda10048-1.0.fw) is only loaded after
>> mythbackend sends them a tuning command for the first time, so there
>> is no need to wait for that firmware to be loaded.  Doing the firmware
>> load means that the first tuning of the HVR-2200 tuners will likely
>> take a bit longer - it is possible that mythbackend may timeout
>> waiting for a response from the tuner and that may be why it seems to
>> try those tuners several times as it starts up:
>>
>> Aug 14 10:09:53 mythpc mythbackend: mythbackend[3168]: I CoreContext
>> recorders/dvbchannel.cpp:712 (Tune)
>> DVBChan[1](/dev/dvb/adapter0/frontend0): Next tuning after less than
>> 1000ms. Delaying by 1000ms
>> Aug 14 10:09:58 mythpc mythbackend: mythbackend[3168]: I CoreContext
>> recorders/dvbchannel.cpp:712 (Tune)
>> DVBChan[3](/dev/dvb/adapter1/frontend0): Next tuning after less than
>> 1000ms. Delaying by 1000ms
>> Aug 14 10:09:59 mythpc mythbackend: mythbackend[3168]: I CoreContext
>> recorders/dvbchannel.cpp:712 (Tune)
>> DVBChan[1](/dev/dvb/adapter0/frontend0): Next tuning after less than
>> 1000ms. Delaying by 1000ms
>> Aug 14 10:10:00 mythpc mythbackend: mythbackend[3168]: I CoreContext
>> recorders/dvbchannel.cpp:712 (Tune)
>> DVBChan[1](/dev/dvb/adapter0/frontend0): Next tuning after less than
>> 1000ms. Delaying by 1000ms
>> Aug 14 10:10:02 mythpc mythbackend: mythbackend[3168]: I CoreContext
>> recorders/dvbchannel.cpp:712 (Tune)
>> DVBChan[3](/dev/dvb/adapter1/frontend0): Next tuning after less than
>> 1000ms. Delaying by 1000ms
>> Aug 14 10:10:03 mythpc mythbackend: mythbackend[3168]: I CoreContext
>> recorders/dvbchannel.cpp:712 (Tune)
>> DVBChan[3](/dev/dvb/adapter1/frontend0): Next tuning after less than
>> 1000ms. Delaying by 1000ms
>>
>> That looks a bit worrying compared to what I see with my /dev/dvb
>> tuners, where each tuner gets tuned only once, in tuner number order.
>> But if the tuners are working afterward, that is all that matters.
>
>
>
>> I really appreciate your research into this problem.
>I did some searching on the forums to see what other commentaries on 16.04  
>were saying and found one that matched my experience very closely.One  
>major concern that has nagged at me is the way the set up pages seems to  
>behave when setting up capture cards.Even before the tuners are set  
>up,trying to get the "set up new capture card" page to open takes 3 to 5  
>minutes.
>To me this says there are bugs in this part of the software.A post on  
>Ubuntu forums describes the same behaviour and goes on to say that Mythtv  
>would lose the capture card setup every week and it would have to be  
>redone.That poster decided to use 14.04.
>I certainly don't want to risk introducing that kind of instability into  
>my PVR.
>
>So I decided to see how 14.04.5 (with mythtv 0.28) installs.This was  
>dream.It took about 3 hours to get up and running with no hardware related  
>problems.
>Interestingly, 14.04.5 boots as fast as 16.04 on the SSD - in about 18  
>seconds.So if the boot in 16.04 was too fast for the cards to  
>initialise,why is it not a problem for 14.04.5?
>
>I'm inclined to stick with 14.04 and hope 16.04 gets ironed out in the  
>next 2 years.

The use of systemd in 16.04 is what makes the difference in booting.
Systemd works completely differently from upstart, and allows things
to start better in parallel, rather than having to wait and start in
sequence.  But because of that, you can get race conditions if you do
not set up the startup conditions properly.  The mythbackend startup
conditions in the package as installed are only OK for systems that do
not have external access to MythTV and which have very standard
tuners.  Putting in the right conditions (which is where I hope we
have arrived at now) should allow mythbackend to start at the right
time.

The reports about mythtv-setup taking ages to get to the new capture
card page are likely what I saw when I first tried 0.28 on 16.04. That
is a problem with the code supporting the new VBox network tuners,
where the code involved takes a very long time to decide that there
are no VBoxes on the network.  In my case, that happened because I had
not set up my networking on my new test box at that point, so it was
not working properly.  Once I sorted out my network setup, the long
timeout waiting for VBox detection went away, although it still takes
longer than it used to.  The reason for the problem was quite visible
in the xterm window that pops up to run mythtv-setup.real - I could
see the messages about VBoxes when I swapped to that window while I
was waiting for mythtv-setup.real to open the new tuner page. VBox
support is new in 0.28 and I think this problem is just a bug, but I
have not checked to see if it has been reported yet.

I have not heard anything about 16.04 losing its capture card setup
though.  That is pretty major and I would have expected to see reports
about it on the mythtv-users mailing list by now, but there has been
nothing.

Using 14.04.05 and 0.28 should be fine for a while, if you want to do
that, but the kernel will not be updated as I think the 14.04.5 kernel
is the last one from 15.10, which is no longer supported.  The 14.04.0
kernel (3.13.x) is still being updated and will be as long as 14.04 is
being supported.  You would not be avoiding the VBox problem or the
possible loss of capture card setup if you use 0.28 on 14.04.  And as
you are using an SSD, 16.04 is recommended for better support for
that.  SSDs do work in 14.04, but you do not get all the latest tweaks
without a 4.x kernel.

What sort of SSD do you have?  In 14.04, there are problems running
the fstrim command or the fstab discard option (one or other of which
is necessary for using an SSD).  Only a limited range of SSDs are
supported, so you need to check if yours is supported.  This comment
is present in the /etc/cron.weekly/fstrim file in 14.04 and is gone in
16.04:

# This only runs on Intel and Samsung SSDs by default, as some SSDs
with faulty
# firmware may encounter data loss problems when running fstrim under
high I/O
# load (e. g.  https://launchpad.net/bugs/1259829). You can append the
# --no-model-check option here to disable the vendor check and run
fstrim on
# all SSD drives.