[mythtvnz] partitions

[Sam Hadley-Jones]

> 
> I'm really surprised at your freeze up issues. I'm using ext3 and slow
> deletes and no longer have any playback issues.
> 
> Did you have a chance to run some performance tests on the raid5 array
> like I suggested as i'm wondering if you have other issues.
> 
> --------------------------------------------
> Steven Ellis - Technical Director
> OpenMedia Limited - The Home of myPVR
> email   - steven at openmedia.co.nz
> website - http://www.openmedia.co.nz
> 

Here's some test results. A word of warning to any newbies here that would
like to do similar benchmarking; dd is a very powerful command and can kill
data indiscriminately with the most minor typo with no
confirmation/warning. 

Just a recap, this is 5x 500GB Western Digital Green Power SATA drives in
RAID 5 as /dev/md0. Three are connected to the motherboard SATA connectors
and the remaining two to a PCIe SATA card. LVM storage group on top with a
single PV.  

mounted:
/dev/mapper/storage-main on /storage type reiserfs (rw)
/dev/mapper/storage-tv on /tv type xfs (rw,allocsize=512m)

Raw drive member of array:

sam at mediabox:~$ sudo dd if=/dev/sda of=/dev/null bs=1048576 count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 16.8546 s, 63.7 MB/s

Raw RAID5 array:

sam at mediabox:~$ sudo dd if=/dev/md0 of=/dev/null bs=1048576 count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 13.9099 s, 77.2 MB/s

LVM LV:

sam at mediabox:~$ sudo dd if=/dev/storage/main of=/dev/null bs=1048576
count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 21.7457 s, 49.4 MB/s

Filesystem on LV 'main' random file read:

sam at mediabox:~$ sudo dd if=/storage/videos/Movies/Gangs\ of\ New\ York.avi
of=/dev/null bs=1048576 count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 26.1345 s, 41.1 MB/s

Filesystem on LV 'main' write:

sam at mediabox:~$ sudo dd of=/storage/testfile if=/dev/zero bs=1048576
count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 27.2666 s, 39.4 MB/s

Deleting file on 'main':

sam at mediabox:~$ time sudo rm /storage/testfile 

real	0m1.260s
user	0m0.008s
sys	0m1.084s

Filesystem on LV 'tv' write:

sam at mediabox:~$ sudo dd of=/tv/testfile if=/dev/zero bs=1048576
count=10241024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 14.4985 s, 74.1 MB/s

Deleting file on 'tv':

sam at mediabox:~$ time sudo rm /tv/testfile

real	0m0.653s
user	0m0.000s
sys	0m0.488s

Filesystem on LV 'tv' random file read:

sam at mediabox:~$ sudo dd if=/tv/2920_20090514175900.mpg of=/dev/null
bs=1048576 count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 13.2303 s, 81.2 MB/s


Conclusion: Inconclusive. The last test showed a higher throughput on the
XFS filesystem than on the underlying LV container. I must admit, this is
not a very controlled test and the backend may have been accessing other
parts of the array at the same time.

That said, I think its safe to assume that the RAID 5 array is a serious
bottleneck. Theoretical speed of the array should be close to 240MB/s. I'm
thinking its either the RAID 5 parity computation or serious latency issues
between the motherboard controller and the PCIe controller.

Others may not see having high throughput as being important. However,
manipulating this much data, whether it be copying, hashing, etc, can take
hours if not days.

Sam.