[mythtvnz] partitions

[Steven Ellis]

On 14/05/2009, at 8:26 PM, Sam Hadley-Jones wrote:

>>
>> I'm really surprised at your freeze up issues. I'm using ext3 and  
>> slow
>> deletes and no longer have any playback issues.
>>
>> Did you have a chance to run some performance tests on the raid5  
>> array
>> like I suggested as i'm wondering if you have other issues.
>>
>> --------------------------------------------
>> Steven Ellis - Technical Director
>> OpenMedia Limited - The Home of myPVR
>> email   - steven at openmedia.co.nz
>> website - http://www.openmedia.co.nz
>>
>
> Here's some test results. A word of warning to any newbies here that  
> would
> like to do similar benchmarking; dd is a very powerful command and  
> can kill
> data indiscriminately with the most minor typo with no
> confirmation/warning.
>
> Just a recap, this is 5x 500GB Western Digital Green Power SATA  
> drives in
> RAID 5 as /dev/md0. Three are connected to the motherboard SATA  
> connectors
> and the remaining two to a PCIe SATA card. LVM storage group on top  
> with a
> single PV.
>
> mounted:
> /dev/mapper/storage-main on /storage type reiserfs (rw)
> /dev/mapper/storage-tv on /tv type xfs (rw,allocsize=512m)
>
> Raw drive member of array:
>
> sam at mediabox:~$ sudo dd if=/dev/sda of=/dev/null bs=1048576 count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 16.8546 s, 63.7 MB/s
>
> Raw RAID5 array:
>
> sam at mediabox:~$ sudo dd if=/dev/md0 of=/dev/null bs=1048576 count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 13.9099 s, 77.2 MB/s
>
> LVM LV:
>
> sam at mediabox:~$ sudo dd if=/dev/storage/main of=/dev/null bs=1048576
> count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 21.7457 s, 49.4 MB/s
>
> Filesystem on LV 'main' random file read:
>
> sam at mediabox:~$ sudo dd if=/storage/videos/Movies/Gangs\ of\ New\  
> York.avi
> of=/dev/null bs=1048576 count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 26.1345 s, 41.1 MB/s
>
> Filesystem on LV 'main' write:
>
> sam at mediabox:~$ sudo dd of=/storage/testfile if=/dev/zero bs=1048576
> count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 27.2666 s, 39.4 MB/s
>
> Deleting file on 'main':
>
> sam at mediabox:~$ time sudo rm /storage/testfile
>
> real	0m1.260s
> user	0m0.008s
> sys	0m1.084s
>
> Filesystem on LV 'tv' write:
>
> sam at mediabox:~$ sudo dd of=/tv/testfile if=/dev/zero bs=1048576
> count=10241024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 14.4985 s, 74.1 MB/s
>
> Deleting file on 'tv':
>
> sam at mediabox:~$ time sudo rm /tv/testfile
>
> real	0m0.653s
> user	0m0.000s
> sys	0m0.488s
>
> Filesystem on LV 'tv' random file read:
>
> sam at mediabox:~$ sudo dd if=/tv/2920_20090514175900.mpg of=/dev/null
> bs=1048576 count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 13.2303 s, 81.2 MB/s
>
>
> Conclusion: Inconclusive. The last test showed a higher throughput  
> on the
> XFS filesystem than on the underlying LV container. I must admit,  
> this is
> not a very controlled test and the backend may have been accessing  
> other
> parts of the array at the same time.
>
> That said, I think its safe to assume that the RAID 5 array is a  
> serious
> bottleneck. Theoretical speed of the array should be close to 240MB/ 
> s. I'm
> thinking its either the RAID 5 parity computation or serious latency  
> issues
> between the motherboard controller and the PCIe controller.
>
> Others may not see having high throughput as being important. However,
> manipulating this much data, whether it be copying, hashing, etc,  
> can take
> hours if not days.
>

Don't expect to see 240MB/s. Your numbers are quite sane. In general I  
see raid 5 as 33% faster than the individual drives for a 4 drive array.

Even with hardware raid I seldom see things much faster.


Steven Ellis - Technical Director
OpenMedia Limited
email   - steven at openmedia.co.nz
website - http://www.openmedia.co.nz

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ourshack.com/pipermail/mythtvnz/attachments/20090514/ceeae0ff/attachment-0001.htm