Is there any know-how about creating filesystems on top of RAID1? Both XFS and ext4 top out at 35 MB/s for me even though the block device below it is capable of 135 MB/s (yes, 4x difference!) This even with an absurdly large I/O request sizes like 512KB.
The disks are old-ish SATA drives with 512-byte sectors. XFS seem to use 512b sectors while ext4 picked 4k. They're connected via USB DAS, with dm-integrity on top of each disk, which are then combined into mdadm raid1, and encrypted with LUKS.
I confirmed with fio that the slowdown happens on the filesystem level, not LUKS or anywhere below it.
The only advice I found online is about alignment, but with 512KB requests it shouldn't be an issue because that's way larger than any of the block/sector sizes involved. I must be missing something, but what is it?
#mdadm #raid #ext4 #xfs #AskFedi
Minoru
npub156…3qpa2
2025-01-10 18:38:01 UTC
Author Public Key
npub156gexupvlr084wdkwg0rr8dsctuzskra0f6mjmjae2mzwduqjjcsw3qpa2Seen on
wss://relay.momostr.pinkPublished at
2025-01-10 18:38:01 UTCEvent JSON
{
"id": "74149706e6cf3cc3ecf03e20e4e66671a58e44aa9a6fdceeb312154d1d2f965f",
"pubkey": "a69193702cf8de7ab9b6721e319db0c2f828587d7a75b96e5dcab627378094b1",
"created_at": 1736534281,
"kind": 1,
"tags": [
[
"t",
"raid"
],
[
"t",
"askfedi"
],
[
"t",
"mdadm"
],
[
"proxy",
"https://functional.cafe/@minoru/113805510681931812",
"web"
],
[
"t",
"xfs"
],
[
"t",
"ext4"
],
[
"proxy",
"https://functional.cafe/users/minoru/statuses/113805510681931812",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://functional.cafe/users/minoru/statuses/113805510681931812",
"pink.momostr"
],
[
"-"
]
],
"content": "Is there any know-how about creating filesystems on top of RAID1? Both XFS and ext4 top out at 35 MB/s for me even though the block device below it is capable of 135 MB/s (yes, 4x difference!) This even with an absurdly large I/O request sizes like 512KB.\n\nThe disks are old-ish SATA drives with 512-byte sectors. XFS seem to use 512b sectors while ext4 picked 4k. They're connected via USB DAS, with dm-integrity on top of each disk, which are then combined into mdadm raid1, and encrypted with LUKS.\n\nI confirmed with fio that the slowdown happens on the filesystem level, not LUKS or anywhere below it.\n\nThe only advice I found online is about alignment, but with 512KB requests it shouldn't be an issue because that's way larger than any of the block/sector sizes involved. I must be missing something, but what is it?\n\n#mdadm #raid #ext4 #xfs #AskFedi",
"sig": "cebb82034d72bd672e4d741b3a1ffb21fdc2505854cb55d6d91dade11bb977a95d7fc82cd29be06851df145e88e820c2c9e45f5faefaec08d52490c776f869ae"
}