Raid 配置的规划
安装 TiDB 集群的物理机配置如下：
| 组件 | 配置描述 |
|---|---|
| CPU | 2 * Intel Xeon Gold 5218R(2.1GHz, 20Core) |
| 内存 | 384GB |
| 系统盘 | 2 * 480GB SATA SSD RAID1 |
| 数据盘 | 14 * 1.92TB SATA SSD RAID10 |
用 14 * 1.92TB SATA SSD 组成 RAID10 后，减去 RAID 的元数据和 1024 的转换损失，实际共 12819716 MB 可用。因为所有的节点都配置了 SSD 磁盘，且数据规模较大，所以需要配置成 TiDB 和 TiKV 的混合部署模式。
目前我们共配置 80 VCPU 的主机，按 16 个 VCPU 一个组件，大概可以配置 5 个组件，每台物理主机共部署 4 个 TiKV 和 1 个 TiDB。按一个 TiKV 约 2 TB 的容量规划，配置一个 3 TB 的空间，保证空间占用在 80% 以下。所以 12819716 MB 的容量空间分成 4 个 3 TB 的空间，挂接在 /data{0123}，和 1 个 500 GB 的空间，挂接在 /deploy。
混合部署的 CPU 和内存配置，可以参考文档 https://docs.pingcap.com/zh/tidb/dev/hybrid-deployment-topology 。
在操作系统中可以使用 RAID 卡的命令行管理工具，实现在线的 RAID 重配置，免去了重启和控制台访问等问题。
目前 x86 服务器中，主要使用 Microchip 的 SmartROC 系列（收购 adaptec 公司后发布的产品）和 Broadcom 的 Megaraid 系列（收购 avago 公司后发布的产品)，以下以这两款 RAID 卡配置为例。
Microchip Adaptec RAID arcconf
RAID 卡背景信息
使用操作系统命令查看 RAID 卡型号是 ZTE 中兴服务器 R5300 G4 的 RAID 卡 SDPSA_L-18i
[host bin]$ sudo dmidecode |grep -i "product name"
Product Name: R5300 G4
Product Name: R5300G4
[host bin]$ sudo dmidecode |grep -i RAID
[host bin]$ sudo lspci |grep -i RAID
[host bin]$ sudo dmesg |grep -i RAID
[ 6.360527] smartpqi 0000:3c:00.0: added 14:1:0:0 4000000000000000 Direct-Access ZTE LOGICAL VOLUME SSDSmartPathCap- En- RAID-1(1+0)
[ 6.386016] smartpqi 0000:3c:00.0: added 14:1:0:1 4000000100000000 Direct-Access ZTE LOGICAL VOLUME SSDSmartPathCap- En- RAID-1(1+0)
[ 6.396947] smartpqi 0000:3c:00.0: added 14:2:0:0 0000000000000000 RAID ZTE SDPSA_L-18i 2G
[ 6.398251] scsi 14:2:0:0: RAID ZTE SDPSA_L-18i 2G 5.00 PQ: 0 ANSI: 5
通过搜索可以查到 SDPSA_L-18i 实际上是 Microchip 的 SmartROC3100 RAID 卡。
核对产品资料确认使用 arcconf 命令行工具可以配置 RAID。
https://ww1.microchip.com/downloads/en/DeviceDoc/00003022A.pdf
软件下载位置
https://storage.microsemi.com/en-us/speed/raid/storage_manager/arcconf_B25611_zip.php
参考产品手册进行配置
https://download.adaptec.com/pdfs/user_guides/adaptec_cli_smarthba_smartraid_11_22_ug.pdf
查看 RAID 卡和数据盘信息
主机配置有两个 vd，vd1 是 14 块盘配置的 RAID 10，需要删除 vd1。
[root@host /tmp/arcconf/linux_x64]# ./arcconf list 1
Controllers found: 1
----------------------------------------------------------------------
Controller information
----------------------------------------------------------------------
Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN
----------------------------------------------------------------------
Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F
----------------------------------------------------------------------
Array Information
----------------------------------------------------------------------
Array ID : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
Array 0 : Ok (SATA SSD, 915725 MB, 0 MB)
Array 1 : Ok (SATA SSD, 25639884 MB, 1 MB)
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical ID : Status (RAID, Interface, Size MB) Name
----------------------------------------------------------------------
Logical 0 : Optimal (1, Data, 457830 MB) os
Logical 1 : Optimal (10, Data, 12819716 MB) vd
----------------------------------------------------------------------
maxCache information
----------------------------------------------------------------------
Array ID : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
No maxCache device created
----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location]
----------------------------------------------------------------------
Physical 0,0 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)]
Physical 0,1 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)]
Physical 0,2 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)]
Physical 0,3 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)]
Physical 0,4 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)]
Physical 0,5 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)]
Physical 0,6 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)]
Physical 0,7 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)]
Physical 0,8 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)]
Physical 0,9 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)]
Physical 0,10 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)]
Physical 0,11 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)]
Physical 0,12 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)]
Physical 0,13 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)]
Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)]
Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)]
Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable]
Command completed successfully.
[root@host /tmp/arcconf/linux_x64]# ./arcconf getconfig 1 ld 1
Controllers found: 1
--------------------------------------------------------
Logical device information
--------------------------------------------------------
Logical Device number 1
Logical Device name : vd
Disk Name : /dev/sdb
Block Size of member drives : 512 Bytes
Array : 1
RAID level : 10
Status of Logical Device : Optimal
Size : 12819716 MB
Stripe-unit size : 256 KB
Full Stripe Size : 1792 KB
Interface Type : SATA SSD
Device Type : Data
Boot Type : None
Heads : 255
Sectors Per Track : 32
Cylinders : 65535
Caching : Enabled
Mount Points : Not Mounted
LD Acceleration Method : Controller Cache
Logical Device On Boot Connector : No
Volume Unique Identifier : 600508B1001CD9EDCFD7F2DC2F11B617
--------------------------------------------------------
Logical Device segment information
--------------------------------------------------------
Group 0, Segment 0 : Present (1831420MB, SATA, SSD, Connector:0, Device:0) 222638F89768
中间输出略
Group 6, Segment 1 : Present (1831420MB, SATA, SSD, Connector:3, Device:13) 22253910CA2E
Command completed successfully.
[root@host /tmp/arcconf/linux_x64]#
删除 VD1
删除 vd1，14 块盘是 ready 状态。
[root@host /tmp/arcconf/linux_x64]# ./arcconf delete 1 logicaldrive 1
Controllers found: 1
WARNING: Deleting this logical device will automatically delete array 1 because it is the only logical device present on that array.
All data in logical device 1 will be lost.
Delete the logical device?
Press y, then ENTER to continue or press ENTER to abort: y
Deleting: logical device 1 ("vd")
Command completed successfully.
[root@host /tmp/arcconf/linux_x64]# ./arcconf list 1
Controllers found: 1
----------------------------------------------------------------------
Controller information
----------------------------------------------------------------------
Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN
----------------------------------------------------------------------
Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F
----------------------------------------------------------------------
Array Information
----------------------------------------------------------------------
Array ID : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
Array 0 : Ok (SATA SSD, 915725 MB, 0 MB)
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical ID : Status (RAID, Interface, Size MB) Name
----------------------------------------------------------------------
Logical 0 : Optimal (1, Data, 457830 MB) os
----------------------------------------------------------------------
maxCache information
----------------------------------------------------------------------
Array ID : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
No maxCache device created
----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location]
----------------------------------------------------------------------
Physical 0,0 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)]
Physical 0,1 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)]
Physical 0,2 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)]
Physical 0,3 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)]
Physical 0,4 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)]
Physical 0,5 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)]
Physical 0,6 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)]
Physical 0,7 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)]
Physical 0,8 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)]
Physical 0,9 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)]
Physical 0,10 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)]
Physical 0,11 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)]
Physical 0,12 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)]
Physical 0,13 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)]
Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)]
Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)]
Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable]
Command completed successfully.
创建第一个 VD
使用 14 块创建 RAID 10 ,并划分出第一个 3070000 MB 的 LUN。
命令参数 |
值 |
|---|---|
arcconf create 1 logicaldrive |
在 控制器 1 上创建 LD |
name / VD 名字 |
data0 |
LDcache / 缓存启用 |
LON |
VD 容量 |
3070000 |
RAID 级别 |
10 |
磁盘从 0 到 13 |
0 0,0 1,0 2,0 3,0 4,0 5,0 6,0 7,0 8,0 9,0 10,0 11,0 12,0 13 |
[root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data0 LDcache LON 3070000 10 0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 10 0 11 0 12 0 13
Controllers found: 1
Do you want to add a logical device to the configuration?
Press y, then ENTER to continue or press ENTER to abort: y
Creating logical device: data0
Command completed successfully.
[root@host /tmp/arcconf/linux_x64]# ./arcconf list 1
Controllers found: 1
----------------------------------------------------------------------
Controller information
----------------------------------------------------------------------
Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN
----------------------------------------------------------------------
Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F
----------------------------------------------------------------------
Array Information
----------------------------------------------------------------------
Array ID : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
Array 0 : Ok (SATA SSD, 915725 MB, 0 MB)
Array 1 : Ok (SATA SSD, 25639884 MB, 19499435 MB)
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical ID : Status (RAID, Interface, Size MB) Name
----------------------------------------------------------------------
Logical 0 : Optimal (1, Data, 457830 MB) os
Logical 1 : Optimal (10, Data, 3070000 MB) data0
----------------------------------------------------------------------
maxCache information
----------------------------------------------------------------------
Array ID : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
No maxCache device created
----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location]
----------------------------------------------------------------------
Physical 0,0 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)]
Physical 0,1 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)]
Physical 0,2 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)]
Physical 0,3 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)]
Physical 0,4 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)]
Physical 0,5 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)]
Physical 0,6 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)]
Physical 0,7 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)]
Physical 0,8 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)]
Physical 0,9 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)]
Physical 0,10 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)]
Physical 0,11 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)]
Physical 0,12 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)]
Physical 0,13 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)]
Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)]
Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)]
Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable]
Command completed successfully.
[root@host /tmp/arcconf/linux_x64]# fdisk -l
Disk /dev/sda: 480.1 GB, 480070426624 bytes, 937637552 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 262144 bytes
Disk label type: dos
Disk identifier: 0x000a9444
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 937506479 468752216 83 Linux
/dev/sda2 937506480 937637551 65536 c W95 FAT32 (LBA)
Partition 2 does not start on physical sector boundary.
Disk /dev/sdb: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes
[root@host /tmp/arcconf/linux_x64]# parted -s -a optimal /dev/sdb mklabel gpt -- mkpart primary ext4 1 3200GB
Warning: The resulting partition is not properly aligned for best performance.
附： “Warning: The resulting partition is not properly aligned for best performance.” 处理方法
http://rainbow.chard.org/2013/01/30/how-to-align-partitions-for-best-performance-using-parted/
[root@host /tmp/arcconf/linux_x64]# cat /sys/block/sdb/queue/optimal_io_size
1835008
[root@host /tmp/arcconf/linux_x64]# cat /sys/block/sdb/alignment_offset
0
[root@host /tmp/arcconf/linux_x64]# parted -s -a optimal /dev/sdb mklabel gpt -- mkpart primary ext4 3584s 100%
[root@host /tmp/arcconf/linux_x64]# mkfs.ext4 /dev/sdb1
[root@host /tmp/arcconf/linux_x64]#
创建后续 VD
在 14 块创建 RAID 10 的 ARRAY 磁盘组继续划分出其他 LUN。
命令 |
注释 |
|---|---|
arcconf create 1 logicaldrive |
在 控制器 1 上创建 LD |
name /VD 名字 |
data1~data4 |
LDcache / 缓存启用 |
LON |
VD 容量 |
3070000 * 3 + 499000 * 1 |
RAID 级别 |
10 |
ARRAY / 现有 RAID 组 |
1（第 1 个 LD 创建时自动定义的 ARRAY） |
| noprompt | 不交互，建议删除操作不使用。 |
[root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data1 LDcache LON 3070000 10 ARRAY 1
Controllers found: 1
Do you want to add a logical device to the configuration?
Press y, then ENTER to continue or press ENTER to abort: y
Creating logical device: data1
Command completed successfully.
[root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data2 LDcache LON 3070000 10 ARRAY 1 noprompt
[root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data3 LDcache LON 3070000 10 ARRAY 1 noprompt
[root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data4 LDcache LON 499000 10 ARRAY 1 noprompt
[root@host /tmp/arcconf/linux_x64]# fdisk -l
Disk /dev/sda: 480.1 GB, 480070426624 bytes, 937637552 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 262144 bytes
Disk label type: dos
Disk identifier: 0x000a9444
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 937506479 468752216 83 Linux
/dev/sda2 937506480 937637551 65536 c W95 FAT32 (LBA)
Partition 2 does not start on physical sector boundary.
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.
Disk /dev/sdb: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes
Disk label type: gpt
Disk identifier: 08A14123-EF5C-4775-A0C0-2DE80827BD40
# Start End Size Type Name
1 3584 6287357439 3T Microsoft basic primary
Disk /dev/sdc: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes
Disk /dev/sdd: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes
Disk /dev/sde: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes
Disk /dev/sdf: 523.2 GB, 523239424000 bytes, 1021952000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes
后续分区格式化挂接略
Broadcom avago MegaRAID storcli
RAID 卡背景信息
使用操作系统命令查看 RAID 卡型号是超聚变（原华为 x86）服务器 2288H V5 的 OEM Megaraid RAID 卡
# dmidecode |grep -Ei "manufacturer|product name"
Manufacturer: XFUSION
Product Name: 2288H V5
# dmesg|grep -i megaraid
...
[ 7.846978] scsi host0: Avago SAS based MegaRAID driver
...
下载和安装 storcli 管理工具
-
在“DOWNLOADS”页签中单击“Management Software and Tools”。
-
打开管理软件和工具列表。
-
下载最新版本的“StorCLI”工具。
-
解压后得到适用于不同操作系统的工具包，安装 linux 的 rpm 包。
使用 storcli 管理工具
storcli 管理工具的使用教程较多，简单列举如下：
- 查看 RAID 卡配置和当前磁盘 RAID 配置
通常 v1 是数据磁盘。
./storcli64 /c0 show
./storcli64 /c0/v1 show all
- 删除 v1
./storcli64 /c0/v1 del
- 按需添加 v1 等多个 LUN
./storcli64 /c0 add vd r10 size=2000GB drives=252:0-13