0
0
0
0
专栏/.../

物理机安装 TiKV 时 RAID 卡在线配置方式

 pepezzzz  发表于  2023-03-10

Raid 配置的规划

安装 TiDB 集群的物理机配置如下:

组件 配置描述
CPU 2 * Intel Xeon Gold 5218R(2.1GHz, 20Core)
内存 384GB
系统盘 2 * 480GB SATA SSD RAID1
数据盘 14 * 1.92TB SATA SSD RAID10

用 14 * 1.92TB SATA SSD 组成 RAID10 后,减去 RAID 的元数据和 1024 的转换损失,实际共 12819716 MB 可用。因为所有的节点都配置了 SSD 磁盘,且数据规模较大,所以需要配置成 TiDB 和 TiKV 的混合部署模式。

目前我们共配置 80 VCPU 的主机,按 16 个 VCPU 一个组件,大概可以配置 5 个组件,每台物理主机共部署 4 个 TiKV 和 1 个 TiDB。按一个 TiKV 约 2 TB 的容量规划,配置一个 3 TB 的空间,保证空间占用在 80% 以下。所以 12819716 MB 的容量空间分成 4 个 3 TB 的空间,挂接在 /data{0123},和 1 个 500 GB 的空间,挂接在 /deploy。

混合部署的 CPU 和内存配置,可以参考文档 https://docs.pingcap.com/zh/tidb/dev/hybrid-deployment-topology

在操作系统中可以使用 RAID 卡的命令行管理工具,实现在线的 RAID 重配置,免去了重启和控制台访问等问题。

目前 x86 服务器中,主要使用 Microchip 的 SmartROC 系列(收购 adaptec 公司后发布的产品)和 Broadcom 的 Megaraid 系列(收购 avago 公司后发布的产品),以下以这两款 RAID 卡配置为例。

Microchip Adaptec RAID arcconf 

RAID 卡背景信息

使用操作系统命令查看 RAID 卡型号是 ZTE 中兴服务器 R5300 G4 的 RAID 卡 SDPSA_L-18i

[host bin]$ sudo dmidecode  |grep -i "product name"
        Product Name: R5300 G4
        Product Name: R5300G4
[host bin]$ sudo dmidecode |grep -i  RAID
[host bin]$ sudo lspci |grep -i RAID
[host bin]$ sudo dmesg |grep -i RAID
[    6.360527] smartpqi 0000:3c:00.0: added 14:1:0:0 4000000000000000 Direct-Access     ZTE      LOGICAL VOLUME   SSDSmartPathCap- En- RAID-1(1+0) 
[    6.386016] smartpqi 0000:3c:00.0: added 14:1:0:1 4000000100000000 Direct-Access     ZTE      LOGICAL VOLUME   SSDSmartPathCap- En- RAID-1(1+0) 
[    6.396947] smartpqi 0000:3c:00.0: added 14:2:0:0 0000000000000000 RAID              ZTE      SDPSA_L-18i 2G   
[    6.398251] scsi 14:2:0:0: RAID              ZTE      SDPSA_L-18i 2G   5.00 PQ: 0 ANSI: 5

通过搜索可以查到 SDPSA_L-18i 实际上是 Microchip 的 SmartROC3100 RAID 卡。

https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=vsanio&productid=50320&vcl=true

核对产品资料确认使用 arcconf 命令行工具可以配置 RAID。

https://ww1.microchip.com/downloads/en/DeviceDoc/00003022A.pdf

软件下载位置

https://storage.microsemi.com/en-us/speed/raid/storage_manager/arcconf_B25611_zip.php

参考产品手册进行配置

https://download.adaptec.com/pdfs/user_guides/adaptec_cli_smarthba_smartraid_11_22_ug.pdf

查看 RAID 卡和数据盘信息

主机配置有两个 vd,vd1 是 14 块盘配置的 RAID 10,需要删除 vd1。


[root@host /tmp/arcconf/linux_x64]#  ./arcconf list 1
Controllers found: 1
----------------------------------------------------------------------
Controller information
----------------------------------------------------------------------
   Controller ID             : Status, Slot, Mode, Name, SerialNumber, WWN
----------------------------------------------------------------------
   Controller 1:             : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F

----------------------------------------------------------------------
Array Information
----------------------------------------------------------------------
   Array ID                  : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
   Array 0                   : Ok (SATA SSD, 915725 MB, 0 MB)
   Array 1                   : Ok (SATA SSD, 25639884 MB, 1 MB)

----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
   Logical ID                : Status (RAID, Interface, Size MB) Name
----------------------------------------------------------------------
   Logical 0                 : Optimal (1, Data, 457830 MB) os
   Logical 1                 : Optimal (10, Data, 12819716 MB) vd

----------------------------------------------------------------------
maxCache information
----------------------------------------------------------------------
   Array ID                  : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
   No maxCache device created

----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
   Physical ID               : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location]
----------------------------------------------------------------------
   Physical 0,0              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)]
   Physical 0,1              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)]
   Physical 0,2              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)]
   Physical 0,3              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)]
   Physical 0,4              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)]
   Physical 0,5              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)]
   Physical 0,6              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)]
   Physical 0,7              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)]
   Physical 0,8              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)]
   Physical 0,9              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)]
   Physical 0,10             : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)]
   Physical 0,11             : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)]
   Physical 0,12             : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)]
   Physical 0,13             : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)]
   Physical 0,16             : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)]
   Physical 0,17             : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)]
   Physical 2,0              : Ready (SES2, Not Applicable, Not Applicable, ZTE     , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable]


Command completed successfully.


[root@host /tmp/arcconf/linux_x64]#  ./arcconf getconfig 1 ld 1
Controllers found: 1
--------------------------------------------------------
Logical device information
--------------------------------------------------------
Logical Device number 1
   Logical Device name                        : vd
   Disk Name                                  : /dev/sdb
   Block Size of member drives                : 512 Bytes
   Array                                      : 1
   RAID level                                 : 10
   Status of Logical Device                   : Optimal
   Size                                       : 12819716 MB
   Stripe-unit size                           : 256 KB
   Full Stripe Size                           : 1792 KB
   Interface Type                             : SATA SSD
   Device Type                                : Data
   Boot Type                                  : None
   Heads                                      : 255
   Sectors Per Track                          : 32
   Cylinders                                  : 65535
   Caching                                    : Enabled
   Mount Points                               : Not Mounted
   LD Acceleration Method                     : Controller Cache
   Logical Device On Boot Connector           : No
   Volume Unique Identifier                   : 600508B1001CD9EDCFD7F2DC2F11B617
--------------------------------------------------------
   Logical Device segment information
--------------------------------------------------------
   Group 0, Segment 0                         : Present (1831420MB, SATA, SSD, Connector:0, Device:0) 222638F89768
 中间输出略
   Group 6, Segment 1                         : Present (1831420MB, SATA, SSD, Connector:3, Device:13) 22253910CA2E




Command completed successfully.
[root@host /tmp/arcconf/linux_x64]#  

删除 VD1

删除 vd1,14 块盘是 ready 状态。



[root@host /tmp/arcconf/linux_x64]#  ./arcconf delete 1 logicaldrive 1
Controllers found: 1

WARNING: Deleting this logical device will automatically delete array 1 because it is the only logical device present on that array.
All data in logical device 1 will be lost.
Delete the logical device?
Press y, then ENTER to continue or press ENTER to abort: y

Deleting: logical device 1 ("vd")

Command completed successfully.


[root@host /tmp/arcconf/linux_x64]#  ./arcconf list 1
Controllers found: 1
----------------------------------------------------------------------
Controller information
----------------------------------------------------------------------
   Controller ID             : Status, Slot, Mode, Name, SerialNumber, WWN
----------------------------------------------------------------------
   Controller 1:             : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F

----------------------------------------------------------------------
Array Information
----------------------------------------------------------------------
   Array ID                  : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
   Array 0                   : Ok (SATA SSD, 915725 MB, 0 MB)

----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
   Logical ID                : Status (RAID, Interface, Size MB) Name
----------------------------------------------------------------------
   Logical 0                 : Optimal (1, Data, 457830 MB) os

----------------------------------------------------------------------
maxCache information
----------------------------------------------------------------------
   Array ID                  : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
   No maxCache device created

----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
   Physical ID               : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location]
----------------------------------------------------------------------
   Physical 0,0              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)]
   Physical 0,1              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)]
   Physical 0,2              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)]
   Physical 0,3              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)]
   Physical 0,4              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)]
   Physical 0,5              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)]
   Physical 0,6              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)]
   Physical 0,7              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)]
   Physical 0,8              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)]
   Physical 0,9              : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)]
   Physical 0,10             : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)]
   Physical 0,11             : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)]
   Physical 0,12             : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)]
   Physical 0,13             : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)]
   Physical 0,16             : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)]
   Physical 0,17             : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)]
   Physical 2,0              : Ready (SES2, Not Applicable, Not Applicable, ZTE     , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable]


Command completed successfully.

创建第一个 VD

使用 14 块创建 RAID 10 ,并划分出第一个 3070000 MB 的 LUN。

命令参数

arcconf create 1 logicaldrive

在 控制器 1 上创建 LD

name / VD 名字

data0

LDcache / 缓存启用

LON

VD 容量

3070000

RAID 级别

10

磁盘从 0 到 13

0 0,0 1,0 2,0 3,0 4,0 5,0 6,0 7,0 8,0 9,0 10,0 11,0 12,0 13


[root@host /tmp/arcconf/linux_x64]#  ./arcconf create 1 logicaldrive name data0 LDcache LON 3070000 10 0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 10 0 11 0 12 0 13
Controllers found: 1

Do you want to add a logical device to the configuration?
Press y, then ENTER to continue or press ENTER to abort: y

Creating logical device: data0

Command completed successfully.


[root@host /tmp/arcconf/linux_x64]#  ./arcconf list 1
Controllers found: 1
----------------------------------------------------------------------
Controller information
----------------------------------------------------------------------
   Controller ID             : Status, Slot, Mode, Name, SerialNumber, WWN
----------------------------------------------------------------------
   Controller 1:             : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F

----------------------------------------------------------------------
Array Information
----------------------------------------------------------------------
   Array ID                  : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
   Array 0                   : Ok (SATA SSD, 915725 MB, 0 MB)
   Array 1                   : Ok (SATA SSD, 25639884 MB, 19499435 MB)

----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
   Logical ID                : Status (RAID, Interface, Size MB) Name
----------------------------------------------------------------------
   Logical 0                 : Optimal (1, Data, 457830 MB) os
   Logical 1                 : Optimal (10, Data, 3070000 MB) data0

----------------------------------------------------------------------
maxCache information
----------------------------------------------------------------------
   Array ID                  : Status (Interface, TotalSize MB, FreeSpace MB)
----------------------------------------------------------------------
   No maxCache device created

----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
   Physical ID               : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location]
----------------------------------------------------------------------
   Physical 0,0              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)]
   Physical 0,1              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)]
   Physical 0,2              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)]
   Physical 0,3              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)]
   Physical 0,4              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)]
   Physical 0,5              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)]
   Physical 0,6              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)]
   Physical 0,7              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)]
   Physical 0,8              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)]
   Physical 0,9              : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)]
   Physical 0,10             : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)]
   Physical 0,11             : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)]
   Physical 0,12             : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)]
   Physical 0,13             : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)]
   Physical 0,16             : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)]
   Physical 0,17             : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)]
   Physical 2,0              : Ready (SES2, Not Applicable, Not Applicable, ZTE     , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable]


Command completed successfully.
[root@host /tmp/arcconf/linux_x64]#  fdisk -l

Disk /dev/sda: 480.1 GB, 480070426624 bytes, 937637552 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 262144 bytes
Disk label type: dos
Disk identifier: 0x000a9444

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   937506479   468752216   83  Linux
/dev/sda2       937506480   937637551       65536    c  W95 FAT32 (LBA)
Partition 2 does not start on physical sector boundary.

Disk /dev/sdb: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes

[root@host /tmp/arcconf/linux_x64]#  parted -s -a optimal /dev/sdb mklabel gpt -- mkpart primary ext4 1 3200GB
Warning: The resulting partition is not properly aligned for best performance.

附: “Warning: The resulting partition is not properly aligned for best performance.” 处理方法

http://rainbow.chard.org/2013/01/30/how-to-align-partitions-for-best-performance-using-parted/

[root@host /tmp/arcconf/linux_x64]#  cat /sys/block/sdb/queue/optimal_io_size 
1835008
[root@host /tmp/arcconf/linux_x64]#  cat /sys/block/sdb/alignment_offset 
0
[root@host /tmp/arcconf/linux_x64]#  parted -s -a optimal /dev/sdb mklabel gpt -- mkpart primary ext4 3584s 100%

[root@host /tmp/arcconf/linux_x64]#  mkfs.ext4 /dev/sdb1

[root@host /tmp/arcconf/linux_x64]# 

创建后续 VD

在 14 块创建 RAID 10 的 ARRAY 磁盘组继续划分出其他 LUN。

命令

注释

arcconf create 1 logicaldrive

在 控制器 1 上创建 LD

name /VD 名字

data1~data4

LDcache / 缓存启用

LON

VD 容量

3070000 * 3 + 499000 * 1

RAID 级别

10

ARRAY / 现有 RAID 组

1(第 1 个 LD 创建时自动定义的 ARRAY)

noprompt 不交互,建议删除操作不使用。

[root@host /tmp/arcconf/linux_x64]#  ./arcconf create 1 logicaldrive name data1 LDcache LON 3070000 10  ARRAY 1
Controllers found: 1

Do you want to add a logical device to the configuration?
Press y, then ENTER to continue or press ENTER to abort: y

Creating logical device: data1

Command completed successfully.
[root@host /tmp/arcconf/linux_x64]#  ./arcconf create 1 logicaldrive name data2 LDcache LON 3070000 10  ARRAY 1 noprompt

[root@host /tmp/arcconf/linux_x64]#  ./arcconf create 1 logicaldrive name data3 LDcache LON 3070000 10  ARRAY 1 noprompt

[root@host /tmp/arcconf/linux_x64]#  ./arcconf create 1 logicaldrive name data4 LDcache LON 499000 10  ARRAY 1 noprompt

[root@host /tmp/arcconf/linux_x64]#  fdisk -l

Disk /dev/sda: 480.1 GB, 480070426624 bytes, 937637552 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 262144 bytes
Disk label type: dos
Disk identifier: 0x000a9444

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   937506479   468752216   83  Linux
/dev/sda2       937506480   937637551       65536    c  W95 FAT32 (LBA)
Partition 2 does not start on physical sector boundary.
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sdb: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes
Disk label type: gpt
Disk identifier: 08A14123-EF5C-4775-A0C0-2DE80827BD40


#         Start          End    Size  Type            Name
 1         3584   6287357439      3T  Microsoft basic primary

Disk /dev/sdc: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes


Disk /dev/sdd: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes


Disk /dev/sde: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes


Disk /dev/sdf: 523.2 GB, 523239424000 bytes, 1021952000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 262144 bytes / 1835008 bytes

后续分区格式化挂接略

Broadcom avago MegaRAID storcli 

RAID 卡背景信息

使用操作系统命令查看 RAID 卡型号是超聚变(原华为 x86)服务器 2288H V5 的 OEM Megaraid RAID 卡

# dmidecode |grep -Ei "manufacturer|product name"    
    Manufacturer: XFUSION
    Product Name: 2288H V5
# dmesg|grep -i megaraid
...
[    7.846978] scsi host0: Avago SAS based MegaRAID driver
...

下载和安装 storcli 管理工具

  1. 登录BROADCOM网站上RAID控制卡的页面

  2. 在“DOWNLOADS”页签中单击“Management Software and Tools”。

  3. 打开管理软件和工具列表。

  4. 下载最新版本的“StorCLI”工具。

  5. 解压后得到适用于不同操作系统的工具包,安装 linux 的 rpm 包。

使用 storcli 管理工具

storcli 管理工具的使用教程较多,简单列举如下:

  1. 查看 RAID 卡配置和当前磁盘 RAID 配置

通常 v1 是数据磁盘。

./storcli64 /c0 show
./storcli64 /c0/v1 show all
  1. 删除 v1

./storcli64 /c0/v1 del

  1. 按需添加 v1 等多个 LUN

./storcli64 /c0 add vd r10 size=2000GB drives=252:0-13

0
0
0
0

版权声明:本文为 TiDB 社区用户原创文章,遵循 CC BY-NC-SA 4.0 版权协议,转载请附上原文出处链接和本声明。

评论
暂无评论