disk 相关,du,df,scsi,fdisk,sfdisk,dd,iostat,hdpram, iotop；lvs；fio；sync

文章目录

• 问题与挑战
• /sys/bus/pci/rescan
• /etc/fstab
• dd
• • bs=BYTES
  • cbs=BYTES
  • conv=CONVS
  • count=N
  • 帮助，版本
  • 需要注意的事情
• hdparm
• • 示例
• hexdump -C /dev/sda2 | less
• lsblk
• lvs
• sfdisk
• • --show-geometry
  • 代码分析
• fdisk
• du
• • --max-depth=1
  • 错误 cannot access
  • 问题1
• df
• scsi kernel module
• iostat
• iotop
• • 实例
• udev management tool
• fio
• sync

问题与挑战

在云环境下经常遇到的磁盘读写变慢的问题，就需要各种磁盘的工具，来检查磁盘的IO的实际情况。
之前遇到过一个情况，ceph的HDD磁盘很慢，导致系统安装出现问题，原因在于安装系统时有各种各样的定时器，很容易就超时了。

/sys/bus/pci/rescan

强制pci重新扫描
echo “1” > /sys/bus/pci/rescan

/etc/fstab

系统的静态文件系统配置。在系统启动时，systemd的服务storage.target会使用这个文件，加载文件系统。

经典的记录示例：
LABEL=t-home2 /home ext4 defaults,auto_da_alloc 0 2

blkid -o device -t LABEL=t-home2
可以查看 当前标签下的设备；查看设备的命令还可以使用lsblk -f。

dd

date;dd if=/dev/zero of=/var/test.out bs=1024 count=1048576;date;rm /var/test.out;
这个命令可以转换和复制文件。

bs=BYTES

一次读写的最大字节数，默认是512字节；会覆盖 ibs和obs

cbs=BYTES

一次转换的字节数

conv=CONVS

convert the file as per the comma separated symbol list

count=N

          copy only N input blocksibs=BYTESread up to BYTES bytes at a time (default: 512)if=FILEread from FILE instead of stdiniflag=FLAGSread as per the comma separated symbol listobs=BYTESwrite BYTES bytes at a time (default: 512)of=FILEwrite to FILE instead of stdoutoflag=FLAGSwrite as per the comma separated symbol listseek=N skip N obs-sized blocks at start of outputskip=N skip N ibs-sized blocks at start of inputstatus=LEVELThe LEVEL of information to print to stderr; 'none'suppresses everything but error messages, 'noxfer'suppresses the final transfer statistics, 'progress' showsperiodic transfer statisticsN and BYTES may be followed by the following multiplicativesuffixes: c=1, w=2, b=512, kB=1000, K=1024, MB=1000*1000,M=1024*1024, xM=M, GB=1000*1000*1000, G=1024*1024*1024, and so onfor T, P, E, Z, Y.  Binary prefixes can be used, too: KiB=K,MiB=M, and so on.Each CONV symbol may be:ascii  from EBCDIC to ASCIIebcdic from ASCII to EBCDICibm    from ASCII to alternate EBCDICblock  pad newline-terminated records with spaces to cbs-sizeunblockreplace trailing spaces in cbs-size records with newlinelcase  change upper case to lower caseucase  change lower case to upper casesparse try to seek rather than write all-NUL output blocksswab   swap every pair of input bytessync   pad every input block with NULs to ibs-size; when usedwith block or unblock, pad with spaces rather than NULsexcl   fail if the output file already existsnocreatdo not create the output filenotruncdo not truncate the output filenoerrorcontinue after read errorsfdatasyncphysically write output file data before finishingfsync  likewise, but also write metadataEach FLAG symbol may be:append append mode (makes sense only for output; conv=notrunc  suggested)direct use direct I/O for datadirectory fail unless a directorydsync  use synchronized I/O for datasync   likewise, but also for metadatafullblock accumulate full blocks of input (iflag only)nonblock use non-blocking I/Onoatime do not update access timenocache Request to drop cache.  See also oflag=syncnoctty do not assign controlling terminal from filenofollowdo not follow symlinkscount_bytestreat 'count=N' as a byte count (iflag only)skip_bytestreat 'skip=N' as a byte count (iflag only)seek_bytestreat 'seek=N' as a byte count (oflag only)Sending a USR1 signal to a running 'dd' process makes it printI/O statistics to standard error and then resume copying.

帮助，版本

–help 显示帮助
–version 显示版本

需要注意的事情

for i in {1…4}; do dd bs=1M count=1024 if=/dev/zero of=simple-container-benchmarks-writetest conv=fdatasync; done
这个命令在执行测试时，第一次和第二次的结果可能比后续的好，原因是一开始系统的iowait非常小，处理的也就快。后续的操作，使得iowait的值保持平稳只会，速率就会保持在一个相对较小，而且稳定的数值。

hdparm

查询，设置SATA/IDE设备参数。

[root@10 ~]# which hdparm
/usr/sbin/hdparm
[root@10 ~]# rpm -qf /usr/sbin/hdparm
hdparm-9.54-4.el8.x86_64

示例

#hdparm --direct -t -T /dev/vda3
/dev/vda3:
Timing O_DIRECT cached reads: 160 MB in 2.05 seconds = 77.91 MB/sec
Timing O_DIRECT disk reads: 242 MB in 3.06 seconds = 79.21 MB/sec

hexdump -C /dev/sda2 | less

[root@10 ~]# hexdump -C /dev/sda2 | less
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…|
*
00000200 4c 41 42 45 4c 4f 4e 45 01 00 00 00 00 00 00 00 |LABELONE…|
00000210 6f 72 05 3a 20 00 00 00 4c 56 4d 32 20 30 30 31 |or.: …LVM2 001|
00000220 79 37 76 43 7a 67 6e 66 57 51 51 33 66 31 38 7a |y7vCzgnfWQQ3f18z|
00000230 64 66 39 62 61 75 38 58 7a 4f 72 51 49 47 42 50 |df9bau8XzOrQIGBP|
00000240 00 00 f0 bf 09 00 00 00 00 00 10 00 00 00 00 00 |…|
00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…|

lsblk

lsblk - 展示block设备
用法： lsblk [options] [device…]
lsblk可用查询所有可用的块设备或者命令里指定的设备。lsblk通过读取sysfs文件系统和udev数据库来收集信息。如果udev数据库不存在，或者lsblk在编译时没有配置udev支持选项，就从块设备来获取标签，UUID和文件系统类型信息。当然是需要root权限。
这个命令默认是按照树的格式打印出所有块设备（除了RAM磁盘）。Use lsblk --help to get a list of all available columns.

# lsblk -f
NAME   FSTYPE  LABEL        UUID                                 MOUNTPOINT
sda
|-sda1 ext4    1@-mg     c9304d5e-cb17-4c2d-a49b-
`-sda3 ext4    2ap1@-mg 5c63eb84-6752-4071-950c-
sdb
|-sdb1 ext4                 212e64f0-0881-468f-ad41- /
`-sdb3 ext4                 994b9ec0-d669-4a75-8ca0- /app1

lvs

lvs — Display information about logical volumes
[root@10 ~]# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root rl -wi-ao---- <36.94g
swap rl -wi-ao---- <2.06g

sfdisk

/* returns: 0 on success, <0 on error, 1 successfully stop sfdisk */
这个命令的返回值，0是成功，1是成功停止sfdisk，其他值是错误。所以在写脚本时，需要注意1的情况。

设置debug 信息
export SFDISK_DEBUG=all

[root@rhel-cleanmodules ~]# sfdisk -v
sfdisk from util-linux 2.32.1

参数
-A, --activate device [partition-number…]
Switch on the bootable flag for the specified partitions and switch off the bootable flag on all unspecified partitions. The special placeholder ‘-’ may be used instead of the partition numbers to switch off the bootable flag on all partitions.
If no partition-number is specified, then list the partitions with an enabled flag.

sfdisk -A /dev/sda 1, 将/dev/sda1 设置为启动磁盘分区。和之前的格式不一样，之前是 sfdisk -A1 /dev/sda, 1 在-A之后。

–show-geometry

#sfdisk --show-geometry /dev/sda
/dev/sda: 21504 cylinders, 255 heads, 2 sectors/track

代码分析

从环境变量设置debug level
main-》sfdisk_init-》sfdiskprog_init_debug-》__UL_INIT_DEBUG_FROM_ENV

#define __UL_INIT_DEBUG_FROM_ENV(lib, pref, mask, env) \do { \const char *envstr = mask ? NULL : getenv(# env); \__UL_INIT_DEBUG_FROM_STRING(lib, pref, mask, envstr); \} while (0)

fdisk

du

estimate file space usage， Summarize disk usage of the set of FILEs, recursively for directories.

du -h tutorials
du -ah tutorials
du -sh, summary

–max-depth=1

-d, --max-depth=N
print the total for a directory (or file, with --all) only if it is N or fewer levels below the command
line argument; --max-depth=0 is the same as --summarize

错误 cannot access

du: cannot access ‘/proc/21419/task/21419/fd/4’: No such file or directory
这个的原因是线程、进程 21419已经被销毁，或者应用的fd被释放。
一般使用du 查看proc目录时会出现这个错误，21419可能是du本身运行的一个进程号。

问题1

在linux和solaris系统上du输出的结果不一样，原因可能是文件系统不一致，占用的磁盘空间有差异。
如果可以使用 下面的选项，来查询实际文件大小来做比较。
–apparent-size
print apparent sizes, rather than disk usage; although the apparent size is usually smaller, it may be larger due to holes in (‘sparse’) files, internal fragmentation, indirect blocks, and the like

df

展示磁盘文件的使用情况。
df -h
df -k
-k like --block-size=1K

STATION_B (root) : /usr/mfs/log> df -k
Filesystem 1024-blocks Used Available Capacity Mounted on
root_domain#root 512000 500812 4968 100% /
/proc 0 0 0 100% /proc
usr_domain#usr 1536000 1123961 390880 75% /usr
var_domain#var 1024000 556762 456328 55% /var

scsi kernel module

#modprobe ./kernel/drivers/scsi/sd_mod.ko.xz
modprobe: FATAL: Module ./kernel/drivers/scsi/sd_mod.ko.xz not found in directory /lib/modules/4.18.0-305.7.1.el8_4.x86_64

[ 6.383346] scsi 2:0:0:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 ANSI: 2
[ 6.424291] scsi 2:0:0:0: Attached scsi generic sg0 type 0
[ 6.436515] sd_mod: Unknown symbol t10_pi_type1_crc (err 0)
[ 6.436556] sd_mod: Unknown symbol t10_pi_type3_crc (err 0)
[ 6.436560] sd_mod: Unknown symbol t10_pi_type3_ip (err 0)
[ 6.436610] sd_mod: Unknown symbol t10_pi_type1_ip (err 0)
依赖t10-pi 模块

/block/t10-pi.c
const struct blk_integrity_profile t10_pi_type1_crc = {.name			= "T10-DIF-TYPE1-CRC",.generate_fn		= t10_pi_type1_generate_crc,.verify_fn		= t10_pi_type1_verify_crc,.ext_ops		= &t10_pi_type1_ops,
};

iostat

#rpm -qf /usr/bin/iostat
sysstat-11.7.3-6.el8.x86_64

git clone .git

问题，显示第一条数据有点不准。第一次的数据要比其他的数据要大。

#iostat -d -x -k 1 10
Linux 4.18.0-348.20.1.el8_5.x86_64 (vl2048-oam-a)       06/20/22        _x86_64_        (2 CPU)Device  r/s     w/s     rkB/s     wkB/s   rrqm/s   wrqm/s  %rrqm  %wrqm r_await w_await aqu-sz rareq-sz wareq-sz  svctm  %util
vda     1.57   59.02    100.12    649.75     0.01    22.64   0.90  27.73   11.21    6.98   0.43    63.62    11.01   3.97  24.04
vdb     0.01    0.00      0.24      0.00     0.00     0.00   0.00   0.00    0.57    0.00   0.00    24.58     0.00   0.50   0.00Device r/s     w/s     rkB/s     wkB/s   rrqm/s   wrqm/s  %rrqm  %wrqm r_await w_await aqu-sz rareq-sz wareq-sz  svctm  %util
vda    0.00   19.00      0.00     64.00     0.00     0.00   0.00   0.00    0.00    2.32   0.04     0.00     3.37   0.95   1.80
vdb    0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00

iotop

类似于CPU top命令的，检测IO的top命令。
可以显示出哪一个线程、进程占用的IO多。
[root@10 ~]# rpm -ql iotop-0.6-17.el8.noarch
/usr/lib/python3.6/site-packages/iotop
python写的脚本，也是按照系统提高的实时的统计数据，进行展示。
otal DISK READ : 0.00 B/s | Total DISK WRITE : 19.41 K/s
Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 54.36 K/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
4009 be/3 root 0.00 B 4.00 K 0.00 % 93.11 % [jbd2/vda1-7] ///这里可以看出这个进程 IO 很大，内核的进程

实例

iotop -b -o -n 5 –d 2 -P > 222.txt
-o: don’t display those process without I/O
-P: only show process
-b: run in background/non-interactive mode
-n 5: run for 5 round
-d 2: refresh interval 2S

Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ :       0.00 B/s | Total DISK WRITE :       3.93 K/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
b' 195168 be/4 root        0.00 B/s    3.93 K/s ?unavailable?  platform-python -s /usr/sbin/iotop -b -o -n 5 -P'
Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       2.50 M/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
b' 188124 be/4 root        0.00 B/s    0.00 B/s ?unavailable?  [kworker/u8:1-writeback]'
Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND

udev management tool

udevadm trigger --subsystem-match=block
-s, --subsystem-match=SUBSYSTEM
Trigger events for devices which belong to a matching subsystem. This option can be specified multiple times and supports shell style pattern matching.

fio

sync

sync - Synchronize cached writes to persistent storage
If one or more files are specified, sync only them, or their containing file systems.
-d, --data sync only file data, no unneeded metadata
-f, --file-system sync the file systems that contain the files
–help display this help and exit
–version output version information and exit

disk 相关,du,df,scsi,fdisk,sfdisk,dd,iostat,hdpram, iotop；lvs；fio；sync

文章目录

• 问题与挑战
• /sys/bus/pci/rescan
• /etc/fstab
• dd
• • bs=BYTES
  • cbs=BYTES
  • conv=CONVS
  • count=N
  • 帮助，版本
  • 需要注意的事情
• hdparm
• • 示例
• hexdump -C /dev/sda2 | less
• lsblk
• lvs
• sfdisk
• • --show-geometry
  • 代码分析
• fdisk
• du
• • --max-depth=1
  • 错误 cannot access
  • 问题1
• df
• scsi kernel module
• iostat
• iotop
• • 实例
• udev management tool
• fio
• sync

问题与挑战

在云环境下经常遇到的磁盘读写变慢的问题，就需要各种磁盘的工具，来检查磁盘的IO的实际情况。
之前遇到过一个情况，ceph的HDD磁盘很慢，导致系统安装出现问题，原因在于安装系统时有各种各样的定时器，很容易就超时了。

/sys/bus/pci/rescan

强制pci重新扫描
echo “1” > /sys/bus/pci/rescan

/etc/fstab

系统的静态文件系统配置。在系统启动时，systemd的服务storage.target会使用这个文件，加载文件系统。

经典的记录示例：
LABEL=t-home2 /home ext4 defaults,auto_da_alloc 0 2

blkid -o device -t LABEL=t-home2
可以查看 当前标签下的设备；查看设备的命令还可以使用lsblk -f。

dd

date;dd if=/dev/zero of=/var/test.out bs=1024 count=1048576;date;rm /var/test.out;
这个命令可以转换和复制文件。

bs=BYTES

一次读写的最大字节数，默认是512字节；会覆盖 ibs和obs

cbs=BYTES

一次转换的字节数

conv=CONVS

convert the file as per the comma separated symbol list

count=N

          copy only N input blocksibs=BYTESread up to BYTES bytes at a time (default: 512)if=FILEread from FILE instead of stdiniflag=FLAGSread as per the comma separated symbol listobs=BYTESwrite BYTES bytes at a time (default: 512)of=FILEwrite to FILE instead of stdoutoflag=FLAGSwrite as per the comma separated symbol listseek=N skip N obs-sized blocks at start of outputskip=N skip N ibs-sized blocks at start of inputstatus=LEVELThe LEVEL of information to print to stderr; 'none'suppresses everything but error messages, 'noxfer'suppresses the final transfer statistics, 'progress' showsperiodic transfer statisticsN and BYTES may be followed by the following multiplicativesuffixes: c=1, w=2, b=512, kB=1000, K=1024, MB=1000*1000,M=1024*1024, xM=M, GB=1000*1000*1000, G=1024*1024*1024, and so onfor T, P, E, Z, Y.  Binary prefixes can be used, too: KiB=K,MiB=M, and so on.Each CONV symbol may be:ascii  from EBCDIC to ASCIIebcdic from ASCII to EBCDICibm    from ASCII to alternate EBCDICblock  pad newline-terminated records with spaces to cbs-sizeunblockreplace trailing spaces in cbs-size records with newlinelcase  change upper case to lower caseucase  change lower case to upper casesparse try to seek rather than write all-NUL output blocksswab   swap every pair of input bytessync   pad every input block with NULs to ibs-size; when usedwith block or unblock, pad with spaces rather than NULsexcl   fail if the output file already existsnocreatdo not create the output filenotruncdo not truncate the output filenoerrorcontinue after read errorsfdatasyncphysically write output file data before finishingfsync  likewise, but also write metadataEach FLAG symbol may be:append append mode (makes sense only for output; conv=notrunc  suggested)direct use direct I/O for datadirectory fail unless a directorydsync  use synchronized I/O for datasync   likewise, but also for metadatafullblock accumulate full blocks of input (iflag only)nonblock use non-blocking I/Onoatime do not update access timenocache Request to drop cache.  See also oflag=syncnoctty do not assign controlling terminal from filenofollowdo not follow symlinkscount_bytestreat 'count=N' as a byte count (iflag only)skip_bytestreat 'skip=N' as a byte count (iflag only)seek_bytestreat 'seek=N' as a byte count (oflag only)Sending a USR1 signal to a running 'dd' process makes it printI/O statistics to standard error and then resume copying.

帮助，版本

–help 显示帮助
–version 显示版本

需要注意的事情

for i in {1…4}; do dd bs=1M count=1024 if=/dev/zero of=simple-container-benchmarks-writetest conv=fdatasync; done
这个命令在执行测试时，第一次和第二次的结果可能比后续的好，原因是一开始系统的iowait非常小，处理的也就快。后续的操作，使得iowait的值保持平稳只会，速率就会保持在一个相对较小，而且稳定的数值。

hdparm

查询，设置SATA/IDE设备参数。

[root@10 ~]# which hdparm
/usr/sbin/hdparm
[root@10 ~]# rpm -qf /usr/sbin/hdparm
hdparm-9.54-4.el8.x86_64

示例

#hdparm --direct -t -T /dev/vda3
/dev/vda3:
Timing O_DIRECT cached reads: 160 MB in 2.05 seconds = 77.91 MB/sec
Timing O_DIRECT disk reads: 242 MB in 3.06 seconds = 79.21 MB/sec

hexdump -C /dev/sda2 | less

[root@10 ~]# hexdump -C /dev/sda2 | less
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…|
*
00000200 4c 41 42 45 4c 4f 4e 45 01 00 00 00 00 00 00 00 |LABELONE…|
00000210 6f 72 05 3a 20 00 00 00 4c 56 4d 32 20 30 30 31 |or.: …LVM2 001|
00000220 79 37 76 43 7a 67 6e 66 57 51 51 33 66 31 38 7a |y7vCzgnfWQQ3f18z|
00000230 64 66 39 62 61 75 38 58 7a 4f 72 51 49 47 42 50 |df9bau8XzOrQIGBP|
00000240 00 00 f0 bf 09 00 00 00 00 00 10 00 00 00 00 00 |…|
00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |…|

lsblk

lsblk - 展示block设备
用法： lsblk [options] [device…]
lsblk可用查询所有可用的块设备或者命令里指定的设备。lsblk通过读取sysfs文件系统和udev数据库来收集信息。如果udev数据库不存在，或者lsblk在编译时没有配置udev支持选项，就从块设备来获取标签，UUID和文件系统类型信息。当然是需要root权限。
这个命令默认是按照树的格式打印出所有块设备（除了RAM磁盘）。Use lsblk --help to get a list of all available columns.

# lsblk -f
NAME   FSTYPE  LABEL        UUID                                 MOUNTPOINT
sda
|-sda1 ext4    1@-mg     c9304d5e-cb17-4c2d-a49b-
`-sda3 ext4    2ap1@-mg 5c63eb84-6752-4071-950c-
sdb
|-sdb1 ext4                 212e64f0-0881-468f-ad41- /
`-sdb3 ext4                 994b9ec0-d669-4a75-8ca0- /app1

lvs

lvs — Display information about logical volumes
[root@10 ~]# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root rl -wi-ao---- <36.94g
swap rl -wi-ao---- <2.06g

sfdisk

/* returns: 0 on success, <0 on error, 1 successfully stop sfdisk */
这个命令的返回值，0是成功，1是成功停止sfdisk，其他值是错误。所以在写脚本时，需要注意1的情况。

设置debug 信息
export SFDISK_DEBUG=all

[root@rhel-cleanmodules ~]# sfdisk -v
sfdisk from util-linux 2.32.1

参数
-A, --activate device [partition-number…]
Switch on the bootable flag for the specified partitions and switch off the bootable flag on all unspecified partitions. The special placeholder ‘-’ may be used instead of the partition numbers to switch off the bootable flag on all partitions.
If no partition-number is specified, then list the partitions with an enabled flag.

sfdisk -A /dev/sda 1, 将/dev/sda1 设置为启动磁盘分区。和之前的格式不一样，之前是 sfdisk -A1 /dev/sda, 1 在-A之后。

–show-geometry

#sfdisk --show-geometry /dev/sda
/dev/sda: 21504 cylinders, 255 heads, 2 sectors/track

代码分析

从环境变量设置debug level
main-》sfdisk_init-》sfdiskprog_init_debug-》__UL_INIT_DEBUG_FROM_ENV

#define __UL_INIT_DEBUG_FROM_ENV(lib, pref, mask, env) \do { \const char *envstr = mask ? NULL : getenv(# env); \__UL_INIT_DEBUG_FROM_STRING(lib, pref, mask, envstr); \} while (0)

fdisk

du

estimate file space usage， Summarize disk usage of the set of FILEs, recursively for directories.

du -h tutorials
du -ah tutorials
du -sh, summary

–max-depth=1

-d, --max-depth=N
print the total for a directory (or file, with --all) only if it is N or fewer levels below the command
line argument; --max-depth=0 is the same as --summarize

错误 cannot access

du: cannot access ‘/proc/21419/task/21419/fd/4’: No such file or directory
这个的原因是线程、进程 21419已经被销毁，或者应用的fd被释放。
一般使用du 查看proc目录时会出现这个错误，21419可能是du本身运行的一个进程号。

问题1

在linux和solaris系统上du输出的结果不一样，原因可能是文件系统不一致，占用的磁盘空间有差异。
如果可以使用 下面的选项，来查询实际文件大小来做比较。
–apparent-size
print apparent sizes, rather than disk usage; although the apparent size is usually smaller, it may be larger due to holes in (‘sparse’) files, internal fragmentation, indirect blocks, and the like

df

展示磁盘文件的使用情况。
df -h
df -k
-k like --block-size=1K

STATION_B (root) : /usr/mfs/log> df -k
Filesystem 1024-blocks Used Available Capacity Mounted on
root_domain#root 512000 500812 4968 100% /
/proc 0 0 0 100% /proc
usr_domain#usr 1536000 1123961 390880 75% /usr
var_domain#var 1024000 556762 456328 55% /var

scsi kernel module

#modprobe ./kernel/drivers/scsi/sd_mod.ko.xz
modprobe: FATAL: Module ./kernel/drivers/scsi/sd_mod.ko.xz not found in directory /lib/modules/4.18.0-305.7.1.el8_4.x86_64

[ 6.383346] scsi 2:0:0:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 ANSI: 2
[ 6.424291] scsi 2:0:0:0: Attached scsi generic sg0 type 0
[ 6.436515] sd_mod: Unknown symbol t10_pi_type1_crc (err 0)
[ 6.436556] sd_mod: Unknown symbol t10_pi_type3_crc (err 0)
[ 6.436560] sd_mod: Unknown symbol t10_pi_type3_ip (err 0)
[ 6.436610] sd_mod: Unknown symbol t10_pi_type1_ip (err 0)
依赖t10-pi 模块

/block/t10-pi.c
const struct blk_integrity_profile t10_pi_type1_crc = {.name			= "T10-DIF-TYPE1-CRC",.generate_fn		= t10_pi_type1_generate_crc,.verify_fn		= t10_pi_type1_verify_crc,.ext_ops		= &t10_pi_type1_ops,
};

iostat

#rpm -qf /usr/bin/iostat
sysstat-11.7.3-6.el8.x86_64

git clone .git

问题，显示第一条数据有点不准。第一次的数据要比其他的数据要大。

#iostat -d -x -k 1 10
Linux 4.18.0-348.20.1.el8_5.x86_64 (vl2048-oam-a)       06/20/22        _x86_64_        (2 CPU)Device  r/s     w/s     rkB/s     wkB/s   rrqm/s   wrqm/s  %rrqm  %wrqm r_await w_await aqu-sz rareq-sz wareq-sz  svctm  %util
vda     1.57   59.02    100.12    649.75     0.01    22.64   0.90  27.73   11.21    6.98   0.43    63.62    11.01   3.97  24.04
vdb     0.01    0.00      0.24      0.00     0.00     0.00   0.00   0.00    0.57    0.00   0.00    24.58     0.00   0.50   0.00Device r/s     w/s     rkB/s     wkB/s   rrqm/s   wrqm/s  %rrqm  %wrqm r_await w_await aqu-sz rareq-sz wareq-sz  svctm  %util
vda    0.00   19.00      0.00     64.00     0.00     0.00   0.00   0.00    0.00    2.32   0.04     0.00     3.37   0.95   1.80
vdb    0.00    0.00      0.00      0.00     0.00     0.00   0.00   0.00    0.00    0.00   0.00     0.00     0.00   0.00   0.00

iotop

类似于CPU top命令的，检测IO的top命令。
可以显示出哪一个线程、进程占用的IO多。
[root@10 ~]# rpm -ql iotop-0.6-17.el8.noarch
/usr/lib/python3.6/site-packages/iotop
python写的脚本，也是按照系统提高的实时的统计数据，进行展示。
otal DISK READ : 0.00 B/s | Total DISK WRITE : 19.41 K/s
Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 54.36 K/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
4009 be/3 root 0.00 B 4.00 K 0.00 % 93.11 % [jbd2/vda1-7] ///这里可以看出这个进程 IO 很大，内核的进程

实例

iotop -b -o -n 5 –d 2 -P > 222.txt
-o: don’t display those process without I/O
-P: only show process
-b: run in background/non-interactive mode
-n 5: run for 5 round
-d 2: refresh interval 2S

Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ :       0.00 B/s | Total DISK WRITE :       3.93 K/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
b' 195168 be/4 root        0.00 B/s    3.93 K/s ?unavailable?  platform-python -s /usr/sbin/iotop -b -o -n 5 -P'
Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       2.50 M/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
b' 188124 be/4 root        0.00 B/s    0.00 B/s ?unavailable?  [kworker/u8:1-writeback]'
Total DISK READ :       0.00 B/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/sPID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND

udev management tool

udevadm trigger --subsystem-match=block
-s, --subsystem-match=SUBSYSTEM
Trigger events for devices which belong to a matching subsystem. This option can be specified multiple times and supports shell style pattern matching.

fio

sync

sync - Synchronize cached writes to persistent storage
If one or more files are specified, sync only them, or their containing file systems.
-d, --data sync only file data, no unneeded metadata
-f, --file-system sync the file systems that contain the files
–help display this help and exit
–version output version information and exit